Remote Jupyter Notebook/ Jupyter Lab cannot connect to Kernel without ssh tunnelling - jupyter-notebook

I am getting a problem in opening remote Jupyter Lab/ Jupyter Notebook, it is not connecting to the kernel, I've mentioned the setup steps, problem coming and attempted fixed below. Any solutions, suggestions, or directions are greatly welcomed. Thanks in advance.
Setup Steps Followed:
I am using a fresh Anaconda 3 install in Ubuntu 20.04 LTS. I set up the Jupyter Notebook/ Jupyter Lab to be accessed remotely using the method provided in this answer (IP of that computer is 172.17.25.75), I've quoted the steps below:
Step 1. jupyter notebook --generate-config
Step 2. gedit /home/koushik/.jupyter/jupyter_notebook_config.py and edit the values c.NotebookApp.allow_origin = '*' and
c.NotebookApp.ip = '0.0.0.0
Step 3. Allow port sudo ufw allow 8888
Step 4. Set Password jupyter notebook password
Step 5. Start Jupyter Lab using jupyter lab --no-browser or Start Jupyter Notebook using jupyter notebook --no-browser
And access the Jupyter Lab/ Jupyter Notebook from my computer using
http://172.17.25.75:8888 in the browser.
I am behind my Institue proxy server. When I connect to Jupyter Lab or Jupyter Notebook using ssh tunneling ssh -N -f -L localhost:8889:localhost:8888 zuddler#172.17.25.75 and access localhost:8889 everything works fine. But when I am not tunneling and accessing them using IP, the following errors occur.
Problem with Jupyter Lab:
Upon running the command jupyter lab --no-browser below is the terminal output:
(base) zuddler#silplab:~$ jupyter lab --no-browser
[I 11:42:09.625 LabApp] JupyterLab extension loaded from /home/zuddler/anaconda3/lib/python3.8/site-packages/jupyterlab
[I 11:42:09.625 LabApp] JupyterLab application directory is /home/zuddler/anaconda3/share/jupyter/lab
[I 11:42:09.627 LabApp] Serving notebooks from local directory: /home/zuddler
[I 11:42:09.627 LabApp] Jupyter Notebook 6.1.4 is running at:
[I 11:42:09.627 LabApp] http://silplab:8888/
[I 11:42:09.627 LabApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 11:42:15.449 LabApp] Build is up to date
Upon opening http://172.17.25.75:8888 in the browser and starting a python 3 notebook, it asked to select a kernel
screenshot 1
upon selecting the kernel Python 3, it says
**Error Starting Kernel**
Invalid Response: 405 Method Not Allowed
screenshot 2
but a kernel was started and the terminal showed the below line
[I 11:46:47.547 LabApp] Kernel started: 572fa891-918c-486c-997a-96dd19651968, name: python3
which is also evident from the below screenshot
screenshot 3
upon selecting the marked kernel, it gives the same error mentioned in screenshot 2.
Upon executing jupyter kernelspec list, one kernel detail was returned
(base) zuddler#silplab:~$ jupyter kernelspec list
Available kernels:
python3 /home/zuddler/anaconda3/share/jupyter/kernels/python3
Problem with Jupyter Notebook:
Upon running the command jupyter notebook --no-browser below is the terminal output:
(base) zuddler#silplab:~$ jupyter notebook --no-browser
[I 11:55:20.472 NotebookApp] JupyterLab extension loaded from /home/zuddler/anaconda3/lib/python3.8/site-packages/jupyterlab
[I 11:55:20.472 NotebookApp] JupyterLab application directory is /home/zuddler/anaconda3/share/jupyter/lab
[I 11:55:20.474 NotebookApp] Serving notebooks from local directory: /home/zuddler
[I 11:55:20.474 NotebookApp] Jupyter Notebook 6.1.4 is running at:
[I 11:55:20.474 NotebookApp] http://silplab:8888/
[I 11:55:20.474 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
Upon opening http://172.17.25.75:8888 in the browser and starting the previously created python 3 notebook, it gave the error Kernel not found asked to select a kernel
screenshot 4
on clicking Set Kernel, it gave a connection failed error as below
screenshot 5
but again a kernel was started and the terminal showed the below line (similar to Jupyter Lab)
[I 11:55:42.447 NotebookApp] Kernel started: d6af7843-cf02-43ce-adc2-14a7659ee4ff, name: python3
Again, upon executing jupyter kernelspec list, one kernel detail was returned
(base) zuddler#silplab:~$ jupyter kernelspec list
Available kernels:
python3 /home/zuddler/anaconda3/share/jupyter/kernels/python3
Attempted Fixed
Try 1. Tried updating jupyter using conda update jupyter, it returned
(base) zuddler#silplab:~$ conda update jupyter
Collecting package metadata (current_repodata.json): done
Solving environment: done
# All requested packages already installed.
So, I tried to check if I have the lastest versions of conda and jupyter and it returned:
(base) zuddler#silplab:~$ jupyter --version
jupyter core : 4.6.3
jupyter-notebook : 6.1.4
qtconsole : 4.7.7
ipython : 7.19.0
ipykernel : 5.3.4
jupyter client : 6.1.7
jupyter lab : 2.2.6
nbconvert : 6.0.7
ipywidgets : 7.5.1
nbformat : 5.0.8
traitlets : 5.0.5
(base) zuddler#silplab:~$ conda --version
conda 4.9.2
To check if there is some path issue, I checked using which as mentioned here and it returned
(base) zuddler#silplab:~$ which jupyter-notebook
/home/zuddler/anaconda3/bin/jupyter-notebook
(base) zuddler#silplab:~$ which jupyter-lab
/home/zuddler/anaconda3/bin/jupyter-lab
(base) zuddler#silplab:~$ which python
/home/zuddler/anaconda3/bin/python
so, the paths were correct.
I also set c.NotebookApp.allow_remote_access = True in .jupyter/jupyter_notebook_config.py file (as mentioned here), but it did not fix the issue.
Try 2. To be sure that the kernel had the correct python path, I checked the kernel.json file, the path to python 3 was correct
(base) zuddler#silplab:~$ cat /home/zuddler/anaconda3/share/jupyter/kernels/python3/kernel.json
{
"argv": [
"/home/zuddler/anaconda3/bin/python",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "Python 3",
"language": "python"
}
I further tried to rename the kernel folder as mentioned in this jupyter-notebook doc
A path ending in ipykernel/resources, like in the example above, is this default kernel. The default often does what you want, so if the python3 kernelspec points somewhere else and you can’t start a Python kernel, try deleting or renaming that kernelspec folder to expose the default.
the executed commands and outputs are below, it revealed the default kernel:
(base) zuddler#silplab:~$ cd /home/zuddler/anaconda3/share/jupyter/kernels/
(base) zuddler#silplab:~/anaconda3/share/jupyter/kernels$ ls
python3
(base) zuddler#silplab:~/anaconda3/share/jupyter/kernels$ mv python3 python3_bak
(base) zuddler#silplab:~/anaconda3/share/jupyter/kernels$ cd ~
(base) zuddler#silplab:~$ jupyter kernelspec list
Available kernels:
python3 /home/zuddler/anaconda3/lib/python3.8/site-packages/ipykernel/resources
python3_bak /home/zuddler/anaconda3/share/jupyter/kernels/python3_bak
but I am getting the same errors.
Try 3. I also tried to follow the suggestions referenced in this question, but to no benefit.

Related

Installed jupyter notebook without anaconda have error like "zsh: command not found: jupyter"

I installed jupyter notebook without anaconda.
(Maybe I think that I installed this with pip3 command.)
In terminal, I can't execute jupyter notebook with this command
jupyter notebook
but another command can execute notebook
python3 -m notebook
I added path in .zshrc file. but nothing changed.
Jupyter path
pip3 show jupyter | grep Location
Location: /Users/myName/Library/Python/3.9/lib/python/site-packages
In my zshrc file
export PATH=/Users/myName/Library/Python/3.9/lib/python/site-packages:$PATH
How can I use the command "jupyter notebook" in terminal?

jupyter-lab does not load jupyter_nbextensions_configurator (but jupyter-notebook does)

Having failed to resolve this after a day and a half of solid effort, I feel I am losing my mind: I have checked config files, rebuilt environments half-a-dozen times, enabled everything possibly relevant, consulted numerous S.O. posts, & github issues and I still can't get it to work.
I want to use certain non-core notebook extensions - and be able to debug using xeus-python (in an environment set up for TensorFlow with CUDA - this part at least has always been OK)
I have tried numerous combinations of python versions, notebook, conda/pip for installation of jupyter etc. but jupyter-lab will not load jupyter_nbextensions_configurator.
However! I discovered by accident that jupyter-notebook does load it (see line 4) - and suddenly I have something to hang a question on. When I run jupyter-notebook, the output begins...
(TF250_PY3810_xeus) A:\My Documents\IT\Programming\Python\Perceptilabs\Batch Files & Shortcuts>jupyter-notebook
[I 10:54:00.733 NotebookApp] Writing notebook server cookie secret to C:\Users\Julian\AppData\Roaming\jupyter\runtime\notebook_cookie_secret
[W 10:54:00.992 NotebookApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
[I 10:54:01.017 NotebookApp] [jupyter_nbextensions_configurator] enabled 0.4.1
[W 2021-07-01 10:54:01.703 LabApp] 'notebook_dir' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2021-07-01 10:54:01.704 LabApp] 'password_required' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2021-07-01 10:54:01.704 LabApp] 'token' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2021-07-01 10:54:01.712 LabApp] JupyterLab extension loaded from C:\Users\Julian\anaconda3\envs\TF250_PY3810_xeus\lib\site-packages\jupyterlab
[I 2021-07-01 10:54:01.712 LabApp] JupyterLab application directory is C:\Users\Julian\anaconda3\envs\TF250_PY3810_xeus\share\jupyter\lab
[I 10:54:01.717 NotebookApp] Serving notebooks from local directory: A:/My Documents
[I 10:54:01.718 NotebookApp] Jupyter Notebook 6.1.1 is running at:
[I 10:54:01.718 NotebookApp] http://localhost:8888/
[I 10:54:01.718 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
(Unfortunately, the debugger seems unavailable in jupyter notebook and ipyfilechooser seemed not to work - note that ipyfilechooser does work in the same environment under jupyter lab!)
Question!
Given this, why isn't jupyter_nbextensions_configurator loaded by jupyter-lab and how do I fix it?
I used a specific notebook version in the end but have no idea whether this is in fact important, since the discovery action was not performed before... but I do wonder whether a specific jupyter version might also be required.
(Side issue/bonus question: why are lab & notebook behaviours different in other ways too e.g. debugger, ipyfilechooser?)
Addtional Info
System: Win 10, 64-bit
I also tried explicitly using
jupyter nbextensions_configurator enable --user/--system/--sys-prefix
even if I used the conda installation that should have taken care of this. Obviously to no avail.
The environment in which is created by (silent, no prompts) batch file as follows (most non-functional content stripped out for posting):
set envName=TF250_PY3810_xeus
echo Conda stuff - create and activate environment %envName%, add xeus-python and cuda support
call conda create -y --name %envName% python==3.8.10
call C:\Users\Julian\anaconda3\Scripts\activate.bat %envName%
call conda install -y -c conda-forge xeus-python
call conda install -y -c conda-forge cudatoolkit=11.2.2
call conda install -y -c conda-forge cudnn=8.1.0.77
echo Conda stuff - install jupyterlab, notebook (versioned!), extensions stuff
call conda install -y -c conda-forge jupyterlab notebook==6.1.1
call conda install -y -c conda-forge jupyter_nbextensions_configurator
call conda install -y -c conda-forge jupyter_contrib_nbextensions
#REM pip installations. NB tensorflow_decision_forests is linux only 30/06/2021 so can't install yet
echo Pip stuff - install TensorFlow, seaborn, etc.
pip install -q --upgrade pip setuptools
pip install -q --no-input tensorflow-gpu==2.5.0 tensorflow-probability tensorflow-addons
pip install -q --no-input seaborn tabulate pandoc
pip install -q ipyfilechooser tqdm
#REM Alternative jupyter PIP setup.
#REM pip install -q jupyterlab jupyter_nbextensions_configurator jupyter_contrib_nbextensions
#REM jupyter contrib nbextension install --user
#REM jupyter nbextensions_configurator enable --user
#REM jupyter lab

How to use conda from Jupyter Lab terminal?

Question in the title. What I've tried: Installing nb_conda and nb_conda_kernels. This did allow me to choose the environment my kernel runs in. However, I am unable to use conda from the terminal inside the Jupyter Lab notebook.
Before launching Jupyter Lab:
(base)
wesle#DESKTOP-VV4V3ND MINGW64 ~/Jupyter Notebooks/jn-anomaly-detect (master)
$ conda activate general-ml
(general-ml)
wesle#DESKTOP-VV4V3ND MINGW64 ~/Jupyter Notebooks/jn-anomaly-detect (master)
$ jupyter lab
From Jupyter Lab terminal:
Change the kernel to run in the general-ml conda environment, then open up a notebook terminal:
wesle#DESKTOP-VV4V3ND MINGW64 ~/Jupyter Notebooks/jn-anomaly-detect (master)
$ conda info -e
bash: conda: command not found
as if it doesn't exist at all. Is there a known solution to this?
Edit:
I noticed that packages installed from Anaconda Navigator GUI are reflected live in Jupyter Notebook. This is an undesirable workaround as it is not efficient to open a new program, navigate through the gui, search for a package then install. All of this could be done from the terminal in 3 seconds.

jupyter notebook cannot open notebook

I am using miniconda version 4.7.10 and the latest version of jupyter notebook. I can launch jupyter notebook and navigate thorough my folders. I can even open a terminal in jupyter and run python code. However, when I try to open or create a .ipynb fileJjupyter gets stuck and I cannot even stop the process. I am using Ubuntu and I have tried with the latest and previous versions of jupyter notebook.
Since you are on Ubuntu, have you tried with docker? It's a one-off install that gives you access to the Docker Hub with a vast array of ready-made, tested applications complete with all necessary dependencies.
In case of Jupyter Lab/Notebook:
docker run -it -p 8888:8888 -v $PWD:/home/jovyan jupyter/datascience-notebook

`jupyter notebook` gives error: `"Could not open static file ''"` on macOS

I haven't changed my system configuration, But I'm spotting this error for the first time today.
I've reported it here: https://github.com/jupyter/notebook/issues/4871
> jupyter notebook
[I 10:44:20.102 NotebookApp] JupyterLab extension loaded from /usr/local/anaconda3/lib/python3.7/site-packages/jupyterlab
[I 10:44:20.102 NotebookApp] JupyterLab application directory is /usr/local/anaconda3/share/jupyter/lab
[I 10:44:20.104 NotebookApp] Serving notebooks from local directory: /Users/pi
[I 10:44:20.104 NotebookApp] The Jupyter Notebook is running at:
[I 10:44:20.104 NotebookApp] http://localhost:8888/?token=586797fb9049c0faea24f2583c4de32c08d45c89051fb07d
[I 10:44:20.104 NotebookApp] or http://127.0.0.1:8888/?token=586797fb9049c0faea24f2583c4de32c08d45c89051fb07d
[I 10:44:20.104 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 10:44:20.110 NotebookApp]
To access the notebook, open this file in a browser:
file:///Users/pi/Library/Jupyter/runtime/nbserver-65385-open.html
Or copy and paste one of these URLs:
http://localhost:8888/?token=586797fb9049c0faea24f2583c4de32c08d45c89051fb07d
or http://127.0.0.1:8888/?token=586797fb9049c0faea24f2583c4de32c08d45c89051fb07d
[E 10:44:21.457 NotebookApp] Could not open static file ''
[W 10:44:21.512 NotebookApp] 404 GET /static/components/react/react-dom.production.min.js (::1) 9.02ms referer=http://localhost:8888/tree?token=BLA
[W 10:44:21.548 NotebookApp] 404 GET /static/components/react/react-dom.production.min.js (::1) 0.99ms referer=http://localhost:8888/tree?token=BLA
Set
Looks like this issue was fixed in Jupyter 6.0.1
So the question becomes: can I force-install jupyter 6.0.1?
As the initial question has now provoked a second question, I now ask this new question here: How to force `conda` to install the latest version of `jupyter`?
Alternatively I can manually provide the missing file, but I'm not sure where. I've asked here: Where does Jupyter install site-packages on macOS?
Research:
https://github.com/jupyter/notebook/pull/4772 "add missing react-dom js to package data #4772" on 6 Aug 2019
minrk added this to the 6.0.1 milestone on 18 Jul
Ok, so can I get Jupyter Notebook 6.0.1?
brew cask install anaconda downloads ~/Library/Caches/Homebrew/downloads/{LONG HEX}--Anaconda3-2019.07-MacOSX-x86_64 which is July, and conda --version reports conda 4.7.10. But this is for Anaconda which is the Package Manager.
> conda list | grep jupy
jupyter 1.0.0 py37_7
jupyter_client 5.3.1 py_0
jupyter_console 6.0.0 py37_0
jupyter_core 4.5.0 py_0
jupyterlab 1.0.2 py37hf63ae98_0
jupyterlab_server 1.0.0 py_0
So that's a bit confusing. No jupyter notebook here.
> which jupyter
/usr/local/anaconda3/bin/jupyter
> jupyter --version
jupyter core : 4.5.0
jupyter-notebook : 6.0.0
qtconsole : 4.5.1
ipython : 7.6.1
ipykernel : 5.1.1
jupyter client : 5.3.1
jupyter lab : 1.0.2
nbconvert : 5.5.0
ipywidgets : 7.5.0
nbformat : 4.4.0
traitlets : 4.3.2
Ok, so it appears jupyter-notebook is in jupyter which is maintained by Anaconda.
Can we update this?
https://jupyter.readthedocs.io/en/latest/projects/upgrade-notebook.html
> conda update jupyter
:
Alas jupyter --version is still 6.0.0
I fixed this by updating both jupyter on pip and pip3 (just to be safe) and this fixed the problem
using both
pip install --upgrade jupyter
and
pip3 install --upgrade jupyter --no-cache-dir
I believe you can do this in the terminal as well as in conda's terminal (since conda envs also have pip)
As per Where does Jupyter install site-packages on macOS?, I locate where on my system jupyter is searching for this missing file:
> find / -path '*/static/components' 2>/dev/null
/usr/local/anaconda3/pkgs/notebook-6.0.0-py37_0/lib/python3.7/site-packages/notebook/static/components
/usr/local/anaconda3/lib/python3.7/site-packages/notebook/static/components
And as per https://github.com/jupyter/notebook/pull/4772#issuecomment-515794823, if I download that file and deposit it in the second location, i.e. creating:
/usr/local/anaconda3/lib/python3.7/site-packages/notebook/static/components/react/react-dom.production.min.js
... now jupyter notebook launches without errors.
(NOTE: Being cautious I have also copied it into the first location. But that doesn't seem to have any effect.)

Resources