I am connected to a Linode terminal via SSH.
I have started a Jupyter Notebook server from the command line.
But I want to do some other tasks in the command line, keeping the NB server running. I can't work out how to do this without stopping the Jupyter Notebook server. It says:
[I 05:55:05.523 NotebookApp] Use Control-C to stop this server and shut
down all kernels (twice to skip confirmation).
Since you mentioned Linode, I'm assuming it is on a linux/unix system.
Possbile options:
Send process to background
send process to background: jupyter notebook &
do some work on terminal
close jupyter with pkill jupyter (this should have the same effect as
a ctrl+c keypress)
WARNING: this will kill all jupyter instances
use a terminal multiplexer like tmux
just use multiple ssh sessions (not really elegant imo)
Overall, I think the option that would be easiest and most flexible is option 2.
Related
I can join the port sucessfully by
ssh -N -L 8888:127.0.0.1:8888 server_ip
But if I just join the server, and then use jupyter notebook, I can't open the port sucessfully.
ssh server_ip
jupyter notebook --port=8888
Error:
The port 8888 is already in use, trying another port.
What is the reason and how to solve it?
You can check which application is currently running on port 8888 by following commands on Linux machine. If in case it happens to be an existing jupyter, you may try killing that process and start a new one.
Check current process running in 8888
lsof -i:8888
To kill the current process.
kill $(lsof -t -i:8888)
To forcefully kill the current process
kill -9 $(lsof -t -i:8888)
If for some reason you wish not to kill the current process, you may try launching jupyter on a different port.
jupyter notebook --port=8889
Sounds like you have some other process on port 8888.
You can either:
1) Kill the process on port 8888 and then launch jupyter-notebook on port 8888
2) Launch jupyter notebook on a different port and then use ssh to connect to the new port
I have several long running scripts (in Jupyter Notebook) on a remote Google Cloud Compute Instance.
If I lose the ssh connection, I cannot reconnect to the (running) Notebook without stopping those running scripts--executing within the Notebook.
It seems that closing my macbook, will sever my connection to the remote (running) jupyter notebook. Is there some way to reconnect without stopping the script?
On Google Cloud, Jupyter is still running. I just can't connect to the notebook executing the code––without stopping code execution.
I'm sure other Jupyter users have figured this out :)
Thanks in advance
My GCloud Tunneling Script
gcloud compute ssh --zone us-central1-c my-compute-instance -- -N -p 22 -D localhost:5000
Bash Script that Launches Chrome
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome
"localhost:22"
--proxy-server="socks5://localhost:5000"
--host-resolver-rules="MAP * 0.0.0 , EXCLUDE localhost"
--user-data-dir=/tmp/
Nohup that launches Jupyter on Gcloud
nohup jupyter notebook --no-browser > log.txt 2>&1 &
On my Sierra-os macbook, no proxy settings (System Preferences) are enabled
On Google Cloud, I'm NOT using a static ip, just an ephemeral ip.
Much appreciation in advance
What do you mean by "cannot reconnect" ? Do you mean you can't see the notebook interface anymore ? (In which case this is likely a google cloud question).Or do you mean you can't run code or see previous results ?
If the second, this is a known issue, the jupyter team is working on it; The way to go around that is to wrap your code in Python Futures, that store intermediate code; thus re-accessing the future will not trigger re-computation, but will show you intermediate results.
I connect to a remote server using ssh -L but if I close the laptop lid or the connection is lost, the jupyter notebook is disconnected.
After I reconnect to the remote server, the "last" session is lost.
What can be done to make it persistent?
Could screen help with it?
On the remote server, you should open your jupyter in a screen session, it will make it persistent if you lose the connection to the server and resume it.
On your computer: ssh -L xxxx:localhost:yyyy server.
screen.
jupyter notebook --no-browser --port=yyyy. [on remote server]
In your browser: localhost:xxxx.
To disconnect manually and reconnect:
Exit the screen window: control + a and then d.
Disconnect from the server: control + d
And reconnect ssh -L xxxx:localhost:yyyy.
Optionally, you can reopen the screen window, though unnecessary, using screen -r.
Go back to your notebook or reopen localhost:xxxx.
The standard usage for persisting Jupyter server sessions is the use of nohup and &; in your remote server with IP address xx.xx.xx.xx:
nohup jupyter notebook --no-browser --ip xx.xx.xx.xx --port yyyy &
Now, even if you switch off your laptop or lose the connection, you will be always able to reconnect by pointing your browser at xx.xx.xx.xx:yyyy
Adding to #BiBi's answer...
Instead of screen I could recommend you to take a look at tmux. Especially, if you combine tmux with the Tmux Plugin Manager and install Tmux Resurrect, even after reboots of your remote server you will be able to go back to your previous Tmux sessions.
Shortcuts for tmux are somewhat equal to those of screens, just that control + a is replaced by control + b. Of course, tmux allows you to configure your custom shortcuts.
BiBi's answer is correct. But I had cases where my ssh connection terminated unexpectedly and the port forwarding no longer worked when trying to reconnect. Probably there was some dangling process on the remote machine, not sure.
Anyway, in these cases I used socat to proxy between two local ports on the remote machine:
# jupyter notebook/lab running in screen on port yyyy, then your connection dies...
ssh -L xxxx:localhost:zzzz
socat tcp-listen:zzzz,reuseaddr,fork tcp:localhost:yyyy
This way you can avoid restarting jupyter on a different port
Use the nohup command to keep jupyter running even after exiting the shell or terminal. Type the following command in the specified locations.
In remote server nohup jupyter notebook --no-browser --port=8085 > my.log 2>&1 < /dev/null &. This runs jupyter in port 8085 and any stdout would be present in my.log
In local ssh -NL 8085:localhost:8085 username#xx.xx.xx.xx. If port needs to be specified, you can use ssh -NL 8085:localhost:8085 -p xxxx username#xx.xx.xx.xx
In browser http://127.0.0.1:8085/
Sometimes port 8085 may be occupied in the remote server, in such cases try it with another port but make sure you use the same port number in the local while tunneling.
I am writing a developer tool, part of which will launch a Jupyter notebook in the background with output sent to a particular file, such as
jupyter notebook --ip 0.0.0.0 --no-browser --allow-root \
>> ${NOTEBOOK_LOGFILE} 2>&1 &
However, I still want the notebook's start-up information to be printed to the console via stdout. Such as
[I 18:25:33.166 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
[I 18:25:33.189 NotebookApp] Serving notebooks from local directory: /faces
[I 18:25:33.189 NotebookApp] 0 active kernels
[I 18:25:33.189 NotebookApp] The Jupyter Notebook is running at: http://0.0.0.0:8888/?token=b02f25972...
[I 18:25:33.189 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 18:25:33.189 NotebookApp]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://0.0.0.0:8888/?token=b02f25972...
so that users can still see which URL connection string they need.
I have tried to cat this file after the notebook command but this has some downsides.
The time it takes for the notebook to launch and print the message is variable, and using a combination of sleep along with cat the log file is undesirable, because if there's a rare delay in start-up time, cat of the log file might print nothing because the file is empty.
On the other hand, I don't want to set the sleep time to an overly high number, because then users will have to wait too long at startup.
I have also tried tail -f ${NOTEBOOK_LOGFILE} | grep -n 10 (because the start-up lines will be the first 10). This is promising, but the notebook server does not append a newline to each line until the next line is incoming. This means if you wait for 10 lines, the tail process will hang until some other message is logged to the log file (producing the 10th newline).
How can I ensure that the start-up information is displayed to stdout in a timely fashion from when the notebook outputs this information, while still redirecting notebook output into a log file?
I figured a hack to do it with tail and head, but would be interested in something simpler.
(tail -f -n +1 ${NOTEBOOK_LOGFILE} | head -n 5);
This relies on the fact that the connection URL is also printed among the first 5 lines, and so it won't matter about the lack of newline that keeps head waiting if you try to extract it from lines 9 and 10.
I am using emacs-snapshot with the ssh.el package, following the instructions from the ess manual.
There are a few ways to open an R session, but this is how I do it:
open emacs
C-x C-f /server:dir/file.R this puts me in ESS [S] mode
Type 'plot(1)'
C-c C-n to run
emacs asks for starting directory, and I choose the /server:dir/
I would like for a figure to pop up but it wont.
This also doesn't work when using ess-remote in shell or tramp mode, but it does work if I set the starting directory to my local desktop.
Any advice much appreciated. My current workaround is to print the file to pdf and then open pdf in DocView mode, but this takes a few extra steps and is slow.
I do it the other way around:
ssh -X some.server.com to connect to a remote server with x11 forwarding.
emacsclient -nw to restart an Emacs session that is already running
plot(cumsum(rnorm(100))) in R as usual
Then the plot windows appears on the initial machine I ssh'ed away from.
Edit: As a follow-up to the comment: This works for any emacs, either emacs or emacs-snapshot. For a long time I used (server-start) in the ~/.emacs but now I prefer that (just once) lauch emacs --daemon after which I can then connect to via emacsclient (which also exists as emacsclient-snapshot). I really like this -- it gives me Emacs around R in a persistent session that I connect, disconnect and reconnect to.
I selected Dirk's answer because he pointed me in the right direction, and especially for lowering the energy of activation required to visualize my data, but here I am going to give the details of how I got this to work on my desktop.
1) set ssh keypairs (I had previously done this, full instructions for Ubuntu here)
mkdir ~/.ssh
chmod 700 ~/.ssh
ssh-keygen -t rsa
ssh-copy-id username#hostname
2) include the following in ~/.ssh/config
Host any_server_nickname
HostName hostname
User username
ForwardX11 yes
3) open emacs on local machine
4) C-x C-f
5) /any_server_nickname:dir/file.R for files in home directory or /any_server_nickname:/path/to/file.R
6) plot(1)
7) C-x C-b to evaluate entire buffer.