Main question
I do a lot of interactive work in R using SSH and tmux. I connect using the -Y flag, e.g.
ssh -Y user#server
Then, I typically start a new tmux session:
tmux new -s name
When I first connect, my plots work fine. However, my connection often gets interrupted or I might disconnect for lunch. For this case, I have a line in my .bashrc that saves my current display:
if [ ! -f ~/.my.display ]; then
touch ~/.my.display
fi
display=$(echo $DISPLAY)
if [[ $display != "" ]]; then
echo "export DISPLAY="$DISPLAY > ~/.my.display
fi
source ~/.my.display
After disconnecting, when I tmux into my interactive R session, I typically need to reset my X11 window:
# reconnect to tmux session
ssh -Y user#server
tmux a -t name
# reset x11 window
graphics.off()
display = gsub(".*DISPLAY.", "", tail(readLines("~/.my.display"), n = 1))
x11(display, type = "Xlib")
This works most of the time. However, around 25% of the time, my graphics.off() call hangs for 30 minutes, which is incredibly frustrating. I often need to kill my session, losing all of my work.
Is there a way to call graphics.off() without it hanging for so long? Ideally, it would try this command for like 10 seconds, then throw an error if there is a problem.
Bonus
Within tmux, I often connect to other user accounts using SSH -Y. Therefore, the sequence of commands is:
# connect laptop -> user1
ssh -Y user1#server1
tmux a -t name
# connect user1 -> user2
ssh -Y user2#server2
I am having trouble making interactive plots with user2's account. Right now, my solution is to save user2's DISPLAY variable to their .my.display. Before plotting with user2 through tmux, I first SSH directly into user2's account, which properly properly configures their .my.display file. Then, I follow the same steps as before: graphics.off() and starting a new x11 window.
This seems unnecessarily complicated, so my second question is if there is a cleaner way to set this up?
I am using XQuartz 2.7.11 on Mac OS 10.12.6
Related
I'm trying to detect the presence of iTerm2 and if I run the following in iTerm2 (echo -n $'\e[5n'; read -s -t 0.1 line; printf '%q\n' "$line") the terminal responds with $'\033'\[ITERM2\ 3.2.1n$'\033'\[0n
However, if I am running a tmux session in the terminal, then tmux responds and gives me nothing.
Any idea how I can ask tmux to query the physical terminal to report its status?
Footnotes
Here is a description of [5n in the tmux source: https://github.com/tmux/tmux/blob/486ce9b09855ae30a2bf5e576cb6f7ad37792699/tools/ansicode.txt#L577
This might be iTerm2 only, since I haven't seen a response on any other terminal
According to ft in freenode's #tmux (and as seen in this Super User answer), you can use:
'\ePtmux;\e" STUFF_FOR_THE_TERMINAL_HERE "\e\\'
So, it would be something like:
echo -n $'\ePtmux;\e\e[5n\e\\'
Use case
I run command over a bastion in SSH and since the connection can be dropped we are using tmux or screen.
For short commands I'm using send-keys, let's say I'm using a command like kubectl get pods. I'd like to keep the output of this command but terminate the tmux session.
Using remain-on-exit is not what I'm looking for as it keeps a dead window.
I've seen those question already
https://unix.stackexchange.com/questions/17116/prevent-pane-window-from-closing-when-command-completes-tmux
TMUX: how to make new window stay when start shell-command quits?
I'd like to keep the output of this command but terminate the tmux session.
I use this in my ~/.tmux.conf, and now when I exit my running shell, pane output is saved to unique log file:
set -g remain-on-exit
set-hook pane-died 'capture-pane -S - -E - ; save-buffer "$HOME/logs/tmux/tmux-saved.#{host_short}-#{session_id}:#{window_id}:#{pane_id}-#{pane_pid}-#{client_activity}.log"; delete-buffer; kill-pane'
I'm not sure it's exactly what you are looking for, but you could use the capture-pane command to save the contents of a pane (subject to what is still in the pane's history) to a file.
I´m a happy user of byobu, but recently I noted that I cannot attach anymore to open sessions.
With ps aux | grep tmux I can clearly see many tmux processes, but unfortunatly, when I try to attach with:
tmux attach
byobu attach
I get no results but a no session error. Moreover, with byobu-select-session I got a failed to connect to server instead.
There is a commant to connect tmux to a given socket, which I found using
lsof -U | grep '^tmux'. But still no session attached. My session files are in /tmp/user/tmux-1000/default, but I can see some sockets being used.
From ps aux I can see that byobu launches tmux with: tmux -2 -f /usr/share/byobu/profiles/tmuxrc new-session -n - /usr/bin/byobu-shell
Unfortunately, either with byobu -S path or byobu -L socketname I am not able to attach to previously open session, and byobu simply start a new session.
I run into a similar situation caused by accidentally removing the tmux socket in /tmp. The method described here solved the problem for me (either killall -SIGUSR1 tmux or kill -USR1 $PID_FOR_RUNNING_TMUX).
I have RStudio server installed on a remote aws server (ubuntu) and want to run several projects at the same time (one of which takes lots of time to finish). On Windows there is a simple GUI solution like 'Open Project in New Window'. Is there something similar for rstudio server?
Simple question, but failed to find a solution except this related question for Macs, which offers
Run multiple rstudio sessions using projects
but how?
While running batch scripts is certainly a good option, it's not the only solution. Sometimes you may still want interactive use in different sessions rather than having to do everything as batch scripts.
Nothing stops you from running multiple instances of RStudio server on your Ubuntu server on different ports. (I find this particularly easy to do by launching RStudio through docker, as outlined here. Because an instance will keep running even when you close the browser window, you can easily launch several instances and switch between them. You'll just have to login again when you switch.
Unfortunately, RStudio-server still prevents you having multiple instances open in the browser at the same time (see the help forum). This is not a big issue as you just have to log in again, but you can work around it by using different browsers.
EDIT: Multiple instances are fine, as long as they are not on the same browser, same browser-user AND on the same IP address. e.g. a session on 127.0.0.1 and another on 0.0.0.0 would be fine. More importantly, the instances keep on running even if they are not 'open', so this really isn't a problem. The only thing to note about this is you would have to log back in to access the instance.
As for projects, you'll see you can switch between projects using the 'projects' button on the top right, but while this will preserve your other sessions I do not think the it actually supports simultaneous code execution. You need multiple instances of the R environment running to actually do that.
UPDATE 2020 Okay, it's now 2020 and there's lots of ways to do this.
For running scripts or functions in a new R environment, check out:
the callr package
The RStudio jobs panel
Run new R sessions or scripts from one or more terminal sessions in the RStudio terminal panel
Log out and log in to the RStudio-server as a different user (requires multiple users to be set up in the container, obviously not a good workflow for a single user but just noting that many different users can access the same RStudio server instance no problem.
Of course, spinning up multiple docker sessions on different ports is still a good option as well. Note that many of the ways listed above still do not allow you to restart the main R session, which prevents you from reloading installed packages, switching between projects, etc, which is clearly not ideal. I think it would be fantastic if switching between projects in an RStudio (server) session would allow jobs in the previously active project to keep running in the background, but have no idea if that's in the cards for the open source version.
Often you don't need several instances of Rstudio - in this case just save your code in .R file and launch it using ubuntu command prompt (maybe using screen)
Rscript script.R
That will launch a separate R session which will do the work without freezing your Rstudio. You can pass arguments too, for example
# script.R -
args <- commandArgs(trailingOnly = TRUE)
if (length(args) == 0) {
start = '2015-08-01'
} else {
start = args[1]
}
console -
Rscript script.R 2015-11-01
I think you need R Studio Server Pro to be able to log in with multiple users/sessions.
You can see the comparison table below for reference.
https://www.rstudio.com/products/rstudio-server-pro/
Installing another instance of rstudio server is less than ideal.
Linux server admins, fear not. You just need root access or a kind admin.
Create a group to use: groupadd Rwarrior
Create an additional user with same home directory as your primary Rstudio login:
useradd -d /home/user1 user2
Add primary and new user into Rwarrior group:
gpasswd -a user2 Rwarrior
gpasswd -a user1 Rwarrior
Take care of the permissions for your primary home directory:
cd /home
chown -R user1:Rwarrior /home/user1
chmod -R 770 /home/user1
chmod g+s /home/user1
Set password for the new user:
passwd user2
Open a new browser window in incognito/private browsing mode and login to Rstudio with the new user you created. Enjoy.
I run multiple RStudio servers by isolating them in Singularity instances. Download the Singularity image with the command singularity pull shub://nickjer/singularity-rstudio
I use two scripts:
run-rserver.sh:
Find a free port
#!/bin/env bash
set -ue
thisdir="$(dirname "${BASH_SOURCE[0]}")"
# Return 0 if the port $1 is free, else return 1
is_port_free(){
port="$1"
set +e
netstat -an |
grep --color=none "^tcp.*LISTEN\s*$" | \
awk '{gsub("^.*:","",$4);print $4}' | \
grep -q "^$port\$"
r="$?"
set -e
if [ "$r" = 0 ]; then return 1; else return 0; fi
}
# Find a free port
find_free_port(){
local lower_port="$1"
local upper_port="$2"
for ((port=lower_port; port <= upper_port; port++)); do
if is_port_free "$port"; then r=free; else r=used; fi
if [ "$r" = "used" -a "$port" = "$upper_port" ]; then
echo "Ports $lower_port to $upper_port are all in use" >&2
exit 1
fi
if [ "$r" = "free" ]; then break; fi
done
echo $port
}
port=$(find_free_port 8080 8200)
echo "Access RStudio Server on http://localhost:$port" >&2
"$thisdir/cexec" \
rserver \
--www-address 127.0.0.1 \
--www-port $port
cexec:
Create a dedicated config directory for each instance
Create a dedicated temporary directory for each instance
Use the singularity instance mechanism to avoid that forked R sessions are adopted by PID 1 and stay around after the rserver has shut down. Instead, they become children of the Singularity instance and are killed when that shuts down.
Map the current directory to the directory /data inside the container and set that as home folder (this step might not be nessecary if you don't care about reproducible paths on every machine)
#!/usr/bin/env bash
# Execute a command in the container
set -ue
if [ "${1-}" = "--help" ]; then
echo <<EOF
Usage: cexec command [args...]
Execute `command` in the container. This script starts the Singularity
container and executes the given command therein. The project root is mapped
to the folder `/data` inside the container. Moreover, a temporary directory
is provided at `/tmp` that is removed after the end of the script.
EOF
exit 0
fi
thisdir="$(dirname "${BASH_SOURCE[0]}")"
container="rserver_200403.sif"
# Create a temporary directory
tmpdir="$(mktemp -d -t cexec-XXXXXXXX)"
# We delete this directory afterwards, so its important that $tmpdir
# really has the path to an empty, temporary dir, and nothing else!
# (for example empty string or home dir)
if [[ ! "$tmpdir" || ! -d "$tmpdir" ]]; then
echo "Error: Could not create temp dir $tmpdir"
exit 1
fi
# check if temp dir is empty (this might be superfluous, see
# https://codereview.stackexchange.com/questions/238439)
tmpcontent="$(ls -A "$tmpdir")"
if [ ! -z "$tmpcontent" ]; then
echo "Error: Temp dir '$tmpdir' is not empty"
exit 1
fi
# Start Singularity instance
instancename="$(basename "$tmpdir")"
# Maybe also superfluous (like above)
rundir="$(readlink -f "$thisdir/.run/$instancename")"
if [ -e "$rundir" ]; then
echo "Error: Runtime directory '$rundir' exists already!" >&2
exit 1
fi
mkdir -p "$rundir"
singularity instance start \
--contain \
-W "$tmpdir" \
-H "$thisdir:/data" \
-B "$rundir:/data/.rstudio" \
-B "$thisdir/.rstudio/monitored/user-settings:/data/.rstudio/monitored/user-settings" \
"$container" \
"$instancename"
# Delete the temporary directory after the end of the script
trap "singularity instance stop '$instancename'; rm -rf '$tmpdir'; rm -rf '$rundir'" EXIT
singularity exec \
--pwd "/data" \
"instance://$instancename" \
"$#"
I am using emacs-snapshot with the ssh.el package, following the instructions from the ess manual.
There are a few ways to open an R session, but this is how I do it:
open emacs
C-x C-f /server:dir/file.R this puts me in ESS [S] mode
Type 'plot(1)'
C-c C-n to run
emacs asks for starting directory, and I choose the /server:dir/
I would like for a figure to pop up but it wont.
This also doesn't work when using ess-remote in shell or tramp mode, but it does work if I set the starting directory to my local desktop.
Any advice much appreciated. My current workaround is to print the file to pdf and then open pdf in DocView mode, but this takes a few extra steps and is slow.
I do it the other way around:
ssh -X some.server.com to connect to a remote server with x11 forwarding.
emacsclient -nw to restart an Emacs session that is already running
plot(cumsum(rnorm(100))) in R as usual
Then the plot windows appears on the initial machine I ssh'ed away from.
Edit: As a follow-up to the comment: This works for any emacs, either emacs or emacs-snapshot. For a long time I used (server-start) in the ~/.emacs but now I prefer that (just once) lauch emacs --daemon after which I can then connect to via emacsclient (which also exists as emacsclient-snapshot). I really like this -- it gives me Emacs around R in a persistent session that I connect, disconnect and reconnect to.
I selected Dirk's answer because he pointed me in the right direction, and especially for lowering the energy of activation required to visualize my data, but here I am going to give the details of how I got this to work on my desktop.
1) set ssh keypairs (I had previously done this, full instructions for Ubuntu here)
mkdir ~/.ssh
chmod 700 ~/.ssh
ssh-keygen -t rsa
ssh-copy-id username#hostname
2) include the following in ~/.ssh/config
Host any_server_nickname
HostName hostname
User username
ForwardX11 yes
3) open emacs on local machine
4) C-x C-f
5) /any_server_nickname:dir/file.R for files in home directory or /any_server_nickname:/path/to/file.R
6) plot(1)
7) C-x C-b to evaluate entire buffer.