I am able to invoke sudo command from R / R Studio to provide specific permission to a path in ubuntu machine.
Code:
system('sudo -kS chmod -R 755 PATH',input="*********")
Here input is the password of sudo user. Now, I need to execute the same system command in Shiny App hosted in shiny server. By default shiny-server runs applications as the "shiny" user, which has lower privilege. What possible permission can be provided to "shiny" user so that this sudo code gets executed in shiny server as well.Thus , looking for ways to run system command from shiny server as well.
Try eco
system(paste0("echo -e 'yourinputhere\n' | sudo -kS chmod -R 755 PATH'))
sudo -s option runs the shell specified by the SHELL environment variable if it is set or the shell as specified.
Related
My shiny app needs to download file from ftp, I can download using R console. For example:
R console:
library(curl)
setwd("/srv/shiny-server/PowerSeq_app/www") curl_download("https://d396qusza40orc.cloudfront.net/getdata%2Fdata%2Fss06hid.csv",destfile="reviews2.csv")`
You can see the file is download in the directory:
However, when I deploy app using shiny server, the error occurs:
Shiny server:
Error in curl_download: Failed to open file /srv/shiny-server/PowerSeq_app/www/reviews2.csv.
I assumed this is file path issue, I also turned to this link, but I still cannot get it resolved. Many Thanks in advance.
Finally I realized it's user privilege issue, we have to change the privilege of user "shiny":
groupadd shiny-apps
usermod -aG shiny-apps shiny
chown -R shiny:shiny-apps /srv/shiny-server
chmod g+w /srv/shiny-server
chmod g+s /srv/shiny-server
I tried to dir.create on a path that I need sudo access to create directories. But I can't seem to do it from Rstudio server instance of Rstudio that I access from the browser, even though I started the Rstudio server using sudo rstudio-server start.
Is there a way to give my Rstudio instance sudo powers?
When you use the RStudio Server web client, it executes local scripts as the "rstudio-server" user on your Linux machine (run cut -d: -f1 /etc/passwd to list local users and it will appear there). You need to ensure that the rstudio-server user and the user you log onto the client with has rwx permissions across the directories you want to make changes in from RStudio.
sudo setfacl -m u:rstudio-server:rwx /path/
sudo setfacl -m u:localuser:rwx /path/
I built a Docker Image for an R Shiny App and ran the corresponding container with Docker Toolbox on Windows 10 Home. When trying to open the App with my web browser, only the index is shown. I don't know why the app isn't executed.
The log shows me this:
*** warning - no files are being watched ***
[2019-08-12T15:34:42.688] [INFO] shiny-server - Shiny Server v1.5.12.1 (Node.js v10.15.3)
[2019-08-12T15:34:42.704] [INFO] shiny-server - Using config file "/etc/shiny-server/shiny-server.conf"
[2019-08-12T15:34:43.100] [INFO] shiny-server - Starting listener on http://[::]:3838
I already specified the app host-to-container path by executing the following command which refers to a docker hub image:
docker run --rm -p 3838:3838 -v /C/Docker/App/:/srv/shinyserver/ -v /C/Docker/shinylog:/var/log/shiny-server/ didsh123/ps_app:heatmap
My Docker File looks like the following:
# get shiny serves plus tidyverse packages image
FROM rocker/shiny-verse:latest
# system libraries of general use
RUN apt-get update && apt-get install -y \
sudo \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev \
libxt-dev \
libssl-dev \
libssh2-1-dev
##Install R packages that are required--> were already succesfull
RUN R -e "install.packages(c('shinydashboard','shiny', 'plotly', 'dplyr', 'magrittr'))"
#Heatmap related packages
RUN R -e "install.packages('gpclib', type='source')"
RUN R -e "install.packages('rgeos', type='source')"
RUN R -e "install.packages('rgdal', type='source')"
# copy app to image
COPY ./App /srv/shiny-server/App
# add .conf file to image/container to preserve log file
COPY ./shiny-server.conf /etc/shiny-server/shiny-server.conf
##When run image and create a container, this container will listen on port 3838
EXPOSE 3838
###Avoiding running as root --> run container as user instead
# allow permission
RUN sudo chown -R shiny:shiny /srv/shiny-server
# execute in the following as user --> imortant to give permission before that step
USER shiny
##run app
CMD ["/usr/bin/shiny-server.sh"]
So when I address the docker ip and the assessed port in the browser, the app should run there, but only the index is displayed. I use the following line:
http://192.168.99.100:3838/App/
I'm glad for every hint or advice. I'm new to Docker, so I'm also happy for detailed explanations.
To use shiny with docker, I suggest you use the golem package. golem provides a framework for builing shiny apps. If you have an app developed according to their framework, the function golem::add_dockerfile() can be used to create dockerfiles automatically.
If you are not interested in a framework, You can still have a look at the source for add_dockerfile() to see how they manage the deployment. Their strategy is to use shiny::runApp() with the port argument. Therefore, shiny-server is not necessary in this case.
The Dockerfile in golem looks roughly like this
FROM rocker/tidyverse:3.6.1
RUN R -e 'install.packages("shiny")'
COPY app.R /app.R
EXPOSE 3838
CMD R -e 'shiny::runApp("app.R", port = 3838, host = "0.0.0.0")'
This will make the app available on port 3838. Of course, you will have to install any other R packages and system dependencies.
Additional tips
To increase reproducibility, I would suggest you use remotes::install_version() instead of install.packages().
If you are going to deploy several applications with similar dependencies (for example shinydashboard), it makes sense to write your own base image that can be used in place of rocker/tidyverse:3.6.1. This way, your builds will be much quicker.
Minimal Reproducible Example
Create an empty directory (can be called anything you want)
Inside it, create two things:
i. A file called Dockerfile
ii. An empty directory called app
Place your shiny app inside the directory called app.
Your shiny app can be a single app.R file, or, for older shiny apps, ui.R and server.R. Either way is fine (see here for more on that).
If unsure about any of the above, just copy the files found here.
Place this in Dockerfile
FROM rocker/shiny:latest
COPY ./app/* /srv/shiny-server/
CMD ["/usr/bin/shiny-server"]
In the terminal, cd to the root of the empty directory you created in step 1, and build the image with
docker build -t shinyimage .
Run the container with
docker run -p 3838:3838 shinyimage
Finally, visit this url to see your shiny app: http://localhost:3838/
Here's a copy of all of the above in case anything's unclear.
Check the logs for any useful information? And exec into the container to verify if the App content is copied to the correct location.
Because the way /App content is copied looks incorrect
The content of /App is copied into the image to /srv/shiny-server/App during the build phase and you are trying to override /srv/shiny-server content using -v option when running the container.
Looks like during runtime the App data copied is being overwritten.
Try without -v /C/Docker/App/:/srv/shinyserver/ or use -v /C/Docker/App/:/srv/shinyserver/App/
docker run --rm -p 3838:3838 -v /C/Docker/shinylog:/var/log/shiny-server/ didsh123/ps_app:heatmap
I have a containerized R Shiny app that, when run locally with docker run --rm -p 3838:3838 [image] works as expected. The "landing" page appears when I go to localhost:3838 and all is good. However, when this container is deployed to AWS Fargate, things break down. The container appears to start and run, but there's no webpage being served on 3838 even though all ports are pointed to 3838 in Fargate.
I'm using dockerfile
FROM rocker/verse:3.5.1
LABEL Steven "email"
## Add shiny capabilities to container
RUN export ADD=shiny && bash /etc/cont-init.d/add
## Update and install
RUN tlmgr update --self
RUN tlmgr install beamer translator
## Add R packages
RUN R -e "install.packages(c('shiny', 'googleAuthR', 'dplyr', 'googleAnalyticsR', 'knitr', 'rmarkdown', 'jsonlite', 'scales', 'ggplot2', 'reshape2', 'Cairo', 'tinytex'), repos = 'https://cran.rstudio.com/')"
#Copy app dir and them dirs to their respective locations
COPY app /srv/shiny-server/ga-reporter
COPY app/report/themes/SwCustom /opt/TinyTeX/texmf-dist/tex/latex/beamer/
#Force texlive to find my custom beamer thems
RUN texhash
EXPOSE 3838
## Add shiny-server information
COPY shiny-server.sh /usr/bin/shiny-server.sh
COPY shiny-customized.config /etc/shiny-server/shiny-server.conf
## Add dos2unix to eliminate Win-style line-endings and run
RUN apt-get update && apt-get install -y dos2unix
RUN dos2unix /usr/bin/shiny-server.sh && apt-get --purge remove -y dos2unix && rm -rf /var/lib/apt/lists/*
RUN ["chmod", "+x", "/usr/bin/shiny-server.sh"]
CMD ["/usr/bin/shiny-server.sh"]
and shiny-server.conf
# Instruct Shiny Server to run applications as the user "shiny"
run_as shiny;
# Define a server that listens on port 3838
server {
listen 3838;
# Define a location at the base URL
location / {
# Host the directory of Shiny Apps stored in this directory
app_dir /srv/shiny-server/ga-reporter;
# Log all Shiny output to files in this directory
log_dir /var/log/shiny-server;
# When a user visits the base URL rather than a particular application,
# an index of the applications available in this directory will be shown.
directory_index on;
}
}
with shiny-server.sh
#!/bin/sh
# ShinyServer: Make sure the directory for individual app logs exists
mkdir -p /var/log/shiny-server
chown -R shiny.shiny /var/log/shiny-server
# RUN ShinyServer
exec shiny-server >> /var/log/shiny-server.log 2>&1
I have edited the .conf file to display the app (i.e., location /) in /srv/shiny-server/ga-reporter which is also where I've copied the app_dir in the Dockerfile. Shiny is listening on port 3838 and should serve the page there. Again, this happens locally but not when deployed to AWS Fargate. I've tried logging Shiny logs to stdout by using the first answer provided here but have had no luck seeing any errors generated. Server "health checking" is only offered in the "pro" version so I can't check to see if the server is actually running.
On AWS, the container starts and appears to function normally (i.e., the "normal" start up logs appear):
but there is simply no page displayed at the location I expect it to be served.
I found another Shiny app that is on dockerhub running under the same configuration as the Fargate cluster but have had no luck trying to implement anything in the shiny-server.conf or the shiny-server.sh files located there.
What am I missing? Everything on Fargate is pointed to listening on 3838; there must be something I'm missing in the .conf file for this to be failing when deployed.
EDIT
I can't bash in to the running container on Fargate because I don't have access to the server on which docker is running.
Fargate has a UI that accepts host and container ports:
EDIT 2 (2018-08-27)
The engineer that was deploying this has been able to resolve the issue:
"it was the port change, I forgot to can the port on the ALB’s security group, I only updated the cluster’s inbound rules
so the cluster was allowing connections, but the ALB security group wasn’t letting it out"
So I've set up an Azure Data Science Virtual Machine on Linux (Ubuntu) and I've executed the following on the terminal to enable Remote R workspace, RStudio Server, R Server Operationalization and hadoop:
sudo apt update
sudo apt -y upgrade
# Hadoop is installed but doesn't seem to appear on the PATH or have its environment variable set by default
sudo echo "" >> ~/.bashrc
sudo echo "export PATH="'$'"PATH:/opt/hadoop/hadoop-2.7.4/bin" >> ~/.bashrc
sudo echo "export HADOOP_HOME=/opt/hadoop/hadoop-2.7.4" >> ~/.bashrc
#
source ~/.bashrc
#Setting up a password as none exists to begin with because of private key selection in the installation
#RStudio Server requires a password though
"MyPassword\nMyPassword\n" | sudo passwd sshuser
#Unfortunately hadoop fails on Data Science Virtual Machine
#error: mkdir: Call From IM-DSonUbuntu/192.168.5.4 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
# hadoop fs -mkdir /user/RevoShare/rserve2
# hadoop fs -chmod uog+rwx /user/RevoShare/rserve2
sudo mkdir -p /var/RevoShare/rserve2
sudo chmod uog+rwx /var/RevoShare/rserve2
# hadoop fs -mkdir /user/RevoShare/sshuser
# hadoop fs -chmod uog+rwx /user/RevoShare/sshuser
sudo mkdir -p /var/RevoShare/sshuser
sudo chmod uog+rwx /var/RevoShare/sshuser
#Setting up R Server Operationalisation
cd /opt/microsoft/mlserver/9.2.1/o16n
sudo dotnet Microsoft.MLServer.Utils.AdminUtil/Microsoft.MLServer.Utils.AdminUtil.dll -silentoneboxinstall MyPassword
#They say this Data Science Virtual Machine already has RStudio Server, but even though the port 8787 is open, it's nowhere to be found! So installing it now, and after the installation it's accessible by refreshing the page that failed before.
#Perhaps it's not installed then? Or a service is not running like it shoudl?
#https://www.rstudio.com/products/rstudio/download-server/
wget https://download2.rstudio.org/rstudio-server-1.1.414-amd64.deb
yes | sudo gdebi rstudio-server-1.1.414-amd64.deb
#They are small, leave them for debug reasons - lets have evidence the script run thus far.
#sudo rm rstudio-server-1.1.414-amd64.deb
# Remote R workspace Service needs dotnet sdk
curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
sudo mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
sudo sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial main" > /etc/apt/sources.list.d/dotnetdev.list'
sudo apt update
sudo apt -y install dotnet-sdk-2.0.0
sudo apt install libxml2-dev
#Downloading and installing the Remote R service
wget -O rtvs-daemon.tar.gz https://aka.ms/r-remote-services-linux-binary-current
tar -xvzf rtvs-daemon.tar.gz
sudo ./rtvs-install -s
sudo systemctl enable rtvsd
sudo systemctl start rtvsd
#sudo rm rtvs-daemon.tar.gz
#sudo rm rtvs-install
#Fixing Remote R: For some reason, even though 'sudo systemctl enable rtvsd' runs, after every reboot the service won't become automatically active. So let's fix that.
wget https://sa0im0general.blob.core.windows.net/general-blob-container/StartRemoteRAfterReboot.sh
sudo mv StartRemoteRAfterReboot.sh /var/RevoShare/StartRemoteRAfterReboot.sh
sudo /sbin/shutdown -r 5
sudo chown root /etc/rc.local
sudo chmod 755 /etc/rc.local
sudo systemctl enable rc-local.service
sudo -s
sudo find /etc/ -name "rc.local" -exec sed -i 's/exit 0//g' {} \;
sudo echo "" >> /etc/rc.local
sudo echo "sh /var/RevoShare/StartRemoteRAfterReboot.sh" >> /etc/rc.local
sudo echo "exit 0" >> /etc/rc.local
exit
I've also tried, one by one, these, to see if it makes any difference to the RStudio Server (it didn't, but even if it did, I want a global solution to work on Remote R Workspace Service and R Server Operationalisation as well, not only RStudio Server):
#Configuring RStudio Server to see the R Server R
sudo echo "rsession-which-r=/opt/microsoft/mlserver/9.2.1/bin/R/R" >> /etc/rstudio/rserver.conf
export RSTUDIO_WHICH_R=/opt/microsoft/mlserver/9.2.1/bin/R/R
sudo echo "RSTUDIO_WHICH_R=/opt/microsoft/mlserver/9.2.1/bin/R/R" >> ~/.profile
source ~/.profile
sudo echo "RSTUDIO_WHICH_R=/opt/microsoft/mlserver/9.2.1/bin/R/R" >> ~/.bashrc
source ~/.bashrc
sudo echo "PATH=$PATH:/opt/microsoft/mlserver/9.2.1/bin/R" >> ~/.bashrc
export PATH=$PATH:/opt/microsoft/mlserver/9.2.1/bin/R
source ~/.bashrc
The problem is that even though "which R" points to R Server's R, i.e. typing "sudo R" will show the message "Loading Microsoft R Server packages, version 9.2.1." and will load packages like RevoScaleR, everything else fails to do so.
Accessing the RStudio Server with http://THE-IP-GOES-HERE.westeurope.cloudapp.azure.com:8787 and logging in with the initial user ("sshuser") (or with any other user for that matter) will NOT load R Server and RevoScaleR rx functions are unavailable
Using my local Visual Studio 2017 to access the remote workspace via "Add connection" on "Workspaces" tab loads MRO and says:
Installed R versions:
[0] Microsoft R Open '3.4.1.1347' (Default)
And finally, when I use R Server's Operationalisation and log in with "mrsdeploy" package's "remoteLogin()" R Server packages like RevoScaleR are not loaded again, so things like "rxSummary(~., data=iris)" fail with error 'could not find function "rxSummary"'
The exact same thing happened when I deployed from azure a "Machine Learning Server 9.2.1 on Linux (Ubuntu)".
I don't want to just use the regular open source R, I want to be able to use the R Server - that's why I deployed this VM. How can I make it so that everything loads R Server's R, not Microsoft R Open? (Like I'm able to do from terminal using "R")
As a result of my having tried all of this and the fact that R Server is loaded in the console, my mind now goes to permissions. Could it be that by default the Data Science VM doesn't have the correct permissions to allow these?
I'm at a loss
RStudio Server is installed on the Ubuntu DSVM, but the service is disabled by default as it does not support SSL. You can enable it with systemctl enable rstudio-server, then start it with systemctl start rstudio-server.
RStudio Server uses the same R as Microsoft R Server, but the .libPaths are different, which is why you cannot load the MRS packages. You will need to manually set the .libPaths so they match.