I want to use a little bit more GPU memory than my local Nvidia 2GB GPU.
The project is hosted on Gitlab and I am able to checkout the project via git commands through a notebook.
But then the troubles start:
The project is checked out to a folder called /content/
In that project-root there are several other folders, namely Python packages. They can not be found by the Python interpreter, even though I manipulated sys.path.
After the checkout, I changed the type of the session from CPU to GPU. The previously checked out project has gone. After setting the session type back to CPU the folder project-root was again available in /content. When checking out to session GPU, I get frequent warings that I should switch back to CPU because I do not use a GPU. Well, still trying to get the project running on Collab.
When I have Jupyter notebooks in the project-root, I can not open and run them. Notebooks need to be placed in some Collab notebooks folder. How can I use notebooks from a git-controlled project? Which use own Python packages which are available in project-root?
How can I address at least issue #1 and #3?
Related
I'm trying to get a shiny app deployed on Shiny Server. I can do that without any issues, but when trying to deploy an app that has a number of dependencies (remote and local) we keep running into issues.
We used renv to track the dependencies (on the Windows dev box) and rebuild it from scratch on the Linux prod box, but even though the dependencies are rebuilt and some get loaded, others do not. The .Rprofile of the user running the app is pointing to the renv activation script.
For the sake of clarity, we need and want all the R code to be built from the source code on the Linux box.
What is the best or standard way (or even a poor way that works) to deploy the libraries for a shiny app to the shiny server? Is renv even the right tool for this scenario or is there a better tool?
I've tried reading the shiny server documentation and the closes it only mentions that it uses the .Rprofile of the user running the app, but there doesn't seem to be any sort of guide on the best way to deploy dependent libraries.
This renv documentation discusses some reproducibility caveats:
system dependencies, and
changes in CRAN (e.g. a binary no longer being available).
Since you are moving from a Windows to a Linux system your packages may have unmet system dependencies (things that need to be installed outside of R) that you didn't encounter in Windows. For example, rJava is required for some of the Excel-related R packages, and getting its related system dependencies installed and working on Linux can sometime be a challenge. You can use the RStudio Package Manager Website to figure out what system dependencies are required for different R packages for your particular Linux OS. Also, the error messages you get when running these apps on Linux should point you in the right direction. These system dependencies are what you'll have to manage yourself since renv doesn't.
But for a more production-level solution you can try Docker and ShinyProxy. For apps with many dependencies or especially external dependencies (e.g. Python, SQL, etc.) you can guarantee more reproducibility using Docker. ShinyProxy can be used to host apps built into docker images. This is more work, but you ensure the entire system is reproducible, not just the R version and R packages. ShinyProxy also adds additional hosting capabilities like user authentication.
I am trying to share my Jupyter notebook with my team and want us all to coauth on the same notebook? Is it possible ?
Yes, JupyterLab 3.1 introduces Real Time Collaboration mode:
Ensure you have JupyterLab 3.1 or newer installed, and for convenience install jupyterlab-link-share:
pip install -U "jupyterlab>=3.1" jupyterlab-link-share
Add c.LabApp.collaborative = True setting to your config file (which is
jupyter_server_config.py or jupyter_notebook_config.py depending on the server you use)
After restarting JupyterLab open the new Share menu and choose Share Jupyter Server Link, copy the link and send it to your collaborator. Of course, the serve needs to be accessible for them (in the same network, or publicly available).
You can try it out on binder: (here using this public gist).
JupyterHub users may need to perform additional configuration (as discussed on the discourse here) because there are additional permissions/authentication questions to be dealt with.
Can R be run from a CD-ROM drive? The computer is a stand-alone (no network or Internet connection) and I can't install anything on it, nor can I use a flash drive.
Thanks.
What do you mean by "can't install"?
You don't need to install R, you can just run it from a folder copied from somewhere else. If you have hard disk storage on the PC then you can copy C:\Program Files\R from one machine onto a CD-ROM, then take the CD-ROM to the cripplebox, copy it to wherever you store your files and run it from there. Worst case scenario is you have to change the R_HOME environment variable. Works for Linux and Windows (you didnt say what OS you are on).
...unless your sysadmins have disabled executable permissions for your hard disk storage. Which is a real BOFH thing to do.
...but if they've done that I'd also suspect they've disabled executables from CD-ROM too.
...and if you don't have any writable hard disk storage, how the heck are you going to do any analysis?
...the real fix may be to kick the sysadmins until you tell them you can't do your job without R installed on the machine.
You may have trouble with packages, but otherwise, the instructions for installing R on a USB key should be pertinent.
I have an R script which I want to deploy so that it's idiot-proof, one click runs it etc. Unfortunately I don't have the means to pay for a server, and the environment in which it needs to run does not allow the installation of new software, only portable style apps can be run. (School computers) My script also relies on several non-base packages.
Is there any way to deploy R and my script in an easy to run way so it can be used off a usb stick?
You can install R on a USB drive and use it on any computer running the same OS. If you're using Windows, see question 2.6 of the R for Windows FAQ.
If you made the USB stick a bootable disk environment (say linux) with R installed on it, you could boot off it and do it that way.
I am developing OpenCL code on a linux cluster through SSH -
are there any tools that would make this process easier, i.e.
something like NVIDIA Parallel Nsight for OpenCL ?
No there is no such tool, though you might try developing your code using ordinary computer and post production versions there..
If the computer where you perform development is also running Linux, you can easily mount a remote folder as local. In a Gnome environment, open Nautilus (the file manager), click File => Connect to server, chose SSH, fill the required parameters, and you have a remote folder as local.
You can then use any IDE you want to develop code, and maybe perform simple runs, tests and debugs if the OpenCL tools (compiler, debugger) you're using remotely are also installed locally. However, To compile and properly run the code on the cluster, you need to use the ssh client on the command line.