I am looking for the best way to execute code on a distant machine. Ideally, I am looking for a solution such as Cuda which provides the opportunity to allocate executions on GPU or CPU, but for distinct machine.
I tried distinct ways to do that :
I connect my machines with ssh, export my script, execut it. No particular issue, but not very handy. But maybe this solution could be optimise. Because I open my ssh connection with the terminal, or termius.
I try another way with mosh, same outcomes, but quicker.
Currently, I am working on a Spyder kernel to have a direct link in the place of execution.
I've seen there is also a possibility with a nohup connection, but I have to work on this solution to understand well the possibilities.
Everything works well, but I am looking for a more convenient solution.
Thank you in advance for your answers !
You could either use sshfs along to ssh to mount the remote filesystem on your machine it's easier than always copy the code by hand, if so I would recommend to use screen or something like that that if the connection breaks it offers no problems.
Personal I like to work with Visual Studio Code and the ssh fs extension for this purpose.
An other alternative is to work with X2Go. X2Go enables you to access a graphical desktop of a computer over a low bandwidth (or high bandwidth) connection.
Related
I have R Studio on Windows. It works fine before I connect to the internet through my VPN. After I connect commands start to hang, autocomplete can take forever, and simple operations like 4 + 4 can take one minute or more.
I have a feeling the studio is making connections under the hood. I would like to disable all of these connections no matter what.
I have experienced the same thing - I am assuming your actual work is on a file server and not local. I found the culprit in my case to not be RStudio directly, but rather the RStudio project file. RStudio will create a generally small, hidden folder in the directory named .Rproj.user with some settings. This, living on the file server, caused constant read/writes through my VPN connection.
I unfortunately had to either (a) move projects off the file server into a local copy (not bad since I can use a company GitLab), or (b) delete the .Rproj and .Rproj.user folders from the project directory on the server and use here::here() or something like that as a work around in my workflow.
As another possibility, I have seen installations of R itself done onto a personal server drive instead of locally. This has been done to avoid needing administrator privileges to install. This is not a great idea and can also result in extremely slow performance through a VPN connection. You can check to see where R is installed as well. Sounds like this is not the problem though based on what you described.
Maybe it is something else, but this is what I found last week for me based on a very similar experience.
I am trying to implement a Selenium test to perform automated actions on a website (looping through pages). I am using R and RSelenium package as well as a PostgreSQL database using DBI package. All this using EC2 AWS server.
My problem is that after a few minutes that the script was launched, my RStudio session freezes (as well as my Linux session) and I can see a message like "cannot allocate memory".
So this is clearly a memory issue without a doubt, and by doing top I could see that my Selenium docker was using most of the resources.
But my question is how can I reduce the amount of memory used by the Selenium test?
IMHO there is no practical way for a test to use less memory than the memory required by the given test. You can try to simplify the given test by breaking it up into 2 or more tests. Check for memory leaks, as suggested in another answer.
It would be much easier to use the next largest instance type with more memory, and shut down the instance when not in use to save money, if that is an issue.
Don't forget drive.close() in your code, if you don't close your driver, you will have a lot instance of Chrome.
I have access to (not authority over) a computing cluster which has R installed. Is there a way for me to use R-Studio on my local computer -- but have the code running on the cluster via SSH?
To clarify -- No I don't really have non-SSH access, no I can't install R-Studio (server or desktop) on the cluster.
In line with the hackish options #hrbrmstr mentioned...
If your aim is to run mostly non-interactive code, then you can probably establish an n-node parallel::makePSOCKcluster() on the remote machines and run each of your commands via parallel like commands. Similarly, you could use package::svSocket, see this neat demo on YouTube for more details than fit in a reasonable answer.
But, given that you said RStudio, I suspect you are thinking of interactive use, and the above would be doable (but painful). Nothing I know of will let you just pretend that the remote machine is the local machine (which is a pity to be sure). However you might be able to hack something together, with sink() etc and a server and client side loop, e.g. How to connect two computers using R?.
I need to automate actions I do on a Remote Desktop. The au3recoder (which comes with Autoit) doesn't seem to record anything on the RDP.
Yes, Autoit works on RDP machines if you execute the file on that machine. If you want to use it from your own PC over the RDP connection that could get difficult.
You can't detect windows over RDP what makes most of the AutoIt functions useless.
You could click on positions with MouseClick(). It's not the most elegant version but it works for little things.
The Au3 Recorder never worked for my anyways, so you could also hardcode it yourself.
I recomend that you copy a file to your RDP machine to use all AutoIt functions.
Of course it is possible.
And you could use most of the "AutoIt" functions because they are C++ functions which windows have them all.
The most accurate way to do it is to use CMD through AutoIt to get most of the informations.
However the most elegant way would be using PowerShell.
I'm trying to setup automated GUI tests in ESXi Virtual Machines using TestComplete. The problem, as I understand it, is that when no remote desktop connection is made to the ESXi virtual machine, then it is impossible for TestComplete to perform screen captures and therefore automate the GUI testing. As far as I understand it, this is due to the fact that Windows does not generate any user interface when nobody is viewing it.
I'm sure other have experienced this problem. How did you solve it ? Are you using a third party computer which automatically launch remote desktop connections prior to running the tests ?
Would it be possible to launch a remote desktop from a head-less virtual machine to another to fake somebody viewing ?
Any other smarter solutions I haven't thought about ?
You should be able to log in to Windows on the VM's console using the vSphere client, then close vSphere, and Windows will still believe the user is viewing the console. Simple as that. :)
So there shouldn't be a need to involve remote desktop in the mix.
As long as your tests then run as that logged-in Windows user, you should be fine.
This technique has always worked like a charm for me with certain Watir, Selenium, and MS UI Automation tests that depend on having an interactive desktop.
If you need to reboot the VM automatically before/during the test, instead of logging in manually in the vSphere client, you can make Windows log in as an arbitrary user automatically - check the "control userpasswords2" command, or you can use the Sysinternals app "Autologin":
http://technet.microsoft.com/en-us/sysinternals/bb963905
Only catch with this technique is that you need to be able to launch your tests while not viewing the console on the VM, but it sounds like you've already taken care of that?
If you need a solution for launching your tests remotely, I highly recommend using Jenkins or Hudson to kick off tests/collect results from the VM. Jenkins has changed my life in this regard.
You may consider using the Network Suites functionality of TestComplete:
http://smartbear.com/support/viewarticle/16849/
It can open Remote Desktop connections on its own, control tests on remote PCs, and pull the logs back to the "master" project. This feature is designed to be used for distributed tests, and looks like it's just what you need.
As for opening RDP to a head-less VM, it should not be a problem - it's up to Windows to "think" about this. You just open RDP and it works even if there is no monitor attached to the remote PC/VM.
I hope this helps,
Alex
You can always use VNC with checking the option "Do nothing" when disconnecting viewer. This way you'll trick windows to generate the image.