I need to run some commands on some remote Solaris/Linux servers and collect their output in a log file on my local server.
Currently, I'm using a simple Expect script, residing on the local server to fire the commands on the target system. I then redirect the output of the expect script to a log file, like this,
/usr/local/bin/expect script.exp >> logfile.txt
However, this is proving to be very unreliable as the connection to the server fluctuates a lot, leading to incomplete logs and hung scripts.
Is there a better and more reliable way to go about this task?
I have implemented fedorqui's answer,
Created a (shell) script that runs the required commands on the target servers.
Deployed this script to all servers.
Executed this script via expect, from my local (central) server.
Finally collected logs individually from each server after successful completion, and processed them.
The solution has been working fine without a glitch till now.
Related
I have been working on my R studio session hosted by a Linux server and recently, ran a piece of code that was taking way too long to execute and I decided to kill it.
Here is the sequence of steps that I took - none of them helped me restore the health of my session.
1) Hit the stop button on R studio and be patient.
2) Ssh into my Linux server and ran the following command to kill all the processes running with my userid
killall -u myuserid
3) Removed the.RData,.Renviron,.Rhistory files from my workspace.
4) Ran the following R command via the Linux server for garbage collection
gc(reset=TRUE)
4) Restarted the entire Linux server.
I am running out of ideas and would really appreciate any other suggestions before I take more drastic steps like revoking access and granting it again(not sure if that would be the right fix)
Note: The browser window freezes every time I login, and it happens only for my R studio session, the rest of the users in the same network have no issues.
I solved this problem - Rstudio-serverfreezing. I think it was a network problem since I couldn't receive any response from calling "~~~~~~.cache.js". In this case, you can find out "~~~~~~~~~.cache.js" no response with pushing key before you click log-in button.
Anyway, here is my way.
Reset your Network with following orders
you can insert these into cmd terminal as an admin mode.
netsh winsock reset
netsh int ip reset
Reboot
The IP information may be erased. So if you're using fixed IP address, fill the blanks with as-is IP address.
That's all.
You may follow this way to recover the connection.
I am performing an install on a server, which requires the server to reboot to complete the installation. My query is how to find out when the reboot completes so that I can run a basic smoke test on the server and confirm the deployment status.
This is a non-trivial task in general, and the solution will be depending on the operating system of the server that's restarted.
In general, it will be best to let the server decide itself on when the startup process has completed, and have it send some notification to interested entities.
If you can't do that, you can check for services that are typically available after reboot (like, HTTP port reachable, exported file shares mountable,...). Adding some additional time on top of that may be useful.
On the other, more unreliable end of possible solutions, you can simply wait for a certain amount of time that's typically required for completing a reboot.
If I am writing a program (in my case a python program using paramiko) that executes multiple Tasks on a remote Server, for example, remove some files, then later run a program, then later move some files, is it better to make a single ssh Client and leave it open for the duration, or should I treat ssh Connections more like files and databases and open and Close them each time I execute a Task?
We have a single threaded application and has to remain single threaded for some reasons. We have a large volume of runtime debug log using log4cxx. What's the best way to do the logging without impact the performance of the application?
I know log4cxx can send log to
* local files directly. This can slow down the application quite a lot. Profiling already confirmed this.
* sending to local syslog. I suppose that this may not be much different from sending to local files directly.
* sending to SocketAppender. I tested using log4j's SimpleSocketServer. The performance is no better than writing files directly. I haven't looked at source code for SimpleSocketServer. The most naive implementation is that SocketAppender is using blocking write, while SimpleSocketServer uses blocking reads from socket and only do next read when it finished writing current log entries. This will perform the same as logging to local file directly.
I guess that we'll have to write a logging server, with 1 threading reads from socket and put the msg on the queue and another thread writes the msg to log file.
There is likely such a free logging server somewhere on the internet. Before start searching for such a server or write one ourselves, are there any other solutions?
Thanks.
I have a long running job that I'd like to run using EC2 + RStudio. I setup the EC2 instance then setup RStudio as a page on my web browser. I need to physically move my laptop that I use to setup the connection and run the web browser throughout the course of the day and my job gets terminated in RStudio but the instance is still running on the EC2 dashboard.
Is there a way to keep a job running without maintaining an active connection?
Does it have to be started / controlled via RStudio?
If you make your task a "normal" R script, executed via Rscript or littler, then you can run them from the shell ... and get to
use old-school tools like nohup, batch or at to control running in the background
use tools like screen, tmux or byobu to maintain one or multiple sessions in which you launch the jobs, and connect / disconnect / reconnect at leisure.
RStudio Server works in similar ways but AFAICT limits you to a single user per user / machine -- which makes perfect sense for interactive work but is limiting if you have a need for multiple sessions.
FWIW, I like byobu with tmux a lot for this.
My original concern that it needed to maintain a live connection was incorrect. It turns out the error was from running out of memory, it just coincided with being disconnected from the internet connection.
An instance is started from the AWS dashboard and stopped or terminated from there also. As long as it is still running it can be accessed from an RStudio tab by copying the public DNS to the address bar on the web page and logging in again.