Ubuntu 10.10 64bit athalon, gnome
My basic scenario is I'm connecting to a VPN service (via newtworkmanager pptp protocol) and I'm transferring private data (hence VPN). The service goes down intermittantly and that's alright, probably due to my ISP/OS/VPN. What is not good is that my applications will then continue to transmit data via the eth0 default route and thats not cool. After some looking around I'm suspecting the best way to deal with this is to post scripts into /etc/NetworkManager/dispatcher.d. In short, the networkmanager service will execute scripts in this directory (and pass arguments to the scripts) when anything about the network changes.
My problem is that I can't get any of my scripts to execute. They all have, per the manpage, 0755 permissions and owned by root, but when I change the network state by unplugging ethernet cable, my scripts don't execute. I can execute them from the command line, but not automatically via the dispatcher....
an example script:
#!/bin/sh -e
exec /usr/bin/wmctrl -c qBittorrent
exit 0
This script is intentionally simple for testing purposes..
I can post whatever else would be helpful.
i'm using the syntax killall -9 any_application_name_here and that's working just fine. I imagine the script didn't have access to the binary wmctrl. I think that bash interpreter in this case will only execute bash binaries.
So, in a nutshell, if you want to control your VPN traffic based on network events, one way is to post scripts to /etc/NetworkManager/dispatcher.d and use binaries that are in bash's default path.
Related
I would like to connect via ssh to certain equipment in a network.
The requisites are:
It must run a command and capture the output of the ssh session in R (or in bash, or any other programming language, but I would prefer it in R language)
It must enter a plain-text password (as this equipment hasn't been accessed before, and can't be changed with a rsa keypair), so the ssh.utils package doesn't meet this requirement
sshpass can't be used, as I have noticed that it doesn't work for some devices I tested.
I've read all this posts but I can't find an effective way to perform it: link 1, link 2, link 3, link 4
I know the requirements are hard to accomplish, but thank you for your effort!
EDIT:
Sorry if I didn't make myself understandable. I mean I work locally in R and I want to connect to +3000 devices in all of my network via ssh. It is Ubiquiti equipment, and the only open ports are 80 and 22.
If ssh doesn't work, I will use the RSelenium package for R and extract info from port 80. But first I will try with ssh pory 22 as it is a lot more efficient than opening an emulated browser.
The big problem in all these Ubiquiti equipment is that they have a password to log in. That's why requisite No.2 is needed. When I must enter a server that I know, I spend time setting up the rsa keypair so that I don't have to enter a password everytime I connect to a specific server, but it's impossible (or at least, for me it's impossible) to configure all +3000 Ubiquiti equipment with these keypairs.
That's why I don't use snmp, for example, as this equipment maybe they have it activated or not, or the snmp configuration is mistaken. I mean, I have to use something that's activated by default, and in a way, ordered. And only port 80 and port 22 are activated and I know all the user's and password's equipment.
And sshpass is an utility in UNIX/Linux like this link explains that works for servers but doesn't work for Ubiquiti equipment, as long as I've tested it. So I can't use it.
The command I need to extract the output from is mca-status. Simply by entering that into the console makes it print some stats I will like to get from the Ubiquiti equipment.
Correct me, please, if I am wrong in something I've posted. Thanks.
I think you have this wrong. I also have no idea what you are trying to say in point 2, and I have not idea what point 3 is supposed to say.
Now: ssh is a authentication mechanism allowing you (trusted) access to another machine and the ability to run a command. This can be as simple as
edd#max:~$ ssh bud Rscript -e '2+2'
[1] 4
edd#max:~$
where I invoke R (or rather, Rscript) on the machine 'bud' (my desktop) from a session on the machine 'max' (my server). That command could be anything including something which writes to temporary or permanent files. You can then retrieve those files via scp.
Authentication is handled independently -- on Unix we often use ssh-agent which run in the background and against you authenticate on login.
Finally I solved it using the rPython package and the python's paramiko module, as there was no way to do it purely via R.
library(rPython)
python.exec(python.code = c("import paramiko",
"ssh = paramiko.SSHClient()",
"ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())",
sprintf('ssh.connect("%s", username="USER", password="PASSWORD") ', IP),
'stdin, stdout, stderr = ssh.exec_command("mca-status")',
'stats = stdout.readlines()'))
I work with another company that is moving from regular FTP to SFTP for their connections. They have informed me that my existing username and password are the same, but that I now need to connect via SFTP on port 22. I have a couple of PHP scripts that do some basic things, like connect to their site, get a directory listing, and upload and download a file each day. Those all work fine on FTP, so I need to just swap out the protocol to SFTP.
After doing some research, the consensus seems to be that phpseclib is the easiest and most robust way to perform SFTP using PHP. My server is running Linux and Apache. I downloaded the library and tried to run just the basic example given by phpseclib, but I get an error:
Notice: SSH_AUTH_SOCK not found in System/SSH/Agent.php on line 244
When I look at Agent.php, I see that the script is looking for SSH_AUTH_SOCK to be defined in either $_SERVER or $_ENV. I must be missing something obvious, but I have no idea how to get past this error.
You are trying to connect to SSH authentication agent (ssh-agent).
When the agent is run, it exports the SSH_AUTH_SOCK environment variable.
What is obviously not happening in your case.
But you didn't tell us, why are you trying to use the agent, if at all. And what did you do to set it up, if anything. So it's difficult to give you a more concrete advice.
See also How is SSH_AUTH_SOCK setup and used by ssh-agent?
Run the bellow command:
eval ssh-agent -s
It works fine for me.
I have always used the os:cmd/1 method to call operating system routines. Now, i know that erlang has an ssh application. I would like to know how i can use this module to ssh into a SOLARIS server, run a command and collect the reply. I believe that such an operation would be handled asynchronously. I need an example using the ssh application built into Erlang doing this:
Now, at times we setup SSH KEYS between servers to prevent password prompt especially if one is using a script to execute tasks on remote servers. i am intending to write many Erlang programs or escripts that will interact with many remote servers within our environment. i need a complete example and explanation on how ssh with and/or without password prompt can be handled using erlang ssh application. NOTE: In the screen shot above, the two servers had SSH KEYS set up and so there is no password prompt when ssh is initiated from any of the two.
The correct erlang native API to achieve this is not ssh, which only implements a user-interactive shell for ssh, but instead use ssh_connection. Take a look at ssh_connection:exec/4
To be more complete, use ssh:connect to establish a connection and then using the handler returned from it to connect with ssh_connection:exec/4
I didn't try it myself and can't provide a complete example but the documentation seems to be a good starting point.
Every month we send reports to a server using FTP. We run a query on a database to create the files then use the ftp functionality in LabVIEW to do the transfer. This runs on a Windows system.
This works fine but now we have to switch to using SFTP and the CopSSH package has been recommended. As LabVIEW has no native SFTP functionality we are looking at how we can use the sftp.exe application from CopSSH.
From the command prompt we have set up the encryption and made the initial connection using sftp username#host and entered the password. This has been confirmed by the team on the server side so connection to the server is set up. Now we just use sftp username#host and no password is required.
Where we are struggling is how to initiate the transfer from our LabVIEW code. We are able to call system commands using the System Exec VI but is there a way to pass a list of functions to the SFTP executable?
The commands used to transfer the files when we type it at the command prompt are:
sftp username#host
put c:/Data/File1.txt remoteFile1
put c:/Data/File2.txt remoteFile2
put c:/Data/File3.txt remoteFile3
quit
This works from the command prompt but I am looking to just call the sftp executable with a list of files to transfer. I don't think this would be specific to LabVIEW as you could use a batch file to run from a scheduled job.
LabVIEW can call ActiveX and .net but we really need to use this specific application.
I have been using WinSCP which has a command line version, winscp.com. It supports sftp and allows synchronize, keepuptodate, get, put and delete on folders and files. One word of warning, keepuptodate depends on an unbroken connection. Although WinSCP can remake a connection automatically, keepuptodate cannot. I suspect it is based on Microsoft's .NET SystemIO FileSystemWatcher. I therefore do a regular synchronize to keep a mirror of my source folder tree on the remote target.
If copssh's sftp.exe is a command line utility, and System Exec in your version of LabVIEW has the 'standard input' terminal (present at least since 8.5), you should be able to simply wire the commands you want sftp.exe to run into the standard input terminal.
If that doesn't work for some reason, could you use PuTTY instead of copssh? The documentation for PuTTY's PSFTP component says that it can execute a sequence of commands in a script file using the -b command line switch, e.g.
psftp user#hostname -b myscript.scr
so you could have your LabVIEW program create the script file then run it with System Exec.
You are mixing SSH and SFTP. SSH opens a secure connection, but SFTP is a separate protocol which is run over SSH connection and requires a separate tunnel. In OpenSSH (and it's Windows Port, copSSH) it's sftp.exe application that does SFTP.
Now about FTP vs SFTP. Please check an article that explains the difference between SFTP and FTP(S). If LabView supports FTP, this doesn't help you when you need to perform SFTP transfers.
I don't know whether you can use external ActiveX controls in LabView. If you can, you are welcome to check our SFTP ActiveX control, that will let you do the transfer. If all you can do is call external application, then you'd have to use copSSH's sftp.exe.
I created a program that iterates over a bunch of files and invokes for some of them:
scp <file> user#host:<remotefile>
However, in my case, there may be thousands of small files that need to transferred, and scp is opening a new ssh connection for each of them, which has quite some overhead.
I was wondering if there is no solution where I keep one process running that maintains the connection and I can send it "requests" to copy over single files.
Ideally, I'm looking for a combination of some sender and receiver program, such that I can start a single process (1) at the beginning:
ssh user#host receiverprogram
And for each file, I invoke a command (2):
senderprogram <file> <remotefile>
and pipe the output of (2) to the input of (1), and this would cause the file to be transferred. In the end, I can just send process (1) some signal to terminate.
Preferably the sender and receiver programs are open source C programs for Unix. They may communicate using a socket instead of a pipe, or any other creative solution.
However, it is an important constraint that each file gets transferred at the moment I iterate over it: it is not acceptable to collect a list of files and then invoke one instance of scp to transfer all the files at once at the end. Also, I have only simple shell access to the receiving host.
Update: I found a solution for the problem of the connection overhead using the multiplexing features of ssh, see my own answer below. Yet, I'm starting a bounty because I'm curious to find if there exists a sender/receiver program as I describe here. It seems there should exist something that can be used, e.g. xmodem/ymodem/zmodem?
I found a solution from another angle. Since version 3.9, OpenSSH supports session multiplexing: a single connection can carry multiple login or file transfer sessions. This avoids the set-up cost per connection.
For the case of the question, I can first open a connection with sets up a control master (-M) with a socket (-S) in a specific location. I don't need a session (-N).
ssh user#host -M -S /tmp/%r#%h:%p -N
Next, I can invoke scp for each file and instruct it to use the same socket:
scp -o 'ControlPath /tmp/%r#%h:%p' <file> user#host:<remotefile>
This command starts copying almost instantaneously!
You can also use the control socket for normal ssh connections, which will then open immediately:
ssh user#host -S /tmp/%r#%h:%p
If the control socket is no longer available (e.g. because you killed the master), this falls back to a normal connection. More information is available in this article.
This way would work, and for other things, this general approach is more or less right.
(
iterate over file list
for each matching file
echo filename
) | cpio -H newc -o | ssh remotehost cd location \&\& | cpio -H newc -imud
It might work to use sftp instead of scp, and to place it into batch mode. Make the batch command file a pipe or UNIX domain socket and feed commands to it as you want them executed.
Security on this might be a little tricky at the client end.
Have you tried sshfs?
You could:
sshfs remote_user#remote_host:/remote_dir /mnt/local_dir
Where
/remote_dir was the directory you want to send files to on the system you are sshing into
/mnt/local_dir was the local mount location
With this setup you can just cp a file into the local_dir and it would be sent over sftp to remote_host in its remote_dir
Note that there is a single connection, so there is little in the way of overhead
You may need to use the flag -o ServerAliveInterval=15 to maintain an indefinite connection
You will need to have fuse installed locally and an SSH server supporting (and configured for) sftp
May be you are looking for this:
ZSSH
zssh (Zmodem SSH) is a program for interactively transferring files to a remote machine while using the secure shell (ssh). It is intended to be a convenient alternative to scp , allowing to transfer files without having to open another session and re-authenticate oneself.
Use rsync over ssh if you can collect all the files to send in a single directory (or hierarchy of directories).
If you don't have all the files in a single place, please give some more informations as to what you want to achieve and why you can't pack all the files into an archive and send that over. Why is it so vital that each file is sent immediately? Would it be OK if the file was sent with a short delay (like when 4K worth of data has accumulated)?
It's a nice little problem. I'm not aware of a prepackaged solution, but you could do a lot with simple shell scripts. I'd try this at the receiver:
#!/bin/ksh
# this is receiverprogram
while true
do
typeset -i length
read filename # read filename sent by sender below
read size # read size of file sent
read -N $size contents # read all the bytes of the file
print -n "$contents" > "$filename"
done
At the sender side I would create a named pipe and read from the pipe, e.g.,
mkfifo $HOME/my-connection
ssh remotehost receiver-script < $HOME/my-connection
Then to send a file I'd try this script
#!/bin/ksh
# this is senderprogram
FIFO=$HOME/my-connection
localname="$1"
remotename="$2"
print "$remotename" > $FIFO
size=$(stat -c %s "$localname")
print "$size" > $FIFO
cat "$localname" > $FIFO
If the file size is large you probably don't want to read it at one go, so something on the order of
BUFSIZ=8192
rm -f "$filename"
while ((size >= BUFSIZ)); do
read -N $BUFSIZE buffer
print -n "$buffer" >> "$filename"
size=$((size - BUFSIZ))
done
read -N $size buffer
print -n "$contents" >> "$filename"
Eventually you'll want to extend the script so you can pass through chmod and chgrp commands. Since you trust the sending code, it's probably easiest to structure the thing so that the receiver simply calls shell eval on each line, then send stuff like
print filename='"'"$remotename"'"' > $FIFO
print "read_and_copy_bytes " '$filename' "$size" > $FIFO
and then define a local function read_and_copy_bytes. Getting the quoting right is a bear, but otherwise it should be straightforward.
Of course, none of this has been tested! But I hope it gives you some useful ideas.
Seems like a job for tar? Pipe its output to ssh, and on the other side pipe the ssh output back to tar.
I think that the GNOME desktop uses a single SSH connection when accessing a share through SFTP (SSH). I'm guessing that this is what's happening because I see a single SSH process when I access a remote share this way. So if this is true you should be able to use the same program for this purpose.
The new version of GNOME used GVFS through GIO in order to perform all kind of I/O through different backends. The Ubuntu package gvfs-bin provides various command line utilities that let you manipulate the backends from the command line.
First you will need to mount your SSH folder:
gvfs-mount sftp://user#host/
And then you can use the gvfs-copy to copy your files. I think that all file transfers will be performed through a single SSH process. You can even use ps to see which process is being used.
If you feel more adventurous you can even write your own program in C or in some other high level language that provides an API to GIO.
One option is Conch is a SSH client and server implementation written in Python using the Twsited framework. You could use it to write a tool which accepts requests via some other protocol (HTTP or Unix domain sockets, FTP, SSH or whatever) and triggers file transfers over a long running SSH connection. In fact, I have several programs in production which use this technique to avoid multiple SSH connection setups.
There was a very similar question here a couple of weeks ago. The accepted answer proposed to open a tunnel when ssh'ing to the remote machine and to use that tunnel for scp transfers.
Perhapse CurlFTPFS might be a valid solution for you.
It looks like it just mounts an external computer's folder to your computer via SFTP. Once that's done, you should be able to use your regular cp commands and everything will be done securely.
Unfortunately I was not able to test it out myself, but let me know if it works for ya!
Edit 1: I have been able to download and test it. As I feared it does require that the client have a FTP server. However, I have found another program which does has exactly the same concept as what you are looking for. sshfs allows you to connect to your client computer without needing any special server. Once you have mounted one of their folders, you can use your normal cp commands to move whatever files you need to more. Once you are done, it should then be a smile matter of umount /path/to/mounted/folder. Let me know how this works out!
rsync -avlzp user#remotemachine:/path/to/files /path/to/this/folder
This will use SSH to transfer files, in a non-slow way
Keep it simple, write a little wrapper script that does something like this.
tar the files
send the tar-file
untar on the other side
Something like this:
tar -cvzf test.tgz files ....
scp test.tgz user#other.site.com:.
ssh user#other.site.com tar -xzvf test.tgz
/Johan