Trying to connect to SFTP connection where there are few CSV files placed in folders and want to download multiple files at once
The directory is exports/payroll_exports.
There are 3 files there for now:
vani_payroll_bayshore_21323_232.csv
vani_payroll_bayshore_21344_256.csv
vani_payroll_bayshore_124523_888.csv
How to use a get command where I can write get vani_payroll_bayshore%.csv?
I plan to add the get command to a script like this:
option batch on
option confirm off
open sftp:.... -hostkey= "...."
cd exports/payroll_exports
lcd "\\....."
get ( don't know how to write the wildcard syntax)
close
exit
WinSCP uses the common file wildcard syntax as most other applications.
To match anything of any length, use *.
get vani_payroll_bayshore*.csv
See WinSCP masks documentation.
Related
I need to write a script that synchronizes local files with a remote machine.
My file structure is:
ProjectFolder/
.git/
input/
output/
classes/
main.py
readme.md
I need to synchronize everything, but:
completely ignore .git folder
ignore files in input and output folders, but copy the folder
So far my code is:
open sftp://me:password#server -hostkey="XXXXXXXX"
option batch abort
option confirm off
synchronize remote "C:\Users\MYNAME\Documents\MY FOLDER\Python Projects\ProjectFolder" "/home/MYNAME/py_proj/ProjectFolder" -filemask="|C:\Users\MYNAME\Documents\MY FOLDER\Python Projects\ProjectFolder\.git"
close
exit
First question: it doesn't seems to work.
Second question, how to add mask for input and output folder if I have spaces in file paths?
Thanks to all in advance.
Masks for directories have to end with a slash.
To exclude files in a specific folder, use something like */folder/*
-filemask="|.git\;*/input/*;*/output/*"
TLDR; Convert the bash line to download sftp files get Inbox/* to c++ or python. We do not have execute permissions on Inbox directory.
I am trying to read the files present in a directory in a remote server through SFTP. The catch is that I only had read and write permissions on the directory and not execute. This means any method that requires opening (cding) into the folder would fail. I need to read the file names since they are variable. From what I understand ls does not require execute privs. If I can get a list of files in the directory then reading then would be fine. Here is the directory structure:
Inbox
--file-a.txt
--file_b.txt
...
I have tried libssh but sftp_readdir required a handle of the open directory. I also looked at paramiko for python but that too requires to open the directory to read the file names.
I am able to do this in bash using send "get Inbox/* ${destination_dir}". Is there anyway I can use a similar pattern match but on c++ or python?
Also, I cannot execute bash commands through my binary. Does anyone know of any library in python or c++ (preferred) that would support this?
I have not posted here in a while so please excuse me if I am not following the formatting. I will learn from your suggestions. Thank you!
I want to download only new files from one SFTP server using WinSCP.
Suppose, I have 10 files in source and destination today.
Tomorrow one new file may be added to the source. In this scenario, I want to copy the new file only into destination.
I am using below script:
open sftp_connection
cd /path
option transfer binary
get "*.txt" localpath
close
exit
By using above, I am able to copy all files, but I want only new files which are not available in destination.
Thanks,
Srikrishna.
The easiest solution is to add -neweronly switch to your get command:
get -neweronly *.txt C:\local\path\*
For a very similar results, you can also use synchronize command:
synchronize local . C:\local\path -filemask=*.txt
See also WinSCP article on Downloading the most recent file.
I have a file loaded in R that I want to move to a samba share
It is something like
write(some-file, file = "|smbclient -U user //ip password")
It connects to the samba but then (I think) the output is "executed" in the smb: \> and I don't want the file to be executed, I don't know how to pass the file to the destination with a putfunction inside smbclient.
Edit: This is not the same problem as the first post. The first post is solved and answered by me. The point there was connecting to samba. Now I'm already connected to it but the write() function doesn't make a file, instead it pipes out the words separately. I just wanted to know how to make it create a file in a sentence.
I found the answer by changing the philosophy:
First, I write the file locally, like
write(some-file, there)
Then I use the system() function to call smbclient and put the file already written
system("smbclient -U user //ip/dir password -c \"put some-file some-file\"")
My script is more complex and it's inside a Shiny app but in summary that's the solution
Let's imagine we have some script 'm12' (I've just invented this name) that runs
on Linux computers. If it is situated in your $PATH, you can easily run it
from the console like this:
m12
It will work with the default parameters. But you can customize the work of
this script by running it something like:
m12 --enable_feature --select=3
It is great and it will work. But I want to create a config file ~/.m12rc so I
will not need to specify --enable_feature --select=3 every time I run it.
It can be easily done.
The difficult part is starting here.
So, I have ~/.m12rc config file, but I what to start m12 without parameters that
are stored in that config file. What is the Unix way to do this? Should I run
script like this:
m12 --ignore_config
or there is better solution?
Next. Let's imagine I have a config file ~/.m12rc and I want some parameters from that
file, but want to change them a bit. How should I run the script and how the
script should work?
And the last question. Is it a good idea for script to first look for .m12rc
in the current directory, then in ~/ and then in /etc?
I'm asking all these questions because I what to implement config files in my
small script and I want to make the correct decisions about the design.
The book 'The Art of Unix Programming' by E S Raymond discusses such issues.
You can override the config file with --config-file=/dev/null.
You would normally use the order:
System-wide configuration (/etc/m12/m12rc, or just /etc/m12).
User's personal configuration (~/.m12rc)
Local directory configuration (./.m12rc)
Command-line options
with each later-listed item overriding earlier listed items. You should be able to specify the configuration file to read on the command line; arguably, that should be given precedence over other options. Think about --no-system-config or --no-user-config or --no-local-config. Many scripts do not warrant a system config file. Most scripts I've developed would not use both local config and user config. But that's the way my mind works.
The way I package standard options is to have a script in $HOME/bin (say m12a) that does it for me:
#!/bin/sh
exec m12 --enable_feature --select=3 "$#"
If I want those options, I run m12a. If I want some other options, I run raw m12 with the requisite options. I have multiple hundreds of files in my personal bin directory (about 500 on my main machine, a Mac; some of those are executables, but many are scripts).
Let me share my experience. I normally source config file at the beginning of the script. In the config file I also handle all the parameter switches:
DEFAULT_USER=blabla
while getopts ":u" do
case $opt in
u)
export APP_USER=$OPTARG
;;
esac
done
export APP_USER=${APP_USER-$DEFAULT_USER}
Then within the script I just use variables, this let me to have number of script having same input parameters.
In your case I imaging you would move "getopts" section to script and after it source the config file (if there was no switch to skip sourcing).
You should not put yours script config file to etc, it will require root privilidge to do that, and you simple can live with config file in home.
If you would like anyway to put your script for sharing with other users, it should go to /usr/share...
Another solution use thor (ruby gem), its way simpler to handle input parameter, avoiding work to get same result in bash e.g. getopts support only single letter switches.