I am unsure why is this rsync command is not syncing?
rsync -v -e root#ec2-X.compute-1.amazonaws.com:/var/log/apache2/USAGE-log.txt splunk-rync-logs/log.txt
I see this returned after that command which appears OK.
building file list ... done
-rw-r--r-- 0 2012/03/26 19:28:00 log.txt
sent 28 bytes received 12 bytes 80.00 bytes/sec
total size is 0 speedup is 0.00
BUT no data is added to the local file that is supposed to be being synced with the remote file:
ls -al
total 0
drwxr-xr-x 3 bd staff 102 Mar 26 19:28 .
drwxr-xr-x+ 54 bd staff 1836 Mar 26 19:28 ..
-rwxrwxrwx 1 bd staff 0 Mar 26 19:28 log.txt
Any advice?
The syntax for the -e option (rsync version 3.0.8 protocol version 30) is:
-e, --rsh=COMMAND specify the remote shell to use
For use such as -e 'ssh -p 2234'.
Maybe you have a different version, but that's where I'd start looking.
Related
Why can't I delete this directory in the Z shell? What number is expected?
I have tried googling "number expected" and z shell to no avail.
drwxr-xr-x 3 me staff 96 Jan 31 09:49 PycharmProjects
drwxr-xr-x+ 3 me staff 96 Apr 21 2015 Sites
drwxr-xr-x# 13 me staff 416 Jul 1 2019 iCloud Drive (Archive)
drwx---rwx# 39 me staff 1248 Dec 5 12:28 iCloud Drive (Archive) 2
me#BigT ~ % rmdir -rf iCloud Drive (Archive)
**zsh: number expected**
My guess is the spaces and parenthesis are making ZSH try to expand / evaluate your expression, which is invalid. Try surrounding the name of the folder with quotes, like so:
rm -rf "iCloud Drive (Archive)"
I'm trying to understand the purpose of the --file-type argument of the UNIX ls command.
I didn't find any example using ls --file-type on the web – neither was I able to make sense of the --file-type argument by issuing ls --file-type myself.
What's the purpose of ls --file-type? Would anyone please give me a helpful example of ls --file-type making the purpose of this argument more clear to me?
Quoting from info ls:
-F
Append a character to each file name indicating the file type.
Also, for regular files that are executable, append *. The file
type indicators are / for directories, # for symbolic links,
| for FIFOs, = for sockets, > for doors, and nothing for
regular files.
--file-type is just like -F, except that it doesn't append * for executable files:
$ ln -s /etc/passwd foo
$ touch bar; chmod +x bar
$ ls
foo bar
$ ls -F
foo# bar*
$ ls --file-type
foo# bar
...
...
The --file-type option is similar to the -F option.
-F, --classify
append indicator (one of */=>#|) to entries
--file-type
likewise, except do not append `*'
Source.
Here's an example of ls --file-tye /
bin# boot/ dev/ etc/ home/ lib# lib64# lost+found/ media/ mnt/ opt/ proc/ root/ run/ sbin# srv/ sys/ tmp/ usr/ var/
as compared to the output of ls -l /
lrwxrwxrwx 1 root root 7 Jan 19 02:32 bin -> usr/bin
drwxr-xr-x 4 root root 4096 Jan 1 1970 boot
drwxr-xr-x 24 root root 4220 Mar 26 19:56 dev
drwxr-xr-x 88 root root 4096 Mar 26 21:15 etc
drwxr-xr-x 4 root root 4096 Apr 5 2020 home
lrwxrwxrwx 1 root root 7 Jan 19 02:32 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Jan 19 02:32 lib64 -> usr/lib
drwx------ 2 root root 16384 Apr 5 2020 lost+found
drwxr-xr-x 2 root root 4096 Dec 11 20:19 media
drwxr-xr-x 7 root root 4096 Sep 8 2020 mnt
drwxr-xr-x 9 root root 4096 Feb 9 13:58 opt
dr-xr-xr-x 335 root root 0 Mar 26 19:56 proc
drwxr-x--- 9 root root 4096 Feb 11 13:18 root
drwxr-xr-x 28 root root 700 Mar 26 21:15 run
lrwxrwxrwx 1 root root 7 Jan 19 02:32 sbin -> usr/bin
drwxr-xr-x 4 root root 4096 Apr 5 2020 srv
dr-xr-xr-x 13 root root 0 Mar 26 19:56 sys
drwxrwxrwt 12 root root 320 Mar 26 22:11 tmp
drwxr-xr-x 9 root root 4096 Mar 25 11:06 usr
drwxr-xr-x 12 root root 4096 Mar 25 11:07 var
As you see the symbol "#" indicates symbolic-links (l). Directories (d) and regular files (f) are not flagged by the --file-type argument.
Issuing ls --file-type in my home directory is identical to ls ~ since it only contains directories and regular files and hence the output of the former doesn't contain any indicators.
See also the answer of PesaThe for a full list of the indicator symbols.
I'm working on the following question for my class (in Unix) and I'm running into some problems.
Using the grep command and regular expressions, create a pipe-based
chain of commands that will list all files in the default directory
that others can read or write.
This is the command I have so far and it will correctly match the right lines, but I only want to to print the name of the file that has the desired permissions.
ls -la | grep "^.......rw.*:\d\d\s(.*)$"
here is the result of just typing ls -la by the way which is what I'm "greping"
total 344
drwxr-xr-x 10 cameronpattisall staff 340 Apr 7 11:21 .
drwxr-xr-x 7 cameronpattisall staff 238 Apr 7 11:22 ..
-rw-r--r-- 1 cameronpattisall staff 6148 Apr 7 11:21 .DS_Store
-rw-r--r-- 1 cameronpattisall staff 11591 Apr 7 11:21 DifferencesToTagged.txt
-rw-r--r-- 1 cameronpattisall staff 4291 Apr 7 11:21 functions.py
-rw-r--r-- 1 cameronpattisall staff 6080 Apr 7 11:21 ls.txt
-rw-r--r-- 1 cameronpattisall staff 5511 Apr 7 11:21 prog4.py
-rw-r--r-- 1 cameronpattisall staff 345 Apr 7 11:21 shared.py
-rw-r--r-- 1 cameronpattisall staff 1832 Apr 7 11:21 subversion-log.txt
-rw-r--r-- 1 cameronpattisall staff 122295 Apr 7 11:21 testcases.txt
I guess the solution is to use:
grep -E "^-.{6}(r|.w)"
The structure of the first column is "-/d" (file/dir) + xxx (owner) + xxx (group) + xxx (others).
So you want to match those starting with - (file) and containing in the "gorup" part either r.. or .w..
To print just the file names, then you can pipe to grep -o '[^ ]*$'. This will print the last block of text. Note this is terrible fragile, since in a file name with spaces it will just print the last one.
Test
I stored your ls -la in the file a:
$ grep -E "^-.{6}(r|.w)" a | grep -o '[^ ]*$'
.DS_Store
DifferencesToTagged.txt
functions.py
ls.txt
prog4.py
shared.py
subversion-log.txt
testcases.txt
If you're already matching the right lines, you can append something like
| tr -s ' ' | cut -d ' ' -f 9- to your pipeline.
The tr -s ' ' replaces multiple sequential spaces with a single space; the cut command removes everything except the ninth field onwards from each line, with fields delimited by single space characters.
I need a shell script to search and replace a file. Details are below. Pl help
Basically everyday i get some files into my directory.Like below i get 100 files
AllResponses_11003_6_20_2013.txt
AllResponses_11004_6_20_2013.txt
AllResponses_11005_6_20_2013.txt
AllResponses_11006_6_20_2013.txt
AllResponses_11007_6_20_2013.txt
AllResponses_11008_6_20_2013.txt
AllResponses_11009_6_20_2013.txt
AllResponses_11010_6_20_2013.txt
AllResponses_11011_6_20_2013.txt
AllResponses_11012_6_20_2013.txt
Among those i need to copy 5 files to another directory based on file number(11003,11004,11005,11006,11007)
AllResponses_11003_6_20_2013.txt
AllResponses_11004_6_20_2013.txt
AllResponses_11005_6_20_2013.txt
AllResponses_11006_6_20_2013.txt
AllResponses_11007_6_20_2013.txt
If not find , then need to replace with 0 byte files for those 5 files into another directory.
Pl help
But how to pass num as 11003,11004,11005,11006,11007 among hundreds of numbers like 11003 to 11100
Pl help...
export SRCDIR=/informat/PowerCenter/9.1.0/server/infa_shared/SrcFiles/CSI/historical
export TGTDIR=/informat/PowerCenter/9.1.0/server/infa_shared/SrcFiles/CSI/incoming
export FILEDT=6_15_2013
export FILEDT=$(date +"%-m_%-d_%Y")
looping for to search and copy files
for FILE_NUM in "$#";
do
GET_FNAME="AllResponses_"${FILE_NUM}"_"${FILEDT}"*.txt"
if [ -f ${GET_FNAME} ]; then
cp ${SRCDIR}/${GET_FNAME} ${TGTDIR}
else
echo "File ${GET_FNAME} is missing in ${SRCDIR}"
touch ${TGTDIR}/AllResponses_${FILE_NUM}_${FILEDT}.txt
echo "Created ${GET_FNAME} touch file in ${TGTDIR}"
fi done
iam done like above and executing as ksh -x csi_file_copy_bala.ksh 11003 11004 99999
but its always going to else clause..please help me...
my files nales looks like...AllResponses_11004_6_11_20132_18_00AM1.txt
Pl help me...as iam running out of time
Thanks in advance
Assuming by shell u mean bash:
Skeleton to start with:
luk32:~/projects/tests$ cat ./process_files.sh
#!/bin/bash
DEST=./copies
for num in "$#"; do
file="AllResponses_"$num"_6_20_2013.txt"
if [ -f $file ]; then
cp $file $DEST
else
touch $DEST/$file
fi
done;
It takes numbers as arguments, then tries to find a file with given pattern in current working directory. If found copy to destination folder, else touch the file.
You will probably have to tinker a little bit to get friendlier than hard-coded date handling.
Example:
luk32:~/projects/tests$ ls -l
total 40116
-rw-r--r-- 1 luk32 luk32 4 cze 21 11:33 AllResponses_1_6_20_2013.txt
-rw-r--r-- 1 luk32 luk32 5 cze 21 11:33 AllResponses_3_6_20_2013.txt
-rw-r--r-- 1 luk32 luk32 0 cze 21 11:32 AllResponses_4_6_20_2013.txt
luk32:~/projects/tests$ ls -l ./copies/
total 0
luk32:~/projects/tests$ ./process_files.sh 1 2 3 4
luk32:~/projects/tests$ ls -l ./copies/
total 8
-rw-r--r-- 1 luk32 luk32 4 cze 21 11:35 AllResponses_1_6_20_2013.txt
-rw-r--r-- 1 luk32 luk32 0 cze 21 11:35 AllResponses_2_6_20_2013.txt
-rw-r--r-- 1 luk32 luk32 5 cze 21 11:35 AllResponses_3_6_20_2013.txt
-rw-r--r-- 1 luk32 luk32 0 cze 21 11:35 AllResponses_4_6_20_2013.txt
Just wondering is there any way to create a temporary sandbox environment for running commands?
My requirement is that i am hosting a webservice in unix and i need to execute a commanline tool to return output to the webservice client. since i am passing the values received from the client to the commanline tool, i would like to execute the commands in sandboxed environment.
I'm not sure but, you can try to use the "chroot" command to create new "root" enviroment, for example,
If you have the directory structure and you want to "protect" the "license" file,
/
/etc
+ license
/bin
+ ls
/lib
+ ...
You can create a chroot enviroment as
itily#openzooey:~$ mkdir chroot_example
itily#openzooey:~$ cd chroot_example/
itily#openzooey:~/chroot_example$ mkdir -p usr/lib lib bin etc
itily#openzooey:~/chroot_example$ cd bin/
itily#openzooey:~/chroot_example/bin$ cp /bin/ls .
itily#openzooey:~/chroot_example/bin$ ldd ls
libsec.so.1 => /lib/libsec.so.1
libnvpair.so.1 => /lib/libnvpair.so.1
libcmdutils.so.1 => /lib/libcmdutils.so.1
libcurses.so.1 => /lib/libcurses.so.1
libc.so.1 => /lib/libc.so.1
libavl.so.1 => /lib/libavl.so.1
libidmap.so.1 => /usr/lib/libidmap.so.1
libnsl.so.1 => /lib/libnsl.so.1
libuutil.so.1 => /lib/libuutil.so.1
libmp.so.2 => /lib/libmp.so.2
libmd.so.1 => /lib/libmd.so.1
libm.so.2 => /lib/libm.so.2
Now populate the "shared lib" required by the ls command (using the ldd we know which are the required shared libs
itily#openzooey:~/chroot_example/bin$ ldd ls |awk '{print "cp "$3" lib/"}'
cp /lib/libsec.so.1 lib/
cp /lib/libnvpair.so.1 lib/
cp /lib/libcmdutils.so.1 lib/
cp /lib/libcurses.so.1 lib/
cp /lib/libc.so.1 lib/
cp /lib/libavl.so.1 lib/
cp /usr/lib/libidmap.so.1 lib/
cp /lib/libnsl.so.1 lib/
cp /lib/libuutil.so.1 lib/
cp /lib/libmp.so.2 lib/
cp /lib/libmd.so.1 lib/
cp /lib/libm.so.2 lib/
Now we need to copy to our new "lib" and usr/lib directory
itily#openzooey:~/chroot_example/bin$ cd ..
itily#openzooey:~/chroot_example$ ldd /bin/ls |awk '{print "cp "$3" lib/"}'|bash
itily#openzooey:~/chroot_example$ ls -ltr
total 9
drwxr-xr-x 2 itily staff 2 dic 22 14:37 etc
drwxr-xr-x 2 itily staff 3 dic 22 14:37 bin
drwxr-xr-x 2 itily staff 14 dic 22 14:38 lib
itily#openzooey:~/chroot_example$ cp /usr/lib/libidmap.so.1 usr/lib/
itily#openzooey:~/chroot_example$ cp /usr/lib/ld.so.1 usr/lib/
itily#openzooey:~/chroot_example$ cd lib/
itily#openzooey:~/chroot_example/lib$ ls -l
total 7615
-rwxr-xr-x 1 itily staff 14044 dic 22 14:38 libavl.so.1
-rwxr-xr-x 1 itily staff 1721400 dic 22 14:38 libc.so.1
-rwxr-xr-x 1 itily staff 26748 dic 22 14:38 libcmdutils.so.1
-rwxr-xr-x 1 itily staff 293876 dic 22 14:38 libcurses.so.1
-rwxr-xr-x 1 itily staff 97852 dic 22 14:38 libidmap.so.1
-rwxr-xr-x 1 itily staff 398704 dic 22 14:38 libm.so.2
-rwxr-xr-x 1 itily staff 87164 dic 22 14:38 libmd.so.1
-rwxr-xr-x 1 itily staff 25140 dic 22 14:38 libmp.so.2
-rwxr-xr-x 1 itily staff 648776 dic 22 14:38 libnsl.so.1
-rwxr-xr-x 1 itily staff 74776 dic 22 14:38 libnvpair.so.1
-rwxr-xr-x 1 itily staff 97500 dic 22 14:38 libsec.so.1
-rwxr-xr-x 1 itily staff 49556 dic 22 14:38 libuutil.so.1
itily#openzooey:~/chroot_example/lib$ cd ..
So, the final structure is
itily#openzooey:~/chroot_example$ ls -l
total 12
drwxr-xr-x 2 itily staff 3 dic 22 14:37 bin
drwxr-xr-x 2 itily staff 5 ene 10 20:43 etc
drwxr-xr-x 2 itily staff 14 ene 10 20:48 lib
drwxr-xr-x 3 itily staff 3 ene 10 20:40 usr
Also you need the group, passwd and other files
itily#openzooey:~/chroot_example$ echo "this is a test" > etc/license
itily#openzooey:~/chroot_example$ cd etc/
itily#openzooey:~/chroot_example/etc$ cat /etc/group |grep staff > group
itily#openzooey:~/chroot_example/etc$ cat /etc/passwd |grep itily > passwd
Now you can run the chroot command, but if you try to run as non root you will get the error
itily#openzooey:~$ chroot chroot_example bin/ls /etc
chroot: cannot change root directory to chroot_example: Not owner
So, you need to run as root
itily#openzooey:~$ pfexec chroot chroot_example /bin/ls -l /etc
total 6
-rw-r--r-- 1 101 10 11 Jan 10 19:43 group
-rw-r--r-- 1 101 10 18 Jan 10 19:42 license
-rw-r--r-- 1 101 10 49 Jan 10 19:43 passwd
I hope it's what you are looking for ...
Urko,
You didn't say which Unix. Solaris zones do that easily. Here's an example how to set one up:
http://www.solarisinternals.com/wiki/index.php/Zones
I routinely develop in sandboxes in Debian Linux and like to use chroots so that I don't destroy my host machine in case I do something stupid by accident (which happens a lot). I put together a set of scripts for Debian and Ubuntu for me to create chroots (both cross-distro and x86 in x86-64) which you may find helpful.
https://github.com/vtonehundred/vroot