I have a directory full of files with names such as:
file_name_is_001
file_name_001
file_name_is_002
file_name_002
file_name_is_003
file_name_003
I want to copy only the files that don't contain 'is'. I'm not sure how to do this. I have tried to search for it, but can't seem to google the right phrase to find the results.
Details depend on operating system, shell, etc.
For a unix system a quite verbose but easy to understand approach could look like this (please mind that I didn't test it):
mkdir some_temporary_directory
mv *_is_* some_temporary_directory
cp * where_ever_you_want_to_copy_it
mv some_temporary_directory/* .
rmdir some_temporary_directory
You can do this using bash. First, here's a command to get you a list of files that don't contain the text _is_:
ls | grep -v "_is_"
This takes the output of ls and matches all values with DO NOT contain _is_ using grep -v.
In order to then copy these files, we need to turn the lines output by grep into arguments of cp. We can do this using xargs:
ls | grep -v "_is_" | xargs -J % cp % new_folder
From the xargs man page, it is a tool to "build and execute command lines from standard input".
I have a file that contains the version name and version number. The contents of the first file looks as-
File1-
<Line contains the name of product1>
package_name0_9_8 >= 1.2.3x-4.5.6
package_name0_9_8-32bit >= 3.6.1g-3.5.1
package_name0_9_8-xx >= 6.3.2v-3.0.4
<Line contains the name of product2>
anotherpackage_name0_9_8 >= 3.5.6u-3.6.5
And,
File2.xml-
<package name="package_name0_9_8" version="1.2.3x-4.4.4"/>
<package name="package_name0_9_8-32bit" version="3.6.1g-3.4.0"/>
.
.
Is there a way to check the existance of package_name that is present in File1 with the package_name of File2 and check if the corresponding version of package_name in File1 with that of corresponding version of package_name of File2?
I am frank that I am pretty much weak in concatenating the 'grep' and 'awk' commands along with options to be used here. Please help out.
for a in $(sed -n '/>=/p' File1.txt | grep -o '^[^ ]*'); do for b in $(sed -n "/^$a /{s/.*>=\(.*\)$/\1/p}" File1.txt); do ((! $(grep -c "$a.*$b" File2.txt))) && (echo "$a $b" >> missing_pkgs.txt); done; done;
this is a quick one liner - you could print it out a bit prettier
the way this works is nested for loop that grabs both pieces separate into variables (you could do that with read and put them in on one loop if you want) and then just counts the occurences in the second file with grep and whenever there is a count of zero it will reverse the value making the test (()) turn true and echo the missing packages to the file missing_pkgs.txt
here is another quick one liner that does the same thing except more efficient with one loop and variables loaded via read
while read each; do read a b < <(echo $each) && ((! $(grep -c "$a.*$b" File2.txt))) && (echo "$a $b" >> missing_pkgs.txt); done < <(awk '/>=/{ print $1" "$3 }' File1.txt)
more simplified:
while read a b; do ((! $(grep -c "$a.*$b" File2.txt))) && (echo "$a $b" >> missing_pkgs.txt); done < <(awk '/>=/{ print $1" "$3 }' File1.txt)
sed -n 's².*²s#<package name="\\(&"/>#\\1 Present#p²;s/ *>= */\\)" *version="/p' File1 > /tmp/File1.sed
sed -n -f /tmp/File1.sed File2
rm /tmp/File1.sed
not in on instruction like awk could do, but do the job (posix version so --posix on GNU sed
you could change the output message that is the \\1 Present text where \\1 will the be the package name (with few modification, version could also be used)
It looks like you already got a much shorter solution in a format closer to what you desired. However, since I asked if a Python solution would work, and you said yes, check out the code here:
http://pastebin.com/F5LYrmea
(I haven't debugged it more than a little, but it seems to work on at least a little more than your example files. I released the code to the public domain. CC-BY-SA isn't a software license, according to the makers of CC; so, that's why I didn't post it here, as posting it here would give it that license. Plus, you get syntax highlighting specific to Python at the link provided.)
Basically, it's a lot of complicated text parsing. Not much of an algorithm to explain. It gets the contents of both files, strips out the packages, their versions and the operands (puts all those in a dictionary for use later), and loops through lines of the other file and compares versions; then it tells you which ones match and which ones don't.
Can someone explain me the meaning of this line :
wget -q -O - $URL | sed 's/<str name="id">/\&/g' > tmp.$$ 2>/dev/null
After hitting the URL, it's searching for the pattern
but what's the meaning of tmp.$$ here.
$$ is the current process id in most of the commonly used unix based shells . It's probably being used to create a unique filename so multiple instances can run the same time and not squash each others temp files.
Making a bash script, and I am trying to figure out a way to find out what my symbolic link points to. Right now, I am doing it with this, but this only works if my symlink is in the current directory. Is there a way to find out what my symlink is pointing to if it is in another directory?
ls -l "symlink" | cut -d'>' -f2
If you have the readlink(1) utility (part of GNU coreutils), it does what you want. Otherwise you are kinda up a creek, I'm not aware of any straightforward & portable equivalent.
On a BSD toolchain, I am doing:
stat -f %Y <filename>
For example:
% ln -sf /bsd ~/blah
% stat -f %Y ~/blah
/bsd
On a GNU toolchain it is not so easy, you can use something like:
$ stat -c %N /usr/bin/firefox
`/usr/bin/firefox' -> `../lib/firefox-3.6.12/firefox.sh'
Then, use awk/cut and sed to extract and remove junk quotes.
Or a messier solution is to use ls -al and either awk/cut to extract the column you need.
Is it possible when listing a directory to view numerical Unix permissions such as 644, rather than the symbolic output -rw-rw-r-- ?
Thanks.
it almost can ..
ls -l | awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/) \
*2^(8-i));if(k)printf("%0o ",k);print}'
Closest I can think of (keeping it simple enough) is stat, assuming you know which files you're looking for. If you don't, * can find most of them:
/usr/bin$ stat -c '%a %n' *
755 [
755 a2p
755 a2ps
755 aclocal
...
It handles sticky, suid and company out of the box:
$ stat -c '%a %n' /tmp /usr/bin/sudo
1777 /tmp
4755 /usr/bin/sudo
you can just use GNU find.
find . -printf "%m:%f\n"
You can use the following command
stat -c "%a %n" *
Also you can use any filename or directoryname instead of * to get a specific result.
On Mac, you can use
stat -f '%A %N' *
Use this to display the Unix numerical permission values (octal values) and file name.
stat -c '%a %n' *
Use this to display the Unix numerical permission values (octal values) and the folder's sgid and sticky bit, user name of the owner, group name, total size in bytes and file name.
stat -c '%a %A %U %G %s %n' *
Add %y if you need time of last modification in human-readable format. For more options see stat.
Better version using an Alias
Using an alias is a more efficient way to accomplish what you need and it also includes color. The following displays your results organized by group directories first, display in color, print sizes in human readable format (e.g., 1K 234M 2G) edit your ~/.bashrc and add an alias for your account or globally by editing /etc/profile.d/custom.sh
Typing cls displays your new LS command results.
alias cls="ls -lha --color=always -F --group-directories-first |awk '{k=0;s=0;for(i=0;i<=8;i++){;k+=((substr(\$1,i+2,1)~/[rwxst]/)*2^(8-i));};j=4;for(i=4;i<=10;i+=3){;s+=((substr(\$1,i,1)~/[stST]/)*j);j/=2;};if(k){;printf(\"%0o%0o \",s,k);};print;}'"
Folder Tree
While you are editing your bashrc or custom.sh include the following alias to see a graphical representation where typing lstree will display your current folder tree structure
alias lstree="ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'"
It would display:
|-scripts
|--mod_cache_disk
|--mod_cache_d
|---logs
|-run_win
|-scripts.tar.gz
#The MYYN
wow, nice awk! But what about suid, sgid and sticky bit?
You have to extend your filter with s and t, otherwise they will not count and you get the wrong result. To calculate the octal number for this special flags, the procedure is the same but the index is at 4 7 and 10. the possible flags for files with execute bit set are ---s--s--t amd for files with no execute bit set are ---S--S--T
ls -l | awk '{
k = 0
s = 0
for( i = 0; i <= 8; i++ )
{
k += ( ( substr( $1, i+2, 1 ) ~ /[rwxst]/ ) * 2 ^( 8 - i ) )
}
j = 4
for( i = 4; i <= 10; i += 3 )
{
s += ( ( substr( $1, i, 1 ) ~ /[stST]/ ) * j )
j/=2
}
if ( k )
{
printf( "%0o%0o ", s, k )
}
print
}'
For test:
touch blah
chmod 7444 blah
will result in:
7444 -r-Sr-Sr-T 1 cheko cheko 0 2009-12-05 01:03 blah
and
touch blah
chmod 7555 blah
will give:
7555 -r-sr-sr-t 1 cheko cheko 0 2009-12-05 01:03 blah
You don't use ls to get a file's permission information. You use the stat command. It will give you the numerical values you want. The "Unix Way" says that you should invent your own script using ls (or 'echo *') and stat and whatever else you like to give the information in the format you desire.
Building off of the chosen answer and the suggestion to use an alias, I converted it to a function so that passing a directory to list is possible.
# ls, with chmod-like permissions and more.
# #param $1 The directory to ls
function lls {
LLS_PATH=$1
ls -AHl $LLS_PATH | awk "{k=0;for(i=0;i<=8;i++)k+=((substr(\$1,i+2,1)~/[rwx]/) \
*2^(8-i));if(k)printf(\"%0o \",k);print}"
}
Solution
It is strange that still nobody mentioned the (quote) "modern replacement for ls" - an alternative and quite powerful tool exa.
You can easily achieve the desired output by using exa command along with the -l (which is equivalent to ls's -l) and the --octal-permissions options.
Example
Here is a simple example of listing the contents of a user's root directory (/) on a macOS machine using exa command and the --octal-permissions option:
exa -lh --octal-permissions /
Result:
Notice how besides the nice colorful output exa can also show the headers for each column thanks to the -h option (long form is --header).
Read man exa or the official online documentation for more information about how to customize the desired output according to your specific needs.
Considering the question specifies UNIX, not Linux, use of a stat binary is not necessary. The solution below works on a very old UNIX, though a shell other than sh (i.e. bash) was necessary. It is a derivation of glenn jackman's perl stat solution. It seems like an alternative worth exploring for conciseness.
$ alias lls='llsfn () { while test $# -gt 0; do perl -s -e \
'\''#fields = stat "$f"; printf "%04o\t", $fields[2] & 07777'\'' \
-- -f=$1; ls -ld $1; shift; done; unset -f llsf; }; llsfn'
$ lls /tmp /etc/resolv.conf
1777 drwxrwxrwt 7 sys sys 246272 Nov 5 15:10 /tmp
0644 -rw-r--r-- 1 bin bin 74 Sep 20 23:48 /etc/resolv.conf
The alias was developed using information in this answer
The whole answer is a modified version of a solution in this answer
After reading MANY answers here, following links provided in comments back to the original UNIX way of doing this, and while wanting to combined what was offered here, as well as the tips and tricks I've already learned, I came up with a new solution.
First off, I used to use this alias, to give me column headers:
alias l='echo "Dir Size|Perms|Link Count|Owner|Group|Size|Mod. Time|Name"; ls -haFl --time-style=long-iso --color=always --group-directories-first --format=long'
After combining this, with AWK, I first learned I had to alter the awk command, when using the "-s" option for ls, as this shows size in the first column, and you need to then read and parse the second (no longer first) column of data.
The ISSUE I found, was when you then provide input to " l " i.e., you are at a path, and type: l, fine, but what if you type "l" then a directory name? what if you want to list out everything in a subdirectory? This wasn't able to happen, as the alias did not handle input. I was able to handle this with a function. Then AWk broke, which I was able to handle with a sub-alias.
Combining the two worked perfectly.
Added to my .bashrc file
function _bestLS() {
echo 'MODE|Dir Size|Perms|Link Count|Owner|Group|Size|Mod. Time|Name';
if [ "$*" == '' ]; then
alias _awk4ls="awk '{k=0;s=0;for(i=0;i<=8;i++){;k+=((substr(\$2,i+2,1)~/[rwxst]/)*2^(8-i));};j=4;for(i=4;i<=10;i+=3){;s+=((substr(\$1,i,1)~/[stST]/)*j);j/=2;};if(k){;printf(\"%0o%0o \",s,k);};print;}'";
output=`ls -shaFl --time-style=long-iso --color=always -F --group-directories-first --format=long | _awk4ls`;
else
output=`ls -shaFl --time-style=long-iso --color=always -F --group-directories-first --format=long $* | _awk4ls`;
fi
echo "$output"
}
alias l="_bestLS"
Observe (please note, it is a lower case letter L, not a numeric 1, which look the same in this site's font):
>>l
>>l [SOME DIRECTORY]
P.S. Please excuse my very long (3 line) prompt (PS1)