I'm trying to find out what is the best way to use the wild cards in Python. I'm trying to migrate this piece of .ks code to .py. If any file in a folder exists. I was thinking using these alternatives like:
glob
os.path.exists = but this will turn true or false
os.path.isfile
len?
Here is part my code.
#this is .ks
fail_flag=0
#Checking more than one file of any kind
files=$(ls /opt/flag/*)
for file in $file
do
if [ -f $file ] ; then
fail_flag=1
fi
done
If I understood you correctly, you can use python glob.
Something like that:
import glob
if glob.glob('//opt//flag//*.*'):
return True
Here is the docs - https://docs.python.org/2/library/glob.html
Related
I want to write a recursive function which exits on 2 conditions.
Let's say I want to make a directory and ask the user for input. He may enter something like this:
Valid: /existing-dir1/existing-DIR2/non-existence-dir1/non-existence-dir2
Invalid: /existing-dir1/existing-FILE1/non-existence-dir1/non-existence-dir2
To loop through the filename, I have the function dirname() which take /foo/bar and returns /foo. I also have function exist() to check if a filename exist and isdir() to see if it is a file or directory.
Basically, I need to loop recursively from the end of the filename, ignore non-existence nodes, and check if any node is a file - which is invalid. The recursion ends when one of the 2 conditions happens, whichever comes first:
A file is found
dirname() returns /
I am not familiar with recursion, and 2 conditions is a bit too much for me. I am using POSIX script but code samples in C++ / Java / C# are all good.
Edit: I know I can do a mkdir -p and get its status code, but it will create the directory. Nontheless, I want to do that in recursion for the purpose of learning.
In JS, you might write the recursion like this:
const isValid = (path) =>
path === '/'
? 'valid'
: exist(path) && !isdir(path)
? 'invalid'
: isValid(dirname(path))
You might be able to skip the exist check depending upon how your isdir function works.
This should show the logic, but I'm not sure how to write this in your Posix script environment.
I solved it myself. Written in POSIX script, but it is quite easy to read and port to other languages:
RecursivelyCheckFilename ()
{
if [ -e "$1" ] # if exists
then
if [ -d "$1" ] # if is directory
then
if [ "$1" = "/" ]
then
return 0;
else
RecursivelyCheckFilename "$(dirname -- "$1")";
return $?; # returns the value returned by previous function
fi
else
return 1;
fi
else
RecursivelyCheckFilename "$(dirname -- "$1")";
return $?;
fi
}
There are still a few issues with filename ending with a trailing slash, which I must find a way to deal with. But the code above is how I want it to work.
I have a c shell script which has the following two lines, it creates a directory and copies some files into it. My question is the following - the files being copied look like this abc.hello, abc.name, abc.date, etc... How can i strip the abc and just copy them over as .hello, .name, .date.. and so forth. I'm new to this.. any help will be appreciated!
mkdir -p $home_dir$param
cp /usr/share/skel/* $home_dir$param
You're looking for something like basename:
In Bash, for example, you could get the base name, file suffix like this:
filepath=/my/folder/readme.txt
filename=$(basename "$filepath") # $filename == "readme.txt"
extension="${filename##*.}" # $extension == "txt"
rootname="${filename%.*}" # $rootname == "readme"
ADDENDUM:
The key takeaway is "basename". Refer to the "man basename" page I linked to above. Here's another example that should make things clearer:
basename readme.txt .txt # prints "readme"
"basename" is a standard *nix command. It works in any shell; it's available on most any platform.
Going forward, I would strongly discourage you from writing scripts in csh, if you can avoid it:
bash vs csh vs others - which is better for application maintenance?
Csh Programming Considered Harmful
I use UNIX fairly infrequently so I apologize if this seems like an easy question. I am trying to loop through subdirectories and files, then generate an output from the specific files that the loop grabs, then pipe an output to a file in another directory whos name will be identifiable from the input file. SO far I have:
for file in /home/sub_directory1/samples/SSTC*/
do
samtools depth -r chr9:218026635-21994999 < $file > /home/sub_directory_2/level_2/${file}_out
done
I was hoping to generate an output from file_1_novoalign.bam in sub_directory1/samples/SSTC*/ and to send that output to /home/sub_directory_2/level_2/ as an output file called file_1_novoalign_out.bam however it doesn't work - it says 'bash: /home/sub_directory_2/level_2/file_1_novoalign.bam.out: No such file or directory'.
I would ideally like to be able to strip off the '_novoalign.bam' part of the outfile and replace with '_out.txt'. I'm sure this will be easy for a regular unix user but I have searched and can't find a quick answer and don't really have time to spend ages searching. Thanks in advance for any suggestions building on the code I have so far or any alternate suggestions are welcome.
p.s. I don't have permission to write files to the directory containing the input folders
Beneath an explanation for filenames without spaces, keeping it simple.
When you want files, not directories, you should end your for-loop with * and not */.
When you only want to process files ending with _novoalign.bam, you should tell this to unix.
The easiest way is using sed for replacing a part of the string with sed.
A dollar-sign is for the end of the string. The total script will be
OUTDIR=/home/sub_directory_2/level_2
for file in /home/sub_directory1/samples/SSTC/*_novoalign.bam; do
echo Debug: Inputfile including path: ${file}
OUTPUTFILE=$(basename $file | sed -e 's/_novoalign.bam$/_out.txt/')
echo Debug: Outputfile without path: ${OUTPUTFILE}
samtools depth -r chr9:218026635-21994999 < ${file} > ${OUTDIR}/${OUTPUTFILE}
done
Note 1:
You can use parameter expansion like file=${fullfile##*/} to get the filename without path, but you will forget the syntax in one hour.
Easier to remember are basename and dirname, but you still have to do some processing.
Note 2:
When your script first changes the directory to /home/sub_directory_2/level_2 you can skip the basename call.
When all the files in the dir are to be processed, you can use the asterisk.
When all files have at most one underscore, you can use cut.
You might want to add some error handling. When you want the STDERR from samtools in your outputfile, add 2>&1.
These will turn your script into
OUTDIR=/home/sub_directory_2/level_2
cd /home/sub_directory1/samples/SSTC
for file in *; do
echo Debug: Inputfile: ${file}
OUTPUTFILE="$(basename $file | cut -d_ -f1)_out.txt"
echo Debug: Outputfile: ${OUTPUTFILE}
samtools depth -r chr9:218026635-21994999 < ${file} > ${OUTDIR}/${OUTPUTFILE} 2>&1
done
Looking to make my ~ a cleaner place, I would like to move as much user configuration files into $XDG_CONFIG_HOME, which is ~/.config by default. So I would like to store all my zsh user files in $XDG_CONFIG_HOME/zsh/. So far already have this:
% ls $XDG_CONFIG_HOME/zsh/
histfile zsh_cache zshrc
Easy, you just have to fill your ~/.zshrc. Now the trickiest part seems to make zsh read directly $XDG_CONFIG_HOME/zsh/zshrc without sourcing it from ~/.zshrc. How would you proceed?
One may edit /etc/zsh/zshenv to set $XDG_CONFIG_HOME directories and $ZDOTDIR. This require write privilegies on this files though.
So provided that $HOME is defined when zsh read it (I don't know if it's the case), you may add to your /etc/zsh/zshenv:
if [[ -z "$XDG_CONFIG_HOME" ]]
then
export XDG_CONFIG_HOME="$HOME/.config/"
fi
if [[ -d "$XDG_CONFIG_HOME/zsh" ]]
then
export ZDOTDIR="$XDG_CONFIG_HOME/zsh/"
fi
It is good practice to not put a / at the end of any variable holding a certain path.
For example, $XDG_CONFIG_HOME/zsh translates to "$HOME/.config//zsh" and the / repeats because XDG_CONFIG_HOME ends with a /.
So I think your answer should be -
if [[ -z "$XDG_CONFIG_HOME" ]]
then
export XDG_CONFIG_HOME="$HOME/.config"
fi
if [[ -d "$XDG_CONFIG_HOME/zsh" ]]
then
export ZDOTDIR="$XDG_CONFIG_HOME/zsh"
fi
Variation to psychoslave's answer which uses ${HOME}/.zshenv to initiate the environment. No root access needed.
export XDG_CONFIG_HOME=${XDG_CONFIG_HOME:=${HOME}/.config}
export ZDOTDIR=${ZDOTDIR:=${XDG_CONFIG_HOME}/zsh}
source $ZDOTDIR/.zshenv
This was discussed on this thread on the zsh-users mailing list.
You may want to consider saving history in XDG_DATA_HOME. Specifications can be found at XDG Base Directory Specification.
Write a wrapper for zsh that executes zsh after setting the environment variable ZDOTDIR to where you want zsh to look for the config files.
See: http://zsh.sourceforge.net/Intro/intro_3.html
Is there a standard way in a unixesque (sh/bash/zsh) system to execute a group of scripts as if the group of scripts was one script? (Think index.html). The point is to avoid additional helper scripts like you usually find and keep small programs self sufficient and easier to maintain.
Say I have two (in bold) ruby scripts.
/bin /bin/foo_master /bin/foo_master/main
/bin/foo_master/helper.rb
So now when I execute foo_master
seo#macbook ~ $foo_master [/bin/foo_master/main]: Make
new friends, but keep the old. [/bin/foo_master/helper.rb]: One
is silver and the other gold.
If you're trying to do this without creating a helper script, the typical way to do this would just be to execute both (note: I'll use : $; to represent the shell prompt):
: $; ./main; ./helper.rb
Now, if you're trying to capture the output of both into a file, say, then you can group these into a subshell, with parenthesis, and capture the output of the subshell as if it was a single command, like so:
: $; (./main; ./helper.rb) > index.html
Is this what you're after? I'm a little unclear on what your final goal is. If you want to make this a heavily repeatable thing, then one probably would want to create a wrapper command... but if you just want to run two commands as one, you can do one of the above two options, and it should work for most cases. (Feel free to expand the question, though, if I'm missing what you're after.)
I figured out how to do this in a semi-standard complaint fashion.
I used the eval syntax in shell scripting to lambda evaluate the $PATH at runtime. So in my /etc/.zshrc
$REALPATH = $PATH
$PATH = $REALPATH:`find_paths`
where find_paths is a function that recursively searches the $PATH directories for folders (pseudocode below)
(foreach path in $PATH => ls -d -- */)
So we go from this:
seo#macbook $ echo $PATH
/bin/:/usr/bin/
To this, automagically:
seo#macbook $ echo $PATH
/bin/:/usr/bin/:/bin/foo_master/
Now I just rename main to "foo_master" and voilà! Self contained executable, dare I say "app".
Yep that's an easy one!
#!/bin/bash
/bin/foo_master/main
/bin/foo_master/helper.rb
Save the file as foo_master.sh and type this in the shell:
seo#macbook ~ $sudo chmod +x foo_master.sh
Then to run type:
seo#macbook ~ $./foo_master.sh
EDIT:
The reason that an index.html file is served at any given directory is because the HTTP Server explicitly looks for one. (In server config files you can specify names of files to look for to server like index.html i.e. index.php index.htm foo.html etc). Thus it is not magical. At some point, a "helper script" is explicitly looking for files. I don't think writing a script like above is a step you can skip.