I am working on a project were we need to load flat file eg : (Gemstone_20220325.csv) I have given the source name as (Gemstone_*.csv) in script to search for the file in the path.
But it is failing with error , No such file .
Is that anything I am missing . Any idea on this is much appreciated .
You need to put either exact name or use a file list with the name of the file and then use indirect file type in the session that is reading the file.
You can use a pre session shell command like this ls -1 Gemstone_*.csv>/infa/home/tmp/Gemstone_filelist.txt. Or you can create a shell script too with this command for better control.
in the session that is reading this file, set the property to indirect file type and mention /infa/home/tmp/Gemstone_filelist.txt as file to be extracted.
Infa will pick files one by one and process them.
Once the file gets processed, delete it using a post session command task rm -f Gemstone_*..
When creating filepaths and URLs, I noticed that many times the path starts with ./ or ~/.
What is the difference between filepaths that start with ./ and ~/?
What do each of them mean?
For the sake of completeness ...
Just path is a file or directory named path in the current directory.
./path is a file or directory named path in the current directory, with the directory spelled out. The dot directory . represents the current directory, and path is the name of the file or directory within this directory.
~/path is a shorthand for $HOME/path where $HOME is a variable which refers to your home directory. Typically your home directory will be somewhere like /home/you or /Users/you where you is your account name. (The command echo "$HOME" will display your home directory.) The expanded value is an absolute path (unless you have messed up the value of $HOME thoroughly), as indicated by the initial slash.
/path is an absolute path which refers to a file or directory named path which is in the root directory /. Every file on Unix is ultimately somewhere in the directory tree which starts with the root directory.
A file name which begins with $ includes the value of a shell variable in its name (like for example $HOME above); you have to know the value of that variable to determine whether it ends up containing a relative or an absolute path. Similarly, ~ at the beginning of a file name gets replaced ("expanded") by the shell to a different string, as outlined above.
(Technically, it's possible for a file name to begin with a literal dollar sign or tilde, too; you would then have to quote or backslash-escape that character to avoid having the shell expand it to something else. This is rather inconvenient, so these file names tend to be rare in practice.)
In the following exposition, we refer to the result of any such replacements, and ignore the complication of possible quoting.
Every file name which begins with / is an absolute path (aka full path) which explains how to reach a particular node starting from the root directory. For example, /var/tmp/you/reminder.txt refers to a file or directory reminder.txt (probably a file, judging from the name; but Unix doesn't care what you call your files or directories) which is in the directory you which is in the directory tmp which is in the directory var which is in the root directory.
Every file name which doesn't begin with / is a relative path which indicates how to reach a particular file or directory starting from the current directory. The special directory .. is the parent directory (that is, the directory which contains this directory) and the special directory . is the current directory. So path/there refers to the file or directory there inside the directory path in the current directory; and (hover the mouse over the gray area to display the spoiler)
there/.././and/back/.. is a (wicked complicated) way to refer to the directory and in the current directory, where we traverse the there directory and then move back to the current directory; then stay in the current directory; then refer to the directory back inside the directory and, but then move back to the parent directory of that, ending up with ./and.
In addition to ~/ for the current user's home directory, some shells and applications allow the notation ~them/ to refer to the home directory of the user account them. Also, some web server configurations allow each user to have a public web site in their directory ~/public_html and the URL notation http://server/~them/ would serve up the site of the user account them for outside visitors.
The current directory is a convenience which the shell provides so you don't have to type long paths all the time. You can, if you want to.
/bin/ls /home/you/Documents/unix-101/directories.txt
is a longwinded but perfectly valid way to say (assuming you are in your home directory),
ls Documents/unix-101/directories.txt
You could also say
cd Documents/unix-101
ls directories.txt
and until you cd again, all your commands will run in this directory.
See What exactly is current working directory? for a longer exposition of this related concept.
A "directory" is sometimes called a "folder" by people who are not yet old enough to prefer the former.
Tangentially, don't confuse the directory name . with the Bourne shell command which comprises a single dot (also known by its Bash alias source). The command
. ./scriptname
runs the commands from the file ./scriptname in the context of the current shell instance, as opposed to in a separate subshell (which is what just ./scriptname does). In other words, this command line invokes the dot command on a file scriptname in the dot directory.
The Bourne shell (and derivatives like Bash, Zsh, etc) use single quotes to prevent variable expansion and wildcard expansion, and double quotes to permit variable expansion, but inhibit wildcard expansion in a string. The quoting rules on Windows are different, and generally use double quotes to keep whitespace-separated values as a single string (and % instead of $ for variable substitutions).
./ means "starting from the current directory". . refers to the current working directory, so something like ./foo.bar would be looking for a file called foo.bar in the current directory. (As a side note, .. means refers to the parent directory of the current directory. So ../foo.bar would be looking for that file one directory above.)
~/ means "starting from the home directory". This could have different meanings in different scenarios. For example, in a Unix environment ~/foo.bar would be looking for a file called foo.bar in your home directory, something like /home/totzam/foo.bar. In many web applications, ~/foo.bar would be looking for a file called foo.bar in the web application root, something like /var/http/mywebapp/foo.bar.
./ is the current directory
~/ is the home directory of the current user
./ means that path is relative to your current position.
~/ means that path is relative to your home directory.
I will explain a simple example of it. As developers mentioned:
./ is current directory.
~/ is the home directory of the current user.
How both of the file path expressions can help us? Suppose you want to execute a script (.sh) and you're in the same directory where file exists then you can simply do it ./filename.sh
I mostly use ~/ to access my home directory files like .bashrc when I want to add any config in it. It's easier since the file path expression (for home directory) feels much easier and makes accessibility to the file from anywhere, without worrying about the path or changing the path.
. represents current directory
.. represents the parent directory
~ represents the home directory for the current user. Home directory is also represented by HOME env variable. you can do echo $HOME on the shell to see it.
These are generally used to specify relative paths. The / in the end of each notation is a separator that you can use when using these notations together.
Ex:
$ cd ../.. # Go 2 directories backwards
$ cd ~ # Takes you to $HOME directory
$ cd . # Does nothing :) As it literally means go to the directory that you are already present in.
$ cd ~/dir1 $ go to `$HOME/dir1`
On Unix, in any directory if you do ls -a you would see that . and .. will be mentioned (even for empty directory). Like mentioned, these have special meaning and are generated by default in Unix systems and are generally helpful to specify relative paths (i.e, path to a different directory relative to your current directory)
cd command is harmless. So, just play around by combining notations with cd command. You will eventually get a grip of them.
I' m trying to zip a folder that contains some .json files plus subfolders with more .json files. I need to zip the parent folder with everything included without containing any of the parent or subfolders paths. Is there any way I can do this?
Thank you
EDIT:
I want this:
pending_provider/722c9cb2-268b-4e4a-9000-f7f65e586011-version/1d296136-ac87-424e-89c4-682a63c7b853.json (deflated 68%)
But not this:
pending_provider/722c9cb2-268b-4e4a-9000-f7f65e586011-version/ (stored 0%)
I want to avoid the "stor" compression type which only saves the folder path. I want only the "defN"
So the -j doesn't help me a lot
Thanks
If you don't want any paths at all you could use the -j option. Below is the man page entry for the option.
-j
--junk-paths
Store just the name of a saved file (junk the path), and do not store
directory names. By default, zip will store the full path (relative to
the current directory).
If you just want to exclude directories
-D
--no-dir-entries
Do not create entries in the zip archive for directories.
I want to create a command that runs an executable created by compiling a c program. I couldn't find a proper solution. Let's say I have a file named myprogram.c and compile it and have myprogram as . I want to type myprogram in any folder in my system and run it. How can I achieve this?
First find out what your PATH is
echo $PATH
For you this outputs
/sbin:/usr/sbin:/bin:/usr/bin:/usr/pkg/sbin:/usr/pkg/bin/usr/X11R7/bin:usr/X11R6/bin:/usr/local/sbin:/usr/local/bin
Then assuming your program is in the /usr/myprog directory, append /usr/myprog to your PATH (don't forget to separate directories with a colon :)
export PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/pkg/sbin:/usr/pkg/bin/usr/X11R7/bin:usr/X11R6/bin:/usr/local/sbin:/usr/local/bin:/usr/myprog
Doing this tells the system when you don't specify an absolute path (like ./myprogram) to look in all the directories in PATH. It is good to add the absolute path of your executable to PATH because adding . to your PATH is frowned upon by some (see this question).
You have to add it in your PATH from your shell rc file
You place the executable into a directory that your shell already searches for programs, or you add your program's location to that list.
the $PATH environment variable contains this information. You can add myProgram's location to it, i.e. export PATH=$PATH:/new/dir, or just print out $PATH and copy myProgram into one of the paths listed there already.
I am attempting to use rsync to copy files, but I want to not copy hidden files and folders, and there is one ordinary file I want excluded from the file transfer. I believe I am eliminating the hidden folders with the --exclude="./" and I believe I am excluding the hidden file with the --exclude file path option. If I eliminate the --exclude file path option, I don't get any errors, but that file is copied, which I do not want. If I eliminate the --excluude="./" the hidden files are copied, which I do not want either. What am I doing wrong?
mbp:~ username $ rsync —-exclude /Users/username/work/java/textsearch/settings/search_config.properties --exclude=".*/" -avz /Users/username/work/java/ root#remote.local:/usr/local/java/ -n
building file list ... rsync: link_stat "/Users/username/?\#200\#224-exclude" failed: No such file or directory (2)
done
sent 9560 bytes received 20 bytes 6386.67 bytes/sec
total size is 17461760 speedup is 1822.73
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9]
1) What is /Users/username/?#200#224-exclude and why is rsync looking for it?
2) How do I get rsync to copy everything except the hidden folders/files and the specified file?
If this is an exact copy of the command line, the "--" in front of exclude is not using the correct characters. Delete this and replace with double minus. What happens is, that rsync doesn't recognize the option, instead searching the user directory for the file "—-exclude"