I have many part-00001, part-00002, ... files.
I want to use this way:
for ((i=0;i<1000;i++)); do <some command> <formatted string with i>; done.
How can I format "part-000xx"-like string with number i in zsh?
It could be done with:
typeset -Z 5 i (using builtin typeset -Z [N])
printf "part-%05d" $i (using builtin printf "%05d" $i)
${(l:5::0:)i} (using parameter expansion flags l:expr::string1:string2:)
like below:
typeset -Z 5 j
for ((i=0;i<1000;i++)); do
# <some command> <formatted string with i>
j=$i; echo "part-$j" # use $j here for sure the effects of below 2 lines
echo "$(printf "part-%05d" $i)"
echo "part-${(l:5::0:)j}"
done
# This outputs below:
# >> part-00000
# >> part-00000
# >> part-00000
# >> part-00001
# >> part-00001
# >> part-00001
# >> ...
# >> part-00999
Here is the description for each item.
typeset
-Z [N]
Specially handled if set along with the -L flag. Otherwise, similar to -R, except that leading zeros are used for padding instead of blanks if the first non-blank character is a digit. Numeric parameters are specially handled: they are always eligible for padding with zeroes, and the zeroes are inserted at an appropriate place in the output.
-- zshbuiltins(1), typeset, Shell Builtin Commands
printf
Print the arguments according to the format specification. Formatting rules are the same as used in C.
-- zshubuiltins(1), printf, Shell Builtin Commands
l:expr::string1::string2:
Pad the resulting words on the left. Each word will be truncated if required and placed in a field expr characters wide.
The arguments :string1: and :string2: are optional; neither, the first, or both may be given. Note that the same pairs of delimiters must be used for each of the three arguments. The space to the left will be filled with string1 (concatenated as often as needed) or spaces if string1 is not given. If both string1 and string2 are given, string2 is inserted once directly to the left of each word, truncated if necessary, before string1 is used to produce any remaining padding.
-- zshexpn(1), Parameter Expansion Flags
How can i compare values of each word for a given string
Sample string
i am over here
i am programmer (this line must start just after first lines i )
Requirement:
Need to check on which column(place) 1st string starts and have to ensure the second line must start from the same column(place).
Check first non blank like this:
lead=$(head -n1 file | grep -o '^ *')
col=$(echo $lead | wc -c)
((col--))
echo "column: $col"
Now remove first blanks of second row and add the appropriate amount of blanks.
sed -i "2s/^ */$lead/" file
Example:
$ cat file
haha
some test
$ bash script.sh
colummn: 6
$ cat file
haha
some test
Here script.sh contains the above lines of code.
I have a SESSION_TOKEN which gets generated dynamically every 30 mins. Its character length is greater than 530 and approximately 536 characters will be there in it.
How can i split this string in UNIX scripting. Need help.
You can use the "cut" utility for this kind of fixed length work:
echo "AAAABBBBCCCCDDDDEEEEFFFFGGGGHHHHIIIIJJJJKKK" | cut -c 10-20
CCCDDDDEEEE
The -c means "select by character" and the "10-20" says which characters to select.
You can also select by byte (using -b) which might make a difference if your data has some unusual encoding.
In your case, where you want to do multiple chunks of the same string, something like:
bradh#saxicola:~$ export somethingToChop="AAAABBBBCCCCDDDDEEEEFFFFGGGGHHHHIIIIJJJJKKK"
bradh#saxicola:~$ echo $somethingToChop | cut -c 1-10
AAAABBBBCC
bradh#saxicola:~$ echo $somethingToChop | cut -c 11-20
CCDDDDEEEE
bradh#saxicola:~$ echo $somethingToChop | cut -c 20-
EFFFFGGGGHHHHIIIIJJJJKKK
Would probably be the easiest to understand.
Bash variable expansion has substring operations built in:
$ string="abcdefghijklmnopqrstuvwxyz";
$ first=${string:0:8}
$ second=${string:8:8}
$ third=${string:16}
$ echo $first, $second, $third
abcdefgh, ijklmnop, qrstuvwxyz
I am using the following sdiff command to get the side-by-side difference of two files. Column width is given as one of the options
sdiff -w170 /tmp/captureFile /tmp/referenceFile (or diff -y )
if i use -w 130 then some characters are stripped. They do not appear in output even on next line. They are lost.
And if -w 170 is used then due to extra characters in the left column, right column is shifted and so few of its characters are seen in the left column part due to screen width being smaller.
So is there any option not to strip off the characters and have then on the next line in the same column of the sdiff command output?
What you are seeing (obviously) is either line truncation (-w 130) or line wrap (-w 170) relative to the line length in your terminal session. I don't believe there is an option to do what you desire. I've used sdiff a lot & tend to use a terminal/CLI that supports changing font sizes.
Shrink the font to something still readable & then maximise the window if possible.
Something else I've done is to 'fold' the two files before comparison to have a shorter line length - depends if you're on Linux or some Unix distro. but fold should be there.
Here is a quick and dirty script I wrote to implement #David Victor's suggestion :
$ cat SDIFF
if [ ! -n "${COLUMNS}" ]
then
echo COLUMNS is not exported !!!
echo run :
echo export COLUMNS
exit 1
fi
if [ ! -f "$1" -o ! -f "$2" ]
then
echo usage: $0 file1 file2
exit 1
fi
H=$(((${COLUMNS} - 3) / 2))
F1=$(mktemp)
F2=$(mktemp)
trap "rm $F1 $F2" 0
fold -s -w $H $1 > $F1
fold -s -w $H $2 > $F2
sdiff -w ${COLUMNS} $F1 $F2 | less
$
I would like to generate a random filename in unix shell (say tcshell). The filename should consist of random 32 hex letters, e.g.:
c7fdfc8f409c548a10a0a89a791417c5
(to which I will add whatever is neccesary). The point is being able to do it only in shell without resorting to a program.
Assuming you are on a linux, the following should work:
cat /dev/urandom | tr -cd 'a-f0-9' | head -c 32
This is only pseudo-random if your system runs low on entropy, but is (on linux) guaranteed to terminate. If you require genuinely random data, cat /dev/random instead of /dev/urandom. This change will make your code block until enough entropy is available to produce truly random output, so it might slow down your code. For most uses, the output of /dev/urandom is sufficiently random.
If you on OS X or another BSD, you need to modify it to the following:
cat /dev/urandom | env LC_CTYPE=C tr -cd 'a-f0-9' | head -c 32
why do not use unix mktemp command:
$ TMPFILE=`mktemp tmp.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX` && echo $TMPFILE
tmp.MnxEsPDsNUjrzDIiPhnWZKmlAXAO8983
One command, no pipe, no loop:
hexdump -n 16 -v -e '/1 "%02X"' -e '/16 "\n"' /dev/urandom
If you don't need the newline, for example when you're using it in a variable:
hexdump -n 16 -v -e '/1 "%02X"' /dev/urandom
Using "16" generates 32 hex digits.
uuidgen generates exactly this, except you have to remove hyphens. So I found this to be the most elegant (at least to me) way of achieving this. It should work on linux and OS X out of the box.
uuidgen | tr -d '-'
As you probably noticed from each of the answers, you generally have to "resort to a program".
However, without using any external executables, in Bash and ksh:
string=''; for i in {0..31}; do string+=$(printf "%x" $(($RANDOM%16)) ); done; echo $string
in zsh:
string=''; for i in {0..31}; do string+=$(printf "%x" $(($RANDOM%16)) ); dummy=$RANDOM; done; echo $string
Change the lower case x in the format string to an upper case X to make the alphabetic hex characters upper case.
Here's another way to do it in Bash but without an explicit loop:
printf -v string '%X' $(printf '%.2s ' $((RANDOM%16))' '{00..31})
In the following, "first" and "second" printf refers to the order in which they're executed rather than the order in which they appear in the line.
This technique uses brace expansion to produce a list of 32 random numbers mod 16 each followed by a space and one of the numbers in the range in braces followed by another space (e.g. 11 00). For each element of that list, the first printf strips off all but the first two characters using its format string (%.2) leaving either single digits followed by a space each or two digits. The space in the format string ensures that there is then at least one space between each output number.
The command substitution containing the first printf is not quoted so that word splitting is performed and each number goes to the second printf as a separate argument. There, the numbers are converted to hex by the %X format string and they are appended to each other without spaces (since there aren't any in the format string) and the result is stored in the variable named string.
When printf receives more arguments than its format string accounts for, the format is applied to each argument in turn until they are all consumed. If there are fewer arguments, the unmatched format string (portion) is ignored, but that doesn't apply in this case.
I tested it in Bash 3.2, 4.4 and 5.0-alpha. But it doesn't work in zsh (5.2) or ksh (93u+) because RANDOM only gets evaluated once in the brace expansion in those shells.
Note that because of using the mod operator on a value that ranges from 0 to 32767 the distribution of digits using the snippets could be skewed (not to mention the fact that the numbers are pseudo random in the first place). However, since we're using mod 16 and 32768 is divisible by 16, that won't be a problem here.
In any case, the correct way to do this is using mktemp as in Oleg Razgulyaev's answer.
Tested in zsh, should work with any BASH compatible shell!
#!/bin/zsh
SUM=`md5sum <<EOF
$RANDOM
EOF`
FN=`echo $SUM | awk '// { print $1 }'`
echo "Your new filename: $FN"
Example:
$ zsh ranhash.sh
Your new filename: 2485938240bf200c26bb356bbbb0fa32
$ zsh ranhash.sh
Your new filename: ad25cb21bea35eba879bf3fc12581cc9
Yet another way[tm].
R=$(echo $RANDOM $RANDOM $RANDOM $RANDOM $RANDOM | md5 | cut -c -8)
FILENAME="abcdef-$R"
This answer is very similar to fmarks, so I cannot really take credit for it, but I found the cat and tr command combinations quite slow, and I found this version quite a bit faster. You need hexdump.
hexdump -e '/1 "%02x"' -n32 < /dev/urandom
Another thing you can add is running the date command as follows:
date +%S%N
Reads nonosecond time and the result adds a lot of randomness.
The first answer is good but why fork cat if not required.
tr -dc 'a-f0-9' < /dev/urandom | head -c32
Grab 16 bytes from /dev/random, convert them to hex, take the first line, remove the address, remove the spaces.
head /dev/random -c16 | od -tx1 -w16 | head -n1 | cut -d' ' -f2- | tr -d ' '
Assuming that "without resorting to a program" means "using only programs that are readily available", of course.
If you have openssl in your system you can use it for generating random hex (also it can be -base64) strings with defined length. I found it pretty simple and usable in cron in one line jobs.
openssl rand -hex 32
8c5a7515837d7f0b19e7e6fa4c448400e70ffec88ecd811a3dce3272947cb452
Hope to add a (maybe) better solution to this topic.
Notice: this only works with bash4 and some implement of mktemp(for example, the GNU one)
Try this
fn=$(mktemp -u -t 'XXXXXX')
echo ${fn/\/tmp\//}
This one is twice as faster as head /dev/urandom | tr -cd 'a-f0-9' | head -c 32, and eight times as faster as cat /dev/urandom | tr -cd 'a-f0-9' | head -c 32.
Benchmark:
With mktemp:
#!/bin/bash
# a.sh
for (( i = 0; i < 1000; i++ ))
do
fn=$(mktemp -u -t 'XXXXXX')
echo ${fn/\/tmp\//} > /dev/null
done
time ./a.sh
./a.sh 0.36s user 1.97s system 99% cpu 2.333 total
And the other:
#!/bin/bash
# b.sh
for (( i = 0; i < 1000; i++ ))
do
cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 32 > /dev/null
done
time ./b.sh
./b.sh 0.52s user 20.61s system 113% cpu 18.653 total
If you are on Linux, then Python will come pre-installed. So you can go for something similar to the below:
python -c "import uuid; print str(uuid.uuid1())"
If you don't like the dashes, then use replace function as shown below
python -c "import uuid; print str(uuid.uuid1()).replace('-','')"