Zsh filling associative array with read from file leads to strange separation - zsh

I have a textfile generated by
find Path -printf '%s\t%p\n' > textfile
When I do
declare -A DICT;
while IFS='\t' read -r SIZE PFAD
do DICT[$SIZE]=$PFAD
done < ../Listen/textfile
the content of DICT surprises me:
print "${(#k)DICT}"
shows, that the keys of DICT are not just the SIZE of the files, but consist of
SIZE\tRoot_of_PFAD/2_letters_of_following_directory.
The values contain the rest of the line = Rest of the path with the filename.
Looks to me as if read separates the lines by '\t+9 characters'

IFS=$(printf '\t')
seems to have done the trick.
#Gairfowl hinted in the right direction.
I hadn't grasped, that the tenth character in the path was a t.
Thank you very much!

Related

Generate an associative array from command output

I am writing a zsh completion function to complete IDs from a database. There is a program listnotes which outputs a list like this:
bf848bf6-63d2-474b-a2c0-e7e3c4865ce8 Note Title
aba21e55-22c6-4c50-8bf6-bf3b337468e2 Another one
09ead915-bf2d-449d-a943-ff589e79794a yet another "one"
...
How do I generate an associative array note_ids from the output of the listnotes command such that I get an associative array like this?
( bf848bf6-63d2-474b-a2c0-e7e3c4865ce8 "Note Title" aba21e55-22c6-4c50-8bf6-bf3b337468e2 "Another one" 09ead915-bf2d-449d-a943-ff589e79794a "yet another \"one\"" )
Note that there may be whitespace in the keys. I tried to generate something with sed:
note_ids=($(listnotes | sed 's/^\(.*\) \(.*\)$/\1 "\2"/'))
but quoting strings like this doesn’t seem to work, and double quotes in the title make it even more difficult.
Try something like
typeset -A note_ids
for line in ${(f)"$(listnotes)"}; do
note_ids+=(${line%% *} ${line#* })
done
${(f)PARAM}: split the result of the expansion of $PARAM at newlines
"$(listnotes)": put the output of listnotes verbatim into the expansion.
for line in LIST: iterate over the items in LIST as split by ${(f)…}.
note_ids+=(key value): add key-value pair to an the associative array note_ids
${line%% *}: cut the largest portion matching " *" (a space followed by anything) from the end of the expansion of line. So remove everying after including the first space, leaving only the key.
${line#* }: cut the smallest portion matching "* " (anything followed by three spaces) from the beginning of the expansion of $line. So remove the key and the three spaces used as separator.
Instead of using the parameter expansion flag (f) you could also read the output of listnotes line by line with read:
listnotes | while read; do
note_ids+=(${REPLY%% *} ${REPLY#* })
done
Unless specified otherwise read puts the read values into the REPLY parameter.

.ksh paste user input value into dataset

Good morning.
First things first: I know next to nothing about shell scripting in Unix, so please pardon my naivety.
Here's what I'd like to do, and I think it's relatively simple: I would like to create a .ksh file to do two things: 1) take a user-provided numerical value (argument) and paste it into a new column at the end of a dataset (a separate .txt file), and 2) execute a different .ksh script.
I envision calling this script at the Unix prompt, with the input value added thereafter. Something like, "paste_and_run.ksh 58", where 58 would populate a new, final (un-headered) column in an existing dataset (specifically, it'd populate the 77th column).
To be perfectly honest, I'm not even sure where to start with this, so any input would be very appreciated. Apologies for the lack of code within the question. Please let me know if I can offer any more detail, and thank you for taking a look.
I have found the answer: the "nawk" command.
TheNumber=$3
PE_Infile=$1
Where the above variables correspond to the third and first arguments from the command line, respectively. "PE_Infile" represents the file (with full path) to be manipulated, and "TheNumber" represents the number to populate the final column. Then:
nawk -F"|" -v TheNewNumber=$TheNumber '{print $0 "|" TheNewNumber/10000}' $PE_Infile > $BinFolder/Temp_Input.txt
Here, the -F"|" dictates the delimiter, and the -v dictates what is to be added. For reasons unknown to myself, the declaration of a new varible (TheNewNumber) was necessary to perform the arithmetic manipulation within the print statement. print $0 means that the whole line would be printed, while tacking the "|" symbol and the value of the command line input divided by 10000 to the end. Finally, we have the input file and an output file (Temp_PE_Input.txt, within a path represented by the $Binfolder variable).
Running the desired script afterward was as simple as typing out the script name (with path), and adding corresponding arguments ($2 $3) afterward as needed, each separated by a space.

How to use a variable to find a specific line in a file

I am using a variable("Address") that is read in from a file("finaladdress.txt"). The variable has info that I need to use to find a line from another file(cityfile.txt). The variable named "Address" as mentioned above contains the string "2320 Wisconsin NE, WA" I need to use that to grep the cityfile.txt. The grep part is the main problem. It tells me that Wisconsin: no such file exists, NE: no such file exists and so on. I hope I explained clearly enough here is the code snippet.
<code>
read Address < finaladdress.txt
grep $Address cityfile.txt
</code>
Quote your variable with double quotes:
grep "$Address" cityfile.txt
If you want know only whether the row exists, you should enclose the variable because of the spaces.
grep "$Address" cityfile.txt

Decrypt many PDFs in one go using pdftk

I have 10 PDFs that ask for a user password to open. I know that password. I want to keep them in a decrypted format. Their filenames follow the form:
static_part.dynamic_part_like_date.pdf
I want to convert all the 10 files. I can give a * after the static part and work on all of them, but I also want the corresponding output filenames. So there has to be a way to capture the dynamic part of the filename and then use it in the output filename.
The normal way of doing this for one file is:
pdftk secured.pdf input_pw foopass output unsecured.pdf
I want to do something like:
pdftk var=secured*.pdf input_pw foopass output unsecured+var.pdf
Thanks.
Your request is a little ambiguous, but here are some ideas that might help you.
Assuming 1 of your 10 files is
# static_part.dynamic_part_like_date.pdf
# SalesReport.20110416.pdf (YYYYMMDD)
And you want only the SalesReport.pdf converted as unsecured, you can use a shell script to achieve your requirement:
# make a file with the following contents,
# then make it executable with `chmod 755 pdfFixer.sh`
# the .../bin/bash has to be the first line the file.
$ cat pdfFixer.sh
#!/bin/bash
# call the script like $ pdfFixer.sh staticPart.*.pdf
# ( not '$' char in your command, that is the cmd-line prompt in this example,
# yours may look different )
# use a variable to hold the password you want to use
pw=foopass
for file in ${#} ; do
# %%.* strips off everything after the first '.' char
unsecuredName=${file%%.*}.pdf
#your example : pdftk secured.pdf input_pw foopass output unsecured.pdf
#converts to
pdftk ${file} input_pw ${foopass} output ${unsecuredName}.pdf
done
You may find that you need to modify the %.* thing to
strip less from end, (use %.*) to strip just the last '.' and all chars after (strip from right).
strip from the fron (use #*.) to just the static part, leaving the dynamic part OR
strip from the front (use ##*.) to strip everything until the last '.' char.
It will really be much easier for you to figure out what you need at the cmd-line.
Set a variable with 1 sample fileName
myTestFileName=staticPart.dynamicPart.pdf
and then use echo combined with the variable modifiers to see the results.
echo ${myTestFileName##*.}
echo ${myTestFileName#*.}
echo ${myTestFileName##.*}
echo ${myTestFileName#.*}
echo ${myTestFileName%%.*}
etc.
Also notice how I combine a modified variable value with a plain string (.pdf), in unsecuredName=${file%%.*}.pdf
IHTH

replace a string in huge file by incrementing with it number of appearance in file?

I have a file of 500MB and it has strings :
string_1 ..... string_500,
I need generate the copy of this file which has :
string_501.......string_1000
I need to do this up to string_500000, what's the best way to solve this?
If it is literally as you describe, a constant string with variable suffixes then just generate the new file forgetting the old one.
If it's actually
wibble_1 something_2 that_3 changes_4 randomly_5
the I'd read and parse the thing in perl
If you just want to generate a sequence of strings (string_501 to string_500000), you can do this:
for i in `seq 501 500000`
do
echo string_${i}
done

Resources