I was wondering if there is a way to change the values of variables inside of a compiled IDL .pro file using a bunch of .txt files that contain the values.
For example I have 3 .txt files with 2 lines in each file. Lets just call them:
1. C:\input1.txt
2. C:\input2.txt
3. C:\input3.txt
Where the contents are something like:
hello
world
And the .pro I have looks like this:
pro tst1
common vars, a, b
infile = 'C:\input1.txt'
a =''
b =''
openr,lun, infile, /get_lun
readf,lun,a
readf,lun,b
end
pro tst2
common vars, a, b
tst1
print,a, b
end
What I want to do is change the infile variable for each iteration until all 3 input.txt files have been read.
Unfortunately, I do need to have the common blocks and the infile in the first pro. I am trying to automate a big nasty .pro I got (like my simple example) and its driving me a bit bonkers.
Something like:
pro tst1, infile
common vars, a, b
a = ''
b = ''
openr, lun, infile, /get_lun
readf, lun, a
readf, lun, b
free_lun, lun
end
pro tst2
common vars, a, b
for i = 1, 3 do begin
tst1, string(i, format='(%"C:\infile%d.txt")')
print,a, b
endfor
end
Related
I'm trying to build a diff viewer plugin for my text editor (Kakoune). I'm comparing two files and marking any lines that are different across two panes.
However the two views don't scroll simultaneously. My idea is to get a list of line numbers that correspond to each other, so I can place the cursor correctly when switching panes, or scroll the secondary window when the primary one scrolls, etc.
So - Is there a way to get a list of corresponding numbers from commandline diff?
I'm hoping for something along the following example: Given file A and B, the output should tell me which line numbers (of the ones that didn't change) would correspond.
File A File B Output
1: hello 1: hello 1:1
2: world 2: another 2:3
3: this 3: world 3:4
4: is 4: this 4:5
5: a 5: is
6: test 6: eof
The goal is that when I scroll to line 4 in file A, I'll know to scroll file B such that it's line 5 is rendered at the same position.
Doesn't have to be Gnu diff, but should only use tools that are available on most/all linux boxes.
I can get some of the way using GNU diff, but then need a python script to post-process it to turn a group-based output into line-based.
#!/usr/bin/env python
import sys
import subprocess
file1, file2 = sys.argv[1:]
cmd = ["diff",
"--changed-group-format=",
"--unchanged-group-format=%df %dl %dF %dL,",
file1,
file2]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
output = p.communicate()[0]
for item in output.split(",")[:-1]:
start1, end1, start2, end2 = [int(s) for s in item.split()]
n = end1 - start1 + 1
assert(n == end2 - start2 + 1) # unchanged group, should be same length in each file
for i in range(n):
print("{}:{}".format(start1 + i, start2 + i))
gives:
$ ./compare.py fileA fileB
1:1
2:3
3:4
4:5
I want to split single line into multiple line of 8 bytes each. And I am using the fold command and since this file contains the special characters the fold command does not work and it breaks in the middle of multibyte character.
File Content
あいbbえおかcc髙①こさし㈱㈱ちつて髙aabbc
Command Used
fold -b8 dummy_file.dat
Appreciate any help on this.
The problem here is that your text contains multi-bytes characters that will be broken by the fold command if we split them on 2 lines.
echo "あいbbえおかcc髙①こさし㈱㈱ちつて髙aabbc" | fold -b8
あいbb
えお��
�cc髙��
�こさ�
��㈱㈱
ちつ��
�髙aabb
c
If you want to have 8 characters per line you can use the following sed command:
echo "あいbbえおかcc髙①こさし㈱㈱ちつて髙aabbc" | sed 's/.\{8\}/&\n/g'
あいbbえおかc
c髙①こさし㈱㈱
ちつて髙aabb
c
that add a breakline after each occurrence of 8 characters.
If you do not want to display 8 characters but want to constraint each line to be at most 8 bytes without breaking the text content then you can use the python script:
import sys
def utf8len(s):
return len(s.encode('utf-8'))
entry = unicode(sys.stdin.read(),'utf-8')
tmp = ''
for c in entry:
if utf8len(tmp)+utf8len(c) > 8:
print tmp
tmp = c
elif utf8len(tmp)+utf8len(c) == 8:
print tmp,c
tmp = ''
else:
tmp += c
if tmp:
print tmp
output:
echo -n "あいbbえおかcc髙①こさし㈱㈱ちつて髙aabbc" | python max8bytes.py
あいb b
えお
かcc 髙
①こ
さし
㈱㈱
ちつ
て髙a a
bbc
Explanations:
You define a function that will count how many bytes you have per char.
You read char by char stdin and you avoid to have more than 8 bytes on the same line. If you do not want to have less than you can add some spaces char at the end of each line.
I have a list of files that do have the identical filename but are in different subfolders. The values in the files are seperated with a tab.
I would like to attach to all of the files "test.txt" an additional first column with the foldername and if merge to one file in the end (they all have the same header for the columns).
The most important command though would be the merging.
I have tried to many commands now that did not work so I guess I am missing an essential step with awk...
Current structure is:
mainfolder
|_>Folder1
|_>test.txt
|->Folder2
|_>test.txt
.
.
.
This is where I would like to get to per file before merging all of the,
#Name Count FragCount Type Left LeftB Right RightB Support FRPM LeftBD LeftBE RightBD RightBE annots
RFP1A 13 10 REF RFP1A_ins chr3:3124352:+ RFP1A_ins chr3:5234143:+ confirmed 0.86 TA 1.454 AC 1.564 ["INTRACHROM."]
#Samplename #Name Count FragCount Type Left LeftB Right RightB Support FRPM LeftBD LeftBE RightBD RightBE annots
Sample1 RFP1A 13 10 REF RFP1A_ins chr3:3124352:+ RFP1A_ins chr3:5234143:+ confirmed 0.86 TA 1.454 AC 1.564 ["INTRACHROM."]
Thanks so much!!
D
I believe this might do the trick:
$ cd mainfolder
$ awk '(NR==1){sub("#","#Samplename\t"); print} # print header
(FNR==1){next} # skip header
{print substr(FILENAME,1,match(FILENAME,"/")-1)"\t"$0 } # add directory
' */test.txt > /path/to/newfile.txt
I am banging my head against the wall when trying to perform a drop duplicate for time series, base on the value of a datetime index.
My function is the following:
def csv_import_merge_T(f):
dfsT = [pd.read_csv(fp, index_col=[0], parse_dates=[0], dayfirst=True, names=['datetime','temp','rh'], header=0) for fp in files]
dfT = pd.concat(dfsT)
#print dfT.head(); print dfT.index; print dfT.dtypes
dfT.drop_duplicates(subset=index, inplace=True)
dfT.resample('H').bfill()
return dfT
which is called by:
inputcsvT = ['./input_csv/A08_KI_T*.csv']
for csvnameT in inputcsvT:
files = glob.glob(csvnameT)
print ('___'); print (files)
t = csv_import_merge_T(files)
print csvT
I receive the error
NameError: global name 'index' is not defined
what is wrong?
UPDATE:
The issue appear to arise when csv input files (which are to be concatenated) are overlapped.
inputcsvT = ['./input_csv/A08_KI_T*.csv'] gets files
A08_KI_T5
28/05/2015 17:00,22.973,24.021
...
08/10/2015 13:30,24.368,45.974
A08_KI_T6
08/10/2015 14:00,24.779,41.526
...
10/02/2016 17:00,22.326,41.83
and it runs correctly, whereas:
inputcsvT = ['./input_csv/A08_LR_T*.csv'] gathers
A08_LR_T5
28/05/2015 17:00,22.493,25.62
...
08/10/2015 13:30,24.296,44.596
A08_LR_T6
28/05/2015 17:00,22.493,25.62
...
10/02/2016 17:15,21.991,38.45
which leads to an error.
IIUC you can call reset_index and then drop_duplicates and then set_index again:
In [304]:
df = pd.DataFrame(data=np.random.randn(5,3), index=list('aabcd'))
df
Out[304]:
0 1 2
a 0.918546 -0.621496 -0.210479
a -1.154838 -2.282168 -0.060182
b 2.512519 -0.771701 -0.328421
c -0.583990 -0.460282 1.294791
d -1.018002 0.826218 0.110252
In [308]:
df.reset_index().drop_duplicates('index').set_index('index')
Out[308]:
0 1 2
index
a 0.918546 -0.621496 -0.210479
b 2.512519 -0.771701 -0.328421
c -0.583990 -0.460282 1.294791
d -1.018002 0.826218 0.110252
EDIT
Actually there is a simpler method is to call duplicated on the index and invert it:
In [309]:
df[~df.index.duplicated()]
Out[308]:
0 1 2
index
a 0.918546 -0.621496 -0.210479
b 2.512519 -0.771701 -0.328421
c -0.583990 -0.460282 1.294791
d -1.018002 0.826218 0.110252
I wrote this code :
1 #!/bin/tcsh
2
3 set myFiles = `ls`
4 set i = 1;
5 echo "argc is $#argv"
6 while ($i <= $#argv)
7 $myFiles = `echo $myFiles | tr "$argv[$i]" " "`
8 echo "argv now is $argv[$i]"
9 echo "my files are : $myFiles"
10 # i++;
11 end
12 echo "my files post proccess are $myFiles"
13 foreach name ($myFiles)
14 set temp = `cat $name`
15 echo "temp is : $temp"
16 unset temp
17 end
This piece should get a list of file names within the current folder, and print the content of the files that are not specified
IE : folder has the files : A B C D E
and the input is : A B C
so the content of D E will be printed.
now the logic is right, but I have some syntactic issues regarding line 7 (the tr)
I've tried with sed as well, but I get "permission denied" to the console for some reason, and I really don't know how to fix it.
So the help I need is actually syntactic regarding assigning a variable with commands output plus including other variables within those commands.
Hope that's alright..
THANKS !
Please note that tr will replace all matching characters, so if your input includes "A", it will replace all "A" with " " in all file names returned by ls.
There is a much cleaner solution. You want to find all files, exclude files matching the input and print what is left. Here you go:
#!/bin/tcsh
set exclude_names = ""
# if any argument is passed in, add it as "! -name $arg"
foreach arg ( $* )
set exclude_names = "$exclude_names ! -name $arg"
end
# Find all files in the current dir, excluding the input
# then print out the filtered set of files
find . -maxdepth 1 -type f $exclude_names -exec cat {} +