Converting unknown binary data into series of numbers? (with a known example) - unix

Question

I'm trying to find a way to convert files in a little-used archaic file format into something human readable...
As an example, od -x myfile gives:
0000000 2800 4620 1000 461e c800 461d a000 461e
0000020 8000 461e 2800 461e 5000 461f b800 461e
0000040 b800 461d 4000 461c a000 461e 3800 4620
0000060 f800 4621 7800 462a e000 4622 2800 463c
0000100 2000 464a 1000 4654 8c00 4693 5000 4661
0000120 7000 46ac 6c00 46d1 a400 4695 3c00 470a
0000140 b000 46ca 7400 46e9 c200 471b 9400 469e
0000160 9c00 4709 cc00 4719 4000 46b0 6400 46cc
...
which I know corresponds to these integers:
10250 10116 10098 10152 10144 10122 10196 10158
10094 10000 10152 10254 10366 10910 10424 12042
12936 13572 18886 14420 22072 ...
but I have no idea how to convert one to the other!!
Many many thanks to anyone who can help.
If possible, general tips for what to try/where to begin in this situation would also be appreciated.
Update: I put the full binary file online here http://pastebin.com/YL2ApExG and the numbers it corresponds to here http://pastebin.com/gXNntsaJ
In the hex dump, it seems to alternate between four digits, presumably they correspond to the numbers I want? separated either by 4600 or 4700. Unfortunately, I don't know where to go from here!
Someone else asked below: the binary file is a .dat file generated by an old spectroscopy program... it's 1336 bytes and corresponds to 334 integers, so it's four bytes per integer.

Answer 1

Well this is what you can do -
Step I: Do the od -x of the file and redirect it to a temp file (eg. hexdump.txt)
od -x myfile > hexdump.txt
Step II: You will now have a text file that contains hexadecimal values which you can view using the cat command. Something like this -
[jaypal~/Temp]$ cat hexdump.txt
0000000 2800 4620 1000 461e c800 461d a000 461e
0000020 8000 461e 2800 461e 5000 461f b800 461e
0000040 b800 461d 4000 461c a000 461e 3800 4620
0000060 f800 4621 7800 462a e000 4622 2800 463c
0000100 2000 464a 1000 4654 8c00 4693 5000 4661
0000120 7000 46ac 6c00 46d1 a400 4695 3c00 470a
0000140 b000 46ca 7400 46e9 c200 471b 9400 469e
0000160 9c00 4709 cc00 4719 4000 46b0 6400 46cc
Step III: The first column isn't really important to you. Columns 2 thru 9 are important. We will now strip the file using AWK so that you can convert it to decimal. We will add space so that we can consider each value as an individual field. We will also add "0x" to it so that we can pass it as a hexadecimal value.
[jaypal~/Temp]$ awk '{for (i=2;i<=NF;i++) printf "0x"$i" "}' hexdump.txt > hexdump1.txt
[jaypal~/Temp]$ cat hexdump1.txt
0x2800 0x4620 0x1000 0x461e 0xc800 0x461d 0xa000 0x461e 0x8000 0x461e 0x2800 0x461e 0x5000 0x461f 0xb800 0x461e 0xb800 0x461d 0x4000 0x461c 0xa000 0x461e 0x3800 0x4620 0xf800 0x4621 0x7800 0x462a 0xe000 0x4622 0x2800 0x463c 0x2000 0x464a 0x1000 0x4654 0x8c00 0x4693 0x5000 0x4661 0x7000 0x46ac 0x6c00 0x46d1 0xa400 0x4695 0x3c00 0x470a 0xb000 0x46ca 0x7400 0x46e9 0xc200 0x471b 0x9400 0x469e 0x9c00 0x4709 0xcc00 0x4719 0x4000 0x46b0 0x6400 0x46cc
Step IV: Now we will convert each hexadecimal value into decimal using printf function with AWK.
[jaypal~/Temp]$ gawk --non-decimal-data '{ for (i=1;i<=NF;i++) printf ("%05d ", $i)}' hexdump1.txt > hexdump2.txt
[jaypal~/Temp]$ cat hexdump2.txt
10240 17952 04096 17950 51200 17949 40960 17950 32768 17950 10240 17950 20480 17951 47104 17950 47104 17949 16384 17948 40960 17950 14336 17952 63488 17953 30720 17962 57344 17954 10240 17980 08192 17994 04096 18004 35840 18067 20480 18017 28672 18092 27648 18129 41984 18069 15360 18186 45056 18122 29696 18153 49664 18203 37888 18078 39936 18185 52224 18201 16384 18096 25600 18124
Step V: Formatting to make it easily readable
[jaypal~/Temp]$ sed 's/.\{48\}/&\n/g' < hexdump2.txt > hexdump3.txt
[jaypal~/Temp]$ cat hexdump3.txt
10240 17952 04096 17950 51200 17949 40960 17950
32768 17950 10240 17950 20480 17951 47104 17950
47104 17949 16384 17948 40960 17950 14336 17952
63488 17953 30720 17962 57344 17954 10240 17980
08192 17994 04096 18004 35840 18067 20480 18017
28672 18092 27648 18129 41984 18069 15360 18186
45056 18122 29696 18153 49664 18203 37888 18078
39936 18185 52224 18201 16384 18096 25600 18124