I have to read data from an archaic fixed-width table. Some of the fields are in the IBM packed decimal Format, and I have no idea how to get the numbers.
for example, one line of my Input file might look like this:
140101 72 ‘ëm§1 1
where ‘ëm§1 is actually the packed decimal representation of the number 3153947. SAS can read this field correctly via INPUT gew pd4.4
Anyone any expericnes with this issue?
Related
When I run a proc print where segment ID equals 1234 the output shows segment ID 1235. SAS actually changes the last 4 digits of a 19 digit number. Contents shows the field in a num 8 formatted as a char 20. I just pull the data and print with no additional formatting or processing.
If I run a SQL statement in a different software package where segment ID equals 1234 (the exact same record) the results show 1234 (no change to the last 4). The other vars pulled with the query exactly match those of SAS except for the segment ID.
My best guess is it's a formatting issue even though the field should be large enough, 20 > 19.
Second guess is some sort of encryption on the field. Typically if I don't have proper access a field would be blank. But I am unfamiliar with this new data source.
I'll try adding a specific format to my SAS datapull for that field but would love to hear any other suggestions.
Thank you!
PROC PRINT is not the issue. You cannot store 19 decimal digits exactly as a number in SAS. SAS stores numbers as 64-bit floating point numbers. The maximum number of decimal digits that can be represented as consecutive integers is 15. After that the binary representation will not have enough bits to exactly represent every decimal string.
Check this description about precision from the documentation: https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/lrcon/p0ji1unv6thm0dn1gp4t01a1u0g6.htm
You should store such things as character strings. I doubt that you need to do any arithmetic with those values.
If you are getting the data from a remote database use the DBSASTYPE= dataset option to control what type of SAS variable is created.
Contents shows the field in a num 8 formatted as a char 20. I just pull the data and print with no additional formatting or processing.
That doesn't make sense. A numeric variable shouldn't have a character format.
I think you'll need to re-read the source field as a character from the source. Not sure where you "just pull the data" from, but you'll need to modify that step to ensure it gets brought over as a character in the first place otherwise you'll have that issue no matter what.
You cannot fix the issue with the data as is, as far as I know.
*issue with numeric;
data have;
input segmentID;
format segmentID 32.;
segmentIDChar=put(SegmentID, 32.);
cards;
1234567890123456789
;;;;
run;
proc print data=have;
run;
*no issue with character fields;
data have;
length segmentID $20.;
input segmentID $;
format segmentID $32.;
cards;
1234567890123456789
;;;;
run;
proc print data=have;
run;
Results:
Numeric Isssue
Obs segmentID segmentIDChar
1 1234567890123456768 1234567890123456768
Character Issue
Obs segmentID
1 1234567890123456789
I'd like to be compare binary or hexadecimal numbers to a range in R.
Specifically, I have hex codes of aircrafts (e.g. 4067F7) and I'd like to know which country that code is allocated to (e.g. UK). The country allocations are given as binary ranges.
cf. the table here: http://www.kloth.net/radio/icao24alloc.php
I figured that I can convert from hex to binary with hex2bin but after that I'm stuck as I can't figure out how to compare the binaries to the ranges.
Thanks!
I have a project about Visible Light Communication. I need to get user input (e.g sentence) and convert this sentence into binary values so that I can turn on-off the LED.
So far I managed to transmit data but i am having issue with how to convert this string into binary.
One of the ideas that i came up with is to convert this string into 1D array with 1s and 0s. Each 8 binary element in the array represents the ascii equvialant. However i couldnt manage to achieve this task.
if you guys give me another method to complete this, it'll be greatly appriciated.
I'm sure there is a better way of doing this, but i couldnt find it so far.
If the text is only A-Z and a-z i.e. Without any special characters like spaces, periods, commas, $&()#, etc. then you can encode them with your own encoding e.g. A=0 B=1 .... a= 26 ... z = 51. Doing this, you will need 6 bits to represent each character.
On the other hand, you could use more advanced algorithms like Huffman coding to encode each character.
However, if you're having problems with transmitting the ASCII representation, you'll have a problem with those two options. I'd say stick with the ASCII encoding till you have a stronger reason to try something else.
I wanted to know the structure of an unknown binary file generated by Fortran routine. For the same I downloaded hex editor. I am fairly new to the whole concept. I can see some character strings in the conversion tool. However, the rest is just dots and junk characters.
I tried with some online converter but it only converts to the decimal systems. Is there any possible way to figure out that certain hex represents integer and real?
I also referred to following thread, but I couldn't get much out of it.
Hex editor for viewing combined string and float data
Any help would be much appreciated. Thanks
The short answer is no. If you really know nothing of the format, then you are stuck. You might see some "obvious" text, in some language, but beyond that, it's pretty much impossible. Your Hex editor reads the file as a group of bytes, and displays, usually, ASCII equivalents beside the hex values. If a byte is not a printable ASCII character, it usually displays a .
So, text aside, if you see a value $31 in the file, you have no way of knowing if this represents a single character ('1'), or is part of a 2 byte word, or a 4 byte long, or indeed an 8 byte floating point number.
In general, that is going to be pretty hard! Some of the following ideas may help.
Can you give the FORTRAN program some inputs that make it change the length/amount of output data? E.g. can you make it produce 1 unit of output, then 8 units of output, then 64 units of output - by unit I mean the number of values it outputs if that is what it does. If so, you can plot output file length against number of units of output and the intercept will tell you how many bytes the header is, if any. So, for example, if you can make it produce one number and you get an output file that is 24 bytes long, and when you make it produce 2 numbers the output file is 28 bytes long, you might deduce that there is a 20 byte header at the start and then 4 bytes per number.
Can you generate some known output? E.g. Can you make it produce zero, or 256 as an output, if so, you can search for zeroes and FF in your output file and see if you can locate them.
Make it generate a zero (or some other number) and save the output file, then make it generate a one (or some other, different number) and save the output file, then difference the files to deduce where that number is located. Or just make it produce any two different numbers and see how much of the output file changes.
Do you have another program that can understand the file? Or some way of knowing any numbers in the file? If so, get those numbers and convert them into hex and look for their positions in the hex dump.
I am trying to write a python program that takes a binary STL file and parses out all the vertices, but I can't figure out the binary files with a hex editor. I have read a few tutorial on hex and I understand that each 2 hex digits represents a byte and you add these together in even numbers to represent numbers or characters, but when I look at what they make it still never makes sense. I have found the format of a binary stl file here (http://en.wikipedia.org/wiki/STL_(file_format)). But the hex editor just spits out random characters like #, a whole lot of . (what do these do?), #, ", *, ect. How can I find which hex characters represent which vertices, which are the spacing characters, which are the normal characters, ect.
search out the documentation about the struct package in python
This is how you want to read binary data