I have the following 3x2 Int16 matrix as "test_matrix":
10 4
10 8
4 10
And I am expecting a binary output of 12 bytes
0x0A 0x04 0x0A 0x08 0x04 0x0A
I tried the following option:
write("test.bin", htol(test_matrix))
And the output becomes
What I have found are:
The matrix gets serialized (which is what I want)
The matrix gets transposed (which I don't want...)
The integers become 64 bits instead of 16 bits
The first 15 bytes are useless bytes for me
Any idea how should I export serialized matrix into binary correctly..?
To answer your questions:
Ad 2) the matrix is not transposed - Julia uses column major order like e.g. Fortran. You can use transpose to transpose the matrix if you want row major order.
Ad 3) htol works only because you are on little endian machine; on big endian it would error - use htol.(test_matrix) instead to broadcast it. Also most probably you actually have 64-bit integers stored in your matrix.
With these comments it works as you expected on my machine:
julia> test_matrix = Int16[10 4; 10 8; 4 10]
3×2 Array{Int16,2}:
10 4
10 8
4 10
julia> write("test.bin", htol.(transpose(test_matrix)))
12
julia> stat("test.bin")
StatStruct(mode=0o100666, size=12)
julia> read("test.bin")
12-element Array{UInt8,1}:
0x0a
0x00
0x04
0x00
0x0a
0x00
0x08
0x00
0x04
0x00
0x0a
0x00
(if you get a different result when running your code can you please specify what Julia version, what OS and what machine you are working on?)
Related
How can I know what these bunch of hex code means?
02 00 A0 E3 1E FF 2F E1
Any convertor of these codes to decimal code like 1,2,3 etc or vice versa like deciaml code to this type of hex code?
Thanks
This is my first response in stack overflow. So here goes...
What Hex Code (a.k.a. hexadecimal) represents purely depends on its context, or what does it mean to the program or machine. It could be a string, machine code (assembly language), flags, pointers to memory, data, part of an image or whatever. And this is dependent on the processor where this code is located also.
Each 2-digit hex code is a byte and represents decimal number (0-255 or 00-FF), half of a byte or 1 digit hex code is called a nibble.
Converting Hex Code to decimal is trivial. Convert from decimal to hex, not as trivial.
There are many calculators that have this functionality built in.
0-9 => 0 – 9, A=10, B=11, C=12, D=13, E=14, F=15.
Now, if you want to convert a 2 digit number like 12 hex (i.e. 0x12 or 12h ). Here is the formula.
(16 x 1) + (1 x 2) = 18 (decimal)
A four-digit hexadecimal 4A3E =>
(4096 x 4) + (256 x 10) + (16 x 3) + (1 x 14) = 19006 (decimal)
An integer in C# is 4 bytes, so your example hex code could also represent 2 integers in C#. Or it could be simply 1 number in C# called a “long” which is 8 bytes and could represent a number between:
0 to 18,446,744,073,709,551,615 unsigned long OR
-9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 signed long
Also something to note hex code also represent characters called ASCII (pronounced a·skee) This is an internal mapping within the microprocessor and could be different. 00 is not mapped so it typically means the end of the string.
Hex codes like that could represent a binary number. You could paste "0200A0E31EFF2FE1" into a converter like this to find out that the decimal representation of that number is "144292085413916641", for example.
But, from the way that your hex codes are grouped, it appears that you're looking at binary data, rather than a single integer that's represented in hexadecimal. When hex codes are grouped in pairs, each group of two characters represents one byte. https://en.wikipedia.org/wiki/Hexadecimal#Written_representation
I'm a Java Programmer.
I happend to begin learning about Encryption. SHA
and I found, for example SHA-256 print 256 bit length output.
I tried for example "Hello World", and I got result "A591A6D40BF420404A011733CFB7B190D62C65BF0BCDA32B57B277D9AD9F146E"
I expected 32 length word. because 1 byte = 8 bit. so 256/8 = 32.
But Why is that's length is 64?
Please teach me. and Any Answers or commnent will be apprieciated. Thank you.
Each byte is two Hexadecimal digits.
One byte, two nibbles: 1111 1110 results in two Hexadecimal digits: FE
The length of the SHA 256 is 64 in Hexadecimal because it contains 32 bytes represented by 64 Hexadecimal digits.
Im reading a book about network protocol structures.
There is an illustration in a chapter about variable-length quantity, which I dont fully understand.(see attachment)
The subject is to convert different numbers to variable-length 7-bit integers.
The first line shows that 0x3F is stored in a single octet as 0x3F.
The second line shows that 0x80 is stored in two octets one as 0x80 and second as 0x01.
However I dont understand why its not 0x81 in the first octet and 0x00 in the second.
Because according to wikipedia, converting numbers into variable-length 7bit integers goes as follows:
Represent the value in binary notation (e.g. 137 as 10001001)
Break it up in groups of 7 bits starting from the lowest significant bit (e.g. 137 as 0000001 0001001). This is equivalent to representing the number in base 128.
Take the lowest 7 bits and that gives you the least significant byte (0000 1001). This byte comes last.
For all the other groups of 7 bits (in the example, this is 000 0001), set the MSB to 1 (which gives 1000 0001 in our example). Thus 137 becomes 1000 0001 0000 1001 where the bits in boldface are something we added. These added bits denote if there is another byte to follow or not. Thus, by definition, the very last byte of a variable length integer will have 0 as its MSB.
So lets do these steps for 0x80:
binary notation: 1000 0000
in groups of 7 bits starting from LSB: 0000001 0000000
and 4. set MSB as described: 1000 0001 0000 0000
Converting that binary number into two hex octets, gives me 0x81 and 0x00.
Which leads me to the question: Is there a printing fail in the book or did I missunderstood something?
Which book is that?
There may be many possible encoding schemes. One of them could go like this:
1. Represent the value in binary notation (e.g. 0x80 as 10000000)
2. Break it up in groups of 7 bits starting from the lowest significant bit: 0000001 0000000
3. Start with the lowest 7 bits: if this is *not* the last group of 7 bits, then set MSB: 10000000; if it's the last, then leave it
alone: 00000001
4. Output starting LSB first: 10000000 00000001, i.e. 0x80 0x01
So what does the book say? What encoding scheme are they using?
I'm programming my Arduino micro controller and I found some code for accepting accelerometer sensor data for later use. I can understand all but the following code. I'd like to have some intuition as to what is happening but after all my searching and reading I can't wrap my head around what is going on and truly understand.
I have taken a class in C++ and we did very little with bitwise operations or bit shifting or whatever you'd like to call it. Let me try to explain what I think I understand and you can correct me where it is needed.
So:
I think we are storing a value in x, pretty sure in fact.
It appears that the data in array "buff", slot number 1, is being set to the datatype of integer.
The value in slot 1 is being bit shifted 8 places to the left.(does this point to buff slot 0?)
This new value is being compared to the data in buff slot 0 and if either bits are true then the bit in the data stored in x will also be true so, 0 and 1 = 1, 0 and 0 = 0 and 1 and 0 = 1 in the end stored value.
The code does this for all three axis: x, y, z but I'm not sure why...I need help. I want full understanding before I progress.
//each axis reading comes in 10 bit resolution, ie 2 bytes.
// Least Significant Byte first!!
//thus we are converting both bytes in to one int
x = (((int)buff[1]) << 8) | buff[0];
y = (((int)buff[3]) << 8) | buff[2];
z = (((int)buff[5]) << 8) | buff[4];
This code is being used to convert the raw accelerometer data (in an array of 6 bytes) into three 10-bit integer values. As the comment says, the data is LSB first. That is:
buff[0] // least significant 8 bits of x data
buff[1] // most significant 2 bits of x data
buff[2] // least significant 8 bits of y data
buff[3] // most significant 2 bits of y data
buff[4] // least significant 8 bits of z data
buff[5] // most significant 2 bits of z data
It's using bitwise operators two put the two parts together into a single variable. The (int) typecasts are unnecessary and (IMHO) confusing. This simplified expression:
x = (buff[1] << 8) | buff[0];
Takes the data in buff[1], and shifts it left 8 bits, and then puts the 8 bits from buff[0] in the space so created. Let's label the 10 bits a through j for example's sake:
buff[0] = cdefghij
buff[1] = 000000ab
Then:
buff[1] << 8 = ab00000000
And:
buff[1] << 8 | buff[0] = abcdefghij
The value in slot 1 is being bit shifted 8 places to the left.(does this point to buff slot 0?)
Nah. Bitwise operators ain't pointer arithmetic, don't confuse the two. Shifting by N places to the left is (roughly) equivalent with multiplying by 2 to the Nth power (except some corner cases in C, but let's not talk about those yet).
This new value is being compared to the data in buff slot 0 and if either bits are true then the bit in the data stored in x will also be true
No. | is not the logical OR operator (that would be ||) but the bitwise OR one. All the code does is combining the two bytes in buff[0] and buff[1] into a single 2-byte integer, where buff[1] denotes the MSB of the number.
The device result is in 6 bytes and the bytes need to be rearranged into 3 integers (having values that can only take up 10 bits at most).
So the first two bytes look like this:
00: xxxx xxxx <- binary value
01: ???? ??xx
The ??? part isn't part of the result because the xxx part comprise the 10 bits. I guess the hardware is built in such a way that the ??? part is all zero bits.
To get this into a single integer variable, we need all 8 of the low bits plus the upper-order 2 bits, shifted left by 8 position so they don't interfere with the low order 8 bits. The logical OR (| - vertical bar) will join those two parts into a single integer that looks like this:
x: ???? ??xx xxxx xxxx <- binary value of a single 16 bit integer
Actually it doesn't matter how big the 'int' is (in bits) as the remaining bits (beyond that 16) will be zero in this case.
to expand and clarify the reply by Carl Norum.
The (int) typecast is required because the source is a byte. The bitshift is performed on the source datatype before the result is saved into X. Therefore it must be cast to at least 16 bits (an int) in order to bitshift 8 bits and retain all the data before the OR operation is executed and the result saved.
What the code is not telling you is if this should be an unsigned int or if there is a sign in the bit data. I'd expect -ve data is possible with an Accelerometer.
It's been a while since my assembly class in college (20 years to be exact).
When someone gives you a number, say 19444, and says that X is bits 15 through 8 and Y are bits 7 through 0... how do I calculate values of X and Y?
I promise this is not homework, just a software guy unwisely trying to do some firmware programming.
First of all convert the input number to hexadecimal:
19444 => 0x4BF4
Hex is convenient because every 4 binary bits are one hex digit. Hence, every 2 hex digits are 8 bits, or a byte. Now assuming traditional little-endian notation (look it up!), bits 7 downto 0 are the low byte, bits 15 downto 8 are the high byte:
[7:0] => 0xF4
[15:8] => 0x4B
Using your preferred language, you can get the least significant byte by using a bitwise AND:
Y = 19444 & 0xff
or, the more mathematical:
Y = 19444 % 256
Now, for the most significant byte you can use bit shifts (if the number is larget than two byte, apply the first stage again):
X = 19444 >> 8
(The following assumes C notation). In general, to access the value in bits N through M, where N is the smaller value and the bits are numbered from 0, use:
(value >> N) & (1U << (M - N + 1)) - 1;
So for bits 0..7, use:
(value >> 0) & (1U << 8) - 1
and for bits 8..15, use:
(value >> 8) & (1U << 8) - 1
Note that for the case where "N through M" is the entire width of the type, you can't use the shift as written.
Also, mind the byte order (wheter the most significant byte comes first).
When given bit positions (like "15 through 8"), by convention bit 0 is the least significant bit of the binary number. If you're dealing with a 16-bit number, then bit 15 is the most significant bit.
One hexadecimal digit corresponds to 4 binary digits. So hex FF is 11111111 in binary. Bitwise AND is often used to "mask out" a certain collection of bits.
Nearly all processors provide some form of bitwise shifting. For example, shifting 1010001 right by 4 bits gives you 101.
Combining all this, in C you would typically do something like this:
unsigned short int num;
unsigned char x, y;
num = 19444;
y = num & 0xff; //use bitwise AND to get 8 least-sig bits
x = num >> 8; //right-shift by 8 bits to get 8 most-sig bits