I'm trying to decode this Firebird blob to extract decimal numbers from it (not sure exactly what format they'll be in).
Some context is the blob is storing vibration spectrum data charting amplitude against frequency. I'm pretty sure that the blob only contains the amplitude data though. Here's an example blob export for a small test spectra i generated:
0000803F0000004000004040000080400000A0400000C0400000E0400000004100001041000020410000304100004041000050410000604100007041000080410000884100009041000098410000A0410000A8410000B0410000B8410000C0410000C8410000D0410000D8410000E0410000E8410000F0410000F84100000042000004420000084200000C4200001042000014420000184200001C4200002042000000006666663FA4707D3F77BE7F3F72F97F3F58FF7F3F0000803F0000C03F0000004000002040000040400000604000008040000088400000904000009840CDCC9C400000A0400000C84200007A4400401C46
As far as i can tell, it looks like each number is represented by 4 bytes of data which is hexadecimal in this export. I know it's 4 bytes per value because of how it lines up with my test set below. I also think that potentially the first 2 bytes are the fractional part, and the last 2 being the whole numbers? I think it might use a scaling factor as well. Here is my test set (same as above, just reformatted), with the actual values (amplitudes):
Actual Value Blob Section
1 0000803F
2 00000040
3 00004040
4 00008040
5 0000A040
6 0000C040
7 0000E040
8 00000041
9 00001041
10 00002041
11 00003041
12 00004041
13 00005041
14 00006041
15 00007041
16 00008041
17 00008841
18 00009041
19 00009841
20 0000A041
21 0000A841
22 0000B041
23 0000B841
24 0000C041
25 0000C841
26 0000D041
27 0000D841
28 0000E041
29 0000E841
30 0000F041
31 0000F841
32 00000042
33 00000442
34 00000842
35 00000C42
36 00001042
37 00001442
38 00001842
39 00001C42
40 00002042
0 00000000
0.9 6666663F
0.99 A4707D3F
0.999 77BE7F3F
0.9999 72F97F3F
0.99999 58FF7F3F
1 0000803F
1.5 0000C03F
2 00000040
2.5 00002040
3 00004040
3.5 00006040
4 00008040
4.25 00008840
4.5 00009040
4.75 00009840
4.9 CDCC9C40
5 0000A040
100 0000C842
1000 00007A44
10000 00401C46
Its pretty obvious that its not just a straight hexadecimal - decimal conversion, but i feel like this is something an expert would be able to recognize. Any help or pointers on how to decode this 4 bytes of hex back to a number value would be much appreciated!
That is industry-standard 4-bytes floating point format (single float).
https://www.h-schmidt.net/FloatConverter/IEEE754.html
https://en.wikipedia.org/wiki/Single-precision_floating-point_format
Of course, bytes order should be accounted for too (you see it visually reversed in your dump above, comparing to normal writing of hexadecimal integer numbers on the site above).
https://en.wikipedia.org/wiki/Endianness
Related
Need some help to determine how is this date/time being encoded
Here are some examples (known date only):
2B5F0200 -> 31/10/2021
2B9F0200 -> 31/12/2021
2C3F0200 -> 31/01/2022
I don't understand how this datetime format works.
First, since the last two bytes are the same in all cases, let's focus on the first two bytes. Look at them in binary:
0x2B5F: 0b_0010_1011_0101_1111
0x2B9F: 0b_0010_1011_1001_1111
0x2C3F: 0b_0010_1100_0011_1111
Next, consider the binary representation for the numbers in the dates. In some date formats, months are 0-based (January is 0), in others they're 1-based, so include both.
21: 0b_1_0101
22: 0b_1_0110
10: 0b_1010, 0b_1001
12: 0b_1010, 0b_1001
01: 0b_0001, 0b_0000
31: 0b_1_1111
By inspection, each of these binary numerals appears in the appropriate date. 31 is the last 5 bits. The next 4 bits are 10, 12 and 1. 21 and 22 show up in the first 7 bits (for 100 years, you'll need at least 7 bits).
21 10 31
0x2B5F: 0b_0010101_1010_11111
21 12 31
0x2B9F: 0b_0010101_1100_11111
22 1 31
0x2C3F: 0b_0010110_0001_11111
The format is thus a packed bit-field:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-------------+-------+---------+
| year (YY) | month | day |
+-------------+-------+---------+
Or, as bit masks:
year: 0xFE00
month: 0x01E0
day: 0x001F
I need to find the interpolated value for consumption from the speed and weather.
I have tried approx function but it is only for 2 variables, wont accept three or more.
Speed weather fuel
10 2 30
12 3 35
14 8 38
15 9 65
need to find fuel for speed_new = 13 and weather = 7.
approx(x=Speed,y=Fuel,z=Weather,xout= speed_new,rule = 2)$y #need to also mention the weather
I have list of data.frame, where I need to do transformation for .score column. However, I implemented helper function for this transformation. After I call .helperFunc for my input list of data.frame, but I got weird pvalue format in first, third data.frame. How to normalize rather big decimal to simple scientific number ? Can anyone tell me how to make this happen easily ?
toy data :
savedDF <- list(
bar = data.frame(.start=c(12,21,37), .stop=c(14,29,45), .score=c(5,69,14)),
cat = data.frame(.start=c(18,42,18,42,81), .stop=c(27,46,27,46,114), .score=c(15,5,15,5,134)),
foo = data.frame(.start=c(3,3,33,3,33,91), .stop=c(26,26,42,26,42,107), .score=c(22,22,6,22,6,7))
)
I got this weird output:
> .savedDF
$bar
.start .stop .score p.value
1 12 14 5 0.000010000000000000000817488438054070343241619411855936050415039062500
2 21 29 69 0.000000000000000000000000000000000000000000000000000000000000000000001
3 37 45 14 0.000000000000009999999999999999990459020882127560980734415352344512939
$cat
.start .stop .score p.value
1 18 27 15 1e-15
2 42 46 5 1e-05
3 18 27 15 1e-15
4 42 46 5 1e-05
5 81 114 134 1e-134
$foo
.start .stop .score p.value
1 3 26 22 0.0000000000000000000001
2 3 26 22 0.0000000000000000000001
3 33 42 6 0.0000010000000000000000
4 3 26 22 0.0000000000000000000001
5 33 42 6 0.0000010000000000000000
6 91 107 7 0.0000001000000000000000
I don't know what happen this, only second data.frame' format is desired. How can I normalize p.value column as simple as possible ?
last column of cat is considered to be desired format, or more precise but simple scientific number is also fit for me.
How can I make this normalization for unexpectedly long decimal numbers ? How can I achieve my desired output ? Any idea ? Thanks a lot
0 is the default scipen option. (See ?options for more details.) You apparently have changed the option to 100, which tells R to use decimal notation unless it is 100 characters longer than scientific notation. To get back to the default, run the line
options(scipen = 0)
As to "So in my function, I could add this option as well?" - you shouldn't do that. Doing it in your script is fine, but not in a function. Functions really shouldn't set user options. That's likely how you got in to this mess - some function you used probably rudely ran options(scipen = 100) and changed your options without you being aware.
Related: the opposite question How to disable scientific notation in R?
I'm running R on a Mac (OS X). I have a rather large data frame (imported from a csv-file) that I'm working with:
dim(mydf)
[1] 75848 9
I'm trying to analyse it and find ways of breaking it up into smaller parts, so I need to print at least parts of it out to get an overview from time to time.
However, when I have printed it, R (version 3.1.2) starts working extremely slowly to the point where I just have to give up and restart it. Then R works normally until I have printed something large to the console again.
I have tried ´gc()´ and ´rm(list = ls())´, but it doesn't improve the speed - and I guess it wouldn't as it seems to be the printing to the console and not the size of the data frame that causes the slowness (clogging up memory?).
Is there anything I can do to prevent R from becoming so slow, or do I just have to choose between restarting frequently or giving up printing my data to the console?
Thanks!
Same as you, I wanted to get an overview of my data. But just a little more then the ‘head’ function would give me. So I wrote a small function that would give me the head, middle, and tail of a dataset.
hmt <- function(x){ # head, middle, tail of data set
if(class(x) == "data.frame"){
middle <- round(nrow(x)*0.5)
middle <- x[(middle-3):(middle+3),]
data <- rbind(head(x),middle,tail(x))
}
return(data)
}
hmt(cars)
And the result:
speed dist
1 4 2
2 4 10
3 7 4
4 7 22
5 8 16
6 9 10
22 14 60
23 14 80
24 15 20
25 15 26
26 15 54
27 16 32
28 16 40
45 23 54
46 24 70
47 24 92
48 24 93
49 24 120
50 25 85
Hope this is of any help to you.
I encountered this problem while preparing for a test.
A hash table of length 10 uses open addressing with hash function h(k)=k mod 10, and linear probing. After inserting 6 values into an empty hash table,
the table is as shown below
0 |
1 |
2 | 42
3 | 23
4 | 34
5 | 52
6 | 46
7 | 33
8 |
9 |
How many different insertion sequences of the key values using the same hash
function and linear probing will result in the hash table shown above?
(A) 10 (B) 20 (C) 30 (D) 40
Answer given in Solutions: c
Could someone please explain how the answer is to be calculated? TIA
Since, 33 mod 10 = 3 and it is stored in 7th position, we know that it must have come after 23,34,52,46 and as 52 came before 33 hence, so did 42 (same hash value). We've established that 33 comes last in the sequences.
For the rest of the numbers there are 2 cases :
(1) Considering 52 came after 42,23,34 (as they are stored according to their hash value) which can be rearranged in 3! ways. That means 52 came 4th position and 46 came in 5th.
(2) Considering 52 came after 42,23,34,46 which can be rearranged in 4! ways. That means 52 came at 5th position.
So, total number of insertion sequences = 4! + 3! = 30
In a valid insertion sequence, the elements 42, 23 and 34 should appear before 52 and 33, and 46 must appear before 33.
Here number of different sequences = 3! x 5 = 30
In the above expression, 3! is for elements 42, 23 and 34 as they can appear in any order, and 5 is for element 46 as it can appear at 5 different places.