I'm implementing UART ROM write to the Fujitsu's mcu and im stucked with CheckSum calculation, the writing requires 2 CRC bytes at the end of the 512 byte package. But how to calculate them? i cant get it.
I'm using the original Fujitsu Programmer to check the crc it calculates on each of my data transactions, for exmaple:
if i'm sending 512 byte ( all zeros and first byte 0x11)
1100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Then the programmer calcualte crc as:
I was checking Fujitsu's datasheet, and found this:
But i cant understand anything, im really bad at Further Mathematics, can someone direct me on the rigt path? What Checksum type is it?
/////////////////////////////////////
EDIT
The formula is not related to the checksum i need :(
But some guy shared some info about BOOTROM of fujitsu and accrording to 0x30 and 0x31 which is the same answer from my mcu aswell for page write, he has same crc algorithm.
he said its crc16 with polynome 0x1021
and sahred screen of his function:
and seems like its init 0x0000
i have checked all known crc16 with polynome 0x1021 and init 0x0000, cant get the answer i need (for 0x11 as example)
/////////////////////////////////////
EDIT2
There are some more examples:
1)
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
gives me 0x8108 CRC
00 02 ( rest 520 bytes are ZEROS, dont want to spamm here) gives me 0x0200
if i put the same values but in the end of the package:
511 bytes zeros and 512 byte is 0x01 it gives me (0x8108 crc):
same as above but with 02 ( 511 bytes zeros and 512 is 0x02) gives me 0x8318 CRC:
i can check any data which can possibly help to figure it out
Your checksum reference produces a one byte result. Not two bytes. However those two bytes are generated, it is not done with what you found. Where or how did you get the 0x1e and 0xf0? Are you sure those bytes represent a checksum?
Update:
Given the additional examples, all I can say is that it is not a CRC. There is some evidence that it is a checksum, since the same value was obtained from a 01 at the start and a 01 at the end. However there is also evidence against it, since the same is not true when moving a 02 in the data.
Related
RSA Encryption/Decryption Error
I am having a problem with an RSA encryption algorithm. I have an app (APP1) on an embedded device. The device has
a proprietary (old) embedded OS. I have 1 RSA crypto function ("rsa_calc") that I am using. In APP1
it has the public key of an RSA Key Pair. My issue is periodically, some encryptions can't be decrypted
in the receiving application APP2 (modern application/os). I am using BP-TOOLS Crypto Calculator to verify
results. I have included the trace of 2 encryptions below. The first works, i.e. it can be decrypted using
BP-TOOLS Crypto Calculator. The Second cannot. The vast majority of encryptions verify, but every now and again
some do not.
The issue appears to be based on the input data to be encrypted. Taking my traces, I found a working value, and a
non-working value. And performed the encryptions. I get the same results. So basically encrypting some data works,
other not so much. Please see traces below.
This leads me to believe I am having some padding error. The RSA encrypt function does not document padding schemes.
I am not implementing any padding manually other than using a 128 byte buffer. On BP-TOOLS Crypto Calculator,
I am using "No Padding" on the decryption, it only seems to work when I select that.
If I select "PKCS1", I get padding errors when decrypting. There is no error decrypting the data on BP-TOOLS, I just
don't get clear text/original text.
I am not concerned about the strength of the encryption and not using padding etc. I am just trying to get this to
work consistently. Do I have to implement one of the padding schemes for RSA manually? Does padding make certain
RSA buffers impossible to decrypt (I wouldn't think so, I know it weakens the cipher, don't care)?
Can anyone see some other issue that may be causing my problem?
Thanks for the help
Function Documentation
rsa_calc()
Performs a public key RSA computation. It supports keys up to 2048 bits and exponent values of 2, 3, and 65537.
Prototype:
int rsa_calc(unsigned short * msg, unsigned short * mod, int wds, int exp, unsigned short * result);
Parameters:
msg - Array of unsigned 16-bit integers holding the input.
mod - Array of unsigned 16-bit integers holding the modulus. High bit must be set.
Typically an RSA modulus always has the most-significant bit set. This
implementation verifies and enforces that, because if the bit is not set, it
might cause unexpected behaviors. The high bit of the modulus is the MSB
of the first 16-bit integer of the array.
wds - Number of 16-bit unsigned shorts in msg and mod.
exp - Code for exponent: actual exponent is 2^exp+1. Acceptable values are 0, 1,
16 that correspond to exponents of 2,3, and 65537, respectively.
Return Values:
result Array of 16-bit integers holding the result on exit.
RSA Key Pair
MODULUS:
BEDB6B21E12D2B6EB590EC129FCC847EA4C00BE41CA530A5FA2CCDDE3B7DB3A0F50E6D3E348CD9258D7076973DD01FDC5B7E00F1F714F4E55C650DF88AAA293ED9376D2B0905F589108FB5C2835EA025D036F369891E5F5F3BA4F5E96CF25D164C1B26215B8D9627CDB95C22F00EBF50DF821A984E01309C1677B5D013E2BEEF
PUBLIC EXP(E):
010001
PRIVATE EXP(D):
A6898BC7FA5691C97EC1405D57F6FBBE0E404D9FF4A6E7F64C807FFAE4EA60AD9867C847394F95C340D1DB894934AC3879D54F39D3A203B78791DE48FBA65369B077BD6541AA8200392CA0BF56EEE2AA8478598852BFB537498095C087910176E1E90F92F8564F3012D7DC52B8D7145C40143F51229FE4416CEF50657511E2F1
TRACES
!!!! This Works !!!!
-------------------------
|*** RSAEncryptKey: rsa_calc :: ENTER
|MSG (256):
|0000 01 2A 9E 34 EA 19 63 07 F9 AD A8 9B 02 5B D6 6D - .*.4..c......[.m
|0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|rsa_calc: 0 ::
|rsa_calc result (256):
|0000 2D 0A F1 93 29 6A 47 D7 94 A6 16 32 E8 AD 2D 3B - -...)jG....2..-;
|0010 72 64 7B 23 55 79 1B 71 A1 34 AD E4 E5 E7 E6 FA - rd{#Uy.q.4......
|0020 6D 78 AA 6E 7F 32 D0 BF 26 87 3A 5F 8F B9 23 42 - mx.n2..&.:_..#B
|0030 C7 02 D6 8B 70 24 76 90 59 40 A3 42 D5 B5 2B 00 - ....p$v.Y#.B..+.
|0040 CB 08 A4 C3 07 E7 12 4D 02 C0 2D 3A 43 15 27 41 - .......M..-:C.'A
|0050 17 42 F7 B0 F8 B7 06 B1 25 D9 90 12 88 C1 05 16 - .B..............
|0060 75 64 8E 87 AA 68 82 96 1E 3E 23 F7 32 10 E5 8A - ud...h...>#.2...
|0070 17 D4 39 18 21 FC 5A C3 38 BB 7F CB DA B0 D2 85 - ..9.!.Z.8......
|0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|*** RSAEncryptKey: :: EXIT
!!!! DOES NOT DECRYPT !!!!
-------------------------------------
|*** RSAEncryptKey: rsa_calc :: ENTER
|MSG (256):
|0000 FA 68 E9 19 F5 D2 64 E0 F6 A8 FB 88 DC 67 35 C0 - .h....d......g5.
|0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|rsa_calc: 0 ::
|rsa_calc result (256):
|0000 36 B8 4D F2 76 DA 21 31 A3 8E C8 26 75 84 52 48 - 6.M.v.!1...&u.RH
|0010 3B CE 87 15 58 D9 F0 21 04 CE A0 83 9E 5E 58 31 - ;...X..!.....^X1
|0020 19 CF 82 70 00 BD A2 6D 2C E8 34 F4 27 D1 B2 4C - ...p...m,.4.'..L
|0030 1A F4 19 1D DA C4 B6 CD 10 C2 29 32 B6 85 E7 CB - ..........)2....
|0040 BD 4D 53 70 B3 27 47 B8 87 48 0C FD 86 B8 C0 4C - .MSp.'G..H.....L
|0050 B4 B8 23 4C 11 8C A7 93 41 2E AF E4 FF F1 EC 9A - ..#L....A.......
|0060 44 13 FF DB 9B ED B6 9E E2 2A 11 41 64 C5 2C E0 - D........*.Ad.,.
|0070 FE 4D 05 30 EE 49 9A 7A C5 46 BD 17 2F 87 72 D1 - .M.0.I.z.F../.r.
|0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|00F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
|*** RSAEncryptKey: :: EXIT
Thank you President James K. Polk. Your comments resolved the issue. I wouldn't mind understanding better, so please comment if you can. It appears you are indicating that if I treat the first 16 bytes as a 16 byte integer, then the encryption block cannot be a larger integer than the Modulus, I don't really understand why. I don't really see the relationship between what I want to encrypt versus the key used to encrypt it.
However, shifting the 16 bytes in the encryption block (I shifted it 16 bytes). Every encryption/decryption appears to work (so far anyway).
Thank you for your help.
I tried to analysis the format of FAT32 to use it later in a project containing SD card with FAT32 and a MCU but when I read the data of the SD Card with FAT32 the partition boot sector wasn't at 0 sector but on 0x2000 sector and 0th sector contained that :
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 C0 78 DA EB 00 00 00 82 03 00 0C FE FF FF 00 20 00 00 00 40 9C 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA
is there any explanation for that ?? I'd be very thankful
I already figured it out , the 0x200 offset is due two the hidden sectors preceded the FAT volume partition . the SD regions and Sectors were normally in order starting from 0x2000 sector. still search to find out why windows made that hidden sectors when format the card
I have a data frame of integers, which I would like to convert to bits, and then split each resulting bit sequence into separate columns. I figured I could convert my bits to string and then split the strings on whitespace. However, my method does not work.
I create my data frame:
r1 <- c(24,25,27)
r2 <- c(98,102,4)
model.data <- as.data.frame(rbind(r1,r2))
and then I convert int to bits with:
model.data[] <- lapply(model.data, function(x) {
if(!is.na(x)) as.character(paste(intToBits(x), collapse = " ")) else x
})
However, this is where I encounter my first problem. The second row of my data frame now seems to have the same values as the first one:
model.data$V1
[1] "00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
[2] "00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
This shouldn't be the case, since the values in the original data frame differ. It's the same case for all the columns here (V2, V3).
My second problem is that I can't split the columns. I try this method:
for(i in names(model.data)) {
a.split <- strsplit(model.data[[i]], split = " ")
cbind(model.data,a.split)
}
but it does nothing.
Could you help me figure out what I'm doing wrong? Thanks!
Use the function "separate" from tidyr package
library(tidyr)
# Your data
r1 <- c(24,25,27)
r2 <- c(98,102,4)
# Concatenate r1 and r2
r3 <- c(r1,r2)
# Create a dataframe
model.data <- as.data.frame(r3)
# Check type of r3
class(model.data$r3)
# Transform with "intToBits" function
model.data[] <- lapply(model.data, function(x) {
if(!is.na(x)) as.character(paste(intToBits(x), collapse = " ")) else x
})
nb = length(unlist(strsplit(model.data$r3[1]," ")))
# Transform into character
model.data$r3 <- as.character(model.data$r3)
# Use "separate" function from tidyr, generating nb variable
model.data %>% separate(r3, into = paste0("V",seq(1, nb, by = 1)), sep = " ")
You need to apply over both rows and columns, using ifelse. Interestingly, using lapply does not seem to work. Here an example for the apply solution:
r1 <- c(24,25,27)
r2 <- c(98,102,4)
model.data <- as.data.frame(rbind(r1,r2))
model.data[] <- as.data.frame(apply(model.data, c(1,2), function(x) ifelse(!is.na(x), as.character(paste(intToBits(x), collapse = " ")), x)), stringsAsFactors = FALSE)
> model.data
V1
r1 00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
r2 00 01 00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
V2
r1 01 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
r2 00 01 01 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
V3
r1 01 01 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
r2 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Edit:
To create the new data with one bit per variable, first lapply the strsplit, giving you a nested list as output, then use a nested do.call to first cbind the sublists, then rbind the result:
newCols <- lapply(model.data, function(x) strsplit(x, split = " "))
a.split <- as.data.frame(do.call(rbind, do.call(cbind, newCols)), stringsAsFactors = FALSE)
> a.split
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 V19 V20 V21 V22 V23 V24 V25 V26 V27 V28 V29 V30 V31 V32
1 00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2 00 01 00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3 01 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4 00 01 01 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5 01 01 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Please share your ideas on how we can resolve this exception
We are working on Netconf Over SSH and the SSH library used is Apache Mina and SSHD. Device is the Netconf server and Application is the Netconf Client. As device is behind firewall, application cannot connect to device but the device can connect to the application. so, we are using the session established by the device to the application for all the communication.
Below are the steps followed:
Started NioSocketAcceptor listening/bind to a port number
IoSessionConfig's useReadoperation attribute is set to true
When a tcp connection is accepted, we have a mina core IoSession available, here we call it as a tcpSession.
We wanted to use this mina IoSession for reading and writing netconf messages. Code used to convert mina core session to ssdh IoSession is as follows
new org.apache.sshd.common.io.IoSession() {
#Override
public long getId() {
return tcpSession.getId();
}
#Override
public Object getAttribute(final Object key) {
return tcpSession.getAttribute(key);
}
#Override
public Object setAttribute(final Object key, final Object value) {
return tcpSession.getAttribute(key, value);
}
#Override
public SocketAddress getRemoteAddress() {
return tcpSession.getRemoteAddress();
}
#Override
public SocketAddress getLocalAddress() {
return tcpSession.getLocalAddress();
}
#Override
public IoWriteFuture write(final Buffer buffer) {
final ChannelAsyncOutputStream.IoWriteFutureImpl ioWriteFuture = new ChannelAsyncOutputStream.IoWriteFutureImpl(buffer);
byte[] bytes = buffer.getCompactData();
int capacity = bytes.length + 8;
IoBuffer bytebuffer = IoBuffer.allocate(capacity).setAutoExpand(true);
//bytebuffer.putInt(bytes.length);
bytebuffer.put(bytes);
bytebuffer.flip();
buffer.clear();
final WriteFuture write = tcpSession.write(bytebuffer);
write.addListener(new IoFutureListener<WriteFuture>() {
#Override
public void operationComplete(final WriteFuture future) {
if (future.isWritten()) {
ioWriteFuture.setValue(true);
} else {
// TODO check the value type expected + if exception can go there
ioWriteFuture.setValue(future.getException());
}
}
});
return ioWriteFuture;
}
#Override
public CloseFuture close(final boolean immediately) {
final DefaultCloseFuture defaultCloseFuture = new DefaultCloseFuture(null);
tcpSession.close(immediately).addListener(new IoFutureListener<org.apache.mina.core.future.CloseFuture>() {
#Override
public void operationComplete(final org.apache.mina.core.future.CloseFuture future) {
if(future.isClosed()) {
defaultCloseFuture.setValue(true);
} else {
// TODO check the value type expected
defaultCloseFuture.setValue(false);
}
}
});
return defaultCloseFuture;
}
#Override
public IoService getService() {
throw new UnsupportedOperationException("No service available");
}
#Override
public boolean isClosed() {
return tcpSession.getCloseFuture().isClosed();
}
#Override
public boolean isClosing() {
return tcpSession.isClosing();
}
});
Code used to attach Apache SSHD IoSession with the client is as below
final ConnectFuture connectFuture = new DefaultConnectFuture(null);
ClientSessionImpl session = null;
try {
session = new ClientSessionImpl(sshClient, ioSession);
AbstractSession.attachSession(ioSession, session);
MyAsyncSshHandlerReader async = new MyAsyncSshHandlerReader(ioSession);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
MyAsyncSshHandlerReader is the class used to read the data from tcpSession.
class MyAsyncSshHandlerReader implements IoFutureListener, AutoCloseable {
private final org.apache.sshd.common.io.IoSession ioSession;
public MyAsyncSshHandlerReader(org.apache.sshd.common.io.IoSession ioSession) {
this.ioSession = ioSession;
minaCoreIoSession.read().addListener(this);
}
#Override
public void close() throws Exception {
// TODO Auto-generated method stub
}
#Override
public void operationComplete(ReadFuture future) {
long msgsRead = future.getSession().getReadMessages();
if(future.isRead() && !future.isClosed()){
IoBuffer msg = (IoBuffer)future.getMessage();
msg.flip();
Buffer buf = new Buffer(msg.array());
try {
if(AbstractSession.getSession(ioSession) != null){
AbstractSession.getSession(ioSession).messageReceived(buf);
}
} catch (Exception e) {
e.printStackTrace();
}
minaCoreIoSession.read().addListener(this);
}
}
}
When the client session is created at the application side, below methods are invoked.
sendIdentification
sendKexInit()
Application has also received the identification message and key's exchanged from the device
and identification message is read properly but could not decode the key exchange message, library has throwed SshException "invalid packet length 0"
2015-11-02 13:02:48,831 | INFO | oupCloseable-6-2 | ClientSessionImpl | 210 - org.apache.sshd.core - 0.14.0 | Error decoding packet (invalid length) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 05 fc 07 14 34 d4 4e 0e 2c 22 73 e4 7e 46 41 1b e3 55 27 ef 00 00 00 b7 65 63 64 68 2d 73 68 61 32 2d 6e 69 73 74 70 32 35 36 2c 65 63 64 68 2d 73 68 61 32 2d 6e 69 73 74 70 33 38 34 2c 65 63 64 68 2d 73 68 61 32 2d 6e 69 73 74 70 35 32 31 2c 64 69 66 66 69 65 2d 68 65 6c 6c 6d 61 6e 2d 67 72 6f 75 70 2d 65 78 63 68 61 6e 67 65 2d 73 68 61 32 35 36 2c 64 69 66 66 69 65 2d 68 65 6c 6c 6d 61 6e 2d 67 72 6f 75 70 2d 65 78 63 68 61 6e 67 65 2d 73 68 61 31 2c 64 69 66 66 69 65 2d 68 65 6c 6c 6d 61 6e 2d 67 72 6f 75 70 31 34 2d 73 68 61 31 2c 64 69 66org.apache.sshd.common.SshException: Invalid packet length: 0
at org.apache.sshd.common.session.AbstractSession.decode(AbstractSession.java:719)
at org.apache.sshd.common.session.AbstractSession.messageReceived(AbstractSession.java:308)
at org.opendaylight.controller.netconf.nettyutil.handler.ssh.client.ReversedAsyncSshHandler$MyAsyncSshHandlerReader.operationComplete(ReversedAsyncSshHandler.java:220)
please throw some light on how AbstractSession.decode() method works
From my debugging, rpos and limit variables are holding same value, say, 1536 for example. Is the way I am reading has some issue?
After going through many examples and Apache mina library, I found the issue lies with the way we are reading from mina core IoSession.
Correct way of reading ::
int r=ioBuffer.remaining();
byte[] b=new byte[r];
ioBuffer.get(b,0,r);
Buffer buf = new Buffer(b);
AbstractSession.getSession(tcpSession).messageReceived(buf)
Edit: It turns out that this is a custom checksum algorithm, not a CRC-32. For the curious here is a snippet of C code calculating the 21 FF 1D E4 checksum in the below example.
I am working with some hex files that seem to be protected by a CRC-32 of sorts, but recalculating using the standard CRC-32 and other known parameters failed to produce a match.
All I know is, the data spans 116 bytes and four additional bytes for the checksum. I can produce tons of examples of message-key pairs, I just cannot find any relation between them.
I don't want to fill this post with hex dumps so I pasted a couple more here: http://mathb.in/12246.
11 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 21 FF 1D E4
Could this be a CRC-32 with some strange settings, or is it completely different? What would be a way to determine how the checksum is produced?
Update: I was able to make minute changes of 1 bit in the messages:
00000000000000000000000000000000000000000000000000000000, 32C9A1E6
00000000000000000000000000000000000000000000000000000001, BD25904E
00000000000000000000000000000000000000000000000000000002, B437A286
00000000000000000000000000000000000000000000000000000003, 8AB790EE
00000000000000000000000000000000000000000000000000000004, 2DDC3AEB
00000000000000000000000000000000000000000000000000000005, 208B3859
00000000000000000000000000000000000000000000000000000006, 87E0925C
00000000000000000000000000000000000000000000000000000007, E59391AE
00000000000000000000000000000000000000000000000000000008, B07292FC
00000000000000000000000000000000000000000000000000000009, 830EA655
Changing anything without updating the checksum appropriately creates a message that is rejected.
If this cannot be a CRC-32 due to the superposition principle (exclusive-or check), would there be any way to perform an analysis to find any properties or patterns in pairs of message and checksums?
Assuming that you got the messages and check values correct in your linked examples, then it is not a CRC. For a CRC or any linear function over GF(2) of the message, it should be that if message 1 exclusive-or'ed with message 2 is equal to message 3 exclusive-or'ed with message 4, then the same should be true of the check values.
There is only one byte that differs in your messages. Since 45 ^ 4A == 43 ^ 4C, then the associated check values when exclusive-or'ed should also be equal. They are not.
If you did have a CRC, then you could use reveng to try to tease out the CRC parameters.