It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
Is there any tool to aid in analyzing an unknown hex dump (to find possible datatypes).
What i have is a string, which is a dump of some hex network data. The network data is NOT regular network data, so wirehsark wont be able to identify.
I need a tool that can help me parse the hex and fine the datatypes within.
The hex dump that i do have could look like this:
00 00 00 00 04 B4 FD 43 90 88 BA 44 06 A3 30 C4 00 00 00 00 CE 3B 54 BF 44 27 0F 3F 00 00 00 00 02 00
You can try to convert these hexdumps to regular binary files. There is a tool for this called xxd, see http://linuxcommand.org/man_pages/xxd1.html
You can also convert using custom script. With PHP it could be:
<?php
$in = fopen("input", "rt");
$out = fopen("output", "wb");
while ($line = fgets($in)) {
foreach (explode(" ", $line) as $number) {
fwrite($out, chr(hexdec($number)));
}
}
Then you use classic data recovery tools and scan these binary files for datatypes:
Scalpel
PhotoRec (for multiple platforms)
Foremost
If you're on windows, you may want to look at 010Editor. It supports a pretty rich format template language and can handle very large dumps. It works well for both network dumps and normal file reverse engineering.
Related
I am reading in an old legacy file format (nor ours).
It's clear that this file format was written using serialization, so to make my life easier I am doing the same for reading.
At some point in reading the file, a segment is identified as being a QMetaType::QVariantList.
The subsequent data reads:
01 00 00 00 03 00 7e 00 00 00
(all byte sequences in little endian)
Stepping through the deserialization into QVariantList confirms that 01 00 00 00 identifies the number of items in the list as 1.
Stepping further into the QVariant itself then shows my problem; QVariant reads the QMetaType unconditionally using a 32 bit integer
So 03 00 7e 00 becomes a QMetaType of 8257539, which is clearly wrong.
It would appear that back in the day, QVariant's meta type may have been serialized using 16 bit types, because then the data makes sense:
03 00 becomes 3 (QMetaType::Uint)
7e 00 00 00 becomes 126
Looking at the implementation of QVariant's serialization, there is nothing I can do via QDataStream::setVersion() either to force Qt to read the meta type as 16 bits.
Does anyone have any suggestions on what I can do now?
I am working on FPGA ETHERNET project. My problem is following :a UDP/IP packet sent from FPGA is captured by "wireshark" and it gives me a following warning : "BAD UDP LENGTH 26 > IP PAYLOAD LENGTH Len=18 (Malformed Packet)".
BAD_UDP_LENGTH(PICTURE)
Actually, I am trying to send the following packet :
55555555555555D598EECB9B6EF400123456789008004500002EB3FE0000801108BDA9FE1504A9FE1503FA00FA00001A45630 00102030405060708090A0B0C0D0E0F1011 06A07518
You can see that the last byte of data is 11 and that part of data is not displayed by "wireshark". That part of data is obviously transmitted as I have seen it on oscilloscope. I have tried the number of different Ethernet Packet generators(PacketETH,EthernetUDP,C#) and all of them have generated the same packets (I think no problem lies in packet or packet generators). Also I have captured packets by different network monitoring software "Omnipeek" which gave me the same result as gave "wireshark" : LAST BYTE IS NOT DISPLAYED
If the last byte displayed, I think "wireshark" wouldn't give me that error.
Does anybody know how to solve that problem ?
Here I will give you additional details :
1) I am using Digilent Anvyl FPGA which has LAN8720A-CP-TR tranceiver. I have written code in VHDL and also run simulation in ISIM which gave me a correct result, concretely: all bits are sent successively with relevant values. Besides, I have checked simulation result in reality by probing LAN8720A-CP-TR tranceiver transmit pins by Digilent Electronics Explorer
2) For Ethernet packet generation I am using a simple program which you can download from here EthernetUDP (fpga4fun.com) here is picture of packet generated by that program EthernetUDP picture
If I copy the frame data from the image, I get this:
0000 98 ee cb 9b 6e f4 00 12 34 56 78 90 08 00 45 00
0010 00 2e b3 fe 00 00 80 11 08 bd a9 fe 15 04 a9 fe
0020 15 03 fa 00 fa 00 00 1a 45 63 00 01 02 03 04 05
0030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11
0040
And if I save that in a file called packet.txt then run text2pcap packet.txt packet.pcap, then load the resulting capture file back into Wireshark, I get a completely valid packet including the trailing 0x11 byte and the "bytes on wire" is indicated as 60 bytes instead of 59 bytes.
So somehow in your setup, that last byte didn't get handed off to Wireshark; therefore it simply wasn't captured for some reason, which is the reason why it's not displayed. Why it wasn't captured is an open question. It may have been transmitted as you say, since you can see it on the oscilloscope, but something about it or your capture hardware isn't correct.
This is a snort rule:
alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"INFO web bug 0x0 gif attempt"; flow:from_server,established; content:"Content-type|3A| image/gif"; nocase; content:"GIF"; nocase; distance:0; content:"|01 00 01 00|"; distance:3; within:4; content:"|2C|"; distance:0; content:"|01 00 01 00|"; distance:4; within:4; classtype:misc-activity; sid:2925; rev:2;)
From the alert name, I can see it seems to preventing receive html code including 0x0 gif to avoid the bandwidth depletion. I just want to know what is something like |01 00 01 00| here. Please tell me to help me have a better understand of the whole rule.
|01 00 01 00| is width and height value of the GIF.
content:"|01 00 01 00|"; distance:3; within:4; matched 4bytes after GIF89a
Good luck~
I'm using a microcontroller (PIC18F26J50) to interface with a G.Skill 4GB microSD card.
SD Card initialization is successful and I go from receiving 0x01 (Idle) R1 tokens to 0x00 (Ready) R1 tokens.
Reading a data block works, I am able to read the location of partition 1 and read the first sector of that partition
However when attempting to write a block, I never see a response token. Upon dumping the raw blocks on the card. I see that the data did indeed write but it is not aligned properly...the best way to explain is with an actual picture
This should be filled with 0x01, 0x02, 0x03, and so on up to 0xFF before repeating
This card works absolutely fine in windows. And I'm able to read and write data to it properly.
Investigating, I find that the response I get is 0XCA, if you right-shift that you get 0xE5, a proper response token. The data itself is misaligned one to the left. Additionally, it appears that the two dummy bytes and the token were also written. Correcting for the shift you get:
FF FF FE 00 01 02 03 04 05 06 07 08 09 0A 0B 0C
So I removed the code to write the 2 dummy bytes and the 0xFE token, and holy s*#$ the card starts writing data IMMEDIATELY after the command, which I believe violates spec! Can anyone confirm if this is intended behavior for SDHC cards? Or is this card just running a really s*#$ty SD controller? (The latter I suspect because I have a 16GB card which is working fine)
I just had the exact same problem, which turned out not to be a problem with the SD card, but with my SPI interfacing.
The particular chip I was using (Freescale KL03) will retain the current received byte in the data buffer until you read it, even after you have started sending the next one. I was out of sync, so that each time I was writing an SPI byte, waiting for transmission and then reading from the buffer, I was actually getting the previous result and not the current one. As a consequence there was a single byte lag in every SPI transaction.
Thus, my scope revealed that I was exchanging the following with the card:
MOSI: 58 00 00 00 01 00 00 00 00 7E nn nn nn ...
MISO: FF FF FF FF FF FF FF 00 FF FF FF FF FF ...
which resulted in the misalignment you encountered. It should have been like this:
MOSI: 58 00 00 00 01 00 00 00 7E nn nn nn nn ...
MISO: FF FF FF FF FF FF FF 00 FF FF FF FF FF ...
In summary, ensure that you are sending the 7E immediately after the 00 response to your write command.
We are doing 2 slices per frame encoding using our codec and we are getting good H264 file output when played on VLC player.
But when we RTP packetize that encoded data and stream to VLC, it shows artifacts. If we use one slice per frame, our packetization is ok and stream on VLC also looks good.
We are using FU-A fragmentation, and my encoded file configuration:
resolution: 640x480
framerate: 30fps
bitrate: 800 Kbps
Our encoder is configured to use High Profile, CBR, IDR every 10 frames.
Our encoder output bitstream looks like:
00 00 00 01 67 [DATA] 00 00 00 01 68 [DATA] 00 00 00 01 65 [DATA] 00 00 00 01 65 [DATA] 00 00 00 01 41 [DATA]
So here we have two successive slice NALUs (0x65).
In our RTP pcap, everything looks good -- FU-A fragmentation, marker bit, etc but VLC and ffplay both show a similar type of artifact, as if the upper half of the frame is stretched (vertically).
My pcap file link:
http://www.filedropper.com/rtp
So I reduced the test case to a small, low bitrate (50 Kbps) QCIF stream with no fragmentation and I am still seeing the same problem.
My pcap file link:
http://www.filedropper.com/rtpqcif
Can any expert please look at the pcap file and see what might be causing VLC such trouble to play stream?
Thank you,
Harshal Patel
Although it's a very old question, when aggregating multiple NAL-U with the same timestamp (like in your example), the packetizer should use STAP-A mode and not FU-A. FU-A is made for a single timestamp NAL-U that would not fit in a RTP packet.
You need to solve the packetizer issue and everything will go well.