I have a lot of pcap files containing traces of different JPEG images being transmitted. Each of those images is divided into a couple hundred of packets of base64 encoded data. I have written a program to extract those images.
The problem is that here and there one or a few of these packets are missing.
I would like to recover as much of those images as possible. Currently the extracting software stops whenever a packet is missing and I end up with partial images. When opened in an image viewer, they look normal on the top, but then as soon as a packet is missing the rest of the image is grey due to the missing data.
So I would like to write a program that modifies a pcap file and inserts a dummy packet into the trace whenever a piece of an image is missing. This way, only the portion of the image that is really missing would be greyed out.
What would be an easy way to achieve this? I would appreciate any hints for useful tools/formats etc.
Thanks!
I can easily identify the locations in Wireshark where it would look as follows when following a TCP stream:
...
OJuOhptqVqcJAMzxzXGm7OlhdtuxWB4YtvGVYvd6V4gvUIQNkT7/AErZWaXLy4SpZ2IIA+1V
uHWJbtkPLB3XpXt7bVqMHsi3Cyn06TAI69P61nujdGNdEvCrMW[1368 bytes missing in capture file].eioyN6ANH4fMB3M7YUApIaUTP0r0Hgv8FlIUncjaenvXA/DEBzNSEn4fKUDP
zFegMOZKWkaSIjnt7VTMvgUviY8pvKrpkhBcTISeT0+1cLflTiidyeK7r4g2wfyret7nRpcC
...
But is there a way to automate it? Maybe some other file format that is easier to parse and modify?
You could render what you have into txt, edit it, and then use text2pcap to convert back to pcap.
Related
I'm trying to make a simple program in GNU Radio to help understand (and test) the encryption blocks. I have attached a screenshot of my program. Basically, it takes a picture of a cat from a .png file and sends it to another.png file. I sent it three ways so as to see how it behaved. One way went straight from file to file, one went through only encryption, and one went through encryption and decryption. With the lower half of the program (the encryption and decryption) disabled, it works on the first route, but when I enable the lower half in an attempt to simultaneously do all 3 paths, the first path only sends the top half of the cat image, and the other two don't send any data to the files at all. The image of my program can be found in the link above this post. I'm new to this so my apologies if this was a bad post, but thanks in advance for any help.
I'm trying to decode array of bytes from DSP TrueSpeech to PCM.
When we convert this array as part of streaming (divide it to packets) we can hear some strange "Beep" tones after the decoding.
We tried to decode the entire WAV file in one piece and we didn't get those Beeps.
Currently we are using Alvas.net for it, but we tried also with NAudio and got the same reaults?
My questions:
1)Is anyone familiar with this kind of behavior?
2)Do you have an idea what can we do?
Thanks
Ziv
How are you performing the decode? Often codecs maintain internal state, so it's important that you don't keep closing and re-opening the codec for each block of audio that you receive. In NAudio, that means just one AcmStream/WaveFormatConversionStream that everything you receive is passed through.
Also, make sure it is only compressed audio that is being passed into the codec. Sometimes when you receive audio over the network it is contained within some kind of larger packet that contains timing or encoding metadata (e.g. RTP).
At the bottom line, we have the packet data(array of bytes) which we are sending to decode (return as PCM) and then we're writing the new decoded array of bytes in to the new WAV file.
We're defiantly going to try your suggestion regarding the stream with NAudio.
Regarding the bytes we're working on, they don't contain any garbage. We've wrote a tester that stream the file directly (without network) and got the same beep results.
Our solution is working so well with many other codecs (GSM and etc..) and only in true speech we're having this problem.
Therefore it seems to be like some behavior of True Speech codec, but we didn't find any documentation about it.
Thanks Again
Ziv
My primary objective is to send a mp3 file over network using QDataStream, QTcpServer and QTcpSocket. But I have broken this task to smaller pieces. At first I need to get the mp3 file to the correct format so that It can be "fed" to the data stream.
How am I supposed to accomplish this? I figured it would be the easiest to use Phonon? But the MediaObject doesnt seem to be offering some sort of getData method.
Any help on how am I supposed to do that would be much appreciated. If needed I can explain more about this.
There is no "correct format". Also, your problem is not MP3-specific. You do the same for all files, regardless of what kind of data they contain. You open the file, read bytes from it and send those bytes until there's nothing left to send.
You don't need Phonon or anything MP3-related. You only need to open the file and read bytes from it. You then write those bytes to the socket using the write() function of your QTcpSocket object. You don't even need a QDataStream, since you're only dealing with data that you don't need to parse.
I am trying to make .txt file look like a .jpg file so it can be sent across wi-fi using an Eye-Fi SD card. The card only sends .jpg files for several reasons. One reason is that the transmission path of the picture from an the SD card to the computer looks like:
Camera writes pictures to EYE-FI SD -> EYE-FI connects to local router -> local router uploads to EYE-FI servers -> EYE-FI servers upload to your computer.
[Explanation]
There could be some filter on the server end, so I found some software that allows the user to bypass the eye-fi servers so now I know I am only dealing with the SD card. It's also nice to know that no one else is looking at my files. After some experimentation, I figured out that I can put .jpg files on the card and have them transmitted once a picture is taken. I also found how that the pictures must be named in short format; a name not longer than 8 characters(excluding file extensions), this probably has to do with the fact the card is formatted in fat32 (the card can be reformatted and still works). I tried uploading a .txt file to the card and gave it a similar format, and renamed it as a .jpg file. It did transfer which indicates to me there is probably something other than a file extension which denotes how the file is formatted.
[Questions]
1) Is there someway I can spoof .txt files to make them look like .jpg files?
2) Is there some kind of program I can use (for linux) to play around with values on the card so I can figure out what triggers an upload? Any ideas one what could trigger the upload?
1) Yes, there are hex value in the file that indicate it is a .jpg. If you open up a .jpg file with a hex editor, you will notice that there are header lines that have a bunch of information about how the image was compress, sometimes what made the image, some firmware information etc. In the editor, you can find the string "FF D8" this indicates the beginning of image file. This is followed shortly by "FF C0". The next 6 bytes contain information about the size of the image, and (I am guessing) is used by whatever software displays the image. The end of a jpg file is denoted by the 2 byes "FF D9". Fun fact, I played around with the jpg file I was using and it seems that you can put text after the "FF D9" and still have the jpg operate. I thought this was neat. Source
None of this was needed to get the eye-fi to upload the file though. As I said in my question, the card needs the name of the file to be in short format (which means the title cannot have more than 8 characters) and needs to have an acceptable file extension, in my case I used ".jpg". I wrote a text file, and just saved it as a "text.jpg". I found that there is a minimum size required in order to transfer the file, which is strange.
My hex editor of choice for this was bless, it is good for opening files, but I have yet to figure out if it can open volumes. It doesn't seem like it can.
I'm increasingly looking at using QR codes to transmit binary information, such as images, since it seems whenever I demo my app, it's happening in situations where the WiFi or 3G/4G just doesn't work.
I'm wondering if it's possible to split a binary file up into multiple parts to be encoded by a series of QR codes?
Would this be as simple as splitting up a text file, or would some sort of complex data coherency check be required?
Yes, you could convert any arbitrary file into a series of QR codes,
something like Books2Barcodes.
The standard way of encoding data too big to fit in one QR code is with the "Structured Append Feature" of the QR code standard.
Alas, I hear that most QR encoders or decoders -- such as zxing -- currently do not (yet) support generating or reading such a series of barcodes that use the structured append feature.
QR codes already have a pretty strong internal error correction.
If you are lucky, perhaps splitting up your file with the "split" utility
into pieces small enough to fit into a easily-readable QR code,
then later scanning them in (hopefully) the right order and using "cat" to re-assemble them,
might be adequate for your application.
You surely can store a lot of data in a QR code; it can store 2953 bytes of data, which is nearly twice the size of a standard TCP/IP packet originated on an Ethernet network, so it's pretty powerful.
You will need to define some header for each QR code that describes its position in the stream required to rebuild the data. It'll be something like filename chunk 12 of 96, though encoded in something better than plain text. (Eight bytes for filename, one byte each for chunk number and total number of chunks -- a maximum of 256 QR codes, one simple ten-byte answer, still leaving 2943 bytes per code.)
You will probably also want to use some form of forward error correction such as erasure codes to encode sufficient redundant data to allow for mis-reads of either individual QR codes or entire missing QR codes to be transparently handled well. While you may be able to take an existing library, such as for Reed-Solomon codes to provide the ability to fix mis-reads within a QR code, handling missing QR codes entirely may take significantly more effort on your part.
Using erasure codes will of course reduce the amount of data you can transmit -- instead of all 753,408 bytes (256 * 2943), you will only have 512k or 384k or even less available to your final images -- depending upon what code rate you choose.
I think it is theoretically possible and as simple as splitting up text file. However, you probably need to design some kind of header to know that the data is multi-part and to make sure different parts can be merged together correctly regardless of the order of scanning.
I am assuming that the QR reader library returns raw binary data, and you will you the job of converting it to whatever form you want.
If you want automated creation and transmission, see
gre/qrloop: Encode a big binary blob to a loop of QR codes
maxg0/displaysocket.js: DisplaySocket.js - a JavaScript library for sending data from one device to another via QR ocdes using only a display and a camera
Note - I haven't used either.
See also: How can I publish data from a private network without adding a bidirectional link to another network - Security StackExchange