How to reverse engineer buffer data received from a serial port - serial-port
I am trying to decode some buffer data that I have received from a serial port. To me the data seems to make no sense - I am not even sure if I am splitting the messages up correctly.
The data comes from a concrete crusher and while the concrete is being crushed we get an almost continuous stream of data. I get about 10 "messages" a second (but this might be multiple messages included in each message) and I am splitting them up by waiting 50 ms after each message. The data looks like this:
[0,0,0,224,0,224,0,0,224,0,0,224,0,0,0,0,0,0,0,224,0,0,0,0,224,0,0,0,224,0,224,0,0,224,0,0,0,224,0,0,0,0,0,0,0,224,0,0,0,224,0,0,224,0,0,224,0,224,0,0,224,0,0,0,0,0,0,0,0,224,224,0]
[0,0,0,224,0,224,0,0,224,0,0,224,0,0,0,0,0,0,0,224,0,0,0,0,0,0,0,224,0,224,0,0,0,0,0,0,0,0,0,0,0,0,0,224,0,0,0,224,0,0,224,0,0,224,0,0,0,0,224,0,0,0,0,0,0,0,0,224,224,0]
[0,0,0,224,0,224,0,0,224,0,0,224,0,0,0,0,0,0,0,224,0,224,0,0,224,0,0,0,224,0,224,224,224,0,0,0,224,0,0,0,0,0,0,0,0,224,0,0,0,224,0,224,224,224,0,0,224,0,0,0,224,0,0,0,224,0,0,0,0,0,224,224,0]
as you can see there are no values at all other than 0 and 224...
The last message is:
[0,0,0,0,0,0,224,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,224,0,0,224,0,0,0,0,0,0,0,0,0,0,0,0,0,0,224,0,0,0,0,224,224,0,0,224,0,0,224,0,0,0,224,0,0,0,0,0,0,0,0,0,0,0,224,0,0,0,0,0,224,0,224,224,0,0,224,0,0,0,224,224,0]
the value displayed on the machine was 427.681kN but I can't see any way that this data can produce that.
Each message ends with 224,224,0 so I am wondering if that is the split sequence?
I am getting this data with node-red and this is the format that I can copy it from the debug panel in.
I am very lost so any guidance or directions that I can look in would be much appreciated.
Related
Qt: API to write raw QAudioInput data to file just like QAudioRecorder
I'm trying to monitor an audio input and record the audio to a file but only when a level threshold is exceeded. There seems to be two main options for recording in Qt; QAudioRecorder and QAudioInput.
Long story short: I'm trying to find the API that can take raw audio sample data read from QAudioInput and record it to a file just like QAudioRecorder does, but strangely it doesn't seem to exist. To give an example, the setup of the QAudioRecorder would be something like the following (but instead of specifying a input device with setAudioInput() you pass it sampled bytes):
QAudioEncoderSettings audioSettings;
QAudioRecorder recorder = new QAudioRecorder;
audioSettings.setCodec("audio/PCM");
audioSettings.setQuality(QMultimedia::HighQuality);
recorder.setEncodingSettings(audioSettings);
recorder.setContainerFormat("wav");
recorder.setOutputLocation(QUrl::fromLocalFile("/tmp/test.wav"));
recorder.setAudioInput("default");
recorder.record();
I'm using QAudioInput because I need access to the raw samples. The problem with QAudioInput is, Qt does not seem to provide an easy way to take the raw samples I get out of the QAudioInput and pipe them into a file encoding them along the way. QAudioRecorder does this nicely, but you can't feed raw samples into QAudioRecorder; you just tell it which device to record from and how you want it stored.
Note I tried using QAudioInput and QAudioRecorder together - QAudioInput for the raw access and QAudioRecorder whenever I need to record, but there is two main issues with that: A) Only one of these can be reading a given device at a time. B) I want to record the data at and just before the threshold is exceeded and not just after the threshold is exceeded.
I ended up using QAudioRecorder+QAudioProbe. There are a couple of limitations though: Firstly the attached QAudioProbe only works if QAudioRecorder is actually recording, so I had to write a wrapper over QAudioRecorder to switch on/off recording by switching output device to|from actual_file|/dev/null. Second, as I stated "I want to record the data at and just before the threshold is exceeded and not just after the threshold is exceeded". Well, I had to compromise on that. The probe is used to detect the recording condition, but there is no way to stuff the data from the probe back into the recorder. I mean, I guess you could record to a buffer file in idle state and somehow prepend part of that data ... but the complexity wasn't worth it for me. Aside; there was another issue with QAudioRecorder that motivated me to write a wrapper over it. Basically I found QAudioRecorder::stop() sometimes hangs indefinitely. To get around this, I had to heap allocate a recorder and delete it and init a new one with each new recording.
How to extract raw data from TCP packets using Wireshark
Completely new to Wireshark and wondering how to extract the data from the TCP packets which I receive on wireshark. I am currently using a raspberry pi with grove sensors and getting the values of pressure and temperature. I am sending these values to a server in cloud and it is working. I am using wireshark to trace the packets. Now I want to extract the data (i.e the pressure value and temperature value) from the packets and store them in a file for further implementation. Is there a way to do it? If yes, then can anyone please explain? It will be helpful. Thanks.
Most convenient way is saving whole TCP stream into file. Right click on any TCP packet of desired stream, choose "Follow -> TCP stream" and you will see window for managing stream data. You can choose, which data to save (one-direction, or both), which format will be used for output and so on. Check pict below: The other way is saving data from any packet individually. Since you aren't interested in whole packet, you can select only payload part. Check picture below, where I selected HTTP part of the packet (I marked field with red arrow for clearance). In your case, it will be some field with raw data. Once highlighted the right field, right click on it, choose "copy" and select desired format. Data will be placed into buffer, so you need to paste it into some application. If raw binary is chosen for format, you need application capable of pasting binary data. I use frhed for this task.
MBed/Arduino RS-232 Serial communication issue
I am receiving messages from a CAN interface into my mBed device. The mBed device then parses the information to send out on serial to another device. The information is sent out in the following format. "< msg>xxxxxxxxxxxxxxxxxxx< /msg>" where x = a hex number. The other device receiving this message will receive the information split up in half (i've accounted for this in the code). The problem I'm having is, the messages will fall into the format ..... but there are times where the format is lost, for example: [1]xxxx< /msg>< msg>xxxxx [2]xxxxxxxx< msg>xxxxxxx [3]< /msg>< msg>xxxxxxxxx [4]xxx< /msg>< msg>xxxxxx **Please ignore the space in the msg tag, it was necessary to show on StackOverflow'** The baud rate set 38400bps on the mBed. I'm not using any parity, stop bit, start bit etc as I'm not too familiar with how they work. Can anyone help me how I might fix this loss in format, or am I going to have to include code in the receiving device to handle this? Many thanks in advance!
This is entirely normal, serial ports are not smart enough to recognize XML. You will have to write the code yourself. A basic algorithm is a state machine. Declare a buffer that's large enough to store a complete message. Then: throw everything away you receive until you get '<' store bytes you receive in the buffer until you get '>' check that you got <msg>, back to state 1 if you did not store bytes you receive in the buffer until you get '>' check that you got <msg/>, back to state 1 if you did not process the message, back to state 1 This ensures that you'll properly sync with the bus when you open the port and that you don't care how many bytes you receive in one read() call.
AT command for disable Radio Signal Strength Indication and alike?
Im working on a program to send and recieve SMS using a GSM modem and my computer. I have gotten sending and receiving to work - well sort of. Once in a while my program is sent into a total chrash due to modem is mixing up information about Radio Signal Strength Indication and alike, while also serving my program with the hex code for the message. My code can handle the hex code just fine. but I have seen the following line popup while im decoding a byte stream: ^RSSI: 2 So far I've seen it send out values between 1 and 10. Is there an AT Command that can disable them? I have no need for them. Or alternative: Is there a general syntax for them, so I can filter them out before decoding? Im leaning towards a filter solution. But that would be more easy to implement if I knew whenever modem is sending out on the form: "^SOMETHING: xxx", then It would be nice to know if it is always followed up be a delimiter say for instance "\r".
You should try turning off periodic messages as using AT^CURC=0. Information regarding the AT^CURC command: AT^CURC? Current setting of periodic status messages AT^CURC=? See what you possible values are AT^CURC=0 turn off periodic status messages
The best way to tackle this scenario would be to replace that part of the response with an empty string because otherwise, it will be difficult to check even if the command sent to disable it is working or not. This regex will match all those. You can replace them ideally by an empty string. (\\n|\\r|\\r\\n)\\^.*(\\n|\\r|\\r\\n)
Locating data (like pressure etc) in a raw data stream
I am a newbie to Serial Port Analysis and I would appreciate some help on this. my specific question is... If I have raw data from a serial port analyzer program, how will I locate measures like temperature, pressure, energy etc? What should I look for in the raw data that will help me identify these units of measure? What is the best way the extract relevant data from this raw data? I would be very grateful if you can provide me any help with respect to this. I am unable to figure out how to do this. Thanks a lot.
The best way that I know of to do this is to find the "reset" identifier, also called the "End of Stream" identifier or sequence. I am assuming that the data is a continuous flow not a one-time transmission. If the data is continuously cycling, you need to find where the transmit begins (or ends) and then start metering your capture from there. Most devices will have an associated manual or documentation that give you the end sequence (or optionally the start sequence) and then the method by which they identifier their data. For instance, the device may end a message by sending 4 all zero bytes in a row, then begin again by sending one byte that identifies the sensor, and another two bytes with the data, followed by the next sensor etc. You would then watch the stream for 4 zero byte entries, and then start capturing 3 bytes at a time, one for the sensor and two for the data, until you saw 4 zero byte entries in a row again.