I try to read multipart sms in PDU mode.
The message was in 3 parts
Below are PDUs I got by using command AT+CMGF=0 and AT+CMGL=4
Part1:07914150740250F7440B917130263521F600005140723295528AA005C01B5B0301B2E53C194D46A3C96834196D169BD16833DA8C368BCD62B3D82C368BCD62B3586C169BC566B1596C169BC562B3D82C368BCD62B3D82C368BCD62B3DBEC769BDD66B7D90D328B41663768DC0699DD66B7D96D769BDD66B7D96D76BBCD6EB3DBEC36BBCD6EB3DBEC36BBCD6EF7D96D769BDD67F7D96D769FDD67B7FBEC3EBBCFEEB3DB7D769FDD
Part2:07912160130320F8440B917130263521F600005140723295528AA005C01B5B0302CE6EB3DBEC56AB41D9729E8C26A3D164349A8C368BCD68B4196D469BC566B1596C169BC566B1592C368BCD62B3D82C368BCD62B1596C169BC566B1596C169BC566B1D96D76BBCD6EB3DBEC0699C520B31B346E83CC6EB3DBEC36BBCD6EB3DBEC36BBDD66B7D96D769BDD66B7D96D769BDD66B7FBEC36BBCDEEB3FBEC36BBCFEEB3DB7D769FDD
Part3:07914140540500F9440B917130263521F600005140723295528A1805C01B5B0303CEEEB3DB7D769FDD67B7D96D76ABD5
*According to my understanding, in order to identify if it's a multipart message I have to check if TP-UDHI is set where it's the sixth bit in first octet. In this case it's not set.
*The bold part of the PDU was the Data Header
*I thought in order to indicate this is a concatenate message it has to be 00 instead of C0?
Please correct me if I get it wrong..
Question 1: Why is the TP-UDHI not set in this case first octet was 07?
Question 2: Why the first octet in UDH is not 00 instead of C0?
Ok answering question 1) You missed spotting that right at the start of the PDU there is the SMSC address. So in fact your PDU header octet is 44. This indicates that there is a UDH present in the PDU.
This is the SMSC address:
07914150740250F7
Directly thereafter is the PDU header 44.
Regarding question 2) things get a little more complicated. Right now I am not spotting that the UDH contains any indication of a concatenated SMS. Don't forget that the UDH is not just for concatenated messages. It can contain lots of other information based on the 3GPP ETSI spec 03.40.
After a closer look it looks like either the SMS was strangely encoded on the sending side or the mobile operator messed around with the UDH. You were correct isolating the UDH as:
C01B5B0302
Based upon the byte prior then the UDH length should be 5 bytes. But the first IEI (Information Element) is misleading. C0 defines the IEI as a SC specific IEI and not a concatenated IEI. Then the next by 1B says the IEI data should be 27 bytes long which contradicts the UDH length of 5.
So from my perspective something mangled the UDH (which can happen with mobile operators, sms aggregators, or even bad encoders).
If you would play around with what you have removing C01B and replacing with 0003 to ensure the 8-bit concat reference:
00035b0301
00035B0302
00035B0303
Then you would end up with a UDH telling you that the MR is 91 and the parts a correctly specified.
Related
I am trying to send a mavlink command for instance
GPS_RTCM_DATA ( #233 )
flags uint8_t
len uint8_t
data uint8_t[180] RTCM message (may be fragmented)
https://mavlink.io/en/messages/common.html#GPS_RTCM_DATA
I understand uint8_ would be in a single byte unsigned int.
What does the [180] means?
The uint8_t[180] in the MAVLink GPS_RTCM_DATA message means that the data field can contain up to 180 bytes.
Beware that RTCM messages can be bigger than 180 bytes and be fragmented in
more than one GPS_RTCM_DATA message.
You can check the flags field as stated in the mavlink documentation:
LSB: 1 means message is fragmented, next 2 bits are the fragment ID,
the remaining 5 bits are used for the sequence ID. Messages are only
to be flushed to the GPS when the entire message has been
reconstructed on the autopilot. The fragment ID specifies which order
the fragments should be assembled into a buffer, while the sequence ID
is used to detect a mismatch between different buffers. The buffer is
considered fully reconstructed when either all 4 fragments are
present, or all the fragments before the first fragment with a non
full payload is received. This management is used to ensure that
normal GPS operation doesn't corrupt RTCM data, and to recover from a
unreliable transport delivery order.
I tried every but it doesn't work. Except putting it as a 180 byte arrays. The data might be only 30 bytes for example. But input with the other 150 0x00 bytes in this way, the python program accepts my command. Strangely so. I can't explain why but in this case it works.
Suppose I am receiving packets containing H264 encoded NAL Units, and each NAL has correct START_CODE (0x00 00 00 01).
But in some NALUs, the later Bytes of the NAL may have been corrupted. Is there any standard way to validate whether the information represented by a NALU is correct or not?
And if it can be validated, is there any standard way to correct it according to the H264 standards ?
Further Clarification:
The problem to specify is somewhat like this ->
Server sends a x264 NALU like 0x 00000001abcdefgh and, when I receive
the packet, it contains : 0x 00000001abcmnopdefgh.
This leads to failure of decompression , hence I was looking for a way
to eliminate these bad data bytes.
The only way to validate it is to parse the entire nal and make sure you have the correct number of bits. Even then some errors may not be detected.
No there is no way to correct the errors unless the container/file includes erasure codes.
I am trying to debug a RTMP client that fails to connect to some servers. I'm using Wireshark to capture the packets and compare them with a client that connects successfully (in this case, ffmpeg).
Looking at the captured packets for a successfull connection, I noticed that, when viewing at TCP level, there is an extra byte in the payload (see pics below). The extra byte has value 0xc3 and is placed at byte 0xc3 in the payload.
I Googled the best I could to find information about extra bytes in the TCP payload, but I didn't find anything like this. I tried to look in the TCP spec but no luck either. Where can I find information about this ?
TCP-level view
RTMP-level view
This happens because the message length is larger than the maximum chunk size (as per the RTMP spec, the default maximum chunk size is 128). So if no Set Chunk Size control message was sent before connect (in your case), and the connect message is greater than 128 bytes, the client will split the message into multiple chunks.
0xC3 is the header of the next chunk, looking at the bits of 0xC3 we would have 11 000011. The highest 2 bits specify the format (fmt = 3 in this case, meaning that this next chunk is a type 3 chunk as per the spec). The remaining 6 bits specify the chunk stream ID (in this case 3). So that extra byte you're seeing is the header of a new chunk. The client/server would then have to assemble these chunks to form the complete message.
Suppose you have a specification for sending information on a TCP or UDP stream and you have a sequence of bytes that you receive delimited with STX and EOT bytes. How do you handle for example the EOT byte occurring in the actual data? This is possible I think: most bytes in the message represent numbers in a defined order (i.e. it's not just ascii text in byte form) so EOT is byte 0x04 and this a number that could occur in the data. The specification is unclear on this: should I always look at the last occurrance of EOT and ignore those in between? Other similar specifications I've seen could even handle multiple messages inside the same TCP/UDP message: for example STX some_data EOT STX more_data EOT inside one TCP/UDP message. In this case you can't just look at the last EOT because it's actually 2 separate messages. Do you do some form of escaping then?
How is this sort of thing handled usually? I couldn't find anything on google, but perhaps I'm not using the best search terms.
"Usually" the protocol should be well designed, so that messages either don't contain the delimiter, use an escape mechanism to include the delimiter, or have a known length so that you know where the message ends without having to depend on the delimiter.
If the messages are fixed size integers for example you'll know that EOT encountered within an integer is not a delimiter.
Is it within TCP standard that multiple messages, sent from server to client in a row, will be accepted by client at same order (and bytes of one message will not be scattered within other messages)?
TCP provides an in-order byte stream delivery service. The bytes won't arrive in another order but the number of writes need not be equal to the number of reads.
You will never read bytes in another order than that in which they were sent
You can make no assumptions on "messages". TCP doesn't know about messages, only bytes (see above). Both the sender and the receiver can coalesce and split such "messages"
TCP uses a sequence number to identify each byte of data. The sequence number identifies the order of the bytes sent from each computer so that the data can be reconstructed in order, regardless of any fragmentation, disordering, or packet loss that may occur during transmission.
I agree with #cnicutar.
How are you deserializing the objects? I suspect the problem lies there.
For example if your messages are like
ABCD followed 200 ms later by PQR. It may appear as:
ABC followed by PQR
or ABCDPQR
or even AB followed by CD followed by PQ followed by R.
Basically you cannot make assumptions based on time of receiving the data.
The deserialization logic should know the object boundaries within a stream of bytes. This information should be encoded into the stream by the serialization logic.
If you are using Java, you can use ObjectInputStream & ObjectOutputStream and not be bothered about serialzation issues.
J2ME Polish has a good serialization utility that can be very easily ported to other platforms. I have myself used it in live environment.