How to send string using Qtcpsocket, when using
tcpsocket->write("hello");
tcpsocket->write("world");
etc..
tcpSocket->flush();
tcpSocket->waitForBytesWritten(3000);
it send it in one string "hello world", what i want make it write only one at time, i want make client recive "hello" then "world".
This is not how TCP works. TCP is a byte stream protocol, not a message protocol. You might programatically write N bytes on the sending side, but the remote end might not receive all N bytes at once. In fact, when it does a recv on its end, it might only get 1 byte, N-1 bytes, or some other number of bytes. Issues such as IP fragmentation, TCP segmentation, TCP window size, can influence this.
Further if you write "Hello" and "World" separately to the socket, the message could easily get coalesced (on the sender or receiver side), such that "HelloWorld" is received all at once. Again, because TCP is a byte stream, not a message based protocol.
When you write TCP code, you have to deal with these issues, because they really do happen.
When you want to do:
Each word is a "message". But each message needs it's only encapsulation. Perhaps you could send the messages like this:
tcpsocket->write("hello|");
tcpsocket->write("world|");
Where the trailing | pipe character of each word is the delimiter between each logical word. You could also use a space instead of a pipe char. Or have your own protocol header to indicate the number of bytes to follow. Regardless, it's up to the receiving side to parse the the messages from the byte stream back together to form the application messages.
Related
I am trying to debug a RTMP client that fails to connect to some servers. I'm using Wireshark to capture the packets and compare them with a client that connects successfully (in this case, ffmpeg).
Looking at the captured packets for a successfull connection, I noticed that, when viewing at TCP level, there is an extra byte in the payload (see pics below). The extra byte has value 0xc3 and is placed at byte 0xc3 in the payload.
I Googled the best I could to find information about extra bytes in the TCP payload, but I didn't find anything like this. I tried to look in the TCP spec but no luck either. Where can I find information about this ?
TCP-level view
RTMP-level view
This happens because the message length is larger than the maximum chunk size (as per the RTMP spec, the default maximum chunk size is 128). So if no Set Chunk Size control message was sent before connect (in your case), and the connect message is greater than 128 bytes, the client will split the message into multiple chunks.
0xC3 is the header of the next chunk, looking at the bits of 0xC3 we would have 11 000011. The highest 2 bits specify the format (fmt = 3 in this case, meaning that this next chunk is a type 3 chunk as per the spec). The remaining 6 bits specify the chunk stream ID (in this case 3). So that extra byte you're seeing is the header of a new chunk. The client/server would then have to assemble these chunks to form the complete message.
I'm trying to send and receive messages over TCP using a size of each message appended before the it starts.
Say, First three bytes will be the length and later will the message:
As a small example:
005Hello003Hey002Hi
I'll be using this method to do large messages, but because the buffer size will be a constant integer say, 200 Bytes. So, there is a chance that a complete message may not be received e.g. instead of 005Hello I get 005He nor a complete length may be received e.g. I get 2 bytes of length in message.
So, to get over this problem, I'll need to wait for next message and append it to the incomplete message etc.
My question is: Am I the only one having these difficulties to appending messages to each other, appending lengths etc.. to make them complete Or is this really usually how we need to handle the individual messages on TCP? Or, if there is a better way?
What you're seeing is 100% normal TCP behavior. It is completely expected that you'll loop receiving bytes until you get a "message" (whatever that means in your context). It's part of the work of going from a low-level TCP byte stream to a higher-level concept like "message".
And "usr" is right above. There are higher level abstractions that you may have available. If they're appropriate, use them to avoid reinventing the wheel.
So, there is a chance that a complete message may not be received e.g.
instead of 005Hello I get 005He nor a complete length may be received
e.g. I get 2 bytes of length in message.
Yes. TCP gives you at least one byte per read, that's all.
Or is this really usually how we need to handle the individual messages on TCP? Or, if there is a better way?
Try using higher-level primitives. For example, BinaryReader allows you to read exactly N bytes (it will internally loop). StreamReader lets you forget this peculiarity of TCP as well.
Even better is using even more higher-level abstractions such as HTTP (request/response pattern - very common), protobuf as a serialization format or web services which automate pretty much all transport layer concerns.
Don't do TCP if you can avoid it.
So, to get over this problem, I'll need to wait for next message and append it to the incomplete message etc.
Yep, this is how things are done at the socket level code. For each socket you would like to allocate a buffer of at least the same size as kernel socket receive buffer, so that you can read the entire kernel buffer in one read/recv/resvmsg call. Reading from the socket in a loop may starve other sockets in your application (this is why they changed epoll to be level-triggered by default, because the default edge-triggered forced application writers to read in a loop).
The first incomplete message is always kept in the beginning of the buffer, reading the socket continues at the next free byte in the buffer, so that it automatically appends to the incomplete message.
Once reading is done, normally a higher level callback is called with the pointers to all read data in the buffer. That callback should consume all complete messages in the buffer and return how many bytes it has consumed (may be 0 if there is only an incomplete message). The buffer management code should memmove the remaining unconsumed bytes (if any) to the beginning of the buffer. Alternatively, a ring-buffer can be used to avoid moving those unconsumed bytes, but in this case the higher level code should be able to cope with ring-buffer iterators, which it may be not ready to. Hence keeping the buffer linear may be the most convenient option.
I got a question regarding tcp/ip socket networking. Basically it is there: are there any parts of tcp/ip that I can leverage to help manage multi-packet sends. For example, I want to send a 100 mb binary file which would take something like 70-80 tcp packets. Meanwhile I have a relatively fast polling receive on the other side. Would my receive have to receive each packet it individually and "stitch" together the data packet by packet, looking for some size to be reached(it can look at the opcode and determine size) or is there some way to tell tcp to say "hey I'm sending 100 mb here, let them know when it is finished."
I am using glib's low level socket library (gsocket).
When using a binary encoding like, say protocol buffers, you would wrap the actual payload by inserting a header that would include the information necessary to decode the payload on the other end.
Say appending 8 bytes where the first 4 signify the type of the encoded message and the second four indicate the length of the entire message.
On the receiving side you are then reading this header, that's part of the payload, to determine the message type and length of the message. This lets you combine multiple messages in one payload or split messages across packets and reliably recombine them.
Is it within TCP standard that multiple messages, sent from server to client in a row, will be accepted by client at same order (and bytes of one message will not be scattered within other messages)?
TCP provides an in-order byte stream delivery service. The bytes won't arrive in another order but the number of writes need not be equal to the number of reads.
You will never read bytes in another order than that in which they were sent
You can make no assumptions on "messages". TCP doesn't know about messages, only bytes (see above). Both the sender and the receiver can coalesce and split such "messages"
TCP uses a sequence number to identify each byte of data. The sequence number identifies the order of the bytes sent from each computer so that the data can be reconstructed in order, regardless of any fragmentation, disordering, or packet loss that may occur during transmission.
I agree with #cnicutar.
How are you deserializing the objects? I suspect the problem lies there.
For example if your messages are like
ABCD followed 200 ms later by PQR. It may appear as:
ABC followed by PQR
or ABCDPQR
or even AB followed by CD followed by PQ followed by R.
Basically you cannot make assumptions based on time of receiving the data.
The deserialization logic should know the object boundaries within a stream of bytes. This information should be encoded into the stream by the serialization logic.
If you are using Java, you can use ObjectInputStream & ObjectOutputStream and not be bothered about serialzation issues.
J2ME Polish has a good serialization utility that can be very easily ported to other platforms. I have myself used it in live environment.
Are these codes the same behaviour for the remote:
a:
socket.write("aaaa");
socket.waitForBytesWrite(3000);
socket.write("b");
b:
socket.write("aaaa");
socket.write("b");
I know the first code will get "aaaab" but..
I don't know if the second codes would result in "aabaa" or something else.
They are equivalent (as in, the remote end should receive the same order of data). In your second case if the socket has not finished sending it's current chunk of data, the new data to send will be appended to the end of the internal buffer for later writing.
This assumes, of course, that you're using TCP - if you use UDP, there's no guarantees the packets will arrive in the order you send them.
What type of socket are you using? TCP or UDP?
If you use TCP socket:
First and second lines will result in "aaaab".
If you are using UDP:
First and second lines in a very bad condition will result in "aaaab" or "baaaa". Below code is better to insure the sequence of UDP packets
socket.write("aaaa");
if (socket.waitForBytesWrite(3000))
socket.write("b");