How to send a message of the form [Integer, String] in erlang using gen_tcp.
Eg: I am looking to send messages of the form [25, "Hello"] or [50, "Hi"] over a socket using gen_tcp:send/2.
I tried to do [ID | Msg] but that doesn't help.
Thanks
In Erlang, a string is just a list of Integers, so the list [25, "Hello"]
is actually represented like [25, [72,101,108,108,111]]
The question remains, how to send that information over a Socket in Erlang.
According to the documentation, in Erlang you can send the data as binary or as a list, which you can set either as {mode, binary} or {mode, list} when you create the Socket.
In the case that you are working with binary data (which is advice since sending and receiving binaries is faster than lists), you can do something like:
{ok, Socket} = gen_tcp:connect(Host, Port, [{mode, binary}]).
Data = list_to_binary([25, "Hello"]).
gen_tcp:send(Socket, Data).
Now, if you use your socket in list mode, you just send the list directly:
{ok, Socket} = gen_tcp:connect(Host, Port, [{mode, binary}]).
Data = [25, "Hello"].
gen_tcp:send(Socket, Data).
At the server side, if you receive the data in a Socket which is in list mode, you convert back to your original format with:
[Integer|String] = ListReceived.
If you receive the data in a Socket with binary mode, then you have to transform the data from a Binary to a List, like:
[Integer|String] = binary_to_list(BinaryReceived).
Related
I am working on a simple chat server and client in golang. I am having some trouble with reading messages from the net.Conn. So far this is what I have been doing:
bufio.NewReader(conn).ReadString('\n')
Since the user presses enter to send the message I only have to read until '\n'. But I am now working on encryption and when sending the public keys between client and server the key sometimes contains '\n', which makes it hard to get the whole key. I am just wondering how I can read the whole message instead of stopping at a specific character. Thanks!
A simple option for sending binary data is to use a length prefix. Encode the data size as a 32bit big endian integer, then read that amount of data.
// create the length prefix
prefix := make([]byte, 4)
binary.BigEndian.PutUint32(prefix, uint32(len(message)))
// write the prefix and the data to the stream (checking errors)
_, err := conn.Write(prefix)
_, err = conn.Write(message)
And to read the message
// read the length prefix
prefix := make([]byte, 4)
_, err = io.ReadFull(conn, prefix)
length := binary.BigEndian.Uint32(prefix)
// verify length if there are restrictions
message = make([]byte, int(length))
_, err = io.ReadFull(conn, message)
See also Golang: TCP client/server data delimiter
You can also of course use an existing, well test protocol, like HTTP, IRC, etc. for your messaging needs. The go std library comes with a simple textproto package, or you could opt to enclose the messages in a uniform encoding, like JSON.
I am using Haskell to send a POST HTTP request via the http-client package, but as a response i am getting an error from the remote server. I would like to inspect my request in order to be sure that i am sending the expected parameters in the body.
I managed to bind my request to a variable in the Haskell interpreter, but i can't figure out how to see its body. When i use requestBody on my request i get a value of type RequestBody, but it is not an instance of show.
In the library i can't find any helpful function. There are just many constructors, and some functions related to a Popper type.
How can i inspect that value? This seems a common use case to me, yet the library doesn't seem to support it. Am i using it wrong?
Edit:
I know what Wireshark is and how to use it, but i expect to be able to programmatically inspect what i am sending
RequestBody has five constructors.
data RequestBody
= RequestBodyLBS L.ByteString
| RequestBodyBS S.ByteString
| RequestBodyBuilder Int64 Builder
| RequestBodyStream Int64 (GivesPopper ())
| RequestBodyStreamChunked (GivesPopper ())
First two are wrappers of bytestring, the third is a Blaze.ByteString.Builder which can be easily converted to bytestring, the last two are functions of type (IO ByteString -> IO a) -> IO a as the type synonym showed:
type Popper = IO S.ByteString
type NeedsPopper a = Popper -> IO a
type GivesPopper a = NeedsPopper a -> IO a
feed them >>= print multiple times and you will get the complete request body printed to console.
You should not do this because these GivesPopper () generators can only be executed once: if you extract the content beforehand then the request body will not be sent to the server. That's why they didn't provide a Show instance for RequestBody data type. You'd better capture the buckets containing your http request with e.g. Wireshark.
I have a data structure in my code and i don't see why i shouldn't be able to inspect it. You mention that inspecting the request might somehow consume it. This is weird in a language like Haskell where immutability and explicit side effects are the rule
That's right for most of the types in Haskell, but Popper is actually IO S.ByteString, a bytestring stream (usually) populated by a file handler, while handler in Haskell is an MVar which is not so different from variables in imperative languages: it may be a value or null, may be assigned and reassigned and the side effect is visible globally.
An example from Network.HTTP.Client.MultipartFormData:
streamFile :: FilePath -> GivesPopper ()
streamFile fp np =
withFile fp ReadMode $ np . go
where
-- go :: Handle -> Popper
go h = BS.hGetSome h defaultChunkSize
In my simple TCP client server application, server send repetitively 1 kB message to the client and client send a reply acknowledgement (just send 'ACK') for each packet. Just think this scenario like client and server passing 1 kB messages here and there in a infinite loop.
I send the same message every time and the fist byte (first char) is always 1. But while testing this client and server application in the same machine for a long time, I noticed first character of some of the received messages are something else in the receive buffer and recv function also returned 1024 (1 kB). This is not happen frequently.
This is the how I receive.
char recvBuff[DEFAULT_BUFFER_SIZE];
int iResult = SOCKET_ERROR;
iResult = recv(curSocket, recvBuff, DEFAULT_BUFFER_SIZE, 0);
if (iResult == SOCKET_ERROR)
{
return iResult;
}
if (recvBuff[0] != 1)
{
//malformed receive
}
MessageHeader *q = (MessageHeader*)recvBuff;
message.header = *q; q++;
std::string temp((char*)q, message.header.fragmentSize);
message.message = temp;
Actually the problem is in constructing the temp string. It breaks since the correct fragment size not received. I tried to drop these kind of malformed data. But the problem is there is a gap between last successfully received fragment ID and first successfully received fragment ID after malformed receives. Any idea why these malformed receives happen?
You’re assuming that you’ve received a complete message when the recv() call completes. If this is a TCP connection (as opposed to UDP), it is byte-oriented, and that means that recv() will return whenever there are any bytes available.
Put more explicitly, there is no reason that doing
send (toServerSocket, someMessage, 1024, 0);
on the client side will cause
recv (fromClientSocket, myBuffer, 1024, 0);
to receive 1,024 bytes. It could just as well receive 27 bytes, with the remaining 997 coming from future calls to recv().
What’s happening in your program, then, is that you’re getting one of these short returns, and it’s causing your program to lose sync. with the message stream. How to fix it? Use recv() to read enough of your message that you know the length (or set a fixed length, though that’s inefficient in many cases). Then continue calling recv() into your buffer until you have read at least that many bytes. Note that you might read more bytes than the length of your message — that is, you may read some bytes that belong to the next message, so you will need to keep those in the buffer after processing the current message.
I have a device that sends data to a server.
Data
[ Client ] == > [ Server ]
After the validation on the server I want to return a message:
OK
[ Client ] < == [ Server ]
Is there a standard "OK" message to return? And an "ERROR" message? How does it looks like? (e.g. ":0011", ":110F")
You've got to design an application-level protocol. TCP is a byte stream, so even the definition of "Data" in your client->server piece needs some protocol so that the receiver can know what bytes make up the data (when to stop reading).
A couple of common types of protocols are...
Length-delimited chunks. Every message starts with a 16 or 32-bit length prefix. Then that many bytes follow. The length needs to be in a defined byte order (see htons, ntohs, etc). Everyone who uses this protocol knows to read the length prefix then read that many bytes. Having defined that "chunk" on the network, you might put a header on the contents of the chunk. Maybe a message type (ACK, NAK, Data, etc) followed by some contents.
ASCII newline delimited. Each message is a line of ASCII (or UTF8, etc) text. It ends at a newline. Newline endings for the lines play the same role as the length prefix for the chunks above. You then define what's in each line (like space or comma-delimited ASCII/UTF8/whatever fields). Somewhere in that you'd define what data looks like, ACK, etc.
I'm sure you could come up with other ideas, but that's the basic job: defining your application-level protocol on top of TCP's byte stream.
I've been looking into Mochiweb, but I can't find a way to read the body while I'm receiving the request through the socket.
I'm not limited to Mochiweb, any other HTTP library would be good for me.
I also tried gen_tcp:listen(Port, [{packet, http}]), this way I can read the body/headers while I'm receiving the HTTP request, but I must handle manually the responses and keeping the socket open for more requests, so I prefer not to use this solution.
My intention is to receive request with large bodies and not to wait to receive the full body in order to start reading/processing them.
With mochiweb you can fold over chunks of the request body using Req:stream_body/3.
It expects a chunk handler function as the second argument. This handler is called with
{ChunkSize, BinaryData} and your state for every chunk, as it is received from the socket.
Example (retrieving a [reversed] list of chunks):
MaxChunkSize = 100,
InitialState = [],
ChunkHandler = fun ({_Size, Bin}, State) -> [Bin | State] end,
List = Req:stream_body(MaxChunkSize, ChunkHandler, InitialState),
...