Erlang gen_tcp missing packets? - tcp

Question

I'm developing a mail client adapter for Erlang. I'm having issues when I try to perform a fetch command, Erlang isn't able to get the body's content.
This is output from my terminal, when I'm trying to utilize the command through netcat:
4 FETCH 2 BODY[2]
* 2 FETCH (BODY[2] {1135}
<div>
test content
</div>
)
4 OK FETCH completed.
The only output the gen_tcp server is able to receive is this binary:
<<"* 2 FETCH (BODY[2] {1135}\r\n">>
The source code is here:
-module(mailconnector).
-behaviour(gen_server).
-export([start_link/2, stop/0]).
-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
start_link(Host, imap) ->
gen_server:start_link({local, ?MODULE}, ?MODULE, [Host, 143], []).
stop() ->
gen_server:call(?MODULE, stop).
init([Host, Port]) ->
{ok, Sock} = gen_tcp:connect(Host, Port, [binary, {packet, 0}, {active, true}]),
{ok, {Sock, 0}}.
handle_call(stop, _From, State) ->
{stop, normal, ok, State};
handle_call({login, Username, Password}, _From, State) ->
{NewState, Output} = action(State, string:join(["LOGIN", Username, Password], " ")),
case Output of
{ok, Response, Data} -> Result = {Response, Data};
_ -> Result = false
end,
{reply, Result, NewState};
handle_call(list, _From, State) ->
{NewState, Resp} = action(State, "LIST \"\" \"*\""),
{reply, Resp, NewState};
handle_call({select, MailBox}, _From, State) ->
{NewState, Output} = action(State, string:join(["SELECT", MailBox], " ")),
case Output of
{ok, Response, Data} -> Result = {Response, Data};
_ -> Result = false
end,
{reply, Result, NewState};
handle_call({fetch, Num}, _From, State) ->
{NewState, Output} = action(State, string:join(["FETCH", Num, "BODY[1]"], " ")),
case Output of
{ok, Response, Data} -> Result = {Response, Data};
_ -> Result = false
end,
{reply, Result, NewState};
handle_call(_Command, _From, _State) ->
{reply, not_valid, _State}.
handle_cast(_Command, State) ->
{noreply, State}.
handle_info(_Info, State) ->
{noreply, State}.
terminate(_Reason, _State) ->
ok.
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
action(_State = {Socket, Counter}, Command) ->
NewCount = Counter+1,
CounterAsList = lists:flatten(io_lib:format("~p ", [NewCount])),
Message = list_to_binary(lists:concat([CounterAsList, Command, "\r\n"])),
io:format("~p~n", [Message]),
gen_tcp:send(Socket, Message),
{{Socket, NewCount}, listener(Socket, NewCount)}.
listener(_Sock, Count) ->
receive
{_, _, Reply} ->
io:format("RECEIVED: ~p~n", [Reply]),
Messages = string:tokens(binary_to_list(Reply), "\r\n"),
io:format("~p~n", [Messages]),
find_message(Messages, Count)
after 5000 ->
timeout
end.
process_message(Message, Count) ->
StringCount = lists:flatten(io_lib:format("~p", [Count])),
case [MCount|PureMessage] = string:tokens(Message, " ") of
_M when StringCount == MCount ->
{ok, string:join(PureMessage, " ")};
_ -> [_Command|Output] = PureMessage, {data, string:join(Output, " ")}
end.
find_message(Messages, Count) ->
find_message(Messages, Count, []).
find_message([], _, _) ->
false;
find_message([H|T], Count, Data) ->
case process_message(H, Count) of
{ok, Message} -> {ok, Message, lists:reverse(Data)};
{data, Output} -> find_message(T, Count, [Output|Data])
end.
Thanks a lot for your help.

Answer 1

The following is just a comment and not an answer to your question which I believe rvirding provided above.
Since you are using one of the standard behaviours (gen_server) one would assume that you intend to write an OTP compliant application. If so you should never use a receive expression directly unless you are prepared to handle all possible system messages as well as your application's. In the case of a gen_server or gen_fsm non-system messages are processed by your handle_info/2 callback function. You could use the State variable to hold an indicator of what command you were processing (e.g. login) and have a separate clause for each:
handle_info({tcp,Socket,Reply}, #state{pending = login} = State) ->
...;
handle_info({tcp,Socket,Reply}, #state{pending = list} = State) ->
...;
... however that would become a poor man's finite state machine so you'd be better off porting it to a gen_fsm behaviour callback module where you have a separate state for each (i.e. wait_for_login). Then you can use handle_info/3:
handle_info({tcp,Socket,Reply}, wait_for_login, State) ->
...;
handle_info({tcp,Socket,Reply}, wait_for_list, State) ->
...;

Answer 2

If you are in active mode, the best approach consists in receiving the stream in your handle_info (or in your custom receiving function) using the pattern {tcp, Socket, Msg} and store it in a buffer until it does satisfy a specific pattern matching or it has a specific length, then you flush the buffer as you wish.
As #rvirding said you can't be sure that all your message will be received in a single packet, so you have to handle possible multiple packets. Otherwise you have to use passive mode and the function gen_tcp:recv/2 but keep in mind that this function is blocking.
I suggest you to read this: http://learnyousomeerlang.com/buckets-of-sockets