I'm trying to understand the idea of non-blocking web server and it seems like there is something I miss.
I can understand there are several reasons for "block" web request(psuedocode):
CPU bound
string on_request(arg)
{
DO_SOME_HEAVY_CPU_CALC
return "done";
}
IO bound
string on_request(arg)
{
DO_A_CALL_TO_EXTERNAL_RESOURCE_SUCH_AS_WEB_IO
return "done";
}
sleep
string on_request(arg)
{
sleep(VERY_VERY_LONG_TIME);
return "done";
}
are all the three can benefit from non-blocking server?
how the situation that do benefit from the non-blocking web server really do that?
I mean, when looking at the Tornado server documentation, it seems
like it "free" the thread. I know that a thread can be put to sleep
and wait for a signal from the operation system (at least in Linux),
is this the meaning of "freeing" the thread? is this some higher
level implementation? something that actually create a new thread
that is waiting for new request instead of the "sleeping" one?
Am I missing something here?
Thanks
Basically the way the non-blocking sockets I/O work is by using polling and the state machine. So your scheme for many connections would be something like that:
Create many sockets and make them nonblocking
Switch the state of them to "connect"
Initiate the connect operation on each of them
Poll all of them until some events fire up
Process the fired up events (connection established or connection failed)
Switch the state those established to "sending"
Prepare the Web request in a buffer
Poll "sending" sockets for WRITE operation
send the data for those who got the WRITE event set
For those which have all the data sent, switch the state to "receiving"
Poll "receiving" sockets for READ operation
For those which have the READ event set, perform read and process the read data according to the protocol
Repeat if the protocol is bidirectional, or close the socket if it is not
Of course, at each stage you need to handle errors, and that the state of each socket is different (one may be connecting while another may be already reading).
Regarding polling I have posted an article about how different polling methods work here: http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system-architects/ - I suggest you check it.
To benefit from a non-blocking server, your code must also be non-blocking - you can't just run blocking code on a non-blocking server and expect better performance. For example, you must remove all calls to sleep() and replace them with non-blocking equivalents like IOLoop.add_timeout (which in turn involves restructuring your code to use callbacks or coroutines).
How To Use Linux epoll with Python http://scotdoyle.com/python-epoll-howto.html may give you some points about this topic.
Related
I am implementing a hub/servers MPI application. Each of the servers can get tied up waiting for some data, then they do an MPI Send to the hub. It is relatively simple for me to have the hub waiting around doing a Recv from ANY_SOURCE. The hub can get busy working with the data. What I'm worried about is skipping data from one of the servers. How likely is this scenario:
server 1 and 2 do Send's
hub does Recv and ends up getting data from server 1
while hub busy, server 1 gets more data, does another Send
when hub does its next Recv, it gets the more recent server 1 data rather than the older server2
I don't need a guarantee that the order the Send's occur is the order the ANY_SOURCE processes them (though it would be nice), but if I new in practice it will be close to the order they are sent, I may go with the above. However if it is likely I could skip over data from one of the servers, I need to implement something more complicated. Which I think would be this pattern:
servers each do Send's
hub does an Irecv for each server
hub does a Waitany on all server requests
upon completion of one server request, hub does a Test on all the others
of all the Irecv's that have completed, hub selects the oldest server data (there is timing tag in the server data)
hub communicates with the server it just chose, has it start a new Send, hub a new Irecv
This requires more complex code, and my first effort crashed inside the Waitany call in a way that I'm finding difficult to debug. I am using the Python bindings mpi4py - so I have less control over buffers being used.
It is guaranteed by the MPI standard that the messages are received in the order they are sent (non-overtaking messages). See also this answer to a similar question.
However, there is no guarantee of fairness when receiving from ANY_SOURCE and when there are distinct senders. So yes, it is the responsibility of the programmers to design their own fairness system if the application requires it.
I am writing a Client/Server application in C++ with the help of Boost Asio. I have a working server, and the server workflow is something I understand well.
My client application handles the connect gracefully as shown in Asio examples, after which, it exchanges a handshake with the server. After that however, the users should be able to send requests to the server when and how they want, which is where I have a problem understanding the paradigm.
The initial workflow goes like a little like this:
OnConnected() { SendHandshake() }
SendHandshake() { async.write_some(handshake...), async_read_some(&OnRead) }
OnRead() { ReadServerHandshake() *** }
And users would send messages by using Write(msg):
Write (msg) { async_write_some(msg,&OnWrite), async_Read_some(&OnRead) }
OnWrite() {}
EDIT: Rephrasing the question to be clearer, here is the scenario:
After the initial handshaking is complete, the Client is only used to send requests to the server, on which it will get a reply. So, for instance, a user sends a write. Client waits for the read operation to complete, reads the reply and does something with it. The next user write will only come after, say, 5 minutes. Will the io_service stop working in the meanwhile because there are no outstanding asynchronous operations in between the last reply read and the next write?
On an informative note, you can provide it with io_service::work to stop an io_service from running out of work. This will ensure that the io_service::run never returns until the work object is destroyed.
To control the lifetime of the work object, you can use a shared_ptr pointer and reset it once the work is done, or you can use boost::optional as outlined here.
Of course you still need to handle the case where either the server closes the TCP connection, or the connection dies for whatever reason. To handle this case, one solution would be to have an outstanding async_read on the socket to the server. The read handler should be called with an error_code when/if something goes wrong with the connection. If you have the outstanding read on the connection, you do not need to use the work object.
If you want the IO service to complete a read, you must start a read. If you want to read data any time the client sends it, you must have an asynchronous read operation pending at all times. Otherwise, how would the library know what to do with the data?
I have written a multithreaded application in C. I have two threads created, one for catching all the signals and another for accept()-ing client connections. When I kill the appilcation using killproc, the thread with the accept call is not interrupted. How can I fix that?
The code looks like:
int stop_exec=0;
sigCatcherThread()
{
int sig
sigset_t allsignals;
sigfillset(allsignals);
do{
sigwait(&allsignals, &sig);
if(sig==SIGTERM)
stop_exec=1;
}while(!stop_exec)
}
clientHandler()
{
...
while(!stop_exec)
{
accept(...);
}
main()
{
pthread_create(..., sigCatcherThread,..);
pthread_create(..., clientHandler,...);
}
Here you see the use of interrupted system calls. But the convenience of a signal handling thread is probably higher than the use of interrupted systems calls.
So you need you client handler to block until it can accept an incoming connection or the signal occurs. Waiting for potential input means either signal driven IO -- a path I wouldn't follow -- or select(2) (or pool). But select(2) can wait only on IO. So transform your signal occurrence in IO: open a pipe, have your signal handling thread write to the pipe when SIGQUIT occurs and have your client thread select(2) for the socket and the other end of the pipe.
Only one thread receives a signal targeted to a process. So, it must be not the thread blocked on accept(). See signal concepts for more details.
As already mentioned here, you should probably be using an event loop based on select(). I would suggest using libevent.
There's no need to interrupt the blocking accept call. Just make sure that if the thread does return from accept, say by receiving an actual connection, it won't do anything harmful.
If there's some specific reason you need the accept call to interrupt, explain what it is. Likely there's a simple way to remove the requirement.
What I am trying to solve: have an Erlang TCP server that listens on a specific port (the code should reside in some kind of external facing interface/API) and each incoming connection should be handled by a gen_server (that is even the gen_tcp:accept should be coded inside the gen_server), but I don't actually want to initially spawn a predefined number of processes that accepts an incoming connection). Is that somehow possible ?
Basic Procedure
You should have one static process (implemented as a gen_server or a custom process) that performs the following procedure:
Listens for incoming connections using gen_tcp:accept/1
Every time it returns a connection, tell a supervisor to spawn of a worker process (e.g. another gen_server process)
Get the pid for this process
Call gen_tcp:controlling_process/2 with the newly returned socket and that pid
Send the socket to that process
Note: You must do it in that order, otherwise the new process might use the socket before ownership has been handed over. If this is not done, the old process might get messages related to the socket when the new process has already taken over, resulting in dropped or mishandled packets.
The listening process should only have one responsibility, and that is spawning of workers for new connections. This process will block when calling gen_tcp:accept/1, which is fine because the started workers will handle ongoing connections concurrently. Blocking on accept ensure the quickest response time when new connections are initiated. If the process needs to do other things in-between, gen_tcp:accept/2 could be used with other actions interleaved between timeouts.
Scaling
You can have multiple processes waiting with gen_tcp:accept/1 on a single listening socket, further increasing concurrency and minimizing accept latency.
Another optimization would be to pre-start some socket workers to further minimize latency after accepting the new socket.
Third and final, would be to make your processes more lightweight by implementing the OTP design principles in your own custom processes using proc_lib (more info). However, this you should only do if you benchmark and come to the conclusion that it is the gen_server behavior that slows you down.
The issue with gen_tcp:accept is that it blocks, so if you call it within a gen_server, you block the server from receiving other messages. You can try to avoid this by passing a timeout but that ultimately amounts to a form of polling which is best avoided. Instead, you might try Kevin Smith's gen_nb_server instead; it uses an internal undocumented function prim_inet:async_accept and other prim_inet functions to avoid blocking.
You might want to check out http://github.com/oscarh/gen_tcpd and use the handle_connection function to convert the process you get to a gen_server.
You should use "prim_inet:async_accept(Listen_socket, -1)" as said by Steve.
Now the incoming connection would be accepted by your handle_info callback
(assuming you interface is also a gen_server) as you have used an asynchronous
accept call.
On accepting the connection you can spawn another ger_server(I would recommend
gen_fsm) and make that as the "controlling process" by calling
"gen_tcp:controlling_process(CliSocket, Pid of spwned process)".
After this all the data from socket would be received by that process
rather than by your interface code. Like that a new controlling process
would be spawned for another connection.
As per my understanding a simple TCP server will be coded as follows.
socket() - bind() - listen() - accept() - read() - write()
The clients will be written as follows.
socket() - bind()(Optional) - connect() - write() - read()
Please note the order difference in read() and write() calls between client and server program.
Is it a requirement to always read() before write() in a server program and if, then why?
Thanks,
Naga
That isn't mandatory, but it makes sense for the server to read the request before writing a response. Note that it is necessary to read on both sides often enough to prevent a distributed deadlock: for example, if the both sides are trying to write and not reading, then the buffers in-between will get full and neither one's write will be able to proceed. One solution for this is to have a separate thread which keeps reading, if there is something to read (this applies to both the client and the server).
The simple answer is no. You are free to do whatever you like.
However, I'll caveat that quickly with the fact that most protocols are designed to wait for the client to send something. After all, the server, by nature, serves requests and needs to wait to know what that request is, be it "GET /" or "HELO" or whatever. So, it is fairly natural for a sever to read before writing any response back to the client.
That said, you could if you felt like it dump version information down to the client before you do any reading. To see the effect, connect to your server using telnet.
You can perform them in either order. However, a server will normally generate a response from the read() operation, then write it with the write() operation, so this order makes sense.
If you're handling multiple clients, you should use a multiplexer like select to notify you when clients have data ready to read, so your server won't lock up the every time you try to read() from a client who hasn't sent anything.
It isn't a requirement, server program can write to socket without reading first. But in many cases server program must know what client wants - so it calls read() first.