All port operations in Rebol 3 are asynchronous. The only way I can find to do synchronous communication is calling wait.
But the problem with calling wait in this case is that it will check events for all open ports (even if they are not in the port block passed to wait). Then they call their responding event handlers, but a read/write could be done in one of those event handlers. That could result in recursive calls to "wait".
How do I get around this?
Why don´t you create a kind of "Buffer" function to receive all messages from assyncronous entries and process them as FIFO (first-in, first-out)?
This way you may keep the Assync characteristics of your ports and process them in sync mode.
in cases where there are only asynchronous events and we are in need on synchronous reply, start a timer or sleep for timeout, if the handler or required objective is met then say true, else false and make sure the event gets cancelled /reset for the same if critical.
I think that there are 2 design problems (maybe intrinsic to the tools / solutions at hand).
Wait is doing too much - it will check events for all open ports. In a sound environment, waiting should be implemented only where it is needed: per device, per port, per socket... Creating unnecessary inter-dependencies between shared resources cannot end well - especially knowing that shared resources (even without inter-dependencies) can create a lot of problems.
The event handlers may do too much. An event handler should be as short as possible, and it should only handle the event. If is does more, then the handler is doing too much - especially if involves other shared resources. In many situations, the handler just saves the data which will be lost otherwise; and an asynchronous job will do the more complex things.
You can just use a lock. Cummunication1 can set some global lock state i.e. with a variable (be sure that it's thread safe). locked = true. Then Communication2 can wait until it's unlocked.
loop do
sleep 10ms
break if not locked
end
locked = true
handle_communication()
Related
I know how a process can respond to signals that were sent to it specifically (e.g. SIGINT, SIGTERM, SIGUSR2, etc.). But can a process be notified of signals that were sent to a different process?
Not in standard Unix or POSIX, you cannot be notified for signals sent to another process. See signal(7) and signal-safety(7).
However, waitpid(2) and friends can tell you if a child process has terminated with a signal. And killpg(2) sends a signal to a process group (and kill(2) does also that with a negative target pid). And getrusage(2) can count signals (recieved by some other process). You could also use proc(5) to query information about other processes. And you might use signalfd(2) or ptrace(2) etc....
Signals are a very limited and poor form of inter-process communication. There are better ways.
BTW, sigaction(2) can be used with SA_SIGINFO and then your handler gets a pointer to siginfo_t and another to ucontext_t so you get a lot of information.
Notice that process groups and sessions are related. See also setpgid(2), setsid(2), credentials(7) and also related to terminals and pseudo-ttys (read the tty demystified and about job control).
I guess that your other question is about these. But you don't mention any of them there.
If I call WinUSB_AbortPipe() just as WinUSB_ReadPipe() starts, I get into a deadlock state. I ran the debug trace log that is provided here. Below is the last 5 lines in the log where the problem occurs. I think ReadPipe must have missed the signal, and AbortPipe is waiting for ReadPipe to complete.
[0]4E34.4B58::06/09/2015-15:42:12.528 - IOCTL_WINUSB_READ_PIPE
[0]4E34.4B58::06/09/2015-15:42:12.528 - PIPE129: (00000019) The read has been added to the raw io queue
[0]4E34.4B58::06/09/2015-15:42:12.528 - PIPE129: (00000019) The read is being handled
[2]4E34.4ECC::06/09/2015-15:42:12.529 - IOCTL_WINUSB_ABORT_PIPE
[2]4E34.4B58::06/09/2015-15:42:12.529 - PIPE129: (00000019) Reading 64 bytes from the device
In my design, I have the IN endpoints read asynchronously into buffers. I found that it is best to set the timeout of the read operation to infinite because the driver hates it when I cause STALLs to occur (ran into other issues with that). So I need to have the disconnect sequence cause the threads to wake up to realize that we need to close. Is there any way to safely do that?
My workaround for this is to instead call WinUsb_ResetPipe(). This causes WinUSB_ReadPipe() to unblock, and doesn't seem to lock up as WinUSB_AbortPipe() sometimes does. The only evidence that I have that this works is through successfully running tests over several hours, so I can't guarantee that this is a solution.
I'm trying to understand the idea of non-blocking web server and it seems like there is something I miss.
I can understand there are several reasons for "block" web request(psuedocode):
CPU bound
string on_request(arg)
{
DO_SOME_HEAVY_CPU_CALC
return "done";
}
IO bound
string on_request(arg)
{
DO_A_CALL_TO_EXTERNAL_RESOURCE_SUCH_AS_WEB_IO
return "done";
}
sleep
string on_request(arg)
{
sleep(VERY_VERY_LONG_TIME);
return "done";
}
are all the three can benefit from non-blocking server?
how the situation that do benefit from the non-blocking web server really do that?
I mean, when looking at the Tornado server documentation, it seems
like it "free" the thread. I know that a thread can be put to sleep
and wait for a signal from the operation system (at least in Linux),
is this the meaning of "freeing" the thread? is this some higher
level implementation? something that actually create a new thread
that is waiting for new request instead of the "sleeping" one?
Am I missing something here?
Thanks
Basically the way the non-blocking sockets I/O work is by using polling and the state machine. So your scheme for many connections would be something like that:
Create many sockets and make them nonblocking
Switch the state of them to "connect"
Initiate the connect operation on each of them
Poll all of them until some events fire up
Process the fired up events (connection established or connection failed)
Switch the state those established to "sending"
Prepare the Web request in a buffer
Poll "sending" sockets for WRITE operation
send the data for those who got the WRITE event set
For those which have all the data sent, switch the state to "receiving"
Poll "receiving" sockets for READ operation
For those which have the READ event set, perform read and process the read data according to the protocol
Repeat if the protocol is bidirectional, or close the socket if it is not
Of course, at each stage you need to handle errors, and that the state of each socket is different (one may be connecting while another may be already reading).
Regarding polling I have posted an article about how different polling methods work here: http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system-architects/ - I suggest you check it.
To benefit from a non-blocking server, your code must also be non-blocking - you can't just run blocking code on a non-blocking server and expect better performance. For example, you must remove all calls to sleep() and replace them with non-blocking equivalents like IOLoop.add_timeout (which in turn involves restructuring your code to use callbacks or coroutines).
How To Use Linux epoll with Python http://scotdoyle.com/python-epoll-howto.html may give you some points about this topic.
I have the following question: can I use a signal handler for SIGCHLD and at specific places use waitpid(3) instead?
Here is my scenario: I start a daemon process that listens on a socket (at this point it's irrelevant if it's a TCP or a UNIX socket). Each time a client connects, the daemon forks a child to handle the request and the parent process keeps on accepting incoming connections. The child handling the request needs at some point to execute a command on the server; let's assume in our example that it needs to perform a copy like this:
cp -a /src/folder /dst/folder
In order to do so, the clild forks a new process that uses execl(3) (or execve(3), etc.) to execute the copy command.
In order to control my code better, I would ideally wish to catch the exit status of the child executing the copy with waitpid(3). Moreover, since my daemon process is forking children to handle requests, I need to have a signal handler for SIGCHLD so as to prevent zombie processes from being created.
In my code, I setup a signal handler for SIGCHLD using signal(3), I daemonize my program by forking twice, then I listen on my socket for incoming connections, I fork a process to handle each coming request and my child-process forks a grand-child-process to perform the copy, trying to catch its exit status via waitpid(3).
What happens is that SIGCHLD is caught by my handler when a grand-child-process dies, before waitpid(3) takes action and waitpid(3) returns -1 even though the grand-child-process exits with success.
My first thought was to add:
signal(SIGCHLD, SIG_DFL);
just before forking the child process to handle my connecting clients, without any success. Using SIG_IGN didn't work either.
Is there a suggestion on how to make my scenario work?
Thank you all for your help in advance!
PS. If you need code, I'll post it, but due to its size I decided to do so only if necessary.
PS2. My intention is to use my code in FreeBSD, but my checks are performed in Linux.
EDIT [SOLVED]:
The problem I was facing is solved. The "unexpected" behaviour was caused by my waitpid(3) handling code which was buggy at some point.
Hence, the above method can indeed be used to allow for signal(3) and waitpid(3) coexistence in daemon-like programs.
Thanx for your help and I hope that this method helps someone wishing to accomplish such a thing!
I plan to use MPI to build a solver that supports asynchronous communication. The basic idea is as follows.
Assume there are two parallel processes. Process 1 wants to send good solutions it finds periodically to process 2, and ask for good solutions from process 2 when it needs diversification.
At some point, process 1 uses MPI_send to send a solution to process 2. How to guarantee there is an MPI_Rev matching this MPI_Send, since this send is triggered dynamically?
When process 1 needs a solution, how can it send a request to process 2, and process 2 will notice its request in time?
There are three ways to achieve what you want, although it is not truly asynchronous communication.
1) Use non-blocking send/recvs. Replace your send/recv calls with irecv/isend and wait. The sender can issue an isend and continue working on the next problem. At some point, you will have to issue a mpi-wait to make sure your previous send was received. Your process2 can issue a recv ahead of time using irecv and continue doing its work. Again, at some point you will call mpi-wait to make sure your irecv was received. this may be a bit cumbersome if I understand you requirement correctly.
2) A Elegant way would be to use One-Sided communication. MPI_Put, Get.
3) Restructure your algorithm in such a way that at certain intervals of time, process 1 & 2 exchange information and state.
Depending on the nature of the MPI_* function you call, the send will block until a matching receive has been called by another process, so you need to make sure that's going to happen in your code.
There are also non-blocking function calls MPI_Isend f.ex, which gives you a request-handle which you can check on later to see if the process' send has been received by a matching receive.
Regarding your issue, you could issue a non-blocking receive (MPI_Irecv being the most basic) and check on the status every n seconds depending on your application. The status will then be set to complete when a message has been received and is ready to be read.
If it's time sensitive, use a blocking call while waiting for a message. The blocking mechanism (in OpenMPI at least) uses a spinning poll however, so the waiting process will be eating 100% cpu.