I've been reading about the different types of I/O and started wondering why operating systems don't just implement non-blocking I/O and emulate blocking I/O through busy-waiting.
One reason I thought of could be the waste of CPU cycles.
Any thoughts?
Related
The library of asyncio in Python, and generally, when we talk about asynchronous programming, I always think about doing “concurrent” I/O operations only on the level thread for optimized CPU use.
The library of asyncio has function of asyncio.sleep(seconds), but what disturb me was that sleep operation isn’t I/O operation, sleep operation is done on the kernel level with the CPU hardware without any external devices that can be counted as I/O [my definition for I/O is every hardware except from CPU and RAM].
So why does the asyncio lib (Asynchronous I/O) call this operation as an asynchronous I/O operation?
This is not a network interface controller we send requests to or the hard disk. I don’t have a problem with “concurrent” every operation we can on the level thread. However, the name of I/O in the end of the library makes me feel that it isn’t the proper terminology. I will be happy for clarification.
One more related question, does the terminology of asynchronous programming refer to “concurrent” I/O operations only or every operation, including CPU operations like x = x + 1 on the level thread? (I guess the last operation can be done “concurrently” on the level thread, but this will be unnecessary)
Link:
https://docs.python.org/3/library/asyncio.html
Code snippet:
import asyncio
async def main():
print('Hello ...')
await asyncio.sleep(1)
print('... World!')
asyncio.run(main())
Paraphrasing Wikipedia, "Asynchronous programming" generally refers to the occurrence of events outside of the main program flow and ways of handling such events. As such, asynchronous operations are not necessarily I/O ones.
These asynchronous events are generally handled at the hardware or OS level and it is important to understand that at this level almost anything is asynchronous: jobs are put into queues and scheduled by the OS, then they are regularly polled for completion by the OS which then notifies the main application that the job is done.
Such asynchronous events comprises:
Network requests (multiplexed and polled by the OS),
Timers (managed by hardware timers and interrupts),
Communication with various external devices such as keyboards (hardware interrupts),
Communication with internal devices such as the GPU (jobs are committed to command queues),
etc.
The purpose of the AsyncIO library is to allow the expression of asynchronous programs in a more "structured" and linear way. As such, it wraps many common asynchronous operations such as I/Os and timers into async-await equivalents. AsyncIO is thus not restricted to only asynchronous I/O operations and one can implement an AsyncIO async-await interface to support GPU for example.
To overlap MPI communications and computations, I am working on issuing asynchronous I/O (MPI calls) with user-defined computation function on the data from I/O.
MS-Window's 'Overlap' is not the friend of MPI (It supports for overlapped I/O only for File I/O and Socket communication, but not for MPI operations...)
I cannot find an appropriate MPI API for it, is there anyone with a glimpse on it?
There are no completion callbacks in MPI. Non-blocking operations always return a request handle that must be either be synchronously waited on using MPI_Wait and family or periodically tested using the non-blocking MPI_Test and family.
With the help of either MPI_Waitsome or MPI_Testsome, it is possible to implement a dispatch mechanism that monitors multiple requests and calls specific functions upon their completion. None of the MPI calls has any timeout characteristics though - it is either "wait forever" (MPI_Wait...) or "check without waiting" (MPI_Test...).
I have been though asynchronous I/O is always has a callback form. But recently I discovered some low level implementations are using polling style API.
kqueue
libpq
And this leads me to think that maybe all (or most) asynchronous I/O (any of file, socket, mach-port, etc.) is implemented in a kind of polling manner at last. Maybe the callback form is just an abstraction only for higher-level API.
This could be a silly question, but I don't know how actually most of asynchronous I/O implemented at low level. I just used the system level notifications, and when I see kqueue - which is the system notification, it's a polling style!
How should I understand asynchronous I/O at low-level? How the high-level asynchronous notification is being made from low-level polling system? (if it actually does)
At the lowest (or at least, lowest worth looking at) hardware level, asynchronous operations truly are asynchronous in modern operating systems.
For example, when you read a file from the disk, the operating system translates your call to read to a series of disk operations (seek to location, read blocks X through Y, etc.). On most modern OSes, these commands get written either to special registers, or special locations in main memory, and the disk controller is informed that there are operations pending. The operating system then goes on about its business, and when the disk controller has completed all of the operations assigned to it, it triggers an interrupt, causing the thread that requested the read to pickup where it left off.
Regardless of what type of low-level asynchronous operation you're looking at (disk I/O, network I/O, mouse and keyboard input, etc.), ultimately, there is some stage at which a command is dispatched to hardware, and the "callback" as it were is not executed until the hardware reaches out and informs the OS that it's done, usually in the form of an interrupt.
That's not to say that there aren't some asynchronous operations implemented using polling. One trivial (but naive and costly) way to implement any blocking operation asynchronously is just to spawn a thread that waits for the operation to complete (perhaps polling in a tight loop), and then call the callback when it's finished. Generally speaking, though, common asynchronous operations at the OS level are truly asynchronous.
It's also worth mentioning that just because an API is blocking doesn't mean it's polling: you can put a blocking API on an asynchronous operation, and a non-blocking API on a synchronous operation. With things like select and kqueues, for example, the thread actually just goes to sleep until something interesting happens. That "something interesting" comes in in the form of an interrupt (usually), and that's taken as an indication that the operating system should wake up the relevant threads to continue work. It doesn't just sit there in a tight loop waiting for something to happen.
There really is no way to tell whether a system uses polling or "real" callbacks (like interrupts) just from its API, but yes, there are asynchronous APIs that are truly backed by asynchronous operations.
I have read some lib as libev, all of them use the non-blocking io to handle the network communication. However, in which case the blocking io is used in networking?
For simple programs (e.g. test utility or dedicated client) or when dedicated threads are used.
In the first case there is no point in the extra logic involved with non-blocking I/O, while in the second case the logic is replaced with the logic involving multiple threads where the thread using blocking I/O essentially simulated a dedicated client (or server), this is done at the cost of additional resources for threads and synchronisation, but is often justified, especially when multiple threads are necessary anyway or where threads are cheap in terms of resources.
Non blocking I/O is often used in libraries and other cases where using dedicated threads cannot be justified, for resource, testability or portability concerns. It often boils down to individual taste. The difference often being a matter of style.
Looking at the processmodel element in the Web.Config there are two attributes.
maxWorkerThreads="25"
maxIoThreads="25"
What is the difference between worker threads and I/O threads?
Fundamentally not a lot, it's all about how ASP.NET and IIS allocate I/O wait objects and manage the contention and latency of communicating over the network and transferring data.
I/O threads are set aside as such because they will be doing I/O (as the name implies) and may have to wait for "long" periods of time (hundreds of milliseconds). They also can be optimized and used differently to take advantage of I/O completion port functionality in the Windows kernel. A single I/O thread may be managing multiple completion ports to maintain throughput.
Windows has a lot of capabilities for dealing with I/O blocking whereas ASP.NET/.NET has a plain concept of "Thread". ASP.NET can optimize for I/O by using more of the unmanaged threading capabilities in the OS. You wouldn't want to do this all the time for every thread as you lose a lot of capabilities that .NET gives you which is why there is a distinction between how the threads are intended to be used.
Worker threads are threads upon which regular "work" or just plain code/processing happens. Worker threads are unlikely to block a lot or wait on anything and will be short running and therefore require more aggressive scheduling to maximize processing power and throughput.
[Edit]: I also found this link which is particularly relevant to this question:
http://blogs.msdn.com/ericeil/archive/2008/06/20/windows-i-o-threads-vs-managed-i-o-threads.aspx
Just to add on to chadmyers...
Seems like I/O Threads was the old way ASP.NET serviced requests,
"Requests in IIS 5.0 are typically
serviced over I/O threads, or threads
performing asynchronous I/O because
requests are dispatched to the worker
process using asynchronous writes to a
named pipe."
with IIS6.0 this has changed.
"Thus all requests are now serviced by
worker threads drawn from the CLR
thread pool and never on I/O threads."
Source: http://msdn.microsoft.com/hi-in/magazine/cc164128(en-us).aspx