I have a tokio-based single-threaded async app where using Arcs or other Sync types seems to be an overhead. Because there is no need for synchronization between threads, I am looking for something like tokio::sync::oneshot::channel, Sender and Receiver of which should be !Sync and could be wrapped into Rc instead of Arc.
Are there any specially crafted synchronization primitives for usage in
single-threaded async apps in Rust?
You can take a look at the various Local types in futures-intrusive. E.g. the LocalOneshotChannel requires no mutex.
Related
The library of asyncio in Python, and generally, when we talk about asynchronous programming, I always think about doing “concurrent” I/O operations only on the level thread for optimized CPU use.
The library of asyncio has function of asyncio.sleep(seconds), but what disturb me was that sleep operation isn’t I/O operation, sleep operation is done on the kernel level with the CPU hardware without any external devices that can be counted as I/O [my definition for I/O is every hardware except from CPU and RAM].
So why does the asyncio lib (Asynchronous I/O) call this operation as an asynchronous I/O operation?
This is not a network interface controller we send requests to or the hard disk. I don’t have a problem with “concurrent” every operation we can on the level thread. However, the name of I/O in the end of the library makes me feel that it isn’t the proper terminology. I will be happy for clarification.
One more related question, does the terminology of asynchronous programming refer to “concurrent” I/O operations only or every operation, including CPU operations like x = x + 1 on the level thread? (I guess the last operation can be done “concurrently” on the level thread, but this will be unnecessary)
Link:
https://docs.python.org/3/library/asyncio.html
Code snippet:
import asyncio
async def main():
print('Hello ...')
await asyncio.sleep(1)
print('... World!')
asyncio.run(main())
Paraphrasing Wikipedia, "Asynchronous programming" generally refers to the occurrence of events outside of the main program flow and ways of handling such events. As such, asynchronous operations are not necessarily I/O ones.
These asynchronous events are generally handled at the hardware or OS level and it is important to understand that at this level almost anything is asynchronous: jobs are put into queues and scheduled by the OS, then they are regularly polled for completion by the OS which then notifies the main application that the job is done.
Such asynchronous events comprises:
Network requests (multiplexed and polled by the OS),
Timers (managed by hardware timers and interrupts),
Communication with various external devices such as keyboards (hardware interrupts),
Communication with internal devices such as the GPU (jobs are committed to command queues),
etc.
The purpose of the AsyncIO library is to allow the expression of asynchronous programs in a more "structured" and linear way. As such, it wraps many common asynchronous operations such as I/Os and timers into async-await equivalents. AsyncIO is thus not restricted to only asynchronous I/O operations and one can implement an AsyncIO async-await interface to support GPU for example.
In my understanding, asynchronous can only handle I/O intensive tasks such as reading and writing sockets or files, but can do nothing with CPU intensive tasks such as encryption and compression.
So in Rust Tokio Runtime, I think only need to use spawn_blocking to handle CPU intensive tasks. But I have seen this repo, the example is
#[tokio_02::main]
async fn main() -> Result<()> {
let data = b"example";
let compressed_data = compress(data).await?;
let de_compressed_data = decompress(&compressed_data).await?;
assert_eq!(de_compressed_data, data);
println!("{:?}", String::from_utf8(de_compressed_data).unwrap());
Ok(())
}
This library crates adaptors between compression and async I/O types.
I have 3 questions:
What is the purpose of awaiting compress/decompress?
Are these adaptors necessary or my understanding of asynchrony wrong?
Can I do compression operations directly in Tokio multithreaded runtime? Like this
async fn foo() {
let mut buf = ...;
compress_sync(&mut buf);
async_tcp_stream.write(buf).await;
}
In my understanding, asynchronous can only handle I/O intensive tasks such as reading and writing sockets or files, but can do nothing with CPU intensive tasks such as encryption and compression.
This is misguided. Asynchronous constructs are designed to work with non-blocking operations, irrespective of what kinds of resources are involved underneath. For instance, a future which delegates computation to a separate thread would be a valid use of async/await.
With that said, the reason why asynchronous compression is useful is because they expose I/O adaptors which also work in asynchronous programming. Even though these compression algorithms are primarily CPU-bound, they work in bulks which are fed from an arbitrary reader or writer, meaning that the process may have to wait for I/O to be conducted.
See for example the documentation of bufread::ZstdDecoder
This structure implements an AsyncRead interface and will read compressed data from an underlying stream and emit a stream of uncompressed data.
This is something which you do not get with synchronous byte source adapters such as flate2::bufread::GzDecoder. Even if you just use a compress function, a synchronous version of it would have blocked while waiting for the possibility to read or write data.
See also:
What is the purpose of async/await in Rust?
Using synchronous file-IO library in asynchronous code
I'm trying to convince a coworker of the benefits of using the Session#executeAsync.
However, since we are using the driver from Scala, it would be rather easy to wrap the sync call Session#execute in a Future and that would be all to transform it in an async call. This will be already an improvement because it will give us the opportunity of avoid blocking the current thread (in our case that would represent blocking the threads that handles http requests in play with a huge impact on the number of requests that can be handled concurrently)
I argue that if the work needed to implement an async driver will be wrap it in a Future it won't exist implementations like ReactiveMongo an the Async Api for Cassandra from Datastax.
So,
What are the benefits of using the async api?
How is the async api implemented in Datastax driver and it what libraries and OS features relies on?
What kind of problems were to be solved beyond the asynchronous networks calls? (I mean, implement the async driver must be more than just using java nio)
How is the async api implemented in Datastax driver and it what libraries and OS features relies on?
Datastax java driver based on Netty networking framework. Netty itself based on Event Driven model. Also for some operating systems Netty provides native transports to improve performance e.g. epoll for Linux.
What are the benefits of using the async api?
I'm not a Scala expert but as I know Scala Future based on Threads model (Execution contexts). It means you need to submit a request to another thread to execute the request asynchronously. For IO tasks you just need request another system and wait response from this system. If you have a big number of requests, all threads in your pool will be busy but will not do anything useful. Thread is a fairly expensive resource and it can be a problem to have thousands threads in the same physical resource. Threads are good for parallel calculation tasks but not for IO tasks.
From other hand Datastax java driver based on Event Driven model (Netty). It means the each request will be submitted in event loop queue. For each iteration of event loop, Netty will define the state of request and will execute handlers associated with this request.
This approach avoids of memory usage overhead for threads and allows you to perform thousands of IO requests in the same time. But in this case you should define slow or blocking request callbacks in another thread to avoid blocking of event-loop.
I understood that .net know to use multiple threads for multiple requests.
So, if probably our service wont get more request than the number of threads our server can produce (it look like huge number), the only reason I can see to use async is on single request that do multiple blocking operations which can done in parallel.
Am I right?
Another advantage may be that serve multiple requests with same thread is cheaper than use multiple threads. How significant is this difference?
(note: no UI exists in our service (I saw that there is single thread for this, but it isn't relevant))
thanks!
Am I right?
No, doing multiple independent blocking operations, is the job of Concurrent APIs anyway (though sometimes they need Synchronization (like lock, mutex) to maintain the object state and avoid Race condition), but the usage of Async-Await is to schedule the IO Operations, like File Read / Write, call a remote service or Database Read / Write, which doesn't need a thread, as they are queued on a queue in hardware called IO Completion ports.
Benefits of Async-Await:
Doesn't start a IO operation on a separate Thread, since Thread is a costly resource, in terms memory and resource allocation and would do little precious than wait for IO call to come back. Separate thread shall be used for the compute bound operations, no IO bound.
Free up the UI / caller thread to make it completely responsive to carry out other tasks / operations
This is the evolution of Asynchronous programming model (BeginXX, EndXX), which was fairly complex to understand and implement
Another advantage may be that serve multiple requests with same thread is cheaper than use multiple threads. How significant is this difference?
Its a good strategy depending on the kind of request from caller, if they are compute bound better invoke a Parallel API and finish them fast, IO bound there's Async-Await, only issue with multiple threads is Resource allocation and Context switching, which needs to be factored in, but on other end it efficiently utilize the processor cores, which are fairly under utilized in the current day systems, as you would see most of the time processor is lying idle
I have been though asynchronous I/O is always has a callback form. But recently I discovered some low level implementations are using polling style API.
kqueue
libpq
And this leads me to think that maybe all (or most) asynchronous I/O (any of file, socket, mach-port, etc.) is implemented in a kind of polling manner at last. Maybe the callback form is just an abstraction only for higher-level API.
This could be a silly question, but I don't know how actually most of asynchronous I/O implemented at low level. I just used the system level notifications, and when I see kqueue - which is the system notification, it's a polling style!
How should I understand asynchronous I/O at low-level? How the high-level asynchronous notification is being made from low-level polling system? (if it actually does)
At the lowest (or at least, lowest worth looking at) hardware level, asynchronous operations truly are asynchronous in modern operating systems.
For example, when you read a file from the disk, the operating system translates your call to read to a series of disk operations (seek to location, read blocks X through Y, etc.). On most modern OSes, these commands get written either to special registers, or special locations in main memory, and the disk controller is informed that there are operations pending. The operating system then goes on about its business, and when the disk controller has completed all of the operations assigned to it, it triggers an interrupt, causing the thread that requested the read to pickup where it left off.
Regardless of what type of low-level asynchronous operation you're looking at (disk I/O, network I/O, mouse and keyboard input, etc.), ultimately, there is some stage at which a command is dispatched to hardware, and the "callback" as it were is not executed until the hardware reaches out and informs the OS that it's done, usually in the form of an interrupt.
That's not to say that there aren't some asynchronous operations implemented using polling. One trivial (but naive and costly) way to implement any blocking operation asynchronously is just to spawn a thread that waits for the operation to complete (perhaps polling in a tight loop), and then call the callback when it's finished. Generally speaking, though, common asynchronous operations at the OS level are truly asynchronous.
It's also worth mentioning that just because an API is blocking doesn't mean it's polling: you can put a blocking API on an asynchronous operation, and a non-blocking API on a synchronous operation. With things like select and kqueues, for example, the thread actually just goes to sleep until something interesting happens. That "something interesting" comes in in the form of an interrupt (usually), and that's taken as an indication that the operating system should wake up the relevant threads to continue work. It doesn't just sit there in a tight loop waiting for something to happen.
There really is no way to tell whether a system uses polling or "real" callbacks (like interrupts) just from its API, but yes, there are asynchronous APIs that are truly backed by asynchronous operations.