I would like to replicate the behavior and ergonomics of taking a closure/function as an argument much like map does: iterator.map(|x| ...).
I've noticed that some library code allows passing in async functionality, but this method doesn't allow me to pass in arguments:
pub fn spawn<F, T>(future: F) -> JoinHandle<T>
where
F: Future<Output = T> + Send + 'static,
T: Send + 'static,
spawn(async { foo().await });
I'm hoping to do one of the following:
iterator.map(async |x| {...});
async fn a(x: _) {}
iterator.map(a)
async functions are effectively desugared as returning impl Future. Once you know that, it's a matter of combining existing Rust techniques to accept a function / closure, resulting in a function with two generic types:
use std::future::Future;
async fn example<F, Fut>(f: F)
where
F: FnOnce(i32, i32) -> Fut,
Fut: Future<Output = bool>,
{
f(1, 2).await;
}
This can also be written as
use std::future::Future;
async fn example<Fut>(f: impl FnOnce(i32, i32) -> Fut)
where
Fut: Future<Output = bool>,
{
f(1, 2).await;
}
How do you pass a Rust function as a parameter?
What is the concrete type of a future returned from `async fn`?
What is the purpose of async/await in Rust?
How can I store an async function in a struct and call it from a struct instance?
What is the difference between `|_| async move {}` and `async move |_| {}`
The async |...| expr closure syntax is available on the nightly channel enabling the feature async_closure.
#![feature(async_closure)]
use futures::future;
use futures::Future;
use tokio;
pub struct Bar;
impl Bar {
pub fn map<F, T>(&self, f: F)
where
F: Fn(i32) -> T,
T: Future<Output = Result<i32, i32>> + Send + 'static,
{
tokio::spawn(f(1));
}
}
async fn foo(x: i32) -> Result<i32, i32> {
println!("running foo");
future::ok::<i32, i32>(x).await
}
#[tokio::main]
async fn main() {
let bar = Bar;
let x = 1;
bar.map(foo);
bar.map(async move |x| {
println!("hello from async closure.");
future::ok::<i32, i32>(x).await
});
}
See the 2394-async_await RFC for more detalis
Related
Part 1: What should be the signature of a function returning an async function?
pub async fn some_async_func(arg: &str) {}
// What should be sig here?
pub fn higher_order_func(action: &str) -> ???
{
some_async_func
}
Part 2: What should be the sig, if based on the action parameter, higher_order_func had to return either async_func1 or async_func2.
I am also interested in learning the performance tradeoffs if there are multiple solutions. Please note that I'd like to return the function itself as an fn pointer or an Fn* trait, and not the result of invoking it.
Returning a function
Returning the actual function pointer requires heap allocation and a wrapper:
use std::future::Future;
use std::pin::Pin;
pub async fn some_async_func(arg: &str) {}
pub fn some_async_func_wrapper<'a>(arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
Box::pin(some_async_func(arg))
}
pub fn higher_order_func<'a>(action: &str)
-> fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
some_async_func_wrapper
}
Why boxing? higher_order_func needs to have a concrete return type, which is a function pointer. The pointed function needs to also have a concrete return type, which is impossible for async function since it returns opaque type. In theory, it could be possible to write return type as fn(&'a str) -> impl Future<Output=()> + 'a, but this would require much more guesswork from the compiler and currently is not supported.
If you are OK with Fn instead of fn, you can get rid of the wrapper:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> impl Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
|arg: &'a str| {
Box::pin(some_async_func(arg))
}
}
To return a different function based on action value, you will need to box the closure itself, which is one more heap allocation:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> Box<dyn Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>>
{
if action.starts_with("one") {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_one(arg))
})
} else {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_two(arg))
})
}
}
Alternative: returning a future
To simplify things, consider returning a future itself instead of a function pointer. This is virtually the same, but much nicer and does not require heap allocation:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> impl Future<Output=()> + 'a
{
some_async_func(arg)
}
It might look like, when higher_order_func_future is called, some_async_func is getting executed - but this is not the case. Because of the way async functions work, when you call some_async_func, no user code is getting executed. The function call returns a Future: the actual function body will be executed only when someone awaits the returned future.
You can use the new function almost the same way as the previous function:
// With higher order function returning function pointer
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func(action)(arg).await;
}
// With higher order function returning future
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func_future(action, arg).await;
}
Notice, once more, that in both cases the actual some_async_func body is executed only when the future is awaited.
If you wanted to be able to call different async functions based on action value, you need boxing again:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
if action.starts_with("one") {
Box::pin(some_async_func_one(arg))
} else {
Box::pin(some_async_func_two(arg))
}
}
Still, this is just one heap allocation, so I strongly advise returning a future. The only scenario that I can imagine where the previous solution is better is when you want to save the boxed closure somewhere and use it many times. In this case, excessive allocation happens only once, and you spare some CPU time by dispatching the call based on action only once - when you make the closure.
Ideally, what you'd want is a nested impl trait: -> impl Fn(&str) -> impl Future<Output = ()>. But nested impl trait is not supported. However, you can emulate that using a trait.
The idea is to define a trait that will abstract over the notion of "function returning a future". If our function would take a u32, for example, it could look like:
trait AsyncFn: Fn(u32) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<F, Fut> AsyncFn for F
where
F: Fn(u32) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we would take impl AsyncFn. Trying to apply that naively to &str doesn't work:
error[E0308]: mismatched types
--> src/lib.rs:16:27
|
16 | fn higher_order_func() -> impl AsyncFn {
| ^^^^^^^^^^^^ one type is more general than the other
|
= note: expected associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
found associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
The error may look very strange, but it arises from the fact that async fn returns a future bound by the lifetime of all of its argument, i.e. for a signature async fn foo<'a>(arg: &'a str), the future is not impl Future<Output = ()> but impl Future<Output = ()> + 'a. There is a way to capture this relationship in our trait, we just need to make it generic over the argument and use HRTB:
trait AsyncFn<Arg>: Fn(Arg) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<Arg, F, Fut> AsyncFn<Arg> for F
where
F: Fn(Arg) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we specify the type as:
fn higher_order_func() -> impl for<'a> AsyncFn<&'a str> {
some_async_func
}
In addition to the great accepted answer, depending on your use case it's also possible to "fake" the higher order function and avoid any heap allocations by using a simple macro to expand the wrapper code in-place instead:
pub async fn some_async_func(arg: &str) {}
macro_rules! higher_order_func {
($action: expr) => {
some_async_func
};
}
fn main() {
let future = higher_order_func!("action")("arg");
}
When trying to write a function that receives a async closure, I found it is a little bit tricky.
I know correct version is
pub async fn f1<F, Fut>(f: F) -> u32
where
F: Fn(u32) -> Fut,
Fut: Future<Output = u32>,
{
f(9u32).await
}
However, the where clause in the Rust doc saying that it is just a convenient way to express those type, so I tried to remove the where clause like this:
pub async fn f2(f: Fn(u32) -> Future<Output = u32>) -> u32 {
f(9u32).await
}
To my surprise, Rust refused to compile it, saying that
error: trait objects must include the dyn keyword
Anyone know what is the correct way to remove the where clause?
Any magic is played here by the where clause?
In your first snippet you have to constraints:
F must implement Fn(u32) -> Fut.
Fut must implement Future<Output = u32>.
But in your second snippet there are no constraints. The syntax for an inline constraint is with the impl keyword:
pub async fn f2(f: impl Fn(u32) -> Future<Output = u32>) -> u32 {
f(9u32).await
}
This doesn't work either because the impl applies to the Fn trait only, not to the return trait, so the compiler complains that a naked trait is not allowed, that you should add dyn to it.
Bad advice, because:
pub async fn f2(f: impl Fn(u32) -> dyn Future<Output = u32>) -> u32 {
f(9u32).await
}
will not work either, because dyn Trait types are unsized and cannot be returned.
You could try another impl there, that usually works in return types but:
pub async fn f2(f: impl Fn(u32) -> impl Future<Output = u32>) -> u32 {
f(9u32).await
}
impl Trait` not allowed outside of function and method return types
The only syntax I know of to constraint the return type of an Fn trait is with the where syntax:
pub async fn f2<Fut>(f: impl Fn(u32) -> Fut) -> u32
where
Fut: Future<Output = u32>,
{
f(9u32).await
}
But now that you are in full where mode, I see little reason to keep the inline constraint. Personally I consider bad style having both syntaxes on the same function.
Assuming you're referencing Rust by Example:
The example there doesn't use trait objects (e.g. foo: Box<dyn SomeTrait>) or existential types (e.g. foo: impl SomeTrait) but adds bounds to generic types at the declaration site:
impl <A: TraitB + TraitC, D: TraitE + TraitF> MyTrait<A, D> for YourType {}
// Expressing bounds with a `where` clause
impl <A, D> MyTrait<A, D> for YourType where
A: TraitB + TraitC,
D: TraitE + TraitF {}
In your case, you'd be writing:
pub async fn f2<F: Fn(u32) -> Fut, Fut: std::future::Future<Output = u32>>(f: F) -> u32 {
f(9u32).await
}
instead of putting the Fn(u32) -> Future<Output = u32> bound as the type of f. Note that you also need to declare a generic type Fut: Future<Output = u32> for the returned future from the closure, otherwise you'd be returning a raw trait object again.
As mentioned by #rodrigo, it's also possible to use the impl SomeTrait notation to entirely remove the generic type parameter from the function.
I'd like a function which asynchronously processes a variable amount of (Sink, Stream) tuples.
use futures::channel::mpsc;
use futures::{Sink, Stream, SinkExt, StreamExt};
async fn foo(v: Vec<(Box<dyn Sink<Error = std::io::Error>>, Box<dyn Stream<Item = u8>>)>) {
for (mut tx, mut rx) in v {
let _ = tx.send(0);
let _ = rx.next().await;
}
}
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn std::error::Error>> {
let (tx, mut rx) = mpsc::channel(32);
foo(vec![(Box::new(tx), Box::new(rx))]).await;
Ok(())
}
But I get this compilation error:
error[E0107]: wrong number of type arguments: expected 1, found 0
--> src/main.rs:4:30
|
4 | async fn foo(v: Vec<(Box<dyn Sink<Error = std::io::Error>>, Box<dyn Stream<Item = u8>>)>) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 1 type argument
I was prompted to declare the associated type for the trait object that way by the compiler itself. I'm unsure why it does not accept it.
The compiler wants you to specify the "type argument" of the Sink. This is not the error type, but the type of the item being sent down the sink, as in Sink<Foo>. You specify u8 as the type of the stream, and are sending the value unchanged between one and the other, so you probably want a Sink<u8>.
Once you do that, the compiler will next complain that you need to specify the Error associated type (this time for real). However if you specify std::io::Error, the call to foo() from main() won't compile because the implementation of Sink for mpsc::Sender specifies its own mpsc::SendError as the error type.
Finally, both the sink and the stream need to be pinned so they can live across await points. This is done by using Pin<Box<...>> instead of Box<...> and Box::pin(...) instead of Box::new(...).
With the above changes, a version that compiles looks like this:
use futures::channel::mpsc;
use futures::{Sink, SinkExt, Stream, StreamExt};
use std::pin::Pin;
async fn foo(
v: Vec<(
Pin<Box<dyn Sink<u8, Error = mpsc::SendError>>>,
Pin<Box<dyn Stream<Item = u8>>>,
)>,
) {
for (mut tx, mut rx) in v {
let _ = tx.send(0);
let _ = rx.next().await;
}
}
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn std::error::Error>> {
let (tx, rx) = mpsc::channel(32);
foo(vec![(Box::pin(tx), Box::pin(rx))]).await;
Ok(())
}
I am trying to store async functions in a vector, but it seems like impl cannot be used in the vector type definition:
use std::future::Future;
fn main() {
let mut v: Vec<fn() -> impl Future<Output = ()>> = vec![];
v.push(haha);
}
async fn haha() {
println!("haha");
}
error[E0562]: `impl Trait` not allowed outside of function and inherent method return types
--> src/main.rs:4:28
|
4 | let mut v: Vec<fn() -> impl Future<Output = ()>> = vec![];
| ^^^^^^^^^^^^^^^^^^^^^^^^
How do I write the type inside the vector?
I found that there may be a workaround by using a type alias, so I changed the code:
use std::future::Future;
type Haha = impl Future<Output = ()>;
fn main() {
let mut v: Vec<fn() -> Haha> = vec![];
v.push(haha);
}
async fn haha() {
println!("haha");
}
This doesn't work either; this time the error occurs in the type alias:
error[E0658]: `impl Trait` in type aliases is unstable
--> src/main.rs:3:1
|
3 | type Haha = impl Future<Output = ()>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: for more information, see https://github.com/rust-lang/rust/issues/63063
error[E0308]: mismatched types
--> src/main.rs:8:12
|
8 | v.push(haha);
| ^^^^ expected opaque type, found a different opaque type
|
= note: expected type `fn() -> Haha`
found type `fn() -> impl std::future::Future {haha}`
= note: distinct uses of `impl Trait` result in different opaque types
error: could not find defining uses
--> src/main.rs:3:1
|
3 | type Haha = impl Future<Output = ()>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
How do I fix it?
You cannot use the impl Trait this way. To be able to store different types that implement a trait into the same container you have to use dynamic dispatch, by storing something like Box<dyn Trait>.
In your particular case, you do not specify if you want to store the async functions themselves or the future generated by the async functions, the solution would be somewhat different.
To store just the futures, you write a container such as:
let mut v: Vec<Box<dyn Future<Output = ()>>> = vec![];
And then just call the function, box it and store it in the container:
v.push(Box::new(haha()));
If instead you want to store the async function itself, without calling it, you need a container with a double dyn:
let mut v2: Vec<Box<dyn Fn() -> Box<dyn Future<Output = ()>>>> = vec![];
Now, since your haha function does not implement this Fn trait you need an adaptor. A lambda function will do, but don't forget the double Box:
v2.push(Box::new(|| Box::new(haha())));
Unfortunately, with these solutions you will be able to create the vector, but not to .await for your futures. For that you need the futures to implement the Unpin marker. That guarantees to the compiler that the future will not move while it is running (if it did, the implementation would be totally unsafe). You could add the + Unpin requirement to the futures, but async fn are not Unpin so you could not fill the vector. The easiest way to fix it is to use this handy function from std:
pub fn into_pin(boxed: Box<T>) -> Pin<Box<T>>
for f in v2 {
f().into_pin().await;
}
Unfortunately, it is still unstable. Fortunately, there is a From impl that does exactly the same. So you can just write:
for f in v2 {
Pin::from(f()).await;
}
In your comment below you write this code to wait for the futures:
for f in v2 {
async { f().await }
}
Note that an async block itself will evaluate to another future, so here you are just wrapping each future into another future, but nobody is waiting for that one. Actually you'll get a warning about it:
warning: unused implementer of std::future::Future that must be used.
Remember that in order to properly wait for all the futures you will need an async runtime.
rodrigo's answer is correct, but I'd prefer to use Box::pin and bake the Pin type into the API of the collection. This makes using the Future trait object (or closure trait object producing a Future trait object) easier:
use std::{future::Future, pin::Pin};
type PinFutureObj<Output> = Pin<Box<dyn Future<Output = Output>>>;
async fn collection_of_pinned_future_trait_objects() {
let v: Vec<PinFutureObj<()>> = vec![
Box::pin(haha()),
Box::pin(hehe()),
Box::pin(haha()),
Box::pin(hehe()),
];
for f in v {
f.await
}
}
async fn collection_of_closure_trait_objects() {
let v: Vec<Box<dyn Fn() -> PinFutureObj<()>>> = vec![
Box::new(|| Box::pin(haha())),
Box::new(|| Box::pin(hehe())),
Box::new(|| Box::pin(haha())),
Box::new(|| Box::pin(hehe())),
];
for f in v {
f().await
}
}
async fn haha() {
println!("haha");
}
async fn hehe() {
println!("hehe");
}
I'd also start introducing type aliases for the longer types.
In fact, this type alias already exists in the futures crate as LocalBoxFuture and can be created via FutureExt::boxed_local. There's also BoxFuture produced by FutureExt::boxed which adds common trait bounds.
use futures::future::{FutureExt, LocalBoxFuture}; // 0.3.5
async fn collection_of_pinned_future_trait_objects() {
let v: Vec<LocalBoxFuture<'static, ()>> = vec![
haha().boxed_local(),
hehe().boxed_local(),
haha().boxed_local(),
hehe().boxed_local(),
];
for f in v {
f.await
}
}
async fn collection_of_closure_trait_objects() {
let v: Vec<Box<dyn Fn() -> LocalBoxFuture<'static, ()>>> = vec![
Box::new(|| haha().boxed_local()),
Box::new(|| hehe().boxed_local()),
Box::new(|| haha().boxed_local()),
Box::new(|| hehe().boxed_local()),
];
for f in v {
f().await
}
}
async fn haha() {
println!("haha");
}
async fn hehe() {
println!("hehe");
}
See also:
How can I put an async function into a map in Rust?
Why can impl trait not be used to return multiple / conditional types?
With Tokio's futures, if you want to convert an Error in the causal chain of combinators, you use from_err::<NewType>(). I want the same functionality, but instead for the Item in impl Future<Item = (), Error = ()>.
An example of some of my code:
let mut async_series_client = vec![];
async_series_client.push(Box::new(
SocketHandler::connect(
port,
addr,
handle,
tx_wave,
tx_linear,
KcpSessionManager::new(&handle2).unwrap(),
)
.from_err::<HyxeError>()
.join(tube)
.map_err(|mut err| err.printf()),
));
This returns ((),()) (Side question: does it return a tuple of () because of the join?). I want it to return just (). How can I do this?
Use Future::map. This is a parallel to Option::map, Result::map, and Iterator::map:
use futures::{future, Future}; // 0.1.27
fn some_future() -> impl Future<Item = i32, Error = ()> {
future::ok(42)
}
fn change_item() -> impl Future<Item = String, Error = ()> {
some_future().map(|i| i.to_string())
}
See also Stream::map.
When async/await syntax is stabilized, you may never need to use this combinator again as you can just use normal methods:
async fn some_future() -> i32 {
42
}
async fn change_output() -> String {
some_future().await.to_string()
}
Or Result::map:
async fn some_future() -> Result<i32, ()> {
Ok(42)
}
async fn change_output() -> Result<String, ()> {
some_future().await.map(|i| i.to_string())
}
But it still exists:
use futures::{Future, FutureExt}; // 0.3.0-alpha.16
async fn some_future() -> i32 {
42
}
fn change_output() -> impl Future<Output = String> {
some_future().map(|i| i.to_string())
}