Suppose we have a type that is not Send.
struct NotSend {
field: std::rc::Rc<i32>
}
Then, following async function can still take NotSend as its parameter and compiles well:
async fn func(not_send: NotSend) -> i32 {
0
}
But when I define the same function inside of the trait, then anything that implements it does not compile.
#[async_trait]
trait A {
async fn func(not_send: NotSend) -> i32;
}
struct S {
}
#[async_trait]
impl A for S {
async fn func(not_send: NotSend) -> i32 {
0
}
}
This fails with the following message:
error: future cannot be sent between threads safely
--> src/main.rs:23:46
|
23 | async fn func( not_send: NotSend) -> i32 {
| ______________________________________________^
24 | | 0
25 | | }
| |_____^ future created by async block is not `Send`
|
= help: within `impl Future<Output = i32>`, the trait `Send` is not implemented for `Rc<i32>`
note: captured value is not `Send`
--> src/main.rs:23:20
|
23 | async fn func( not_send: NotSend) -> i32 {
| ^^^^^^^^ has type `NotSend` which is not `Send`
= note: required for the cast to the object type `dyn Future<Output = i32> + Send`
What's so different from the naive function and the function in the trait? Why one does works but not the other? Playground Link
It's because async_trait expands to something like Pin<Box<dyn Future>>. If we want the resulting future to be Send, it needs to be Pin<Box<dyn Future + Send>>. But this forces it to be Send, i.e. will error on non-Send futures. The async_trait crate does not have a way to know whether the future is Send (because the implementation of the trait is different from its declaration, and we need to decide at declaration site whether the future will be Send or not), so it opts to use user-defined annotations.
By default, the generated type is Send since most futures needs to be Send. However, as explained in the documentation, you can use #[async_trait(?Send)] to opt this out:
#[async_trait(?Send)]
trait A {
async fn func(not_send: NotSend) -> i32;
}
struct S {
}
#[async_trait(?Send)]
impl A for S {
async fn func(not_send: NotSend) -> i32 {
0
}
}
Related
The following function compiles without issues:
async fn filter_con<T, O, F>(arr: Vec<T>, predicate: F) -> Vec<T>
where
O: Future<Output = bool>,
F: for Fn(&T) -> O,
{
join_all(arr.into_iter().map(|it| async {
if predicate(&it).await {
Some(it)
} else {
None
}
}))
.await
.into_iter()
.filter_map(|p| p)
.collect::<Vec<_>>()
}
But I can't find a way to call it properly:
let items_filtered = filter_con(items, filter).await;
Error:
error[E0308]: mismatched types
--> src/lifecycle/querier.rs:101:30
|
101 | let items_filtered = filter_con(items_response, filter).await;
| ^^^^^^^^^^ lifetime mismatch
|
= note: expected associated type `<for<'_> fn(&PairInfo) -> impl futures::Future<Output = bool> {lifecycle::querier::filter} as FnOnce<(&PairInfo,)>>::Output`
found associated type `<for<'_> fn(&PairInfo) -> impl futures::Future<Output = bool> {lifecycle::querier::filter} as FnOnce<(&PairInfo,)>>::Output`
= note: the required lifetime does not necessarily outlive the empty lifetime
note: the lifetime requirement is introduced here
--> src/lifecycle/querier.rs:246:26
|
246 | F: for<'a> Fn(&T) -> O,
| ^
Why is this happening?
Full playground here
You have essentially the same problem as in here
To make it easier to understand I'll rewrite/desugar some of your code. Let's start with the predicate:
async fn filter(x: &Test) -> bool{
x.0 >= 50
}
Is functionally equivalent to:
fn filter(x: &Test) -> impl Future<Output=bool> + '_{
async {
x.0 >= 50
}
}
Now it should be obvious that the lifetime of the returned Future is not 'static, but actually it depends on the lifetime of the parameter x: &Test.
This should make the error more understandable - your predicate is returning some future with some lifetime, which is not mentioned anywhere in the generic type definition - you can see that you don't mention it when you define neither O, nor F:
where
O: Future<Output = bool>,
F: for Fn(&T) -> O,
The problem is that, currently the language does not provide any means to say that O's lifetime depends on F's lifetime. The only easy solution I know is to use the escape hatch - box the future, which will allow you to use the same lifetime in F and O. the downside is that, F will be heap allocated:
use futures::future::{join_all, Future};
use futures::future::BoxFuture;
use futures::FutureExt;
#[derive(Debug)]
struct Test(usize);
#[tokio::main]
async fn main() {
let items = vec![Test(10), Test(100), Test(1000)];
let items_filtered = filter_con(items, filter).await;
println!("{:?}", items_filtered); // should print [100, 1000]
}
fn filter(x: &Test) -> BoxFuture<'_, bool>{
async {
x.0 >= 50
}.boxed()
}
async fn filter_con<T, F>(arr: Vec<T>, predicate: F) -> Vec<T>
where
F: for<'a> Fn(&'a T) -> BoxFuture<'a, bool>,
{
join_all(arr.into_iter().map(|it| async{
if predicate(&it).await {
Some(it)
} else {
None
}
}))
.await
.into_iter()
.filter_map(|p| p)
.collect::<Vec<_>>()
}
Here is a link to the Rust Plauground
I have a struct Test I want to implement std::future::Future that would poll function:
use std::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
struct Test;
impl Test {
async fn function(&mut self) {}
}
impl Future for Test {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
match self.function() {
Poll::Pending => Poll::Pending,
Poll::Ready(_) => Poll::Ready(()),
}
}
}
That didn't work:
error[E0308]: mismatched types
--> src/lib.rs:17:13
|
10 | async fn function(&mut self) {}
| - the `Output` of this `async fn`'s expected opaque type
...
17 | Poll::Pending => Poll::Pending,
| ^^^^^^^^^^^^^ expected opaque type, found enum `Poll`
|
= note: expected opaque type `impl Future`
found enum `Poll<_>`
error[E0308]: mismatched types
--> src/lib.rs:18:13
|
10 | async fn function(&mut self) {}
| - the `Output` of this `async fn`'s expected opaque type
...
18 | Poll::Ready(_) => Poll::Ready(()),
| ^^^^^^^^^^^^^^ expected opaque type, found enum `Poll`
|
= note: expected opaque type `impl Future`
found enum `Poll<_>`
I understand that function must be called once, the returned Future must be stored somewhere in the struct, and then the saved future must be polled. I tried this:
struct Test(Option<Box<Pin<dyn Future<Output = ()>>>>);
impl Test {
async fn function(&mut self) {}
fn new() -> Self {
let mut s = Self(None);
s.0 = Some(Box::pin(s.function()));
s
}
}
That also didn't work:
error[E0277]: the size for values of type `(dyn Future<Output = ()> + 'static)` cannot be known at compilation time
--> src/lib.rs:7:13
|
7 | struct Test(Option<Box<Pin<dyn Future<Output = ()>>>>);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `(dyn Future<Output = ()> + 'static)`
After I call function() I have taken a &mut reference of Test, because of that I can't change the Test variable, and therefore can't store the returned Future inside the Test.
I did get an unsafe solution (inspired by this)
struct Test<'a>(Option<BoxFuture<'a, ()>>);
impl Test<'_> {
async fn function(&mut self) {
println!("I'm alive!");
}
fn new() -> Self {
let mut s = Self(None);
s.0 = Some(unsafe { &mut *(&mut s as *mut Self) }.function().boxed());
s
}
}
impl Future for Test<'_> {
type Output = ();
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
self.0.as_mut().unwrap().poll_unpin(cx)
}
}
I hope that there is another way.
Though there are times when you may want to do things similar to what you're trying to accomplish here, they are a rarity. So most people reading this, maybe even OP, may wish to restructure such that struct state and data used for a single async execution are different objects.
To answer your question, yes it is somewhat possible. Unless you want to absolutely resort to unsafe code you will need to use Mutex and Arc. All fields you wish to manipulate inside the async fn will have to be wrapped inside a Mutex and the function itself will accept an Arc<Self>.
I must stress, however, that this is not a beautiful solution and you probably don't want to do this. Depending on your specific case your solution may vary, but my guess of what OP is trying to accomplish while using Streams would be better solved by something similar to this gist that I wrote.
use std::{
future::Future,
pin::Pin,
sync::{Arc, Mutex},
};
struct Test {
state: Mutex<Option<Pin<Box<dyn Future<Output = ()>>>>>,
// if available use your async library's Mutex to `.await` locks on `buffer` instead
buffer: Mutex<Vec<u8>>,
}
impl Test {
async fn function(self: Arc<Self>) {
for i in 0..16u8 {
let data: Vec<u8> = vec![i]; // = fs::read(&format("file-{}.txt", i)).await.unwrap();
let mut buflock = self.buffer.lock().unwrap();
buflock.extend_from_slice(&data);
}
}
pub fn new() -> Arc<Self> {
let s = Arc::new(Self {
state: Default::default(),
buffer: Default::default(),
});
{
// start by trying to aquire a lock to the Mutex of the Box
let mut lock = s.state.lock().unwrap();
// create boxed future
let b = Box::pin(s.clone().function());
// insert value into the mutex
*lock = Some(b);
} // block causes the lock to be released
s
}
}
impl Future for Test {
type Output = ();
fn poll(
self: std::pin::Pin<&mut Self>,
ctx: &mut std::task::Context<'_>,
) -> std::task::Poll<<Self as std::future::Future>::Output> {
let mut lock = self.state.lock().unwrap();
let fut: &mut Pin<Box<dyn Future<Output = ()>>> = lock.as_mut().unwrap();
Future::poll(fut.as_mut(), ctx)
}
}
I'm not sure what you want to achieve and why, but I suspect that you're trying to implement Future for Test based on some ancient tutorial or misunderstanding and just overcomplicating things.
You don't have to implement Future manually. An async function
async fn function(...) {...}
is really just syntax sugar translated behind the scenes into something like
fn function(...) -> Future<()> {...}
All you have to do is to use the result of the function the same way as any future, e.g. use await on it or call block a reactor until it's finished. E.g. based on your first version, you can simply call:
let mut test = Test{};
test.function().await;
UPDATE1
Based on your descriptions I still think you're trying to overcomplicate this minimal working snippet without the need to manually implement Future for anything:
async fn asyncio() { println!("Doing async IO"); }
struct Test {
count: u32,
}
impl Test {
async fn function(&mut self) {
asyncio().await;
self.count += 1;
}
}
#[tokio::main]
async fn main() {
let mut test = Test{count: 0};
test.function().await;
println!("Count: {}", test.count);
}
My goal is to implement a function that returns another function, which returns some trait. To be more specific, the returned function should itself return a Future.
To return a function that returns a concrete type, we obviously can do this:
fn returns_closure() -> impl Fn(i32) -> i32 {
|x| x + 1
}
But what if instead of i32 we want to return a Future?
I tried the following:
use futures::Future;
fn factory() -> (impl Fn() -> impl Future) {
|| async {
// some async code
}
}
This does not work because the second impl keyword is not allowed:
error[E0562] `impl Trait` not allowed outside of function and inherent method return types
What is the best way to solve this issue?
I don't know of any way to do this on stable Rust. However, you can use a type alias for an opaque type (also known as existential type) on Rust nightly like this (playground):
#![feature(type_alias_impl_trait)]
use futures::Future;
type Fut<O> = impl Future<Output = O>;
fn factory<O>() -> impl Fn() -> Fut<O> {
|| async {
todo!()
}
}
I am trying to store async functions in a vector, but it seems like impl cannot be used in the vector type definition:
use std::future::Future;
fn main() {
let mut v: Vec<fn() -> impl Future<Output = ()>> = vec![];
v.push(haha);
}
async fn haha() {
println!("haha");
}
error[E0562]: `impl Trait` not allowed outside of function and inherent method return types
--> src/main.rs:4:28
|
4 | let mut v: Vec<fn() -> impl Future<Output = ()>> = vec![];
| ^^^^^^^^^^^^^^^^^^^^^^^^
How do I write the type inside the vector?
I found that there may be a workaround by using a type alias, so I changed the code:
use std::future::Future;
type Haha = impl Future<Output = ()>;
fn main() {
let mut v: Vec<fn() -> Haha> = vec![];
v.push(haha);
}
async fn haha() {
println!("haha");
}
This doesn't work either; this time the error occurs in the type alias:
error[E0658]: `impl Trait` in type aliases is unstable
--> src/main.rs:3:1
|
3 | type Haha = impl Future<Output = ()>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: for more information, see https://github.com/rust-lang/rust/issues/63063
error[E0308]: mismatched types
--> src/main.rs:8:12
|
8 | v.push(haha);
| ^^^^ expected opaque type, found a different opaque type
|
= note: expected type `fn() -> Haha`
found type `fn() -> impl std::future::Future {haha}`
= note: distinct uses of `impl Trait` result in different opaque types
error: could not find defining uses
--> src/main.rs:3:1
|
3 | type Haha = impl Future<Output = ()>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
How do I fix it?
You cannot use the impl Trait this way. To be able to store different types that implement a trait into the same container you have to use dynamic dispatch, by storing something like Box<dyn Trait>.
In your particular case, you do not specify if you want to store the async functions themselves or the future generated by the async functions, the solution would be somewhat different.
To store just the futures, you write a container such as:
let mut v: Vec<Box<dyn Future<Output = ()>>> = vec![];
And then just call the function, box it and store it in the container:
v.push(Box::new(haha()));
If instead you want to store the async function itself, without calling it, you need a container with a double dyn:
let mut v2: Vec<Box<dyn Fn() -> Box<dyn Future<Output = ()>>>> = vec![];
Now, since your haha function does not implement this Fn trait you need an adaptor. A lambda function will do, but don't forget the double Box:
v2.push(Box::new(|| Box::new(haha())));
Unfortunately, with these solutions you will be able to create the vector, but not to .await for your futures. For that you need the futures to implement the Unpin marker. That guarantees to the compiler that the future will not move while it is running (if it did, the implementation would be totally unsafe). You could add the + Unpin requirement to the futures, but async fn are not Unpin so you could not fill the vector. The easiest way to fix it is to use this handy function from std:
pub fn into_pin(boxed: Box<T>) -> Pin<Box<T>>
for f in v2 {
f().into_pin().await;
}
Unfortunately, it is still unstable. Fortunately, there is a From impl that does exactly the same. So you can just write:
for f in v2 {
Pin::from(f()).await;
}
In your comment below you write this code to wait for the futures:
for f in v2 {
async { f().await }
}
Note that an async block itself will evaluate to another future, so here you are just wrapping each future into another future, but nobody is waiting for that one. Actually you'll get a warning about it:
warning: unused implementer of std::future::Future that must be used.
Remember that in order to properly wait for all the futures you will need an async runtime.
rodrigo's answer is correct, but I'd prefer to use Box::pin and bake the Pin type into the API of the collection. This makes using the Future trait object (or closure trait object producing a Future trait object) easier:
use std::{future::Future, pin::Pin};
type PinFutureObj<Output> = Pin<Box<dyn Future<Output = Output>>>;
async fn collection_of_pinned_future_trait_objects() {
let v: Vec<PinFutureObj<()>> = vec![
Box::pin(haha()),
Box::pin(hehe()),
Box::pin(haha()),
Box::pin(hehe()),
];
for f in v {
f.await
}
}
async fn collection_of_closure_trait_objects() {
let v: Vec<Box<dyn Fn() -> PinFutureObj<()>>> = vec![
Box::new(|| Box::pin(haha())),
Box::new(|| Box::pin(hehe())),
Box::new(|| Box::pin(haha())),
Box::new(|| Box::pin(hehe())),
];
for f in v {
f().await
}
}
async fn haha() {
println!("haha");
}
async fn hehe() {
println!("hehe");
}
I'd also start introducing type aliases for the longer types.
In fact, this type alias already exists in the futures crate as LocalBoxFuture and can be created via FutureExt::boxed_local. There's also BoxFuture produced by FutureExt::boxed which adds common trait bounds.
use futures::future::{FutureExt, LocalBoxFuture}; // 0.3.5
async fn collection_of_pinned_future_trait_objects() {
let v: Vec<LocalBoxFuture<'static, ()>> = vec![
haha().boxed_local(),
hehe().boxed_local(),
haha().boxed_local(),
hehe().boxed_local(),
];
for f in v {
f.await
}
}
async fn collection_of_closure_trait_objects() {
let v: Vec<Box<dyn Fn() -> LocalBoxFuture<'static, ()>>> = vec![
Box::new(|| haha().boxed_local()),
Box::new(|| hehe().boxed_local()),
Box::new(|| haha().boxed_local()),
Box::new(|| hehe().boxed_local()),
];
for f in v {
f().await
}
}
async fn haha() {
println!("haha");
}
async fn hehe() {
println!("hehe");
}
See also:
How can I put an async function into a map in Rust?
Why can impl trait not be used to return multiple / conditional types?
I'd like to take a regular iterator and turn it into a stream so that I can do further stream processing. The trouble is that I may have an iterator or an error to deal with. I think I'm pretty close with this:
#[macro_use]
extern crate log;
extern crate futures; // 0.1.21
extern crate tokio;
use futures::prelude::*;
use futures::{future, stream};
use std::fmt::Debug;
use std::net::{SocketAddr, ToSocketAddrs};
fn resolve(addrs: impl ToSocketAddrs + Debug) -> impl Stream<Item = SocketAddr, Error = ()> {
match addrs.to_socket_addrs() {
Ok(iter) => stream::unfold(iter, |iter| match iter.next() {
Some(a) => Some(future::ok((a, iter))),
None => None,
}),
Err(e) => {
error!("could not resolve socket addresses {:?}: {:?}", addrs, e);
stream::empty()
}
}
}
fn main() {
let task = resolve("1.2.3.4:12345")
.map_err(|e| error!("{:?}", e))
.for_each(|addr| info!("{:?}", addr))
.fold();
tokio::run(task);
}
playground
error[E0308]: match arms have incompatible types
--> src/main.rs:12:5
|
12 | / match addrs.to_socket_addrs() {
13 | | Ok(iter) => stream::unfold(iter, |iter| match iter.next() {
14 | | Some(a) => Some(future::ok((a, iter))),
15 | | None => None,
... |
20 | | }
21 | | }
| |_____^ expected struct `futures::stream::Unfold`, found struct `futures::stream::Empty`
|
= note: expected type `futures::stream::Unfold<<impl ToSocketAddrs + Debug as std::net::ToSocketAddrs>::Iter, [closure#src/main.rs:13:42: 16:10], futures::FutureResult<(std::net::SocketAddr, <impl ToSocketAddrs + Debug as std::net::ToSocketAddrs>::Iter), _>>`
found type `futures::stream::Empty<_, _>`
note: match arm with an incompatible type
--> src/main.rs:17:19
|
17 | Err(e) => {
| ___________________^
18 | | error!("could not resolve socket addresses {:?}: {:?}", addrs, e);
19 | | stream::empty()
20 | | }
| |_________^
error[E0277]: the trait bound `(): futures::Future` is not satisfied
--> src/main.rs:27:10
|
27 | .for_each(|addr| info!("{:?}", addr))
| ^^^^^^^^ the trait `futures::Future` is not implemented for `()`
|
= note: required because of the requirements on the impl of `futures::IntoFuture` for `()`
error[E0599]: no method named `fold` found for type `futures::stream::ForEach<futures::stream::MapErr<impl futures::Stream, [closure#src/main.rs:26:18: 26:39]>, [closure#src/main.rs:27:19: 27:45], ()>` in the current scope
--> src/main.rs:28:10
|
28 | .fold();
| ^^^^
|
= note: the method `fold` exists but the following trait bounds were not satisfied:
`&mut futures::stream::ForEach<futures::stream::MapErr<impl futures::Stream, [closure#src/main.rs:26:18: 26:39]>, [closure#src/main.rs:27:19: 27:45], ()> : futures::Stream`
`&mut futures::stream::ForEach<futures::stream::MapErr<impl futures::Stream, [closure#src/main.rs:26:18: 26:39]>, [closure#src/main.rs:27:19: 27:45], ()> : std::iter::Iterator`
The hint is pretty obvious. The two Results I'm returning from the match differ and should be the same. Now, how can I do that so that I return a stream?
Rust is a statically typed language which means that the return type of a function has to be a single type, known at compile time. You are attempting to return multiple types, decided at runtime.
The closest solution to your original is to always return the Unfold stream:
fn resolve(addrs: impl ToSocketAddrs) -> impl Stream<Item = SocketAddr, Error = ()> {
stream::unfold(addrs.to_socket_addrs(), |r| {
match r {
Ok(mut iter) => iter.next().map(|addr| future::ok((addr, Ok(iter)))),
Err(_) => None,
}
})
}
But why reinvent the wheel?
futures::stream::iter_ok
Converts an Iterator into a Stream which is always ready to yield the next value.
Subsequent versions of the futures crate implement Stream for Either, which makes this very elegant:
fn resolve(addrs: impl ToSocketAddrs) -> impl Stream<Item = SocketAddr, Error = ()> {
match addrs.to_socket_addrs() {
Ok(iter) => stream::iter_ok(iter).left_stream(),
Err(_) => stream::empty().right_stream(),
}
}
It's straightforward to backport this functionality to futures 0.1 (maybe someone should submit it as a PR for those who are stuck on 0.1...):
enum MyEither<L, R> {
Left(L),
Right(R),
}
impl<L, R> Stream for MyEither<L, R>
where
L: Stream,
R: Stream<Item = L::Item, Error = L::Error>,
{
type Item = L::Item;
type Error = L::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self {
MyEither::Left(l) => l.poll(),
MyEither::Right(r) => r.poll(),
}
}
}
trait EitherStreamExt {
fn left_stream<R>(self) -> MyEither<Self, R>
where
Self: Sized;
fn right_stream<L>(self) -> MyEither<L, Self>
where
Self: Sized;
}
impl<S: Stream> EitherStreamExt for S {
fn left_stream<R>(self) -> MyEither<Self, R> {
MyEither::Left(self)
}
fn right_stream<L>(self) -> MyEither<L, Self> {
MyEither::Right(self)
}
}
Even better, use the fact that Result is an iterator and Stream::flatten exists:
fn resolve(addrs: impl ToSocketAddrs) -> impl Stream<Item = SocketAddr, Error = ()> {
stream::iter_ok(addrs.to_socket_addrs())
.map(stream::iter_ok)
.flatten()
}
Or if you really want to print errors:
fn resolve(addrs: impl ToSocketAddrs) -> impl Stream<Item = SocketAddr, Error = ()> {
stream::once(addrs.to_socket_addrs())
.map(stream::iter_ok)
.map_err(|e| eprintln!("err: {}", e))
.flatten()
}
See also:
Conditionally return empty iterator from flat_map
Conditionally iterate over one of several possible iterators
What is the correct way to return an Iterator (or any other trait)?