With Tokio's futures, if you want to convert an Error in the causal chain of combinators, you use from_err::<NewType>(). I want the same functionality, but instead for the Item in impl Future<Item = (), Error = ()>.
An example of some of my code:
let mut async_series_client = vec![];
async_series_client.push(Box::new(
SocketHandler::connect(
port,
addr,
handle,
tx_wave,
tx_linear,
KcpSessionManager::new(&handle2).unwrap(),
)
.from_err::<HyxeError>()
.join(tube)
.map_err(|mut err| err.printf()),
));
This returns ((),()) (Side question: does it return a tuple of () because of the join?). I want it to return just (). How can I do this?
Use Future::map. This is a parallel to Option::map, Result::map, and Iterator::map:
use futures::{future, Future}; // 0.1.27
fn some_future() -> impl Future<Item = i32, Error = ()> {
future::ok(42)
}
fn change_item() -> impl Future<Item = String, Error = ()> {
some_future().map(|i| i.to_string())
}
See also Stream::map.
When async/await syntax is stabilized, you may never need to use this combinator again as you can just use normal methods:
async fn some_future() -> i32 {
42
}
async fn change_output() -> String {
some_future().await.to_string()
}
Or Result::map:
async fn some_future() -> Result<i32, ()> {
Ok(42)
}
async fn change_output() -> Result<String, ()> {
some_future().await.map(|i| i.to_string())
}
But it still exists:
use futures::{Future, FutureExt}; // 0.3.0-alpha.16
async fn some_future() -> i32 {
42
}
fn change_output() -> impl Future<Output = String> {
some_future().map(|i| i.to_string())
}
Related
Part 1: What should be the signature of a function returning an async function?
pub async fn some_async_func(arg: &str) {}
// What should be sig here?
pub fn higher_order_func(action: &str) -> ???
{
some_async_func
}
Part 2: What should be the sig, if based on the action parameter, higher_order_func had to return either async_func1 or async_func2.
I am also interested in learning the performance tradeoffs if there are multiple solutions. Please note that I'd like to return the function itself as an fn pointer or an Fn* trait, and not the result of invoking it.
Returning a function
Returning the actual function pointer requires heap allocation and a wrapper:
use std::future::Future;
use std::pin::Pin;
pub async fn some_async_func(arg: &str) {}
pub fn some_async_func_wrapper<'a>(arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
Box::pin(some_async_func(arg))
}
pub fn higher_order_func<'a>(action: &str)
-> fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
some_async_func_wrapper
}
Why boxing? higher_order_func needs to have a concrete return type, which is a function pointer. The pointed function needs to also have a concrete return type, which is impossible for async function since it returns opaque type. In theory, it could be possible to write return type as fn(&'a str) -> impl Future<Output=()> + 'a, but this would require much more guesswork from the compiler and currently is not supported.
If you are OK with Fn instead of fn, you can get rid of the wrapper:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> impl Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
|arg: &'a str| {
Box::pin(some_async_func(arg))
}
}
To return a different function based on action value, you will need to box the closure itself, which is one more heap allocation:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> Box<dyn Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>>
{
if action.starts_with("one") {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_one(arg))
})
} else {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_two(arg))
})
}
}
Alternative: returning a future
To simplify things, consider returning a future itself instead of a function pointer. This is virtually the same, but much nicer and does not require heap allocation:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> impl Future<Output=()> + 'a
{
some_async_func(arg)
}
It might look like, when higher_order_func_future is called, some_async_func is getting executed - but this is not the case. Because of the way async functions work, when you call some_async_func, no user code is getting executed. The function call returns a Future: the actual function body will be executed only when someone awaits the returned future.
You can use the new function almost the same way as the previous function:
// With higher order function returning function pointer
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func(action)(arg).await;
}
// With higher order function returning future
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func_future(action, arg).await;
}
Notice, once more, that in both cases the actual some_async_func body is executed only when the future is awaited.
If you wanted to be able to call different async functions based on action value, you need boxing again:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
if action.starts_with("one") {
Box::pin(some_async_func_one(arg))
} else {
Box::pin(some_async_func_two(arg))
}
}
Still, this is just one heap allocation, so I strongly advise returning a future. The only scenario that I can imagine where the previous solution is better is when you want to save the boxed closure somewhere and use it many times. In this case, excessive allocation happens only once, and you spare some CPU time by dispatching the call based on action only once - when you make the closure.
Ideally, what you'd want is a nested impl trait: -> impl Fn(&str) -> impl Future<Output = ()>. But nested impl trait is not supported. However, you can emulate that using a trait.
The idea is to define a trait that will abstract over the notion of "function returning a future". If our function would take a u32, for example, it could look like:
trait AsyncFn: Fn(u32) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<F, Fut> AsyncFn for F
where
F: Fn(u32) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we would take impl AsyncFn. Trying to apply that naively to &str doesn't work:
error[E0308]: mismatched types
--> src/lib.rs:16:27
|
16 | fn higher_order_func() -> impl AsyncFn {
| ^^^^^^^^^^^^ one type is more general than the other
|
= note: expected associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
found associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
The error may look very strange, but it arises from the fact that async fn returns a future bound by the lifetime of all of its argument, i.e. for a signature async fn foo<'a>(arg: &'a str), the future is not impl Future<Output = ()> but impl Future<Output = ()> + 'a. There is a way to capture this relationship in our trait, we just need to make it generic over the argument and use HRTB:
trait AsyncFn<Arg>: Fn(Arg) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<Arg, F, Fut> AsyncFn<Arg> for F
where
F: Fn(Arg) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we specify the type as:
fn higher_order_func() -> impl for<'a> AsyncFn<&'a str> {
some_async_func
}
In addition to the great accepted answer, depending on your use case it's also possible to "fake" the higher order function and avoid any heap allocations by using a simple macro to expand the wrapper code in-place instead:
pub async fn some_async_func(arg: &str) {}
macro_rules! higher_order_func {
($action: expr) => {
some_async_func
};
}
fn main() {
let future = higher_order_func!("action")("arg");
}
I'd like a function which asynchronously processes a variable amount of (Sink, Stream) tuples.
use futures::channel::mpsc;
use futures::{Sink, Stream, SinkExt, StreamExt};
async fn foo(v: Vec<(Box<dyn Sink<Error = std::io::Error>>, Box<dyn Stream<Item = u8>>)>) {
for (mut tx, mut rx) in v {
let _ = tx.send(0);
let _ = rx.next().await;
}
}
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn std::error::Error>> {
let (tx, mut rx) = mpsc::channel(32);
foo(vec![(Box::new(tx), Box::new(rx))]).await;
Ok(())
}
But I get this compilation error:
error[E0107]: wrong number of type arguments: expected 1, found 0
--> src/main.rs:4:30
|
4 | async fn foo(v: Vec<(Box<dyn Sink<Error = std::io::Error>>, Box<dyn Stream<Item = u8>>)>) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 1 type argument
I was prompted to declare the associated type for the trait object that way by the compiler itself. I'm unsure why it does not accept it.
The compiler wants you to specify the "type argument" of the Sink. This is not the error type, but the type of the item being sent down the sink, as in Sink<Foo>. You specify u8 as the type of the stream, and are sending the value unchanged between one and the other, so you probably want a Sink<u8>.
Once you do that, the compiler will next complain that you need to specify the Error associated type (this time for real). However if you specify std::io::Error, the call to foo() from main() won't compile because the implementation of Sink for mpsc::Sender specifies its own mpsc::SendError as the error type.
Finally, both the sink and the stream need to be pinned so they can live across await points. This is done by using Pin<Box<...>> instead of Box<...> and Box::pin(...) instead of Box::new(...).
With the above changes, a version that compiles looks like this:
use futures::channel::mpsc;
use futures::{Sink, SinkExt, Stream, StreamExt};
use std::pin::Pin;
async fn foo(
v: Vec<(
Pin<Box<dyn Sink<u8, Error = mpsc::SendError>>>,
Pin<Box<dyn Stream<Item = u8>>>,
)>,
) {
for (mut tx, mut rx) in v {
let _ = tx.send(0);
let _ = rx.next().await;
}
}
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn std::error::Error>> {
let (tx, rx) = mpsc::channel(32);
foo(vec![(Box::pin(tx), Box::pin(rx))]).await;
Ok(())
}
I have a struct Test I want to implement std::future::Future that would poll function:
use std::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
struct Test;
impl Test {
async fn function(&mut self) {}
}
impl Future for Test {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
match self.function() {
Poll::Pending => Poll::Pending,
Poll::Ready(_) => Poll::Ready(()),
}
}
}
That didn't work:
error[E0308]: mismatched types
--> src/lib.rs:17:13
|
10 | async fn function(&mut self) {}
| - the `Output` of this `async fn`'s expected opaque type
...
17 | Poll::Pending => Poll::Pending,
| ^^^^^^^^^^^^^ expected opaque type, found enum `Poll`
|
= note: expected opaque type `impl Future`
found enum `Poll<_>`
error[E0308]: mismatched types
--> src/lib.rs:18:13
|
10 | async fn function(&mut self) {}
| - the `Output` of this `async fn`'s expected opaque type
...
18 | Poll::Ready(_) => Poll::Ready(()),
| ^^^^^^^^^^^^^^ expected opaque type, found enum `Poll`
|
= note: expected opaque type `impl Future`
found enum `Poll<_>`
I understand that function must be called once, the returned Future must be stored somewhere in the struct, and then the saved future must be polled. I tried this:
struct Test(Option<Box<Pin<dyn Future<Output = ()>>>>);
impl Test {
async fn function(&mut self) {}
fn new() -> Self {
let mut s = Self(None);
s.0 = Some(Box::pin(s.function()));
s
}
}
That also didn't work:
error[E0277]: the size for values of type `(dyn Future<Output = ()> + 'static)` cannot be known at compilation time
--> src/lib.rs:7:13
|
7 | struct Test(Option<Box<Pin<dyn Future<Output = ()>>>>);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `(dyn Future<Output = ()> + 'static)`
After I call function() I have taken a &mut reference of Test, because of that I can't change the Test variable, and therefore can't store the returned Future inside the Test.
I did get an unsafe solution (inspired by this)
struct Test<'a>(Option<BoxFuture<'a, ()>>);
impl Test<'_> {
async fn function(&mut self) {
println!("I'm alive!");
}
fn new() -> Self {
let mut s = Self(None);
s.0 = Some(unsafe { &mut *(&mut s as *mut Self) }.function().boxed());
s
}
}
impl Future for Test<'_> {
type Output = ();
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
self.0.as_mut().unwrap().poll_unpin(cx)
}
}
I hope that there is another way.
Though there are times when you may want to do things similar to what you're trying to accomplish here, they are a rarity. So most people reading this, maybe even OP, may wish to restructure such that struct state and data used for a single async execution are different objects.
To answer your question, yes it is somewhat possible. Unless you want to absolutely resort to unsafe code you will need to use Mutex and Arc. All fields you wish to manipulate inside the async fn will have to be wrapped inside a Mutex and the function itself will accept an Arc<Self>.
I must stress, however, that this is not a beautiful solution and you probably don't want to do this. Depending on your specific case your solution may vary, but my guess of what OP is trying to accomplish while using Streams would be better solved by something similar to this gist that I wrote.
use std::{
future::Future,
pin::Pin,
sync::{Arc, Mutex},
};
struct Test {
state: Mutex<Option<Pin<Box<dyn Future<Output = ()>>>>>,
// if available use your async library's Mutex to `.await` locks on `buffer` instead
buffer: Mutex<Vec<u8>>,
}
impl Test {
async fn function(self: Arc<Self>) {
for i in 0..16u8 {
let data: Vec<u8> = vec![i]; // = fs::read(&format("file-{}.txt", i)).await.unwrap();
let mut buflock = self.buffer.lock().unwrap();
buflock.extend_from_slice(&data);
}
}
pub fn new() -> Arc<Self> {
let s = Arc::new(Self {
state: Default::default(),
buffer: Default::default(),
});
{
// start by trying to aquire a lock to the Mutex of the Box
let mut lock = s.state.lock().unwrap();
// create boxed future
let b = Box::pin(s.clone().function());
// insert value into the mutex
*lock = Some(b);
} // block causes the lock to be released
s
}
}
impl Future for Test {
type Output = ();
fn poll(
self: std::pin::Pin<&mut Self>,
ctx: &mut std::task::Context<'_>,
) -> std::task::Poll<<Self as std::future::Future>::Output> {
let mut lock = self.state.lock().unwrap();
let fut: &mut Pin<Box<dyn Future<Output = ()>>> = lock.as_mut().unwrap();
Future::poll(fut.as_mut(), ctx)
}
}
I'm not sure what you want to achieve and why, but I suspect that you're trying to implement Future for Test based on some ancient tutorial or misunderstanding and just overcomplicating things.
You don't have to implement Future manually. An async function
async fn function(...) {...}
is really just syntax sugar translated behind the scenes into something like
fn function(...) -> Future<()> {...}
All you have to do is to use the result of the function the same way as any future, e.g. use await on it or call block a reactor until it's finished. E.g. based on your first version, you can simply call:
let mut test = Test{};
test.function().await;
UPDATE1
Based on your descriptions I still think you're trying to overcomplicate this minimal working snippet without the need to manually implement Future for anything:
async fn asyncio() { println!("Doing async IO"); }
struct Test {
count: u32,
}
impl Test {
async fn function(&mut self) {
asyncio().await;
self.count += 1;
}
}
#[tokio::main]
async fn main() {
let mut test = Test{count: 0};
test.function().await;
println!("Count: {}", test.count);
}
I would like to replicate the behavior and ergonomics of taking a closure/function as an argument much like map does: iterator.map(|x| ...).
I've noticed that some library code allows passing in async functionality, but this method doesn't allow me to pass in arguments:
pub fn spawn<F, T>(future: F) -> JoinHandle<T>
where
F: Future<Output = T> + Send + 'static,
T: Send + 'static,
spawn(async { foo().await });
I'm hoping to do one of the following:
iterator.map(async |x| {...});
async fn a(x: _) {}
iterator.map(a)
async functions are effectively desugared as returning impl Future. Once you know that, it's a matter of combining existing Rust techniques to accept a function / closure, resulting in a function with two generic types:
use std::future::Future;
async fn example<F, Fut>(f: F)
where
F: FnOnce(i32, i32) -> Fut,
Fut: Future<Output = bool>,
{
f(1, 2).await;
}
This can also be written as
use std::future::Future;
async fn example<Fut>(f: impl FnOnce(i32, i32) -> Fut)
where
Fut: Future<Output = bool>,
{
f(1, 2).await;
}
How do you pass a Rust function as a parameter?
What is the concrete type of a future returned from `async fn`?
What is the purpose of async/await in Rust?
How can I store an async function in a struct and call it from a struct instance?
What is the difference between `|_| async move {}` and `async move |_| {}`
The async |...| expr closure syntax is available on the nightly channel enabling the feature async_closure.
#![feature(async_closure)]
use futures::future;
use futures::Future;
use tokio;
pub struct Bar;
impl Bar {
pub fn map<F, T>(&self, f: F)
where
F: Fn(i32) -> T,
T: Future<Output = Result<i32, i32>> + Send + 'static,
{
tokio::spawn(f(1));
}
}
async fn foo(x: i32) -> Result<i32, i32> {
println!("running foo");
future::ok::<i32, i32>(x).await
}
#[tokio::main]
async fn main() {
let bar = Bar;
let x = 1;
bar.map(foo);
bar.map(async move |x| {
println!("hello from async closure.");
future::ok::<i32, i32>(x).await
});
}
See the 2394-async_await RFC for more detalis
In hyper 0.12.33, how do I implement hyper::service::Service for a struct ?
I have tried the following but it is not sufficient as it seems that in 0.12 the Future trait is not provided automatically anymore for a struct that implements Service:
use futures::future::Future;
use hyper::{Body, Request, Response};
struct MyStruct;
impl MyStruct {
pub fn new() -> Self {
MyStruct
}
}
impl hyper::service::Service for MyStruct {
type ReqBody = Body;
type ResBody = Body;
type Error = hyper::Error;
type Future = Box<Future<Item = Response<Body>, Error = hyper::Error>>;
fn call(&mut self, req: Request<Body>) -> Self::Future {
unimplemented!()
}
}
fn main() {
let addr = "0.0.0.0:8080".parse().unwrap();
let server = hyper::Server::bind(&addr)
.serve(|| MyStruct::new())
.map_err(|e| eprintln!("server error: {}", e));
hyper::rt::run(server);
}
gives me the build error message:
Standard Error
Compiling playground v0.0.1 (/playground)
error[E0277]: the trait bound `MyStruct: futures::future::Future` is not satisfied
--> src/main.rs:26:10
|
26 | .serve(|| MyStruct::new())
| ^^^^^ the trait `futures::future::Future` is not implemented for `MyStruct`
|
= note: required because of the requirements on the impl of `hyper::service::make_service::MakeServiceRef<hyper::server::tcp::addr_stream::AddrStream>` for `[closure#src/main.rs:26:16: 26:34]`
error[E0599]: no method named `map_err` found for type `hyper::server::Server<hyper::server::tcp::AddrIncoming, [closure#src/main.rs:26:16: 26:34]>` in the current scope
--> src/main.rs:27:10
|
27 | .map_err(|e| eprintln!("server error: {}", e));
| ^^^^^^^
|
= note: the method `map_err` exists but the following trait bounds were not satisfied:
`&mut hyper::server::Server<hyper::server::tcp::AddrIncoming, [closure#src/main.rs:26:16: 26:34]> : futures::future::Future`
`hyper::server::Server<hyper::server::tcp::AddrIncoming, [closure#src/main.rs:26:16: 26:34]> : futures::future::Future`
This example gives one way. It compiles and runs with v0.14.12
#![deny(warnings)]
use std::task::{Context, Poll};
use futures_util::future;
use hyper::service::Service;
use hyper::{Body, Request, Response, Server};
const ROOT: &str = "/";
#[derive(Debug)]
pub struct Svc;
impl Service<Request<Body>> for Svc {
type Response = Response<Body>;
type Error = hyper::Error;
type Future = future::Ready<Result<Self::Response, Self::Error>>;
fn poll_ready(&mut self, _cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
Ok(()).into()
}
fn call(&mut self, req: Request<Body>) -> Self::Future {
let rsp = Response::builder();
let uri = req.uri();
if uri.path() != ROOT {
let body = Body::from(Vec::new());
let rsp = rsp.status(404).body(body).unwrap();
return future::ok(rsp);
}
let body = Body::from(Vec::from(&b"heyo!"[..]));
let rsp = rsp.status(200).body(body).unwrap();
future::ok(rsp)
}
}
pub struct MakeSvc;
impl<T> Service<T> for MakeSvc {
type Response = Svc;
type Error = std::io::Error;
type Future = future::Ready<Result<Self::Response, Self::Error>>;
fn poll_ready(&mut self, _cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
Ok(()).into()
}
fn call(&mut self, _: T) -> Self::Future {
future::ok(Svc)
}
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// pretty_env_logger::init();
let addr = "127.0.0.1:1337".parse().unwrap();
let server = Server::bind(&addr).serve(MakeSvc);
println!("Listening on http://{}", addr);
server.await?;
Ok(())
}
The indirection (MakeSvc -> Src) appears to follow from the architecture of Hyper, as described in this issue:
There's two steps involved here, and both make use of Service:
The MakeSvc is a Service that creates Svcs for each connection.
The Svc is a Service to handle requests on a single connection.