I am building a server and would like to serve until a oneshot receiver tell me to stop. For that I am using tokios serve_with_shutdown. My understanding is that the service will run until the future signal is ready
pub async fn serve_with_shutdown<F: Future<Output = ()>>(
self,
addr: SocketAddr,
signal: F
) -> Result<(), Error>
How do I pass the oneshot receiver as a signal?
Passing it directly like serve_with_shutdown(some_addr, receiver); gives me unused implementer of futures::Future that must be used.
I tried implementing my own future. Same error.
pub struct OneShotFut {
receiver: Mutex<tokio::sync::oneshot::Receiver<()>>,
}
impl Future for OneShotFut {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let mut rec = self.receiver.lock().unwrap();
if rec.try_recv().is_err() {
return Poll::Pending;
}
Poll::Ready(())
}
}
//...
serve_with_shutdown(some_addr, OneShotFut {
receiver: Mutex::new(receiver),
})
I cannot await the future when passing ti to serve_with_shutdown, since that will directly return ()
Since serve_with_shutdown is an async fn it returns a Future that too must be awaited:
Inside another async fn:
serve_with_shutdown((some_addr, OneShotFut {
receiver: Mutex::new(receiver),
}).await
In addition to .await-ing on the result, you probably don't want to use OneShotFut. It is deficient because you never use the Context to trigger a wake-up. Using a oneshot::Receiver is the way to go, but passing it directly as you tried initially would run into an error:
error[E0271]: type mismatch resolving `<tokio::sync::oneshot::Receiver<_> as Future>::Output == ()`
--> src/main.rs:33:10
|
33 | .serve_with_shutdown(some_addr, receiver)
| ^^^^^^^^^^^^^^^^^^^ expected `()`, found enum `Result`
|
= note: expected unit type `()`
found enum `Result<_, tokio::sync::oneshot::error::RecvError>`
But to ignore any error from the oneshot receiver, which could happen if the sender is destroyed, you simply need to wrap it in an async block like so:
Server::builder()
.add_service(...)
.serve_with_shutdown(some_addr, async { let _ = receiver.await; })
.await?;
Related
Tokio's spawn can only work with a Send future. This makes the following code invalid:
async fn async_foo(v: i32) {}
async fn async_computation() -> Result<i32, Box<dyn std::error::Error>> {
Ok(1)
}
async fn async_start() {
match async_computation().await {
Ok(v) => async_foo(v).await,
_ => unimplemented!(),
};
}
#[tokio::main]
async fn main() {
tokio::spawn(async move {
async_start().await;
});
}
The error is:
future cannot be sent between threads safely
the trait `Send` is not implemented for `dyn std::error::Error`
If I understand correctly: because async_foo(v).await might yield, Rust internally have to save all the context which might be on a different thread - hence the result of async_computation().await must be Send - which dyn std::error::Error is not.
This could be mitigated if the non-Send type can be dropped before the .await, such as:
async fn async_start() {
let result;
match async_computation().await {
Ok(v) => result = v,
_ => return,
};
async_foo(result).await;
}
However once another .await is needed before the non-Send type is dropped, the workarounds are more and more awkward.
What a is a good practice for this - especially when the non-Send type is for generic error handling (the Box<dyn std::error::Error>)? Is there a better type for errors that common IO ops implements (async or not) in Rust? Or there is a better way to group nested async calls?
Most errors are Send so you can just change the return type to:
Box<dyn std::error::Error + Send>
It's also common to have + Sync.
I want to establish an http connection to a Ganache test blockchain.
Going through the GitHub page of the web3 crate I found this example:
#[tokio::main]
async fn main() -> web3::Result<()> {
let _ = env_logger::try_init();
let transport = web3::transports::Http::new("http://localhost:7545")?;
let web3 = web3::Web3::new(transport);
let mut accounts = web3.eth().accounts().await?;
...
Ok(())
}
However I want to implement the connection setup in a function. So I tried the following:
async fn establish_web3_connection_http(url: &str) -> web3::Result<Web3<Http>>{
let transport = web3::transports::Http::new(url)?;
Ok(web3::Web3::new(transport))
}
...
#[tokio::main]
async fn main() -> web3::Result<()> {
let web3_con = establish_web3_connection_http("http://localhost:7545");
println!("Calling accounts.");
let mut accounts = web3_con.eth().accounts().await?;
Ok(())
}
This results in the following error:
Error
I am not sure why I do not return the correct value. There is not error when I
don't call web3_con, so the function seems to be fine.
Is the return value somehow wrong, or how I call it?
establish_web3_connection_http() is an async function, so it returns a future. You're trying to call .eth() on the future, when you probably want to call it on the value produced by the future. You need to await the result of this function:
let web3_con = establish_web3_connection_http("http://localhost:7545").await?;
// ^^^^^^^
However, you don't do any awaiting in establish_web3_connection_http(), so there's no reason it needs to be async in the first place. You could just remove async from its signature instead:
fn establish_web3_connection_http(url: &str) -> web3::Result<Web3<Http>>{
I have a struct Test I want to implement std::future::Future that would poll function:
use std::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
struct Test;
impl Test {
async fn function(&mut self) {}
}
impl Future for Test {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
match self.function() {
Poll::Pending => Poll::Pending,
Poll::Ready(_) => Poll::Ready(()),
}
}
}
That didn't work:
error[E0308]: mismatched types
--> src/lib.rs:17:13
|
10 | async fn function(&mut self) {}
| - the `Output` of this `async fn`'s expected opaque type
...
17 | Poll::Pending => Poll::Pending,
| ^^^^^^^^^^^^^ expected opaque type, found enum `Poll`
|
= note: expected opaque type `impl Future`
found enum `Poll<_>`
error[E0308]: mismatched types
--> src/lib.rs:18:13
|
10 | async fn function(&mut self) {}
| - the `Output` of this `async fn`'s expected opaque type
...
18 | Poll::Ready(_) => Poll::Ready(()),
| ^^^^^^^^^^^^^^ expected opaque type, found enum `Poll`
|
= note: expected opaque type `impl Future`
found enum `Poll<_>`
I understand that function must be called once, the returned Future must be stored somewhere in the struct, and then the saved future must be polled. I tried this:
struct Test(Option<Box<Pin<dyn Future<Output = ()>>>>);
impl Test {
async fn function(&mut self) {}
fn new() -> Self {
let mut s = Self(None);
s.0 = Some(Box::pin(s.function()));
s
}
}
That also didn't work:
error[E0277]: the size for values of type `(dyn Future<Output = ()> + 'static)` cannot be known at compilation time
--> src/lib.rs:7:13
|
7 | struct Test(Option<Box<Pin<dyn Future<Output = ()>>>>);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `(dyn Future<Output = ()> + 'static)`
After I call function() I have taken a &mut reference of Test, because of that I can't change the Test variable, and therefore can't store the returned Future inside the Test.
I did get an unsafe solution (inspired by this)
struct Test<'a>(Option<BoxFuture<'a, ()>>);
impl Test<'_> {
async fn function(&mut self) {
println!("I'm alive!");
}
fn new() -> Self {
let mut s = Self(None);
s.0 = Some(unsafe { &mut *(&mut s as *mut Self) }.function().boxed());
s
}
}
impl Future for Test<'_> {
type Output = ();
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
self.0.as_mut().unwrap().poll_unpin(cx)
}
}
I hope that there is another way.
Though there are times when you may want to do things similar to what you're trying to accomplish here, they are a rarity. So most people reading this, maybe even OP, may wish to restructure such that struct state and data used for a single async execution are different objects.
To answer your question, yes it is somewhat possible. Unless you want to absolutely resort to unsafe code you will need to use Mutex and Arc. All fields you wish to manipulate inside the async fn will have to be wrapped inside a Mutex and the function itself will accept an Arc<Self>.
I must stress, however, that this is not a beautiful solution and you probably don't want to do this. Depending on your specific case your solution may vary, but my guess of what OP is trying to accomplish while using Streams would be better solved by something similar to this gist that I wrote.
use std::{
future::Future,
pin::Pin,
sync::{Arc, Mutex},
};
struct Test {
state: Mutex<Option<Pin<Box<dyn Future<Output = ()>>>>>,
// if available use your async library's Mutex to `.await` locks on `buffer` instead
buffer: Mutex<Vec<u8>>,
}
impl Test {
async fn function(self: Arc<Self>) {
for i in 0..16u8 {
let data: Vec<u8> = vec![i]; // = fs::read(&format("file-{}.txt", i)).await.unwrap();
let mut buflock = self.buffer.lock().unwrap();
buflock.extend_from_slice(&data);
}
}
pub fn new() -> Arc<Self> {
let s = Arc::new(Self {
state: Default::default(),
buffer: Default::default(),
});
{
// start by trying to aquire a lock to the Mutex of the Box
let mut lock = s.state.lock().unwrap();
// create boxed future
let b = Box::pin(s.clone().function());
// insert value into the mutex
*lock = Some(b);
} // block causes the lock to be released
s
}
}
impl Future for Test {
type Output = ();
fn poll(
self: std::pin::Pin<&mut Self>,
ctx: &mut std::task::Context<'_>,
) -> std::task::Poll<<Self as std::future::Future>::Output> {
let mut lock = self.state.lock().unwrap();
let fut: &mut Pin<Box<dyn Future<Output = ()>>> = lock.as_mut().unwrap();
Future::poll(fut.as_mut(), ctx)
}
}
I'm not sure what you want to achieve and why, but I suspect that you're trying to implement Future for Test based on some ancient tutorial or misunderstanding and just overcomplicating things.
You don't have to implement Future manually. An async function
async fn function(...) {...}
is really just syntax sugar translated behind the scenes into something like
fn function(...) -> Future<()> {...}
All you have to do is to use the result of the function the same way as any future, e.g. use await on it or call block a reactor until it's finished. E.g. based on your first version, you can simply call:
let mut test = Test{};
test.function().await;
UPDATE1
Based on your descriptions I still think you're trying to overcomplicate this minimal working snippet without the need to manually implement Future for anything:
async fn asyncio() { println!("Doing async IO"); }
struct Test {
count: u32,
}
impl Test {
async fn function(&mut self) {
asyncio().await;
self.count += 1;
}
}
#[tokio::main]
async fn main() {
let mut test = Test{count: 0};
test.function().await;
println!("Count: {}", test.count);
}
Using warp.rs 0.2.2, let's consider a basic web service with one route for GET /:
#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
let getRoot = warp::get().and(warp::path::end()).and_then(routes::getRoot);
warp::serve(getRoot).run(([0, 0, 0, 0], 3030)).await;
Ok(())
}
My goal is to use ? for error handling in the route handlers, so let's write one that can error and return early in crate::routes:
use crate::errors::ServiceError;
use url::Url;
pub async fn getRoot() -> Result<impl warp::Reply, warp::Rejection> {
let _parsed_url = Url::parse(&"https://whydoesn.it/work?").map_err(ServiceError::from)?;
Ok("Hello world !")
}
This version works.
Here the error that's returned by Url::parse() is a url::ParseError
To convert between error types, from url::ParseError to ServiceError, then from ServiceError to warp::Rejection, I've written some error helpers in crate::errors:
#[derive(thiserror::Error, Debug)]
pub enum ServiceError {
#[error(transparent)]
Other(#[from] anyhow::Error), // source and Display delegate to anyhow::Error
}
impl warp::reject::Reject for ServiceError {}
impl From<ServiceError> for warp::reject::Rejection {
fn from(e: ServiceError) -> Self {
warp::reject::custom(e)
}
}
impl From<url::ParseError> for ServiceError {
fn from(e: url::ParseError) -> Self {
ServiceError::Other(e.into())
}
}
Now, the above works, and I'm trying to shorten the second code block to use ? for error handling directly, and convert automatically from the underlying error (here url::ParseError) to a warp::Rejection.
Here's what I've tried:
use crate::errors::ServiceError;
use url::Url;
pub async fn getRoot() -> Result<impl warp::Reply, ServiceError> {
let _parsed_url = Url::parse(&"https://whydoesn.it/work?")?;
Ok("Hello world !")
}
The url::ParseError returned by Url::Parse will convert fine into a ServiceError to return, but returning a ServiceError from my handler doesn't work.
The first compilation error I get is:
error[E0277]: the trait bound `errors::ServiceError: warp::reject::sealed::CombineRejection<warp::reject::Rejection>` is not satisfied
--> src/main.rs:102:54
|
102 | let getRoot = warp::get().and(warp::path::end()).and_then(routes::getRoot);
| ^^^^^^^^ the trait `warp::reject::sealed::CombineRejection<warp::reject::Rejection>` is not implemented for `errors::ServiceError`
Is there a way I can keep the short error handling using ? only and either:
make ServiceError implement warp::reject::sealed::CombineRejection<warp::reject::Rejection> ?
work around that ?
You can implement From to convert your error type into warp::Rejection using reject::custom. Rejection encapsulates custom types which you can later choose to inspect inside of a recover handler.
This example uses a plain error struct, but if you have an error enum you can match on the variants inside the recovery handler and perform different logic as needed.
use serde::Deserialize;
use snafu::{ensure, Snafu};
use std::convert::Infallible;
use warp::{
filters::{any, query, BoxedFilter},
http::StatusCode,
reject::Reject,
Filter, Rejection, Reply,
};
// A normal error type, created by SNAFU
#[derive(Debug, Snafu)]
#[snafu(display("Expected a value less than 10, but it was {}", value))]
struct LessThanTenError {
value: i32,
}
// A function that might fail
fn validate(value: i32) -> Result<i32, LessThanTenError> {
ensure!(value < 10, LessThanTenContext { value });
Ok(value)
}
// We need a custom type to later extract from the `Rejection`. In
// this case, we can reuse the error type itself.
impl Reject for LessThanTenError {}
// To allow using `?`, we implement a conversion from our error to
// `Rejection`
impl From<LessThanTenError> for Rejection {
fn from(other: LessThanTenError) -> Self {
warp::reject::custom(other)
}
}
#[tokio::main]
async fn main() {
let api = simple_math().recover(report_invalid);
let p: std::net::SocketAddr = "0.0.0.0:8888".parse().unwrap();
warp::serve(api).run(p).await;
}
#[derive(Debug, Deserialize)]
struct QueryParams {
a: i32,
b: i32,
}
fn simple_math() -> BoxedFilter<(impl Reply,)> {
any::any()
.and(query::query())
.and_then(|args: QueryParams| async move {
// Look at us using those question marks!
let a = validate(args.a)?;
let b = validate(args.b)?;
let sum = validate(a + b)?;
// We specify that we are returning an error type of
// `Rejection`, which allows the compiler to know what
// type to convert to when using `?` here.
Ok::<_, Rejection>(format!("The sum is {}", sum))
})
.boxed()
}
async fn report_invalid(r: Rejection) -> Result<impl Reply, Infallible> {
if let Some(e) = r.find::<LessThanTenError>() {
// It was our specific error type, do whatever we want. We
// will just print out the error text.
Ok(warp::reply::with_status(
e.to_string(),
StatusCode::BAD_REQUEST,
))
} else {
// Do prettier error reporting for the default error here.
Ok(warp::reply::with_status(
String::from("Something bad happened"),
StatusCode::INTERNAL_SERVER_ERROR,
))
}
}
[dependencies]
serde = { version = "1.0.118", features = ["derive"] }
snafu = "0.6.10"
tokio = { version = "0.2.23", features = ["full"] }
warp = "0.2.5"
% curl 'http://127.0.0.1:8888'
< HTTP/1.1 500 Internal Server Error
Something bad happened
% curl -v 'http://127.0.0.1:8888?a=1&b=2'
< HTTP/1.1 200 OK
The sum is 3
% curl -v 'http://127.0.0.1:8888?a=6&b=5'
< HTTP/1.1 400 Bad Request
Expected a value less than 10, but it was 11
See also:
Is there a way to do validation as part of a filter in Warp?
When should I implement std::convert::From vs std::convert::Into?
How do you define custom `Error` types in Rust?
From my findings, there are two solutions.
Abandon ? in favor of your own macro that constructs and returns a response if there is an error.
Use PR #458 by cjbassi instead of the mainline release by:
Implementing warp::reply::Reply on your error type so that it converts into the correct user facing error message.
Replace warp = "0.2" with warp = { git = "https://github.com/cjbassi/warp.git", branch = "error"} in your Cargo.toml file
use .map_async instead of .and_then for handlers
Rust has async methods that can be tied to Abortable futures. The documentation says that, when aborted:
the future will complete immediately without making any further progress.
Will the variables owned by the task bound to the future be dropped? If those variables implement drop, will drop be called? If the future has spawned other futures, will all of them be aborted in a chain?
E.g.: In the following snippet, I don't see the destructor happening for the aborted task, but I don't know if it is not called or happens in a separate thread where the print is not shown.
use futures::executor::block_on;
use futures::future::{AbortHandle, Abortable};
struct S {
i: i32,
}
impl Drop for S {
fn drop(&mut self) {
println!("dropping S");
}
}
async fn f() -> i32 {
let s = S { i: 42 };
std::thread::sleep(std::time::Duration::from_secs(2));
s.i
}
fn main() {
println!("first test...");
let (abort_handle, abort_registration) = AbortHandle::new_pair();
let _ = Abortable::new(f(), abort_registration);
abort_handle.abort();
std::thread::sleep(std::time::Duration::from_secs(1));
println!("second test...");
let (_, abort_registration) = AbortHandle::new_pair();
let task = Abortable::new(f(), abort_registration);
block_on(task).unwrap();
std::thread::sleep(std::time::Duration::from_secs(1));
}
playground
Yes, values that have been created will be dropped.
In your first example, the future returned by f is never started, so the S is never created. This means that it cannot be dropped.
In the second example, the value is dropped.
This is more obvious if you both run the future and abort it. Here, I spawn two concurrent futures:
create an S and waits 200ms
wait 100ms and abort future #1
use futures::future::{self, AbortHandle, Abortable};
use std::time::Duration;
use tokio::time;
struct S {
i: i32,
}
impl S {
fn new(i: i32) -> Self {
println!("Creating S {}", i);
S { i }
}
}
impl Drop for S {
fn drop(&mut self) {
println!("Dropping S {}", self.i);
}
}
#[tokio::main]
async fn main() {
let create_s = async {
let s = S::new(42);
time::delay_for(Duration::from_millis(200)).await;
println!("Creating {} done", s.i);
};
let (abort_handle, abort_registration) = AbortHandle::new_pair();
let create_s = Abortable::new(create_s, abort_registration);
let abort_s = async move {
time::delay_for(Duration::from_millis(100)).await;
abort_handle.abort();
};
let c = tokio::spawn(create_s);
let a = tokio::spawn(abort_s);
let (c, a) = future::join(c, a).await;
println!("{:?}, {:?}", c, a);
}
Creating S 42
Dropping S 42
Ok(Err(Aborted)), Ok(())
Note that I've switched to Tokio to be able to use time::delay_for, as you should never use blocking operations in an async function.
See also:
Why does Future::select choose the future with a longer sleep period first?
What is the best approach to encapsulate blocking I/O in future-rs?
If the future has spawned other futures, will all of them be aborted in a chain?
No, when you spawn a future, it is disconnected from where it was spawned.
See also:
What is the purpose of async/await in Rust?