Returning a recursive closure in Rust - recursion

I have the following higher-order function
fn ensure_tonicty(tone_fn: &fn(&f64, &f64) -> bool) -> impl Fn(&Vec<f64>) -> bool {
return |floats: &Vec<f64>| -> bool {
let first = floats.first().unwrap();
let rest = &floats[1..];
fn f(tone_fn: &fn(&f64, &f64) -> bool, prev: &f64, xs: &[f64]) -> bool {
match xs.first() {
Some(x) => tone_fn(prev, x) && f(tone_fn, x, &xs[1..]),
None => true,
}
};
return f(tone_fn, first, rest);
};
}
My goal is to return this lambda. I can't figure out how to effectively use tone_fn here though.
The code above errors out:
error[E0621]: explicit lifetime required in the type of `tone_fn`
--> src/lib.rs:1:56
|
1 | fn ensure_tonicty(tone_fn: &fn(&f64, &f64) -> bool) -> impl Fn(&Vec<f64>) -> bool {
| ----------------------- ^^^^^^^^^^^^^^^^^^^^^^^^^^ lifetime `'static` required
| |
| help: add explicit lifetime `'static` to the type of `tone_fn`: `&'static for<'r, 's> fn(&'r f64, &'s f64) -> bool`
If I try to include a lifetime though, I am not sure how to type impl Fn, and include the lifetime
// where do I write `'a`?
fn ensure_tonicty<'a>(tone_fn: &'a fn(&f64, &f64) -> bool) -> impl Fn(&Vec<f64>) -> bool {
I could write this as a macro and get past this, but I'm curious if there's a way I can do this without going the macro route.

You are using a lot of references, which don't seem necessary, and make it harder to figure this all out:
A fn is already a function pointer, so you can pass them around by value instead of using another layer of references. This is easier because a function pointer is 'static.
All those &f64s are immutable, so could be replaced with f64 without changing the logic. This should be the same speed as (or possibly faster than) using a reference.
Once you do that, you won't have many reference left, and it will be clearer which are causing the problem:
fn ensure_tonicty(tone_fn: fn(f64, f64) -> bool) -> impl Fn(&Vec<f64>) -> bool {
|floats: &Vec<f64>| -> bool {
let first = *floats.first().unwrap();
let rest = &floats[1..];
fn f(tone_fn: fn(f64, f64) -> bool, prev: f64, xs: &[f64]) -> bool {
match xs.first() {
Some(&x) => tone_fn(prev, x) && f(tone_fn, x, &xs[1..]),
None => true,
}
};
f(tone_fn, first, rest);
};
}
Now, the error is:
error[E0373]: closure may outlive the current function, but it borrows `tone_fn`, which is owned by the current function
--> src/lib.rs:2:12
|
2 | return |floats: &Vec<f64>| -> bool {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ may outlive borrowed value `tone_fn`
...
11 | return f(tone_fn, first, rest);
| ------- `tone_fn` is borrowed here
|
note: closure is returned here
--> src/lib.rs:2:12
|
2 | return |floats: &Vec<f64>| -> bool {
| ____________^
3 | | let first = *floats.first().unwrap();
4 | | let rest = &floats[1..];
5 | | fn f(tone_fn: fn(f64, f64) -> bool, prev: f64, xs: &[f64]) -> bool {
... |
11 | | return f(tone_fn, first, rest);
12 | | };
| |_____^
help: to force the closure to take ownership of `tone_fn` (and any other referenced variables), use the `move` keyword
|
2 | return move |floats: &Vec<f64>| -> bool {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The help section tells you exactly how to fix it: make the closure move its environment. The result is:
fn ensure_tonicty(tone_fn: fn(f64, f64) -> bool) -> impl Fn(&[f64]) -> bool {
move |floats: &[f64]| -> bool {
let first = floats[0];
let rest = &floats[1..];
fn f(tone_fn: fn(f64, f64) -> bool, prev: f64, xs: &[f64]) -> bool {
match xs.first() {
Some(&x) => tone_fn(prev, x) && f(tone_fn, x, &xs[1..]),
None => true,
}
};
f(tone_fn, first, rest)
}
}
If you return a closure from another function, you will nearly always need this keyword. Otherwise, any variables mentioned in the closure will be references to values that will go out of scope when the function ends. Using the move keyword moves those values so they go wherever the closure goes.
Also notice the other changes I made, to make the code more idiomatic:
Use expressions instead of return keyword.
Use &[f64] instead of &Vec<f64> in function arguments (see Why is it discouraged to accept a reference to a String (&String), Vec (&Vec), or Box (&Box) as a function argument?).

Related

Using !Send object on async function works, but does not work on trait function

Suppose we have a type that is not Send.
struct NotSend {
field: std::rc::Rc<i32>
}
Then, following async function can still take NotSend as its parameter and compiles well:
async fn func(not_send: NotSend) -> i32 {
0
}
But when I define the same function inside of the trait, then anything that implements it does not compile.
#[async_trait]
trait A {
async fn func(not_send: NotSend) -> i32;
}
struct S {
}
#[async_trait]
impl A for S {
async fn func(not_send: NotSend) -> i32 {
0
}
}
This fails with the following message:
error: future cannot be sent between threads safely
--> src/main.rs:23:46
|
23 | async fn func( not_send: NotSend) -> i32 {
| ______________________________________________^
24 | | 0
25 | | }
| |_____^ future created by async block is not `Send`
|
= help: within `impl Future<Output = i32>`, the trait `Send` is not implemented for `Rc<i32>`
note: captured value is not `Send`
--> src/main.rs:23:20
|
23 | async fn func( not_send: NotSend) -> i32 {
| ^^^^^^^^ has type `NotSend` which is not `Send`
= note: required for the cast to the object type `dyn Future<Output = i32> + Send`
What's so different from the naive function and the function in the trait? Why one does works but not the other? Playground Link
It's because async_trait expands to something like Pin<Box<dyn Future>>. If we want the resulting future to be Send, it needs to be Pin<Box<dyn Future + Send>>. But this forces it to be Send, i.e. will error on non-Send futures. The async_trait crate does not have a way to know whether the future is Send (because the implementation of the trait is different from its declaration, and we need to decide at declaration site whether the future will be Send or not), so it opts to use user-defined annotations.
By default, the generated type is Send since most futures needs to be Send. However, as explained in the documentation, you can use #[async_trait(?Send)] to opt this out:
#[async_trait(?Send)]
trait A {
async fn func(not_send: NotSend) -> i32;
}
struct S {
}
#[async_trait(?Send)]
impl A for S {
async fn func(not_send: NotSend) -> i32 {
0
}
}

Why am I getting a lifetime mismatch?

The following function compiles without issues:
async fn filter_con<T, O, F>(arr: Vec<T>, predicate: F) -> Vec<T>
where
O: Future<Output = bool>,
F: for Fn(&T) -> O,
{
join_all(arr.into_iter().map(|it| async {
if predicate(&it).await {
Some(it)
} else {
None
}
}))
.await
.into_iter()
.filter_map(|p| p)
.collect::<Vec<_>>()
}
But I can't find a way to call it properly:
let items_filtered = filter_con(items, filter).await;
Error:
error[E0308]: mismatched types
--> src/lifecycle/querier.rs:101:30
|
101 | let items_filtered = filter_con(items_response, filter).await;
| ^^^^^^^^^^ lifetime mismatch
|
= note: expected associated type `<for<'_> fn(&PairInfo) -> impl futures::Future<Output = bool> {lifecycle::querier::filter} as FnOnce<(&PairInfo,)>>::Output`
found associated type `<for<'_> fn(&PairInfo) -> impl futures::Future<Output = bool> {lifecycle::querier::filter} as FnOnce<(&PairInfo,)>>::Output`
= note: the required lifetime does not necessarily outlive the empty lifetime
note: the lifetime requirement is introduced here
--> src/lifecycle/querier.rs:246:26
|
246 | F: for<'a> Fn(&T) -> O,
| ^
Why is this happening?
Full playground here
You have essentially the same problem as in here
To make it easier to understand I'll rewrite/desugar some of your code. Let's start with the predicate:
async fn filter(x: &Test) -> bool{
x.0 >= 50
}
Is functionally equivalent to:
fn filter(x: &Test) -> impl Future<Output=bool> + '_{
async {
x.0 >= 50
}
}
Now it should be obvious that the lifetime of the returned Future is not 'static, but actually it depends on the lifetime of the parameter x: &Test.
This should make the error more understandable - your predicate is returning some future with some lifetime, which is not mentioned anywhere in the generic type definition - you can see that you don't mention it when you define neither O, nor F:
where
O: Future<Output = bool>,
F: for Fn(&T) -> O,
The problem is that, currently the language does not provide any means to say that O's lifetime depends on F's lifetime. The only easy solution I know is to use the escape hatch - box the future, which will allow you to use the same lifetime in F and O. the downside is that, F will be heap allocated:
use futures::future::{join_all, Future};
use futures::future::BoxFuture;
use futures::FutureExt;
#[derive(Debug)]
struct Test(usize);
#[tokio::main]
async fn main() {
let items = vec![Test(10), Test(100), Test(1000)];
let items_filtered = filter_con(items, filter).await;
println!("{:?}", items_filtered); // should print [100, 1000]
}
fn filter(x: &Test) -> BoxFuture<'_, bool>{
async {
x.0 >= 50
}.boxed()
}
async fn filter_con<T, F>(arr: Vec<T>, predicate: F) -> Vec<T>
where
F: for<'a> Fn(&'a T) -> BoxFuture<'a, bool>,
{
join_all(arr.into_iter().map(|it| async{
if predicate(&it).await {
Some(it)
} else {
None
}
}))
.await
.into_iter()
.filter_map(|p| p)
.collect::<Vec<_>>()
}
Here is a link to the Rust Plauground

Cannot use `impl Future` to store async function in a vector

I am trying to store async functions in a vector, but it seems like impl cannot be used in the vector type definition:
use std::future::Future;
fn main() {
let mut v: Vec<fn() -> impl Future<Output = ()>> = vec![];
v.push(haha);
}
async fn haha() {
println!("haha");
}
error[E0562]: `impl Trait` not allowed outside of function and inherent method return types
--> src/main.rs:4:28
|
4 | let mut v: Vec<fn() -> impl Future<Output = ()>> = vec![];
| ^^^^^^^^^^^^^^^^^^^^^^^^
How do I write the type inside the vector?
I found that there may be a workaround by using a type alias, so I changed the code:
use std::future::Future;
type Haha = impl Future<Output = ()>;
fn main() {
let mut v: Vec<fn() -> Haha> = vec![];
v.push(haha);
}
async fn haha() {
println!("haha");
}
This doesn't work either; this time the error occurs in the type alias:
error[E0658]: `impl Trait` in type aliases is unstable
--> src/main.rs:3:1
|
3 | type Haha = impl Future<Output = ()>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: for more information, see https://github.com/rust-lang/rust/issues/63063
error[E0308]: mismatched types
--> src/main.rs:8:12
|
8 | v.push(haha);
| ^^^^ expected opaque type, found a different opaque type
|
= note: expected type `fn() -> Haha`
found type `fn() -> impl std::future::Future {haha}`
= note: distinct uses of `impl Trait` result in different opaque types
error: could not find defining uses
--> src/main.rs:3:1
|
3 | type Haha = impl Future<Output = ()>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
How do I fix it?
You cannot use the impl Trait this way. To be able to store different types that implement a trait into the same container you have to use dynamic dispatch, by storing something like Box<dyn Trait>.
In your particular case, you do not specify if you want to store the async functions themselves or the future generated by the async functions, the solution would be somewhat different.
To store just the futures, you write a container such as:
let mut v: Vec<Box<dyn Future<Output = ()>>> = vec![];
And then just call the function, box it and store it in the container:
v.push(Box::new(haha()));
If instead you want to store the async function itself, without calling it, you need a container with a double dyn:
let mut v2: Vec<Box<dyn Fn() -> Box<dyn Future<Output = ()>>>> = vec![];
Now, since your haha function does not implement this Fn trait you need an adaptor. A lambda function will do, but don't forget the double Box:
v2.push(Box::new(|| Box::new(haha())));
Unfortunately, with these solutions you will be able to create the vector, but not to .await for your futures. For that you need the futures to implement the Unpin marker. That guarantees to the compiler that the future will not move while it is running (if it did, the implementation would be totally unsafe). You could add the + Unpin requirement to the futures, but async fn are not Unpin so you could not fill the vector. The easiest way to fix it is to use this handy function from std:
pub fn into_pin(boxed: Box<T>) -> Pin<Box<T>>
for f in v2 {
f().into_pin().await;
}
Unfortunately, it is still unstable. Fortunately, there is a From impl that does exactly the same. So you can just write:
for f in v2 {
Pin::from(f()).await;
}
In your comment below you write this code to wait for the futures:
for f in v2 {
async { f().await }
}
Note that an async block itself will evaluate to another future, so here you are just wrapping each future into another future, but nobody is waiting for that one. Actually you'll get a warning about it:
warning: unused implementer of std::future::Future that must be used.
Remember that in order to properly wait for all the futures you will need an async runtime.
rodrigo's answer is correct, but I'd prefer to use Box::pin and bake the Pin type into the API of the collection. This makes using the Future trait object (or closure trait object producing a Future trait object) easier:
use std::{future::Future, pin::Pin};
type PinFutureObj<Output> = Pin<Box<dyn Future<Output = Output>>>;
async fn collection_of_pinned_future_trait_objects() {
let v: Vec<PinFutureObj<()>> = vec![
Box::pin(haha()),
Box::pin(hehe()),
Box::pin(haha()),
Box::pin(hehe()),
];
for f in v {
f.await
}
}
async fn collection_of_closure_trait_objects() {
let v: Vec<Box<dyn Fn() -> PinFutureObj<()>>> = vec![
Box::new(|| Box::pin(haha())),
Box::new(|| Box::pin(hehe())),
Box::new(|| Box::pin(haha())),
Box::new(|| Box::pin(hehe())),
];
for f in v {
f().await
}
}
async fn haha() {
println!("haha");
}
async fn hehe() {
println!("hehe");
}
I'd also start introducing type aliases for the longer types.
In fact, this type alias already exists in the futures crate as LocalBoxFuture and can be created via FutureExt::boxed_local. There's also BoxFuture produced by FutureExt::boxed which adds common trait bounds.
use futures::future::{FutureExt, LocalBoxFuture}; // 0.3.5
async fn collection_of_pinned_future_trait_objects() {
let v: Vec<LocalBoxFuture<'static, ()>> = vec![
haha().boxed_local(),
hehe().boxed_local(),
haha().boxed_local(),
hehe().boxed_local(),
];
for f in v {
f.await
}
}
async fn collection_of_closure_trait_objects() {
let v: Vec<Box<dyn Fn() -> LocalBoxFuture<'static, ()>>> = vec![
Box::new(|| haha().boxed_local()),
Box::new(|| hehe().boxed_local()),
Box::new(|| haha().boxed_local()),
Box::new(|| hehe().boxed_local()),
];
for f in v {
f().await
}
}
async fn haha() {
println!("haha");
}
async fn hehe() {
println!("hehe");
}
See also:
How can I put an async function into a map in Rust?
Why can impl trait not be used to return multiple / conditional types?

How to asynchronously explore a directory and its sub-directories?

I need to explore a directory and all its sub-directories. I can explore the directory easily with recursion in a synchronous way:
use failure::Error;
use std::fs;
use std::path::Path;
fn main() -> Result<(), Error> {
visit(Path::new("."))
}
fn visit(path: &Path) -> Result<(), Error> {
for e in fs::read_dir(path)? {
let e = e?;
let path = e.path();
if path.is_dir() {
visit(&path)?;
} else if path.is_file() {
println!("File: {:?}", path);
}
}
Ok(())
}
When I try to do the same in an asynchronous manner using tokio_fs:
use failure::Error; // 0.1.6
use futures::Future; // 0.1.29
use std::path::PathBuf;
use tokio::{fs, prelude::*}; // 0.1.22
fn visit(path: PathBuf) -> impl Future<Item = (), Error = Error> {
let task = fs::read_dir(path)
.flatten_stream()
.for_each(|entry| {
println!("{:?}", entry.path());
let path = entry.path();
if path.is_dir() {
let task = visit(entry.path());
tokio::spawn(task.map_err(drop));
}
future::ok(())
})
.map_err(Error::from);
task
}
Playground
I get the following error:
error[E0391]: cycle detected when processing `visit::{{opaque}}#0`
--> src/lib.rs:6:28
|
6 | fn visit(path: PathBuf) -> impl Future<Item = (), Error = Error> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: ...which requires processing `visit`...
--> src/lib.rs:6:1
|
6 | fn visit(path: PathBuf) -> impl Future<Item = (), Error = Error> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= note: ...which requires evaluating trait selection obligation `futures::future::map_err::MapErr<impl futures::future::Future, fn(failure::error::Error) {std::mem::drop::<failure::error::Error>}>: std::marker::Send`...
= note: ...which again requires processing `visit::{{opaque}}#0`, completing the cycle
note: cycle used when checking item types in top-level module
--> src/lib.rs:1:1
|
1 | / use failure::Error; // 0.1.6
2 | | use futures::Future; // 0.1.29
3 | | use std::path::PathBuf;
4 | | use tokio::{fs, prelude::*}; // 0.1.22
... |
20| | task
21| | }
| |_^
error[E0391]: cycle detected when processing `visit::{{opaque}}#0`
--> src/lib.rs:6:28
|
6 | fn visit(path: PathBuf) -> impl Future<Item = (), Error = Error> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: ...which requires processing `visit`...
--> src/lib.rs:6:1
|
6 | fn visit(path: PathBuf) -> impl Future<Item = (), Error = Error> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= note: ...which again requires processing `visit::{{opaque}}#0`, completing the cycle
note: cycle used when checking item types in top-level module
--> src/lib.rs:1:1
|
1 | / use failure::Error; // 0.1.6
2 | | use futures::Future; // 0.1.29
3 | | use std::path::PathBuf;
4 | | use tokio::{fs, prelude::*}; // 0.1.22
... |
20| | task
21| | }
| |_^
What is the correct way of exploring a directory and its sub-directories asynchronously while propagating all the errors?
I would make several modifications to rodrigo's existing answer:
Return a Stream from the function, allowing the caller to do what they need with a given file entry.
Return an impl Stream instead of a Box<dyn Stream>. This leaves room for more flexibility in implementation. For example, a custom type could be created that uses an internal stack instead of the less-efficient recursive types.
Return io::Error from the function to allow the user to deal with any errors.
Accept a impl Into<PathBuf> to allow a nicer API.
Create an inner hidden implementation function that uses concrete types in its API.
Futures 0.3 / Tokio 0.2
In this version, I avoided the deeply recursive calls, keeping a local stack of paths to visit (to_visit).
use futures::{stream, Stream, StreamExt}; // 0.3.1
use std::{io, path::PathBuf};
use tokio::fs::{self, DirEntry}; // 0.2.4
fn visit(path: impl Into<PathBuf>) -> impl Stream<Item = io::Result<DirEntry>> + Send + 'static {
async fn one_level(path: PathBuf, to_visit: &mut Vec<PathBuf>) -> io::Result<Vec<DirEntry>> {
let mut dir = fs::read_dir(path).await?;
let mut files = Vec::new();
while let Some(child) = dir.next_entry().await? {
if child.metadata().await?.is_dir() {
to_visit.push(child.path());
} else {
files.push(child)
}
}
Ok(files)
}
stream::unfold(vec![path.into()], |mut to_visit| {
async {
let path = to_visit.pop()?;
let file_stream = match one_level(path, &mut to_visit).await {
Ok(files) => stream::iter(files).map(Ok).left_stream(),
Err(e) => stream::once(async { Err(e) }).right_stream(),
};
Some((file_stream, to_visit))
}
})
.flatten()
}
#[tokio::main]
async fn main() {
let root_path = std::env::args().nth(1).expect("One argument required");
let paths = visit(root_path);
paths
.for_each(|entry| {
async {
match entry {
Ok(entry) => println!("visiting {:?}", entry),
Err(e) => eprintln!("encountered an error: {}", e),
}
}
})
.await;
}
Futures 0.1 / Tokio 0.1
use std::path::PathBuf;
use tokio::{fs, prelude::*}; // 0.1.22
use tokio_fs::DirEntry; // 1.0.6
fn visit(
path: impl Into<PathBuf>,
) -> impl Stream<Item = DirEntry, Error = std::io::Error> + Send + 'static {
fn visit_inner(
path: PathBuf,
) -> Box<dyn Stream<Item = DirEntry, Error = std::io::Error> + Send + 'static> {
Box::new({
fs::read_dir(path)
.flatten_stream()
.map(|entry| {
let path = entry.path();
if path.is_dir() {
// Optionally include `entry` if you want to
// include directories in the resulting
// stream.
visit_inner(path)
} else {
Box::new(stream::once(Ok(entry)))
}
})
.flatten()
})
}
visit_inner(path.into())
}
fn main() {
tokio::run({
let root_path = std::env::args().nth(1).expect("One argument required");
let paths = visit(root_path);
paths
.then(|entry| {
match entry {
Ok(entry) => println!("visiting {:?}", entry),
Err(e) => eprintln!("encountered an error: {}", e),
};
Ok(())
})
.for_each(|_| Ok(()))
});
}
See also:
How do I synchronously return a value calculated in an asynchronous Future in stable Rust?
Your code has two errors:
First, a function returning impl Trait cannot currently be recursive, because the actual type returned would depend on itself.
To make your example work, you need to return a sized type. The simple candidate is a trait object, that is, a Box<dyn Future<...>>:
fn visit(path: PathBuf) -> Box<dyn Future<Item = (), Error = Error>> {
// ...
let task = visit(entry.path());
tokio::spawn(task.map_err(drop));
// ...
Box::new(task)
}
There is still your second error:
error[E0277]: `dyn futures::future::Future<Item = (), Error = failure::error::Error>` cannot be sent between threads safely
--> src/lib.rs:14:30
|
14 | tokio::spawn(task.map_err(drop));
| ^^^^^^^^^^^^^^^^^^ `dyn futures::future::Future<Item = (), Error = failure::error::Error>` cannot be sent between threads safely
|
::: /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/executor/mod.rs:131:52
|
131 | where F: Future<Item = (), Error = ()> + 'static + Send
| ---- required by this bound in `tokio::executor::spawn`
|
= help: the trait `std::marker::Send` is not implemented for `dyn futures::future::Future<Item = (), Error = failure::error::Error>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::ptr::Unique<dyn futures::future::Future<Item = (), Error = failure::error::Error>>`
= note: required because it appears within the type `std::boxed::Box<dyn futures::future::Future<Item = (), Error = failure::error::Error>>`
= note: required because it appears within the type `futures::future::map_err::MapErr<std::boxed::Box<dyn futures::future::Future<Item = (), Error = failure::error::Error>>, fn(failure::error::Error) {std::mem::drop::<failure::error::Error>}>`
This means that your trait object is not Send so it cannot be scheduled for execution in another thread using tokio::spawn(). Fortunately, this is easy to fix: just add + Send to your trait object:
fn visit(path: PathBuf) -> Box<dyn Future<Item = (), Error = Error> + Send> {
//...
}
See the full code in the Playground.

How do I convert an iterator into a stream on success or an empty stream on failure?

I'd like to take a regular iterator and turn it into a stream so that I can do further stream processing. The trouble is that I may have an iterator or an error to deal with. I think I'm pretty close with this:
#[macro_use]
extern crate log;
extern crate futures; // 0.1.21
extern crate tokio;
use futures::prelude::*;
use futures::{future, stream};
use std::fmt::Debug;
use std::net::{SocketAddr, ToSocketAddrs};
fn resolve(addrs: impl ToSocketAddrs + Debug) -> impl Stream<Item = SocketAddr, Error = ()> {
match addrs.to_socket_addrs() {
Ok(iter) => stream::unfold(iter, |iter| match iter.next() {
Some(a) => Some(future::ok((a, iter))),
None => None,
}),
Err(e) => {
error!("could not resolve socket addresses {:?}: {:?}", addrs, e);
stream::empty()
}
}
}
fn main() {
let task = resolve("1.2.3.4:12345")
.map_err(|e| error!("{:?}", e))
.for_each(|addr| info!("{:?}", addr))
.fold();
tokio::run(task);
}
playground
error[E0308]: match arms have incompatible types
--> src/main.rs:12:5
|
12 | / match addrs.to_socket_addrs() {
13 | | Ok(iter) => stream::unfold(iter, |iter| match iter.next() {
14 | | Some(a) => Some(future::ok((a, iter))),
15 | | None => None,
... |
20 | | }
21 | | }
| |_____^ expected struct `futures::stream::Unfold`, found struct `futures::stream::Empty`
|
= note: expected type `futures::stream::Unfold<<impl ToSocketAddrs + Debug as std::net::ToSocketAddrs>::Iter, [closure#src/main.rs:13:42: 16:10], futures::FutureResult<(std::net::SocketAddr, <impl ToSocketAddrs + Debug as std::net::ToSocketAddrs>::Iter), _>>`
found type `futures::stream::Empty<_, _>`
note: match arm with an incompatible type
--> src/main.rs:17:19
|
17 | Err(e) => {
| ___________________^
18 | | error!("could not resolve socket addresses {:?}: {:?}", addrs, e);
19 | | stream::empty()
20 | | }
| |_________^
error[E0277]: the trait bound `(): futures::Future` is not satisfied
--> src/main.rs:27:10
|
27 | .for_each(|addr| info!("{:?}", addr))
| ^^^^^^^^ the trait `futures::Future` is not implemented for `()`
|
= note: required because of the requirements on the impl of `futures::IntoFuture` for `()`
error[E0599]: no method named `fold` found for type `futures::stream::ForEach<futures::stream::MapErr<impl futures::Stream, [closure#src/main.rs:26:18: 26:39]>, [closure#src/main.rs:27:19: 27:45], ()>` in the current scope
--> src/main.rs:28:10
|
28 | .fold();
| ^^^^
|
= note: the method `fold` exists but the following trait bounds were not satisfied:
`&mut futures::stream::ForEach<futures::stream::MapErr<impl futures::Stream, [closure#src/main.rs:26:18: 26:39]>, [closure#src/main.rs:27:19: 27:45], ()> : futures::Stream`
`&mut futures::stream::ForEach<futures::stream::MapErr<impl futures::Stream, [closure#src/main.rs:26:18: 26:39]>, [closure#src/main.rs:27:19: 27:45], ()> : std::iter::Iterator`
The hint is pretty obvious. The two Results I'm returning from the match differ and should be the same. Now, how can I do that so that I return a stream?
Rust is a statically typed language which means that the return type of a function has to be a single type, known at compile time. You are attempting to return multiple types, decided at runtime.
The closest solution to your original is to always return the Unfold stream:
fn resolve(addrs: impl ToSocketAddrs) -> impl Stream<Item = SocketAddr, Error = ()> {
stream::unfold(addrs.to_socket_addrs(), |r| {
match r {
Ok(mut iter) => iter.next().map(|addr| future::ok((addr, Ok(iter)))),
Err(_) => None,
}
})
}
But why reinvent the wheel?
futures::stream::iter_ok
Converts an Iterator into a Stream which is always ready to yield the next value.
Subsequent versions of the futures crate implement Stream for Either, which makes this very elegant:
fn resolve(addrs: impl ToSocketAddrs) -> impl Stream<Item = SocketAddr, Error = ()> {
match addrs.to_socket_addrs() {
Ok(iter) => stream::iter_ok(iter).left_stream(),
Err(_) => stream::empty().right_stream(),
}
}
It's straightforward to backport this functionality to futures 0.1 (maybe someone should submit it as a PR for those who are stuck on 0.1...):
enum MyEither<L, R> {
Left(L),
Right(R),
}
impl<L, R> Stream for MyEither<L, R>
where
L: Stream,
R: Stream<Item = L::Item, Error = L::Error>,
{
type Item = L::Item;
type Error = L::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self {
MyEither::Left(l) => l.poll(),
MyEither::Right(r) => r.poll(),
}
}
}
trait EitherStreamExt {
fn left_stream<R>(self) -> MyEither<Self, R>
where
Self: Sized;
fn right_stream<L>(self) -> MyEither<L, Self>
where
Self: Sized;
}
impl<S: Stream> EitherStreamExt for S {
fn left_stream<R>(self) -> MyEither<Self, R> {
MyEither::Left(self)
}
fn right_stream<L>(self) -> MyEither<L, Self> {
MyEither::Right(self)
}
}
Even better, use the fact that Result is an iterator and Stream::flatten exists:
fn resolve(addrs: impl ToSocketAddrs) -> impl Stream<Item = SocketAddr, Error = ()> {
stream::iter_ok(addrs.to_socket_addrs())
.map(stream::iter_ok)
.flatten()
}
Or if you really want to print errors:
fn resolve(addrs: impl ToSocketAddrs) -> impl Stream<Item = SocketAddr, Error = ()> {
stream::once(addrs.to_socket_addrs())
.map(stream::iter_ok)
.map_err(|e| eprintln!("err: {}", e))
.flatten()
}
See also:
Conditionally return empty iterator from flat_map
Conditionally iterate over one of several possible iterators
What is the correct way to return an Iterator (or any other trait)?

Resources