I am trying to create a Stream using a camera with a blocking capture method. The blocking call is wrapped with blocking::unblock.
use futures::stream;
use rscam::{Camera, Config};
fn frame_stream() -> impl stream::Stream {
let mut camera = Camera::new("/dev/video0").unwrap();
camera.start(&Config {
interval: (1, 30),
resolution: (1280, 720),
format: b"H264",
..Default::default()
}).unwrap();
stream::unfold(camera, |c| async move {
let frame = blocking::unblock(|| c.capture().unwrap()).await;
Some((frame, c))
})
}
Compiling gives this error message:
error[E0373]: closure may outlive the current function, but it borrows `c`, which is owned by the current function
--> src/lib.rs:15:39
|
15 | let frame = blocking::unblock(|| c.capture().unwrap()).await;
| ^^ - `c` is borrowed here
| |
| may outlive borrowed value `c`
|
note: function requires argument type to outlive `'static`
--> src/lib.rs:15:21
|
15 | let frame = blocking::unblock(|| c.capture().unwrap()).await;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: to force the closure to take ownership of `c` (and any other referenced variables), use the `move` keyword
|
15 | let frame = blocking::unblock(move || c.capture().unwrap()).await;
| ++++
error[E0505]: cannot move out of `c` because it is borrowed
--> src/lib.rs:17:22
|
15 | let frame = blocking::unblock(|| c.capture().unwrap()).await;
| ------------------------------------------
| | | |
| | | borrow occurs due to use in closure
| | borrow of `c` occurs here
| argument requires that `c` is borrowed for `'static`
16 |
17 | Some((frame, c))
| ^ move out of `c` occurs here
How can I guarantee to the compiler that the reference to c taken in the closure will still be valid? I assume it will be, since execution of the closure is awaited before c is returned.
Solution
stream::unfold(camera, |c| async move {
Some(
blocking::unblock(|| {
(c.capture().unwrap(), c)
}).await
)
})
You could move the camera into the inner closure, then return it once the frame capture is complete:
stream::unfold(camera, |c| async move {
Some(blocking::unblock(|| move {
let frame = c.capture().unwrap()).await;
(frame,c)
})
})
.await does not guarantee liveness. This is the general problem of scoped async tasks. Futures can be canceled at any time. Consider:
let future = async {
let local = 123;
blocking::unblock(|| local).await;
};
// Poll `future`, but only once.
futures::block_on(async move { futures::poll!(future) });
We started a task using local data, then dropped the future. The task continues executing but the local data is gone. For this reason, there is currently no sound way to expose an async API allowing using local data similar to scoped threads. You have to use 'static data, for example by wrapping in Arc.
See also blocking issue #4.
Related
I want to repeat a certain user-provided function multiple times in an async context. Thus, I thought about a non-async closure that produces futures which can be consumed/awaited.
A minimal reproducible example is the following:
let counter = Arc::new(AtomicU64::new(0));
let counter_closure = counter.clone();
let future_producer = move || async {
counter_closure.fetch_add(1, Ordering::SeqCst);
};
but this results in this error
562 | let future_producer = move || async {
| _______________________________-------_^
| | | |
| | | return type of closure `impl Future<Output = ()>` contains a lifetime `'2`
| | lifetime `'1` represents this closure's body
563 | | counter_closure.fetch_add(1, Ordering::SeqCst);
564 | | };
| |_________^ returning this value requires that `'1` must outlive `'2`
How can I solve this problem?
The problem is that the closure owns counter_closure but the futures returned by the closure only reference this value owned by the closure. The signature of closures doesn't have a way to express this. It's akin to the "owning iterator" problem -- like how the Iterator trait doesn't permit expression of the case where the iterator dispenses references tied to its own lifetime, the family of closure traits that can be invoked on a reference to a closure (Fn and FnMut) don't have a way to express that the value returned from the closure borrows from the closure itself.
If the compiler permitted this to compile, it would be possible to create a use-after-free situation by dropping the closure while futures it has previously returned still exist. This would destroy counter_closure while it's still borrowed.
You can fix this by cloning the closure-owned counter_closure on each invocation of the closure before the async block, and moving this clone into the future with async move:
let counter = Arc::new(AtomicU64::new(0));
let counter_closure = counter.clone();
let future_producer = move || {
let counter_inner = counter_closure.clone();
async move {
counter_inner.fetch_add(1, Ordering::SeqCst);
}
};
This uncouples the lifetime of the future from the closure by giving the future its own Arc.
I am trying to pass a closure returning Future<Output=bool> to an async function and call this closure as an async predicate (something like an async .filter or other higher-order function).
This predicate receives its input as a reference. I found how to implement it for pure predicates that do not capture their environment:
type BoxFuture<'a, Out> = Pin<Box<dyn Future<Output=Out> + 'a + Send>>;
////////////////////////////////////////////////////////////////////////////////
// 1 -> Closure only uses the inner argument //
////////////////////////////////////////////////////////////////////////////////
/// Run the computation and check that its output is not empty
fn run1<'a>(expected: &'a [u8]) -> impl Future<Output=()> + 'a {
async move {
let is_ok = compute_and_check1(|b: &[u8]| Box::pin(async move {
b.len() > 0 // Only uses the inner argument
}));
let is_ok1 = is_ok.await;
dbg!(is_ok1);
}
}
/// Compute some bytes (may be complex / use async), then check their validity
/// with a user-supplied function, finally do some clean-up and return.
async fn compute_and_check1<F>(check: F) -> bool
where
F: for<'r> FnOnce(&'r [u8]) -> BoxFuture<'r, bool>
{
let bytes = [0u8; 128];
let is_ok = check(&bytes).await;
drop(bytes);
is_ok
}
Playground link
Supporting only non-capturing closures is quite limiting. I would like to use a closure that captures its environment. By changing the bounds of my compute_and_check function, I am able to pass a closure that uses its environment - but not its input:
type BoxFuture<'a, Out> = Pin<Box<dyn Future<Output=Out> + 'a + Send>>;
////////////////////////////////////////////////////////////////////////////////
// 2 -> Closure only uses the outer argument //
////////////////////////////////////////////////////////////////////////////////
/// Run the computation and assume that its output is not empty if `expected` is not empty
fn run2<'a>(expected: &'a [u8]) -> impl Future<Output=()> + 'a {
async move {
let is_ok = compute_and_check2(|b: &[u8]| Box::pin(async move {
expected.len() > 0 // Only uses the environment
}));
let is_ok2 = is_ok.await;
dbg!(is_ok2);
}
}
/// Compute some bytes (may be complex / use async), then check their validity
/// with a user-supplied function, finally do some clean-up and return.
async fn compute_and_check2<'a, F>(check: F) -> bool
where
F: for<'r> FnOnce(&'r [u8]) -> BoxFuture<'a, bool>
{
let bytes = [0u8; 128];
let is_ok = check(&bytes).await;
drop(bytes);
is_ok
}
Playground link
I can write an implementation where the closure uses its input, and one where the closure uses its environment. But not both at the same time.
How can I accept a Future-producing closure that uses references for both its input and environment?
What I would like to write is something like this:
type BoxFuture<'a, Out> = Pin<Box<dyn Future<Output=Out> + 'a + Send>>;
////////////////////////////////////////////////////////////////////////////////
// 3 -> Closure uses both the inner and outer arguments //
////////////////////////////////////////////////////////////////////////////////
/// Run the computation and check its output is the provided expected value
fn run3<'a>(expected: &'a [u8]) -> impl Future<Output=()> + 'a {
async move {
let is_ok = compute_and_check3(|b: &[u8]| Box::pin(async move {
b == expected // Uses both the input and environment
}));
let is_ok2 = is_ok.await;
dbg!(is_ok2);
}
}
/// Compute some bytes (may be complex / use async), then check their validity
/// with a user-supplied function, finally do some clean-up and return.
async fn compute_and_check3<'a, F>(check: F) -> bool
where
F: for<'r> FnOnce(&'r [u8]) -> BoxFuture<'r + 'a, bool>
{
let bytes = [0u8; 128];
let is_ok = check(&bytes).await;
drop(bytes);
is_ok
}
Playground link
This code does not compile because I am asking the closure to return BoxFuture<'r + 'a, bool> but this is not legal syntax:
error[E0226]: only a single explicit lifetime bound is permitted
--> src/main.rs:89:51
|
89 | F: for<'r> FnOnce(&'r [u8]) -> BoxFuture<'r + 'a, bool>
| ^^
From what I understand, the core of my issue is that I need to restrict my Higher-Ranked Trait Bound. Instead of "for any 'r", I want to say "for any 'r that does not outlive 'a" but I don't see how to write it down.
I tried to use two lifetimes and restrictions to my BoxFuture type alias, or define an auxilliary trait but I did not manage to solve this issue because I fail to apply restrictions on the HRTB lifetime.
For completness, here are the errors I get when passing my final closure to compute_and_check1 (only input) and compute_and_check2 (only environment):
Using compute_and_check1 (only input) playground link
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:17:67
|
17 | let is_ok = compute_and_check1(|b: &[u8]| Box::pin(async move {
| ___________________________________________________________________^
18 | | b == expected
19 | | }));
| |_____^
|
note: first, the lifetime cannot outlive the lifetime `'a` as defined on the function body at 15:9...
--> src/main.rs:15:9
|
15 | fn run3<'a>(expected: &'a [u8]) -> impl Future<Output=()> + 'a {
| ^^
note: ...so that the types are compatible
--> src/main.rs:17:67
|
17 | let is_ok = compute_and_check1(|b: &[u8]| Box::pin(async move {
| ___________________________________________________________________^
18 | | b == expected
19 | | }));
| |_____^
= note: expected `(&[u8], &[u8])`
found `(&[u8], &'a [u8])`
note: but, the lifetime must be valid for the anonymous lifetime #1 defined on the body at 17:36...
--> src/main.rs:17:36
|
17 | let is_ok = compute_and_check1(|b: &[u8]| Box::pin(async move {
| ____________________________________^
18 | | b == expected
19 | | }));
| |______^
note: ...so that the expression is assignable
--> src/main.rs:17:47
|
17 | let is_ok = compute_and_check1(|b: &[u8]| Box::pin(async move {
| _______________________________________________^
18 | | b == expected
19 | | }));
| |______^
= note: expected `Pin<Box<dyn Future<Output = bool> + Send>>`
found `Pin<Box<dyn Future<Output = bool> + Send>>`
Using compute_and_check2 (only environment) playground link
error: lifetime may not live long enough
--> src/main.rs:17:47
|
17 | let is_ok = compute_and_check2(|b: &[u8]| Box::pin(async move {
| ________________________________________-____-_^
| | | |
| | | return type of closure is Pin<Box<(dyn Future<Output = bool> + Send + '2)>>
| | let's call the lifetime of this reference `'1`
18 | | b == expected
19 | | }));
| |______^ returning this value requires that `'1` must outlive `'2`
I have also looked into the unboxed_closure nightly feature but did not manage to solve my issue. I would prefer my code to work on stable Rust but it is acceptable for my code to require nightly if it is the only solution.
I am writing a websocket server in Rust using rust-websocket and its Tokio-based async system. I can serve clients just fine, however, I can not figure out how to share mutable state between the clients. Here is some (partial) code demonstrating this issue:
let mut core = Core::new().unwrap();
let handle = core.handle();
let server = Server::bind("localhost:62831", &handle).unwrap();
let mut state = State{
...
};
let f = server.incoming()
.map_err(|InvalidConnection {error, ..}| error)
.for_each(|upgrade, _)| {
let f = upgrade.accept()
.and_then(|s, _| {
let ctx = ClientContext{
// some other per-client values
state: &mut state,
}
...
return s.send(Message::binary(data).into())
.and_then(move |s| Ok(s, ctx)); // this could be the complete wrong way to insert context into the stream
}).and_then(|s, ctx| {
// client handling code here
});
handle.spawn(f
.map_err(...)
.map(...)
);
return Ok(())
});
core.run(f).unwrap();
This code errors with this:
error[E0373]: closure may outlive the current function, but it borrows `**state`, which is owned by the current function
--> src/main.rs:111:27
|
111 | .and_then(|(s, _)| {
| ^^^^^^^^ may outlive borrowed value `**state`
...
114 | state: &mut state,
| ----- `**state` is borrowed here
|
help: to force the closure to take ownership of `**state` (and any other referenced variables), use the `move` keyword, as shown:
| .and_then(move |(s, _)| {
When trying the compiler's suggestion, I get this:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/main.rs:111:27
|
111 | .and_then(move |(s, _)| {
| ^^^^^^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure
error: `state` does not live long enough
--> src/main.rs:114:37
|
114 | state: &mut state,
| ^^^^^ does not live long enough
...
122 | })
| - borrowed value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
I also tried wrapping the state in a RefCell (creating the RefCell right after the state itself), however, the compiler gives a similar move error since it tries to move the RefCell into the closure that creates the client context.
You're pretty close with the RefCell. What you need now is an Rc to wrap that RefCell so you can clone the Rc and not capture the RefCell itself.
let shared_state = Rc::new(RefCell::new(State::new())));
incoming().for_each(move |s, _| {
let shared_state = shared_state.clone(); // Left uncaptured
shared_state.borrow_mut().do_mutable_state_stuff(); // Could panic
});
Note that since you're using Rc's and RefCell's now, you'll likely need to go ahead and convert your ClientContext struct to storing an Rc> instead of a &mut State. It may be possible to keep using &mut State's for some things, but your &mut State's will be tied to the lifetime of the RefMut, and if you keep it alive until the next closure runs, the borrows will panic (or fail if you use the try_ variants).
Also keep in mind if you decide you want to have multiple threads in your reactor, you will just need to change Rc to Arc, and RefCell to Mutex, which is a very natural conversion when it's needed.
This question already has answers here:
How can callbacks with captured mutable variables be treated like normal mutable borrows?
(1 answer)
What's the correct way to implement the equivalent of multiple mutable (statically allocated, statically dispatched, etc.) callbacks in Rust?
(1 answer)
Closed 5 years ago.
I am writing a program that uses hlua to call Lua code that pushes to a Vec. The problem is, I have multiple Lua methods that push to the same Vec. Here is a very simplified version of what I am trying to do:
let mut vec = Vec::new();
{
let mut lua = Lua::new();
// these functions do different things in my actual code
lua.set("pushA", hlua::function1(|s: String| {
vec.push(s);
}));
lua.set("pushB", hlua::function1(|s: String| {
vec.push(s);
}));
lua.execute::<()>("...");
}
The compiler errors with this:
error[E0499]: cannot borrow `vec` as mutable more than once at a time
--> src/main.rs:11:42
|
10 | lua.set("pushA", hlua::function1(|s: String| { vec.push(s); }));
| ----------- --- previous borrow occurs due to use of `vec` in closure
| |
| first mutable borrow occurs here
11 | lua.set("pushB", hlua::function1(|s: String| { vec.push(s); }));
| ^^^^^^^^^^^ --- borrow occurs due to use of `vec` in closure
| |
| second mutable borrow occurs here
...
14 | }
| - first borrow ends here
which I understand, but I can't come up with a solution to my problem where this would not occur in one way or another. I am very new with Rust, so this may be a dumb question.
Is there any way to get the slices of this vector to last long enough so that I can use them in this kind of circular structure?
fn populate_chain(file_path: &str) -> HashMap<String, HashSet<&String>> {
println!("loading...");
let time = util::StopWatch::new();
let mut words = HashMap::new();
{
let f = |mut x: Vec<String>| {
let word = x.pop().unwrap();
words.insert(word, HashSet::new());
};
Csv::process_rows(f, file_path, "\t");
}
let col: Vec<(String, HashSet<&String>)> = words.clone().into_iter().collect();
let m: usize = col.len() - 1;
for i in 0..m {
let ref k: String = col[i].0;
for j in i..m {
let ref nk: String = col[j].0;
if check_link(k, nk) {
words.get_mut(k).unwrap().insert(nk);
words.get_mut(nk).unwrap().insert(k);
}
}
}
time.print_time();
words
}
I'm using the double for loops to chain words together which are related so that they can be quickly looked up later.
Here are the compiler errors...
error: `col` does not live long enough
--> src/main.rs:28:29
|
28 | let ref k: String = col[i].0;
| ^^^ does not live long enough
...
40 | }
| - borrowed value only lives until here
|
note: borrowed value must be valid for the anonymous lifetime #1 defined on the block at 13:72...
--> src/main.rs:13:73
|
13 | fn populate_chain(file_path: &str) -> HashMap<String, HashSet<& String>>{
| ^
error: `col` does not live long enough
--> src/main.rs:30:34
|
30 | let ref nk: String = col[j].0;
| ^^^ does not live long enough
...
40 | }
| - borrowed value only lives until here
|
note: borrowed value must be valid for the anonymous lifetime #1 defined on the block at 13:72...
--> src/main.rs:13:73
|
13 | fn populate_chain(file_path: &str) -> HashMap<String, HashSet<& String>>{
|
I can tell you from your function signature that you're going to have big problems trying to write this function, at least if you want it to contain non-empty HashSets in the result.
fn populate_chain(file_path: &str) -> HashMap<String, HashSet<&String>>
This function signature has references in it; they have elided lifetimes. If you make the inferred lifetimes explicit, it would look like this:
fn populate_chain<'a>(file_path: &'a str) -> HashMap<String, HashSet<&'a String>>
In other words: This function claims that, given some string slice with lifetime 'a, it will give back a collection that holds String objects with lifetime 'a.
But you have no means for allocating such String objects within your code. :(
So, you're stuck; no matter what you put into that function body, you're not going to be able to supply an implementation that returns a non-trivial result with respect to the HashSets.
However, all is not lost. You could, for example, revise your function so that it also takes as an additional parameter a reference to a TypedArena with an appropriate lifetime, and then allocate the strings there. Another (simpler) option would be to use HashSet<String> instead of HashSet<&String>...