This question already has answers here:
How can callbacks with captured mutable variables be treated like normal mutable borrows?
(1 answer)
What's the correct way to implement the equivalent of multiple mutable (statically allocated, statically dispatched, etc.) callbacks in Rust?
(1 answer)
Closed 5 years ago.
I am writing a program that uses hlua to call Lua code that pushes to a Vec. The problem is, I have multiple Lua methods that push to the same Vec. Here is a very simplified version of what I am trying to do:
let mut vec = Vec::new();
{
let mut lua = Lua::new();
// these functions do different things in my actual code
lua.set("pushA", hlua::function1(|s: String| {
vec.push(s);
}));
lua.set("pushB", hlua::function1(|s: String| {
vec.push(s);
}));
lua.execute::<()>("...");
}
The compiler errors with this:
error[E0499]: cannot borrow `vec` as mutable more than once at a time
--> src/main.rs:11:42
|
10 | lua.set("pushA", hlua::function1(|s: String| { vec.push(s); }));
| ----------- --- previous borrow occurs due to use of `vec` in closure
| |
| first mutable borrow occurs here
11 | lua.set("pushB", hlua::function1(|s: String| { vec.push(s); }));
| ^^^^^^^^^^^ --- borrow occurs due to use of `vec` in closure
| |
| second mutable borrow occurs here
...
14 | }
| - first borrow ends here
which I understand, but I can't come up with a solution to my problem where this would not occur in one way or another. I am very new with Rust, so this may be a dumb question.
Related
I want to repeat a certain user-provided function multiple times in an async context. Thus, I thought about a non-async closure that produces futures which can be consumed/awaited.
A minimal reproducible example is the following:
let counter = Arc::new(AtomicU64::new(0));
let counter_closure = counter.clone();
let future_producer = move || async {
counter_closure.fetch_add(1, Ordering::SeqCst);
};
but this results in this error
562 | let future_producer = move || async {
| _______________________________-------_^
| | | |
| | | return type of closure `impl Future<Output = ()>` contains a lifetime `'2`
| | lifetime `'1` represents this closure's body
563 | | counter_closure.fetch_add(1, Ordering::SeqCst);
564 | | };
| |_________^ returning this value requires that `'1` must outlive `'2`
How can I solve this problem?
The problem is that the closure owns counter_closure but the futures returned by the closure only reference this value owned by the closure. The signature of closures doesn't have a way to express this. It's akin to the "owning iterator" problem -- like how the Iterator trait doesn't permit expression of the case where the iterator dispenses references tied to its own lifetime, the family of closure traits that can be invoked on a reference to a closure (Fn and FnMut) don't have a way to express that the value returned from the closure borrows from the closure itself.
If the compiler permitted this to compile, it would be possible to create a use-after-free situation by dropping the closure while futures it has previously returned still exist. This would destroy counter_closure while it's still borrowed.
You can fix this by cloning the closure-owned counter_closure on each invocation of the closure before the async block, and moving this clone into the future with async move:
let counter = Arc::new(AtomicU64::new(0));
let counter_closure = counter.clone();
let future_producer = move || {
let counter_inner = counter_closure.clone();
async move {
counter_inner.fetch_add(1, Ordering::SeqCst);
}
};
This uncouples the lifetime of the future from the closure by giving the future its own Arc.
I am trying to create a Stream using a camera with a blocking capture method. The blocking call is wrapped with blocking::unblock.
use futures::stream;
use rscam::{Camera, Config};
fn frame_stream() -> impl stream::Stream {
let mut camera = Camera::new("/dev/video0").unwrap();
camera.start(&Config {
interval: (1, 30),
resolution: (1280, 720),
format: b"H264",
..Default::default()
}).unwrap();
stream::unfold(camera, |c| async move {
let frame = blocking::unblock(|| c.capture().unwrap()).await;
Some((frame, c))
})
}
Compiling gives this error message:
error[E0373]: closure may outlive the current function, but it borrows `c`, which is owned by the current function
--> src/lib.rs:15:39
|
15 | let frame = blocking::unblock(|| c.capture().unwrap()).await;
| ^^ - `c` is borrowed here
| |
| may outlive borrowed value `c`
|
note: function requires argument type to outlive `'static`
--> src/lib.rs:15:21
|
15 | let frame = blocking::unblock(|| c.capture().unwrap()).await;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: to force the closure to take ownership of `c` (and any other referenced variables), use the `move` keyword
|
15 | let frame = blocking::unblock(move || c.capture().unwrap()).await;
| ++++
error[E0505]: cannot move out of `c` because it is borrowed
--> src/lib.rs:17:22
|
15 | let frame = blocking::unblock(|| c.capture().unwrap()).await;
| ------------------------------------------
| | | |
| | | borrow occurs due to use in closure
| | borrow of `c` occurs here
| argument requires that `c` is borrowed for `'static`
16 |
17 | Some((frame, c))
| ^ move out of `c` occurs here
How can I guarantee to the compiler that the reference to c taken in the closure will still be valid? I assume it will be, since execution of the closure is awaited before c is returned.
Solution
stream::unfold(camera, |c| async move {
Some(
blocking::unblock(|| {
(c.capture().unwrap(), c)
}).await
)
})
You could move the camera into the inner closure, then return it once the frame capture is complete:
stream::unfold(camera, |c| async move {
Some(blocking::unblock(|| move {
let frame = c.capture().unwrap()).await;
(frame,c)
})
})
.await does not guarantee liveness. This is the general problem of scoped async tasks. Futures can be canceled at any time. Consider:
let future = async {
let local = 123;
blocking::unblock(|| local).await;
};
// Poll `future`, but only once.
futures::block_on(async move { futures::poll!(future) });
We started a task using local data, then dropped the future. The task continues executing but the local data is gone. For this reason, there is currently no sound way to expose an async API allowing using local data similar to scoped threads. You have to use 'static data, for example by wrapping in Arc.
See also blocking issue #4.
I have a Result<Vec<f64>, _>. When I try to extract a pointer to the actual f64 array, what I observe is that the array dptr points to is a corrupted version of the expected array (the first 10 bytes have been changed).
Why does this happen, and how I can avoid it?
use std::error::Error;
fn main() {
let res: Result<Vec<f64>, Box<dyn Error>> = Ok(vec![1., 2., 3., 4.]);
let dptr: *const f64 = match res {
Ok(v) => &v[0],
Err(_) => std::ptr::null(),
};
assert_eq!(unsafe { *dptr }, 1.0);
}
Result:
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `0.0`,
right: `1.0`', src/main.rs:9:5
Playground
The behaviour of that program is undefined, which can be seen by running it under Miri, which is a Rust interpreter that can sometimes detect Undefined Behavior. (you can do this in the playground by clicking "Tools" (top right) -> "Miri"):
error: Undefined Behavior: pointer to alloc1039 was dereferenced after this allocation got freed
--> src/main.rs:9:25
|
9 | assert_eq!(unsafe { *dptr }, 1.0);
| ^^^^^ pointer to alloc1039 was dereferenced after this allocation got freed
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
What's happening here is a use-after-free: the Ok(v) => &v[0], line moves data from v (and thus res), causing it to be freed. Later, data used for other variable overwrote the existing data (because using a pointer after the memory it points to is freed is Undefined Behaviour).
If you had tried to read the data out of the res the normal way without unsafe, you would have got a compile-time error for this very issue:
error[E0382]: use of partially moved value: `res`
--> src/main.rs:9:10
|
6 | Ok(v) => &v[0],
| - value partially moved here
...
9 | dbg!(res.unwrap()[0]);
| ^^^ value used here after partial move
|
= note: partial move occurs because value has type `Vec<f64>`, which does not implement the `Copy` trait
help: borrow this field in the pattern to avoid moving `res.0`
|
6 | Ok(ref v) => &v[0],
| +++
(playground)
I'm using serde and serde_json 1.0 to decode data from a base64 string:
fn from_base64_str<T: Deserialize>(string: &str) -> T {
let slice = decode_config(string, URL_SAFE).unwrap();
serde_json::from_slice(&slice).unwrap()
}
When I compile, I got this:
error[E0106]: missing lifetime specifier
--> src/main.rs:6:23
|
6 | fn from_base64_str<T: Deserialize>(string: &str) -> T {
| ^^^^^^^^^^^ expected lifetime parameter
Checking the serde doc, Deserialize is defined as:
pub trait Deserialize<'de>: Sized {
So I added the lifetime:
fn from_base64_str<'de, T: Deserialize<'de>>(string: &str) -> T {
let slice = decode_config(string, URL_SAFE).unwrap();
serde_json::from_slice(&slice).unwrap()
}
The compiler then told me:
error: `slice` does not live long enough
--> src/main.rs:11:29
|
11 | serde_json::from_slice(&slice).unwrap()
| ^^^^^ does not live long enough
12 | }
| - borrowed value only lives until here
|
note: borrowed value must be valid for the lifetime 'de as defined on the body at 9:64...
--> src/main.rs:9:65
|
9 | fn from_base64_str<'de, T: Deserialize<'de>>(string: &str) -> T {
| _________________________________________________________________^ starting here...
10 | | let slice = decode_config(string, URL_SAFE).unwrap();
11 | | serde_json::from_slice(&slice).unwrap()
12 | | }
| |_^ ...ending here
I'm using Rust 1.18.0-nightly (452bf0852 2017-04-19). I only know the very basics of lifetimes in Rust, so I'm very confused by the 'de in trait Deserialize.
How can I fix the lifetime error in such function?
I found an answer from Serde issue 891: I should use DeserializeOwned instead of Deserialize.
This section of the Serde website covers Deserialize bounds in detail.
There are two main ways to write Deserialize trait bounds, whether on an impl block or a function or anywhere else.
<'de, T> where T: Deserialize<'de>
This means "T can be deserialized from some lifetime." The caller gets to decide what lifetime that is. Typically this is used when the caller also provides the data that is being deserialized from, for example in a function like serde_json::from_str. In that case the input data must also have lifetime 'de, for example it could be &'de str.
<T> where T: DeserializeOwned
This means "T can be deserialized from any lifetime." The callee gets to decide what lifetime. Usually this is because the data that is being deserialized from is going to be thrown away before the function returns, so T must not be allowed to borrow from it. For example a function that accepts base64-encoded data as input, decodes it from base64, deserializes a value of type T, then throws away the result of base64 decoding. Another common use of this bound is functions that deserialize from an IO stream, such as serde_json::from_reader.
To say it more technically, the DeserializeOwned trait is equivalent to the higher-rank trait bound for<'de> Deserialize<'de>. The only difference is DeserializeOwned is more intuitive to read. It means T owns all the data that gets deserialized.
I am writing a websocket server in Rust using rust-websocket and its Tokio-based async system. I can serve clients just fine, however, I can not figure out how to share mutable state between the clients. Here is some (partial) code demonstrating this issue:
let mut core = Core::new().unwrap();
let handle = core.handle();
let server = Server::bind("localhost:62831", &handle).unwrap();
let mut state = State{
...
};
let f = server.incoming()
.map_err(|InvalidConnection {error, ..}| error)
.for_each(|upgrade, _)| {
let f = upgrade.accept()
.and_then(|s, _| {
let ctx = ClientContext{
// some other per-client values
state: &mut state,
}
...
return s.send(Message::binary(data).into())
.and_then(move |s| Ok(s, ctx)); // this could be the complete wrong way to insert context into the stream
}).and_then(|s, ctx| {
// client handling code here
});
handle.spawn(f
.map_err(...)
.map(...)
);
return Ok(())
});
core.run(f).unwrap();
This code errors with this:
error[E0373]: closure may outlive the current function, but it borrows `**state`, which is owned by the current function
--> src/main.rs:111:27
|
111 | .and_then(|(s, _)| {
| ^^^^^^^^ may outlive borrowed value `**state`
...
114 | state: &mut state,
| ----- `**state` is borrowed here
|
help: to force the closure to take ownership of `**state` (and any other referenced variables), use the `move` keyword, as shown:
| .and_then(move |(s, _)| {
When trying the compiler's suggestion, I get this:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/main.rs:111:27
|
111 | .and_then(move |(s, _)| {
| ^^^^^^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure
error: `state` does not live long enough
--> src/main.rs:114:37
|
114 | state: &mut state,
| ^^^^^ does not live long enough
...
122 | })
| - borrowed value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
I also tried wrapping the state in a RefCell (creating the RefCell right after the state itself), however, the compiler gives a similar move error since it tries to move the RefCell into the closure that creates the client context.
You're pretty close with the RefCell. What you need now is an Rc to wrap that RefCell so you can clone the Rc and not capture the RefCell itself.
let shared_state = Rc::new(RefCell::new(State::new())));
incoming().for_each(move |s, _| {
let shared_state = shared_state.clone(); // Left uncaptured
shared_state.borrow_mut().do_mutable_state_stuff(); // Could panic
});
Note that since you're using Rc's and RefCell's now, you'll likely need to go ahead and convert your ClientContext struct to storing an Rc> instead of a &mut State. It may be possible to keep using &mut State's for some things, but your &mut State's will be tied to the lifetime of the RefMut, and if you keep it alive until the next closure runs, the borrows will panic (or fail if you use the try_ variants).
Also keep in mind if you decide you want to have multiple threads in your reactor, you will just need to change Rc to Arc, and RefCell to Mutex, which is a very natural conversion when it's needed.