I'm having some issues understanding lifetimes in Rust. It may also be the way I implement my design.
error[E0597]: `request` does not live long enough
--> src/service/session/server.rs:25:23
|
25 | let msg_ref = request.get_ref();
| ^^^^^^^ borrowed value does not live long enough
...
32 | let body: Box<dyn Body> = Box::new(signup);
| ---------------- cast requires that `request` is borrowed for `'static`
...
44 | }
| - `request` dropped here while still borrowed
The main source:
#[tonic::async_trait]
impl Session for SessionImplementation {
async fn signup(
&self,
request: Request<SignupRequest>,
) -> Result<Response<SessionResponse>, Status> {
let msg_ref = request.get_ref();
let signup = TxSignup::new(&msg_ref.name, &msg_ref.addr, &msg_ref.pwd);
let body: Box<dyn Body> = Box::new(signup);
let tx = Transaction::new(body);
let mut tx_signup: Box<dyn Tx> = Box::new(tx);
tx_signup.execute();
let response = SessionResponse {
deadline: 0,
cookie: "".to_string(),
status: 0,
};
Ok(Response::new(response))
}
/* more code */
}
Background
The idea is to have a Transaction, that implements Tx { execute(&self), result(&self) ... };. This Transaction has a parameter body of the type Box<dyn Box>, being the trait Body { /*some fn*/ }. Having this, I'm pretending to implement some kind of hierarchy.
The code above
On line 24 I'm getting some requests of the type SignupRequest (from proto file). This is the implementation of proto's server, using Tonic and Tokio.
After this, I have also an object TxSignup with some parameters of the type &str (set in from line 27 till 29). TxSignup implements the Body trait so I'm able to turn it into a Tx trait, apparently. The Transaction object wraps a Body implementation. I call the execute() function from the given trait Tx. All that explained has been done from line 32 till 35.
The problem
If I replace the &str type from TxSignup by type String it works. However, if I want them to be of the type &str, a lot of "incongruencies" with lifetimes emerge. I want it to be &str because none of these values will change. I think it is better to keep them on the stack instead of in the heap. Am I wrong?
If I want &str, I'm coerced to define TxSignup with <'a>, and here is where I get lost. I get why a lifetime is required, but not why all these problems appear.
As far I do understand, all elements inside the function should have the same lifetime, being killed at the end of its block (line 44). I will never send them outside.
I have tried giving to Body trait also an <'a>, and even to Tx trait (meaning the Transaction object must have one too to match the trait).
Is there any way to make it work? Am I misunderstanding the Trait use and how they work, or this patter design will never work?
Reproduction on GitHub
I have reproduced this same error in my rust-proto repository. Running cargo run should be enough.
I come from Go development and some C++, Java and Python, so I have a way of coding that may not be the most appropriate one using Rust. That's what I want to solve.
Related
Let's begin with a canonical example of Arc
use std::sync::{Arc, Mutex};
use std::thread;
fn main() {
let msg = Arc::new(Mutex::new(String::new()));
let mut handles = Vec::new();
for _ in 1..10 {
let local_msg = Arc::clone(&msg);
handles.push(thread::spawn(move || {
let mut locked = local_msg.lock().unwrap();
locked.push_str("hello, world\n");
}));
}
for handle in handles {
handle.join().unwrap();
}
println!("{}", msg.lock().unwrap());
}
This compiles and runs as expected. Then I realized maybe the Mutex doesn't have to live on the heap and started wondering if I can get rid of Arc and just use a shared reference to a Mutex allocated on the stack. Here is my attempt
use std::sync::Mutex;
use std::thread;
fn main() {
let msg = Mutex::new(String::new());
let mut handles = Vec::new();
for _ in 1..10 {
let local_msg = &msg;
handles.push(thread::spawn(move || {
let mut locked = local_msg.lock().unwrap();
locked.push_str("hello, world\n");
}));
}
for handle in handles {
handle.join().unwrap();
}
println!("{}", msg.lock().unwrap());
}
This one doesn't compile, though
error[E0597]: `msg` does not live long enough
--> src/main.rs:8:25
|
8 | let local_msg = &msg;
| ^^^^ borrowed value does not live long enough
9 | handles.push(thread::spawn(move || {
| ______________________-
10 | | let mut locked = local_msg.lock().unwrap();
11 | | locked.push_str("hello, world\n");
12 | | }));
| |__________- argument requires that `msg` is borrowed for `'static`
...
20 | }
| - `msg` dropped here while still borrowed
error: aborting due to previous error
For more information about this error, try `rustc --explain E0597`.
error: could not compile `hello`
To learn more, run the command again with --verbose.
The compiler complains that local_msg doesn't have a 'static lifetime. Well, it doesn't, so the error makes sense. However, this implies the variable let local_msg = Arc::clone(&msg); in the first snippet has 'static lifetime, otherwise I should get a similar error.
Questions:
How could Arc::clone(&msg) get a 'static lifetime? The value it points to isn't known at compile-time, and could die before the whole program exits.
As a bonus, what about other heap-backed smart pointers like Box and Rc? Do they all have a 'static lifetime because the borrow checker ensures that as long as these pointers are visible, then the addresses they point to are always valid?
The thing the compiler is looking for is a lifetime bound. A lifetime bound of 'a doesn't mean “this type is a reference with lifetime 'a”, but rather “all of the references this type contains have lifetimes of at least 'a”.
(When a lifetime bound is written explicitly, it looks like where T: 'a.)
Thus, any type which does not contain any references (or rather, has no lifetime parameters) automatically satisfies the 'static lifetime bound. If T: 'static, then Arc<T>: 'static (and the same for Box and Rc).
How could Arc::clone(&msg) get a 'static lifetime? The value it points to isn't known at compile-time, and could die before the whole program exits.
It does not point to the value using a reference, so it's fine. The type of your value is Arc<Mutex<String>>; there are no lifetime parameters here because there are no references. If it were, hypothetically, Arc<'a, Mutex<String>> (a lifetime parameter which Arc doesn't actually have), then that type would not satisfy the bound.
The job of Arc (or Rc or Box) is to own the value it points to. Ownership is not a reference and thus not subject to lifetimes.
However, if you had the type Arc<Mutex<&'a str>> then that would not satisfy the bound, because it contains a reference which is not 'static.
I'm very new to Rust and system languages in general. And I'm currently playing around with Rust to explore the language. I've a problem that I cannot fix by myself. And I think I've understanding problem whats going on.
I wan't to store objects that implements the trait BaseStuff in a vector. In Rust not a simple task for me :-).
Here is my example code that won't compile.
trait BaseStuff {}
struct MyStuff {
value: i32,
}
struct AwesomeStuff {
value: f32,
text: String,
}
impl BaseStuff for MyStuff {}
impl BaseStuff for AwesomeStuff {}
struct Holder {
stuff: Vec<BaseStuff>,
}
impl Holder {
fn register(&mut self, data: impl BaseStuff) {
self.stuff.push(data);
}
}
fn main() {
let my_stuff = MyStuff { value: 100 };
let awesome_stuff = AwesomeStuff {
value: 100.0,
text: String::from("I'm so awesome!"),
};
let mut holder = Holder { stuff: vec![] };
holder.register(my_stuff);
}
error[E0277]: the size for values of type (dyn BaseStuff + 'static)
cannot be known at compilation time --> src\main.rs:17:5 | 17 |
stuff: Vec, // unknown size | ^^^^^^^^^^^^^^^^^^^^^
doesn't have a size known at compile-time | = help: the trait
std::marker::Sized is not implemented for (dyn BaseStuff +
'static) = note: to learn more, visit
https://doc.rust-lang.org/book/second-edition/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait
= note: required by std::vec::Vec
error: aborting due to previous error
For more information about this error, try rustc --explain E0277.
error: Could not compile playground.
The compiler message is clear and I understand the message. I can
implement the trait BaseStuff in any struct I want't so its unclear
which size it is. Btw the link isn't helpful because its pointing to
outdated site...
The size of String is also unknown, but the String implements the trait std::marker::Sized and that's why a Vec<String> will work without problems. Is that correct?
In the rust book I read for data types with unknown size I've to store these data on the heap instead of the stack. I changed my code as follows.
struct Holder {
stuff: Vec<Box<BaseStuff>>,
}
impl Holder {
fn register(&mut self, data: impl BaseStuff) {
self.stuff.push(Box::new(data));
}
}
Now I'm hitting a new compiler issue:
error[E0310]: the parameter type impl BaseStuff may not live long
enough --> src\main.rs:22:25 | 21 | fn register(&mut self,
data: impl BaseStuff) { |
--------------- help: consider adding an explicit lifetime bound impl BaseStuff: 'static... 22 | self.stuff.push(Box::new(data));
| ^^^^^^^^^^^^^^ | note: ...so that the
type impl BaseStuff will meet its required lifetime bounds -->
src\main.rs:22:25 | 22 | self.stuff.push(Box::new(data));
| ^^^^^^^^^^^^^^
error: aborting due to previous error
For more information about this error, try rustc --explain E0310.
error: Could not compile playground.
And know I'm out... I read in the book about lifetimes and changed my code a lot with 'a here and 'a in any combination but without luck... I don't want to write down any lifetime definition combination I tried.
I don't understand why I need the lifetime definition. The ownership is moved in any step so for my understanding its clear that Holder struct is the owner for all data. Is it?
How can I correct my code to compile?
Thanks for help.
You almost got it - the issue here is that the type for which BaseStuff is implemented may be a reference (e.g. impl BaseStuff for &SomeType). This means that even though you're passing data by value, the value may be a reference that will be outlived by your Box.
The way to fix this is to add a constraint such that the object has a 'static lifetime, meaning it will either be a value type or a static reference. You can apply this constraint to the trait or the method accepting the trait, depending on your use case.
Applying the constraint to the trait:
trait BaseStuff: 'static {}
Applying the constraint to the method:
impl Holder {
fn register(&mut self, data: impl BaseStuff + 'static) {
self.stuff.push(Box::new(data));
}
}
If you add the 'static constraint to the method, I would recommend also adding it to the Vec to avoid losing type information, like so:
struct Holder {
stuff: Vec<Box<dyn BaseStuff + 'static>>,
}
I'm using serde and serde_json 1.0 to decode data from a base64 string:
fn from_base64_str<T: Deserialize>(string: &str) -> T {
let slice = decode_config(string, URL_SAFE).unwrap();
serde_json::from_slice(&slice).unwrap()
}
When I compile, I got this:
error[E0106]: missing lifetime specifier
--> src/main.rs:6:23
|
6 | fn from_base64_str<T: Deserialize>(string: &str) -> T {
| ^^^^^^^^^^^ expected lifetime parameter
Checking the serde doc, Deserialize is defined as:
pub trait Deserialize<'de>: Sized {
So I added the lifetime:
fn from_base64_str<'de, T: Deserialize<'de>>(string: &str) -> T {
let slice = decode_config(string, URL_SAFE).unwrap();
serde_json::from_slice(&slice).unwrap()
}
The compiler then told me:
error: `slice` does not live long enough
--> src/main.rs:11:29
|
11 | serde_json::from_slice(&slice).unwrap()
| ^^^^^ does not live long enough
12 | }
| - borrowed value only lives until here
|
note: borrowed value must be valid for the lifetime 'de as defined on the body at 9:64...
--> src/main.rs:9:65
|
9 | fn from_base64_str<'de, T: Deserialize<'de>>(string: &str) -> T {
| _________________________________________________________________^ starting here...
10 | | let slice = decode_config(string, URL_SAFE).unwrap();
11 | | serde_json::from_slice(&slice).unwrap()
12 | | }
| |_^ ...ending here
I'm using Rust 1.18.0-nightly (452bf0852 2017-04-19). I only know the very basics of lifetimes in Rust, so I'm very confused by the 'de in trait Deserialize.
How can I fix the lifetime error in such function?
I found an answer from Serde issue 891: I should use DeserializeOwned instead of Deserialize.
This section of the Serde website covers Deserialize bounds in detail.
There are two main ways to write Deserialize trait bounds, whether on an impl block or a function or anywhere else.
<'de, T> where T: Deserialize<'de>
This means "T can be deserialized from some lifetime." The caller gets to decide what lifetime that is. Typically this is used when the caller also provides the data that is being deserialized from, for example in a function like serde_json::from_str. In that case the input data must also have lifetime 'de, for example it could be &'de str.
<T> where T: DeserializeOwned
This means "T can be deserialized from any lifetime." The callee gets to decide what lifetime. Usually this is because the data that is being deserialized from is going to be thrown away before the function returns, so T must not be allowed to borrow from it. For example a function that accepts base64-encoded data as input, decodes it from base64, deserializes a value of type T, then throws away the result of base64 decoding. Another common use of this bound is functions that deserialize from an IO stream, such as serde_json::from_reader.
To say it more technically, the DeserializeOwned trait is equivalent to the higher-rank trait bound for<'de> Deserialize<'de>. The only difference is DeserializeOwned is more intuitive to read. It means T owns all the data that gets deserialized.
I am writing a websocket server in Rust using rust-websocket and its Tokio-based async system. I can serve clients just fine, however, I can not figure out how to share mutable state between the clients. Here is some (partial) code demonstrating this issue:
let mut core = Core::new().unwrap();
let handle = core.handle();
let server = Server::bind("localhost:62831", &handle).unwrap();
let mut state = State{
...
};
let f = server.incoming()
.map_err(|InvalidConnection {error, ..}| error)
.for_each(|upgrade, _)| {
let f = upgrade.accept()
.and_then(|s, _| {
let ctx = ClientContext{
// some other per-client values
state: &mut state,
}
...
return s.send(Message::binary(data).into())
.and_then(move |s| Ok(s, ctx)); // this could be the complete wrong way to insert context into the stream
}).and_then(|s, ctx| {
// client handling code here
});
handle.spawn(f
.map_err(...)
.map(...)
);
return Ok(())
});
core.run(f).unwrap();
This code errors with this:
error[E0373]: closure may outlive the current function, but it borrows `**state`, which is owned by the current function
--> src/main.rs:111:27
|
111 | .and_then(|(s, _)| {
| ^^^^^^^^ may outlive borrowed value `**state`
...
114 | state: &mut state,
| ----- `**state` is borrowed here
|
help: to force the closure to take ownership of `**state` (and any other referenced variables), use the `move` keyword, as shown:
| .and_then(move |(s, _)| {
When trying the compiler's suggestion, I get this:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/main.rs:111:27
|
111 | .and_then(move |(s, _)| {
| ^^^^^^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure
error: `state` does not live long enough
--> src/main.rs:114:37
|
114 | state: &mut state,
| ^^^^^ does not live long enough
...
122 | })
| - borrowed value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
I also tried wrapping the state in a RefCell (creating the RefCell right after the state itself), however, the compiler gives a similar move error since it tries to move the RefCell into the closure that creates the client context.
You're pretty close with the RefCell. What you need now is an Rc to wrap that RefCell so you can clone the Rc and not capture the RefCell itself.
let shared_state = Rc::new(RefCell::new(State::new())));
incoming().for_each(move |s, _| {
let shared_state = shared_state.clone(); // Left uncaptured
shared_state.borrow_mut().do_mutable_state_stuff(); // Could panic
});
Note that since you're using Rc's and RefCell's now, you'll likely need to go ahead and convert your ClientContext struct to storing an Rc> instead of a &mut State. It may be possible to keep using &mut State's for some things, but your &mut State's will be tied to the lifetime of the RefMut, and if you keep it alive until the next closure runs, the borrows will panic (or fail if you use the try_ variants).
Also keep in mind if you decide you want to have multiple threads in your reactor, you will just need to change Rc to Arc, and RefCell to Mutex, which is a very natural conversion when it's needed.
I'm looking for a class much like Vec<RefCell<T>>, in that it is the ultimate owner & allocator of all of its data, yet different pieces of the array can be be mutably borrowed by multiple parties indefinitely.
I emphasize indefinitely because of course pieces of Vec<T> can also be mutably borrowed by multiple parties, but doing so involves making a split which can only be resolved after the parties are done borrowing.
Vec<RefCell<T>> seems to be a world of danger and many ugly if statements checking borrow_state, which seems to be unstable. If you do something wrong, then kablammo! Panic! This is not what a lending library is like. In a lending library, if you ask for a book that isn't there, they tell you "Oh, it's checked out." Nobody dies in an explosion.
So I would like to write code something like this:
let mut a = LendingLibrary::new();
a.push(Foo{x:10});
a.push(Foo{x:11});
let b1 = a.get(0); // <-- b1 is an Option<RefMut<Foo>>
let b2 = a.get(1); // <-- b2 is an Option<RefMut<Foo>>
// the 0th element has already been borrowed, so...
let b3 = a.get(0); // <-- b3 is Option::None
Does such a thing exist? Or is there another canonical way to get this kind of behavior? A kind of "friendly RefCell"?
If the answer happens to be yes, is there also a threadsafe variant?
RefCell is not designed for long-lived borrows. The typical use case is that in a function, you'll borrow the RefCell (either mutably or immutably), work with the value, then release the borrow before returning. I'm curious to know how you're hoping to recover from a borrowed RefCell in a single-threaded context.
The thread-safe equivalent to RefCell is RwLock. It has try_read and try_write functions that do not block or panic if an incompatible lock is still acquired (on any thread, including the current thread). Contrarily to RefCell, it makes sense to just retry later if locking a RwLock fails, since another thread might just happen to have locked it at the same time.
If you end up always using write or try_write, and never read or try_read, then you should probably use the simpler Mutex instead.
#![feature(borrow_state)]
use std::cell::{RefCell, RefMut, BorrowState};
struct LendingLibrary<T> {
items: Vec<RefCell<T>>
}
impl<T> LendingLibrary<T> {
fn new(items: Vec<T>) -> LendingLibrary<T> {
LendingLibrary {
items: items.into_iter().map(|e| RefCell::new(e)).collect()
}
}
fn get(&self, item: usize) -> Option<RefMut<T>> {
self.items.get(item)
.and_then(|cell| match cell.borrow_state() {
BorrowState::Unused => Some(cell.borrow_mut()),
_ => None
})
}
}
fn main() {
let lib = LendingLibrary::new(vec![1, 2, 3]);
let a = lib.get(0); // Some
let b = lib.get(1); // Some
let a2 = lib.get(0); // None
}
This currently requires a nightly release to work.