Suppose I have some data Bar (e.g. database client) which I would like to create only once
but lazily for my structure Foo.
struct Bar;
struct Foo {
bar: Option<Bar>
}
To do this, I check that the field is initialized; if not, I run the async routine.
The result of the routine is then saved as Some to reuse later.
I know, that Option::get_or_insert_with perfectly fits this scenario, but I have to
deal with async, so I do this manually like this.
impl Foo {
pub async fn get_bar(&mut self) -> &Bar {
if let Some(bar) = &self.bar {
return bar;
}
let bar = Self::create_bar().await;
self.bar.insert(bar)
}
/// Long and heavy-resource routine,
/// we want to memoize it.
async fn create_bar() -> Bar {
Bar
}
}
However, this cannot be compiled due to the immutable and mutable borrowing of self.bar.
Is there a way to do this correctly?
Full example.
Interestingly the borrow checker is able to infer better lifetimes by using the ref keyword in your if let so the following works:
pub async fn get_bar(&mut self) -> &Bar {
if let Some(ref bar) = self.bar {
return bar;
}
let bar = Self::create_bar().await;
self.bar.insert(bar)
}
You can use the as_ref() method of Option<T>.
Here's it being used in your get_bar() function:
async fn get_bar(&mut self) -> &Bar {
if self.bar.is_some() {
self.bar.as_ref().unwrap()
} else {
let bar = Self::create_bar().await;
self.bar.insert(bar)
}
}
I posted this as an answer because my reputation is too low. Please let me know if this answer is not suitable.
Related
Is it possible to both read a stream of Futures from a set of JoinHandle<()> tasks and update that set of tasks with new tasks at the same time?
I currently have a Service that runs some long tasks. Only thing is, I would actually like to (if possible) add new tasks in at the same time -- via a flag sent by some type of Receiver channel (not shown below to keep things simple).
Given that in Service::run handles becomes owned by that function, I would lean towards "no", this is not possible. Is this true? If this isn't possible given my setup, is there some way I could tweak the code below to make this possible?
I read in this answer that wrapping HashMap in an Option allows me to use .take() in Service::run since the value needs to be owned in order to call .into_values(). However, problem with this is that .take() consumes the value in the Mutex, leaving None in its wake.
Here is my minimal reproducible example (did not compile this, but should give the idea):
use tokio::{sleep, time::Duration, task::JoinHandle};
use async_std::{Mutex, Arc};
use futures::{
stream::{FuturesUnordered, StreamExt},
Future,
};
type Handles = Arc<Mutex<Option<HashMap<String, JoinHandle<()>>>>>;
fn a_task() -> impl Future<Output = ()> {
async move {
sleep(Duration::from_secs(3)).await;
}
}
fn the_update_task(handles: Handles) -> impl Future<Output = ()> {
async move {
// would like to update `handles` here as I get new data from a channel
// calling .take() in Service::run nukes my handles here :(
}
}
struct Service {
handles: Handles,
}
impl Service {
fn new() -> Self {
let handles = Arc::new(Mutex::new(Some(HashMap::default())));
let handle = tokio::spawn(the_update_task(handles.clone());
Self { handles }
}
async fn add_a_task(&mut self, id: String) {
let handle = tokio::spawn(a_task());
self.handles.lock().await.as_mut().unwrap().insert(id, handle);
}
async fn run(self) {
let Service { handles, .. } = self;
let mut futs = FuturesUnordered::from_iter(
handles.lock().await.take().unwrap().into_values()
);
while let Some(fut) = futs.next().await {
info!("I completed a task! fut:?}");
}
}
}
#[tokio::main]
async fn main() {
let mut srvc = Service::new();
srvc.add_task("1".to_string()).await;
srvc.add_task("2".to_string()).await;
let handle = tokio::spawn(srv.run());
handle.await;
}
I have tried
Using Arc(Mutex(HashMap))
Using Arc(Mutex(Option(HashMap)))
I seem to arrive always at the same conclusion:
I cannot both own handles in Service::run and update handles (even a copy/reference) from other part of the code
Just answering my own question here with the help of #user1937198's comment.
The solution was to update a reference to the FuturesUnordered directly with new tasks, as opposed to being concerned with handles. This simplifies things quite a bit.
use tokio::{sleep, time::Duration, task::JoinHandle};
use async_std::{Mutex, Arc};
use futures::{
stream::{FuturesUnordered, StreamExt},
Future,
};
fn a_task() -> impl Future<Output = ()> {
async move {
sleep(Duration::from_secs(3)).await;
}
}
fn the_update_task(futs: Arc<Mutex<FuturesUnordered>>) -> impl Future<Output = ()> {
async move {
// Just push another task
let fut = tokio::spawn(a_task());
futs.lock().await.push(fut);
}
}
struct Service {
handles: HashMap<String, JoinHandle<()>>,
}
impl Service {
fn new() -> Self {
let handles = HashMap::default();
Self { handles }
}
async fn add_a_task(&mut self, id: String) {
let handle = tokio::spawn(a_task());
self.handles.insert(id, handle);
}
async fn run(self) {
let Service { handles, .. } = self;
let futs = Arc::new(Mutex::new(FuturesUnordered::from_iter(handles.into_values())));
tokio::spawn(the_update_task(futs.clone())).await.unwrap();
while let Some(fut) = futs.lock().await.next().await {
info!("I completed a task! fut:?}");
}
}
}
#[tokio::main]
async fn main() {
let mut srvc = Service::new();
srvc.add_task("1".to_string()).await;
srvc.add_task("2".to_string()).await;
let handle = tokio::spawn(srv.run());
handle.await;
}
I'm quite new to Rust, so this might be quite a beginner question: Assume I want to start some async action that runs in the background and with a function call I want to stop it. The desired API looks like this:
let stop = show_loadbar("loading some data...").await;
// load data...
stop().await;
My code:
pub fn show_loadbar(text: &str) -> Box<dyn FnOnce() -> Box<dyn Future<Output=()>>>
{
let (sender, receiver) = channel::bounded::<()>(0);
let display = task::spawn(async move {
while receiver.try_recv().is_err() {
// show loadbar: xxx.await;
}
// cleanup: yyy.await;
});
// return a function which stops the load bar
Box::new(move || {
Box::new(async {
sender.send(()).await;
display.await;
})
})
}
I played around quite a lot (creating a struct instead of a function and some combinations), but finally, I always get error like this:
error[E0373]: async block may outlive the current function, but it borrows `sender`, which is owned by the current function
--> src/terminal/loading.rs:23:24
|
23 | Box::new(async {
| ________________________^
24 | | sender.send(()).await;
| | ------ `sender` is borrowed here
25 | | display.await;
26 | | })
| |_________^ may outlive borrowed value `sender`
Given the described API, is it even possible to implement the function like this in Rust?
Independent of this, what is the Rust-way to do it? Maybe this interface is absolutely not how it should be done in Rust.
Thank you very much
The immediate error you see can be fixed by changing async to async move, so that it captures sender by value instead of by reference. But trying tu use your code reveals futher issues:
you can't (and probably don't need to) await show_loadbar(), since it's not itself async.
pin the boxed future to be able to await it.
a bounded async_std channel cannot have the capacity of 0 (it panics if given 0);
handle the error returned by sender.send(), e.g. by unwrapping it.
(optionally) get rid of the outer box by returning impl FnOnce(...) instead of Box<dyn FnOnce(...)>.
With these taken into account, the code would look like this:
pub fn show_loadbar(_text: &str) -> impl FnOnce() -> Pin<Box<dyn Future<Output = ()>>> {
let (sender, receiver) = channel::bounded::<()>(1);
let display = task::spawn(async move {
while receiver.try_recv().is_err() {
// show loadbar: xxx.await;
}
// cleanup: yyy.await;
});
// return a function which stops the load bar
|| {
Box::pin(async move {
sender.send(()).await.unwrap();
display.await;
})
}
}
// usage example:
async fn run() {
let cancel = show_loadbar("xxx");
task::sleep(Duration::from_secs(1)).await;
cancel().await;
}
fn main() {
task::block_on(run());
}
In this question someone commented that you could use PhantomData to add a lifetime bound to a raw pointer inside a struct. I thought I'd try doing this on an existing piece of code I've been working on.
Here's our (minimised) starting point. This compiles (playground):
extern crate libc;
use libc::{c_void, free, malloc};
trait Trace {}
struct MyTrace {
#[allow(dead_code)]
buf: *mut c_void,
}
impl MyTrace {
fn new() -> Self {
Self {
buf: unsafe { malloc(128) },
}
}
}
impl Trace for MyTrace {}
impl Drop for MyTrace {
fn drop(&mut self) {
unsafe { free(self.buf) };
}
}
trait Tracer {
fn start(&mut self);
fn stop(&mut self) -> Box<Trace>;
}
struct MyTracer {
trace: Option<MyTrace>,
}
impl MyTracer {
fn new() -> Self {
Self { trace: None }
}
}
impl Tracer for MyTracer {
fn start(&mut self) {
self.trace = Some(MyTrace::new());
// Pretend the buffer is mutated in C here...
}
fn stop(&mut self) -> Box<Trace> {
Box::new(self.trace.take().unwrap())
}
}
fn main() {
let mut tracer = MyTracer::new();
tracer.start();
let _trace = tracer.stop();
println!("Hello, world!");
}
I think that the problem with the above code is that I could in theory move the buf pointer out of a MyTrace and use if after the struct has died. In this case the underlying buffer will have been freed due to the Drop implementation.
By using a PhantomData we can ensure that only references to buf can be obtained, and that the lifetimes of those references are bound to the instances of MyTrace from whence they came.
We can proceed like this (playground):
extern crate libc;
use libc::{c_void, free, malloc};
use std::marker::PhantomData;
trait Trace {}
struct MyTrace<'b> {
#[allow(dead_code)]
buf: *mut c_void,
_phantom: PhantomData<&'b c_void>,
}
impl<'b> MyTrace<'b> {
fn new() -> Self {
Self {
buf: unsafe { malloc(128) },
_phantom: PhantomData,
}
}
}
impl<'b> Trace for MyTrace<'b> {}
impl<'b> Drop for MyTrace<'b> {
fn drop(&mut self) {
unsafe { free(self.buf) };
}
}
trait Tracer {
fn start(&mut self);
fn stop(&mut self) -> Box<Trace>;
}
struct MyTracer<'b> {
trace: Option<MyTrace<'b>>,
}
impl<'b> MyTracer<'b> {
fn new() -> Self {
Self { trace: None }
}
}
impl<'b> Tracer for MyTracer<'b> {
fn start(&mut self) {
self.trace = Some(MyTrace::new());
// Pretend the buffer is mutated in C here...
}
fn stop(&mut self) -> Box<Trace> {
Box::new(self.trace.take().unwrap())
}
}
fn main() {
let mut tracer = MyTracer::new();
tracer.start();
let _trace = tracer.stop();
println!("Hello, world!");
}
But this will give the error:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:53:36
|
53 | Box::new(self.trace.take().unwrap())
| ^^^^^^
|
note: first, the lifetime cannot outlive the lifetime 'b as defined on the impl at 46:1...
--> src/main.rs:46:1
|
46 | impl<'b> Tracer for MyTracer<'b> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= note: ...so that the types are compatible:
expected std::option::Option<MyTrace<'_>>
found std::option::Option<MyTrace<'b>>
= note: but, the lifetime must be valid for the static lifetime...
= note: ...so that the expression is assignable:
expected std::boxed::Box<Trace + 'static>
found std::boxed::Box<Trace>
I have three sub-questions:
Did I understand the motivation for PhantomData in this scenario correctly?
Where is 'static coming from in the error message?
Can this be made to work without changing the interface of stop? Specifically, without adding a lifetime to the return type?
I'm going to ignore your direct question because I believe you arrived at it after misunderstanding several initial steps.
I could in theory move the buf pointer out of a MyTrace and use if after the struct has died
Copy the pointer, not move it, but yes.
By using a PhantomData we can ensure that only references to buf can be obtained
This is not true. It is still equally easy to get a copy of the raw pointer and misuse it even when you add a PhantomData.
Did I understand the motivation for PhantomData in this scenario correctly?
No. PhantomData is used when you want to act like you have a value of some type without actually having it. Pretending to have a reference to something is only useful when there is something to have a reference to. There's no such value to reference in your example.
The Rust docs say something about raw pointers and PhantomData, but I perhaps got it wrong
That example actually shows my point well. The Slice type is intended to behave as if it has a reference to the Vec that it's borrowed from:
fn borrow_vec<'a, T>(vec: &'a Vec<T>) -> Slice<'a, T>
Since this Slice type doesn't actually have a reference, it needs a PhantomData to act like it has a reference. Note that the lifetime 'a isn't just made up out of whole cloth — it's related to an existing value (the Vec). It would cause memory unsafety for the Slice to exist after the Vec has moved, thus it makes sense to include a lifetime of the Vec.
why the commenter in the other question suggested I use PhantomData to improve the type safety of my raw pointer
You can use PhantomData to improve the safety of raw pointers that act as references, but yours doesn't have some existing Rust value to reference. You can also use it for correctness if your pointer owns some value behind the reference, which yours seemingly does. However, since it's a c_void, it's not really useful. You'd usually see it as PhantomData<MyOwnedType>.
Where is 'static coming from in the error message?
Why is adding a lifetime to a trait with the plus operator (Iterator<Item = &Foo> + 'a) needed?
I have a cache-like structure which internally uses a HashMap:
impl Cache {
fn insert(&mut self, k: u32, v: String) {
self.map.insert(k, v);
}
fn borrow(&self, k: u32) -> Option<&String> {
self.map.get(&k)
}
}
Playground with external mutability
Now I need internal mutability. Since HashMap does not implement Copy, my guess is that RefCell is the path to follow. Writing the insert method is straight forward but I encountered problems with the borrow-function. I could return a Ref<String>, but since I'd like to cache the result, I wrote a small Ref-wrapper:
struct CacheRef<'a> {
borrow: Ref<'a, HashMap<u32, String>>,
value: &'a String,
}
This won't work since value references borrow, so the struct can't be constructed. I know that the reference is always valid: The map can't be mutated, because Ref locks the map. Is it safe to use a raw pointer instead of a reference?
struct CacheRef<'a> {
borrow: Ref<'a, HashMap<u32, String>>,
value: *const String,
}
Am I overlooking something here? Are there better (or faster) options? I'm trying to avoid RefCell due to the runtime overhead.
Playground with internal mutability
I'll complement #Shepmaster's safe but not quite as efficient answer with the unsafe version. For this, we'll pack some unsafe code in a utility function.
fn map_option<'a, T, F, U>(r: Ref<'a, T>, f: F) -> Option<Ref<'a, U>>
where
F: FnOnce(&'a T) -> Option<&'a U>
{
let stolen = r.deref() as *const T;
let ur = f(unsafe { &*stolen }).map(|sr| sr as *const U);
match ur {
Some(u) => Some(Ref::map(r, |_| unsafe { &*u })),
None => None
}
}
I'm pretty sure this code is correct. Although the compiler is rather unhappy with the lifetimes, they work out. We just have to inject some raw pointers to make the compiler shut up.
With this, the implementation of borrow becomes trivial:
fn borrow<'a>(&'a self, k: u32) -> Option<Ref<'a, String>> {
map_option(self.map.borrow(), |m| m.get(&k))
}
Updated playground link
The utility function only works for Option<&T>. Other containers (such as Result) would require their own modified copy, or else GATs or HKTs to implement generically.
I'm going to ignore your direct question in favor of a definitely safe alternative:
impl Cache {
fn insert(&self, k: u32, v: String) {
self.map.borrow_mut().insert(k, v);
}
fn borrow<'a>(&'a self, k: u32) -> Option<Ref<'a, String>> {
let borrow = self.map.borrow();
if borrow.contains_key(&k) {
Some(Ref::map(borrow, |hm| {
hm.get(&k).unwrap()
}))
} else {
None
}
}
}
Ref::map allows you to take a Ref<'a, T> and convert it into a Ref<'a, U>. The ugly part of this solution is that we have to lookup in the hashmap twice because I can't figure out how to make the ideal solution work:
Ref::map(borrow, |hm| {
hm.get(&k) // Returns an `Option`, not a `&...`
})
This might require Generic Associated Types (GATs) and even then the return type might be a Ref<Option<T>>.
As mentioned by Shepmaster, it is better to avoid unsafe when possible.
There are multiple possibilities:
Ref::map, with double look-up (as illustrated by Shepmaster's answer),
Ref::map with sentinel value,
Cloning the return value.
Personally, I'd consider the latter first. Store Rc<String> into your map and your method can easily return a Option<Rc<String>> which completely sidesteps the issues:
fn get(&self, k: u32) -> Option<Rc<String>> {
self.map.borrow().get(&k).cloned()
}
As a bonus, your cache is not "locked" any longer while you use the result.
Or, alternatively, you can work-around the fact that Ref::map does not like Option by using a sentinel value:
fn borrow<'a>(&'a self, k: u32) -> Ref<'a, str> {
let borrow = self.map.borrow();
Ref::map(borrow, |map| map.get(&k).map(|s| &s[..]).unwrap_or(""))
}
I'm having a lot of fun playing around with Rust having been a C# programmer for a long time but I have a question around reflection. Maybe I don't need reflection in this case but given that Rust is strongly typed I suspect I do (I would definitely need it in good ol' C#, bless its cotton socks).
I have this situation:
use std::collections::HashMap;
fn invoke_an_unknown_function(
hashmap: HashMap<String, String>,
// Something to denote a function I know nothing about goes here
) {
// For each key in the hash map, assign the value
// to the parameter argument whose name is the key
// and then invoke the function
}
How would I do that? I'm guessing I need to pass in some sort of MethodInfo as the second argument to the function and then poke around with that to get the arguments whose name is the key in the hash map and assign the values but I had a look around for the reflection API and found the following pre-Rust 1.0 documentation:
Module std::reflect
Module std::repr
[rust-dev] Reflection system
None of these give me enough to go on to get started. How would I implement the function I describe above?
Traits are the expected way to implement a fair amount of what reflection is (ab)used for elsewhere.
trait SomeInterface {
fn exposed1(&self, a: &str) -> bool;
fn exposed2(&self, b: i32) -> i32;
}
struct Implementation1 {
value: i32,
has_foo: bool,
}
impl SomeInterface for Implementation1 {
fn exposed1(&self, _a: &str) -> bool {
self.has_foo
}
fn exposed2(&self, b: i32) -> i32 {
self.value * b
}
}
fn test_interface(obj: &dyn SomeInterface) {
println!("{}", obj.exposed2(3));
}
fn main() {
let impl1 = Implementation1 {
value: 1,
has_foo: false,
};
test_interface(&impl1);
}