rust futures::select modifying local variable in loop - asynchronous

Question

I have a struct G like this
struct G { /* some member */ }
impl G {
async fn ref_foo(&self) { /* some code uses G's member */ }
async fn mut_foo(&mut self) { /* some code modifies G's member */ }
}
which is responsible to handle requests from a mpsc::Receiver, like this -- which won't compile:
async fn run_loop(mut rx: impl Stream<Item = Task> + FusedStream + Unpin) {
let mut g = G {};
let mut QS = FuturesUnordered::new();
loop {
select! {
t = rx.select_next_some() => match t {
TR {..} => QS.push(g.ref_foo()),
TM {..} => QS.push(g.mut_foo()),
},
_ = QS.select_next_some() => {},
}
}
}
the above code won't compile due to multiple mutable reference to g.
Target:
What I want is that the loop runs in parallel for any number of ref_foo tasks, and when it needs to run a mut_foo task, it waits until every ref_foo task finish, and then run the mut_foo task, then it can run other tasks as usual.
/ g.ref_foo() \ / ...
| g.ref_foo() | | ...
g.mut_foo() => < g.ref_foo() > => g.mut_foo() => g.mut_foo() => < ...
| g.ref_foo() | | ...
\ g.ref_foo() / \ ...
Additional Infomation:
I used to move implementation of mut_foo to the select loop, and remove async on g.mut_foo() so that no mutable reference would be used in stream QS.
But this implementation is really cubersome and undoubtedly broke G's design.
Just now, I come up with another implementation by make a wrapper:
async fn run_task(mut g: G, t: Task) -> G {
match t {
TR {..} => g.ref_foo().await,
TM {..} => g.mut_foo().await,
};
g
}
while in the select loop:
async fn run_loop(mut rx: impl Stream<Item = Task> + FusedStream + Unpin) {
let g0 = G {};
let mut QS = FuturesUnordered::new();
let mut getter = FuturesUnordered::new();
getter.push(ready(g0));
loop {
select! {
t = rx.select_next_some() => {
let mut g = getter.select_next_some().await;
QS.push(run_task(g, t));
},
mut g = QS.select_next_some() => getter.push(ready(g)),
}
}
}
this one compiles, but it's not so "async" as it can possibly be. In this implementation, ref_foo tasks are also running sequentially.
Question:
Are there more material I should learn to solve this problem? The technics I'm using comes from rust-async-book
Do I HAVE TO use RefCell to solve this problem? IMHO, this should be a trivial problem that can be solved without breaking rust's borrowing rules (by using RefCell).
Can I change my wrap run_task and the select loop so that ref_foo runs in parallel? I have problem in the implementation because G is flowing getter => QS => getter => ..., there's no long-term G instance, and I cannot figure out where I can store it.
Append some of my thoughts:
Since mut_foo can not be run in parallel, I am trying to solve this problem by removing async keyword on mut_foo -- with little progress. The core problem is that, immutable ref to G is needed for parallel running of ref_foo, but I have to get rid of all these immutable ref G when it's time for mut_foo. The fact do not change whether mut_foo is async or not ( or "whether mut_foo returns ref G or not").

Answer 1

I've solved Question 3 with a lot of if statements. I hope there are some more elegant implementations. And, I really appreciate any learning material as stated in Question 1.
here's the full code that compiles (simplified):
use tokio::runtime;
use std::thread;
use std::time::Duration;
use futures::{
select, StreamExt, SinkExt,
future::{ready},
stream::{FusedStream, FuturesUnordered, Stream},
};
struct G;
impl G {
async fn ref_foo(&self) { println!("ref_foo +++"); tokio::time::sleep(Duration::from_millis(500)).await; println!("ref_foo ---"); }
async fn mut_foo(&mut self) { println!("mut_foo +++"); tokio::time::sleep(Duration::from_millis(500)).await; println!("mut_foo ---"); }
}
#[derive(Clone)]
enum Task {
TR,
TM,
}
// wrappers
async fn run_ref_task(g: &G, task: Task) {
match task {
Task::TR => g.ref_foo().await,
_ => {},
};
}
async fn run_mut_task(mut g: G, task: Task) -> G {
match task {
Task::TM => g.mut_foo().await,
_ => {},
};
g
}
async fn run_loop(mut rx: impl Stream<Item = Task> + FusedStream + Unpin) {
let g0 = G;
let mut getter = FuturesUnordered::new();
getter.push(ready(g0));
// the following streams stores only `ready(task)`
let mut mut_tasks = FuturesUnordered::new(); // for tasks that's scheduled in this loop
let mut ref_tasks = FuturesUnordered::new();
let mut mut_delay = FuturesUnordered::new(); // for tasks that's scheduled in next loop
let mut ref_delay = FuturesUnordered::new();
loop {
println!("============ avoid idle loops ============");
let g = getter.select_next_some().await;
{
let mut queries = FuturesUnordered::new(); // where we schedule ref_foo tasks
loop {
println!("------------ avoid idle ref_task loops ------------");
select! {
task = rx.select_next_some() => {
match &task {
Task::TR => ref_delay.push(ready(task)),
Task::TM => mut_tasks.push(ready(task)),
};
if mut_delay.is_empty() && ref_tasks.is_empty() && queries.is_empty() { break; }
},
task = mut_delay.select_next_some() => {
mut_tasks.push(ready(task));
if mut_delay.is_empty() && ref_tasks.is_empty() && queries.is_empty() { break; }
}
task = ref_tasks.select_next_some() => {
queries.push(run_ref_task(&g, task));
}
_ = queries.select_next_some() => {
if mut_delay.is_empty() && ref_tasks.is_empty() && queries.is_empty() { break; }
},
}
}
}
getter.push(ready(g));
{
let mut queries = FuturesUnordered::new(); // where we schedule mut_foo tasks
loop {
println!("------------ avoid idle mut_task loops ------------");
select! {
task = rx.select_next_some() => {
match &task {
Task::TR => ref_tasks.push(ready(task)),
Task::TM => mut_delay.push(ready(task)),
};
if ref_delay.is_empty() && mut_tasks.is_empty() && queries.is_empty() { break; }
},
task = ref_delay.select_next_some() => {
ref_tasks.push(ready(task));
if ref_delay.is_empty() && mut_tasks.is_empty() && queries.is_empty() { break; }
}
g = getter.select_next_some() => {
if let Some(task) = mut_tasks.next().await {
queries.push(run_mut_task(g, task));
} else {
getter.push(ready(g));
if ref_delay.is_empty() && queries.is_empty() { break; }
}
}
g = queries.select_next_some() => {
getter.push(ready(g));
if ref_delay.is_empty() && mut_tasks.is_empty() && queries.is_empty() { break; }
}
}
}
}
}
}
fn main() {
let (mut tx, rx) = futures::channel::mpsc::channel(10000);
let th = thread::spawn(move || thread_main(rx));
let tasks = vec![Task::TR, Task::TR, Task::TM, Task::TM, Task::TR, Task::TR, Task::TR, Task::TM, Task::TM];
let rt = runtime::Builder::new_multi_thread().enable_time().build().unwrap();
rt.block_on(async {
loop {
for task in tasks.clone() {
tx.send(task).await.expect("");
}
tokio::time::sleep(Duration::from_secs(10)).await;
}
});
th.join().expect("");
}
fn thread_main(rx: futures::channel::mpsc::Receiver<Task>) {
let rt = runtime::Builder::new_multi_thread().enable_time().build().unwrap();
rt.block_on(async {
run_loop(rx).await;
});
}