I have closure:
move |p| match Params::parse::<Value>(p) {
Ok(ref v) if v.as_str().is_some() => {
let chain = v.as_str().unwrap();
match sender.send(chain) {
Ok(_) => futures::done(Ok(Value::Null)).boxed(),
Err(err) => futures::failed(JsonRpcError::invalid_params(
format!("Node not responding: {}", err.to_string())))
.boxed(),
}
}
Ok(_) | Err(_) => {
futures::failed(JsonRpcError::invalid_params("Invalidlabel"))
.boxed()
}
};
compilation fails with such error:
the trait bound `std::sync::mpsc::Sender<&str>: std::marker::Sync` is not satisfied in `[closure#src/rpc/mod.rs:223:13: 237:14 sender:std::sync::mpsc::Sender<&str>]`
Isn't move keyword should capture and clone Sender?
The move keyword causes the Sender object to move into the closure, i.e. it is specifically designed to prevent a borrow from occurring. Your error message indicates that sender is not Sync, which indicates that a move did not in fact occur. As noted by #red75prim, one way for this to happen is if sender is a reference, which will cause it to be copied into the closure, and in turn require the underlying Sender object to be Sync.
To send an mpsc Sender into another thread, you need to first clone it, and then move the clone into the closure:
{
let sender = sender.clone();
move |p| match Params::parse::<Value>(p) {
...
}
};
Related
In the documentation for the Send trait, there is a nice example of how something like Rc is not Send, since cloning/dropping in two different threads can cause the reference count to get out of sync.
What is less clear, however, is why holding a binding to a non-Send type across an await point in an async fn causes the generated future to also be non-Send. I was able to find a work around for when the compiler has been too conservative in the work-arounds chapter of the async handbook, but it does not go as far as answering the question that I am asking here.
Perhaps someone could shed some light on this with an example of why having a non-Send type in a Future is ok, but holding it across an await is not?
When you use .await in an async function, the compiler builds a state machine behind the scenes. Each .await introduces a new state (while it waits for something) and the code in between are state transitions (aka tasks), which will be triggered based on some external event (e.g. from IO or a timer etc).
Each task gets scheduled to be executed by the async runtime, which could choose to use a different thread from the previous task. If the state transition is not safe to be sent between threads then the resulting Future is also not Send so that you get a compilation error if you try to execute it in a multi-threaded runtime.
It is completely OK for a Future not to be Send, it just means you can only execute it in a single-threaded runtime.
Perhaps someone could shed some light on this with an example of why having a non-Send type in a Future is ok, but holding it across an await is not?
Consider the following simple example:
async fn add_votes(current: Rc<Cell<i32>>, post: Url) {
let new_votes = get_votes(&post).await;
*current += new_votes;
}
The compiler will construct a state machine like this (simplified):
enum AddVotes {
Initial {
current: Rc<Cell<i32>>,
post: Url,
},
WaitingForGetVotes {
current: Rc<Cell<i32>>,
fut: GetVotesFut,
},
}
impl AddVotes {
fn new(current: Rc<Cell<i32>>, post: Url) {
AddVotes::Initial { current, post }
}
fn poll(&mut self) -> Poll {
match self {
AddVotes::Initial(state) => {
let fut = get_votes(&state.post);
*self = AddVotes::WaitingForGetVotes {
current: state.current,
fut
}
Poll::Pending
}
AddVotes::WaitingForGetVotes(state) => {
if let Poll::Ready(votes) = state.fut.poll() {
*state.current += votes;
Poll::Ready(())
} else {
Poll::Pending
}
}
}
}
}
In a multithreaded runtime, each call to poll could be from a different thread, in which case the runtime would move the AddVotes to the other thread before calling poll on it. This won't work because Rc cannot be sent between threads.
However, if the future just used an Rc within the same state transition, it would be fine, e.g. if votes was just an i32:
async fn add_votes(current: i32, post: Url) -> i32 {
let new_votes = get_votes(&post).await;
// use an Rc for some reason:
let rc = Rc::new(1);
println!("rc value: {:?}", rc);
current + new_votes
}
In which case, the state machine would look like this:
enum AddVotes {
Initial {
current: i32,
post: Url,
},
WaitingForGetVotes {
current: i32,
fut: GetVotesFut,
},
}
The Rc isn't captured in the state machine because it is created and dropped within the state transition (task), so the whole state machine (aka Future) is still Send.
I seem unable to use pnet's datalink library in any asynchronous way. The trouble appears to relate to pinning. This snippet adapted from pnet's main example:
async fn ethernet_channel(i: NetworkInterface) {
// Create Ethernet channel:
let (mut tx, mut rx) = match datalink::channel(&i, Default::default()) {
Ok(Ethernet(tx, rx)) => (tx, rx),
_ => panic!("Error creating channel")
};
loop { // handle inbound Ethernet packets forever
tokio::select! {
Ok(packet) = rx.next() => {}
// TODO: include some arm like an mpsc oneshot to break out, like Tokio's examples
}
}
}
Leads to compiler error: ror[E0599]: no method named 'poll' found for struct 'Pin<&mut std::result::Result<&[u8], std::io::Error>>' in the current scope
The Tokio tutorial covers this exact scenario and includes this remark: "to .await a reference, the value being referenced must be pinned or implement Unpin."
Combined with knowing the return value of the rx.next() function is the Result<&[u8], Error> mentioned in the error, I gather the packet byte array is the culprit "value being referenced" that must be pinned. I've tried many combinations of tokio::pin!(), including what makes the most sense to me based on Tokio's same example, to no avail:
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn Error>> {
for interface in datalink::interfaces() {
let operation = ethernet_channel(interface);
tokio::pin!(operation);
tokio::select! {
_ = &mut operation => {}
}
}
}
I've also tried tokio::pin!() within my "ethernet_channel()" function, before and after datalink::channel(), on both the NetworkInterface and rx. I always end up with the same error as above. Any guidance appreciated, pulling my hair out.
i just started learning Swift a year ago, so please be patient with me :)
i am downloading JSON data with a network call, and as soon as i successfully received those rows, i then continue to clear the rows inside my coreData entity, and rewrite those new rows into coredata..
i am having a hard time understanding this asynchronous procedure..
what i've learned is that i have to use completion handlers, but i still can't use it the way i need to.. especialy when i need to proceed after those 3 steps were executed..
First call from button action:
#IBAction func updateButtonPressed(_ sender: Any) {
self.myCoreData.update() {(success) in // calls my update method
print(success!)
textField.text = success! // not possible bc not in the Mainthread
}
textField.text = "blabla" // gets executed before the result is available
methods:
func update(completion: #escaping (String?) -> Void) { //parent method which calls sub methods
var returnValue = ""
Step1getJson {_ in. // step 1
self.Step2Delete { // step 2
self.Step3Save { // step 3
returnValue = "return Value: \(self.step1Result)"
completion(returnValue)
}
}
}
}
func Step1getJson(completion: #escaping (Bool) -> ()) {
var success = false
if let url = URL(string: "https:foo") {
URLSession.shared.dataTask(with: url) { data, response, error in
guard let data = data else { return }
do {
let parsedJSON = try JSONDecoder().decode([RemoteWire].self, from: data)
print("-- Successfully received \(parsedJSON.count) datarows ")
self.JSON = parsedJSON
self.step1Result = "-- Successfully received \(parsedJSON.count) datarows "
success = true
} catch {
print(error)
}
completion(success)
}.resume()
}
}
func Step2Delete(completion: () -> Void) {
...delete entity rows
completion()
}
func Step3Save(completion: () -> Void) {
.. save new JSON rows to coreData
completion()
}
Everything is working fine that far, and step 2 and step 3 get successfully called when network download has finished..
but how can i proceed after those steps were executed inside my updateButtonPressed function?
if i try to write those results into any UI element inside my completion block, a textField or whatever, i get an error that this has to happen in the main thread, and if i execute it outside the completion block those lines get executed far too early, when no results are available yet.
i feel like i have understanding problem with this, i hope you guys can help me out and guide me in the right direction.
As swift allows any changes or updates in UI element only from main thread, you need to call the main thread to update the UI.
Replace the below code
#IBAction func updateButtonPressed(_ sender: Any) {
self.myCoreData.update() {(success) in // calls my update method
print(success!)
textField.text = success! // not possible bc not in the Mainthread
}
}
with the new code
#IBAction func updateButtonPressed(_ sender: Any) {
self.myCoreData.update() {(success) in // calls my update method
print(success!)
DispatchQueue.main.async {
textField.text = success! // Now possible because it is in main thread
}
}
}
I'm creating a task which will spawn other tasks. Some of them will take some time, so they cannot be awaited, but they can run in parallel:
src/main.rs
use crossbeam::crossbeam_channel::{bounded, select};
#[tokio::main]
async fn main() {
let (s, r) = bounded::<usize>(1);
tokio::spawn(async move {
let mut counter = 0;
loop {
let loop_id = counter.clone();
tokio::spawn(async move { // why this one was not fired?
println!("inner task {}", loop_id);
}); // .await.unwrap(); - solves issue, but this is long task which cannot be awaited
println!("loop {}", loop_id);
select! {
recv(r) -> rr => {
// match rr {
// Ok(ee) => {
// println!("received from channel {}", loop_id);
// tokio::spawn(async move {
// println!("received from channel task {}", loop_id);
// });
// },
// Err(e) => println!("{}", e),
// };
},
// more recv(some_channel) ->
}
counter = counter + 1;
}
});
// let s_clone = s.clone();
// tokio::spawn(async move {
// s_clone.send(2).unwrap();
// });
loop {
// rest of the program
}
}
I've noticed strange behavior. This outputs:
loop 0
I was expecting it to also output inner task 0.
If I send a value to channel, the output will be:
loop 0
inner task 0
loop 1
This is missing inner task 1.
Why is inner task spawned with one loop of delay?
The first time I noticed such behavior with 'received from channel task' delayed one loop, but when I reduced code to prepare sample this started to happen with 'inner task'. It might be worth mentioning that if I write second tokio::spawn right to another, only the last one will have this issue. Is there something I should be aware when calling tokio::spawn and select!? What causes this one loop of delay?
Cargo.toml dependencies
[dependencies]
tokio = { version = "0.2", features = ["full"] }
crossbeam = "0.7"
Rust 1.46, Windows 10
select! is blocking, and the docs for tokio::spawn say:
The spawned task may execute on the current thread, or it may be sent to a different thread to be executed.
In this case, the select! "future" is actually a blocking function, and spawn doesn't use a new thread (either in the first invocation or the one inside the loop).
Because you don't tell tokio that you are going to block, tokio doesn't think another thread is needed (from tokio's perspective, you only have 3 futures which should never block, so why would you need another thread anyway?).
The solution is to use the tokio::task::spawn_blocking for the select!-ing closure (which will no longer be a future, so async move {} is now move || {}).
Now tokio will know that this function actually blocks, and will move it to another thread (while keeping all the actual futures in other execution threads).
use crossbeam::crossbeam_channel::{bounded, select};
#[tokio::main]
async fn main() {
let (s, r) = bounded::<usize>(1);
tokio::task::spawn_blocking(move || {
// ...
});
loop {
// rest of the program
}
}
Link to playground
Another possible solution is to use a non-blocking channel like tokio::sync::mpsc, on which you can use await and get the expected behavior, like this playground example with direct recv().await or with tokio::select!, like this:
use tokio::sync::mpsc;
#[tokio::main]
async fn main() {
let (mut s, mut r) = mpsc::channel::<usize>(1);
tokio::spawn(async move {
loop {
// ...
tokio::select! {
Some(i) = r.recv() => {
println!("got = {}", i);
}
}
}
});
loop {
// rest of the program
}
}
Link to playground
Preface: I have done my research and know that it is really not a good idea/nor is it idiomatic Rust to have one. Completely open to suggestions of other ways to solve this issue.
Background: I have a console application that connects to a websocket and once connected successfully, the server sends a "Connected" message. I have the sender, and the receiver is separate threads and all is working great. After the connect() call a loop begins and places a prompt in the terminal, signaling that the application is ready to receive input from the user.
Problem: The issue is that the current flow of execution calls connect, and then immediately displays the prompt, and then the application receives the message from the server stating it is connected.
How I would solve this in higher level languages: Place a global bool (we'll call it ready) and once the application is "ready" then display the prompt.
How I think this might look in Rust:
//Possible global ready flag with 3 states (true, false, None)
let ready: Option<&mut bool> = None;
fn main(){
welcome_message(); //Displays a "Connecting..." message to the user
//These are special callback I created and basically when the
//message is received the `connected` is called.
//If there was an error getting the message (service is down)
//then `not_connected` is called. *This is working code*
let p = mylib::Promise::new(connected, not_connected);
//Call connect and start websocket send and receive threads
mylib::connect(p);
//Loop for user input
loop {
match ready {
Some(x) => {
if x == true { //If ready is true, display the prompt
match prompt_input() {
true => {},
false => break,
}
} else {
return; //If ready is false, quit the program
}
},
None => {} //Ready is None, so continue waiting
}
}
}
fn connected() -> &mut bool{
println!("Connected to Service! Please enter a command. (hint: help)\n\n");
true
}
fn not_connected() -> &mut bool{
println!("Connection to Service failed :(");
false
}
Question:
How would you solve this issue in Rust? I have tried passing it around to all the libraries method calls, but hit some major issues about borrowing an immutable object in a FnOnce() closure.
It really sounds like you want to have two threads that are communicating via channels. Check out this example:
use std::thread;
use std::sync::mpsc;
use std::time::Duration;
enum ConsoleEvent {
Connected,
Disconnected,
}
fn main() {
let (console_tx, console_rx) = mpsc::channel();
let socket = thread::spawn(move || {
println!("socket: started!");
// pretend we are taking time to connect
thread::sleep(Duration::from_millis(300));
println!("socket: connected!");
console_tx.send(ConsoleEvent::Connected).unwrap();
// pretend we are taking time to transfer
thread::sleep(Duration::from_millis(300));
println!("socket: disconnected!");
console_tx.send(ConsoleEvent::Disconnected).unwrap();
println!("socket: closed!");
});
let console = thread::spawn(move || {
println!("console: started!");
for msg in console_rx.iter() {
match msg {
ConsoleEvent::Connected => println!("console: I'm connected!"),
ConsoleEvent::Disconnected => {
println!("console: I'm disconnected!");
break;
}
}
}
});
socket.join().expect("Unable to join socket thread");
console.join().expect("Unable to join console thread");
}
Here, there are 3 threads at play:
The main thread.
A thread to read from the "socket".
A thread to interface with the user.
Each of these threads can maintain it's own non-shared state. This allows reasoning about each thread to be easier. The threads use a channel to send updates between them safely. The data that crosses threads is encapsulated in an enum.
When I run this, I get
socket: started!
console: started!
socket: connected!
console: I'm connected!
socket: disconnected!
socket: closed!
console: I'm disconnected!