cannot move out of a captured variable in an async `Fn` closure - asynchronous

Here is my code. In this program, I want to create a simple websocket server. When user sends a request to the ws://{url}/, the browser will establish a websocket connection with the server.
use std::{collections::HashMap, sync::Arc};
use async_std::{prelude::*, sync::Mutex};
use tide_websockets::WebSocket;
use uuid::Uuid;
#[async_std::main]
async fn main() {
let connections = Arc::new(Mutex::new(HashMap::new()));
let mut app = tide::new();
app.at("/").get(WebSocket::new(move |_, mut stream| async move {
let uuid = Uuid::new_v4();
// Add the connection to clients when opening a new connection
connections.lock().await.insert(uuid, stream.clone());
// Waiting for the connection to be closed
while let Some(Ok(_)) = stream.next().await {}
// Remove the connection from clients when it is closed
connections.lock().await.remove(&uuid);
Ok(())
}));
// app.bind(url).await
}
When I tried to compile this program, the rustc said:
error[E0507]: cannot move out of `connections`, a captured variable in an `Fn` closure
--> src/main.rs:11:57
|
9 | let connections = Arc::new(Mutex::new(HashMap::new()));
| ----------- captured outer variable
10 | let mut app = tide::new();
11 | app.at("/").get(WebSocket::new(move |_, mut stream| async move {
| ____________________________________--------------------_^
| | |
| | captured by this `Fn` closure
12 | | let uuid = Uuid::new_v4();
13 | |
14 | | // Add the connection to clients when opening a new connection
15 | | connections.lock().await.insert(uuid, stream.clone());
| | -----------
| | |
| | variable moved due to use in generator
| | move occurs because `connections` has type `Arc<async_std::sync::Mutex<HashMap<Uuid, WebSocketConnection>>>`, which does not implement the `Copy` trait
... |
23 | | Ok(())
24 | | }));
| |_____^ move out of `connections` occurs here
For more information about this error, try `rustc --explain E0507`.
error: could not compile `mre` due to previous error
And this is the definition of the Websocket::new method (no sure if it's useful):
impl<S, H, Fut> WebSocket<S, H>
where
S: Send + Sync + Clone + 'static,
H: Fn(Request<S>, WebSocketConnection) -> Fut + Sync + Send + 'static,
Fut: Future<Output = Result<()>> + Send + 'static,
{
/// Build a new WebSocket with a handler function that
pub fn new(handler: H) -> Self {
Self {
handler: Arc::new(handler),
ghostly_apparition: PhantomData,
protocols: Default::default(),
}
}
// ...
}
I tried searching this problem before posting this question. Most of the answers are either irrelevant, or need to modify the source code of the method (Websocket::new method here). But this method is not written by me but is from a third-party crate. Is there still any way to resolve this problem?

The argument of WebSocket::new() has to be an Fn closure, meaning, it must be callable repeatedly.
In your code, however, it internally uses the connections variable inside of an async move, meaning it moves the variable into the async block. This can for obvious reasons only be done once.
It's easy to fix, though. Instead of moving the entire connections variable in, you need to create a new Arc reference of the connections variable and move that one into the async move. So every invocation gets its own copy of it, making it compatible with Fn.
Here is a compiling version:
use std::{collections::HashMap, sync::Arc};
use async_std::{prelude::*, sync::Mutex};
use tide_websockets::WebSocket;
use uuid::Uuid;
#[async_std::main]
async fn main() {
let connections = Arc::new(Mutex::new(HashMap::new()));
let mut app = tide::new();
app.at("/").get(WebSocket::new(move |_, mut stream| {
let connections = Arc::clone(&connections);
async move {
let uuid = Uuid::new_v4();
// Add the connection to clients when opening a new connection
connections.lock().await.insert(uuid, stream.clone());
// Waiting for the connection to be closed
while let Some(Ok(_)) = stream.next().await {}
// Remove the connection from clients when it is closed
connections.lock().await.remove(&uuid);
Ok(())
}
}));
// app.bind(url).await
}

Related

Streaming results using Rust actix-web, lifetime issue

I am having issue to use Stream with actix-web using bellow code:
fn format_csv_row(row: tiberius::Row) -> Result<web::Bytes, ServerError> { ... }
#[get("/stream/")]
async fn get_stream(
db_pool: web::Data<bb8::Pool<TiberiusConnectionManager>>,
) -> Result<HttpResponse, ServerError> {
// Get connection
let mut conn = db_pool
.get()
.await
.map_err(|e| ServerError::PoolUnavailable)?;
// Execute query
let stream = conn
.query("SELECT * FROM table", &[])
.await
.map_err(|e| ServerError::QueryFail)?;
// Build a stream from SQL results
let stream = stream.map(|x| format_csv_row(x.unwrap()));
Ok(HttpResponse::Ok().streaming(stream))
}
Here are the methods I call in this function:
bb8::Pool::get
tiberius::Client::query
actix-web::HttpResponseBuilder::stream
The compiler complains with the following errors:
error[E0597]: `db_pool` does not live long enough
--> src/routes/trades_stream.rs:88:20
|
88 | let mut conn = db_pool
| -^^^^^^
| |
| ____________________borrowed value does not live long enough
| |
89 | | .get()
| |______________- argument requires that `db_pool` is borrowed for `'static`
...
102 | }
| - `db_pool` dropped here while still borrowed
error[E0597]: `conn` does not live long enough
--> src/routes/trades_stream.rs:94:18
|
94 | let stream = conn
| -^^^
| |
| __________________borrowed value does not live long enough
| |
95 | | .query("SELECT * FROM table", &[])
| |__________________________________________- argument requires that `conn` is borrowed for `'static`
...
102 | }
| - `conn` dropped here while still borrowed
error: aborting due to 2 previous errors
I understand that connection and pool are dropped while streaming is still in progress.
How can I modify my code to make this works ?
Is it possible to add explicit lifetime for db_pool and conn to make them match stream ?
To make this work, we need to change the code such that the stream has ownership of the connection it is reading from, and due to how bb8 is written, you also need ownership of a handle to the pool. The best way to do this is to use the async-stream crate.
I believe something like this should do it:
use async_stream::try_stream;
fn format_csv_row(row: tiberius::Row) -> Result<web::Bytes, ServerError> { ... }
#[get("/stream/")]
async fn get_stream(
db_pool: web::Data<bb8::Pool<TiberiusConnectionManager>>,
) -> Result<HttpResponse, ServerError> {
// Cloning a bb8 pool gives a new handle to the same pool.
let db_pool = db_pool.clone();
let stream = try_stream! {
// Get connection
let mut conn = db_pool
.get()
.await
.map_err(|e| ServerError::PoolUnavailable)?;
// Execute query
let stream = conn
.query("SELECT * FROM table", &[])
.await
.map_err(|e| ServerError::QueryFail)?;
while let Some(row) = stream.next().await {
yield format_csv_row(row?)?;
}
};
Ok(HttpResponse::Ok().streaming(Box::pin(stream)))
}
You might need another map_err on the row? part.
Is it possible to add explicit lifetime for db_pool and conn to make them match stream ?
No, you don't change how long things live by setting lifetimes. Instead, you change the lifetimes by changing the structure of the code in a way such that it lives long enough.

Error: Could not find main or io in tokio, invalid return type `impl Future`

I'm on my way of converting to Rust from the ML family, but I'm finding it hard at some strange places I'm not used to having problems.
I'm trying to use hyper for http handling but can't seem to get tokio to work.
I have tried to copy paste this example:
use hyper::{body::HttpBody as _, Client};
use tokio::io::{self, AsyncWriteExt as _};
type Result<T> = std::result::Result<T, Box<dyn std::error::Error + Send + Sync>>;
#[tokio::main]
async fn main() -> Result<()> {
// ...
fetch_url(url).await
}
async fn fetch_url(url: hyper::Uri) -> Result<()> {
// ...
Ok(())
}
Here is my Cargo.toml:
[package]
name = "projectname"
version = "0.1.0"
authors = ["username"]
edition = "2018"
[dependencies]
hyper = "0.14.4"
tokio = "1.2.0"
It is complaining that it can't find the io crate, and that main has an invalid type impl Future, and that it can't find main in tokio:
error[E0433]: failed to resolve: could not find `main` in `tokio`
--> src/main.rs:9:10
|
9 | #[tokio::main]
| ^^^^ could not find `main` in `tokio`
error[E0277]: `main` has invalid return type `impl Future`
--> src/main.rs:10:20
|
10 | async fn main() -> Result<()> {
| ^^^^^^^^^^ `main` can only return types that implement `Termination`
error[E0432]: unresolved import `hyper::Client`
--> src/main.rs:3:34
|
3 | use hyper::{body::HttpBody as _, Client};
| ^^^^^^ no `Client` in the root
error[E0425]: cannot find function `stdout` in module `io`
--> src/main.rs:45:13
|
45 | io::stdout().write_all(&chunk).await?;
| ^^^^^^ not found in `io`
|
error[E0432]: unresolved import `tokio::io::AsyncWriteExt`
--> src/main.rs:4:23
|
4 | use tokio::io::{self, AsyncWriteExt as _};
| -------------^^^^^
| |
| no `AsyncWriteExt` in `io`
| help: a similar name exists in the module: `AsyncWrite`
Is #[tokio::main] and client not in hyper?
The tokio::main macro converts an async main to a regular main that spawns a runtime. However, because the macro is not found is scope, it cannot transform your main function, and the compiler is complaining that your main has an invalid return type of impl Future. To fix this, you have to enable the required features to import the main macro:
tokio = { version = "1.2.0", features = ["rt", "macros"] }
You also have to enable the io-util feature to access io::AsyncWriteExt, and the io-std feature to access io::stdout. To simplify this, tokio provides the full feature flag, which will enable all optional features:
tokio = { version = "1.2.0", features = ["full"] }
You also need hyper's client and http feature flags to resolve the Client import:
hyper = { version = "0.14.4", features = ["client", "http1", "http2"] }

Using `DeserializedOwned` trait in a oneshot channel: object safety error

The following code:
trait ClientResponse: DeserializeOwned + Send + Sized {}
struct ClientMsg {
...
resp: oneshot::Sender<Box<dyn ClientResponse>>
}
async fn client_thread(rx: mpsc::Receiver<ClientMsg>, client: reqwest::Client, base_url: Url) -> Result<(), Box<dyn Error>> {
while let Some(msg) = rx.recv().await {
...
let response = client.get(url).send().await?.json().await?;
msg.resp.send(response);
}
}
Fails with error:
error[E0038]: the trait `ClientResponse` cannot be made into an object
--> src/main.rs:16:11
|
16 | resp: oneshot::Sender<Box<dyn ClientResponse>>
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `ClientResponse` cannot be made into an object
|
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
--> src/main.rs:12:23
|
12 | trait ClientResponse: DeserializeOwned + Send + Sized {}
| -------------- ^^^^^^^^^^^^^^^^ ^^^^^ ...because it requires `Self: Sized`
| | |
| | ...because it requires `Self: Sized`
| this trait cannot be made into an object...
As you can see, I tried adding the Sized trait as a super trait after reading the compiler error, but it still gives me the same error. I'm not sure how else to approach this problem, since I want a client thread that can deserialize responses into types decided by the senders.
There is a crate exactly for making serde traits object safe called erased-serde

How to execute async lua code with rlua-async?

I'm trying to run/script my existing rust async code with rlua-async. Sadly it is not well documented and has no examples but I have prevailed in getting my async functions defined but I have trouble getting my lua code executed in an async way.
I have created a minimal repository to reproduce the problem here
use rlua::{Lua};
use rlua_async::{ChunkExt, ContextExt};
#[actix_rt::main]
async fn main() {
let lua_code = "my.asyncfunc(42)";
let lua = Lua::new();
lua.context(|lua_ctx| {
let globals = lua_ctx.globals();
let map_table = lua_ctx.create_table().unwrap();
map_table
.set(
"asyncfunc",
lua_ctx
.create_async_function(
|_ctx,
param:
u32
| async move {
println!("async function called {}", param);
Ok(())
}).unwrap()).unwrap();
globals.set("my", map_table).unwrap();
});
lua.context(|lua_context| async move {
let chunk = lua_context
.load(&lua_code);
chunk.exec_async(lua_context).await.unwrap();
})
.await;
println!("finished");
}
But I'm getting this error message:
error: lifetime may not live long enough
--> src\main.rs:28:31
|
28 | lua.context(|lua_context| async move {
| __________________------------_^
| | | |
| | | return type of closure is impl Future
| | has type `LuaContext<'1>`
29 | | let chunk = lua_context
30 | | .load(&lua_code);
31 | | chunk.exec_async(lua_context).await.unwrap();
32 | | })
| |_____^ returning this value requires that `'1` must outlive `'2`
I really don't get what the error is trying to tell me and there is no helpful tips or even documentation linked.
The closure is somehow different from the closure body and needs lifetime annotations? But why and how...?
EDIT: if I instead call the code without async like this:
lua.context(|lua_context| {
let chunk = lua_context.load(&lua_code);
chunk.exec().unwrap();
});
it compiles but I get the following panic on runtime:
thread 'main' panicked at 'cannot access a scoped thread local variable without calling `set` first', C:\Users\ahallmann\.cargo\registry\src\github.com-1ecc6299db9ec823\scoped-tls-1.0.0\src\lib.rs:168:9
Everything works fine if I define the function with create_function.
I figured it out with the kind help of the author of rlua-async.
The issue is with the actix-rt itself as it requires 'static lifetimes for the block_on call.
It works fine if you use either futures or tokio instead:
tokio::runtime::Runtime::new()
.unwrap()
.block_on(chunk.exec_async(ctx))
See https://github.com/actix/actix-net/issues/201
or
https://github.com/Ekleog/rlua-async/issues/1 for further information.

Can you convert a mut i8 into an i32?

I am attempting to build a small terminal emulator and am running into some interesting type conflicts with libc. When I am attempting to set up the slave portion of the pty connection I need to create the slave with a system call to ptsname() in order to get the name for the pts so I can access it. However, I get a type error saying that libc::ptsname() requires an i32 for the input. This is in direct conflict with the man page that says it should be passed a file descriptor. I'm just wondering if I can convert the libc::c_int that I have for a file descriptor into a i32 to pass into ptsname.
The code is as follows :
use libc::{self, c_int, grantpt, posix_openpt, ptsname, unlockpt, O_RDWR};
use std::os::unix::io::FromRawFd;
use std::process::{Child, Command, Stdio};
#[derive(Debug)]
pub struct Pty {
process: Child,
fd: i32,
}
fn create_pty(process: &str) -> Pty {
let master: c_int;
unsafe {
// create master/slave pair of fd
master = posix_openpt(O_RDWR);
if master == -1 {
panic!("Failed to posix_openpt");
}
// set slave ownership and mode as master
let mut result = grantpt(master);
if result == -1 {
panic!("Failed to grantpt");
}
// unlock slave
result = unlockpt(master);
if result == -1 {
panic!("Failed to unlockpt");
}
}
let slave: c_int = ptsname(master as i32);
slave = libc::open(slave);
let mut builder = Command::new(process);
match builder.spawn() {
Ok(process) => {
let pty = Pty {
process,
fd: master,
};
pty
}
Err(e) => {
panic!("Failed to create pty: {}", e);
}
}
}
fn main() {
let shell = "/bin/bish";
let pty = create_pty(shell);
println!("{:?}", pty);
}
and the console output(The second error can be ignored for now):
error[E0308]: mismatched types
--> src/main.rs:42:24
|
42 | let slave: c_int = ptsname(master as i32);
| ^^^^^^^^^^^^^^^^^^^^^^ expected i32, found *-ptr
|
= note: expected type `i32`
found type `*mut i8`
error[E0060]: this function takes at least 2 parameters but 1 parameter was supplied
--> src/main.rs:43:13
|
43 | slave = libc::open(slave);
| ^^^^^^^^^^^^^^^^^ expected at least 2 parameters
error: aborting due to 2 previous errors
Some errors have detailed explanations: E0060, E0308.
For more information about an error, try `rustc --explain E0060`.
error: could not compile `experiment`.
It's not saying that it requires an input of i32, but rather that you're asking that ptsname(master as i32); has the type i32. This might be a bit confusing because c_int is an alias for i32, so it sounds like it's asking for an unrelated type.
The problem is that you're giving slave the type c_int, when ptsname returns *mut c_char (c_char is also an alias, this time for i8).

Resources