I am trying to create an async Rust client that connects to a WebSocket server on one end and a Bluetooth dongle on the other end. In between there will be some logic to filter messages.
I am using rust-websocket's async module for the WebSocket side.
I would like to isolate the three components (websocket send/receive, bluetooth send/receive, message processing). In order to do this, I want to use a futures::sync::mpsc channel as a Sink in order to pass messages from the WebSocket receiver to the message processing portion of the loop. Below is a simplified portion of my code:
const CONNECTION: &'static str = "ws://127.0.0.1:4000/socket/websocket";
fn main() {
let mut core = Core::new().unwrap();
let (ws_send, ws_recv) = mpsc::channel(100);
let (ws_to_main, main_from_ws) = mpsc::channel(100);
let ws_future = ClientBuilder::new(CONNECTION)
.unwrap()
.add_protocol("rust-websocket")
.async_connect_insecure(&core.handle())
.and_then(|(duplex, _)| {
let (mut sink, stream) = duplex.split();
stream
.filter_map(|message| {
println!("Received Message: {:?}", message);
match message {
OwnedMessage::Close(e) => Some(OwnedMessage::Close(e)),
OwnedMessage::Ping(d) => Some(OwnedMessage::Pong(d)),
OwnedMessage::Text(msg) => Some(OwnedMessage::Text(msg)),
_ => None,
}
})
.forward(ws_to_main)
});
let result = core.run(ws_future).unwrap();
}
When I run this code, however, I get several error messages which look like this:
error[E0277]: the trait bound `websocket::WebSocketError: std::convert::From<futures::sync::mpsc::SendError<websocket::OwnedMessage>>` is not satisfied
--> src/main.rs:103:22
|
103 | .forward(ws_to_main)
| ^^^^^^^ the trait `std::convert::From<futures::sync::mpsc::SendError<websocket::OwnedMessage>>` is not implemented for `websocket::WebSocketError`
|
= help: the following implementations were found:
<websocket::WebSocketError as std::convert::From<std::io::Error>>
<websocket::WebSocketError as std::convert::From<hyper::error::Error>>
<websocket::WebSocketError as std::convert::From<websocket::client::ParseError>>
<websocket::WebSocketError as std::convert::From<native_tls::Error>>
and 5 others
I've tried to hack it a bit with a .map_err which transforms the errors into a SendError, but the SendError struct is private.
A quick solution is converting the error type of the steam and sink to the same one:
stream
.map_err(|err| eprintln!("stream error: {}", err))
.forward(sink.sink_map_err(|err| eprintln!("sink error: {}", err)))
Related
Issue
I have a client that needs to send the following custom data structure to an API:
#[derive(Serialize, Deserialize)]
pub struct FheSum {
pub server_keys: ServerKey,
pub value1: FheUint8,
pub value2: FheUint8,
}
The code for the client is the following:
let fhe_post: FheSum = FheSum {
server_keys: server_keys.to_owned(),
value1: value_api.to_owned(),
value2: value_api_2.to_owned(),
};
let client = reqwest::blocking::Client::builder()
.timeout(None)
.build().unwrap();
let response = client
.post("http://127.0.0.1:8000/computesum")
.json(&fhe_post)
.send().unwrap();
let response_json: Result<FheSumResult, reqwest::Error> = response.json();
match response_json {
Ok(j) => {
let result_api: u8 = FheUint8::decrypt(&j.result, &client_keys);
println!("Final Result: {}", result_api)
},
Err(e) => println!("{:}", e),
};
In the API, I have the following definition of an HttpServer:
HttpServer::new(|| {
let json_cfg = actix_web::web::JsonConfig::default()
.limit(std::usize::MAX);
App::new()
.app_data(json_cfg)
.service(integers::computesum)
})
.client_disconnect_timeout(std::time::Duration::from_secs(3000))
.client_request_timeout(std::time::Duration::from_secs(3000))
.max_connection_rate(std::usize::MAX)
.bind(("127.0.0.1", 8000))?
.run()
.await
And the associated endpoint the client is trying to access:
#[post("/computesum")]
pub async fn computesum(req: Json<FheSum>) -> HttpResponse {
let req: FheSum = req.into_inner();
let recovered: FheSum = FheSum::new(
req.server_keys,
req.value1,
req.value2,
);
set_server_key(recovered.server_keys);
let result_api_enc: FheSumResult = FheSumResult::new(recovered.value1 + recovered.value2);
HttpResponse::Ok()
.content_type(ContentType::json())
.json(&result_api_enc)
}
Problem
The structs are the same in both the client and the server. This code works when using common data types such as Strings. The issue is when using this data structures. The memory occupied, obtained with mem::size_of_val which returns the size in bytes, is the following:
Size 1: 2488
Size 2: 32
Size 3: 32
The result has been obtained in bytes, so, given the limit established in the HttpServer, this shouldn't be an issue. Timeouts have also been set at much higher values than commonly needed.
Even with this changes, the client always shows Killed, and doesn't display the answer from the server, not giving any clues on what the problem might be.
The client is killing the process before being able to process the server's response. I want to find a way to send these custom data types to the HTTP server without the connection closing before the operation has finished.
I have already tried different libraries for the client such as the acw crate, apart from reqwest and the result is the same. I have also tried not using reqwest in blocking mode, and the error persists.
I'm writing a "hello world" HTTP server with Hyper, however I am not able to find the Server and rt modules when trying to import them.
When invoking cargo run, then I see this error message:
26 | let server = hyper::Server::bind(&addr).serve(router);
| ^^^^^^ could not find `Server` in `hyper`
I must be missing something obvious about Rust and Hyper. What I am trying to do is writing something as dry/simple as possible with just the HTTP layer and some basic routes. I would like to include as little as possible 3rd party dependencies e.g avoiding Tokio which I think involves async behavior, but I am not sure about the context as I am new to Rust.
Looks like I must use futures, so I included this dependency and perhaps futures only work with the async reserved word (which I am not sure if it comes from Tokio or Rust itself).
What confuses me is that in the Hyper examples I do see imports like use hyper::{Body, Request, Response, Server};, so that Server thing must be there, somewhere.
These are the dependencies in Cargo.toml:
hyper = "0.14.12"
serde_json = "1.0.67"
futures = "0.3.17"
This is the code in main.rs:
use futures::future;
use hyper::service::service_fn;
use hyper::{Body, Method, Response, StatusCode};
use serde_json::json;
fn main() {
let router = || {
service_fn(|req| match (req.method(), req.uri().path()) {
(&Method::GET, "/foo") => {
let mut res = Response::new(
Body::from(json!({"message": "bar"}).to_string())
);
future::ok(res)
},
(_, _) => {
let mut res = Response::new(
Body::from(json!({"content": "route not found"}).to_string())
);
*res.status_mut() = StatusCode::NOT_FOUND;
future::ok(res)
}
})
};
let addr = "127.0.0.1:8080".parse::<std::net::SocketAddr>().unwrap();
let server = hyper::Server::bind(&addr).serve(router); // <== this line fails to compile
// hyper::rt::run(server.map_err(|e| {
// eprintln!("server error: {}", e);
// }));
}
How do I make the code above compile and run?
According to documentation, you are missing one module namespace in your call hyper::server::Server:
let server = hyper::server::Server::bind(&addr).serve(router)
In order to use server you need to activate the feature flag in cargo:
hyper = { version = "0.14.12", features = ["server"] }
I'm building an application for a toy problem to learn more about SAFE. I have some background processes running server-side and occasionally they need to send a message unprompted to the connected clients. This means that I need a reference to the SocketHub from outside of any particular request.
Currently I have a mutable variable which I pass a value to when the Channel is joined:
let mainChannel = channel {
join (fun ctx socketId ->
task {
printfn "Connected! Main Socket Id: %O" socketId
let hub = ctx.GetService<Channels.ISocketHub>()
webSocketHub <- Some hub // Passing the reference to a mutable variable
task {
do! Task.Delay 500
let m = (socketId |> (SetChannelSocketId >> GameData))
do! (harderSendMessage socketId "message" m "Problem sending SocketId")
} |> ignore
return Channels.Ok })
}
However, it seems to me like there should be a better way to get access to the hub - I just can't figure it out.
I am trying to write an udp client in rust which establishes a socket connection to a remote server, should listen for incoming messages(and then process the data), while also be able to send messages and then disconnects after a given time. I would like to use the new async/await syntax in tokio and spawn a task that takes care of reading incoming/processing the incoming messages, while keeping the socket in the main task to send messages in parallel, especially at the end the protocol to close the connection.
How can I avoid moving the socket into the spawned task? Is there a way to borrow it in that task maybe trough a reference. I looked through answers to similar questions but could not understand it as they apply to the version of tokio without the new syntax and as I am an absolute beginner in rust.
I can move the socket into the spawned function, but then it is of course no longer available to the code outside, which needs to send messages in parallel.
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let remote_addr: SocketAddr = "...:xxxx".parse()?;
let local_addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(0,0,0,0)), 0);
let mut socket = UdpSocket::bind(&local_addr)?;
socket.connect(&remote_addr)?;
// do some protocol work with the socket to establish a connection
tokio::spawn(async move {
let mut buf = [0; 1024];
loop {
let l = match socket.recv(&mut buf).await {
// socket closed
Ok(l) if l == 0 => {
println!("socket closed");
return;
},
Ok(l) => l,
Err(e) => {
println!("failed to read from socket; err = {:?}", e);
return;
}
};
let data = buf[..l].to_vec();
println!("Received {} bytes:\n{:#x?}", l, data);
}
});
// here I would like to use the socket again to send messages and to do the disconnect protocol, i.e.
let len = socket.recv(.....
When I use the socket afterwards, I get the error that the variable moved due to use in generator and gets dropped at the end of the spawn task (which it should not). Later use of socket says value borrowed after move, which is clear, but how can I avoid it?
I would appreciate if somebody could help me with this beginner question, especially in the context of the new async/await syntax of tokio. Thanks!
Well, I solved the problem in a different way. Tokio's UdpSocket can be split into a receiving and a sending part. I run both of them in separate task and then use multiple tokio::mpsc::channel to communicate between the two task and the main task.
I am trying to spawn a server and connect to it on a different thread. I know Rust has blocking I/O, but I feel like I should be able to connect a server in a different thread. I do not have a lot of knowledge in threads. The end game is to connect to this server across a network. That is what I am simulating with the player_stream TCPStream. The player_stream will wait until there is something in its buffer. Once something has been written there, it will respond back to the server. As is, the program will not terminate.
use std::net::{TcpListener, TcpStream};
use std::io::{BufReader,BufWriter};
use std::io::Write;
use std::io::Read;
use std::thread;
fn main() {
thread::spawn(move || {
start_server();
});
let player_stream = TcpStream::connect("127.0.0.1:8000").expect("Couldn't connect");
let mut reader = BufReader::new(&player_stream);
let mut response = String::new();
reader.read_to_string(&mut response);
println!("Player received {}", response);
let mut writer = BufWriter::new(&player_stream);
writer.write_all("NAME".as_bytes());
}
fn start_server() {
let listener = TcpListener::bind("127.0.0.1:8000").unwrap();
fn handle_client(stream: TcpStream) {
println!("Client connected");
let mut writer = BufWriter::new(&stream);
writer.write_all("Red".as_bytes());
let mut reader = BufReader::new(&stream);
let mut response = String::new();
reader.read_to_string(&mut response);
println!("Server received {}", response);
}
// accept connections
for stream in listener.incoming() {
match stream {
Ok(stream) => {
handle_client(stream);
}
Err(e) => { panic!("{}",e) }
}
}
}
First off, don't ignore warnings. You have 4 errors of the type warning: unused result which must be used. Every single one of those could be cases where your code is failing and you wouldn't even know it. Use expect liberally!
Second, you have an open client read socket and you ask to "read all the data until the end into a string". What determines the end? In this case, it's when the socket is closed; so when is that?
Trick question!
The client's read socket closes when the server's write socket closes.
The server's write socket closes when the server's read socket closes.
The server's read socket closes when the the client's write socket closes.
So when does that happen? Because there's no code that does it specifically, it will close when the socket is dropped, so:
The client's write socket closes when the the client ends.
Thus the deadlock. The issue could be fixed by explicitly closing the write half of the socket:
stream.shutdown(std::net::Shutdown::Write).expect("could not shutdown");
Third, you are writing into a BufWriter. Review the documentation for it:
A BufWriter keeps an in-memory buffer of data and writes it to an underlying writer in large, infrequent batches.
The buffer will be written out when the writer is dropped.
The BufWriter is dropped at the end of the scope, after you've tried to read the response. That's another deadlock.
In the end, you need to establish a protocol for how to delimit messages sent back and forth. A simple but very limited solution is to have a line-oriented protocol: every message fits on one line ending with a newline character.
If you choose that, you can use read_to_line instead. I've also used BufWriter::flush to force the data to be sent down the wire; you could have also encapsulated writer in a block so it is dropped earlier or explicitly call drop(writer).
use std::net::{TcpListener, TcpStream};
use std::io::{BufReader, BufWriter, Write, BufRead};
use std::thread;
fn main() {
thread::spawn(start_server);
let player_stream = TcpStream::connect("127.0.0.1:8000").expect("Couldn't connect");
let mut reader = BufReader::new(&player_stream);
let mut response = String::new();
reader.read_line(&mut response).expect("Could not read");
println!("Player received >{}<", response.trim());
let mut writer = BufWriter::new(&player_stream);
writer.write_all("NAME\n".as_bytes()).expect("Could not write");
}
fn start_server() {
let listener = TcpListener::bind("127.0.0.1:8000").unwrap();
fn handle_client(stream: TcpStream) {
println!("Client connected");
let mut writer = BufWriter::new(&stream);
writer.write_all("Red\n".as_bytes()).expect("could not write");
writer.flush().expect("could not flush");
let mut reader = BufReader::new(&stream);
let mut response = String::new();
reader.read_line(&mut response).expect("could not read");
println!("Server received {}", response);
}
for stream in listener.incoming() {
let stream = stream.expect("Unable to accept");
handle_client(stream);
}
}
You'll note that the program doesn't always print out the server's response. That's because the main thread exiting exits the program.
You mentioned that your real case uses XML, which can have newlines embedded in it, making a line-oriented protocol unsuitable. Another common protocol is to send a length before sending the data itself. There are many possible implementations for this. At a previous job, we sent XML in this fashion. We started with an ASCII-encoded newline-terminated string of the length before the data itself. In that case, having the readability of the length as a string was a benefit. You could also choose to send a number of bytes that can be interpreted according to some endianness as a 2's compliment number.
See also:
Rust echo server and client using futures blocks itself forever