I'm following the Make a simple TCP server example in the nightly docs. I've connected via telnet and I'd like to see data sent line by line. Right now I'm read_to_string and I only get data when I close the telnet connection. I'd like to be able to read each line at the server after the user in the telnet session hits enter.
This is what I came up after reading the docs for BufferedReader.
fn handle_client(mut stream: TcpStream) {
let wresult = stream.write_line("Welcome.");
match wresult {
Err(e) => {
println!("error writing: {}", e);
}
_ => {}
}
let mut reader = BufferedReader::new(stream);
loop {
let result = reader.read_line();
match result {
Ok(data) => {
println!("{}", data.as_slice().trim());
}
Err(e) => {
println!("error reading: {}", e);
break;
}
}
}
}
Related
I'm new to network programming and thread in Rust so I may be missing something obvious here. I've been following along with this trying to build a simple chat application. Only, he does it with the standard library and I'm trying to do it with tokio. The functionality is very simple: Client sends a message to Server, Server acknowledges it and sends it back to the Client. Here's my code for the client and server, stripped down as much as I can:
server.rs
#[tokio::main]
async fn main() {
let server = TcpListener::bind("127.0.0.1:7878").await.unwrap();
let mut clients = vec![];
let (tx, mut rx) = mpsc::channel(32);
loop {
if let Ok((socket, addr)) = server.accept().await {
let tx = tx.clone();
let (mut reader, writer) = split(socket);
clients.push(writer);
tokio::spawn(async move {
loop {
let mut buffer = vec![0; 1024];
reader.read(&mut buffer).await.unwrap();
//get message written by the client and print it
//then transmit it on the channel
let msg = buffer.into_iter().take_while(|&x| x != 0).collect::<Vec<_>>();
let msg = String::from_utf8(msg).expect("Invalid utf8 message");
println!("{}: {:?}", addr, msg);
match tx.send(msg).await {
Ok(_) => { ()}
Err(_) => { println!("Error");}
}
}
});
}
//write each message received back to its client
if let Some(msg) = rx.recv().await {
clients = clients.into_iter().filter_map(|mut x| {
println!("writing: {:?}", &msg);
x.write(&msg.clone().into_bytes());
Some(x)
}).collect::<Vec<_>>();
}
}
}
client.rs
#[tokio::main]
async fn main() {
let client = TcpStream::connect("127.0.0.1:7878").await.unwrap();
let (tx, mut rx) = mpsc::channel::<String>(32);
tokio::spawn(async move {
loop {
let mut buffer = vec![0; 1024];
// get message sent by the server and print it
match client.try_read(&mut buffer) {
Ok(_) => {
let msg = buffer.into_iter().take_while(|&x| x != 0).collect::<Vec<_>>();
println!("Received from server: {:?}", msg);
}
Err(ref err) if err.kind() == io::ErrorKind::WouldBlock => {
()
}
Err(_) => {
println!("Connection with server was severed");
break;
}
}
// get message transmitted from user input loop
// then write it to the server
match rx.try_recv() {
Ok(message) => {
let mut buffer = message.clone().into_bytes();
buffer.resize(1024, 0);
match client.try_write(&buffer) {
Ok(_) => { println!("Write successful");}
Err(_) => { println!("Write error");}
}
}
Err(TryRecvError::Empty) => (),
_ => break
}
}
} );
// user input loop here
// takes user message and transmits it on the channel
}
Sending to the server works fine, and the server appears to be successfully writing as indicated by its output:
127.0.0.1:55346: "test message"
writing: "test message"
The issue is the client never reads back the message from the server, instead getting WouldBlock errors every time it hits the match client.try_read(&mut buffer) block.
If I stop the server while keeping the client running, the client is suddenly flooded with successful reads of empty messages:
Received from server: []
Received from server: []
Received from server: []
Received from server: []
Received from server: []
Received from server: []
Received from server: []
Received from server: []
...
Can anyone tell me what's going on?
Here's what happens in your server:
Wait for a client to connect.
When the client is connected, spawn a background task to receive from the client.
Try to read from the channel, since it is very unlikely that the client has already sent anything at this point the channel is empty.
Loop → wait for another client to connect.
While waiting for another client, the background task receives the message from the first client and sends it to the channel, but the main task is blocked waiting for another client and never tries to read again from the channel.
Easiest way to get it to work is to get rid of the channel in the server and simply echo the message from the spawned task.
Another solution is to spawn an independent task to process the channel and write to the clients.
As for what happens when you kill the server: once the connection is lost attempting to read from the socket does not return an error but instead returns an empty buffer.
I'm trying to implement a simple TCP messaging protocol in Rust with the standard TCP Sockets. The protocol is like so:
[C0 FF EE EE] header
[XX XX] type U16LE
[XX XX] size U16LE
... (size - 8) bytes of arbitrary data ...
I have come up with the following code:
const REQUEST_HEADER: [u8; 4] = [0xC0, 0xFF, 0xEE, 0xEE];
pub fn run_server(host: &str, port: &str) {
let listener = TcpListener::bind(format!("{}:{}", host, port))
.expect("Could not bind to the requested host/port combination.");
for stream in listener.incoming() {
thread::spawn(|| {
let stream = stream.unwrap();
handle_client(stream);
});
}
}
fn handle_client(stream: TcpStream) {
stream.set_nodelay(true).expect("set_nodelay call failed");
loop {
match get_packet(stream) {
Ok(deframed) => {
match deframed.msgid {
Some(FrameType::Hello) => say_hello(stream),
Some(FrameType::Text) => {
println!("-- Got text message: {:X?}", deframed.content);
save_text(deframed);
},
Some(FrameType::Goodbye) => {
println!("-- You say goodbye, and I say hello");
break; // end of connection
},
_ => println!("!! I don't know this packet type")
}
},
Err(x) => {
match x {
1 => println!("!! Malformed frame header"),
2 => println!("!! Client gone?"),
_ => println!("!! Unknown error")
}
break;
}
}
}
}
// Read one packet from the stream
fn get_packet(reader: TcpStream) -> Result<MyFrame, u8> {
// read the packet header in here
let mut heading = [0u8; 0x8];
match reader.read_exact(&mut heading) {
Ok(_) => {
// check if header is OK
if heading[..4] == REQUEST_HEADER
{
// extract metadata
let mid: u16 = ((heading[4] as u16) << 0)
+ ((heading[5] as u16) << 8);
let len: u16 = ((heading[6] as u16) << 0)
+ ((heading[7] as u16) << 8);
let remain: u16 = len - 0x8;
println!("-> Pkt: 0x{:X}, length: 0x{:X}, remain: 0x{:X}", mid, len, remain);
let mut frame = vec![0; remain as usize];
reader.read_exact(&mut frame).expect("Read packet failed");
let deframed = MyFrame {
msgid: FromPrimitive::from_u16(mid),
content: frame
};
Ok(deframed)
} else {
println!("!! Expected packet header but got {:X?}", heading.to_vec());
Err(1)
}
},
_ => Err(2)
}
}
This is the Python client:
import socket, sys
sock = socket.socket()
sock.connect(("127.0.0.1", 12345))
f = open("message.bin",'rb')
dat = f.read()
sock.sendall(dat)
while True:
sock.recv(64)
When the program receives a multiple messages in a row, they seem to be processed fine. If the client sends a message and waits for a response, the program gets stuck at reading the frame content until ~16 more bytes are received. As a result, the program cannot send a response to the client.
I can tell it's stuck in there and not somewhere else because the ->Pkt log line appears with the correct metadata but it doesn't continue further processing despite the client having already sent everything.
I've tried replacing read_exact with read but it just keeps being stuck there. Once the client software drops the connection the message is suddenly processed as normal.
Is this a design problem or am I missing out a setting I need to change on the socket?
From the rust std net library:
let listener = TcpListener::bind(("127.0.0.1", port)).unwrap();
info!("Opened socket on localhost port {}", port);
// accept connections and process them serially
for stream in listener.incoming() {
break;
}
info!("closed socket");
How does one make the listener stop listening? It says in the API that when the listener is dropped, it stops. But how do we drop it if incoming() is a blocking call? Preferably without external crates like tokio/mio.
You'll want to put the TcpListener into non-blocking mode using the set_nonblocking() method, like so:
use std::io;
use std::net::TcpListener;
let listener = TcpListener::bind("127.0.0.1:7878").unwrap();
listener.set_nonblocking(true).expect("Cannot set non-blocking");
for stream in listener.incoming() {
match stream {
Ok(s) => {
// do something with the TcpStream
handle_connection(s);
}
Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
// Decide if we should exit
break;
// Decide if we should try to accept a connection again
continue;
}
Err(e) => panic!("encountered IO error: {}", e),
}
}
Instead of waiting for a connection, the incoming() call will immediately return a Result<> type. If Result is Ok(), then a connection was made and you can process it. If the Result is Err(WouldBlock), this isn't actually an error, there just wasn't a connection pending at the exact moment incoming() checked the socket.
Note that in the WouldBlock case, you may want to put a sleep() or something before continuing, otherwise your program will rapidly poll the incoming() function checking for a connection, resulting in high CPU usage.
Code example adapted from here
The standard library doesn't provide an API for this, but there are a few strategies you can use to work around it:
Shut down reads on the socket
You can use platform-specific APIs to shutdown reads on the socket which will cause the incoming iterator to return an error. You can then break out of handling connections when the error is received. For example, on a Unix system:
use std::net::TcpListener;
use std::os::unix::io::AsRawFd;
use std::thread;
let listener = TcpListener::bind("localhost:0")?;
let fd = listener.as_raw_fd();
let handle = thread::spawn(move || {
for connection in listener.incoming() {
match connection {
Ok(connection) => { /* handle connection */ }
Err(_) => break,
}
});
libc::shutdown(fd, libc::SHUT_RD);
handle.join();
Force the listener to wake up
Another (cross-platform) trick is to set a variable indicating that you want to stop listening, and then connect to the socket yourself to force the listening thread to wake up. When the listening thread wakes up, it checks the "stop listening" variable, and then exits cleanly if it's set.
use std::net::{TcpListener, TcpStream};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::thread;
let listener = TcpListener::bind("localhost:0")?;
let local_addr = listener.local_addr()?;
let shutdown = Arc::new(AtomicBool::new(false));
let server_shutdown = shutdown.clone();
let handle = thread::spawn(move || {
for connection in listener.incoming() {
if server_shutdown.load(Ordering::Relaxed) {
return;
}
match connection {
Ok(connection) => { /* handle connection */ }
Err(_) => break,
}
}
});
shutdown.store(true, Ordering::Relaxed);
let _ = TcpStream::connect(local_addr);
handle.join().unwrap();
You can poll your socket with an eventfd, which used for signaling.
I wrote a helper for this.
let shutdown = EventFd::new();
let listener = TcpListener::bind("0.0.0.0:12345")?;
let incoming = CancellableIncoming::new(&listener, &shutdown);
for stream in incoming {
// Your logic
}
// While in other thread
shutdown.add(1); // Light the shutdown signal, now your incoming loop exits gracefully.
use nix;
use nix::poll::{poll, PollFd, PollFlags};
use nix::sys::eventfd::{eventfd, EfdFlags};
use nix::unistd::{close, write};
use std;
use std::net::{TcpListener, TcpStream};
use std::os::unix::io::{AsRawFd, RawFd};
pub struct EventFd {
fd: RawFd,
}
impl EventFd {
pub fn new() -> Self {
EventFd {
fd: eventfd(0, EfdFlags::empty()).unwrap(),
}
}
pub fn add(&self, v: i64) -> nix::Result<usize> {
let b = v.to_le_bytes();
write(self.fd, &b)
}
}
impl AsRawFd for EventFd {
fn as_raw_fd(&self) -> RawFd {
self.fd
}
}
impl Drop for EventFd {
fn drop(&mut self) {
let _ = close(self.fd);
}
}
// -----
//
pub struct CancellableIncoming<'a> {
listener: &'a TcpListener,
eventfd: &'a EventFd,
}
impl<'a> CancellableIncoming<'a> {
pub fn new(listener: &'a TcpListener, eventfd: &'a EventFd) -> Self {
Self { listener, eventfd }
}
}
impl<'a> Iterator for CancellableIncoming<'a> {
type Item = std::io::Result<TcpStream>;
fn next(&mut self) -> Option<std::io::Result<TcpStream>> {
use nix::errno::Errno;
let fd = self.listener.as_raw_fd();
let evfd = self.eventfd.as_raw_fd();
let mut poll_fds = vec![
PollFd::new(fd, PollFlags::POLLIN),
PollFd::new(evfd, PollFlags::POLLIN),
];
loop {
match poll(&mut poll_fds, -1) {
Ok(_) => break,
Err(nix::Error::Sys(Errno::EINTR)) => continue,
_ => panic!("Error polling"),
}
}
if poll_fds[0].revents().unwrap() == PollFlags::POLLIN {
Some(self.listener.accept().map(|p| p.0))
} else if poll_fds[1].revents().unwrap() == PollFlags::POLLIN {
None
} else {
panic!("Can't be!");
}
}
}
I want to implement a futures::Stream for reading and parsing the standard output of a child subprocess.
What I'm doing at the moment:
spawn subprocess and obtain its stdout via std::process methods: let child = Command::new(...).stdout(Stdio.pipe()).spawn().expect(...)
add AsyncRead and BufRead to stdout:
let stdout = BufReader::new(tokio_io::io::AllowStdIo::new(
child.stdout.expect("Failed to open stdout"),
));
declare a wrapper struct for stdout:
struct MyStream<Io: AsyncRead + BufRead> {
io: Io,
}
implement Stream:
impl<Io: AsyncRead + BufRead> Stream for MyStream<Io> {
type Item = Message;
type Error = Error;
fn poll(&mut self) -> Poll<Option<Message>, Error> {
let mut line = String::new();
let n = try_nb!(self.io.read_line(&mut line));
if n == 0 {
return Ok(None.into());
}
//...read & parse further
}
}
The problem is that AllowStdIo doesn't make ChildStdout magically asynchronous and the self.io.read_line call still blocks.
I guess I need to pass something different instead of Stdio::pipe() to have it asynchronous, but what? Or is there a different solution for that?
This question is different from What is the best approach to encapsulate blocking I/O in future-rs? because I want to get asynchronous I/O for the specific case of a subprocess and not solve the problem of encapsulation of synchronous I/O.
Update: I'm using tokio = "0.1.3" to leverage its runtime feature and using tokio-process is not an option at the moment (https://github.com/alexcrichton/tokio-process/issues/27)
The tokio-process crate provides you with a CommandExt trait that allows you to spawn a command asynchronously.
The resulting Child has a getter for ChildStdout which implements Read and is non-blocking.
Wrapping tokio_process::ChildStdout into AllowStdIo as you did in your example should make it work!
Here is my version using tokio::process
let mut child = match Command::new(&args.run[0])
.args(parameters)
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.kill_on_drop(true)
.spawn()
{
Ok(c) => c,
Err(e) => panic!("Unable to start process `{}`. {}", args.run[0], e),
};
let stdout = child.stdout.take().expect("child did not have a handle to stdout");
let stderr = child.stderr.take().expect("child did not have a handle to stderr");
let mut stdout_reader = BufReader::new(stdout).lines();
let mut stderr_reader = BufReader::new(stderr).lines();
loop {
tokio::select! {
result = stdout_reader.next_line() => {
match result {
Ok(Some(line)) => println!("Stdout: {}", line),
Err(_) => break,
_ => (),
}
}
result = stderr_reader.next_line() => {
match result {
Ok(Some(line)) => println!("Stderr: {}", line),
Err(_) => break,
_ => (),
}
}
result = child.wait() => {
match result {
Ok(exit_code) => println!("Child process exited with {}", exit_code),
_ => (),
}
break // child process exited
}
};
}
I'd like to create a curl-like function in Rust.
So far, here is the code I use:
match Url::parse(url) {
Ok(u) => {
match TcpStream::connect(u.host.as_slice(), 80) {
Ok(mut socket) => {
let req = format!(
"GET {:s} HTTP/1.1\r\nHost: {:s}\r\nAccept: */*\r\nContent-Length: 0\r\nContent-Type: aplication/x-www-form-urlencoded\r\n",
u.path.path.as_slice(), u.host
);
socket.write(req.as_bytes());
match socket.read_to_string() {
Ok(res) => println!("{}", res.as_slice()),
Err(e) => fail!("Error: {}", e)
};
drop(socket);
},
Err(e) => fail!("Error: {}", e)
};
},
Err(e) => fail!("Error: {}", e)
};
The problem is, if the HTTP request is properly formatted,
println!("{}", res.as_slice())
will not display anything, with any url.
If it's not well formatted the same code will display 403 or 400 errors. Is that supposed to mean that the socket var is well populated?
Where could the problem come from? The server or the request?
By the way, the code being a little too long, I'd like to use the macro try! but it raises an error when doing, for instance:
try!(TcpStream::connect(u.host.as_slice(), 80)
It looks like your request really is incorrect, but not in a way the server could tell you.
GET {:s} HTTP/1.1\r\nHost: {:s}\r\nAccept: */*\r\nContent-Length: 0\r\nContent-Type: aplication/x-www-form-urlencoded\r\n
You see, an HTTP request is supposed to be ended in two newlines, that is, two \r\n sequences. Also GET request cannot have body (though most web servers will probably ignore that):
GET {:s} HTTP/1.1\r\nHost: {:s}\r\nAccept: */*\r\n\r\n
This request should work.
As for try!, it seems that you're trying to use it in a main() function or something like this, right? try! is intended to return from the function upon error, that is, this:
try!(TcpStream::connect(u.host.as_slice(), 80))
is rewritten into
match TcpStream::connect(u.host.as_slice(), 80) {
Ok(s) => s,
Err(e) => return Err(e)
}
If your function does not return Result, it won't work. So your code will look better like this:
extern crate url;
use std::io::{IoResult, File, TcpStream};
use url::Url;
fn download_file(url: &Url, file: &Path) -> IoResult<()> {
let mut socket = try!(TcpStream::connect(url.host.as_slice(), 80));
let req = format!(
"GET {:s} HTTP/1.1\r\nHost: {:s}\r\nAccept: */*\r\n\r\n",
url.path.path.as_slice(), url.host
);
try!(socket.write(req.as_bytes()));
let res = try!(socket.read_to_end());
let mut file = File::create(file);
try!(file.write(res.as_slice()));
Ok(())
}
fn main() {
let url = "http://kitten.jpg.to/";
match Url::parse(url) {
Ok(url) => match download_file(&url, &Path::new("/tmp/kitten_link.http")) {
Ok(_) => println!("Download successful"),
Err(e) => fail!("Error: {}", e)
},
Err(e) => fail!("Error: {}", e)
}
}
BTW, you also don't need to drop sockets or other things explicitly. They are destroyed automatically when they go out of scope. I've also changed read_to_string() to read_to_end() because binary files will not be correct after they are passed through a String.