I've been reading through the Rust documentation and made it to section 4.26 before looking at the libraries included. std::net::TcpStream caught my eye but I don't understand the following lines:
let _ = stream.write(&[1]);
let _ = stream.read(&mut [0; 128]);
I have seen [0; 128] before under Vectors as vec![0;10] // Ten 0s so I can see that a buffer of 128 0s is passed in. The documentation for read says "Pull some bytes from this source into the specified buffer, returning how many bytes were read." so how can you access the data that was read into the buffer?
The comment in the code indicates that the result is ignored:
let _ = stream.read(&mut [0; 128]); // ignore here too
To get the data, you need to create a variable:
let mut buffer = [0; 128];
let _ = stream.read(&mut buffer);
// The data is in buffer.
Related
I have a TCP Client in rust, which should communicate with a Java Server. I got the basics working and can send bytearrays between them.
But for the bytearray buffer, I need to know the length of the bytearray. But I don't know I should obtain it. At the moment, I only have a fixed size for the buffer right now.
My Rust code looks like this:
loop {
let mut buffer = vec![0; 12]; //fixed buffer length
let n = stream.read(&mut buffer).await;
let text = from_utf8(&buffer).unwrap();
println!("{}", text);
}
In Java, you can send the size of the buffer directly as an Integer with DataInputStream. Is there any option to do that in rust?
For example, this is how I'm doing it in Java:
public String readMsg(Socket socket) throws IOException {
DataInputStream in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
byte[] bytes = new byte[in.readInt()]; //dynamic buffer length
in.readFully(bytes);
return new String(bytes, StandardCharsets.US_ASCII);
}
What you want to know is a property of the protocol that you are using. It's not a property of the programming language you use. Based on your Java code it seems like you are using a protocol which sends a 4 byte length field before the message data (signed/unsigned?).
If that is the case you can handle reading the message the same way in Rust:
1. Read the 4 bytes in order to obtain the length information
2. Read the remaining data
3. Deserialize the data
fn read_message(stream: Read) -> io::Result<String> {
let mut buffer = [0u8; 4];
// Read the length information
stream.read_exact(&mut buffer[..])?;
// Deserialize the length
let size = u32::from_be_bytes(buffer);
// Allocate a buffer for the message
// Be sure to check against a maximum size before doing this in production
let mut payload = vec![0; size];
stream.read_exact(&mut payload[..]).await;
// Convert the buffer into a string
let text = String::from_utf8(payload).map_err(/* omitted */)?;
println!("{}", text);
Ok(text)
}
This obviously is only correct if your protocol uses length prefixed messages with a 4byte unsigned int prefix. This is something that you need to check.
My goal is to read some bytes from a TcpStream in order to parse the data in each message and build a struct from it.
loop {
let mut buf: Vec<u8> = Vec::new();
let len = stream.read(&mut buf)?;
if 0 == len {
//Disconnected
}
println!("read() -> {}", len);
}
Like in Python, I thought the stream.read() would block until it received some data.
So I've set up a server that calls the loop you see above for each incoming connection. I've then tried to connect to the server with netcat; netcat connects successfully to the server and blocks on the stream.read(), which is what I want; but as soon as I send some data, read() returns 0.
I've also tried doing something similar with stream.read_to_end() but it only appears to only return when the connection is closed.
How can I read from the TcpStream, message per message, knowing that each message can have a different, unknown, size ?
You're getting caught with your pants down by an underlying technicality of Vec more than by std::io::Read, although they both interact in this particular case.
The definition and documentation of Read states:
If the return value of this method is Ok(n), then it must be guaranteed that 0 <= n <= buf.len(). A nonzero n value indicates that the buffer buf has been filled in with n bytes of data from this source. If n is 0, then it can indicate one of two scenarios:
The important part is bolded.
When you define a new Vec the way you did, it starts with a capacity of zero. This means that the underlying slice (that you will use as a buffer) has a length of zero. As a result, since it must be guaranteed that 0 <= n <= buf.len() and since buf.len() is zero, your read() call immediately returns with 0 bytes read.
To "fix" this, you can either assign a default set of elements to your Vec (Vec::new().resize(1024, 0)), or just use an array from the get-go (let mut buffer:[u8; 1024] = [0; 1024])
I want to read data from a TCP stream but it results in an empty Vec:
extern crate net2;
use net2::TcpBuilder;
use std::io::Read;
use std::io::Write;
use std::io::BufReader;
let tcp = TcpBuilder::new_v4().unwrap();
let mut stream = tcp.connect("127.0.0.1:3306").unwrap();
let mut buf = Vec::with_capacity(1024);
stream.read(&mut buf);
println!("{:?}", buf); // prints []
When I use stream.read_to_end the buffer is filled but this takes way too long.
In Python I can do something like
import socket
TCP_IP = '127.0.0.1'
TCP_PORT = 3306
BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
#s.send(MESSAGE)
data = s.recv(BUFFER_SIZE)
s.close()
print "received data:", data
How can I achieve this in Rust?
The two methods you tried don't work for different reasons:
read(): "does not provide any guarantees about whether it blocks waiting for data". In general, read() is unreliable from a users perspective and should only be used as a building block for higher level functions, like read_to_end().
But maybe more importantly, you have a bug in your code: you create your vector via with_capacity() which reserves memory internally, but doesn't change the length of the vector. It is still empty! When you now slice it like &buf, you pass an empty slice to read(), thus read() cannot read any actual data. To fix that, the elements of your vector need to be initialized: let mut buf = vec![0; 1024] or something like that.
read_to_end(): calls read() repeatedly until EOF is encountered. This doesn't really make sense in most TCP stream situations.
So what should you use instead? In your Python code you read a specific number of bytes into a buffer. You can do that in Rust, too: read_exact(). It works like this:
const BUFFER_SIZE: usize = 1024;
let mut stream = ...;
let mut buf = [0; BUFFER_SIZE];
stream.read_exact(&mut buf);
println!("{:?}", buf);
You could also use take(). That way you can use read_to_end():
const BUFFER_SIZE: usize = 1024;
let mut stream = ...;
let mut buf = Vec::with_capacity(BUFFER_SIZE);
stream.take(BUFFER_SIZE).read_to_end(&mut buf);
println!("{:?}", buf);
If you want to use the stream multiple times, you probably want to use by_ref() before calling take().
The two code snippets are not equivalent though! Please read the documentation for more details.
I am experimenting with an API where the callers provide a Vec<Point> which they want me to fill in with data. They can allocate room in the vector by creating it with with_capacity then push out its size by doing push(Point{...}). Then I will fill the underlying buffer with bytes from the disk, switch their endian representation as necessary, and then provide it back as a Vec<Point>.
Here is the function which takes their vector and fills it with data. The problem is that transmute only works when the types are of the same size, a Point is 12 bytes and transmute throws away 11 of those bytes.
fn read_points(&self, offset: u64, points: &mut [point::Point]) {
let mut file = self.handle.borrow_mut();
file.seek(SeekFrom::Start(offset)).unwrap();
// bleep bloorp. danger!
let points_buf : &mut [u8] = unsafe { mem::transmute(points) };
file.read(points_buf).unwrap();
// should take the 12 bytes and do the endian swaps
for mut chunk in points_buf.chunks_mut(point::POINT_SIZE) {
let point = point::buf_to_point(chunk);
let buf : &mut [u8] = &mut chunk;
point::fill_buf(buf, point.timestamp, point.value);
}
}
Can this API be done in Rust or should I should I switch to doing safer but slower copy operations?
The memory representation of &mut [T] is (*mut T, usize) where usize is the number of T elements in the slice, not the number of bytes. So transmuting a slice of say 20 points gives you a slice of 20 bytes.
You have to compute the correct number of bytes:
let n_bytes = points.len() * std::mem::size_of::<Point>();
let points_buf = std::slice::from_raw_parts_mut(points.as_mut_ptr(), n_bytes);
(Then of course deal of all the rest of the unsafety.)
I want to create a Vec<T> and make some room for it, but I don't know how to do it, and, to my surprise, there is almost nothing in the official documentation about this basic type.
let mut v: Vec<i32> = Vec<i32>(SIZE); // How do I do this ?
for i in 0..SIZE {
v[i] = i;
}
I know I can create an empty Vec<T> and fill it with pushes, but I don't want to do that since I don't always know, when writing a value at index i, if a value was already inserted there yet. I don't want to write, for obvious performance reasons, something like :
if i >= len(v) {
v.push(x);
} else {
v[i] = x;
}
And, of course, I can't use the vec! syntax either.
While vec![elem; count] from the accepted answer is sufficient to create a vector with all elements equal to the same value, there are other convenience functions.
Vec::with_capacity() creates a vector with the given capacity but with zero length. It means that until this capacity is reached, push() calls won't reallocate the vector, making push() essentially free:
fn main() {
let mut v = Vec::with_capacity(10);
for i in 0..10 {
v.push(i);
}
println!("{:?}", v);
}
You can also easily collect() a vector from an iterator. Example:
fn main() {
let v: Vec<_> = (1..10).collect();
println!("{:?}", v);
}
And finally, sometimes your vector contains values of primitive type and is supposed to be used as a buffer (e.g. in network communication). In this case you can use Vec::with_capacity() + set_len() unsafe method:
fn main() {
let mut v = Vec::with_capacity(10);
unsafe { v.set_len(10); }
for i in 0..10 {
v[i] = i;
}
println!("{:?}", v);
}
Note that you have to be extra careful if your vector contains values with destructors or references - it's easy to get a destructor run over a uninitialized piece of memory or to get an invalid reference this way. It will also work right if you only use initialized part of the vector (you have to track it yourself now). To read about all the possible dangers of uninitialized memory, you can read the documentation of mem::uninitialized().
You can use the first syntax of the vec! macro, specifically vec![elem; count]. For example:
vec![1; 10]
will create a Vec<_> containing 10 1s (the type _ will be determined later or default to i32). The elem given to the macro must implement Clone. The count can be a variable, too.
There is the Vec::resize method:
fn resize(&mut self, new_len: usize, value: T)
This code resizes an empty vector to 1024 elements by filling with the value 7:
let mut vec: Vec<i32> = Vec::new();
vec.resize(1024, 7);