My goal is to read some bytes from a TcpStream in order to parse the data in each message and build a struct from it.
loop {
let mut buf: Vec<u8> = Vec::new();
let len = stream.read(&mut buf)?;
if 0 == len {
//Disconnected
}
println!("read() -> {}", len);
}
Like in Python, I thought the stream.read() would block until it received some data.
So I've set up a server that calls the loop you see above for each incoming connection. I've then tried to connect to the server with netcat; netcat connects successfully to the server and blocks on the stream.read(), which is what I want; but as soon as I send some data, read() returns 0.
I've also tried doing something similar with stream.read_to_end() but it only appears to only return when the connection is closed.
How can I read from the TcpStream, message per message, knowing that each message can have a different, unknown, size ?
You're getting caught with your pants down by an underlying technicality of Vec more than by std::io::Read, although they both interact in this particular case.
The definition and documentation of Read states:
If the return value of this method is Ok(n), then it must be guaranteed that 0 <= n <= buf.len(). A nonzero n value indicates that the buffer buf has been filled in with n bytes of data from this source. If n is 0, then it can indicate one of two scenarios:
The important part is bolded.
When you define a new Vec the way you did, it starts with a capacity of zero. This means that the underlying slice (that you will use as a buffer) has a length of zero. As a result, since it must be guaranteed that 0 <= n <= buf.len() and since buf.len() is zero, your read() call immediately returns with 0 bytes read.
To "fix" this, you can either assign a default set of elements to your Vec (Vec::new().resize(1024, 0)), or just use an array from the get-go (let mut buffer:[u8; 1024] = [0; 1024])
Related
I'm new to Rust programing and I have a bit of difficulty when this language is different from C Example, I have a C function as follows:
bool check(char* data, int size){
int i;
for(i = 0; i < size; i++){
if( data[i] != 0x00){
return false;
}
}
return true;
}
How can I convert this function to Rust? I tried it like C, but it has Errors :((
First off, I assume that you want to use as little unsafe code as possible. Otherwise there really isn't any reason to use Rust in the first place, as you forfeit all the advantages it brings you.
Depending on what data represents, there are multiple ways to transfer this to Rust.
First off: Using pointer and length as two separate arguments is not possible in Rust without unsafe. It has the same concept, though; it's called slices. A slice is exactly the same as a pointer-size combination, just that the compiler understands it and checks it for correctness at compile time.
That said, a char* in C could actually be one of four things. Each of those things map to different types in Rust:
Binary data whose deallocation is taken care of somewhere else (in Rust terms: borrowed data)
maps to &[u8], a slice. The actual content of the slice is:
the address of the data as *u8 (hidden from the user)
the length of the data as usize
Binary data that has to be deallocated within this function after using it (in Rust terms: owned data)
maps to Vec<u8>; as soon as it goes out of scope the data is deleted
actual content is:
the address of the data as *u8 (hidden from the user)
the length of the data as usize
the size of the allocation as usize. This allows for efficient push()/pop() operations. It is guaranteed that the length of the data does not exceed the size of the allocation.
A string whose deallocation is taken care of somewhere else (in Rust terms: a borrowed string)
maps to &str, a so called string slice.
This is identical to &[u8] with the additional compile time guarantee that it contains valid UTF-8 data.
A string that has to be deallocated within this function after using it (in Rust terms: an owned string)
maps to String
same as Vec<u8> with the additional compile time guarantee that it contains valid UTF-8 data.
You can create &[u8] references from Vec<u8>'s and &str references from Strings.
Now this is the point where I have to make an assumption. Because the function that you posted checks if all of the elements of data are zero, and returns false if if finds a non-zero element, I assume the content of data is binary data. And because your function does not contain a free call, I assume it is borrowed data.
With that knowledge, this is how the given function would translate to Rust:
fn check(data: &[u8]) -> bool {
for d in data {
if *d != 0x00 {
return false;
}
}
true
}
fn main() {
let x = vec![0, 0, 0];
println!("Check {:?}: {}", x, check(&x));
let y = vec![0, 1, 0];
println!("Check {:?}: {}", y, check(&y));
}
Check [0, 0, 0]: true
Check [0, 1, 0]: false
This is quite a direct translation; it's not really idiomatic to use for loops a lot in Rust. Good Rust code is mostly iterator based; iterators are most of the time zero-cost abstraction that can get compiled very efficiently.
This is how your code would look like if rewritten based on iterators:
fn check(data: &[u8]) -> bool {
data.iter().all(|el| *el == 0x00)
}
fn main() {
let x = vec![0, 0, 0];
println!("Check {:?}: {}", x, check(&x));
let y = vec![0, 1, 0];
println!("Check {:?}: {}", y, check(&y));
}
Check [0, 0, 0]: true
Check [0, 1, 0]: false
The reason this is more idiomatic is that it's a lot easier to read for someone who hasn't written it. It clearly says "return true if all elements are equal to zero". The for based code needs a second to think about to understand if its "all elements are zero", "any element is zero", "all elements are non-zero" or "any element is non-zero".
Note that both versions compile to the exact same bytecode.
Also note that, unlike the C version, the Rust borrow checker guarantees at compile time that data is valid. It's impossible in Rust (without unsafe) to produce a double free, a use-after-free, an out-of-bounds array access or any other kind of undefined behaviour that would cause memory corruption.
This is also the reason why Rust doesn't do pointers without unsafe - it needs the length of the data to check out-of-bounds errors at runtime. That means, accessing data via [] operator is a little more costly in Rust (as it does perform an out-of-bounds check every time), which is the reason why iterator based programming is a thing. Iterators can iterate over data a lot more efficient than directly accessing it via [] operators.
https://golang.org/src/net/http/server.go#L139
I would've expected the signature to be Write([]byte) error, not Write([]byte) (int, error). I also can't find any good explanation from looking through usages, and the documentation comment doesn't explain the return values.
What does the returned int mean?
The ResponseWriter.Write() method is to implement the general io.Writer interface, so a value of http.ResponseWriter can be used / passed to any utility function that accepts / works on an io.Writer.
io.Writer has exactly one Write() method and it details exactly the "contract" of Write, what it should return and how it should work:
type Writer interface {
Write(p []byte) (n int, err error)
}
Write writes len(p) bytes from p to the underlying data stream. It returns the number of bytes written from p (0 <= n <= len(p)) and any error encountered that caused the write to stop early. Write must return a non-nil error if it returns n < len(p).
I thought rust makes another data on the heap memory when modifying the string. Therefore I expected a pointer address would change when I push a value to the string variable.
fn main() {
let mut hello = String::from("hello");
println!("{:?}", hello.as_ptr()); // 0x7fcfa7c01be0
hello.push_str(", world!");
println!("{:?}", hello.as_ptr()); // 0x7fcfa7c01be0
}
However, the result shows it's not. The address of the pointers was not changed, so I tested it with vector type.
fn main() {
let mut numbers = vec![1, 2, 3];
println!("{:?}", numbers.as_ptr()); // 0x7ffac4401be0
numbers.push(4);
println!("{:?}", numbers.as_ptr()); // 0x7ffac4401ce0
}
The pointer address of the vector variable was changed when modifying it. What is the difference between the memory of string and vector type?
Vec<T> and String may maintain extra space to avoid allocating on every push operation. This provides amortized O(1) time for push operations.
It happens to be the case that the vec! macro is guaranteed to create a vector without such extra space, while String::from(&str) does not have such a guarantee.
See https://doc.rust-lang.org/std/vec/struct.Vec.html#capacity-and-reallocation for more details.
A String is like a Vec<T>¹ in that it has both a length and a capacity. If the capacity of the current allocation is big enough to hold the new string, the underlying buffer does not need to be reallocated. The documentation for Vec<T> explains it this way:
The capacity of a vector is the amount of space allocated for any future elements that will be added onto the vector. This is not to be confused with the length of a vector, which specifies the number of actual elements within the vector. If a vector's length exceeds its capacity, its capacity will automatically be increased, but its elements will have to be reallocated.
For example, a vector with capacity 10 and length 0 would be an empty vector with space for 10 more elements. Pushing 10 or fewer elements onto the vector will not change its capacity or cause reallocation to occur.
However, even if the capacity does change, the pointer value is still not guaranteed to move. The system allocator itself may be able to resize the allocation without moving it if there is enough unallocated space adjacent to it. That appears to be what's happening in your code. If you print the capacity along with the pointer, you can observe this behavior:
let mut hello = String::from("hello");
for _ in 0..10 {
println!("({:3}) {:?}", hello.capacity(), hello.as_ptr()); // 0x7fcfa7c01be0
hello.push_str(", world!");
}
( 5) 0x557624d8da40
( 13) 0x557624d8da40
( 26) 0x557624d8dba0
( 52) 0x557624d8dba0
( 52) 0x557624d8dba0
( 52) 0x557624d8dba0
(104) 0x557624d8dba0
(104) 0x557624d8dba0
(104) 0x557624d8dba0
(104) 0x557624d8dba0
In this example, the buffer was resized 4 times, but the contents were only moved once.
¹ Actually, a String is a newtyped Vec<u8>, which explains why they work the same.
Is it safe to call set_len on Vec that has declared capacity? Like this:
let vec = unsafe {
let temp = Vec::with_capacity(N);
temp.set_len(N);
temp
}
I need my Vector to be of size N before any elements are to be added.
Looking at docs:
https://doc.rust-lang.org/collections/vec/struct.Vec.html#capacity-and-reallocation
https://doc.rust-lang.org/collections/vec/struct.Vec.html#method.with_capacity
https://doc.rust-lang.org/collections/vec/struct.Vec.html#method.set_len
I'm a bit confused. Docs say that with_capacity doesn't change length and set_len says that caller must insure vector has proper length. So is this safe?
The reason I need this is because I was looking for a way to declare a mutable buffer (&mut [T]) of size N and Vec seems to fit the bill the best. I just wanted to avoid having my types implement Clone that vec![0;n] would bring.
The docs are just a little ambiguously stated. The wording could be better. Your code example is as "safe" as the following stack-equivalent:
let mut arr: [T; N] = mem::uninitialized();
Which means that as long as you write to an element of the array before reading it you are fine. If you read before writing, you open the door to nasal demons and memory unsafety.
I just wanted to avoid clone that vec![0;n] would bring.
llvm will optimize this to a single memset.
If by "I need my Vector to be of size N" you mean you need memory to be allocated for 10 elements, with_capacity is already doing that.
If you mean you want to have a vector with length 10 (not sure why you would, though...) you need to initialize it with an initial value.
i.e.:
let mut temp: Vec<i32> = Vec::with_capacity(10); // allocate room in memory for
// 10 elements. The vector has
// initial capacity 10, length will be the
// number of elements you push into it
// (initially 0)
v.push(1); // now length is 1, capacity still 10
vs
let mut v: Vec<i32> = vec![0; 10]; // create a vector with 10 elements
// initialized to 0. You can mutate
// those in place later.
// At this point, length = capacity = 10
v[0] = 1; // mutating first element to 1.
// length and capacity are both still 10
http://play.golang.org/p/uRHG-Th_2P
I am having hard time understanding the concept of channel
package main
import (
"fmt"
)
func Fibonacci(limit int, chnvar chan int) {
x, y := 0, 1
for i := 0; i < limit; i++ {
chnvar <- x
x, y = y, x+y
}
close(chnvar)
v, ok := <-chnvar
fmt.Println(v, ok)
}
func main() {
chn := make(chan int, 10)
go Fibonacci(cap(chn), chn)
for elem := range chn {
fmt.Printf("%v ", elem)
}
}
//1 1 2 3 5 8 13 21 34
1)
How do I get false value from the line
v, ok := <-chnvar
It says false if there are no more values to get.
and also false if the channel is closed.
But in this case, the channel is closed but(?) still get the true value.
And if I take out the close, it panics.
How and why it returns true here?
2)
The line
go Fibonacci(cap(chn), chn)
also runs without goroutine.
What is the difference? Just matter of performance.
Thanks in advance
Your Fibonacci function stuffs 10 values into the channel (which has a buffer of 10 values), and then closes it. Assuming the v, ok <- chnvar statement executes before the main goroutine reads everything out of the channel (very likely, but not guaranteed), there will be a value to read so ok will be true.
If you remove the close call, the for loop in the main goroutine will eventually empty the channel's buffer and block waiting for more data. Since there is no other goroutine active to write to the channel, the runtime detects this as a deadlock.
Your sample program runs with Fibonacci called directly (not as a goroutine) because the channel it writes to is buffered, and it never overruns the buffer. Therefore it can complete without blocking and allows execution to continue to the rest of the main function.
If the channel was not buffered, or you wrote more values than would fit in the buffer, then Fibonacci would block waiting for some other goroutine to read something from the channel.
1)
The Go specification states for channel receive operations (my emphasis):
x, ok := <-ch
The value of ok is true if the value received was delivered by a successful send operation to the channel, or false if it is a zero value generated because the channel is closed and empty.
That is, because the buffered channel is not empty and you have successfully received a value (0), ok will be true. You won't receive false until the channel has been emptied.
2)
By running the Fibonacci(cap(chn), chn) in it's own Go routine, main can start receiving and processing (printing out) out values while the Fibonacci function is still feeding new values to the channel.
In your case, this probably never happens since the function will have filled up the buffer and completed before main gets a chance to process anything.
If it would not be running in a Go routine, Fibonacci would first need to produce all the values before they can be processed further by main.