Is it safe to call set_len on Vec that has declared capacity? Like this:
let vec = unsafe {
let temp = Vec::with_capacity(N);
temp.set_len(N);
temp
}
I need my Vector to be of size N before any elements are to be added.
Looking at docs:
https://doc.rust-lang.org/collections/vec/struct.Vec.html#capacity-and-reallocation
https://doc.rust-lang.org/collections/vec/struct.Vec.html#method.with_capacity
https://doc.rust-lang.org/collections/vec/struct.Vec.html#method.set_len
I'm a bit confused. Docs say that with_capacity doesn't change length and set_len says that caller must insure vector has proper length. So is this safe?
The reason I need this is because I was looking for a way to declare a mutable buffer (&mut [T]) of size N and Vec seems to fit the bill the best. I just wanted to avoid having my types implement Clone that vec![0;n] would bring.
The docs are just a little ambiguously stated. The wording could be better. Your code example is as "safe" as the following stack-equivalent:
let mut arr: [T; N] = mem::uninitialized();
Which means that as long as you write to an element of the array before reading it you are fine. If you read before writing, you open the door to nasal demons and memory unsafety.
I just wanted to avoid clone that vec![0;n] would bring.
llvm will optimize this to a single memset.
If by "I need my Vector to be of size N" you mean you need memory to be allocated for 10 elements, with_capacity is already doing that.
If you mean you want to have a vector with length 10 (not sure why you would, though...) you need to initialize it with an initial value.
i.e.:
let mut temp: Vec<i32> = Vec::with_capacity(10); // allocate room in memory for
// 10 elements. The vector has
// initial capacity 10, length will be the
// number of elements you push into it
// (initially 0)
v.push(1); // now length is 1, capacity still 10
vs
let mut v: Vec<i32> = vec![0; 10]; // create a vector with 10 elements
// initialized to 0. You can mutate
// those in place later.
// At this point, length = capacity = 10
v[0] = 1; // mutating first element to 1.
// length and capacity are both still 10
Related
I'm new to Rust programing and I have a bit of difficulty when this language is different from C Example, I have a C function as follows:
bool check(char* data, int size){
int i;
for(i = 0; i < size; i++){
if( data[i] != 0x00){
return false;
}
}
return true;
}
How can I convert this function to Rust? I tried it like C, but it has Errors :((
First off, I assume that you want to use as little unsafe code as possible. Otherwise there really isn't any reason to use Rust in the first place, as you forfeit all the advantages it brings you.
Depending on what data represents, there are multiple ways to transfer this to Rust.
First off: Using pointer and length as two separate arguments is not possible in Rust without unsafe. It has the same concept, though; it's called slices. A slice is exactly the same as a pointer-size combination, just that the compiler understands it and checks it for correctness at compile time.
That said, a char* in C could actually be one of four things. Each of those things map to different types in Rust:
Binary data whose deallocation is taken care of somewhere else (in Rust terms: borrowed data)
maps to &[u8], a slice. The actual content of the slice is:
the address of the data as *u8 (hidden from the user)
the length of the data as usize
Binary data that has to be deallocated within this function after using it (in Rust terms: owned data)
maps to Vec<u8>; as soon as it goes out of scope the data is deleted
actual content is:
the address of the data as *u8 (hidden from the user)
the length of the data as usize
the size of the allocation as usize. This allows for efficient push()/pop() operations. It is guaranteed that the length of the data does not exceed the size of the allocation.
A string whose deallocation is taken care of somewhere else (in Rust terms: a borrowed string)
maps to &str, a so called string slice.
This is identical to &[u8] with the additional compile time guarantee that it contains valid UTF-8 data.
A string that has to be deallocated within this function after using it (in Rust terms: an owned string)
maps to String
same as Vec<u8> with the additional compile time guarantee that it contains valid UTF-8 data.
You can create &[u8] references from Vec<u8>'s and &str references from Strings.
Now this is the point where I have to make an assumption. Because the function that you posted checks if all of the elements of data are zero, and returns false if if finds a non-zero element, I assume the content of data is binary data. And because your function does not contain a free call, I assume it is borrowed data.
With that knowledge, this is how the given function would translate to Rust:
fn check(data: &[u8]) -> bool {
for d in data {
if *d != 0x00 {
return false;
}
}
true
}
fn main() {
let x = vec![0, 0, 0];
println!("Check {:?}: {}", x, check(&x));
let y = vec![0, 1, 0];
println!("Check {:?}: {}", y, check(&y));
}
Check [0, 0, 0]: true
Check [0, 1, 0]: false
This is quite a direct translation; it's not really idiomatic to use for loops a lot in Rust. Good Rust code is mostly iterator based; iterators are most of the time zero-cost abstraction that can get compiled very efficiently.
This is how your code would look like if rewritten based on iterators:
fn check(data: &[u8]) -> bool {
data.iter().all(|el| *el == 0x00)
}
fn main() {
let x = vec![0, 0, 0];
println!("Check {:?}: {}", x, check(&x));
let y = vec![0, 1, 0];
println!("Check {:?}: {}", y, check(&y));
}
Check [0, 0, 0]: true
Check [0, 1, 0]: false
The reason this is more idiomatic is that it's a lot easier to read for someone who hasn't written it. It clearly says "return true if all elements are equal to zero". The for based code needs a second to think about to understand if its "all elements are zero", "any element is zero", "all elements are non-zero" or "any element is non-zero".
Note that both versions compile to the exact same bytecode.
Also note that, unlike the C version, the Rust borrow checker guarantees at compile time that data is valid. It's impossible in Rust (without unsafe) to produce a double free, a use-after-free, an out-of-bounds array access or any other kind of undefined behaviour that would cause memory corruption.
This is also the reason why Rust doesn't do pointers without unsafe - it needs the length of the data to check out-of-bounds errors at runtime. That means, accessing data via [] operator is a little more costly in Rust (as it does perform an out-of-bounds check every time), which is the reason why iterator based programming is a thing. Iterators can iterate over data a lot more efficient than directly accessing it via [] operators.
I thought rust makes another data on the heap memory when modifying the string. Therefore I expected a pointer address would change when I push a value to the string variable.
fn main() {
let mut hello = String::from("hello");
println!("{:?}", hello.as_ptr()); // 0x7fcfa7c01be0
hello.push_str(", world!");
println!("{:?}", hello.as_ptr()); // 0x7fcfa7c01be0
}
However, the result shows it's not. The address of the pointers was not changed, so I tested it with vector type.
fn main() {
let mut numbers = vec![1, 2, 3];
println!("{:?}", numbers.as_ptr()); // 0x7ffac4401be0
numbers.push(4);
println!("{:?}", numbers.as_ptr()); // 0x7ffac4401ce0
}
The pointer address of the vector variable was changed when modifying it. What is the difference between the memory of string and vector type?
Vec<T> and String may maintain extra space to avoid allocating on every push operation. This provides amortized O(1) time for push operations.
It happens to be the case that the vec! macro is guaranteed to create a vector without such extra space, while String::from(&str) does not have such a guarantee.
See https://doc.rust-lang.org/std/vec/struct.Vec.html#capacity-and-reallocation for more details.
A String is like a Vec<T>¹ in that it has both a length and a capacity. If the capacity of the current allocation is big enough to hold the new string, the underlying buffer does not need to be reallocated. The documentation for Vec<T> explains it this way:
The capacity of a vector is the amount of space allocated for any future elements that will be added onto the vector. This is not to be confused with the length of a vector, which specifies the number of actual elements within the vector. If a vector's length exceeds its capacity, its capacity will automatically be increased, but its elements will have to be reallocated.
For example, a vector with capacity 10 and length 0 would be an empty vector with space for 10 more elements. Pushing 10 or fewer elements onto the vector will not change its capacity or cause reallocation to occur.
However, even if the capacity does change, the pointer value is still not guaranteed to move. The system allocator itself may be able to resize the allocation without moving it if there is enough unallocated space adjacent to it. That appears to be what's happening in your code. If you print the capacity along with the pointer, you can observe this behavior:
let mut hello = String::from("hello");
for _ in 0..10 {
println!("({:3}) {:?}", hello.capacity(), hello.as_ptr()); // 0x7fcfa7c01be0
hello.push_str(", world!");
}
( 5) 0x557624d8da40
( 13) 0x557624d8da40
( 26) 0x557624d8dba0
( 52) 0x557624d8dba0
( 52) 0x557624d8dba0
( 52) 0x557624d8dba0
(104) 0x557624d8dba0
(104) 0x557624d8dba0
(104) 0x557624d8dba0
(104) 0x557624d8dba0
In this example, the buffer was resized 4 times, but the contents were only moved once.
¹ Actually, a String is a newtyped Vec<u8>, which explains why they work the same.
My goal is to read some bytes from a TcpStream in order to parse the data in each message and build a struct from it.
loop {
let mut buf: Vec<u8> = Vec::new();
let len = stream.read(&mut buf)?;
if 0 == len {
//Disconnected
}
println!("read() -> {}", len);
}
Like in Python, I thought the stream.read() would block until it received some data.
So I've set up a server that calls the loop you see above for each incoming connection. I've then tried to connect to the server with netcat; netcat connects successfully to the server and blocks on the stream.read(), which is what I want; but as soon as I send some data, read() returns 0.
I've also tried doing something similar with stream.read_to_end() but it only appears to only return when the connection is closed.
How can I read from the TcpStream, message per message, knowing that each message can have a different, unknown, size ?
You're getting caught with your pants down by an underlying technicality of Vec more than by std::io::Read, although they both interact in this particular case.
The definition and documentation of Read states:
If the return value of this method is Ok(n), then it must be guaranteed that 0 <= n <= buf.len(). A nonzero n value indicates that the buffer buf has been filled in with n bytes of data from this source. If n is 0, then it can indicate one of two scenarios:
The important part is bolded.
When you define a new Vec the way you did, it starts with a capacity of zero. This means that the underlying slice (that you will use as a buffer) has a length of zero. As a result, since it must be guaranteed that 0 <= n <= buf.len() and since buf.len() is zero, your read() call immediately returns with 0 bytes read.
To "fix" this, you can either assign a default set of elements to your Vec (Vec::new().resize(1024, 0)), or just use an array from the get-go (let mut buffer:[u8; 1024] = [0; 1024])
I want to limit a map to be maximum X bytes. It seems there is no straightforward way of computing the byte length of a map though.
"encoding/binary" package has a nice Size function, but it only works for slices or "fixed values", not for maps.
I could try to get all key/value pairs from the map, infer their type (if it's a map[string]interface{}) and compute the length - but that would be both cumbersome and probably incorrect (because that would exclude the "internal" Go cost of the map itself - managing pointers to elements etc).
Any suggested way of doing this? Preferably a code example.
This is the definition for a map header:
// A header for a Go map.
type hmap struct {
// Note: the format of the Hmap is encoded in ../../cmd/gc/reflect.c and
// ../reflect/type.go. Don't change this structure without also changing that code!
count int // # live cells == size of map. Must be first (used by len() builtin)
flags uint32
hash0 uint32 // hash seed
B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
oldbuckets unsafe.Pointer // previous bucket array of half the size, non-nil only when growing
nevacuate uintptr // progress counter for evacuation (buckets less than this have been evacuated)
}
Calculating its size is pretty straightforward (unsafe.Sizeof).
This is the definition for each individual bucket the map points to:
// A bucket for a Go map.
type bmap struct {
tophash [bucketCnt]uint8
// Followed by bucketCnt keys and then bucketCnt values.
// NOTE: packing all the keys together and then all the values together makes the
// code a bit more complicated than alternating key/value/key/value/... but it allows
// us to eliminate padding which would be needed for, e.g., map[int64]int8.
// Followed by an overflow pointer.
}
bucketCnt is a constant defined as:
bucketCnt = 1 << bucketCntBits // equals decimal 8
bucketCntBits = 3
The final calculation would be:
unsafe.Sizeof(hmap) + (len(theMap) * 8) + (len(theMap) * 8 * unsafe.Sizeof(x)) + (len(theMap) * 8 * unsafe.Sizeof(y))
Where theMap is your map value, x is a value of the map's key type and y a value of the map's value type.
You'll have to share the hmap structure with your package via assembly, analogously to thunk.s in the runtime.
I want to create a Vec<T> and make some room for it, but I don't know how to do it, and, to my surprise, there is almost nothing in the official documentation about this basic type.
let mut v: Vec<i32> = Vec<i32>(SIZE); // How do I do this ?
for i in 0..SIZE {
v[i] = i;
}
I know I can create an empty Vec<T> and fill it with pushes, but I don't want to do that since I don't always know, when writing a value at index i, if a value was already inserted there yet. I don't want to write, for obvious performance reasons, something like :
if i >= len(v) {
v.push(x);
} else {
v[i] = x;
}
And, of course, I can't use the vec! syntax either.
While vec![elem; count] from the accepted answer is sufficient to create a vector with all elements equal to the same value, there are other convenience functions.
Vec::with_capacity() creates a vector with the given capacity but with zero length. It means that until this capacity is reached, push() calls won't reallocate the vector, making push() essentially free:
fn main() {
let mut v = Vec::with_capacity(10);
for i in 0..10 {
v.push(i);
}
println!("{:?}", v);
}
You can also easily collect() a vector from an iterator. Example:
fn main() {
let v: Vec<_> = (1..10).collect();
println!("{:?}", v);
}
And finally, sometimes your vector contains values of primitive type and is supposed to be used as a buffer (e.g. in network communication). In this case you can use Vec::with_capacity() + set_len() unsafe method:
fn main() {
let mut v = Vec::with_capacity(10);
unsafe { v.set_len(10); }
for i in 0..10 {
v[i] = i;
}
println!("{:?}", v);
}
Note that you have to be extra careful if your vector contains values with destructors or references - it's easy to get a destructor run over a uninitialized piece of memory or to get an invalid reference this way. It will also work right if you only use initialized part of the vector (you have to track it yourself now). To read about all the possible dangers of uninitialized memory, you can read the documentation of mem::uninitialized().
You can use the first syntax of the vec! macro, specifically vec![elem; count]. For example:
vec![1; 10]
will create a Vec<_> containing 10 1s (the type _ will be determined later or default to i32). The elem given to the macro must implement Clone. The count can be a variable, too.
There is the Vec::resize method:
fn resize(&mut self, new_len: usize, value: T)
This code resizes an empty vector to 1024 elements by filling with the value 7:
let mut vec: Vec<i32> = Vec::new();
vec.resize(1024, 7);