I am experimenting with an API where the callers provide a Vec<Point> which they want me to fill in with data. They can allocate room in the vector by creating it with with_capacity then push out its size by doing push(Point{...}). Then I will fill the underlying buffer with bytes from the disk, switch their endian representation as necessary, and then provide it back as a Vec<Point>.
Here is the function which takes their vector and fills it with data. The problem is that transmute only works when the types are of the same size, a Point is 12 bytes and transmute throws away 11 of those bytes.
fn read_points(&self, offset: u64, points: &mut [point::Point]) {
let mut file = self.handle.borrow_mut();
file.seek(SeekFrom::Start(offset)).unwrap();
// bleep bloorp. danger!
let points_buf : &mut [u8] = unsafe { mem::transmute(points) };
file.read(points_buf).unwrap();
// should take the 12 bytes and do the endian swaps
for mut chunk in points_buf.chunks_mut(point::POINT_SIZE) {
let point = point::buf_to_point(chunk);
let buf : &mut [u8] = &mut chunk;
point::fill_buf(buf, point.timestamp, point.value);
}
}
Can this API be done in Rust or should I should I switch to doing safer but slower copy operations?
The memory representation of &mut [T] is (*mut T, usize) where usize is the number of T elements in the slice, not the number of bytes. So transmuting a slice of say 20 points gives you a slice of 20 bytes.
You have to compute the correct number of bytes:
let n_bytes = points.len() * std::mem::size_of::<Point>();
let points_buf = std::slice::from_raw_parts_mut(points.as_mut_ptr(), n_bytes);
(Then of course deal of all the rest of the unsafety.)
Related
In my code, I have a type that is aligned to 1 byte, and a function that requires a type that is aligned to 8 bytes. The following hypothetical code shows this usage:
fn use_bar(bar: &mut [u64; 25]) {
unimplemented!()
}
fn main() {
let mut foo: [u8; 200] = get_foo();
unsafe {
// Option 1
use_bar(mem::transmute::<&mut [u8; 200], &mut [u64; 25]>::(&mut foo));
// Option 2
use_bar(&mut *(&mut foo as *mut [u8; 200] as *mut [u64; 25]));
}
}
Unfortunately, this doesn't necessarily work. If you ask clippy about the first option, it will tell you that transmuting references is a bad thing to do. Option 2 may work, however, it will then tell you that the alignment requirements for [u64; 25] are more strict (8 byte alignment) than for [u8; 200] (1 byte alignment) so this may cause undefined behaviour.
Since I don't control the type returned by get_foo(), is there any way I can force foo to be 8 byte aligned? (other than wrapping it in a struct that is properly aligned)
Use align_to to get an aligned slice.
To get data aligned in the first place, you can use a wrapper with #[repr(align(x))]:
#[repr(align(8))]
struct Wrapper([u8; 200]);
I want to create a C FFI API for my crate, but it's not clear how safe it is to cast pointers. Pseudocode:
#[no_mangle]
extern "C" fn f(...) -> *mut c_void {
let t: Box<T> = ...;
let p = Box::into_raw(t);
p as *mut c_void
}
This works as expected, but how safe is it? In C or C++, there is special void * pointer and the C++ standard declares that it is safe to cast to it. Potentially, sizeof(void *) may be not equal sizeof(T *), but there is a guarantee that sizeof(void *) >= sizeof(T *).
What about Rust? Is there any guarantee about the std::mem::size_of of a pointer or safe casting between pointers? Or do all pointers have equal size by implementation, equal to usize?
By "universal", I mean that you can convert X * without losing anything. I do not care about type information; I care about different sizes of pointers to different things, like near/far pointers in the 16-bit days.
4.10 says
The result of converting a "pointer to cv T" to a "pointer to cv void" points to the start of the storage location where the object of type T resides,
It is impossible that sizeof(void *) < sizeof(T *), because then it is impossible to have real address of storage location.
No.
Rust's raw pointers (and references) currently come in two flavors:
thin (one native-sized integer in size)
fat (two native-sized integers in size)
use std::mem;
fn main() {
println!("{}", mem::size_of::<*const u8>()); // 8
println!("{}", mem::size_of::<*const [u8]>()); // 16
}
There's no type that allows storing both; even the Big Hammer of mem::transmute won't work:
use std::mem;
unsafe fn example(mut thin: *const u8, mut fat: *const [u8]) {
fat = mem::transmute(thin);
thin = mem::transmute(fat);
}
error[E0512]: cannot transmute between types of different sizes, or dependently-sized types
--> src/main.rs:4:11
|
4 | fat = mem::transmute(thin);
| ^^^^^^^^^^^^^^
|
= note: source type: `*const u8` (64 bits)
= note: target type: `*const [u8]` (128 bits)
error[E0512]: cannot transmute between types of different sizes, or dependently-sized types
--> src/main.rs:5:12
|
5 | thin = mem::transmute(fat);
| ^^^^^^^^^^^^^^
|
= note: source type: `*const [u8]` (128 bits)
= note: target type: `*const u8` (64 bits)
Since the layout of fat pointers is a Rust-specific concept, they should never be accessed via FFI. This means that only thin pointers should be used, all of which have a uniform known size.
For those types, you should use an opaque pointer to provide better type safety. You could also use *const () or *const libc::c_void.
See also:
What's the Rust idiom to define a field pointing to a C opaque pointer?
Why can comparing two seemingly equal pointers with == return false?
How do I pass a closure through raw pointers as an argument to a C function?
In C or C++, there is special void * pointer and the C++ standard declares that it is safe to cast to it.
This isn't always true:
Why can't I cast a function pointer to (void *)?
I was writing unsafe snippet of code to emulate how C would have allocated a memory for an int. The code allocates memory for an isize type, assign 0 to the pointer variable, then infinitely increment the value at the address every second
use std::process;
use std::{thread, time};
use std::alloc::{alloc, Layout};
fn main() {
unsafe {
let layout = Layout::new::<isize>();
let p = alloc(layout);
println!("({}) address pointed to by p: {:p}", process::id(), p);
*(p as *mut isize) = 0;
loop {
thread::sleep(time::Duration::from_millis(1000));
*(p as *mut isize) += 1;
println!("({}) p: {}", process::id(), *(p as *mut isize));
}
}
}
However, for each *(p as *mut size) I could have replaced with just *p without the compiler complaining (i.e. *p = 0;), thus I assumed it was correctly inferred. I need help explaining the difference and in which case the explicit type annotation may be required.
It's not correctly inferred. alloc returns a pointer to u8, which can be set to 0 or incremented, so the compiler doesn't complain. However it will wrap at 255 instead of wrapping at 4G (or whatever if you're on a 64-bit system).
I want to read data from a TCP stream but it results in an empty Vec:
extern crate net2;
use net2::TcpBuilder;
use std::io::Read;
use std::io::Write;
use std::io::BufReader;
let tcp = TcpBuilder::new_v4().unwrap();
let mut stream = tcp.connect("127.0.0.1:3306").unwrap();
let mut buf = Vec::with_capacity(1024);
stream.read(&mut buf);
println!("{:?}", buf); // prints []
When I use stream.read_to_end the buffer is filled but this takes way too long.
In Python I can do something like
import socket
TCP_IP = '127.0.0.1'
TCP_PORT = 3306
BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
#s.send(MESSAGE)
data = s.recv(BUFFER_SIZE)
s.close()
print "received data:", data
How can I achieve this in Rust?
The two methods you tried don't work for different reasons:
read(): "does not provide any guarantees about whether it blocks waiting for data". In general, read() is unreliable from a users perspective and should only be used as a building block for higher level functions, like read_to_end().
But maybe more importantly, you have a bug in your code: you create your vector via with_capacity() which reserves memory internally, but doesn't change the length of the vector. It is still empty! When you now slice it like &buf, you pass an empty slice to read(), thus read() cannot read any actual data. To fix that, the elements of your vector need to be initialized: let mut buf = vec![0; 1024] or something like that.
read_to_end(): calls read() repeatedly until EOF is encountered. This doesn't really make sense in most TCP stream situations.
So what should you use instead? In your Python code you read a specific number of bytes into a buffer. You can do that in Rust, too: read_exact(). It works like this:
const BUFFER_SIZE: usize = 1024;
let mut stream = ...;
let mut buf = [0; BUFFER_SIZE];
stream.read_exact(&mut buf);
println!("{:?}", buf);
You could also use take(). That way you can use read_to_end():
const BUFFER_SIZE: usize = 1024;
let mut stream = ...;
let mut buf = Vec::with_capacity(BUFFER_SIZE);
stream.take(BUFFER_SIZE).read_to_end(&mut buf);
println!("{:?}", buf);
If you want to use the stream multiple times, you probably want to use by_ref() before calling take().
The two code snippets are not equivalent though! Please read the documentation for more details.
I want to create a Vec<T> and make some room for it, but I don't know how to do it, and, to my surprise, there is almost nothing in the official documentation about this basic type.
let mut v: Vec<i32> = Vec<i32>(SIZE); // How do I do this ?
for i in 0..SIZE {
v[i] = i;
}
I know I can create an empty Vec<T> and fill it with pushes, but I don't want to do that since I don't always know, when writing a value at index i, if a value was already inserted there yet. I don't want to write, for obvious performance reasons, something like :
if i >= len(v) {
v.push(x);
} else {
v[i] = x;
}
And, of course, I can't use the vec! syntax either.
While vec![elem; count] from the accepted answer is sufficient to create a vector with all elements equal to the same value, there are other convenience functions.
Vec::with_capacity() creates a vector with the given capacity but with zero length. It means that until this capacity is reached, push() calls won't reallocate the vector, making push() essentially free:
fn main() {
let mut v = Vec::with_capacity(10);
for i in 0..10 {
v.push(i);
}
println!("{:?}", v);
}
You can also easily collect() a vector from an iterator. Example:
fn main() {
let v: Vec<_> = (1..10).collect();
println!("{:?}", v);
}
And finally, sometimes your vector contains values of primitive type and is supposed to be used as a buffer (e.g. in network communication). In this case you can use Vec::with_capacity() + set_len() unsafe method:
fn main() {
let mut v = Vec::with_capacity(10);
unsafe { v.set_len(10); }
for i in 0..10 {
v[i] = i;
}
println!("{:?}", v);
}
Note that you have to be extra careful if your vector contains values with destructors or references - it's easy to get a destructor run over a uninitialized piece of memory or to get an invalid reference this way. It will also work right if you only use initialized part of the vector (you have to track it yourself now). To read about all the possible dangers of uninitialized memory, you can read the documentation of mem::uninitialized().
You can use the first syntax of the vec! macro, specifically vec![elem; count]. For example:
vec![1; 10]
will create a Vec<_> containing 10 1s (the type _ will be determined later or default to i32). The elem given to the macro must implement Clone. The count can be a variable, too.
There is the Vec::resize method:
fn resize(&mut self, new_len: usize, value: T)
This code resizes an empty vector to 1024 elements by filling with the value 7:
let mut vec: Vec<i32> = Vec::new();
vec.resize(1024, 7);