In game development, it is common to write "baked" assets that are ready to be used by reading their data into memory and using a relocation table to adjust all pointers by the base offset of the buffer used to cache the asset in memory, then directly dereferencing a pointer into that buffer as a particular type. It is similar to relocatable addresses for executable code. This allows for zero-copy reads at runtime, which is significantly faster than parsing the asset into an in-memory format (e.g. parsing JSON to a struct with serde). For example, Halo: Combat Evolved loads all .map data in retail builds to the virtual address 0x40440000 and all pointers in the data on disk are already offset by that base address.
While this is somewhat straightforward in C and C++, Rust's safety rules present a challenge. Not all bitpatterns are safe for all values, and there are stricter alignment requirements for structures. Given the following constraints,
All structures represented in the buffer are repr(C), all bitpatterns for non-pointer fields in them are considered valid representations, and may not necessarily have alignment requirements (ala the zerocopy crate's FromBytes and Unaligned traits)
Magic fields can be used and checked to ensure the validity of the referenced structures from a higher struct
The pointer to the data is wrapped with std::pin::Pin to prevent the data from being moved
The access of the relocated data does not necessarily have to be safe Rust (e.g. dereferencing unchecked pointers), but the pointers themselves can be verified by relocatable offsets accompanying the asset data
How could I implement offset relocation within the loaded asset's buffer, to ensure that the internal pointers of the root structure and all its values are within the bounds of the allocation?
Related
There are [] operations which are similar to dereferencing in high-level languages. So does that mean that every variable and register name is just a pointer or are pointers a high-level languages idea and have no use in Assembly?
Pointers are a useful concept in asm, but I wouldn't say that symbol names are truly pointers. They're addresses, but they aren't pointers because there's no storage holding them (except metadata, and embedded into the machine code), and thus you can't modify them. A pointer can be incremented.
Variables are a high-level concept that doesn't truly exist in assembly. Asm has labels which you can put at any byte position in any section, including .data, .text, or whatever. Along with directives like dd, you can reserve space for a global variable and attach a symbol to it. (But a variable's value can temporarily be in a register, or for its whole lifetime if it's a local variable. The high-level concept of a variable doesn't have to map to static storage with a label.)
Types like integer vs. pointer also don't really exist in assembly; everything is just bytes that you can load into an integer, XMM, or even x87 FP register. (And you don't really have to think of that as type-punning to integer and back if you use eax to copy the bytes of a float from one memory location to another, you're just loading and storing to copy bytes around.)
But on the other hand, a pointer is a low-enough level concept still be highly relevant in assembly. We have the stack pointer (RSP) which usually holds a valid address, pointing to some memory we're using as stack space. (You can use the RSP register to hold values that aren't valid addresses, though. In that case you're not using it as a pointer. But at any time you could execute a push instruction, or mov eax, [rsp], and cause an exception from the invalid address.)
A pointer is an object that holds the address of another object. (I'm using "object" in C terms here: any byte[s] of storage that you can access, including something like an int. Not objected as in object-oriented programming.) So a pointer is basically a type of integer data, especially in assembly for a flat memory model where there aren't multiple components to it. For a segmented memory model, a seg:off far pointer is a pair of integers.
So any valid address stored anywhere in register or memory can usefully be thought of as a pointer.
But no, a symbol defined by a label is not a pointer. Conceptually, I think it's important to think of it as just a label. A pointer is itself an object (some bytes of storage), and thus can be modified. e.g. increment a pointer. But a symbol is just a way to reference some fixed position.
In C terms, a symbol is like a char symbol[], not a char *symbol = NULL; If you use bare symbol, you get the address. Like mov edi, symbol in NASM syntax. (Or mov edi, OFFSET symbol in GNU .intel_syntax or MASM. See also How to load address of function or label into register for practical considerations like using RIP-relative LEA if 32-bit absolute addresses don't work.)
You can deref any symbol in asm to access the bytes there, whether that's mov eax, [main] to load the first 4 bytes of machine code of that function, or mov eax, [global_arr + rdi*8] to index into an array, or any other x86 addressing mode. (Caveat: 32-bit absolute addresses no longer allowed in x86-64 Linux? for that last example).
But you can't do arr++; that makes no sense. There is no storage anywhere holding that address. It's embedded into the machine code of your program at each site that uses it. It's not a pointer. (Note that C arr[4] = 1 compiles to different asm depending on char *arr; vs. char arr[], but in asm you have to manually load the pointer value from wherever it's stored, and then deref it with some offset.)
If you have a label in asm whose address you want to use in C, but that isn't attached to some bytes of storage, you usually want to declare it as extern const char end_of_data_section[]; or whatever.
So for example you can do size_t data_size = data_end - data_start; and get the size in bytes as a link-time constant, if you arranged for those symbols to be at the end/start of your .data section. With a linker script or with global data_end / data_end: in your NASM source. Probably at the same address as some other symbol, for the start.
Assembly language doesn't have variables, pointers or type checking. It has addresses and registers (and no type checking).
A variable (e.g. in C) is a higher level thing - a way to abstract the location of a piece of data so that you don't have to care if the compiler felt like putting it in a register or in memory. A variable also has a type; which is used to detect some kinds of bugs, and used to allow the compiler to automatically convert data to a different type for you.
A pointer (e.g. in C) is a variable (see above). The main difference between a pointer and a "not pointer" is the type of data it contains - for a pointer the variable typically contains an address, but a pointer is not just an address, it's the address of something with a type. This is important - if it was just an address then the compiler wouldn't know how big it is, couldn't detect some kinds of bugs, and couldn't automatically convert data (e.g. consider what would happen for int foo = *pointer_to_char; if the compiler didn't know what type of data the pointer points to).
Suppose a process is going to send a number of arrays of different sizes but of the same type to another process in a single communication, so that the receiver builds the same arrays in its memory. Prior to the communication the receiver doesn't know the number of arrays and their sizes. So it seems to me that though the task can be done quite easily with MPI_Pack and MPI_Unpack, it cannot be done by creating a new datatype because the receiver doesn't know enough. Can this be regarded as an advantage of MPI_PACK over derived datatypes?
There is some passage in the official document of MPI which may be referring to this:
The pack/unpack routines are provided for compatibility with previous libraries. Also, they provide some functionality that is not otherwise available in MPI. For instance, a message can be received in several parts, where the receive operation done on a later part may depend on the content of a former part.
You are absolutely right. The way I phrase it is that with MPI_Pack I can make "self-documenting messages". First you store an integer that says how many elements are coming up, then you pack those elements. The receiver inspects that first int, then unpacks the elements. The only catch is that the receiver needs to know an upper bound on the number of bytes in the pack buffer, but you can do that with a separate message, or a MPI_Probe.
There is of course the matter that unpacking a packed message is way slower than straight copying out of a buffer.
Another advantage to packing is that it makes heterogeneous data much easier to handle. The MPI_Type_struct is rather a bother.
Can someone translate this into something that makes sense for me:
Converts the pointer to a raw pointer (without the tag).
What is the difference between a pointer and a raw pointer?
The Stack Overflow raw-pointer tag says neither "smart" nor "shared" which again is mystifying.
What are Crossbeam's Shared::as_raw's "tags" all about?
crossbeam_epoch::Shared is a smart pointer. That is, a pointer plus extra stuff. In C++ or Rust, smart pointer is the term used for a pointer wrapper which adds any of the following:
Ownership information
Lifetime information
Packing extra data in unused bits
Copy-on-write behavior
Reference counting
In that context, a raw pointer is just the wrapped pointer, without all the extra stuff.
crossbeam_epoch::Shared fits (among others) in the “Packing extra data in unused bits” category above. Most data in modern computers is aligned, that is, addresses are a multiple of some power of two. This means that all low bits of the addresses are always 0. One can use that fact to store a few extra bits of information in a pointer.
This extra data is called tag by this particular library, however that term isn't as common as raw pointer.
I am playing deep learning with opencl, the output size of the tensor is fixed.
In cuda, I can use zero copy via cudaMallocHost, this can be called in the initialization. And I can read the output of the tensor from the host without explicitly calling cudaMemcpy.
It's very efficient since it's called only one time over the entire execution of my program. I don't need to call cudaMallocHost every time after forwarding.
And when I try to implement zero copy in opencl, in some implementations they call clEnqueueMapBuffer and clEnqueueUnmapMemObject every time after forwarding when you want to read the output of the tensor.
Here is the example (https://github.com/alibaba/MNN/blob/master/source/backend/opencl/core/OpenCLBackend.cpp#L291).
But I find that the overhead of clEnqueueMapBuffer can not be neglected, sometimes the latency is quite large.
Is this really suggested way to do so? Can I call clEnqueueMapBuffer only one time in the lifetime of my program and call clEnqueueUnmapMemObject one time when the end of my program? is there any issue to do so?
If your OpenCL implementation supports Shared Virtual Memory (introduced in 2.0), that feature allows you to do something similar, and much more.
For OpenCL 1.x, unless your OpenCL implementation makes any guarantees above and beyond the standard (which I'd expect it to do via an extension), you must unmap a buffer before a kernel gets write access to it, and likewise, you must not allow a kernel to read from it while it is mapped for writing.
This is explained in the clEnqueueMapBuffer specification:
Reads and writes by a kernel executing on a device to a memory region(s) mapped for writing are undefined.
The behavior of writes by a kernel executing on a device to a mapped region of a memory object is undefined.
In version 1.2, this was expanded, but the gist is the same:
If a memory object is currently mapped for writing, the application must ensure that the memory
object is unmapped before any enqueued kernels or commands that read from or write to this
memory object or any of its associated memory objects (sub-buffer or 1D image buffer objects)
or its parent object (if the memory object is a sub-buffer or 1D image buffer object) begin
execution; otherwise the behavior is undefined.
If a memory object is currently mapped for reading, the application must ensure that the memory
object is unmapped before any enqueued kernels or commands that write to this memory object
or any of its associated memory objects (sub-buffer or 1D image buffer objects) or its parent
object (if the memory object is a sub-buffer or 1D image buffer object) begin execution;
otherwise the behavior is undefined.
If you find that map/unmap has a high overhead, you are probably not hitting a zero-copy code path in your OpenCL implementation, and the driver is actually copying the memory contents. If in doubt, check with your implementation vendor to see how they recommend you implement zero-copy buffers in OpenCL. Zero-copy buffers are not guaranteed by the standard.
I am using PIN to instrument my application binary and generating a list of addresses (more specifically memory reads) made by the application. I have an instrumentation routine, which passes the IARG_MEMORYREAD_SIZE, IARG_MEMORYREAD_EA as arguments. However,I want to infer the type information of the application variable based on memory size being read.
For example,
If PIN observes a memory read of 4 bytes, how can I conclude what type of data is being accessed. Is it int/float ? Similarly, for 8 byte data, how would I know if the data is a double variable or a pointer type variable.
There is no way you can infer the type of the operand just by its size. I even doubt you can do it in a reliable manner with the instruction.