When we call a function, we will make use of Stack Pointer and Framer Pointer.
I know the function and initial position of SP, but how about FP?
The is dependent on architecture, and should be documented in the ABI manual.
Typically, functions do not rely on the frame pointer being properly initialized before the prologue, but rather just store the value on the stack and subsequently set the frame pointer to the current stack pointer (so by following the frame pointer, you find the frame pointer for the previous function at a fixed offset).
Keep in mind that the frame pointer is optional most of the time (you need it if you do not have debugging information and want to follow the call stack, or if the current function used alloca, otherwise it is just a register going to waste.
Related
I'm relatively new to Rust. I was working on some lock-free algorithms, and started playing around with manually managing memory, something similar to C++ new/delete. I noticed a couple different ways that do this throughout the standard library components, but I want to really understand the differences and use cases of each. Here's what it seems like to me:
ManuallyDrop<Box<T>> will prevent Box's destructor from running. I can save a raw pointer to the ManuallyDrop element, and have the actual element go out of scope (what would normally be dropped in Rust) without being dropped. I can later call ManuallyDrop::drop(&mut *ptr) to drop this value manually.
I can also dereference the ManuallyDrop<Box<T>> element, save a raw pointer to just the Box<T>, and later call std::ptr::drop_in_place(box_ptr). This is supposed to destroy the Boxitself and drop the heap-allocated T.
Looking at the ManuallyDrop::drop implementation, it looks those are literally doing the exact same thing. Since ManuallyDrop is zero cost and just stores a value in it's struct, is there any difference in the above two approaches?
I can also call std::alloc::Global.dealloc(...), which looks like it will deallocate the memory block without calling drop. So if I call this on a pointer to Box<T>, it'll deallocate the heap pointer, but won't call drop, so T will still be lying around on the heap. I could call it on a pointer to T itself, which will remove T.
From exploring the standard library, it looks like Global.dealloc gets called in the raw_vec implementation to actually remove the heap-allocated array that Vec points to. This makes sense, since it's literally trying to remove a block of memory.
Rc has a drop implementation that looks roughly like this:
// destroy the contained object
ptr::drop_in_place(self.ptr.as_mut());
// remove the implicit "strong weak" pointer now that we've
// destroyed the contents.
self.dec_weak();
if self.weak() == 0 {
Global.dealloc(self.ptr.cast(), Layout::for_value(self.ptr.as_ref()));
}
I don't really understand why it needs both the dealloc and the drop_in_place. What does the dealloc add that the drop_in_place doesn't do?
Also, if I just save a raw pointer to a heap-allocated value by doing something like Box::new(5).into_raw(), does my pointer now control that memory allocation. As in, will it remain alive until I explicitly call ptr::drop_in_place()?
Finally, when I was playing with all this, I ran into a strange issue. After running ManuallyDrop::drop or ptr::drop_in_place on my raw pointer, I then tried running println! on the pointer's dereferenced value. Sometimes I get a scary heap error and my test fails, which is what I would expect. Other times, it just prints the same value, as if no drops happened. I also tried running ManuallyDrop::drop multiple times on the exact same value, and same thing. Sometimes a heap error, sometimes totally fine, and the same value prints out.
What is happening here?
If you come from C++, you can think of drop_in_place as calling the destructor manually, and dealloc as calling old C free.
They serve different purposes:
drop_in_place just calls Drop::drop, that releases the resources held by your type.
dealloc frees the memory pointed to by a pointer, previously allocated with alloc.
You seem to think that drop_in_place also frees the memory, but that is not the case. I think your confusion arises because Box<T> contains a dynamically allocated object, so its Box::drop implementation does release the memory used by that object, after calling its drop_in_place, of course.
That is what you see in the Rc implementation, first it calls the drop_in_place (destructor) of the inner object, then it releases the memory.
About what happens if you call drop_in_place several times in a row... well, the function is unsafe for a reason: you most likely get Uundefined Behavior. From the docs:
...if T is not Copy, using the pointed-to value after calling drop_in_place can cause undefined behavior.
Note the can cause. I think it is perfectly possible to write a type that allows calling drop several times, but it doesn't sound like such a good idea.
one thing I was recently thinking about was how a computer finds his variables. When we run a program, the program will create multiple layers in the stack, one layer for every new scope it opens and put either the variable value or a pointer in case of storage in heap in this scope. When the scope is done, it and all its variables will be destroyed. But how does a computer know where its variables are? And which ones to use if the same variables occur to be present more often.
How I imagine it, the computer searches the scope it is in like an array and if it doesn't find the variable it follows the stack downwards like a linked list and searches the next scope like an array.
That leads to the assumption that a global variable is the slowest to use since it has to traverse all the way back to the last scope. So it has a computational time from a * n (a = average amount of variables per scope, n = amount of scopes). If I now assume that my code is recursive and within the recursive function calls on a global variable (let's say I have defined the variable const PI = 3.1416 and I use it in every recursion), then it would traverse it backwards again for every single call and if my recursive function takes 1000 recursion, then it does that 1000 times.
But on the other hand, while learning about recursion, I have never heard that referring to variables that are not found inside the recursive scope is to be avoided if possible. Therefore I wonder if I am right with my thoughts. Can someone please shed some light on the issue.
You got it the other way around: scopes, frames, heaps don't make variables, variables make scopes, frames, heaps.
Both are a bit of a stretch actually but my point is to avoid focusing on the lifetime of a variable (that's what terms like heap and stack really mean) and instead take a look under the hood.
Memory is a form of storage where each cell is assigned a number, the cell is called word and the number is called address.
The set of addresses is called address space, an address space is usually a range of addresses or a union of ranges of addresses.
The compiler assumes the program data will be loaded at a specific address, say X, and that the there is enough memory after X (i.e. X+1, X+2, X+3, ..., all exists) for all the data.
Variables are then laid out sequentially from X onward, it is the job of the compiler to keep the association between the address X+k and the variable instance.
Note that a variable may be instanced more than one time, calling a function twice or recursion are both examples of that.
In the first case, the two instances can share the same address X+k since they are don't overlap in time (by the time the second instance is alive, the first is over).
In the second case, the two instances overlap in time and two addresses must be used.
So we see that it is the lifetime of a variable that affects how the mapping between the variable name and its address (a.k.a. the allocation of the variable) is done.
Two common strategies are:
A stack
We start from an address X+b and allocates new instances at successive addresses X+b+1, X+b+2, etc.
The current address (e.g. X+b+54) is stored somewhere (it is the stack pointer).
When we want to free a variable we set the stack pointer back (e.g. from X+b+54 to X+b+53).
We can see that it's impossible to free a variable that is not the last allocated.
This allows for a very fast allocation/deallocation and naturally fits the need of a function frame that holds the local variables: when a function is invoked the new variables are allocated, when it ends they are removed.
From what we noted above, we see that if f calls g (i.e. f is the parent of g) then the variables of f cannot be deallocated before those of g.
This again naturally fits the semantics of functions.
The heap
This strategy dynamically allocate a variable instance at an address X+o.
The runtime reserves a block of addresses and manages their status (free, occupied), when asked, it can give a free address and mark it occupied.
This is useful to allocate an object whose size depends on the user input, for example.
The heap (static)
Some variables have the lifespan of the program but their size and number is known a compile time.
In this case, the compiler simply assigns each instance a unique address X+i.
They cannot be deallocated, they are loaded in memory in batch along with the program code and stay there until the program is unloaded.
I left behind some details, like the fact that the stack most often than not grows from bigger to lower addresses (so it can be put at the farthest edge of the memory) and that variables occupy more than one address.
Some programming languages, especially interpreted ones, don't associate addresses to variable instances, instead, they keep a map between the variable name (properly qualified) and the variable value, this way the lifespan of a variable can be controlled in many particular ways (see Closure in Javascript).
Global variables are allocated in the static heap, only one instance is present (only one address).
Each recursive function that uses it always references directly to the sole instance because the unique address is known at compile time.
Local variables in a function are allocated in the stack and each invocation of a function (recursive or not) uses a new set of instances (the addresses don't need to be the same each time, but they could).
Simply put, there is no lookup, variables are allocated so that the code can access them once compiler (either relatively, in the stack, or absolutely, in the heap).
Sorry if my question seems stupid. My background is in PHP, Ruby, Python, Lua and similar languages, and I have no understanding of pointers in real-life scenarios.
From what I've read on the Internet and what I've got as responses in a question I asked (When is a pointer idiomatic?), I have understood that:
Pointers should be used when copying large data. Instead of getting the whole object hierarchy, receive its address and access it.
Pointers have to be used when you have a function on a struct that modifies it.
So, pointers seem like a great thing: I should just always get them as function arguments because they are so lightweight, and it's okay if I somehow end up not needing to modify anything on the struct.
However, looking at that statement intuitively, I can feel that it sounds very creepy, and yet I don't know why.
So, as someone who is designing a struct and its related functions, or just functions, when should I receive a pointer? When should I receive a value, and why?
In other words, when should my NewAuthor method return &Author{ ... }, and when should it return Author{ ... }? When should my function get a pointer to an author as an argument, and when should it just get the value (a copy) of type Author?
There's tradeoffs for both pointers and values.
Generally speaking, pointers will point to some other region of memory in the system. Be it the stack of the function that wants to pass a pointer to a local variable or some place on the heap.
func A() {
i := 25
B(&i) // A sets up stack frame to call B,
// it copies the address of i so B can look it up later.
// At this point, i is equal to 30
}
func B(i *int){
// Here, i points to A's stack frame.
// For this to execute, I look at my variable "i",
// see the memory address it points to, then look at that to get the value of 25.
// That address may be on another page of memory,
// causing me to have to look it up from main memory (which is slow).
println(10 + (*i))
// Since I have the address to A's local variable, I can modify it.
*i = 30
}
Pointers require me to de-reference them constantly whenever I was to see the data it points to. Sometimes you don't care. Other times it matters a lot. It really depends on the application.
If that pointer has to be de-referenced a lot (ie: you pass in a number to use in a bunch of different calcs), then you keep paying the cost.
Compared to using values:
func A() {
i := 25
B(i) // A sets up the stack frame to call B, copying in the value 25
// i is still 25, because A gave B a copy of the value, and not the address.
}
func B(i int){
// Here, i is simply on the stack. I don't have to do anything to use it.
println(10 + i)
// Since i here is a value on B's stack, modifications are not visible outside B's scpe
i = 30
}
Since there's nothing to dereference, it's basically free to use the local variable.
The down side of passing values happens if those values are large because copying data to the stack isn't free.
For an int it's a wash because pointers are "int" sized. For a struct, or an array, you are copying all the data.
Also, large objects on the stack can make the stack extra big. Go handles this well with stack re-allocation, but in high performance scenarios, it may be too much of an impact to performance.
There's a data safety aspect as well (can't modify something I pass by value), but I don't feel that is usually an issue in most code bases.
Basically, if your problem was already solvable by ruby, python or other language without value types, then these performance nuances don't super-matter.
In general, passing structs as pointers will usually do "the right thing" while learning the language.
For all other types, or things that you want to keep as read-only, pass values.
There are exceptions to that rule, but it's best that you learn those as needs arise rather than try to redefine your world all at once. If that makes sense.
Simply you can use pointers anywhere you want, sometimes you don't want to change your data. It may stand for abstract data, and you don't want to explicitly copy the data. Just pass by value and let compiler do its job.
Go doesn't allow taking the address of a map member:
// if I do this:
p := &mm["abc"]
// Syntax Error - cannot take the address of mm["abc"]
The rationale is that if Go allows taking this address, when the map backstore grows or shinks, the address can become invalid, confusing the user.
But Go slice gets relocated when it outgrows its capacity, yet, Go allows us to take the address of a slice element:
a := make([]Test, 5)
a[0] = Test{1, "dsfds"}
a[1] = Test{2, "sdfd"}
a[2] = Test{3, "dsf"}
addr1 := reflect.ValueOf(&a[2]).Pointer()
fmt.Println("Address of a[2]: ", addr1)
a = append(a, Test{4, "ssdf"})
addrx := reflect.ValueOf(&a[2]).Pointer()
fmt.Println("Address of a[2] After Append:", addrx)
// Note after append, the first address is invalid
Address of a[2]: 833358258224
Address of a[2] After Append: 833358266416
Why is Go designed like this? What is special about taking address of slice element?
There is a major difference between slices and maps: Slices are backed by a backing array and maps are not.
If a map grows or shrinks a potential pointer to a map element may become a dangling pointer pointing into nowhere (uninitialised memory). The problem here is not "confusion of the user" but that it would break a major design element of Go: No dangling pointers.
If a slice runs out of capacity a new, larger backing array is created and the old backing array is copied into the new; and the old backing array remains existing. Thus any pointers obtained from the "ungrown" slice pointing into the old backing array are still valid pointers to valid memory.
If you have a slice still pointing to the old backing array (e.g. because you made a copy of the slice before growing the slice beyond its capacity) you still access the old backing array. This has less to do with pointers of slice elements, but slices being views into arrays and the arrays being copied during slice growth.
Note that there is no "reducing the backing array of a slice" during slice shrinkage.
A fundamental difference between map and slice is that a map is a dynamic data structure that moves the values that it contains as it grows. The specific implementation of Go map may even grow incrementally, a little bit during insert and delete operations until all values are moved to a bigger memory structure. So you may delete a value and suddenly another value may move. A slice on the other hand is just an interface/pointer to a subarray. A slice never grows. The append function may copy a slice into another slice with more capacity, but it leaves the old slice intact and is also a function instead of just an indexing operator.
In the words of the map implementor himself:
https://www.youtube.com/watch?v=Tl7mi9QmLns&feature=youtu.be&t=21m45s
"It interferes with this growing procedure, so if I take the address
of some entry in the bucket, and then I keep that entry around for a
long time and in the meantime the map grows, then all of a sudden that
pointer points to an old bucket and not a new bucket and that pointer
is now invalid, so it's hard to provide the ability to take the
address of a value in a map, without constraining how grow works...
C++ grows in a different way, so you can take the address of a bucket"
So, even though &m[x] could have been allowed and would be useful for short-lived operations (do a modification to the value and then not use that pointer again), and in fact the map internally does that, I think the language designers/implementors chose to be on the safe side with map, not allowing &m[x] in order to avoid subtle bugs with programs that might keep the pointer for a long time without realizing then it would point to different data than the programmer thought.
See also Why doesn't Go allow taking the address of map value? for related comments.
I've read a bunch of explanations about the difference between array pointers and map pointers and it all still seems a tad odd.
Consider this: https://go.dev/play/p/uzADxzdq2EP
I can get a pointer to the zeroth array object but after I add another object to the array the original pointer is still there but it no longer points to the zeroth object of the current array. It points to the original value. Sure, it's not pointing to a nil object, it's pointing to the same object, but it's no longer 'correct' for some version of correct.
I'm not sure what my point is here other than it's just...odd.
If we call
factorial(n){
if(n==0)
return 1;
return (n*factorial(n-1));
}
so in stack it will 5,4,3,2,1 after that it will pop 1 and 2 multiply them and push them to the stack. So who is responsible to all that stuff?
And other than 5,4,3,2,1 is any other elements are there in the stack?
Call stack is used in recursion.
From wikipedia: A call stack is a stack data structure that stores information about the active subroutines of a computer program. This kind of stack is also known as an execution stack, control stack, run-time stack, or machine stack, and is often shortened to just "the stack".
Here you can find a nice explanation on what the recursion is and how it works.
Answer borrowed from Java Basics - How call works.
Evaluate arguments left-to-right. If an argument is a simple variable or a literal value, there is no need to evaluate it. When an expression is used, the expression must be evaluated before the call can be made.
Push a new stack frame on the call stack. When a method is called, memory is required to store the following information.
Parameter and local variable storage. The storage that is needed for each of the parameters and local variables is reserved in the stack frame.
Where to continue execution when the called method returns. You don't have to worry about this; it's automatically saved for you.
Other working storage needed by the method may be required. You don't have to do anything about this because it's handled automatically.
Initialize the parameters. When the arguments are evaluated, they are assigned to the local parameters in the called method.
Execute the method. After the stack frame for this method has been initialized, execution starts with the first statement and continues as normal. Execution may call on other methods, which will push and pop their own stack frames on the call stack.
Return from the method. When a return statement is encountered, or the end of a void method is reached, the method returns. For non-void methods, the return value is passed back to the calling method. The stack frame storage for the called method is popped off the call stack. Popping something off the stack is really efficient - a pointer is simply moved to previous stack frame. This means that the current stack frame can be reused by other methods. Execution is continued in the called method immediately after where the call took place.
You can also find some good answers here.