My Frama-C plug-in creates some varinfos with makeGlobalVar ~logic:true name type. These varinfos do not exist in the AST (they are placeholders for the results of calls to allocating functions in the target program, created “dynamically” during the analysis). If my plug-in takes care not to keep any strong pointer onto these varinfos, will they have a chance to be garbage-collected? Or are they registered in a data structure with strong pointers? If so, would it be possible to make that data structure weak? OCaml does not have the variety of weak data structure found in the literature for other languages, but there is nothing a periodical explicit pass to clean up empty stubs cannot fix.
Now that I think about it, I may not even have to create a varinfo. But it is a bit late to change my plug-in now. What I use of the varinfo is a name and a representation of a C type. Function makeGlobalVar offers a guarantee of unicity for the name, which is nice, I guess, as long as it does not create a strong pointer to it or to part of it in the process.
Context:
Say that you are writing a C interpreter to execute C programs that call malloc() and free(). If the target program does not have a memory leak (it frees everything it allocates and never holds too much memory), you would like the interpreter to behave the same.
If you don't explicitely register the varinfos into one of the Globals table, Frama-C won't do it for you (and in fact, if you do, you're supposed to add their declaration in the AST and vice-versa), so I guess that you are safe here. The only visible side-effect as far as the kernel is concerned should be the incrementation of the Vid counter. Note however that makeGlobalVar itself does not guarantee the unicity of the vname, but only of the vid field.
Related
The information about Ada.Containers.Functional_Maps in the GNAT documentation is quite—let's say—abstruse.
First, it says this:
…these containers can still be used safely.
In the second paragraph, it seems to me that you cannot free the memory allocated for those objects once the program exits the context where they are created. I am understanding that you could run into a memory leak. Am I right?
They are also memory consuming, as the allocated memory is not reclaimed when the container is no longer referenced.
Read the next two sentences in the doc:
Thus, they should in general be used in ghost code and annotations, so that they can be removed from the final executable. The specification of this unit is compatible with SPARK 2014.
Because the specification of Ada.Containers.Functional_Maps is compatible with SPARK, it may help to examine it in the context of related SPARK Libraries with regard to proof, testing and annotation. In particular,
The functional maps, sets and vectors are unbounded collections of indefinite elements that are neither controlled nor limited. While they are inefficient with regard to memory, they are simple, immutable and useful "to model user defined data structures."
The functional containers can be used in Ghost Code, "parts of the code that are only meant for specification and verification", as suggested here. This related example illustrates a ghost function.
it seems to me that you cannot free the memory allocated for those
objects once the program exits the context where they are created. I
am understanding that you could run into a memory leak. Am I right?
There are some things that you can do in Ada to manage memory, I would be surprised if (for example) the usage of an instance inside a declare-block were not cleaned-up on the block's exit. — This is, in fact, how some surprisingly robust applications can get away without "dynamically-allocated" memory/values (it's actually heap-allocated, but that's pedantic).
This sort of granular control is really nice, as you can constrain things/usages to specific points. Combined with Ada's good facilities for presenting interfaces, this means that changing some structure to another can be less-painful than it otherwise might be.
As an example of the above, I had a nested key-value map (a JSON object) that was being used to pass parameters around; the method for doing this changed and so I had a string of values (with common-rooted keys) coming in and a procedure that took JSON as input. Obviously what was needed was a "keys&values-to-JSON function, so inside the function I used the multiway-tree container where the leafs represented values and the internal-nodes the keys, the second step was to traverse the tree and create the JSON-object as needed - simple recursion and data-structure selection used to address the problem of adapting the textual key-value pairs of these nested parameters to JSON. — And because the usage of multi-way trees was exclusive to this function, I can be confident that the memory used by the intermediate tree-object I used is released on the function's exit.
I get the impression that Rust is intended to be used in highly safe systems. Then I noticed that raw pointers allow arbitrary pointer arithmetic, and they can cause memory safety and security issues.
Basically, a pointer is an object that refers to another object. In most programming languages (I guess) a pointer is actually just a number that refers to a memory address. Rust's raw pointers are really just that - memory addresses. There are other pointer types in Rust (& references, Box, Rc, Arc), for which the compiler can verify that the memory is valid and contains what the program thinks it contains. This is not the case for raw pointers; they can in principle point to any memory location, regardless of the content. Refer to The Book for more details.
Raw pointers can only be dereferenced inside unsafe blocks. These blocks are a way for the programmer to tell the compiler "I know better than you that this is safe and I promise not to do anything stupid".
It is generally best to avoid raw pointers if possible because the compiler cannot reason about their validity, which makes them unsafe in general. Things that make raw pointers unsafe are the potential to...
access a NULL pointer,
access a dangling (freed or invalid) pointer,
free a pointer multiple times,
All these points boil down to dereferencing the pointer. That is, to use the memory pointed to.
However, using raw pointers without dereferencing them is perfectly safe. This has a use case in finding out if two references point to the same object:
fn is_same(a: &i32, b: &i32) -> bool {
a as *const _ == b as *const _
}
Another use case is the foreign function interface (FFI). If you wrap a C function that takes raw pointers as arguments, there is no way around providing them to the function. This is actually unsafe (as is the whole FFI business), because the function is likely to dereference the pointer. This means you are responsible for making sure the pointer is valid, stays valid, and is not freed multiple times.
Finally, raw pointers are used for optimization. For example, the slice iterator uses raw pointers as internal state. This is faster than indices because they avoid range checks during iteration. However, it is also unsafe as far as the compiler is concerned. The library author needs to pay extra attention, so using raw pointers for optimization always comes at the risk of introducing memory bugs that you normally do not have in rust.
In summary, the three main uses of raw pointers are:
"just numbers" - you never access the memory they point to.
FFI - you pass them outside Rust.
memory-mapped I/O - to trigger I/O actions you need to access hardware registers at fixed addresses.
performance - they can be faster than other options, but the compiler won't enforce safety.
As to when raw pointers should be used, the first three points are straight-forward: You will know when they apply because you have to. The last point is more subtle. As with all optimizations, only use them when the benefit outweighs the effort and risk of using them.
A counter example when not to use raw pointers is whenever the other pointer types (& references, Box, Rc, Arc) do the job.
I basically have two C functions to be used from R, one of which is making some blob and the second which needs to use it. While the user is not supposed to look inside it, I thought it would be reasonable not to do any serialization/conversion to R types and just dump it to an RAWSXP.
Are there any non-obvious disadvantages of this (i.e. except of killing user's console when printing it)?
EDIT: Ok, let's say for instance that I have an array of double/int64/(4 x int16) unions which is a result of some algorithm; I want it to be have a normal R copy semantics to behave naturally from an user's point of view (thus external pointer is rather not an option) but I'm not too eager to serialize it to R objects since it would not be straightforward and would probably end in a significant memory overhead.
If the blob is meant to persist within a single R session then it would be more natural to create, at the C level, an external pointer, and to return that to the user. This is outlined in Writing R Extensions, section 5.13.
One limitation of this approach is that the external pointer does not serialize, so is not saved to disk or, e.g., returned from a parallel job. This is often appropriate when the blob is a reference to a data structure that only makes sense in the context in which it was created (e.g., a file handle) but less so if it is a static data structure. In that case storing the data as a RAWSXP can be appropriate, typically as a slot or element of an S3 or S4 class with print / show methods to hide the gory details from the user. Perhaps the downside is that the RAWSXP is allocated and managed by R, e.g., subject to garbage collection, whereas the content of an external pointer would likely be allocated more directly via Calloc and Free.
As Martin and Josh pointed out, external pointers may be preferable.
Your approach sounds related to what e.g. the bigmemory does: it allocates a chunk of memory putside of R and controls it, thereby circumventing R's memory management and constraints. It doesn't matter for your purposes that bigmemory uses this to pass the memory back to R as a custom data type -- the external pointer makes that possible. Other packages using external pointers are
RODBC for a database connection object, and my RcppDE package which does what DEoptim does but in C++ and thereby allows to user-provided compiled functions in for the optimization, leveraging the Rcpp wrapper to external pointers: the Rcpp::XPtr class.
And as Marting rightly says, it is all in the good manual.
I was wondering if it is possible to create a programming language without explicit memory allocation/deallocation (like C, C++ ...) AND without garbage collection (like Java, C#...) by doing a full analysis at the end of each scope?
The obvious problem is that this would take some time at the end of each scope, but I was wondering if it has become feasible with all the processing power and multiple cores in current CPU's. Do such languages exist already?
I also was wondering if a variant of C++ where smart pointers are the only pointers that can be used, would be exactly such a language (or am I missing some problems with that?).
Edit:
Well after some more research apparently it's this: http://en.wikipedia.org/wiki/Reference_counting
I was wondering why this isn't more popular. The disadvantages listed there don't seem quite serious, the overhead should be that large according to me. A (non-interpreted, properly written from the ground up) language with C family syntax with reference counting seems like a good idea to me.
The biggest problem with reference counting is that it is not a complete solution and is not capable of collecting a cyclic structure. The overhead is incurred every time you set a reference; for many kinds of problems this adds up quickly and can be worse than just waiting for a GC later. (Modern GC is quite advanced and awesome - don't count it down like that!!!)
What you are talking about is nothing special, and it shows up all the time. The C or C++ variant you are looking for is just plain regular C or C++.
For example write your program normally, but constrain yourself not to use any dynamic memory allocation (no new, delete, malloc, or free, or any of their friends, and make sure your libraries do the same), then you have that kind of system. You figure out in advance how much memory you need for everything you could do, and declare that memory statically (either function level static variables, or global variables). The compiler takes care of all the accounting the normal way, nothing special happens at the end of each scope, and no extra computation is necessary.
You can even configure your runtime environment to have a statically allocated stack space (this one isn't really under the compiler's control, more linker and operating system environment). Just figure out how deep your function call chain goes, and how much memory it uses (with a profiler or similar tool), an set it in your link options.
Without dynamic memory allocation (and thus no deallocation through either garbage collection or explicit management), you are limited to the memory you declared when you wrote the program. But that's ok, many programs don't need dynamic memory, and are already written that way. The real need for this shows up in embedded and real-time systems when you absolutely, positively need to know exactly how long an operation will take, how much memory (and other resources) it will use, and that the running time and the use of those resources can't ever change.
The great thing about C and C++ is that the language requires so little from the environment, and gives you the tools to do so much, that smart pointers or statically allocated memory, or even some special scheme that you dream up can be implemented. Requiring the use them, and the constraints you put on yourself just becomes a policy decision. You can enforce that policy with code auditing (use scripts to scan the source or object files and don't permit linking to the dynamic memory libraries)
I recently read a discussion regarding whether managed languages are slower (or faster) than native languages (specifically C# vs C++). One person that contributed to the discussion said that the JIT compilers of managed languages would be able to make optimizations regarding references that simply isn't possible in languages that use pointers.
What I'd like to know is what kind of optimizations that are possible on references and not on pointers?
Note that the discussion was about execution speed, not memory usage.
In C++ there are two advantages of references related to optimization aspects:
A reference is constant (refers to the same variable for its whole lifetime)
Because of this it is easier for the compiler to infer which names refer to the same underlying variables - thus creating optimization opportunities. There is no guarantee that the compiler will do better with references, but it might...
A reference is assumed to refer to something (there is no null reference)
A reference that "refers to nothing" (equivalent to the NULL pointer) can be created, but this is not as easy as creating a NULL pointer. Because of this the check of the reference for NULL can be omitted.
However, none of these advantages carry over directly to managed languages, so I don't see the relevance of that in the context of your discussion topic.
There are some benefits of JIT compilation mentioned in Wikipedia:
JIT code generally offers far better performance than interpreters. In addition, it can in some or many cases offer better performance than static compilation, as many optimizations are only feasible at run-time:
The compilation can be optimized to the targeted CPU and the operating system model where the application runs. For example JIT can choose SSE2 CPU instructions when it detects that the CPU supports them. With a static compiler one must write two versions of the code, possibly using inline assembly.
The system is able to collect statistics about how the program is actually running in the environment it is in, and it can rearrange and recompile for optimum performance. However, some static compilers can also take profile information as input.
The system can do global code optimizations (e.g. inlining of library functions) without losing the advantages of dynamic linking and without the overheads inherent to static compilers and linkers. Specifically, when doing global inline substitutions, a static compiler must insert run-time checks and ensure that a virtual call would occur if the actual class of the object overrides the inlined method.
Although this is possible with statically compiled garbage collected languages, a bytecode system can more easily rearrange memory for better cache utilization.
I can't think of something related directly to the use of references instead of pointers.
In general speak, references make it possible to refer to the same object from different places.
A 'Pointer' is the name of a mechanism to implement references. C++, Pascal, C... have pointers, C++ offers another mechanism (with slightly other use cases) called 'Reference', but essentially these are all implementations of the general referencing concept.
So there is no reason why references are by definition faster/slower than pointers.
The real difference is in using a JIT or a classic 'up front' compiler: the JIT can data take into account that aren't available for the up front compiler. It has nothing to do with the implementation of the concept 'reference'.
Other answers are right.
I would only add that any optimization won't make a hoot of difference unless it is in code where the program counter actually spends much time, like in tight loops that don't contain function calls (such as comparing strings).
An object reference in a managed framework is very different from a passed reference in C++. To understand what makes them special, imagine how the following scenario would be handled, at the machine level, without garbage-collected object references: Method "Foo" returns a string, which is stored into various collections and passed to different pieces of code. Once nothing needs the string any more, it should be possible to reclaim all memory used in storing it, but it's unclear what piece of code will be the last one to use the string.
In a non-GC system, every collection either needs to have its own copy of the string, or else needs to hold something containing a pointer to a shared object which holds the characters in the string. In the latter situation, the shared object needs to somehow know when the last pointer to it gets eliminated. There are a variety of ways this can be handled, but an essential common aspect of all of them is that shared objects need to be notified when pointers to them are copied or destroyed. Such notification requires work.
In a GC system by contrast, programs are decorated with metadata to say which registers or parts of a stack frame will be used at any given time to hold rooted object references. When a garbage collection cycle occurs, the garbage collector will have to parse this data, identify and preserve all live objects, and nuke everything else. At all other times, however, the processor can copy, replace, shuffle, or destroy references in any pattern or sequence it likes, without having to notify any of the objects involved. Note that when using pointer-use notifications in a multi-processor system, if different threads might copy or destroy references to the same object, synchronization code will be required to make the necessary notification thread-safe. By contrast, in a GC system, each processor may change reference variables at any time without having to synchronize its actions with any other processor.