The Racket FFI's documentation has types for _ptr, _cpointer, and _pointer.1
However, the documentation (as of writing this question) does not seem to compare the three different types. Obviously the first two are functions that produce ctype?s, where as the last one is a ctype? itself. But when would I use one type over the other?
1It also has as other types such as _box, _list, _gcpointer, and _cpointer/null. These are all variants of those three functions.
_ptr is a macro that is used to create types that are suitable for function types in which you need to pass data via a pointer passed as an argument (a very common idiom in C).
_pointer is a generic pointer ctype that can be used pretty much wherever a pointer is expected or returned. On the Racket side, it becomes an opaque value that you can't manipulate very easily (you can use ptr-ref if you need it). Note the docs have some caveats about interactions with GC when using this.
_cpointer constructs safer variants of _pointer that use tags to ensure that you don't mix up pointers of different types. It's generally more convenient to use define-cpointer-type instead of manually constructing these. In other words, these help you build abstractions represented by Racket's C pointers. You can do it manually with cpointer-push-tag! and _pointer but that's less convenient.
There's also a blog post I wrote that goes into more detail about some of these pointer issues: http://prl.ccs.neu.edu/blog/2016/06/27/tutorial-using-racket-s-ffi/
Related
I have seen many places the mention that Julia is "Composable". I know that the word itself means:
Composability is a system design principle that deals with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements.
But I am curious what the specific components of Julia are that make it composable. Is it the ability to override base functions with my own implementation?
I guess I'll hazard an answer, though my understanding may be no more complete than yours!
As far as I understand it (in no small part from Stefan's "Unreasonable Effectiveness of Multiple Dispatch" JuliaCon talk as linked by Oscar in the comments), I would say that it is in part:
As you say, the ability override base functions with your own implementation [and, critically, then have it "just work" (be dispatched to) whenever appropriate thanks to multiple dispatch] ...since this means if you make a custom type and define all the fundamental / primitive operations on that type (as in https://docs.julialang.org/en/v1/manual/interfaces/ -- say +-*/ et al. for numeric types, or getindex, setindex! et al. for an array-like type, etc.), then any more complex program built on those fundamentals will also "just work" with your new custom type. And that in turn means your custom type will also work (AKA compose) with other people's packages without any need for (e.g.) explicit compatibility shims as long as people haven't over-constrained their function argument types (which is, incidentally, why over-constraining function argument types is a Julia antipattern )
Following on 1), the fact that so many Base methods are also just plain Julia, so will also work with your new custom type as long as the proper fundamental operations are defined
The fact that Julia's base types and methods are generally performant and convenient enough that in many cases there's no need to do anything custom, so you can just put together blocks that all operate on, e.g., plain Julia arrays or tuples or etc.This last point is perhaps most notable in contrast to a language like Python where, for example, every sufficiently large subset of the ecosystem (numpy, tensorflow, etc.) has their own reimplementation of (e.g.) arrays, which for better performance are all ultimately implemented in some other language entirely (C++, for numpy and TF) and thus probably do not compose with each other.
Coming from Wolfram Mathematica, I like the idea that whenever I pass a variable to a function I am effectively creating a copy of that variable. On the other hand, I am learning that in Julia there are the notions of mutable and immutable types, with the former passed by reference and the latter passed by value. Can somebody explain me the advantage of such a distinction? why arrays are passed by reference? Naively I see this as a bad aspect, since it creates side effects and ruins the possibility to write purely functional code. Where I am wrong in my reasoning? is there a way to make immutable an array, such that when it is passed to a function it is effectively passed by value?
here an example of code
#x is an in INT and so is immutable: it is passed by value
x = 10
function change_value(x)
x = 17
end
change_value(x)
println(x)
#arrays are mutable: they are passed by reference
arr = [1, 2, 3]
function change_array!(A)
A[1] = 20
end
change_array!(arr)
println(arr)
which indeed modifies the array arr
There is a fair bit to respond to here.
First, Julia does not pass-by-reference or pass-by-value. Rather it employs a paradigm known as pass-by-sharing. Quoting the docs:
Function arguments themselves act as new variable bindings (new
locations that can refer to values), but the values they refer to are
identical to the passed values.
Second, you appear to be asking why Julia does not copy arrays when passing them into functions. This is a simple one to answer: Performance. Julia is a performance oriented language. Making a copy every time you pass an array into a function is bad for performance. Every copy operation takes time.
This has some interesting side-effects. For example, you'll notice that a lot of the mature Julia packages (as well as the Base code) consists of many short functions. This code structure is a direct consequence of near-zero overhead to function calls. Languages like Mathematica and MatLab on the other hand tend towards long functions. I have no desire to start a flame war here, so I'll merely state that personally I prefer the Julia style of many short functions.
Third, you are wondering about the potential negative implications of pass-by-sharing. In theory you are correct that this can result in problems when users are unsure whether a function will modify its inputs. There were long discussions about this in the early days of the language, and based on your question, you appear to have worked out that the convention is that functions that modify their arguments have a trailing ! in the function name. Interestingly, this standard is not compulsory so yes, it is in theory possible to end up with a wild-west type scenario where users live in a constant state of uncertainty. In practice this has never been a problem (to my knowledge). The convention of using ! is enforced in Base Julia, and in fact I have never encountered a package that does not adhere to this convention. In summary, yes, it is possible to run into issues when pass-by-sharing, but in practice it has never been a problem, and the performance benefits far outweigh the cost.
Fourth (and finally), you ask whether there is a way to make an array immutable. First things first, I would strongly recommend against hacks to attempt to make native arrays immutable. For example, you could attempt to disable the setindex! function for arrays... but please don't do this. It will break so many things.
As was mentioned in the comments on the question, you could use StaticArrays. However, as Simeon notes in the comments on this answer, there are performance penalties for using static arrays for really big datasets. More than 100 elements and you can run into compilation issues. The main benefit of static arrays really is the optimizations that can be implemented for smaller static arrays.
Another package-based options suggested by phipsgabler in the comments below is FunctionalCollections. This appears to do what you want, although it looks to be only sporadically maintained. Of course, that isn't always a bad thing.
A simpler approach is just to copy arrays in your own code whenever you want to implement pass-by-value. For example:
f!(copy(x))
Just be sure you understand the difference between copy and deepcopy, and when you may need to use the latter. If you're only working with arrays of numbers, you'll never need the latter, and in fact using it will probably drastically slow down your code.
If you wanted to do a bit of work then you could also build your own array type in the spirit of static arrays, but without all the bells and whistles that static arrays entails. For example:
struct MyImmutableArray{T,N}
x::Array{T,N}
end
Base.getindex(y::MyImmutableArray, inds...) = getindex(y.x, inds...)
and similarly you could add any other functions you wanted to this type, while excluding functions like setindex!.
I'm in a situation where I'm modifying an existing compiler written in OCaml. I've added locations to the AST of the compiled language, but it has cause a bunch of bugs, because equality checks that previously succeeded now fail when identical ASTs have a different location attached.
In particular, I'm seeing List.mem return false when it should return true, since it relies on equality.
I'm wondering, is there a way for me to specify that, for any two values of my location type, that = should always return true for any two values of this type?
It would be a ton of work to refactor the entire compiler to use a custom equality everywhere, particularly since many polymorphic functions rely on being able to use = on any type.
There's no existing OCaml mechanism to do what you want.
You can use ppx to write OCaml syntax extensions, and (as I understand it) the behavior can depend on types. So there's some chance you could get things working that way. But it wouldn't be as straightforward as what you're asking for. I suspect you would need to explicitly handle = and any standard functions (like List.mem) that use = implicitly. (Note that I have no experience with ppx.)
I found a description of PPX here: http://ocamllabs.io/doc/ppx.html
Many experienced OCaml programmers avoid the use of built-in polymorphic equality because its behavior is often surprising. So it might be worth converting to a custom comparison function after all.
What an annoying problem to have.
If you are desperate and willing to write a little C code you can change the representation of locations to Custom_tag blocks, which allow customising the behaviour of some of the polymorphic operations. It's a nasty solution, and I suggest you look hard for a better approach before resorting to this one.
One possibility is that most of the compiler does not use locations at all. If so, you might be able to get away with replacing every location in the AST with the same dummy location. That should allow equality to behave as if locations were not there at all. This is rather hacky, and may not be possible if passes later in the compiler make any use of location info.
The 'clean' solution is to define a sane equality operation for ASTs (or to derive one using ppx) and to change the code to use that. As you say, this would be a lot more work.
Julia's parametric types really define a family of types containing different layout in memory. I was wondering if this works also for the names and number of fields in a composite type? A simple example would be something like:
type mytype{Float64}
a::Float64
b::Float64
end
type mytype{Int64}
a::Int64
end
This gives me an error for redefining mytype.
Here, I want to have two fields if mytype's type parameter was Float64 and just one if its Int64. (Actually what I want is more complicated, but this is a basic example). One could imagine having abstract types and <:, etc in the above.
I realize this might not be possible in other languages, but to me it seems the compiler should be able to figure this out much the same way functions can be specialized. After all, real (compiled) code will involve concrete types and everything will be known by the compiler. (for truly dynamical types, perhaps an additional layer of encapsulation would be required in this case?)
Perhaps there is a different/better way of achieving similar results?
You could define the two types separately (mytypeF & mytypeI) and define a new type mytype as the union of the two. Then functions which really could statically determine which type they'd received would be specialized as you requested. But I'm not sure if that's sensible or what you're really after.
This is currently not possible, but the feature has been speculatively proposed as "generated types" in issue #8472. Sebastian's answer is a reasonable work around so long as you take care that the grouped mytype constructor is type-stable. For a more complete example, see how ImmutableArrays.jl programmatically defines a group of types around the abstract ImmutableArray locus.
In Python map() works on any data that follows the sequence protocol. It does The Right Thing^TM whether I feed it a string or a list or even a tuple.
Can't I have my cake in OCaml too? Do I really have no other choice but to look at the collection type I'm using and find a corresponding List.map or an Array.map or a Buffer.map or a String.map? Some of these don't even exist! Is what I'm asking for unusual? I must be missing something.
The closest you will get to this is the module Enum in OCaml Batteries Included (formerly of Extlib). Enum defines maps and folds over Enum.t; you just have to use a conversion to/from Enum.t for your datatype. The conversions can be fairly light-weight, because Enum.t is lazy.
What you really want is Haskell-style type classes, like Foldable and Functor (which generalizes "maps"). The Haskell libraries define instances of Foldable and Functor for lists, arrays, and trees. Another relevant technique is the "Scrap Your Boilerplate" approach to generic programming. Since OCaml doesn't support type classes or higher-kinded polymorphism, I don't think you'd be able to express patterns like these in its type system.
There are two main solutions in OCaml:
Jacques Garrigue already implemented a syntactically-light but inefficient approach for many data structures several years ago. You just wrap the collections in objects that provide a map method. Then you can do collection#map to use the map function for any kind of collection. This is more general than your requirements because it allows different kinds of data structures to be substituted at run time. However, this is not very useful in practice so the approach was never widely adopted.
A syntactically-heavier but efficient, robust and static solution is to use functors to parameterize your code over the data structure you are using. This makes it trivial to reuse your code with different data structures. See Markus Mottl's OCaml translations of Okasaki's book "Purely Functional Data Structures" for some great examples.
If you aren't looking for that kind of power and just want brevity then, of course, you can just create a module alias with a shorter name (e.g. module S = String).
The problem is that each container has a different representation and requires different code for map/reduce to iterate over it. This is why there are separate functions. Most languages provide some sort of general interface for containers (such as the sequence protocol you mentioned) so functions like map/reduce can be implemented abstractly, but this is not done for the types you mentioned.
As long as you define a type t and val compare (: t->t->int) in your module, Map.Make will give you the map you want.