Nim: How to pass an array of varying size to an argument of a foreign function calling into a .dll? - pointers

Bellow is a minimal example when wrapping the OpenAL32.dll. The foreign function alcCreateContext has the argument attrlist which takes a ptr to an array of type ALCint or nil. The issue is the array can be of different lengths depending on the amount of different flags passed in. The array should be organized as [flag, int, flag, int, ...]. How can this be accomplished in a more dynamic way allowing the inclusion of ALC_FREQUENCY for example? The array size is currently hard coded into the procedure and its nasty.
when defined(windows):
{.push cdecl, dynlib: "OpenAL32.dll", importc.}
else:
{.push importc.}
type
ALCint = cint
ALCdevice* = pointer
ALCcontext* = pointer
const
ALC_MONO_SOURCES* = 0x00001010
ALC_STEREO_SOURCES* = 0x00001011
ALC_FREQUENCY* = 0x00001007
proc alcCreateContext*(device: ALCdevice; attrlist: ptr array[0..3, ALCint]): ALCcontext
proc alcOpenDevice*(devicename: cstring): ALCdevice
const attributes = [ALC_MONO_SOURCES.ALCint, 65536.ALCint, ALC_STEREO_SOURCES.ALCint, 65536.ALCint]
discard alcOpenDevice(nil).alcCreateContext(attributes.unsafeAddr)
I experimented with openArray and other containers. Is the solution some sort of cast? This is also the workaround for getting more then 256 sounds out of OpenAL.
Answer from PMunch. Thank You.
The foreign function now wants ptr UncheckedArray[ALCint] and when passing the argument use cast[ptr UncheckedArray[ALCint]](attributes.unsafeAddr)
when defined(windows):
{.push cdecl, dynlib: "OpenAL32.dll", importc.}
else:
{.push importc.}
type
ALCint = cint
ALCdevice* = pointer
ALCcontext* = pointer
const
ALC_MONO_SOURCES* = 0x00001010
ALC_STEREO_SOURCES* = 0x00001011
ALC_FREQUENCY* = 0x00001007
proc alcCreateContext*(device: ALCdevice; attrlist: ptr UncheckedArray[ALCint]): ALCcontext
proc alcOpenDevice*(devicename: cstring): ALCdevice
const attributes = [ALC_MONO_SOURCES.ALCint, 65536.ALCint, ALC_STEREO_SOURCES.ALCint, 65536.ALCint]
discard alcOpenDevice(nil).alcCreateContext(cast[ptr UncheckedArray[ALCint]](attributes.unsafeAddr))

An array in C is simply a pointer to anywhere with one or more contiguous elements of the same type. So to pass a C array to a function you simply need to get such a pointer. Say for example you have a seq of integers then the address of the first element is a C array. Simply do mySeq[0].addr and you're good. Keep the lifecycle of the data in mind though. If Nim doesn't find any more references to the sequence then the memory will get freed. You can also manually get a pointer with create (https://nim-lang.org/docs/system.html#create%2Ctypedesc) and you can cast such pointers to ptr UncheckedArray[T] to be able to use [] on the data in Nim.

Related

Julia immutable struct with mutable list

Here's some broken code:
struct NumberedList
index::Int64
values::Vector{Int64}
NumberedList(i) = new(i, Int64[])
end
function set_values!(list::NumberedList, new_values::Vector{Int64})
list.values = new_values
end
# ---
mylist = NumberedList(1)
set_values!(mylist, [1, 2, 3])
I don't want to do either of these:
I could declare the NumberedList as a mutable struct (making everything mutable)
I could replace set_values! with something like this (copy all the values):
function set_values!(list::NumberedList, new_values::Vector{Int64})
for i in new_values
push!(list.values, i)
end
end
But I'd like to make index immutable, but allow values to be assigned to.
Other notes:
Thread from the Discourse: "Mutable field in immutable type," but this doesn't apply to vectors in this setting.
It depends on what exactly you want to mutate.
If you really want to reassign the field values::Vector{Int64} itself, then you have to use a mutable struct. There's no way around that, because the actual data of the struct changes when you reassign that field.
If you use an immutable struct with a values::Vector{Int64} field, it means that you cannot change which array is contained, but the array itself is mutable and can change its elements (which are not stored in the struct). In this case, you really do have to copy values to it from external arrays, like your example code (though I would point out that your code did not reset the array to an empty one). I personally think this would be cleaner:
function set_values!(list::NumberedList, new_values::Vector{Int64})
empty!(list.values) # reset list.values to Int64[]
append!(list.values, new_values)
end
The thread you linked talks about using Base.Ref. Base.Ref is pretty much THE way to make a field of an immutable struct indirectly act like a mutable field. It works like this: the field cannot change which RefValue{Vector{Int64}} instance is contained, but the instance itself is mutable and can change its reference (again, not stored in the struct) to any Int64 array. You have to use indexing values[] to get to the array, though:
struct NumberedList
index::Int64
values::Ref{Vector{Int64}}
NumberedList(i) = new(i, Int64[])
end
function set_values!(list::NumberedList, new_values::Vector{Int64})
list.values[] = new_values # "reassign" different array to Ref
end
# ---
mylist = NumberedList(1)
set_values!(mylist, [1, 2, 3])

Why I cannot append a value to a structs' slice using a reference?

I Go, I assumed slices were passed by reference, but this seems to work for values
but not for the array itself. For example, If I have this struct:
l := Line{
Points: []Point{
Point{3, 4},
},
}
I can define a variable, which gets passed a reference to the struct's slice
slice := l.Points
And then if I modify it, the original struct referenced by the variable
is going to reflect those modifications.
slice[0].X = 1000
fmt.Printf(
"This value %d is the same as this %d",
slice[0].X,
l.Points[0].X,
)
This differs from the behavior of arrays which, I assume, are passed by value.
So, for example, if I had defined the previous code using an array:
l := Line{
Points: [1]Point{
Point{3, 4},
},
}
arr := l.Points
arr[0].X = 1000
fmt.Println(arr.[0].X != s.Points[0].X) // equals true, original struct is untouched
Then, the l struct wouldn't have been modified.
Now, if I want to modify the slice itself I obviously cannot do this:
slice = append(slice, Point{99, 100})
Since that would only redefine the slice variable, losing the original reference.
I know I can simply do this:
l.Points = append(l.Points, Point{99, 100})
But, in some cases, it is more convenient to have another variable instead of having
to type the whole thing.
I tried this:
*slice = append(*slice, Point{99, 100})
But it doesn't work as I am trying to dereference something that apparently is not a pointer.
I finally tried this:
slice := &l.Points
*slice = append(l.Points, Point{99, 100})
And it works, but I am not sure what is happening. Why is the value of slice not overwritten? How does append works here?
Let's dispense first with a terminology issue. The Go language specification does not use the word reference the way you are using it. Go does however have pointers, and pointers are a form of reference. In addition, slices and maps are kind of special as there's some underlying data—the array underneath a slice, or the storage for a map—that may or may not already exist or be created by declaring or defining a variable whose type is slice of T or map[T1]T2 for some type T or type-pair T1 and T2.1
We can take your usage of the word reference to mean explicit pointer when talking about, e.g.:
func f1(p *int) {
// code ...
}
and the implied pointer when talking about:
func f2(m map[T1]T2) { ... }
func f3(s []T) { ... }
In f1, p really is a pointer: it thus refers to some actual int, or is nil. In f2, m refers to some underlying map, or is nil. In f3, s refers to some underlying array, or is nil.
But if you write:
l := Line{
Points: []Point{
Point{3, 4},
},
}
then you must have written:
type Line struct {
// ... maybe some fields here ...
Points []Point
// ... maybe more fields here ...
}
This Line is a struct type. It is not a slice type; it is not a map type. It contains a slice type but it is not itself one.
You now talk about passing these slices. If you pass l, you're passing the entire struct by value. It's pretty important to distinguish between that, and passing the value of l.Points. The function that receives one of these arguments must declare it with the right type.
For the most part, then, talking about references is just a red herring—a distraction from what's really going on. What we need to know is: What variables are you assigning what values, using what source code?
With all of that out of the way, let's talk about your actual code samples:
l.Points = append(l.Points, Point{99, 100})
This does just what it says:
Pass l.Points to append, which is a built-in as it is somewhat magically type-flexible (vs the rest of Go, where types are pretty rigid). It takes any value of type []T (slice of T, for any valid type T) plus one or more values of type T, and produces a new value of the same type, []T.
Assigns the result to l.Points.
When append does its work, it may:
receive nil (of the given type): in this case, it creates the underlying array, or
receive a non-nil slice: in this case, it writes into the underlying array or discards that array in favor of a new larger-capacity array as needed.2
So in all cases, the underlying array may have, in effect, just been created or replaced. It's therefore important that any other use of the same underlying array be updated appropriately. Assigning the result back to l.Points updates the—presumably one-and-only—slice variable that refers to the underlying array.
We can, however, break these assumptions:
s2 := l.Points
Now l.Points and s2 both refer to the (single) underlying array. Operations that modify that underlying array will, at least potentially, affect both s2 and l.Points.
Your second example is itself OK:
*slice = append(*slice, Point{99, 100})
but you haven't shown how slice itself was declared and/or assigned-to.
Your third example is fine as well:
slice := &l.Points
*slice = append(l.Points, Point{99, 100})
The first of these lines declares-and-initializes slice to point to l.Points. The variable slice therefore has type *[]Point. Its value—the value in slice, that is, rather than that in *slice—is the address of l.Points, which has type []Point.
The value in *slice is the value in l.Points. So you could write:
*slice = append(*slice, Point{99, 100})
here. Since *slice is just another name for l.Points, you can also write:
l.Points = append(*slice, Point{99, 100})
You only need to use *slice if there's some reason that l.Points is not available,3 but you may use *slice if that's more convenient. Reading *slice reads l.Points and updating *slice updates l.Points.
1To see what I mean by may or may not be created here, consider:
var s []int
vs:
var s = []int{42}
The first leaves s == nil while the second creates an underlying array with the capacity to hold the one int value 42, holding the one int value 42, so that s != nil.
2It's not clear to me whether there is a promise never to write on an existing slice-array whose capacity is greater than its current length, but not sufficient to hold the final result. That is, can append first append 10 objects to the existing underlying array, then discover that it needs a bigger array and expand the underlying array? The difference is observable if there are other slice values referring to the existing underlying array.
3Here, a classic example would occur if you have reason to pass l.Points or &l.Points to some existing (pre-written) function:
If you need pass l.Points—the slice value—to some existing function, that existing function cannot change the slice value, but could change the underlying array. That's probably a bad plan, so if it does do this, make sure that this is OK! If it only reads the slice and underlying array, that's a lot safer.
If you need to pass &l.Points—a value that points to the slice value—to some existing function, that existing function can change both the slice, and the underlying array.
If you're writing a new function, it's up to you to write it in whatever manner is most appropriate. If you're only going to read the slice and underlying array, you can take a value of type []Point. If you intend to update the slice in place, you should take a value of type *[]Point—pointer to slice of Point.
Append returns a new slice that may modify the original backing array of the initial slice. The original slice will still point to the original backing array, not the new one (which may or may not be in the same place in memory)
For example (playground)
slice := []int{1,2,3}
fmt.Println(len(slice))
// Output: 3
newSlice := append(slice, 4)
fmt.Println(len(newSlice))
// Output: 4
fmt.Println(len(slice))
// Output: 3
While a slice can be described as a "fat pointer to an array", it is not a pointer and therefore you can't dereference it, which is why you get an error.
By creating a pointer to a slice, and using append as you did above, you are setting the slice the pointer points to to the "new" slice returned by append.
For more information, check out Go Slice Usage And Internals
Your first attempt didn't work because slices are not pointers, they can be considered reference types. Append will modify the underlying array if it has enough capacity, otherwise it returns a new slice.
You can achieve what you want with a combination of your two attempts.
playground
l := Line{
Points: []Point{
Point{3, 4},
},
}
slice := &l.Points
for i := 0; i < 100; i++ {
*slice = append(*slice, Point{99 + i, 100 + i})
}
fmt.Println(l.Points)
I know that this might be sacrilegious, but, for me, it is useful to think of slices
as structs.
type Slice struct {
len int
cap int
Array *[n]T // Pointer to array of type T
}
Since in languages like C, the [] operator is also a dereferencing operator, we can think that every time we are accessing a slice, we are actually dereferencing the underlying array and assigning some value to it. That is:
var s []int
s[0] = 1
Might be thought of as equivalent to (in pseudo-code):
var s Slice
*s.Array[0] = 1
That is why we can say that slices are "pointers". For that reason, it can modify its underlying array like this:
myArray := [3]int{1,1,1}
mySlice := myArray[0:1]
mySlice = append(mySlice, 2, 3) // myArray == mySlice
Modifying mySlice also modifies myArray, since the slice stores a pointer to the array and, on appending, we are dereferencing that pointer.
This behavior, nonetheless, is not always like this. If we exceed the capacity of the original array, a new array is created and the original array is left untouched.
myArray := [3]int{1,1,1}
mySlice := myArray[0:1]
mySlice = append(mySlice, 2, 3, 4, 5) // myArray != mySlice
The confusion arises when we try to treat the slice itself as an actual pointer. Since we can modify an underlying array by appending to it, we are led to believe that in this case:
sliceCopy := mySlice
sliceCopy = append(sliceCopy, 6)
both slices, slice and sliceCopy are the same, but they are not. We have to explicitly pass a reference to the memory address of the slice (using the & operator) in order to modify it. That is:
sliceAddress := &mySlice
*sliceAddress = append(mySlice, 6) // or append(*sliceAddress, 6)
See also
https://forum.golangbridge.org/t/slice-pass-as-value-or-pointer/2866/4
https://blog.golang.org/go-slices-usage-and-internals
https://appliedgo.net/slices/

How does a pointer to a struct or array value in Go work?

Considering the following Go struct:
type Person struct {
Name string
Age int
Country string
}
I have encountered numerious times the following use:
p := &Person{"Adam", 33, "Argentina"}
Yet I can not see the point in pointing to a struct value, and I wonder, how does it differ from:
n := &999 // Error
My questions are:
How is it even possible to point to a value, even if it is a struct or array and not a primitive like a string or int? Strange enough, the following doesn't contribute to my understanding:
fmt.Println(p, &p) // outputs: &{Adam 33 Argentina} 0xc042084018
Why would a programmer want to declare a struct instance by a pointer? What could you achieve doing so?
&Person{} is a language "construct", it's part of the spec: it allocates a new variable of Person type, and provides you the address of that anonymous variable.
Spec: Composite literals:
Taking the address of a composite literal generates a pointer to a unique variable initialized with the literal's value.
Also: Spec: Variables:
Calling the built-in function new or taking the address of a composite literal allocates storage for a variable at run time.
&999 is not allowed by the language spec. The possible operands of the address operators are listed in the Spec: Address operators:
The operand must be addressable, that is, either a variable, pointer indirection, or slice indexing operation; or a field selector of an addressable struct operand; or an array indexing operation of an addressable array. As an exception to the addressability requirement, x may also be a (possibly parenthesized) composite literal.
p := Person{} creates a new variable p whose type will be Person. p := &Person{} creates a new variable whose type will be *Person.
See possible duplicate: How do I do a literal *int64 in Go?
When you print the values with the fmt package, it has certain rules how to print values of different types:
For compound objects, the elements are printed using these rules, recursively, laid out like this:
struct: {field0 field1 ...}
array, slice: [elem0 elem1 ...]
maps: map[key1:value1 key2:value2]
pointer to above: &{}, &[], &map[]
When you use fmt.Println(), the default formatting rules will be applied, which for a value of type *int is the %p verb, which will print the memory address in hexadecimal format, but for a pointer to struct it prints the struct value prepended with an & sign (&{}). You can read more about it in related question: Difference between golang pointers
If you want to print the pointed value, dereference the pointer and pass the pointed value, e.g.:
var p = new(int)
*p = 12
fmt.Println(*p) // Prints 12
As to why to create a pointer to a value (and not a value), see these related questions:
Pointers vs. values in parameters and return values
Why should constructor of Go return address?
Go, X does not implement Y (... method has a pointer receiver)

Does Go (deep) copy keys when inserting into a map?

I have a map with complex keys - for example, 2D arrays:
m := make(map[[2][3]int]int)
When I insert a new key into the map, does Go make a deep copy of the key?
a := [2][3]int{{1, 2, 3}, {4, 5, 6}}
m[a] = 1
In other words, if I change the array a after using it as a map key, does the map still contain the old value of a?
Short answer, it is copied.
By specification, Arrays are value types.
Go's arrays are values. An array variable denotes the entire array; it is not a pointer to the first array element (as would be the case in C). This means that when you assign or pass around an array value you will make a copy of its contents. (To avoid the copy you could pass a pointer to the array, but then that's a pointer to an array, not an array.)
https://blog.golang.org/go-slices-usage-and-internals
See for yourself:
https://play.golang.org/p/fEUYWwN-pm
package main
import (
"fmt"
)
func main() {
m := make(map[[2][3]int]int)
a := [2][3]int{{1, 2, 3}, {4, 5, 6}}
fmt.Printf("Pointer to a: %p\n", &a)
m[a] = 1
for k, _ := range m {
fmt.Printf("Pointer to k: %p\n", &k)
}
}
The pointers do not match.
EDIT: The real reason is when inserting into a map, the key value is copied. Or, you can continue to just remember the rule above: arrays are value types and their reuse denotes a copy. Either works here. :)
Arrays are always passed by value, so, yes in this case Go will make a deep copy of the key.
From the language spec
The comparison operators == and != must be fully defined for operands of the key type; thus the key type must not be a function, map, or slice. If the key type is an interface type, these comparison operators must be defined for the dynamic key values; failure will cause a run-time panic.
The keys are copied into the map. Excluding map and slice as valid keys means that the keys can't change. Note that go doesn't follow pointers if you define a map type with a pointer as a key (eg map[*int]int) it compares the pointers directly.

Pointers in ABAP (like this in java)

Could anyone tell me how to define a pointer in ABAP OO?
In Java I have no problems with it, eg. this.name or this.SomeMethod().
Probably you are asking about so called self reference.
In ABAP it is available by using keyword me.
Example in Java: this.someMethod();
Example in ABAP: me->someMethod( ).
ABAP uses field symbols. They are defined like:
FIELD-SYMBOLS:
, " TYPE any.
TYPE file_table.
If you want to dereference it, you need to do it using another field symbol like this:
ASSIGN str_mfrnr TO <str1>.
This Stores the value of str_mfrnr into the field symbol. If this is formatted as a work area like 'wa_itab-my_column', will now contain this string.
Next, assign the location to another FS:
ASSIGN (<str1>) TO <tmfrnr>.
now points to wa_itab-my_column. If you perform:
<tmfrnr> = some_value.
the location pointed to by now contains the value in some_value.
ABAP pointers are more like C pointers, you have to know whether you are referenceing the value or the location.
Here's a small report I wrote a while ago to wrap my head around it. I think this is how it works:
REPORT zpointers.
* Similar to C:
***************
* int *pointer;
* int value = 1.
* pointer = &value
* int deref = *pointer
*this is the variable
DATA int TYPE i VALUE 10.
*this is the pointer, or the reference to a memory address
DATA pointer_i TYPE REF TO i.
*this is the dereferenced value, or the var that points to the
*value stored in a particular memory address
FIELD-SYMBOLS <int> TYPE i.
*the memory address of variable 'int' is now assigned to
*variable 'pointer_i'.
GET REFERENCE OF int INTO pointer_i.
*you can access the pointer by dereferencing it to a field symbol.
ASSIGN pointer_i->* TO <int>.

Resources