I have a type definition corresponding to a C-struct as follows:
type fakeCStruct
a::Uint8;
b::Uint32;
end
var=fakeCStruct(3,4);
How can I pass a pointer to this type as an input argument (i.e. struct CStruct *) for a c-function in ccall?
The old documentation (v0.3) suggest using &var in the input argument list of ccall. However that statement is deleted in v0.4 documentation (also, &var does not work in v0.4).
Use Ref{fakeCStruct}:
r = Ref(fakeCStruct(3, 4))
ccall((:somefunction, "lib"), Void, (Ref{fakeCStruct},), r)
From the Julia docs in the System Independent table here:
C name: | Julia base type
T* (where T represents an appropriately defined type) | Ref{T}
Allocate memory for the Julia object, then pass it by pointer:
type fakeCStruct
a::UInt8;
b::UInt32;
end
var = fakeCStruct(3,4)
var_p = Ptr{fakeCStruct}(pointer_from_objref(var))
ccall((:somefunc, "lib"), Void, (Ptr{fakeCStruct},), var_p)
(Using Ref{} doesn't work for me... using Julia 0.4.0)
Although I'm definitely no expert on this I iterated to some code which works for me. Hopefully this will help people on their way. In the example foo.c is compiled to a shared library (.dll in my case) and called from foo.jl.
foo.c
struct mystruct {
int a;
int b;
};
typedef struct mystruct mystruct_t;
int recvstruct(mystruct_t* st) {
printf("C %u, %u\n", st->a, st->b);
return 0;
}
mystruct_t* updatestruct(mystruct_t* st) {
// here we need to return the pointer, since Julia seems to
// pass a pointer to a copy of the struct
st->a = 10;
st->b = 11;
return st;
}
foo.jl
struct mystruct
a::Int32
b::Int32
end
function let_c_print_struct()
# Note that the function call automatically converts `mystruct(3, 4)` to `::Ref(..)`.
ccall((:recvstruct, libfile), Int32, (Ref{mystruct},), mystruct(3, 4))
end
function let_c_update_struct()
st = mystruct(5, 6)
ret = ccall((:updatestruct, libfile), Ref{ mystruct }, (Ref{ mystruct }, ), st)
#show st # st = mystruct(5, 6)
#show ret[] # ret[] = mystruct(10, 11)
end
Regarding lifetime of data inside the shared library: I found that objects stored in memory remain available (I would guess that they are outside of the scope of the garbage collector). Four observations support that the data remains available:
Julia keeps the library open unless it is explicitly closed.
The Julia language source code uses unsafe_string (dereferencing pointer) on values returning from a ccall. See search results for unsafe_string(ccall(.
I found that allocating data (even structs containing pointers) in C and passing references to this data around in Julia does work.
I found that sockets remain open.
Related
I have a Julia struct:
struct WindChillCalc
location::Tuple;
w_underground_url::String;
WindChillCalc(location, wug) = new(location, w_underground_url);
end
How do I hard code w_underground_url to contain "someString" upon the constructor of WindChillCalc being called?
Try something like below
struct testStruct
x::Real
y::String
testStruct(x,y) = new(x,"printThis")
end
test = testStruct(1,"")
test2 = testStruct(2,"")
println(test.y)
println(test2.y)
It prints "printThis" for any object.
Just write for example:
struct WindChillCalc{T}
location::T;
w_underground_url::String;
WindChillCalc(location::T) where {T <: NTuple{2, Real}} =
new{T}(location, "some string");
end
and now Julia automatically creates a concrete type for you:
julia> WindChillCalc((1, 2.5))
WindChillCalc{Tuple{Int64,Float64}}((1, 2.5), "some string")
Note that I have restricted the type of the parameter to be a two element tuple where each element is a Real. You could of course use another restriction (or use no restriction).
With this approach your code will be fast as during compile time Julia will know exact types of all fields in your struct.
package main
import "fmt"
func main() {
p := new(map[string]int)
m := make(map[string]int)
m["in m"] = 2
(*p)["in p"] = 1
fmt.Println(m)
fmt.Println(*p)
}
The above code gives an error panic: assignment to entry in nil map. If I print *p before inserting pairs into it, the output is correct. It seems I just can't modify *p?
Both new and make are used to allocate memory in a program, but they work differently. new(T, args) zeros memory and returns the memory address (a value of type *T) but does not initialize that memory. make(T, args) on the other hand initializes a value of type T. A map need to be initialized because, while a map can be empty, there is still the structure of the map itself, which is non-zero and therefore needs to be initialized before use.
From Effective Go:
The built-in function make(T, args) serves a purpose different from new(T). It creates slices, maps, and channels only, and it returns an initialized (not zeroed) value of type T (not *T). The reason for the distinction is that these three types represent, under the covers, references to data structures that must be initialized before use. A slice, for example, is a three-item descriptor containing a pointer to the data (inside an array), the length, and the capacity, and until those items are initialized, the slice is nil. For slices, maps, and channels, make initializes the internal data structure and prepares the value for use.
This is not directly an issue with new keyword. You would get the same behavior, if you did not initialize your map, when declaring it with var keyword, for example:
var a map[string]int
a["z"] = 10
The way to fix this is to initialize the map:
var a map[string]int
a = map[string]int{}
a["z"] = 10
And it works the same way, with new keyword:
p := new(map[string]int)
*p = map[string]int{}
(*p)["in p"] = 1
The reason make(map[string]int) does what you expect is that the map is declared and initialized.
Go Playground
The following is the Deref example from The Rust Programming Language except I've added another assertion.
Why does the assert_eq with the deref also equal 'a'? Why do I need a * once I've manually called deref?
use std::ops::Deref;
struct DerefExample<T> {
value: T,
}
impl<T> Deref for DerefExample<T> {
type Target = T;
fn deref(&self) -> &T {
&self.value
}
}
fn main() {
let x = DerefExample { value: 'a' };
assert_eq!('a', *x.deref()); // this is true
// assert_eq!('a', x.deref()); // this is a compile error
assert_eq!('a', *x); // this is also true
println!("ok");
}
If I uncomment the line, I get this error:
error[E0308]: mismatched types
--> src/main.rs:18:5
|
18 | assert_eq!('a', x.deref());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected char, found &char
|
= note: expected type `char`
found type `&char`
= help: here are some functions which might fulfill your needs:
- .to_ascii_lowercase()
- .to_ascii_uppercase()
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
First, let's spell out the generic types for your specific example: 'a' is char, so we have:
impl Deref for DerefExample<char> {
type Target = char;
fn deref(&self) -> &char {
&self.value
}
}
Notably, the return type of deref is a reference to a char. Thus it shouldn't be surprising that, when you use just x.deref(), the result is a &char rather than a char. Remember, at that point deref is just another normal method — it's just implicitly invoked as part of some language-provided special syntax. *x, for example, will call deref and dereference the result, when applicable. x.char_method() and fn_taking_char(&x) will also call deref some number of times and then do something further with the result.
Why does deref return a reference to begin with, you ask? Isn't that circular? Well, no, it isn't circular: it reduces library-defined smart pointers to the built-in type &T which the compiler already knows how to dereference. By returning a reference instead of a value, you avoid a copy/move (which may not always be possible!) and allow &*x (or &x when it's coerced) to refer to the actual char that DerefExample holds rather than a temporary copy.
See also:
Why is the return type of Deref::deref itself a reference?
I'm trying the build a generic currying function that's look like:
package curry
import (
"fmt"
"reflect"
)
// Function
type fn interface{}
// Function parameter
type pr interface{}
// It return the curried function
func It(f fn, p ...pr) (fn, error) {
// examine the concret type of the function f
if reflect.ValueOf(f).Kind() == reflect.Func {
// Get the slice of input and output parameters type
} else {
return nil, fmt.Errorf("%s", "takes a function as a first parameter")
}
// _, _ = f, p
return nil, nil
}
Is it possible to extract the slice of input and output parameters types as []reflect.Type of the function f ?
You can use reflect.Type.In(int) and reflect.Type.Out(int), there are corresponding methods called NumIn() int and NumOut() int that give you the number of inputs/outputs.
However, keep in mind a few caveats:
To correctly extract the function for an arbitrary signature, you'll need an infinite number of cases. You'll have to switch over every single In and Out in turn to correctly get the type to extract.
You can't dynamically create a function anyway. There's no FuncOf method to go with SliceOf, MapOf, etc. You'll have to hand code the curried versions anyway.
Using reflection to emulate generics is generally considered a Bad Idea™.
If you absolutely have to do something like this, I'd heavily recommend making an interface and having each implementation do the currying itself, rather than trying to hack it "generically" for all cases, which will never work as of Go 1.2.1.
Go 1.5 will add a function that could help here.
(review 1996, commit e1c1fa2 by Dave (okdave))
// FuncOf returns the function type with the given argument and result types.
// For example if k represents int and e represents string,
// FuncOf([]Type{k}, []Type{e}, false) represents func(int) string.
//
// The variadic argument controls whether the function is variadic. FuncOf
// panics if the in[len(in)-1] does not represent a slice and variadic is
// true.
func FuncOf(in, out []Type, variadic bool) Type
The test cases include this intriguing code:
v := MakeFunc(FuncOf([]Type{TypeOf(K(""))}, []Type{TypeOf(V(0))}, false), fn)
outs := v.Call([]Value{ValueOf(K("gopher"))})
I'm having trouble wrapping my head around how pointers, slices, and interfaces interact in Go. This is what I currently have coded up:
type Loader interface {
Load(string, string)
}
type Foo struct {
a, b string
}
type FooList []Foo
func (l FooList) Load(a, b string) {
l = append(l, Foo{a, b})
// l contains 1 Foo here
}
func Load(list Loader) {
list.Load("1", "2")
// list is still nil here
}
Given this setup, I then try to do the following:
var list FooList
Load(list)
fmt.Println(list)
However, list is always nil here. My FooList.Load function does add an element to the l slice, but that's as far as it gets. The list in Load continues to be nil. I think I should be able to just pass the reference to my slice around and have things append to it. I'm obviously missing something on how to get it to work though.
(Code in http://play.golang.org/p/uuRKjtxs9D)
If you intend your method to make changes, you probably want to use a pointer receiver.
// We also define a method Load on a FooList pointer receiver.
func (l *FooList) Load(a, b string) {
*l = append(*l, Foo{a, b})
}
This has a consequence, though, that a FooList value won't itself satisfy the Loader interface.
var list FooList
Load(list) // You should see a compiler error at this point.
A pointer to a FooList value, though, will satisfy the Loader interface.
var list FooList
Load(&list)
Complete code below:
package main
import "fmt"
/////////////////////////////
type Loader interface {
Load(string, string)
}
func Load(list Loader) {
list.Load("1", "2")
}
/////////////////////////////
type Foo struct {
a, b string
}
// We define a FooList to be a slice of Foo.
type FooList []Foo
// We also define a method Load on a FooList pointer receiver.
func (l *FooList) Load(a, b string) {
*l = append(*l, Foo{a, b})
}
// Given that we've defined the method with a pointer receiver, then a plain
// old FooList won't satisfy the Loader interface... but a FooList pointer will.
func main() {
var list FooList
Load(&list)
fmt.Println(list)
}
I'm going to simplify the problem so it's easier to understand. What is being done there is very similar to this, which also does not work (you can run it here):
type myInt int
func (a myInt) increment() { a = a + 1 }
func increment(b myInt) { b.increment() }
func main() {
var c myInt = 42
increment(c)
fmt.Println(c) // => 42
}
The reason why this does not work is because Go passes parameters by value, as the documentation describes:
In a function call, the function value and arguments are evaluated in the usual
order. After they are evaluated, the parameters of the call are passed by value
to the function and the called function begins execution.
In practice, this means that each of a, b, and c in the example above are pointing to different int variables, with a and b being copies of the initial c value.
To fix it, we must use pointers so that we can refer to the same area of memory (runnable here):
type myInt int
func (a *myInt) increment() { *a = *a + 1 }
func increment(b *myInt) { b.increment() }
func main() {
var c myInt = 42
increment(&c)
fmt.Println(c) // => 43
}
Now a and b are both pointers that contain the address of variable c, allowing their respective logic to change the original value. Note that the documented behavior still holds here: a and b are still copies of the original value, but the original value provided as a parameter to the increment function is the address of c.
The case for slices is no different than this. They are references, but the reference itself is provided as a parameter by value, so if you change the reference, the call site will not observe the change since they are different variables.
There's also a different way to make it work, though: implementing an API that resembles that of the standard append function. Again using the simpler example, we might implement increment without mutating the original value, and without using a pointer, by returning the changed value instead:
func increment(i int) int { return i+1 }
You can see that technique used in a number of places in the standard library, such as the strconv.AppendInt function.
It's worth keeping a mental model of how Go's data structures are implemented. That usually makes it easier to reason about behaviour like this.
http://research.swtch.com/godata is a good introduction to the high-level view.
Go is pass-by-value. This is true for both parameters and receivers. If you need to assign to the slice value, you need to use a pointer.
Then I read somewhere that you shouldn't pass pointers to slices since
they are already references
This is not entirely true, and is missing part of the story.
When we say something is a "reference type", including a map type, a channel type, etc., we mean that it is actually a pointer to an internal data structure. For example, you can think of a map type as basically defined as:
// pseudocode
type map *SomeInternalMapStructure
So to modify the "contents" of the associative array, you don't need to assign to a map variable; you can pass a map variable by value and that function can change the contents of the associative array pointed to by the map variable, and it will be visible to the caller. This makes sense when you realize it's a pointer to some internal data structure. You would only assign to a map variable if you want to change which internal associative array you want it to point to.
However, a slice is more complicated. It is a pointer (to an internal array), plus the length and capacity, two integers. So basically, you can think of it as:
// pseudocode
type slice struct {
underlyingArray uintptr
length int
capacity int
}
So it's not "just" a pointer. It is a pointer with respect to the underlying array. But the length and capacity are "value" parts of the slice type.
So if you just need to change an element of the slice, then yes, it acts like a reference type, in that you can pass the slice by value and have the function change an element and it's visible to the caller.
However, when you append() (which is what you're doing in the question), it's different. First, appending affects the length of the slice, and length is one of the direct parts of the slice, not behind a pointer. Second, appending may produce a different underlying array (if the capacity of the original underlying array is not enough, it allocates a new one); thus the array pointer part of the slice might also be changed. Thus it is necessary to change the slice value. (This is why append() returns something.) In this sense, it cannot be regarded as a reference type, because we are not just "changing what it points to"; we are changing the slice directly.