I need to pass a pointer to a map to another function using the reflect package. Think of encoding/json.Unmarshal which accepts only pointers. Below is some code of what I need to do using reflection:
myTyp := &MyType{}
json.Unmarshal(jsonPayload1, myTyp)
m := make(map[string]*MyType)
m["someKey"] = myTyp
// Dump it again to Unmarshal. It will overwrite it only if
// there is a `someKey` key.
x := json.Unmarshal(jsonPayload2, &m)
So far I have the following:
....
m := reflect.MakeMap(reflect.MapOf(reflect.TypeOf(""), reflect.TypeOf(typ))
x := json.Unmarshal(jsonPayload2, m.Interface())
The issue with this code is that it's making a map of map[string]MyType but there is no way I can create a pointer for it. If I try to make the pointer [2] directly it panics with reflect.MakeMap of non-map type.
Why the map must be allocated:
I need the map allocated so that I can set myTyp on a specific key. myTyp is decoded using multiple decoding sources so I don't know what fields were decoded with each source but I know that Unmarshal overwrites only keys that match the json content/keys so it's safe to use the same variable to multiple Unmarshal functions.
[2] reflect.MakeMap(reflect.PtrTo(reflect.MapOf(reflect.TypeOf("")), reflect.TypeOf(typ))
For a function like json.Unamrshal, you don't need the map, you only need the correct type pointer, which you can make with reflect.New. The json package will allocate the map as needed.
m := reflect.New(reflect.MapOf(reflect.TypeOf(""), reflect.TypeOf(typ)))
If you need to allocate the map, use the Set method to assign it to the pointer.
http://play.golang.org/p/hnOhh8242i
mapType := reflect.MapOf(reflect.TypeOf(""), reflect.TypeOf(typ))
ptr := reflect.New(mapType)
ptr.Elem().Set(reflect.MakeMap(mapType))
Related
When the formal parameter is map, assigning a value directly to a formal parameter cannot change the actual argument, but if you add a new key and value to the formal parameter, the actual argument outside the function can also be seen. Why is that?
I don't understand the output value of the following code, and the formal parameters are different from the actual parameters.
unc main() {
t := map[int]int{
1: 1,
}
fmt.Println(unsafe.Pointer(&t))
copysss(t)
fmt.Println(t)
}
func copysss(m map[int]int) {
//pointer := unsafe.Pointer(&m)
//fmt.Println(pointer)
m = map[int]int{
1: 2,
}
}
stdout :0xc000086010
map[1:1]
func main() {
t := map[int]int{
1: 1,
}
fmt.Println(unsafe.Pointer(&t))
copysss(t)
fmt.Println(t)
}
func copysss(m map[int]int) {
//pointer := unsafe.Pointer(&m)
//fmt.Println(pointer)
m[1] = 2
}
stdout :0xc00007a010
map[1:2]
func main() {
t := map[int]int{
1: 1,
}
fmt.Println(unsafe.Pointer(&t))
copysss(t)
fmt.Println(t)
}
func copysss(m map[int]int) {
pointer := unsafe.Pointer(&m)
fmt.Println(pointer)
m[1] = 2
}
stdout:0xc00008a008
0xc00008a018
map[1:2]
I want to know if the parameter is a value or a pointer.
The parameter is both a value and a pointer.
Wait.. whut?
Yes, a map (and slices, for that matter) are types, pretty similar to what you would implement. Think of a map like this:
type map struct {
// meta information on the map
meta struct{
keyT type
valueT type
len int
}
value *hashTable // pointer to the underlying data structure
}
So in your first function, where you reassign m, you're passing a copy of the struct above (pass by value), and you're assigning a new map to it, creating a new hashtable pointer in the process. The variable in the function scope is updated, but the one you passed still holds a reference to the original map, and with it, the pointer to the original map is preserved.
In the second snippet, you're accessing the underlying hash table (a copy of the pointer, but the pointer points to the same memory). You're directly manipulating the original map, because you're just changing the contents of the memory.
So TL;DR
A map is a value, containing meta information of what the map looks like, and a pointer to the actual data stored inside. The pointer is passed by value, like anything else (same way pointers are passed by value in C/C++), but of course, dereferencing a pointer means you're changing the values in memory directly.
Careful...
Like I said, slices work pretty much in the same way:
type slice struct {
meta struct {
type T
len, cap int
}
value *array // yes, it's a pointer to an underlying array
}
The underlying array is of say, a slice of ints will be [10]int if the cap of the slice is 10, regardless of the length. A slice is managed by the go runtime, so if you exceed the capacity, a new array is allocated (twice the cap of the previous one), the existing data is copied over, and the slice value field is set to point to the new array. That's the reason why append returns the slice that you're appending to, the underlying pointer may have changed etc.. you can find more in-depth information on this.
The thing you have to be careful with is that a function like this:
func update(s []int) {
for i, v := range s {
s[i] = v*2
}
}
will behave much in the same way as the function you have were you're assigning m[1] = 2, but once you start appending, the runtime is free to move the underlying array around, and point to a new memory address. So bottom line: maps and slices have an internal pointer, which can produce side-effects, but you're better off avoiding bugs/ambiguities. Go supports multiple return values, so just return a slice if you set about changing it.
Notes:
In your attempt to figure out what a map is (reference, value, pointer...), I noticed you tried this:
pointer := unsafe.Pointer(&m)
fmt.Println(pointer)
What you're doing there, is actually printing the address of the argument variable, not any address that actually corresponds to the map itself. the argument passed to unsafe.Pointer isn't of the type map[int]int, but rather it's of type *map[int]int.
Personally, I think there's too much confusion around passing by value vs passing by . Go works exactly like C in this regard, just like C, absolutely everything is passed by value. It just so happens that this value can sometimes be a memory address (pointer).
More details (references)
Slices: usage & internals
Maps Note: there's some confusion caused by this one, as pointers, slices, and maps are referred to as *reference types*, but as explained by others, and elsewhere, this is not to be confused with C++ references
In Go, map is a reference type. This means that the map actually resides in the heap and variable is just a pointer to that.
The map is passed by copy. You can change the local copy in your function, but this will not be reflected in caller's scope.
But, since the map variable is a pointer to the unique map residing in the heap, every change can be seen by any variable that points to the same map.
This article can clarify the concept: https://www.ardanlabs.com/blog/2014/12/using-pointers-in-go.html.
As far as I understand, types slice and map are similar in many ways in Go. They both reference (or container) types. In terms of abstract data types, they represent an array and an associative array, respectively.
However, their behaviour is quite different.
var s []int
var m map[int]int
While we can use a declared slice immediately (append new items or reslice it), we cannot do anything with a newly declared map. We have to call make function and initialize a map explicitly. Therefore, if some struct contains a map we have to write a constructor function for the struct.
So, the question is why it is not possible to add some syntaсtic sugar and both allocate and initialize the memory when declaring a map.
I did google the question, learnt a new word "avtovivification", but still failing to see the reason.
I am not talking about struct literal. Yes, you can explicitly initialize a map by providing values such as m := map[int]int{1: 1}. However, if you have some struct:
package main
import (
"fmt"
)
type SomeStruct struct {
someField map[int]int
someField2 []int
}
func main() {
s := SomeStruct{}
s.someField2 = append(s.someField2, -1) // OK
s.someField[0] = -1 // panic: assignment to entry in nil map
fmt.Println(s)
}
It is not possible to use a struct immediately (with default values for all fields). One has to create a constructor function for SomeStruct which has to initialize a map explicitly.
While we can use a declared slice immediately (append new items or reslice it), we cannot do anything with a newly declared map. We have to call make function and initialize a map explicitly. Therefore, if some struct contains a map we have to write a constructor function for the struct.
That's not true. Default value–or more precisely zero value–for both slices and maps is nil. You may do the "same" with a nil map as you can do with a nil slice. You can check length of a nil map, you can index a nil map (result will be the zero value of the value type of the map), e.g. the following are all working:
var m map[int]int
fmt.Println(m == nil) // Prints true
fmt.Println(len(m)) // Prints 0
fmt.Println(m[2]) // Prints 0
Try it on the Go Playground.
What you "feel" more about the zero-value slice is that you may add values to it. This is true, but under the hood a new slice will be allocated using the exact make() builtin function that you'd have to call for a map in order to add entries to it, and you have to (re)assign the returned slice. So a zero-value slice is "no more ready for use" than a zero-value map. append() just takes care of necessary (re)allocation and copying over. We could have an "equivalent" addEntry() function to which you could pass a map value and the key-value pairs, and if the passed map is nil, it could allocate a new map value and return it. If you don't call append(), you can't add values to a nil slice, just as you can't add entries to a nil map.
The primary reason that the zero value for slices and maps is nil (and not an initialized slice or map) is performance and efficiency. It is very often that a map or slice value (either variable or a struct field) will never get used, or not right away, and so if they would be allocated at declaration, that would be a waste of memory (and some CPU) resources, not to mention it gives more job to the garbage collector. Also if the zero value would be an initialized value, it would often be insufficient (e.g. a 0-size slice cannot hold any elements), and often it would be discarded as you add new elements to it (so the initial allocation would be a complete waste).
Yes, there are cases when you do want to use slices and maps right away, in which cases you may call make() yourself, or use a composite literal. You may also use the special form of make() where you supply the (initial) capacity for maps, avoiding future restructuring of the map internals (which usually requires non-negligible computation). An automatic non-nil default value could not guess what capacity you'd require.
You can! What you're looking for is:
package main
import "fmt"
func main() {
v := map[int]int{}
v[1] = 1
v[2] = 2
fmt.Println(v)
}
:= is declare and assign, where as var is simply declare.
Supposedly maps are reference types in Go, so when returning them from functions, you don't need to pass as a pointer to the map in order for the changes to be visible outside the function body. But what if said map is returned from a method on a non-pointer struct?
For example:
type ExampleMapHolder struct {
theUnexportedMap map[string]int
}
func (emp ExampleMapHolder) TheMap() map[string]int {
return emp.theUnexportedMap
}
If I make a call to TheMap(), and then modify a value in it, will this change be visible elsewhere even though the receiver is not a pointer? I imagine it would return a reference to a map that belonged to a copy of ExampleMapHolder, but haven't been able to find an explicit answer in the docs.
Why won't you just check it?
emp := ExampleMapHolder{make(map[string]int)}
m := emp.TheMap()
m["a"] = 1
fmt.Println(emp) // Prints {map[a:1]}
Playground: http://play.golang.org/p/jGZqFr97_y
I'm getting this return value from a function call in the "reflect" package:
< map[string]string Value >.
Wondering if I can access the actual map inside the return value and if so, how?
EDIT:
So this is where I'm making the call which returns the Value object.
It returns [< map[string]string Value >] to which I grab the first object in that array. However, I'm not sure how to convert [< map[string]string Value >] into a regular map.
view_args := reflect.ValueOf(&controller_ref).MethodByName(action_name).Call(in)
Most reflect Value objects can be converted back to a interface{} value using the .Interface() method.
After obtaining this value, you can assert it back to the map you want. Example (play):
m := map[string]int{"foo": 1, "bar": 3}
v := reflect.ValueOf(m)
i := v.Interface()
a := i.(map[string]int)
println(a["foo"]) // 1
In the example above, m is your original map and v is the reflected value. The interface value i, acquired by the Interface method is asserted to be of type map[string]int and this value is used as such in the last line.
To turn the value in a reflect.Value into an interface{}, you use iface := v.Interface(). Then, to access that, you use a type assertion or type switch.
If you know you're getting a map[string]string the assertion is simply m := iface.(map[string]string). If there's a handful of possibilities, the type switch to handle them all looks like:
switch item := iface.(type) {
case map[string]string:
fmt.Println("it's a map, and key \"key\" is", item["key"])
case string:
fmt.Println("it's a string:", item)
default:
// optional--code that runs if it's none of the above types
// could use reflect to access the object if that makes sense
// or could do an error return or panic if appropriate
fmt.Println("unknown type")
}
Of course, that only works if you can write out all the concrete types you're interested out in the code. If you don't know the possible types at compile time, you have to use methods like v.MapKeys() and v.MapIndex(key) to work more with the reflect.Value, and, in my experience, that involves a long time looking at the reflect docs and is often verbose and pretty tricky.
I'm currently getting desperate over the behaviour of golangs reflect package, which to me doesn't seem consistent at all.
1) As far as I understand it, a reflect.Value seems to carry a pointer to the underlying value.
E.g. if I call
var s string
v1 := reflect.ValueOf(&s).Elem()
v2 := v1
v2.SetString("Hello World!")
fmt.Println(s)
It prints me "Hello World!".
However, this doesn't seem to hold true for a reflect.Value obtained by a call to Field().
val := ... //Assign a reflect.Value to it
nextval := val.Field(0) //Make sure that Field exists and is of type map
nextval = reflect.MakeMap(reflect.MapOf(KEY, ELEM))
nextval.SetMapIndex(Some_value_of_type_KEY, Something_of_type_ELEM)
fmt.Println(nextval.MapKeys()
fmt.Println(val.Field(index).MapKeys())
This prints
[Some_value_of_type_KEY]
[]
which is a major annoyance. Anyone knows why this is the case?
===================================================
2) Consider the function
func Test(v interface{}) {
val := reflect.ValueOf(v)
if val.Kind() != reflect.Struct {
fmt.Println("It is a struct")
}
}
If I call it with any struct as an argument it prints "This is a struct".
However, I won't be able to assign new values to stuff inside v by using val,
due to the value not being addressable. Working around by the following:
func Test(v interface{}) {
val := reflect.ValueOf(&v).Elem()
if val.Kind() != reflect.Struct {
fmt.Println("This never get's printed!")
}
}
According to the doc, I would assume, that by taking the '&' I use a pointer to v and by the call of Elem() I get the element it points to, therefore val.Kind() should still return the same thing. It doesn't. val.Kind() now is a reflect.Interface.
Is there a way of not having to go
valForTestingKind := reflect.ValueOf(v)
valForSettingNewValue := reflect.ValueOf(&v).Elem()
as this somehow feels wrong.
Part 1:
By assigning to nextval, you are breaking its association with the original val. Instead, use the Set() method.
nextval.Set(reflect.MakeMap(reflect.MapOf(KEY, ELEM)))
Set() is the equivalent of assignment in the reflection world. Of course, you must make sure it is assignable using reflect.ValueOf(&v).Elem() as you do in your first code example.
Part 2:
The issue here is that you have another level of indirection. v is of type interface{} and has a concrete value whose type is of Kind struct. Just like with every function that accepts an interface typed parameter, when you call reflect.ValueOf, the parameter is automatically converted to that type. However, converting an interface to another interface results in the concrete value being reboxed in the new interface type. The information of the type before it was reboxed is lost. As an example, a function that accepts an io.Writer would not know that the calling function considered it an io.ReaderWriter.
In this context, it means that reflect.ValueOf cannot tell if you passed an os.File (some struct) or a file boxed in an interface{}. It assumes you passed an os.File and shows you the Kind "struct".
However, when you pass a pointer to an interface{}, you are passing an interface{} variable that can be modified. You are not passing the underlying concrete type and that has important consequences. You can .Set() anything, not just what the original concrete type allows. You also can't edit individual fields as anything in an interface{} is not assignable. If the concrete type is in fact a pointer, you can do a fourth dereference (.Elem()) and modify fields from there.
So, what does this mean in terms of code?
//let v = an interface{} with a concrete type of SomeStruct
val := reflect.ValueOf(&v).Elem()
fmt.Println(val.Elem().Kind()) // struct
val.Elem().Field(0).Set(10) // PANIC! Field isn't assignable.
val.Set("a string which is not a SomeStruct")
fmt.Println(val.Elem().Kind()) // string
I made an example here: http://play.golang.org/p/6MULn3KoNh
I want to talk about your second block of code:
val := ... //Assign a reflect.Value to it
nextval := val.Field(0) //Make sure that Field exists and is of type map
nextval = reflect.MakeMap(reflect.MapOf(KEY, ELEM))
nextval.SetMapIndex(Some_value_of_type_KEY, Something_of_type_ELEM)
fmt.Println(nextval.MapKeys()
fmt.Println(val.Field(index).MapKeys())
On the third line, you are reassigning a new, different object to the variable nextval. Shouldn't you call some kind of setting method on nextval instead of reassigning it? In your first example, you called SetString but in this example you are just reassigning the variable and that might be why the behavior is different. After you reassign the variable, nextval will no longer be connected in any way to val.Field(0). Also, what is index?
If this does not explain your problem, please edit the question to contain a short, self-contained, correct, compilable example ( SSCCE ). I want to be able to post it into the text box on the front page of golang.org in order to see the problem. You should always post an SSCCE when possible.
You have not shown a complete and compilable code. Do you pass a pointer to a struct or do you pass the struct by value? In the later case reflection cannot mutate it.
Values stored in a map are not addressable even when not using reflection.
http://play.golang.org/p/wYLeJ3W4R2
http://play.golang.org/p/ttUGBVh1lc
https://groups.google.com/forum/#!topic/golang-nuts/jzjEXoc9FwU
https://groups.google.com/forum/#!topic/golang-nuts/V_5kwzwKJAY