How to get underlying value from a reflect.Value in golang? - reflection

So I found some code that help me get started with reflection in Go (golang), but I'm having trouble getting a the underlying value so that I can basically create a map[string]string from a struct and it's fields.
Eventually, I'd like to make the result into a map[string]interface{}, but this one issue is kind of blocking me.
The code I have at the moment:
package main
import (
"fmt"
"reflect"
)
type Foo struct {
FirstName string `tag_name:"tag 1"`
LastName string `tag_name:"tag 2"`
Age int `tag_name:"tag 3"`
}
func inspect(f interface{}) map[string]string {
m := make(map[string]string)
val := reflect.ValueOf(f).Elem()
for i := 0; i < val.NumField(); i++ {
valueField := val.Field(i)
typeField := val.Type().Field(i)
f := valueField.Interface()
val := reflect.ValueOf(f)
m[typeField.Name] = val.String()
}
return m
}
func dump(m map[string]string) {
for k, v := range m {
fmt.Printf("%s : %s\n", k, v)
}
}
func main() {
f := &Foo{
FirstName: "Drew",
LastName: "Olson",
Age: 30,
}
a := inspect(f)
dump(a)
}
The output from running the code:
FirstName : Drew
LastName : Olson
Age : <int Value>
From what I understand the output for FirstName and LastName are actual reflect.Value objects but for strings the String() method on value just outputs the underlying String. I'd like to either get the int and change it into a string, but from the relfect package documentation I'm not immediately seeing how that's done.
Soo.... How do I get the underlying value from a reflect.Value in golang?

A good example of how to parse values is the fmt package. See this code.
Using the mentioned code to match your problem would look like this:
switch val.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
m[typeField.Name] = strconv.FormatInt(val.Int(), 10)
case reflect.String:
m[typeField.Name] = val.String()
// etc...
}
Basically you need to check for all available Kinds.

It looks like you're on the right track. The problem I see with your code is it makes assumptions about the values, meaning when do you call Elem() and how many times (to resolve pointers). In order to know this you need to look at the reflect.Kind. Is the value a reflect.Ptr? Then use Elem().
Once you have the value from val.Interface() / val.String() / val.Int() you can convert your values as needed. What you use is going to depend on reflect.Kind. To convert an int to/from string you need to use the strconv package.
The encoding/json and encoding/xml packages do this kind of work already. The source code provides some great examples. For example, take a look at copyValue in encoding/xml/read.go and marshalSimple in encoding/xml/marshal.go.

This should be easier to do with Go 1.5 (August 2015)
See review 8731 and commit 049b89d by Rob Pike (robpike):
fmt: treat reflect.Value specially - as the value it holds
This would allow you to print the actual value of a Reflect.Value() argument:
When a reflect.Value is passed to Printf (etc.), fmt called the String method, which does not disclose its contents.
To get the contents, one could call Value.Interface(), but that is illegal
if the Value is not exported or otherwise forbidden.
This CL improves the situation with a trivial change to the fmt package: when we see a reflect.Value as an argument, we treat it exactly as we treat a reflect.Value we make inside the package.
This means that we always print the contents of the Value as if that was the argument to Printf.
This is arguably a breaking change but I think it is a genuine improvement and no greater a break than many other tweaks we have made to formatted output from this package.

Another simple solution can be ,
flavorName = fmt.Sprintf("%v",strct)
" fmt.Sprintf() " will return the value which can be stored in a variable.

Related

Pointer problems

TL;DR Somehow, I am appending a pointer to a list instead of the object within a for loop of objects so at the end the entire slice is composed of the same object multiple times. I just don't know how to fix that.
The Long Way
I am still having a super hard time trying to figure out pointers in go. I posted a question yesterday and got some help but now I am stuck on a slightly different issue in the same piece of code.
I am working with gocql and cqlr go packages to try and bit a small object mapper for my Cassandra queries. Essentially the problem I am having is I am appending what appears to be a pointer to an object, not a new instance of the obj to the array. How do I fix that? I have tried adding & and * in front of value but that doesn't seem to work. How do I fix these? The bind function needs an & according to their docs.
Code
type Query struct {
query string
values interface{}
attempts int
maxAttempts int
structType reflect.Type
}
func (query Query) RetryingQuery() (results []interface{}) {
var q *gocql.Query
if query.values != nil {
q = c.Session.Query(query.query, query.values)
} else {
q = c.Session.Query(query.query)
}
bindQuery := cqlr.BindQuery(q)
value := reflect.New(query.structType).Interface()
for bindQuery.Scan(value) {
fmt.Println(value)
results = append(results, value)
}
return
}
The docs ask for var value type then in bind you would pass &value. I quoted the docs below.
var t Tweet
var s []Tweet
for b.Scan(&t) {
// Application specific code goes here
append(s, t)
}
The issue is I cannot directly go var value query.structType to define its type then pass the reference of that to bindQuery.Scan().
What is printed
&{result1 x86_64 24 3.2.0-74-generic Linux}
&{result2 x86_64 24 3.19.0-25-generic Linux}
&{result3 x86_64 4 3.13.0-48-generic Linux}
&{result4 x86_64 2 3.13.0-62-generic Linux}
&{result5 x86_64 4 3.13.0-48-generic Linux}
What is in the slice
Spoiler, it is result5 repeated over and over. I understand that I am just appending the pointer to same object to the list and that every loop iteration the object is changed and that changes all the results in the slice to that new object. I just don't know how to fix it.
[{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"},{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"},{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"},{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"},{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"}]
Well I can at least tell you what you're doing. bindQuery takes a pointer. It changes the value stored at the address.
What you're essentially doing is this:
package main
import "fmt"
func main() {
var q int
myInts := make([]*int, 0, 5)
for i := 0; i < 5; i++ {
q = i
fmt.Printf("%d ", q)
myInts = append(myInts, &q)
}
fmt.Printf("\n")
for _, value := range myInts {
fmt.Printf("%d ", *value)
}
fmt.Printf("\n")
fmt.Println(myInts)
}
Which, as you can probably guess, gives you this:
0 1 2 3 4
4 4 4 4 4
[0x104382e0 0x104382e0 0x104382e0 0x104382e0 0x104382e0]
Things get a little more confusing with reflect. You can get your type as an interface, but that is it (unless you want to play with unsafe). An interface, in simple terms, contains a pointer to the original type underneath (and some other stuff). So in your function you are passing a pointer (and some other stuff). Then you're appending the pointer. It might be nice just to get concrete and type switch your interface. I assume you know what types it could be. In which case you'd have to have something along these lines:
package main
import (
"fmt"
"reflect"
)
type foo struct {
fooval string
}
type bar struct {
barval string
}
func main() {
f1 := foo{"hi"}
f2 := &foo{"hi"}
b1 := bar{"bye"}
b2 := &bar{"bye"}
doSomething(f1)
doSomething(f2)
doSomething(b1)
doSomething(b2)
}
func doSomething(i interface{}) {
n := reflect.TypeOf(i)
// get a new one
newn := reflect.New(n).Interface()
// find out what we got and handle each case
switch t := newn.(type) {
case **foo:
*t = &foo{"hi!"}
fmt.Printf("It was a **foo, here is the address %p and here is the value %v\n", *t, **t)
case **bar:
*t = &bar{"bye :("}
fmt.Printf("It was a **bar, here is the address %p and here is the value %v\n", *t, **t)
case *foo:
t = &foo{"hey!"}
fmt.Printf("It was a *foo, here is the address %p and here is the value %v\n", t, *t)
case *bar:
t = &bar{"ahh!"}
fmt.Printf("It was a *bar, here is the address %p and here is the value %v\n", t, *t)
default:
panic("AHHHH")
}
}
You could also just keep calling value = reflect.New(query.structType).Interface() inside of the loop which will give you new interfaces every time. Reassigning value after every append. Last time through the loop would make one extra though..

How to switch on reflect.Type?

I have managed to do this, but it does not look efficient:
var t reflect.Type
switch t {
case reflect.TypeOf(([]uint8)(nil)):
// handle []uint8 array type
}
First question, are you sure you want to switch on reflect.Type and not use a type switch? Example:
switch x := y.(type) {
case []uint8:
// x is now a []uint8
}
Assuming that will not work for your situation, my recommendation is to make those package variables. Example:
var uint8SliceType = reflect.TypeOf(([]uint8)(nil))
func Foo() {
var t reflect.Type
switch t {
case uint8SliceType:
// handle []uint8 array type
}
}
you may not need reflect if you are just trying to detect type.
switch t := myVar.(type){
case []uint8:
// t is []uint8
case *Foo:
// t is *Foo
default:
panic("unknown type")
}
What are you actually trying to accomplish?
The answer to the initial question How to switch on reflect.Type? is: You can’t. However, you can do it with reflect.Value.
Given a variable v interface{} you can call reflect.TypeOf(v) and reflect.ValueOf(v), which return a reflect.Type or reflect.Value, resp.
If the type of v is not interface{} then these function calls will convert it to interface{}.
reflect.Type contains various run-time information about the type, but it does not contain anything usable to retrieve the type of v itself as needed in a type switch.
Hovewer, reflect.Value provides it through its Interface() method, which returns the underlying value as interface{}. This you can use in a type switch or type assertion.
import "fmt"
import "reflect"
var v int
var rt reflect.Type = reflect.TypeOf(v)
fmt.Println(rt.String(), " has awesome properties: Its alignment is",
rt.Align(), ", it has", rt.Size(), "bytes, is it even comparable?",
rt.Comparable())
// … but reflect.Type won’t tell us what the real type is :(
// Let’s see if reflect.Value can help us.
var rv reflect.Value = reflect.ValueOf(v)
// Here we go:
vi := rv.Interface()
switch vi.(type) {
// Mission accomplished.
}
Perhaps it helps to clarify a few points which may cause confusion about dynamic typing in Go. At least I was confused by this for quite some time.
reflect vs. interface{}
In Go there are two systems of run-time generics:
In the language: interface{}, useful for type switches/assertions,
In the library: The reflect package, useful for inspection of run-time generic types and values of such.
These two systems are separated worlds, and things that are possible with one are impossible with the other. For example, Given an interface{}, it is in plain Go (with safe code) impossible to, say, if the value is an array or slice, regardless of its element type, then get the value of the i-th element. One needs to use reflect in order to do that. Conversely, with reflect it is impossible to make a type switch or assertion: convert it to interface{}, then you can do that.
There are only very few points of an interface between these systems. In one direction it is the TypeOf() and ValueOf() functions which accept interface{} and return a reflect struct. In the other direction it is Value.Interface().
It is a bit counter-intuitive that one needs a Value, not a Type, to do a type switch. At least this is somewhat consistent with the fact that one needs a value construct a Type by calling TypeOf().
reflect.Kind
Both reflect.Type and reflect.Value have a Kind() method. Some suggest using the value these methods return, of type reflect.Kind, to imitate a type switch.
While this may be useful in certain situations, it is not a replacement for a type switch. For example, using Kind one cannot distinguish between int64 and time.Duration because the latter is defined as
type Duration int64
Kind is useful to tell if a type is any kind of struct, array, slice etc., regardless of the types it is composed of. This is not possible to find out with a type switch.
(Side note. I had the same question and found no answer here helpful so I went to figure it out myself. The repeated counter-question “why are you doing this?”, followed by unrelated answers did not help me either. I have a good reason why I want to do it precisely this way.)
This might work.
switch t := reflect.TypeOf(a).String() {
case "[]uint8":
default:
}
As others have said, it's not clear what you are trying to achieve by switching on reflect.Type However, I came across this question when probably trying to do something similar, so I will give you my solution in case it answers your question.
As captncraig said, a simple type switch could be done on a interface{} variable without needing to use reflect.
func TypeSwitch(val interface{}) {
switch val.(type) {
case int:
fmt.Println("int with value", val)
case string:
fmt.Println("string with value ", val)
case []uint8:
fmt.Println("Slice of uint8 with value", val)
default:
fmt.Println("Unhandled", "with value", val)
}
}
However, going beyond this, the usefulness of reflection in the context of the original question could be in a function that accepts a struct with arbitrarily typed fields, and then uses a type switch to process the field according to its type. It is not necessary to switch directly on reflect.Type, as the type can be extracted by reflect and then a standard type switch will work. For example:
type test struct {
I int
S string
Us []uint8
}
func (t *test) SetIndexedField(index int, value interface{}) {
e := reflect.ValueOf(t).Elem()
p := e.Field(index)
v := p.Interface()
typeOfF := e.Field(index).Type()
switch v.(type) {
case int:
p.SetInt(int64(value.(int)))
case string:
p.SetString(value.(string))
case []uint8:
p.SetBytes(value.([]uint8))
default:
fmt.Println("Unsupported", typeOfF, v, value)
}
}
The following examples demonstrate the use of this function:
var t = test{10, "test string", []uint8 {1, 2, 3, 4}}
fmt.Println(t)
(&t).SetIndexedField(0, 5)
(&t).SetIndexedField(1, "new string")
(&t).SetIndexedField(2, []uint8 {8, 9})
fmt.Println(t)
(A few points on reflection in go:
It is necessary to export the struct fields for reflect to be able to use them, hence the capitalisation of the field names
In order to modify the field values, it would be necessary to use a pointer to the struct as in this example function
Elem() is used to "dereference" the pointer in reflect
)
Well, I did this by first transfer it to interface and then use the.(type)
ty := reflect.TypeOf(*c)
vl := reflect.ValueOf(*c)
for i:=0;i<ty.NumField();i++{
switch vl.Field(i).Interface().(type) {
case string:
fmt.Printf("Type: %s Value: %s \n",ty.Field(i).Name,vl.Field(i).String())
case int:
fmt.Printf("Type: %s Value: %d \n",ty.Field(i).Name,vl.Field(i).Int())
}
}

How would you access the underlying array passed to a function expecting an empty interface in Go?

So let's say that we have a function of the following form:
func WorkMagic(obj interface{}) interface{} {
switch t := obj.(type) {
case string:
// Do string magic
default:
// Do slice magic
}
...
}
I am expecting obj to be either a string or a slice, which I can ascertain via the switch. In the case of a slice, I want to be able to do ordering work on any arbitrary slice, regardless of type. Seems like the best way to accomplish this is using the unsafe package in a similar fashion to that discussed in this article.
Here however, the function accepts a specific type of slice ([]string), whereas I would like to be able to work on any slice. So the question is, given that I am accepting an empty interface as input, how might I access the underlying slice / array using unsafe.Pointer so as to be able to loop through and modify which value is associate with which index?
You'll want to use reflection. It enables you to work generically without giving up type and memory safety like unsafe would. Read the Go blog's Laws of Reflection.
func actOnSlices(i interface{}) {
v := reflect.ValueOf(i)
for v.Kind() == reflect.Ptr { // dereference pointers
v = v.Elem()
}
if v.Kind() != reflect.Slice { // ensure you actually got a slice
panic("given argument is not a slice")
}
// do slice stuff
}
Edit to answer your second question:
Yes – this can be done: elements of a slice are adressable and hence settable. See the following working example:
package main
import (
"fmt"
"reflect"
)
func main() {
s := []string{"foo", "bar"}
fmt.Println(swapIndexes(s, 0, 1)) // prints [bar foo]
}
func swapIndexes(i interface{}, x, y int) interface{} {
v := reflect.ValueOf(i)
for v.Kind() == reflect.Ptr { // dereference pointers
v = v.Elem()
}
if v.Kind() != reflect.Slice { // ensure you actually got a slice
panic("given argument is not a slice")
}
t := v.Index(x).Interface()
v.Index(x).Set(v.Index(y))
v.Index(y).Set(reflect.ValueOf(t))
return v.Interface()
}
Edit to answer your third question:
The unsafe package is not something you'll encounter much in user-land code. It exists to implement certain features (e.g. reflection, C interaction) that need to circumvent Go's safety guarantees to work. Using unsafe is unsafe, as the name suggests, because you can mess up big time without even realizing. By using unsafe, you're incurring in a big trade-off, so it better be worth it. Quoting #twotwotwo:
The downside of unsafe is that if you mess up you're in the old days of segfaults, memory corruption, and buffer-overflow security holes.
Also, as #twotwotwo suggested; it's more "Go-like" to repeat code than using reflection to achieve genericity.
To Go's type-system, []string and []int are two completely separate and unrelated types. just as int and string would be. The relation (both are slices) is obvious only to the programmer. There is no way of expressing "a slice" without saying a slice of what.

Convert Value type to Map in Golang?

I'm getting this return value from a function call in the "reflect" package:
< map[string]string Value >.
Wondering if I can access the actual map inside the return value and if so, how?
EDIT:
So this is where I'm making the call which returns the Value object.
It returns [< map[string]string Value >] to which I grab the first object in that array. However, I'm not sure how to convert [< map[string]string Value >] into a regular map.
view_args := reflect.ValueOf(&controller_ref).MethodByName(action_name).Call(in)
Most reflect Value objects can be converted back to a interface{} value using the .Interface() method.
After obtaining this value, you can assert it back to the map you want. Example (play):
m := map[string]int{"foo": 1, "bar": 3}
v := reflect.ValueOf(m)
i := v.Interface()
a := i.(map[string]int)
println(a["foo"]) // 1
In the example above, m is your original map and v is the reflected value. The interface value i, acquired by the Interface method is asserted to be of type map[string]int and this value is used as such in the last line.
To turn the value in a reflect.Value into an interface{}, you use iface := v.Interface(). Then, to access that, you use a type assertion or type switch.
If you know you're getting a map[string]string the assertion is simply m := iface.(map[string]string). If there's a handful of possibilities, the type switch to handle them all looks like:
switch item := iface.(type) {
case map[string]string:
fmt.Println("it's a map, and key \"key\" is", item["key"])
case string:
fmt.Println("it's a string:", item)
default:
// optional--code that runs if it's none of the above types
// could use reflect to access the object if that makes sense
// or could do an error return or panic if appropriate
fmt.Println("unknown type")
}
Of course, that only works if you can write out all the concrete types you're interested out in the code. If you don't know the possible types at compile time, you have to use methods like v.MapKeys() and v.MapIndex(key) to work more with the reflect.Value, and, in my experience, that involves a long time looking at the reflect docs and is often verbose and pretty tricky.

Golang reflection.Value behaviour

I'm currently getting desperate over the behaviour of golangs reflect package, which to me doesn't seem consistent at all.
1) As far as I understand it, a reflect.Value seems to carry a pointer to the underlying value.
E.g. if I call
var s string
v1 := reflect.ValueOf(&s).Elem()
v2 := v1
v2.SetString("Hello World!")
fmt.Println(s)
It prints me "Hello World!".
However, this doesn't seem to hold true for a reflect.Value obtained by a call to Field().
val := ... //Assign a reflect.Value to it
nextval := val.Field(0) //Make sure that Field exists and is of type map
nextval = reflect.MakeMap(reflect.MapOf(KEY, ELEM))
nextval.SetMapIndex(Some_value_of_type_KEY, Something_of_type_ELEM)
fmt.Println(nextval.MapKeys()
fmt.Println(val.Field(index).MapKeys())
This prints
[Some_value_of_type_KEY]
[]
which is a major annoyance. Anyone knows why this is the case?
===================================================
2) Consider the function
func Test(v interface{}) {
val := reflect.ValueOf(v)
if val.Kind() != reflect.Struct {
fmt.Println("It is a struct")
}
}
If I call it with any struct as an argument it prints "This is a struct".
However, I won't be able to assign new values to stuff inside v by using val,
due to the value not being addressable. Working around by the following:
func Test(v interface{}) {
val := reflect.ValueOf(&v).Elem()
if val.Kind() != reflect.Struct {
fmt.Println("This never get's printed!")
}
}
According to the doc, I would assume, that by taking the '&' I use a pointer to v and by the call of Elem() I get the element it points to, therefore val.Kind() should still return the same thing. It doesn't. val.Kind() now is a reflect.Interface.
Is there a way of not having to go
valForTestingKind := reflect.ValueOf(v)
valForSettingNewValue := reflect.ValueOf(&v).Elem()
as this somehow feels wrong.
Part 1:
By assigning to nextval, you are breaking its association with the original val. Instead, use the Set() method.
nextval.Set(reflect.MakeMap(reflect.MapOf(KEY, ELEM)))
Set() is the equivalent of assignment in the reflection world. Of course, you must make sure it is assignable using reflect.ValueOf(&v).Elem() as you do in your first code example.
Part 2:
The issue here is that you have another level of indirection. v is of type interface{} and has a concrete value whose type is of Kind struct. Just like with every function that accepts an interface typed parameter, when you call reflect.ValueOf, the parameter is automatically converted to that type. However, converting an interface to another interface results in the concrete value being reboxed in the new interface type. The information of the type before it was reboxed is lost. As an example, a function that accepts an io.Writer would not know that the calling function considered it an io.ReaderWriter.
In this context, it means that reflect.ValueOf cannot tell if you passed an os.File (some struct) or a file boxed in an interface{}. It assumes you passed an os.File and shows you the Kind "struct".
However, when you pass a pointer to an interface{}, you are passing an interface{} variable that can be modified. You are not passing the underlying concrete type and that has important consequences. You can .Set() anything, not just what the original concrete type allows. You also can't edit individual fields as anything in an interface{} is not assignable. If the concrete type is in fact a pointer, you can do a fourth dereference (.Elem()) and modify fields from there.
So, what does this mean in terms of code?
//let v = an interface{} with a concrete type of SomeStruct
val := reflect.ValueOf(&v).Elem()
fmt.Println(val.Elem().Kind()) // struct
val.Elem().Field(0).Set(10) // PANIC! Field isn't assignable.
val.Set("a string which is not a SomeStruct")
fmt.Println(val.Elem().Kind()) // string
I made an example here: http://play.golang.org/p/6MULn3KoNh
I want to talk about your second block of code:
val := ... //Assign a reflect.Value to it
nextval := val.Field(0) //Make sure that Field exists and is of type map
nextval = reflect.MakeMap(reflect.MapOf(KEY, ELEM))
nextval.SetMapIndex(Some_value_of_type_KEY, Something_of_type_ELEM)
fmt.Println(nextval.MapKeys()
fmt.Println(val.Field(index).MapKeys())
On the third line, you are reassigning a new, different object to the variable nextval. Shouldn't you call some kind of setting method on nextval instead of reassigning it? In your first example, you called SetString but in this example you are just reassigning the variable and that might be why the behavior is different. After you reassign the variable, nextval will no longer be connected in any way to val.Field(0). Also, what is index?
If this does not explain your problem, please edit the question to contain a short, self-contained, correct, compilable example ( SSCCE ). I want to be able to post it into the text box on the front page of golang.org in order to see the problem. You should always post an SSCCE when possible.
You have not shown a complete and compilable code. Do you pass a pointer to a struct or do you pass the struct by value? In the later case reflection cannot mutate it.
Values stored in a map are not addressable even when not using reflection.
http://play.golang.org/p/wYLeJ3W4R2
http://play.golang.org/p/ttUGBVh1lc
https://groups.google.com/forum/#!topic/golang-nuts/jzjEXoc9FwU
https://groups.google.com/forum/#!topic/golang-nuts/V_5kwzwKJAY

Resources