Using Go's reflect package, is there a way to set a pointer in a struct if the pointer is nil? Looking at the reflect package, if reflect.Value.CanSet() is false then any Set() calls will yield a panic. Is there another way around this, only using the reflect package, not depending on any direct reference to the struct? For example, how can I set the empty last name as "Johnson" in the code below?
package main
import (
"fmt"
"reflect"
)
type Person struct {
FirstName *string
LastName *string
}
func main() {
p := Person{}
firstName := "Ryan"
p.FirstName = &firstName
rp := reflect.ValueOf(p)
fmt.Println("rp:", rp)
for i := 0; i < rp.NumField(); i++ {
fmt.Printf("%s: Pointer: %d CanSet: %t\n", rp.Type().Field(i).Name, rp.Field(i).Pointer(), rp.Field(i).Elem().CanSet())
}
rp.Field(0).Elem().SetString("Brian")
fmt.Println(*p.FirstName)
// Yields nil pointer error
// rp.Field(1).Elem().SetString("Johnson")
// fmt.Println(*p.LastName)
fmt.Println(rp.Field(1).Type())
fmt.Println(rp.Field(1).CanSet())
// fmt.Println(rp.Field(1).Elem().Type()) // nil pointer here also
fmt.Println(rp.Field(1).Elem().CanSet())
}
See in Golang Playground
You first need a pointer to a Person{}, because you need to set a value to the LastName Field.
p := &Person{}
Then you can set a valid pointer value to the LastName field, which will allow you to set the string value:
rp.Field(1).Set(reflect.New(rp.Field(1).Type().Elem()))
rp.Field(1).Elem().SetString("Jones")
https://play.golang.org/p/f5MjpkDI2H
Related
The task is pretty simple. I have struct for model Foo, and one for it's representation:
type Foo struct {
FooId string
Bar string
Baz *string
Salt int64
}
type FooView struct {
FooId *string `json: "foo_id"`
Bar *string `json: "bar"`
Baz *string `json: "baz"`
}
As you may see, there is Salt field which I want to hide, change JSON field names, and do all the fields optional. The target method should fill FooView using Foo like this:
func MirrorFoo(foo Foo) (*FooView, error) {
return &FooView{
FooId: &foo.FooId,
Bar: &foo.Bar,
Baz: foo.Baz,
}, nil
}
And now, I want to do the same with Go reflect:
func Mirror(src interface{}, dstType reflect.Type) (interface{}, error) {
zeroValue := reflect.Value{}
srcValue := reflect.ValueOf(src)
srcType := srcValue.Type()
dstValue := reflect.New(dstType)
dstValueElem := dstValue.Elem()
for i := 0; i < srcType.NumField(); i++ {
srcTypeField := srcType.Field(i)
srcValueField := srcValue.FieldByName(srcTypeField.Name)
dstField := dstValueElem.FieldByName(srcTypeField.Name)
// if current source field exists in destination type
if dstField != zeroValue {
srcValueField := srcValue.Field(i)
if dstField.Kind() == reflect.Ptr && srcValueField.Kind() != reflect.Ptr {
panic("???")
} else {
dstField.Set(srcValueField)
}
}
}
return dstValue.Interface(), nil
}
This code works fine when FooId is uuid.UUID in both types, but it fails when source is uuid.UUID and destination is *uuid.UUID, and now do not know how to overcome this.
Somehow I need to do analog of dstField.Set(reflect.ValueOf(&uuid.UUID{}...)) bit everything I've tried does not works. Any ideas?
Yep, Addr() worked for me, but without
reflect.Copy()
It's for arrays. I've instantiated new value using reflect.New() and .Set() current value. Address became available. It's completely black magic. Thanks for all.
Given this struct
type A struct {
b *B
c string
d string
}
I want to check with reflect which fields have a struct behind the pointer type, so specifically I want to create a condition that would only be true when iterating over field b. From what I've tried using the reflect documentation I always end up with an invalid Value kind which doesn't allow me to go further as every subsequent method panics.
package main
import (
"fmt"
"reflect"
)
type A struct {
b *B
c string
d string
}
type B struct {
}
func main() {
val := reflect.ValueOf(A{})
for i := 0; i < val.Type().NumField(); i++ {
if val.Field(i).Kind() == reflect.Ptr {
fmt.Println(reflect.Indirect(val.Field(i)).Kind())
}
fmt.Println(val.Field(i).Kind())
}
}
https://play.golang.org/p/oRry3ZubRxI
You get invalid value, because the val.b pointer field is nil, and you can't dereference a nil pointer. If you want your code to work, you have to initialize it with a valid pointer:
val := reflect.ValueOf(A{b: &B{}})
With this change it works and outputs (try it on the Go Playground):
struct
ptr
string
If you want it to work without having to initialize the pointer, then you have to work on the types and not values:
val := reflect.ValueOf(A{})
t := val.Type()
for i := 0; i < t.NumField(); i++ {
if ft := t.Field(i).Type; ft.Kind() == reflect.Ptr {
fmt.Println(ft.Elem().Kind())
}
fmt.Println(t.Field(i).Type.Kind())
}
This outputs the same, try this one on the Go Playground.
When checking the reflect.Kind() of a slice, reflect correctly identifies it as a slice when not stored inside a pointer:
package main
import (
"fmt"
"reflect"
)
type Dog struct {
Name string
Age int
}
func main() {
var dogs []Dog
rDogs := reflect.ValueOf(dogs)
fmt.Println(rDogs.Kind())
}
Output:
slice
However, when a slice is stored inside a pointer, reflect.Kind() identifies it as a struct:
package main
import (
"fmt"
"reflect"
)
type Dog struct {
Name string
Age int
}
func main() {
dogs1 := make([]Dog, 0)
pointer := &dogs1
printPointerValueKind(pointer)
var dogs2 []Dog
pointer = &dogs2
printPointerValueKind(pointer)
}
func printPointerValueKind(pointer interface{}) {
if pointer != nil {
rPointer := reflect.ValueOf(pointer)
if rPointer.Kind() == reflect.Ptr {
value := rPointer.Elem()
rValue := reflect.ValueOf(value)
fmt.Println(rValue.Kind())
}
}
}
Output:
struct
struct
My questions are:
Why does this happen?
Is there a way to store a slice inside a pointer and still have reflect.Kind() identify it as a slice?
1. (reflect.Value).Elem() returns a reflect.Value, which is a struct, and which does not need to be passed to reflect.ValueOf again if you're looking for the kind of the type that the value holds.
2. rPointer.Elem().Kind() should be enough for what you need.
I noticed, quite by accident, that I can successfully pass both a pointer to a struct, and a pointer to a pointer to a struct to json.Unmarshal(), and both work just fine:
package main
import (
"testing"
"encoding/json"
)
type Person struct {
Name string
Age int
}
func TestMarshaling(t *testing.T) {
foo := &Person{Name: "bob", Age: 23}
// marshal it to bytes
b, err := json.Marshal(foo)
if err != nil {
t.Error(err)
}
bar := &Person{} // pointer to new, empty struct
err = json.Unmarshal(b, bar) // unmarshal to bar, which is a *Person
if err != nil {
t.Error(err)
}
testBob(t, bar) // ok
bar = &Person{} // pointer to new, empty struct
err = json.Unmarshal(b, &bar) // wait a minute, passing in a **Person, yet it still works?
if err != nil {
t.Error(err)
}
testBob(t, bar) // ok
}
func testBob(t *testing.T, person *Person) {
if person.Name != "bob" || person.Age != 23 {
t.Error("not equal")
}
}
I was really surprised that the second one (unmarshal to **Person) worked.
What's going on in json.Unmarshal()? Is it dereferencing the pointers until it finds a struct?
The documentation offers:
To unmarshal JSON into a pointer, Unmarshal first handles the case of
the JSON being the JSON literal null. In that case, Unmarshal sets
the pointer to nil. Otherwise, Unmarshal unmarshals the JSON into the
value pointed at by the pointer
It seems to be doing a bit more than that. What's really going on?
Fleshing out my question more: how does it know to automatically dereference my pointer to a pointer? The documentation says it will unmarshal "into the value pointed at by the pointer". Since the value of my pointer is in fact another pointer, and has no Name/Age fields, I expected it to stop there.
To be clear: I'm not saying there's a bug or misfeature in Unmarshal(); I'm trying to satisfy my astonishment that it works at all when given a ptr-to-ptr, and avoid any potential pitfalls in my use of it.
The json package has no reason to "stop at a pointer", since a pointer means nothing in json. It has to keep walking the tree in order to find a value to write. Since the json package is going to allow unmarshaling the same value into Type or *Type, it stands to reason that it should be able to unmarshal that into **Type, which is also a valid type in Go.
For a example, if Person were defined using pointers to differentiate between nil and zero values, and you were unmarshaling into a slice of []*Person, the json package needs to follow those pointers, and allocate values if necessary. The same applies if a field in Person were defined as a **string.
type Person struct {
Name **string
Age *int
}
type People []*Person
http://play.golang.org/p/vLq0nJPG5M
The json.Unmarshal implementation takes multiple indirection into account. Check the source here, in particular the decodeState.indirect method:
// indirect walks down v allocating pointers as needed,
// until it gets to a non-pointer.
// if it encounters an Unmarshaler, indirect stops and returns that.
// if decodingNull is true, indirect stops at the last pointer so it can be set to nil.
func (d *decodeState) indirect(v reflect.Value, decodingNull bool) (Unmarshaler, encoding.TextUnmarshaler, reflect.Value) {
// If v is a named type and is addressable,
// start with its address, so that if the type has pointer methods,
// we find them.
if v.Kind() != reflect.Ptr && v.Type().Name() != "" && v.CanAddr() {
v = v.Addr()
}
for {
if v.Kind() == reflect.Interface && !v.IsNil() {
e := v.Elem()
if e.Kind() == reflect.Ptr && !e.IsNil() && (!decodingNull || e.Elem().Kind() == reflect.Ptr) {
v = e
continue
}
}
if v.Kind() != reflect.Ptr {
break
}
//and so on
}
return nil, nil, v
The same method is called when unmarshaling arrays:
func (d *decodeState) array(v reflect.Value) {
u, ut, pv := d.indirect(v, false)
//...
That would have me believe that go can handle double indirection just fine. If nothing else, the json package source is a great example of what the reflect package is all about.
In short, values are checked, if the decoder is dealing with pointers, it will use reflection to work out how many levels of indirection there are, and determine what type the target has/is. The place to start from in the decode source is this: func (d *decodeState) unmarshal(v interface{}) (err error) {, from that point on, it's pretty self-explanatory.
As other answers have said, pointers are followed.
A little weird that this errors (nil pointer), but makes sense when you think about it.
package main
import (
"encoding/json"
"fmt"
"log"
)
type MyStruct struct {
A string `json:"a"`
}
func main() {
data := []byte(`{"a":"foo"}`)
var a *MyStruct
err := json.Unmarshal(data, a) // nil ptr
if err != nil {
log.Fatal(err)
}
fmt.Println(a)
}
But this doesn't error (pointer to nil pointer).
package main
import (
"encoding/json"
"fmt"
"log"
)
type MyStruct struct {
A string `json:"a"`
}
func main() {
data := []byte(`{"a":"foo"}`)
var a *MyStruct
err := json.Unmarshal(data, &a) // **MyStruct, ptr to nil ptr
if err != nil {
log.Fatal(err)
}
fmt.Println(a)
}
https://play.golang.org/p/eI8jqWZOmGW
I want to convert a struct to map in Golang. It would also be nice if I could use the JSON tags as keys in the created map (otherwise defaulting to field name).
Edit Dec 14, 2020
Since structs repo was archived, you can use mapstructure instead.
Edit TL;DR version, Jun 15, 2015
If you want the fast solution for converting a structure to map, see the accepted answer, upvote it and use that package.
Happy coding! :)
Original Post
So far I have this function, I am using the reflect package but I don't understand well how to use the package, please bear with me.
func ConvertToMap(model interface{}) bson.M {
ret := bson.M{}
modelReflect := reflect.ValueOf(model)
if modelReflect.Kind() == reflect.Ptr {
modelReflect = modelReflect.Elem()
}
modelRefType := modelReflect.Type()
fieldsCount := modelReflect.NumField()
var fieldData interface{}
for i := 0; i < fieldsCount; i++ {
field := modelReflect.Field(i)
switch field.Kind() {
case reflect.Struct:
fallthrough
case reflect.Ptr:
fieldData = ConvertToMap(field.Interface())
default:
fieldData = field.Interface()
}
ret[modelRefType.Field(i).Name] = fieldData
}
return ret
}
Also I looked at JSON package source code, because it should contain my needed implementation (or parts of it) but don't understand too much.
I also had need for something like this. I was using an internal package which was converting a struct to a map. I decided to open source it with other struct based high level functions. Have a look:
https://github.com/fatih/structs
It has support for:
Convert struct to a map
Extract the fields of a struct to a []string
Extract the values of a struct to a []values
Check if a struct is initialized or not
Check if a passed interface is a struct or a pointer to struct
You can see some examples here: http://godoc.org/github.com/fatih/structs#pkg-examples
For example converting a struct to a map is a simple:
type Server struct {
Name string
ID int32
Enabled bool
}
s := &Server{
Name: "gopher",
ID: 123456,
Enabled: true,
}
// => {"Name":"gopher", "ID":123456, "Enabled":true}
m := structs.Map(s)
The structs package has support for anonymous (embedded) fields and nested structs. The package provides to filter certain fields via field tags.
From struct to map[string]interface{}
package main
import (
"fmt"
"encoding/json"
)
type MyData struct {
One int
Two string
Three int
}
func main() {
in := &MyData{One: 1, Two: "second"}
var inInterface map[string]interface{}
inrec, _ := json.Marshal(in)
json.Unmarshal(inrec, &inInterface)
// iterate through inrecs
for field, val := range inInterface {
fmt.Println("KV Pair: ", field, val)
}
}
go playground here
Here is a function I've written in the past to convert a struct to a map, using tags as keys
// ToMap converts a struct to a map using the struct's tags.
//
// ToMap uses tags on struct fields to decide which fields to add to the
// returned map.
func ToMap(in interface{}, tag string) (map[string]interface{}, error){
out := make(map[string]interface{})
v := reflect.ValueOf(in)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
// we only accept structs
if v.Kind() != reflect.Struct {
return nil, fmt.Errorf("ToMap only accepts structs; got %T", v)
}
typ := v.Type()
for i := 0; i < v.NumField(); i++ {
// gets us a StructField
fi := typ.Field(i)
if tagv := fi.Tag.Get(tag); tagv != "" {
// set key of map to value in struct field
out[tagv] = v.Field(i).Interface()
}
}
return out, nil
}
Runnable example here.
Note, if you have multiple fields with the same tag value, then you will obviously not be able to store them all within a map. It might be prudent to return an error if that happens.
I like the importable package for the accepted answer, but it does not translate my json aliases. Most of my projects have a helper function/class that I import.
Here is a function that solves my specific problem.
// Converts a struct to a map while maintaining the json alias as keys
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj) // Convert to a json string
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
And in the main, this is how it would be called...
package main
import (
"fmt"
"encoding/json"
"github.com/fatih/structs"
)
type MyStructObject struct {
Email string `json:"email_address"`
}
func main() {
obj := &MyStructObject{Email: "test#test.com"}
// My solution
fmt.Println(StructToMap(obj)) // prints {"email_address": "test#test.com"}
// The currently accepted solution
fmt.Println(structs.Map(obj)) // prints {"Email": "test#test.com"}
}
package main
import (
"fmt"
"reflect"
)
type bill struct {
N1 int
N2 string
n3 string
}
func main() {
a := bill{4, "dhfthf", "fdgdf"}
v := reflect.ValueOf(a)
values := make(map[string]interface{}, v.NumField())
for i := 0; i < v.NumField(); i++ {
if v.Field(i).CanInterface() {
values[v.Type().Field(i).Name] = v.Field(i).Interface()
} else {
fmt.Printf("sorry you have a unexported field (lower case) value you are trying to sneak past. I will not allow it: %v\n", v.Type().Field(i).Name)
}
}
fmt.Println(values)
passObject(&values)
}
func passObject(v1 *map[string]interface{}) {
fmt.Println("yoyo")
}
I'm a bit late but I needed this kind of feature so I wrote this. Can resolve nested structs. By default, uses field names but can also use custom tags. A side effect is that if you set the tagTitle const to json, you could use the json tags you already have.
package main
import (
"fmt"
"reflect"
)
func StructToMap(val interface{}) map[string]interface{} {
//The name of the tag you will use for fields of struct
const tagTitle = "kelvin"
var data map[string]interface{} = make(map[string]interface{})
varType := reflect.TypeOf(val)
if varType.Kind() != reflect.Struct {
// Provided value is not an interface, do what you will with that here
fmt.Println("Not a struct")
return nil
}
value := reflect.ValueOf(val)
for i := 0; i < varType.NumField(); i++ {
if !value.Field(i).CanInterface() {
//Skip unexported fields
continue
}
tag, ok := varType.Field(i).Tag.Lookup(tagTitle)
var fieldName string
if ok && len(tag) > 0 {
fieldName = tag
} else {
fieldName = varType.Field(i).Name
}
if varType.Field(i).Type.Kind() != reflect.Struct {
data[fieldName] = value.Field(i).Interface()
} else {
data[fieldName] = StructToMap(value.Field(i).Interface())
}
}
return data
}
map := Structpb.AsMap()
// map is the map[string]interface{}