I'm trying to do something like this:
Define structs with tags named env:
type Env struct {
Port string `env:"PORT"`
}
Call some function which will get the environment variable names using os.Getenv and put set it in the struct.
Right now, I have this:
package main
import (
"fmt"
"os"
"reflect"
)
func ParseEnv(t interface{}, v interface{}) {
it := reflect.TypeOf(t)
for i := 0; i < it.NumField(); i++ {
field := it.Field(i)
value := os.Getenv(field.Tag.Get("env"))
if value == "" {
continue
}
reflect.ValueOf(v).Elem().FieldByName(field.Name).SetString(value)
}
}
type Env struct {
Port string `env:"PORT"`
DatabaseURL string `env:"DATABASE_URL"`
}
func main() {
os.Setenv("PORT", "8080")
os.Setenv("DATABASE_URL", "postgres://user:pass#host:5432/my-db")
env := Env{}
ParseEnv(env, &env)
fmt.Println(env)
}
http://play.golang.org/p/b8uPPVo4aV
But, as you can see, I have to pass both the reference and the pointer to my function.
While this works, it is very ugly (at least I think it is).
If I try to pass the pointer only, I can't get the type right (because it will be an *interface{}) and, if I pass only the reference, I can't set the values using reflect (even if I could, it would not work).
Is there a sane way of doing this?
Below is a "saner" way of doing what you want. You will notice that, instead of passing in two copies of the struct, we only need a pointer to the struct.
func ParseEnv(val interface{}) {
ptrRef := reflect.ValueOf(val)
if ptrRef.Kind() != reflect.Ptr {
panic("pointer to struct expected")
}
ref := ptrRef.Elem()
if ref.Kind() != reflect.Struct {
panic("pointer to struct expected")
}
refType := ref.Type()
for i := 0; i < refType.NumField(); i++ {
field := refType.Field(i)
value := os.Getenv(field.Tag.Get("env"))
if value == "" {
continue
}
ref.Field(i).SetString(value)
}
}
The above function should be invoked in the following way:
ParseEnv(&env)
Example: https://play.golang.org/p/_BwWz2oUql
Related
Given this struct
type A struct {
b *B
c string
d string
}
I want to check with reflect which fields have a struct behind the pointer type, so specifically I want to create a condition that would only be true when iterating over field b. From what I've tried using the reflect documentation I always end up with an invalid Value kind which doesn't allow me to go further as every subsequent method panics.
package main
import (
"fmt"
"reflect"
)
type A struct {
b *B
c string
d string
}
type B struct {
}
func main() {
val := reflect.ValueOf(A{})
for i := 0; i < val.Type().NumField(); i++ {
if val.Field(i).Kind() == reflect.Ptr {
fmt.Println(reflect.Indirect(val.Field(i)).Kind())
}
fmt.Println(val.Field(i).Kind())
}
}
https://play.golang.org/p/oRry3ZubRxI
You get invalid value, because the val.b pointer field is nil, and you can't dereference a nil pointer. If you want your code to work, you have to initialize it with a valid pointer:
val := reflect.ValueOf(A{b: &B{}})
With this change it works and outputs (try it on the Go Playground):
struct
ptr
string
If you want it to work without having to initialize the pointer, then you have to work on the types and not values:
val := reflect.ValueOf(A{})
t := val.Type()
for i := 0; i < t.NumField(); i++ {
if ft := t.Field(i).Type; ft.Kind() == reflect.Ptr {
fmt.Println(ft.Elem().Kind())
}
fmt.Println(t.Field(i).Type.Kind())
}
This outputs the same, try this one on the Go Playground.
I just can't find the way to get a slice of pointer to each attribute of a given struct. I am using reflection to get my pointers (Thanks to https://stackoverflow.com/a/24348352/6093604)
if valueField.CanAddr() {
address = fmt.Sprintf("0x%X", valueField.Addr().Pointer())
}
As you can see, valueField.Addr().Pointer() returns a pointer addr value, however, using reflection, I would like to get a usable pointer for sql.Rows.Scan()
So what did I do is:
func StructAttributesToPointersSlice(object interface{}) []interface{} {
val := reflect.ValueOf(object)
if val.Kind() == reflect.Interface && !val.IsNil() {
elm := val.Elem()
if elm.Kind() == reflect.Ptr && !elm.IsNil() && elm.Elem().Kind() == reflect.Ptr {
val = elm
}
}
if val.Kind() == reflect.Ptr {
val = val.Elem()
}
var ptrs []interface{}
for i := 0; i < val.NumField(); i++ {
valueField := val.Field(i)
if valueField.Kind() == reflect.Ptr {
valueField = valueField.Elem()
}
if valueField.CanAddr() {
ptrs = append(ptrs, valueField.Addr().Pointer())
}
}
return ptrs
}
But when I try to use it for Scan() sql function:
var values []interface{}
// Iterate over each rows got from the query
for rows.Next() {
ptrs := utils.StructAttributesToPointersSlice(&newObject)
for _, item := range ptrs {
fmt.Println(reflect.TypeOf(item))
}
err = rows.Scan(ptrs...)
if err != nil {
return nil, model.Status{Code: http.StatusInternalServerError, Error: err.Error()}
} else {
values = append(values, newObject)
}
}
I am getting this error:
sql: Scan error on column index 0: destination not a pointer
I know it's because it's not the good type since it's a uintptr, but then how to transform it into usable pointer?
Thanks
Use unsafe.Pointer to convert a uintptr to a pointer of some type. As an example, the following expression converts uintptr u to a T pointer:
(*T)(unsafe.Pointer(u))
This conversion does not help in StructAttributesToPointersSlice because the struct fields can be of any type. Also, the conversion from uintptr is not needed and unsafe.
The expression valueField.Addr() is the reflect.Value for the pointer to the field. Call Interface() to get the actual pointer. To fix the program, change
ptrs = append(ptrs, valueField.Addr().Pointer())
to
ptrs = append(ptrs, valueField.Addr().Interface())
Here's a simplified version of the function:
func StructAttributesToPointersSlice(object interface{}) []interface{} {
v := reflect.ValueOf(object)
if v.Kind() != reflect.Ptr || v.Elem().Kind() != reflect.Struct {
panic("argument must be a pointer to struct")
}
v = v.Elem()
var result []interface{}
for i := 0; i < v.NumField(); i++ {
f := v.Field(i)
if !f.CanSet() {
continue
}
result = append(result, f.Addr().Interface())
}
return result
}
Some notes about this code:
The argument must be a pointer to a struct.
There's no need to call CanAddr on the fields. The check for pointer to struct covers this.
The CanSet() skips over unexported fields. You may want it to panic instead.
The function panics for errors in the caller. Consider returning an error instead.
playground example
I have a struct and I am trying to iterate through all fields to determine whether or not the field is a map (or a pointer to a map). I'm able to compare field kind to reflect.Map for non-pointer fields, but I am having trouble doing the same for pointer fields. If the field has a value I can use Elem() to determine the pointer field type, but if the field is nil that method won't work and I get "invalid." Any ideas on how to accomplish this? Playground is here.
package main
import (
"fmt"
"reflect"
)
type Item struct {
itemMap map[string]interface{}
itemMapPtr *map[string]interface{}
}
func main() {
item := new(Item)
printTypes(*item)
}
func printTypes(item Item) {
itemVal := reflect.ValueOf(item)
for i := 0; i < itemVal.NumField(); i++ {
fieldVal := itemVal.Field(i)
if fieldVal.Kind() == reflect.Ptr {
fieldVal = fieldVal.Elem() // This is only helpful if the field is not nil
}
fmt.Println(fieldVal.Kind())
}
}
Work with types instead of values:
t := reflect.TypeOf(item)
for i := 0; i < t.NumField(); i++ {
ft := t.Field(i).Type
if ft.Kind() == reflect.Ptr {
ft = ft.Elem()
}
fmt.Println(ft.Kind())
}
playground example
I want to convert a struct to map in Golang. It would also be nice if I could use the JSON tags as keys in the created map (otherwise defaulting to field name).
Edit Dec 14, 2020
Since structs repo was archived, you can use mapstructure instead.
Edit TL;DR version, Jun 15, 2015
If you want the fast solution for converting a structure to map, see the accepted answer, upvote it and use that package.
Happy coding! :)
Original Post
So far I have this function, I am using the reflect package but I don't understand well how to use the package, please bear with me.
func ConvertToMap(model interface{}) bson.M {
ret := bson.M{}
modelReflect := reflect.ValueOf(model)
if modelReflect.Kind() == reflect.Ptr {
modelReflect = modelReflect.Elem()
}
modelRefType := modelReflect.Type()
fieldsCount := modelReflect.NumField()
var fieldData interface{}
for i := 0; i < fieldsCount; i++ {
field := modelReflect.Field(i)
switch field.Kind() {
case reflect.Struct:
fallthrough
case reflect.Ptr:
fieldData = ConvertToMap(field.Interface())
default:
fieldData = field.Interface()
}
ret[modelRefType.Field(i).Name] = fieldData
}
return ret
}
Also I looked at JSON package source code, because it should contain my needed implementation (or parts of it) but don't understand too much.
I also had need for something like this. I was using an internal package which was converting a struct to a map. I decided to open source it with other struct based high level functions. Have a look:
https://github.com/fatih/structs
It has support for:
Convert struct to a map
Extract the fields of a struct to a []string
Extract the values of a struct to a []values
Check if a struct is initialized or not
Check if a passed interface is a struct or a pointer to struct
You can see some examples here: http://godoc.org/github.com/fatih/structs#pkg-examples
For example converting a struct to a map is a simple:
type Server struct {
Name string
ID int32
Enabled bool
}
s := &Server{
Name: "gopher",
ID: 123456,
Enabled: true,
}
// => {"Name":"gopher", "ID":123456, "Enabled":true}
m := structs.Map(s)
The structs package has support for anonymous (embedded) fields and nested structs. The package provides to filter certain fields via field tags.
From struct to map[string]interface{}
package main
import (
"fmt"
"encoding/json"
)
type MyData struct {
One int
Two string
Three int
}
func main() {
in := &MyData{One: 1, Two: "second"}
var inInterface map[string]interface{}
inrec, _ := json.Marshal(in)
json.Unmarshal(inrec, &inInterface)
// iterate through inrecs
for field, val := range inInterface {
fmt.Println("KV Pair: ", field, val)
}
}
go playground here
Here is a function I've written in the past to convert a struct to a map, using tags as keys
// ToMap converts a struct to a map using the struct's tags.
//
// ToMap uses tags on struct fields to decide which fields to add to the
// returned map.
func ToMap(in interface{}, tag string) (map[string]interface{}, error){
out := make(map[string]interface{})
v := reflect.ValueOf(in)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
// we only accept structs
if v.Kind() != reflect.Struct {
return nil, fmt.Errorf("ToMap only accepts structs; got %T", v)
}
typ := v.Type()
for i := 0; i < v.NumField(); i++ {
// gets us a StructField
fi := typ.Field(i)
if tagv := fi.Tag.Get(tag); tagv != "" {
// set key of map to value in struct field
out[tagv] = v.Field(i).Interface()
}
}
return out, nil
}
Runnable example here.
Note, if you have multiple fields with the same tag value, then you will obviously not be able to store them all within a map. It might be prudent to return an error if that happens.
I like the importable package for the accepted answer, but it does not translate my json aliases. Most of my projects have a helper function/class that I import.
Here is a function that solves my specific problem.
// Converts a struct to a map while maintaining the json alias as keys
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj) // Convert to a json string
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
And in the main, this is how it would be called...
package main
import (
"fmt"
"encoding/json"
"github.com/fatih/structs"
)
type MyStructObject struct {
Email string `json:"email_address"`
}
func main() {
obj := &MyStructObject{Email: "test#test.com"}
// My solution
fmt.Println(StructToMap(obj)) // prints {"email_address": "test#test.com"}
// The currently accepted solution
fmt.Println(structs.Map(obj)) // prints {"Email": "test#test.com"}
}
package main
import (
"fmt"
"reflect"
)
type bill struct {
N1 int
N2 string
n3 string
}
func main() {
a := bill{4, "dhfthf", "fdgdf"}
v := reflect.ValueOf(a)
values := make(map[string]interface{}, v.NumField())
for i := 0; i < v.NumField(); i++ {
if v.Field(i).CanInterface() {
values[v.Type().Field(i).Name] = v.Field(i).Interface()
} else {
fmt.Printf("sorry you have a unexported field (lower case) value you are trying to sneak past. I will not allow it: %v\n", v.Type().Field(i).Name)
}
}
fmt.Println(values)
passObject(&values)
}
func passObject(v1 *map[string]interface{}) {
fmt.Println("yoyo")
}
I'm a bit late but I needed this kind of feature so I wrote this. Can resolve nested structs. By default, uses field names but can also use custom tags. A side effect is that if you set the tagTitle const to json, you could use the json tags you already have.
package main
import (
"fmt"
"reflect"
)
func StructToMap(val interface{}) map[string]interface{} {
//The name of the tag you will use for fields of struct
const tagTitle = "kelvin"
var data map[string]interface{} = make(map[string]interface{})
varType := reflect.TypeOf(val)
if varType.Kind() != reflect.Struct {
// Provided value is not an interface, do what you will with that here
fmt.Println("Not a struct")
return nil
}
value := reflect.ValueOf(val)
for i := 0; i < varType.NumField(); i++ {
if !value.Field(i).CanInterface() {
//Skip unexported fields
continue
}
tag, ok := varType.Field(i).Tag.Lookup(tagTitle)
var fieldName string
if ok && len(tag) > 0 {
fieldName = tag
} else {
fieldName = varType.Field(i).Name
}
if varType.Field(i).Type.Kind() != reflect.Struct {
data[fieldName] = value.Field(i).Interface()
} else {
data[fieldName] = StructToMap(value.Field(i).Interface())
}
}
return data
}
map := Structpb.AsMap()
// map is the map[string]interface{}
I have the following query builder function:
func CreateQuery(t interface{}, where string) {
var b bytes.Buffer
b.WriteString("SELECT ")
s := reflect.ValueOf(t).Elem()
typeOfT := s.Type()
for i := 0; i < s.NumField() - 1; i++ {
b.WriteString(fmt.Sprintf("%s, ", typeOfT.Field(i).Name))
}
//Last one has no Comma
b.WriteString(fmt.Sprintf("%s ", typeOfT.Field(s.NumField() - 1).Name))
b.WriteString(fmt.Sprintf("FROM %s ", typeOfT.Name()))
b.WriteString(where)
fmt.Println(b.String())
}
There works fine when called as follows:
var dst FooStruct
CreateQuery(&dst, "")
But the following raises a "call of reflect.Value.NumField on slice Value" Panic:
var dst []FooStruct
CreateQuery(&dst, "")
How I can I make the function print the fields of a slice's underlying struct type? It seems like I want the inverse of reflect's SliceOf function.
You can only call NumField or Field methods on a reflect.Type representing a struct (i.e. t.Kind() == reflect.Struct).
If you have a slice type, you can access the contained type via the Elem method, which returns another reflect.Type. If the slice contains a struct, then you can call NumField/Field on this type.
You can iterate over the slice, calling CreateQuery for every query:
func CreateQueries(t interface{}, where string) {
v := reflect.ValueOf(t)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
if v.Kind() == reflect.Array || v.Kind() == reflect.Slice {
for i := 0; i < v.Len(); i++ {
CreateQuery(v.Index(i).Interface(), where)
}
}
}
Using reflect.Value.Index you can access each field separately, calling .Interface() on the
value yields the interface{} type representation of that value, making it suitable to put
it in your CreateQuery function (which expects a interface{} value).