Reflection - restore time.Time instance - reflection

I am developing a program in go that needs to store and retrieve an array of custom structure instances using the sessions package of the Gorilla toolkit. For restoring the custom structure I need to make use of reflection features. The issue is that my structure named Timestamp includes two time.Time instances and I have not been able to restore the instances. Thus, my question is how to restore a time.Time instance.
Below you can see my code for the Timespan structure as well as the code for storing and reading the Timespan array in the session store.
type Timespan struct {
ID uint8;
StartDate time.Time;
EndDate time.Time;
}
func (server *WebServer) setTimespans(writer http.ResponseWriter, request *http.Request, timespans [model.TimespanCount]*model.Timespan) error {
var session *sessions.Session;
var sessionDecodingException error;
session, sessionDecodingException = server.SessionStore.Get(request, authenticationSessionName);
if sessionDecodingException != nil {
return sessionDecodingException;
}
session.Values[sessionTimestamps] = timespans;
return nil;
}
func (server *WebServer) getTimespans(request *http.Request) ([model.TimespanCount]*model.Timespan, error) {
var session *sessions.Session;
var sessionDecodingException error;
session, sessionDecodingException = server.SessionStore.Get(request, authenticationSessionName);
var readTimespans [model.TimespanCount]*model.Timespan;
if sessionDecodingException != nil {
return readTimespans, sessionDecodingException;
}
interfaceValue := reflect.ValueOf(session.Values[sessionTimestamps]);
var actuallyAddedTimespan *model.Timespan;
for counter := 0; counter < model.TimespanCount; counter++ {
actuallyAddedTimespan = &model.Timespan{};
actuallyReflectedTimespan := interfaceValue.Index(counter).Elem();
actuallyAddedTimespan.ID = uint8(actuallyReflectedTimespan.FieldByName("ID").Uint());
//actuallyAddedTimespan.StartDate = actuallyReflectedTimespan.FieldByName("StartDate");
//actuallyAddedTimespan.EndDate = actuallyReflectedTimespan.FieldByName("EndDate");
fmt.Println(actuallyAddedTimespan);
}
return readTimespans, nil;
}

You need to get the interface to the field:
actuallyAddedTimespan.StartDate = actuallyReflectedTimespan.FieldByName("StartDate").Interface().(time.Time)
playground
Personal opinion time, using reflection for this rather than using a simple interface is:
Slow.
Inefficient
Can break easily if you change how your struct looks and forget to update the reflection code.
Example of using an interface:
func main() {
ts := &Timespan{ID: 102, StartDate: time.Now().AddDate(6, 0, 0), EndDate: time.Now().AddDate(8, 0, 0)}
m := map[string]interface{}{
"key": ts,
}
switch v := m["key"].(type) {
case Timespaner:
fmt.Println(v.Value())
default:
fmt.Println("wtfmate?")
}
}
func (ts *Timespan) Value() (id uint8, start, end time.Time) {
return ts.ID, ts.StartDate, ts.EndDate
}
type Timespaner interface {
Value() (id uint8, start, end time.Time)
}
playground

You can use the Interface() function to get the interface{} value from a struct. Then, you can use a type assertion to get the correct type back:
func main() {
t := &Timespan{42, time.Now(), time.Now()}
reflectedPointer := reflect.ValueOf(t)
reflectedTimespan := reflectedPointer.Elem()
var timespan Timespan = reflectedTimespan.Interface().(Timespan)
fmt.Println(*t)
fmt.Println(timespan)
}

Related

Race condition while writing and reading from the map

Following up on old post here.
I am iterating over flatProduct.Catalogs slice and populating my productCatalog concurrent map in golang. I am using upsert method so that I can add only unique productID's into my productCatalog map.
Below code is called by multiple go routines in parallel that is why I am using concurrent map here to populate data into it. This code runs in background to populate data in the concurrent map every 30 seconds.
var productRows []ClientProduct
err = json.Unmarshal(byteSlice, &productRows)
if err != nil {
return err
}
for i := range productRows {
flatProduct, err := r.Convert(spn, productRows[i])
if err != nil {
return err
}
if flatProduct.StatusCode == definitions.DONE {
continue
}
r.products.Set(strconv.Itoa(flatProduct.ProductId, 10), flatProduct)
for _, catalogId := range flatProduct.Catalogs {
catalogValue := strconv.FormatInt(int64(catalogId), 10)
r.productCatalog.Upsert(catalogValue, flatProduct.ProductId, func(exists bool, valueInMap interface{}, newValue interface{}) interface{} {
productID := newValue.(int64)
if valueInMap == nil {
return map[int64]struct{}{productID: {}}
}
oldIDs := valueInMap.(map[int64]struct{})
// value is irrelevant, no need to check if key exists
// I think problem is here
oldIDs[productID] = struct{}{}
return oldIDs
})
}
}
And below are my getters in the same class where above code is there. These getters are used by main application threads to get data from the map or get the whole map.
func (r *clientRepository) GetProductMap() *cmap.ConcurrentMap {
return r.products
}
func (r *clientRepository) GetProductCatalogMap() *cmap.ConcurrentMap {
return r.productCatalog
}
func (r *clientRepository) GetProductData(pid string) *definitions.FlatProduct {
pd, ok := r.products.Get(pid)
if ok {
return pd.(*definitions.FlatProduct)
}
return nil
}
This is how I am reading data from this productCatalog cmap but my system is crashing on the below range statement -
// get productCatalog map which was populated above
catalogProductMap := clientRepo.GetProductCatalogMap()
productIds, ok := catalogProductMap.Get("211")
data, _ := productIds.(map[int64]struct{})
// I get panic here after sometime
for _, pid := range data {
...
}
Error I am getting as - fatal error: concurrent map iteration and map write.
I think issue is r.productCatalog is a concurrentmap, but oldIDs[productID] is a normal map which is causing issues while I am iterating in the for loop above.
How can I fix this race issue I am seeing? One way I can think of is making oldIDs[productID] as concurrent map but if I do that approach then my memory increase by a lot and eventually goes OOM. Below is what I have tried which works and it solves the race condition but it increases the memory by a lot which is not what I want -
r.productCatalog.Upsert(catalogValue, flatProduct.ProductId, func(exists bool, valueInMap interface{}, newValue interface{}) interface{} {
productID := newValue.(int64)
if valueInMap == nil {
// return map[int64]struct{}{productID: {}}
return cmap.New()
}
// oldIDs := valueInMap.(map[int64]struct{})
oldIDs := valueInMap.(cmap.ConcurrentMap)
// value is irrelevant, no need to check if key exists
// oldIDs[productID] = struct{}{}
oldIDs.Set(strconv.FormatInt(productID, 10), struct{}{})
return oldIDs
})
Any other approach I can do which doesn't increase memory and also fixes the race condition I am seeing?
Note
I am still using v1 version of cmap without generics and it deals with strings as keys.
Rather than a plain map[int64]struct{} type, you could define a struct which holds the map and a mutex to control the access to the map:
type myMap struct{
m sync.Mutex
data map[int64]struct{}
}
func (m *myMap) Add(productID int64) {
m.m.Lock()
defer m.m.Unlock()
m.data[productID] = struct{}{}
}
func (m *myMap) List() []int64 {
m.m.Lock()
defer m.m.Unlock()
var res []int64
for id := range m.data {
res = append(res, id)
}
// sort slice if you need
return res
}
With the sample implementation above, you would have to be careful to store *myMap pointers (as opposed to plain myMap structs) in your cmap.ConcurrentMap structure.

Golang Invalid Receiver Type in Method Func

I'm trying to make a simple package to send SSH commands to a server.
I have the following code:
type Connection *ssh.Client
func Connect(addr, user, password string) (conn Connection, err error) {
sshConfig := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{
ssh.Password(password),
},
HostKeyCallback: ssh.HostKeyCallback(func(hostname string, remote net.Addr, key ssh.PublicKey) error { return nil }),
}
conn, err = ssh.Dial("tcp", addr, sshConfig)
return
}
func (conn Connection) SendCommand() ([]byte, error) {
session, err := (*ssh.Client)(conn).NewSession()
// ...
}
My problem is on the two lines func (conn Connection) SendCommand() ([]byte, error) and session, err := (*ssh.Client)(conn).NewSession().
I can't figure out how to use the methods available for *ssh.Client from my overlaying Connection type.
I understand that I need to do some conversion, and using ssh.Client(*conn).NewSession() would work, but it copies the values of the *ssh.Client which doesn't seem to be the right method.
What should do to access the methods available for a *ssh.Client when working with my custom type Connection *ssh.Client type?
You can't declare a new type with a pointer TypeSpec. Also declaring a new type is used specifically to remove the entire method set, so you won't have any of the original methods from the *ssh.Client.
What you want is to use composition by embedding the *ssh.Client in your own struct type:
type Connection struct {
*ssh.Client
}
func Connect(addr, user, password string) (*Connection, error) {
sshConfig := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{
ssh.Password(password),
},
HostKeyCallback: ssh.HostKeyCallback(func(hostname string, remote net.Addr, key ssh.PublicKey) error { return nil }),
}
conn, err = ssh.Dial("tcp", addr, sshConfig)
if err != nil {
return nil, err
}
return &Connection{conn}, nil
}
func (conn *Connection) SendCommand() ([]byte, error) {
session, err := conn.NewSession()
// ...
}
This is the best I can come up with:
type Connection ssh.Client
func (conn *Connection) SendCommand() ([]byte, error) {
(*ssh.Client)(conn).NewSession()
Note that I've changed the type to not be a pointer type (but then I've made a pointer receiver for SendCommand). I'm not sure there's any way to create a function with a pointer type as a receiver.
Another option is to use type aliasing to achieve the desired behavior. I was trying to do something "clever" for readability:
type foo struct {
i int
}
type foo_ptr = *foo
type foo_ptr_slice = []foo_ptr
type foo_ptr_map = map[string]foo_ptr
type foo_ptr_slice_map = map[string]foo_ptr_slice
func (r foo_ptr) dump() {
fmt.Printf("%d\n", r.i)
}
func main() {
// need a map of slice of pointers
var m foo_ptr_map
m = make(foo_ptr_map, 0)
m["test"] = &foo{i: 1}
var m2 foo_ptr_slice_map
m2 = make(foo_ptr_slice_map, 0)
m2["test"] = make(foo_ptr_slice, 0, 10)
m2["test"] = append(m2["test"], &foo{i: 2})
fmt.Printf("%d\n", m["test"].i)
fmt.Printf("%d\n", m2["test"][0].i)
m["test"].dump()
}
I acknowledge that type aliasing is used for large-scale refactoring but this seems like a very good use for readability sake.

Display Foreign Keys of Foreign Keys in Go GORM

I was able to partially solve this with this case
Unfortunately, the Preload() function doesn't seem to work with delving further down in the related object set.
To clarify, I have the following models:
type Room struct {
gorm.Model
Name string
Games []Game `gorm:"ForeignKey:RoomID"`
}
type Game struct {
gorm.Model
RoomID int `gorm:"index"`
Players []Player `gorm:"ForeignKey:GameID"`
}
type Player struct {
gorm.Model
Name string
GameID int `gorm:"index"`
}
When I create a new object with a new Room, Game, and Player object created I get the following data returned (json encoded):
{"Value":{"ID":26,"CreatedAt":"2016-05-15T01:21:22.426234189-07:00","UpdatedAt":"2016-05-15T01:21:22.426234189-07:00","DeletedAt":null,"Name":"foo","Games":[{"ID":17,"CreatedAt":"2016-05-15T01:21:22.427026134-07:00","UpdatedAt":"2016-05-15T01:21:22.427026134-07:00","DeletedAt":null,"RoomID":26,"Turns":null,"Players":[{"ID":4,"CreatedAt":"2016-05-15T01:21:22.427560561-07:00","UpdatedAt":"2016-05-15T01:21:22.427560561-07:00","DeletedAt":null,"Name":"TestPlayer","GameID":17}],"Counter":1,"Assigned":false}],"Testing":false},"Error":null,"RowsAffected":1}
But if I try to query the structure with a preload() function, I get the following:
{"Value":{"ID":26,"CreatedAt":"2016-05-15T01:21:22.426234189-07:00","UpdatedAt":"2016-05-15T01:21:22.426234189-07:00","DeletedAt":null,"Name":"foo","Games":[{"ID":17,"CreatedAt":"2016-05-15T01:21:22.427026134-07:00","UpdatedAt":"2016-05-15T01:21:22.427026134-07:00","DeletedAt":null,"RoomID":26,"Turns":null,"Players":null,"Counter":1,"Assigned":false}],"Testing":false},"Error":null,"RowsAffected":1}
Note that the Players section is now null. Here's my room registration and room query functions:
func RegisterRoom(w http.ResponseWriter, r *http.Request) {
err := r.ParseForm()
if err != nil {
// Handle error
}
// r.PostForm is a map of our POST form values
room := Room{
Name: r.PostFormValue("label"),
Games: []Game{{
Counter: 1,
Players: []Player{{Name: r.PostFormValue("username")}},
}},
}
t := db.Create(&room)
if err := json.NewEncoder(w).Encode(t); err != nil {
panic(err)
}
}
func RoomShow(w http.ResponseWriter, r *http.Request) {
var rm Room
vars := mux.Vars(r)
roomId := vars["roomId"]
id, _ := strconv.Atoi(roomId)
room := db.Preload("Games").First(&rm, id)
result := db.Find(&room)
json.NewEncoder(w).Encode(result)
}
Solved it! Answer was under Nested Preloading smacks forehead

Discover the return type of the current function in go

I have a function that is being generated using reflection and reflect.MakeFunc, so I don't actually have the return type until runtime.
Inside the template function that MakeFunc is using, is there a way to determine the return type of the concrete function being templated?
Essentially, is there a way to determine the return type iof the currently executing function at runtime?
I know about the Out method:
fn.Type().Out(0)
And I can find the return type of a function easily enough?
But is there a way to find the return type of the currently executing function (as opposed to an explicit passed function reference).
You should check fn.Type().Out(0).Kind(), for example:
func main() {
fnTmpl := func(in []reflect.Value) []reflect.Value {
return []reflect.Value{in[0]}
}
makeFn := func(fptr interface{}) {
fn := reflect.ValueOf(fptr).Elem()
fn.Set(reflect.MakeFunc(fn.Type(), fnTmpl))
}
var nFn func(int) int
makeFn(&nFn)
kind := reflect.TypeOf(nFn).Out(0).Kind()
switch kind {
case reflect.Int:
fmt.Println("int")
}
}
In the case you are talking about, the return type of the currently executing function is always []reflect.Type (because that is what a function passed to reflect.MakeFunc must return). What you really want is the return type of the reflect.makeFuncStub function that called your function.
There is no way to get that (except perhaps some strange inspection of the call stack), but you can make an enhanced version of MakeFunc that provides the information:
package main
import (
"fmt"
"reflect"
)
// MakeFunc is like reflect.MakeFunc, but fn has an extra argument, retType, which
// is passed the desired return type.
func MakeFunc(typ reflect.Type, fn func(args []reflect.Value, retType reflect.Type) (results []reflect.Value)) reflect.Value {
if n := typ.NumOut(); n != 1 {
panic("wrong number of return values")
}
rt := typ.Out(0)
return reflect.MakeFunc(typ, func(args []reflect.Value) (results []reflect.Value) {
return fn(args, rt)
})
}
func makeReturnOne(fptr interface{}) {
fn := reflect.ValueOf(fptr).Elem()
fn.Set(MakeFunc(fn.Type(), returnOne))
}
func returnOne(args []reflect.Value, retType reflect.Type) []reflect.Value {
ret := reflect.New(retType).Elem()
switch retType.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
ret.SetInt(1)
case reflect.Float32, reflect.Float64:
ret.SetFloat(1.0)
default:
panic("returnOne only supports int and float types")
}
r := ret.Interface()
fmt.Printf("returning %v as %T\n", r, r)
return []reflect.Value{ret}
}
func main() {
var r1f func() float64
var r1i func() int
makeReturnOne(&r1f)
makeReturnOne(&r1i)
fmt.Println(r1f())
fmt.Println(r1i())
}
I might have misinterpreted what you are trying to achieve, but why not just take the kind of the value you are returning? Modifying OneOfOne's example as follows:
fnTmpl := func(in []reflect.Value) (res []reflect.Value) {
res = []reflect.Value{in[0]}
fmt.Println("Returned:", res[0].Kind())
return res
}
Playground: http://play.golang.org/p/EujmxyGRrI

function for converting a struct to map in Golang

I want to convert a struct to map in Golang. It would also be nice if I could use the JSON tags as keys in the created map (otherwise defaulting to field name).
Edit Dec 14, 2020
Since structs repo was archived, you can use mapstructure instead.
Edit TL;DR version, Jun 15, 2015
If you want the fast solution for converting a structure to map, see the accepted answer, upvote it and use that package.
Happy coding! :)
Original Post
So far I have this function, I am using the reflect package but I don't understand well how to use the package, please bear with me.
func ConvertToMap(model interface{}) bson.M {
ret := bson.M{}
modelReflect := reflect.ValueOf(model)
if modelReflect.Kind() == reflect.Ptr {
modelReflect = modelReflect.Elem()
}
modelRefType := modelReflect.Type()
fieldsCount := modelReflect.NumField()
var fieldData interface{}
for i := 0; i < fieldsCount; i++ {
field := modelReflect.Field(i)
switch field.Kind() {
case reflect.Struct:
fallthrough
case reflect.Ptr:
fieldData = ConvertToMap(field.Interface())
default:
fieldData = field.Interface()
}
ret[modelRefType.Field(i).Name] = fieldData
}
return ret
}
Also I looked at JSON package source code, because it should contain my needed implementation (or parts of it) but don't understand too much.
I also had need for something like this. I was using an internal package which was converting a struct to a map. I decided to open source it with other struct based high level functions. Have a look:
https://github.com/fatih/structs
It has support for:
Convert struct to a map
Extract the fields of a struct to a []string
Extract the values of a struct to a []values
Check if a struct is initialized or not
Check if a passed interface is a struct or a pointer to struct
You can see some examples here: http://godoc.org/github.com/fatih/structs#pkg-examples
For example converting a struct to a map is a simple:
type Server struct {
Name string
ID int32
Enabled bool
}
s := &Server{
Name: "gopher",
ID: 123456,
Enabled: true,
}
// => {"Name":"gopher", "ID":123456, "Enabled":true}
m := structs.Map(s)
The structs package has support for anonymous (embedded) fields and nested structs. The package provides to filter certain fields via field tags.
From struct to map[string]interface{}
package main
import (
"fmt"
"encoding/json"
)
type MyData struct {
One int
Two string
Three int
}
func main() {
in := &MyData{One: 1, Two: "second"}
var inInterface map[string]interface{}
inrec, _ := json.Marshal(in)
json.Unmarshal(inrec, &inInterface)
// iterate through inrecs
for field, val := range inInterface {
fmt.Println("KV Pair: ", field, val)
}
}
go playground here
Here is a function I've written in the past to convert a struct to a map, using tags as keys
// ToMap converts a struct to a map using the struct's tags.
//
// ToMap uses tags on struct fields to decide which fields to add to the
// returned map.
func ToMap(in interface{}, tag string) (map[string]interface{}, error){
out := make(map[string]interface{})
v := reflect.ValueOf(in)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
// we only accept structs
if v.Kind() != reflect.Struct {
return nil, fmt.Errorf("ToMap only accepts structs; got %T", v)
}
typ := v.Type()
for i := 0; i < v.NumField(); i++ {
// gets us a StructField
fi := typ.Field(i)
if tagv := fi.Tag.Get(tag); tagv != "" {
// set key of map to value in struct field
out[tagv] = v.Field(i).Interface()
}
}
return out, nil
}
Runnable example here.
Note, if you have multiple fields with the same tag value, then you will obviously not be able to store them all within a map. It might be prudent to return an error if that happens.
I like the importable package for the accepted answer, but it does not translate my json aliases. Most of my projects have a helper function/class that I import.
Here is a function that solves my specific problem.
// Converts a struct to a map while maintaining the json alias as keys
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj) // Convert to a json string
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
And in the main, this is how it would be called...
package main
import (
"fmt"
"encoding/json"
"github.com/fatih/structs"
)
type MyStructObject struct {
Email string `json:"email_address"`
}
func main() {
obj := &MyStructObject{Email: "test#test.com"}
// My solution
fmt.Println(StructToMap(obj)) // prints {"email_address": "test#test.com"}
// The currently accepted solution
fmt.Println(structs.Map(obj)) // prints {"Email": "test#test.com"}
}
package main
import (
"fmt"
"reflect"
)
type bill struct {
N1 int
N2 string
n3 string
}
func main() {
a := bill{4, "dhfthf", "fdgdf"}
v := reflect.ValueOf(a)
values := make(map[string]interface{}, v.NumField())
for i := 0; i < v.NumField(); i++ {
if v.Field(i).CanInterface() {
values[v.Type().Field(i).Name] = v.Field(i).Interface()
} else {
fmt.Printf("sorry you have a unexported field (lower case) value you are trying to sneak past. I will not allow it: %v\n", v.Type().Field(i).Name)
}
}
fmt.Println(values)
passObject(&values)
}
func passObject(v1 *map[string]interface{}) {
fmt.Println("yoyo")
}
I'm a bit late but I needed this kind of feature so I wrote this. Can resolve nested structs. By default, uses field names but can also use custom tags. A side effect is that if you set the tagTitle const to json, you could use the json tags you already have.
package main
import (
"fmt"
"reflect"
)
func StructToMap(val interface{}) map[string]interface{} {
//The name of the tag you will use for fields of struct
const tagTitle = "kelvin"
var data map[string]interface{} = make(map[string]interface{})
varType := reflect.TypeOf(val)
if varType.Kind() != reflect.Struct {
// Provided value is not an interface, do what you will with that here
fmt.Println("Not a struct")
return nil
}
value := reflect.ValueOf(val)
for i := 0; i < varType.NumField(); i++ {
if !value.Field(i).CanInterface() {
//Skip unexported fields
continue
}
tag, ok := varType.Field(i).Tag.Lookup(tagTitle)
var fieldName string
if ok && len(tag) > 0 {
fieldName = tag
} else {
fieldName = varType.Field(i).Name
}
if varType.Field(i).Type.Kind() != reflect.Struct {
data[fieldName] = value.Field(i).Interface()
} else {
data[fieldName] = StructToMap(value.Field(i).Interface())
}
}
return data
}
map := Structpb.AsMap()
// map is the map[string]interface{}

Resources