Pointer problems - pointers

TL;DR Somehow, I am appending a pointer to a list instead of the object within a for loop of objects so at the end the entire slice is composed of the same object multiple times. I just don't know how to fix that.
The Long Way
I am still having a super hard time trying to figure out pointers in go. I posted a question yesterday and got some help but now I am stuck on a slightly different issue in the same piece of code.
I am working with gocql and cqlr go packages to try and bit a small object mapper for my Cassandra queries. Essentially the problem I am having is I am appending what appears to be a pointer to an object, not a new instance of the obj to the array. How do I fix that? I have tried adding & and * in front of value but that doesn't seem to work. How do I fix these? The bind function needs an & according to their docs.
Code
type Query struct {
query string
values interface{}
attempts int
maxAttempts int
structType reflect.Type
}
func (query Query) RetryingQuery() (results []interface{}) {
var q *gocql.Query
if query.values != nil {
q = c.Session.Query(query.query, query.values)
} else {
q = c.Session.Query(query.query)
}
bindQuery := cqlr.BindQuery(q)
value := reflect.New(query.structType).Interface()
for bindQuery.Scan(value) {
fmt.Println(value)
results = append(results, value)
}
return
}
The docs ask for var value type then in bind you would pass &value. I quoted the docs below.
var t Tweet
var s []Tweet
for b.Scan(&t) {
// Application specific code goes here
append(s, t)
}
The issue is I cannot directly go var value query.structType to define its type then pass the reference of that to bindQuery.Scan().
What is printed
&{result1 x86_64 24 3.2.0-74-generic Linux}
&{result2 x86_64 24 3.19.0-25-generic Linux}
&{result3 x86_64 4 3.13.0-48-generic Linux}
&{result4 x86_64 2 3.13.0-62-generic Linux}
&{result5 x86_64 4 3.13.0-48-generic Linux}
What is in the slice
Spoiler, it is result5 repeated over and over. I understand that I am just appending the pointer to same object to the list and that every loop iteration the object is changed and that changes all the results in the slice to that new object. I just don't know how to fix it.
[{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"},{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"},{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"},{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"},{"hostname":"result5","machine":"x86_64","num_cpus":4,"release":"3.13.0-48-generic","sysname":"Linux"}]

Well I can at least tell you what you're doing. bindQuery takes a pointer. It changes the value stored at the address.
What you're essentially doing is this:
package main
import "fmt"
func main() {
var q int
myInts := make([]*int, 0, 5)
for i := 0; i < 5; i++ {
q = i
fmt.Printf("%d ", q)
myInts = append(myInts, &q)
}
fmt.Printf("\n")
for _, value := range myInts {
fmt.Printf("%d ", *value)
}
fmt.Printf("\n")
fmt.Println(myInts)
}
Which, as you can probably guess, gives you this:
0 1 2 3 4
4 4 4 4 4
[0x104382e0 0x104382e0 0x104382e0 0x104382e0 0x104382e0]
Things get a little more confusing with reflect. You can get your type as an interface, but that is it (unless you want to play with unsafe). An interface, in simple terms, contains a pointer to the original type underneath (and some other stuff). So in your function you are passing a pointer (and some other stuff). Then you're appending the pointer. It might be nice just to get concrete and type switch your interface. I assume you know what types it could be. In which case you'd have to have something along these lines:
package main
import (
"fmt"
"reflect"
)
type foo struct {
fooval string
}
type bar struct {
barval string
}
func main() {
f1 := foo{"hi"}
f2 := &foo{"hi"}
b1 := bar{"bye"}
b2 := &bar{"bye"}
doSomething(f1)
doSomething(f2)
doSomething(b1)
doSomething(b2)
}
func doSomething(i interface{}) {
n := reflect.TypeOf(i)
// get a new one
newn := reflect.New(n).Interface()
// find out what we got and handle each case
switch t := newn.(type) {
case **foo:
*t = &foo{"hi!"}
fmt.Printf("It was a **foo, here is the address %p and here is the value %v\n", *t, **t)
case **bar:
*t = &bar{"bye :("}
fmt.Printf("It was a **bar, here is the address %p and here is the value %v\n", *t, **t)
case *foo:
t = &foo{"hey!"}
fmt.Printf("It was a *foo, here is the address %p and here is the value %v\n", t, *t)
case *bar:
t = &bar{"ahh!"}
fmt.Printf("It was a *bar, here is the address %p and here is the value %v\n", t, *t)
default:
panic("AHHHH")
}
}
You could also just keep calling value = reflect.New(query.structType).Interface() inside of the loop which will give you new interfaces every time. Reassigning value after every append. Last time through the loop would make one extra though..

Related

How to use struct keys in a map?

I am implementing some DB logic without using an ORM.
Can I somehow create a mapping between struct keys and db enums?
type Message struct {
SomeKey string
SomeOtherKey string
}
MessageToDBEnum: = map[ ? MessageKey] string {
SomeKey: "some_key",
SomeOtherKey: "some_other_key"
}
can I later dynamically use the map key (eg when iterating throgh the map) to set/get struct values?
also can I somehow assure that MessageToDBEnum is exhaustive (all public keys of Message are included)?
You essentially want to convert a struct to map and vice versa. There are 3rd party libs to do that, e.g. github.com/mitchellh/mapstructure.
But we can do it ourselves too, it's not that hard. We may use reflection to do it. Without checking errors, here's the essence of the struct to map conversion:
func struct2Map(x interface{}) map[string]interface{} {
m := map[string]interface{}{}
v := reflect.ValueOf(x)
t := reflect.TypeOf(x)
for i := 0; i < v.NumField(); i++ {
m[t.Field(i).Name] = v.Field(i).Interface()
}
return m
}
You may use it like this:
msg := Message{
SomeKey: "v1",
SomeOtherKey: "v2",
}
m := struct2Map(msg)
fmt.Println(m)
Which outputs:
map[SomeKey:v1 SomeOtherKey:v2]
The backward conversion is even simpler, but know that for a function to modify a struct, you must pass a pointer to it. Again, without checking for possible errors, the essence of it is:
func map2Struct(m map[string]interface{}, d interface{}) {
s := reflect.ValueOf(d).Elem()
for k, v := range m {
s.FieldByName(k).Set(reflect.ValueOf(v))
}
}
Using it:
var msg2 Message
map2Struct(m, &msg2)
fmt.Printf("%+v\n", msg2)
Which outputs:
{SomeKey:v1 SomeOtherKey:v2}
Try the examples on the Go Playground.
You may build the validation into these conversion functions, and return an error or use default / zero values when an invalid value is found.

Comparing referenced internal struct field to string type in go

Say I have two structs that define a linked list:
....
....
type node struct {
item interface{}
next *node
}
type LinkedList struct {
first *node
N int
}
...
...
and I want to compare the value of the type of the underlying node, say, in a find function where we check if k == node.item such that:
func (l *LinkedList) find (key interface{}) bool {
result := false
if !l.isEmpty() {
for x:= l.first; x != nil; x = x.next {
if x.item == key {
result = true
break
}
}
return result
}
this will not work for the expected find function because the underlying types are different, hence the func will always return false. We can confirm this upon reflecting the type:
fmt.Println(reflect.TypeOf(key), reflect.TypeOf(x.item))
>>> string, *main.node
Tried workarounds?
I've tried asserting the type but alas this does not work and panics
tmp := x.item.(string)
>>>panic: interface conversion: interface {} is *main.node, not string
This case is the same for using fmt.Sprintf(x.item)
I'm a bit stumped as to where to go from here. Is there a way to do this?
Inserting item to linked list
The following snippet should clarify how insertion is handled
func (l *LinkedList) insertFirst(item interface{}) {
var first *node = new(node)
oldfirst := l.first
first.item = item
first.next = oldfirst
l.first = first
l.N++
}
.....
//which gets called somewhere like
var n *node = new(node)
n.item = item
l.insertFirst(n)
.....wait no theres the error!
----------
burak-serdar you are 100% correct that I am inserting the node in the node!
The interface comparison in find() is a valid comparison and it will work if the type of the key and the type of the value stored in the node are the same. However, evidence points to you adding a node in place of a value.

How to convert interface to struct

Here is the simplified code of a cache. Suppose Container placed in a package, so it don't know about Member.
While I wanna store instances of Member in Container, So I store an empty instance of Member in Container as outerType.
In the Container->GetMysql, I fill a new variable by test values (but, in real world, It fill by data of database, dynamically).
then in the function Put, I store data in items as Cache for next uses. In the Get I get the data stored in the items.
Before this every thing is fine. My problem is where i want to convert result of Get to type of Member m = res.(Member) . How Can I convert it to an instance of Member
I found many question about this subject, but none of them solved my problem
For more detail: I want the Get return data with its pointer of where it stored in items. So if I get some variable of same member, an change in one are shown in others
package main
import (
"fmt"
"reflect"
)
type Member struct {
Id int
Name string
Credit int
Age int
}
type Container struct {
outerType interface{}
items map[string]*interface{}
}
func (cls *Container)GetMysql(s string, a int64) interface{}{
obj := reflect.New(reflect.TypeOf(cls.outerType))
elem := obj.Elem()
//elem := reflect.ValueOf(o).Elem()
if elem.Kind() == reflect.Struct {
f := elem.FieldByName("Name")
f.SetString(s)
f = elem.FieldByName("Credit")
f.SetInt(a)
}
return obj.Interface()
}
func (cls *Container)Get(value string) *interface{}{
return cls.items[value]
}
func (cls *Container)Put(value string, a int64) {
res := cls.GetMysql(value, a)
cls.items[value] = &res
}
func main() {
c := Container{outerType:Member{}}
c.items = make(map[string]*interface{})
c.Put("Jack", 500)
res := c.Get("Jack")
fmt.Println(*res)
m := &Member{}
m = res.(Member) // Here is the problem. How to convert ?
fmt.Println(m)
}
You should hardly ever use pointer to interface. My advice is to never use it, when you'll need it, you'll know.
Instead if you need a pointer to something (so you can have the same pointer at multiple places, and so modifying the pointed value somewhere, it will have effect on the others), "wrap the pointer" in the interface value.
So first modify the items field so that it stores interface{} values instead of pointers:
items map[string]interface{}
This means no restriction: you can pass and store pointers, that's not a problem.
Next modify Get() to return interface{}:
func (cls *Container) Get(value string) interface{}{
return cls.items[value]
}
And also in Put(), don't take the address of an interface{}:
func (cls *Container) Put(value string, a int64) {
res := cls.GetMysql(value, a)
cls.items[value] = res
}
And you have to type-assert *Member from the values returned by Get().
And now testing it:
c := Container{outerType: Member{}}
c.items = make(map[string]interface{})
c.Put("Jack", 500)
res := c.Get("Jack")
fmt.Println(res)
m := res.(*Member) // Here is the problem. How to convert ?
fmt.Println(m)
Output (try it on the Go Playground):
&{0 Jack 500 0}
&{0 Jack 500 0}
Now if you would modify a field of m:
m.Credit = 11
And then get the value form the cache:
fmt.Println(c.Get("Jack"))
We'll see the modified value, even though we did not call Put() (try it on the Go Playground):
&{0 Jack 11 0}

Difference between &Struct{} vs Struct{}

Is there a reason why I should create a struct using &StructName{} instead of Struct{}? I see many examples using the former syntax, even in the Effective Go Page but I really can not understand why.
Additional Notes:
I'm not sure whether I explained my problem well with these two approaches so let me refine my question.
I know that by using the & I will recieve a pointer instead of a value however I would like to know why would I use the &StructName{} instead of the StructName{}. For example, is there any benefits of using:
func NewJob(command string, logger *log.Logger) *Job {
return &Job{command, logger}
}
instead of:
func NewJob(command string, logger *log.Logger) Job {
return Job{command, logger}
}
Well, they will have different behavior. Essentially if you want to modify state using a method on a struct, then you will need a pointer, otherwise a value will be fine. Maybe an example will be better:
package main
import "fmt"
type test_struct struct {
Message string
}
func (t test_struct)Say (){
fmt.Println(t.Message)
}
func (t test_struct)Update(m string){
t.Message = m;
}
func (t * test_struct) SayP(){
fmt.Println(t.Message)
}
func (t* test_struct) UpdateP(m string) {
t.Message = m;
}
func main(){
ts := test_struct{}
ts.Message = "test";
ts.Say()
ts.Update("test2")
ts.Say() // will still output test
tsp := &test_struct{}
tsp.Message = "test"
tsp.SayP();
tsp.UpdateP("test2")
tsp.SayP() // will output test2
}
And you can run it here go playground
Assuming you know the general difference between a pointer and a value:
The first way allocates a struct and assigns a pointer to that allocated struct to the variable p1.
p1 := &StructName{}
The second way allocates a struct and assigns a value (the struct itself) to the variable s.
Then a pointer to that struct may be assigned to another variable (p2 in the following example).
s := StructName{}
p2 := &s

How to get underlying value from a reflect.Value in golang?

So I found some code that help me get started with reflection in Go (golang), but I'm having trouble getting a the underlying value so that I can basically create a map[string]string from a struct and it's fields.
Eventually, I'd like to make the result into a map[string]interface{}, but this one issue is kind of blocking me.
The code I have at the moment:
package main
import (
"fmt"
"reflect"
)
type Foo struct {
FirstName string `tag_name:"tag 1"`
LastName string `tag_name:"tag 2"`
Age int `tag_name:"tag 3"`
}
func inspect(f interface{}) map[string]string {
m := make(map[string]string)
val := reflect.ValueOf(f).Elem()
for i := 0; i < val.NumField(); i++ {
valueField := val.Field(i)
typeField := val.Type().Field(i)
f := valueField.Interface()
val := reflect.ValueOf(f)
m[typeField.Name] = val.String()
}
return m
}
func dump(m map[string]string) {
for k, v := range m {
fmt.Printf("%s : %s\n", k, v)
}
}
func main() {
f := &Foo{
FirstName: "Drew",
LastName: "Olson",
Age: 30,
}
a := inspect(f)
dump(a)
}
The output from running the code:
FirstName : Drew
LastName : Olson
Age : <int Value>
From what I understand the output for FirstName and LastName are actual reflect.Value objects but for strings the String() method on value just outputs the underlying String. I'd like to either get the int and change it into a string, but from the relfect package documentation I'm not immediately seeing how that's done.
Soo.... How do I get the underlying value from a reflect.Value in golang?
A good example of how to parse values is the fmt package. See this code.
Using the mentioned code to match your problem would look like this:
switch val.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
m[typeField.Name] = strconv.FormatInt(val.Int(), 10)
case reflect.String:
m[typeField.Name] = val.String()
// etc...
}
Basically you need to check for all available Kinds.
It looks like you're on the right track. The problem I see with your code is it makes assumptions about the values, meaning when do you call Elem() and how many times (to resolve pointers). In order to know this you need to look at the reflect.Kind. Is the value a reflect.Ptr? Then use Elem().
Once you have the value from val.Interface() / val.String() / val.Int() you can convert your values as needed. What you use is going to depend on reflect.Kind. To convert an int to/from string you need to use the strconv package.
The encoding/json and encoding/xml packages do this kind of work already. The source code provides some great examples. For example, take a look at copyValue in encoding/xml/read.go and marshalSimple in encoding/xml/marshal.go.
This should be easier to do with Go 1.5 (August 2015)
See review 8731 and commit 049b89d by Rob Pike (robpike):
fmt: treat reflect.Value specially - as the value it holds
This would allow you to print the actual value of a Reflect.Value() argument:
When a reflect.Value is passed to Printf (etc.), fmt called the String method, which does not disclose its contents.
To get the contents, one could call Value.Interface(), but that is illegal
if the Value is not exported or otherwise forbidden.
This CL improves the situation with a trivial change to the fmt package: when we see a reflect.Value as an argument, we treat it exactly as we treat a reflect.Value we make inside the package.
This means that we always print the contents of the Value as if that was the argument to Printf.
This is arguably a breaking change but I think it is a genuine improvement and no greater a break than many other tweaks we have made to formatted output from this package.
Another simple solution can be ,
flavorName = fmt.Sprintf("%v",strct)
" fmt.Sprintf() " will return the value which can be stored in a variable.

Resources