Pass struct interface to sqlite exec - sqlite

In Go, I'm trying to pass an interface{} to the statement.Exec() function from go-sqlite3. I'm sure this is a solved problem, but I cannot figure it out.
Basically I have a struct with the row data which I want to pass to a function that will insert it to a sqlite db. The thing is I want to be able to programmatically control what goes into the statement.Exec() function
Here is an excerpt:
type hostRows struct {
domain string
}
type clientRows struct {
name string
}
func main() {
...
data := hostRows{domain: "dom.com"}
insertRow(sqliteDatabase, data)
data2 := clientRows{name: "bob"}
insertRow(sqliteDatabase, data2)
...
}
func insertRow(db *sql.DB, row interface{}) {
insertSQL := "INSERT INTO table(col) VALUES (?)"
statement, _ := db.Prepare(insertSQL)
statement.Exec(row) // here's the issue, how can I extract the element in the interface to pass it to the function for Exec to understand
}
I know that in this example, I could hard code the row type to the struct and type statement.Exec(row.(hostRows).domain), but now the code will break to when the client struct is passed.
here is the deceleration for the Exec function
func (s *Stmt) Exec(args ...interface{}) (Result, error)
I've tried playing with reflect but it hasn't worked for me so far.
My only solution for the moment is using a switch condition that could check and prepare the right command for Exec, but this is less than dodgy.
type hostRows struct {
domain string
}
type clientRows struct {
name string
}
func main() {
...
data := hostRows{domain: "dom.com"}
insertRow(sqliteDatabase, 1, data)
data2 := clientRows{name: "bob"}
insertRow(sqliteDatabase, 2, data2)
...
}
func insertRow(db *sql.DB, i int, row interface{}) {
insertSQL := "INSERT INTO table(col) VALUES (?)"
statement, _ := db.Prepare(insertSQL)
// This basically could be a working solution, but I'm sure there is a better one
switch i {
case 1:
data := row.(hostRows)
statement.Exec(data.domain)
case 2:
data := row.(clientRows)
statement.Exec(data.name)
}
}
edit: corrected the INSERT statement ; forget the columns. corrected statement.Exec(row.domain) to statement.Exec(row.(hostRows).domain)
edit2: added second example

Remember that in order for reflect.Interface() to work, you must export the fields. To achieve what you want using reflection, you could try something like this:
type hostRows struct {
//Should export field to read it using reflect.Value.Interface()
Domain string
}
type clientRows struct {
//Should export field to read it using reflect.Value.Interface()
Name string
}
func insertRow(db *sql.DB, i int, row interface{}) {
rv := reflect.ValueOf(row)
var args []interface{}
for i := 0; i < rv.NumField(); i++ {
args = append(args, rv.Field(i).Interface())
}
db.Exec("Insert Satement...", args...)
}

Related

Pointers confusion in Go

I am having a hard time understanding the below code, I have it working but still don't understand it.
It would really helpful if someone can demystify it
In below code Flags name is not updated to "Changed"
package main
import "fmt"
type Command struct {
Name string
Flags []Flag
}
type Flag struct {
Name string
Shorthand string
}
func getCommand() *Command {
return &Command{Name: "Hello", Flags: []Flag{{"version", "v"}}}
}
func change(cmd *Command) {
for _, flg := range cmd.Flags {
flg.Name = "Changed"
}
}
func main() {
cmd := getCommand()
change(cmd)
fmt.Println(cmd.Flags[0])
}
Correct code. Flags Name is changed to "Changed"
package main
import "fmt"
type Command struct {
Name string
Flags []*Flag
}
type Flag struct {
Name string
Shorthand string
}
func getCommand() *Command {
return &Command{Name: "Hello", Flags: []*Flag{{"version", "v"}}}
}
func change(cmd *Command) {
for _, flg := range cmd.Flags {
flg.Name = "Changed"
}
}
func main() {
cmd := getCommand()
change(cmd)
fmt.Println(cmd.Flags[0])
}
I am sure, my confusion might be trivial, but it has wasted my couple of hours
In your first version, when you do this:
for _, flg := range cmd.Flags {
flg.Name = "Changed"
}
flg is a local variable of type Flag. On each iteration of the for loop, range sets flg to a copy of the next item in cmd.Flags. So when you change that, nothing changes outside that loop.
In your second version, when you do that loop, flg is a local variable of type *Flag, so when range sets it to a copy of the next item in cmd.Flags, flg is pointing to the data you are trying to mutate, and mutating it actually changes that data.
for _, flg := range cmd.Flags {
flg.Name = "Changed"
}
This part is the exact culprit. The first value returned by range is the index(which is ignored here), the second value is a copy of the actual data. When you are using it inside the loop(flg.Name=...), You are assigning the value to Name filed of the copy. You may do this:
for index, _ := range cmd.Flags {
cmd.Flags[index].Name = "Changed"
}
In the second case, you are assigning to Name field of a copy of reference(or memory address) to the Flag, so the value at that memory address get changed.

Initializing Slice of type Struct in Golang

I have the following structure in Golang
type mystruct struct {
Name string
Power int
}
My purpose is to write a function that takes as input a slice of type *mystuct and returns a slice of type int containing the "Power" property taken from the input slice.
my code is presented below:
package main
import (
"fmt"
)
func main() {
obj := make([]*mystruct, 15)
for i, s := range obj {
s.Power = i
}
fmt.Println(extractpowers(obj))
}
func extractpowers(obj []*mystruct) []int {
powers := make([]int, len(obj))
for i, s := range obj {
powers[i] = s.Power
}
return powers
}
My issue is that the obj := make([]*mystruct, 15) creates a slices of 15 *mystruc pointers initialized to nil; which causes the code within the for loop to raise a panic of type "invalid memory or nil pointer dereference".
My question is what is the proper and fastest way to initialize the slice; (the equivalent of var lst = new List(Of mystruct) in .net)
Regards.
Use a composite literal and take its address in the loop:
for i := range obj {
obj[i] = &mystruct{Power: i}
}
Try it on the Go Playground.

Passing structs to functions across packages

In an attempt to become more familiar with go, I am trying to refactor some code which is already working fine.
The original code has three structs:
type ConfigGroup struct {
Interval int
Tprefix string
Target []string
}
type ConfigDefaults struct {
Interval int
Sprefix string
}
type Config struct {
Group map[string]*ConfigGroup
Defaults ConfigDefaults
}
These structs get passed to a function like so:
func runpinger(clientStatsd statsd.Statter, defaults *ConfigDefaults, group *ConfigGroup) {
// some stuff here
}
Now, I've reworked the config (which uses gocfg) to use hcl instead, which seems to provide a cleaner config syntax.
I've moved the config parser into a package, config, with structs that look like this:
type Config struct {
Interval int `hcl:"interval"`
Prefix string `hcl:"prefix"`
Groups []TargetGroups `hcl:"target_group"`
}
type TargetGroups struct {
Name string `hcl:",key"`
Prefix string `hcl:"prefix"`
Interval int `hcl:"interval"`
Targets []Targets `hcl:"target"`
}
type Targets struct {
Address string `hcl:"address"`
Label string `hcl:"label"`
}
and then a function in config package that looks like this:
func Parse(ConfigFile string) (*Config, error) {
result := &Config{}
var errors *multierror.Error
config, err := ioutil.ReadFile(ConfigFile)
if err != nil {
return nil, err
}
hclParseTree, err := hcl.Parse(string(config))
if err != nil {
return nil, err
}
if err := hcl.DecodeObject(&result, hclParseTree); err != nil {
return nil, err
}
return result, errors.ErrorOrNil()
}
Now, in my main package I'd like to pass these structs to the function again. How can I do this across packages?
I tried:
func(runpinger config *config.Config) {
// here
}
But that didn't seem to work. Ideally, I'd like to just pass a pointer to the "sub-struct" (ie the TargetGroups struct) as well, although I'm not sure if that's possible.
You should be able to pass the structs to the main package, just check that you put import "path/to/config" at the top of your file.
The path has to be the full path to your package from your $GOPATH/src/ directory

Reflect: setting a field of a pointer

I'm trying to do something like this:
Define structs with tags named env:
type Env struct {
Port string `env:"PORT"`
}
Call some function which will get the environment variable names using os.Getenv and put set it in the struct.
Right now, I have this:
package main
import (
"fmt"
"os"
"reflect"
)
func ParseEnv(t interface{}, v interface{}) {
it := reflect.TypeOf(t)
for i := 0; i < it.NumField(); i++ {
field := it.Field(i)
value := os.Getenv(field.Tag.Get("env"))
if value == "" {
continue
}
reflect.ValueOf(v).Elem().FieldByName(field.Name).SetString(value)
}
}
type Env struct {
Port string `env:"PORT"`
DatabaseURL string `env:"DATABASE_URL"`
}
func main() {
os.Setenv("PORT", "8080")
os.Setenv("DATABASE_URL", "postgres://user:pass#host:5432/my-db")
env := Env{}
ParseEnv(env, &env)
fmt.Println(env)
}
http://play.golang.org/p/b8uPPVo4aV
But, as you can see, I have to pass both the reference and the pointer to my function.
While this works, it is very ugly (at least I think it is).
If I try to pass the pointer only, I can't get the type right (because it will be an *interface{}) and, if I pass only the reference, I can't set the values using reflect (even if I could, it would not work).
Is there a sane way of doing this?
Below is a "saner" way of doing what you want. You will notice that, instead of passing in two copies of the struct, we only need a pointer to the struct.
func ParseEnv(val interface{}) {
ptrRef := reflect.ValueOf(val)
if ptrRef.Kind() != reflect.Ptr {
panic("pointer to struct expected")
}
ref := ptrRef.Elem()
if ref.Kind() != reflect.Struct {
panic("pointer to struct expected")
}
refType := ref.Type()
for i := 0; i < refType.NumField(); i++ {
field := refType.Field(i)
value := os.Getenv(field.Tag.Get("env"))
if value == "" {
continue
}
ref.Field(i).SetString(value)
}
}
The above function should be invoked in the following way:
ParseEnv(&env)
Example: https://play.golang.org/p/_BwWz2oUql

function for converting a struct to map in Golang

I want to convert a struct to map in Golang. It would also be nice if I could use the JSON tags as keys in the created map (otherwise defaulting to field name).
Edit Dec 14, 2020
Since structs repo was archived, you can use mapstructure instead.
Edit TL;DR version, Jun 15, 2015
If you want the fast solution for converting a structure to map, see the accepted answer, upvote it and use that package.
Happy coding! :)
Original Post
So far I have this function, I am using the reflect package but I don't understand well how to use the package, please bear with me.
func ConvertToMap(model interface{}) bson.M {
ret := bson.M{}
modelReflect := reflect.ValueOf(model)
if modelReflect.Kind() == reflect.Ptr {
modelReflect = modelReflect.Elem()
}
modelRefType := modelReflect.Type()
fieldsCount := modelReflect.NumField()
var fieldData interface{}
for i := 0; i < fieldsCount; i++ {
field := modelReflect.Field(i)
switch field.Kind() {
case reflect.Struct:
fallthrough
case reflect.Ptr:
fieldData = ConvertToMap(field.Interface())
default:
fieldData = field.Interface()
}
ret[modelRefType.Field(i).Name] = fieldData
}
return ret
}
Also I looked at JSON package source code, because it should contain my needed implementation (or parts of it) but don't understand too much.
I also had need for something like this. I was using an internal package which was converting a struct to a map. I decided to open source it with other struct based high level functions. Have a look:
https://github.com/fatih/structs
It has support for:
Convert struct to a map
Extract the fields of a struct to a []string
Extract the values of a struct to a []values
Check if a struct is initialized or not
Check if a passed interface is a struct or a pointer to struct
You can see some examples here: http://godoc.org/github.com/fatih/structs#pkg-examples
For example converting a struct to a map is a simple:
type Server struct {
Name string
ID int32
Enabled bool
}
s := &Server{
Name: "gopher",
ID: 123456,
Enabled: true,
}
// => {"Name":"gopher", "ID":123456, "Enabled":true}
m := structs.Map(s)
The structs package has support for anonymous (embedded) fields and nested structs. The package provides to filter certain fields via field tags.
From struct to map[string]interface{}
package main
import (
"fmt"
"encoding/json"
)
type MyData struct {
One int
Two string
Three int
}
func main() {
in := &MyData{One: 1, Two: "second"}
var inInterface map[string]interface{}
inrec, _ := json.Marshal(in)
json.Unmarshal(inrec, &inInterface)
// iterate through inrecs
for field, val := range inInterface {
fmt.Println("KV Pair: ", field, val)
}
}
go playground here
Here is a function I've written in the past to convert a struct to a map, using tags as keys
// ToMap converts a struct to a map using the struct's tags.
//
// ToMap uses tags on struct fields to decide which fields to add to the
// returned map.
func ToMap(in interface{}, tag string) (map[string]interface{}, error){
out := make(map[string]interface{})
v := reflect.ValueOf(in)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
// we only accept structs
if v.Kind() != reflect.Struct {
return nil, fmt.Errorf("ToMap only accepts structs; got %T", v)
}
typ := v.Type()
for i := 0; i < v.NumField(); i++ {
// gets us a StructField
fi := typ.Field(i)
if tagv := fi.Tag.Get(tag); tagv != "" {
// set key of map to value in struct field
out[tagv] = v.Field(i).Interface()
}
}
return out, nil
}
Runnable example here.
Note, if you have multiple fields with the same tag value, then you will obviously not be able to store them all within a map. It might be prudent to return an error if that happens.
I like the importable package for the accepted answer, but it does not translate my json aliases. Most of my projects have a helper function/class that I import.
Here is a function that solves my specific problem.
// Converts a struct to a map while maintaining the json alias as keys
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj) // Convert to a json string
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
And in the main, this is how it would be called...
package main
import (
"fmt"
"encoding/json"
"github.com/fatih/structs"
)
type MyStructObject struct {
Email string `json:"email_address"`
}
func main() {
obj := &MyStructObject{Email: "test#test.com"}
// My solution
fmt.Println(StructToMap(obj)) // prints {"email_address": "test#test.com"}
// The currently accepted solution
fmt.Println(structs.Map(obj)) // prints {"Email": "test#test.com"}
}
package main
import (
"fmt"
"reflect"
)
type bill struct {
N1 int
N2 string
n3 string
}
func main() {
a := bill{4, "dhfthf", "fdgdf"}
v := reflect.ValueOf(a)
values := make(map[string]interface{}, v.NumField())
for i := 0; i < v.NumField(); i++ {
if v.Field(i).CanInterface() {
values[v.Type().Field(i).Name] = v.Field(i).Interface()
} else {
fmt.Printf("sorry you have a unexported field (lower case) value you are trying to sneak past. I will not allow it: %v\n", v.Type().Field(i).Name)
}
}
fmt.Println(values)
passObject(&values)
}
func passObject(v1 *map[string]interface{}) {
fmt.Println("yoyo")
}
I'm a bit late but I needed this kind of feature so I wrote this. Can resolve nested structs. By default, uses field names but can also use custom tags. A side effect is that if you set the tagTitle const to json, you could use the json tags you already have.
package main
import (
"fmt"
"reflect"
)
func StructToMap(val interface{}) map[string]interface{} {
//The name of the tag you will use for fields of struct
const tagTitle = "kelvin"
var data map[string]interface{} = make(map[string]interface{})
varType := reflect.TypeOf(val)
if varType.Kind() != reflect.Struct {
// Provided value is not an interface, do what you will with that here
fmt.Println("Not a struct")
return nil
}
value := reflect.ValueOf(val)
for i := 0; i < varType.NumField(); i++ {
if !value.Field(i).CanInterface() {
//Skip unexported fields
continue
}
tag, ok := varType.Field(i).Tag.Lookup(tagTitle)
var fieldName string
if ok && len(tag) > 0 {
fieldName = tag
} else {
fieldName = varType.Field(i).Name
}
if varType.Field(i).Type.Kind() != reflect.Struct {
data[fieldName] = value.Field(i).Interface()
} else {
data[fieldName] = StructToMap(value.Field(i).Interface())
}
}
return data
}
map := Structpb.AsMap()
// map is the map[string]interface{}

Resources