I want to specify a function based on a string. I'm getting strings out of a map, in the example below they are the values in function while interating ove the map. Now for example, when the string value function == "networkInfo", I would like to "treat" that value as a real functions' name. It's hard to explain, but I think you guys will know what I mean.
My goal is to remove the switch statement and directly call c.AddFunc(spec, func() { networkInfo() }) where networkInfo is the function name, extracted from string function. I know this is possible, but I don't know how :(. Help is appreciated!
// ScheduleCronjobs starts the scheduler
func ScheduleCronjobs() {
tasks := props.P.GetStringMapString("tasks")
log.Infof("number of tasks: %d", len(tasks))
if len(tasks) != 0 {
c := cron.New()
// for each task, initialize
for function, spec := range tasks {
switch function {
case "networkInfo":
c.AddFunc(spec, func() { networkInfo() })
case "bla":
c.AddFunc(spec, func() { bla() })
default:
log.Errorf("unknown task: %q", function)
}
}
c.Start()
}
// after initialization, send out confirmation message
slack.SendMessage("tasks initialized", props.P.GetString("channel"))
}
Why not something like:
taskDefs := map[string]func(){
"networkInfo": networkInfo,
"bla": bla,
}
for function, spec := range tasks {
if fn, ok := taskDefs[function]; ok {
c.AddFunc(spec, func() { fn() }) // not sure if you need the enclosing func
} else {
log.Errorf("unknown task: %q", function)
}
}
If you do need varying signatures of your funcs then you'd actually need reflection, but if the types of the funcs are all the same, then using this map approach might be a simpler solution, without the overhead of reflection.
The only way I've found to find functions by name in a package is by actually parsing the source files. This repo is an example of finding funcs and storing them in a map with the name as the key.
The Go linker will silently drop unreferenced funcs, so if the only way you're referencing the func is through reflection it would break. That is why the map approach I suggest is superior; it let's the linker know the func is being used.
Related
I am implementing some DB logic without using an ORM.
Can I somehow create a mapping between struct keys and db enums?
type Message struct {
SomeKey string
SomeOtherKey string
}
MessageToDBEnum: = map[ ? MessageKey] string {
SomeKey: "some_key",
SomeOtherKey: "some_other_key"
}
can I later dynamically use the map key (eg when iterating throgh the map) to set/get struct values?
also can I somehow assure that MessageToDBEnum is exhaustive (all public keys of Message are included)?
You essentially want to convert a struct to map and vice versa. There are 3rd party libs to do that, e.g. github.com/mitchellh/mapstructure.
But we can do it ourselves too, it's not that hard. We may use reflection to do it. Without checking errors, here's the essence of the struct to map conversion:
func struct2Map(x interface{}) map[string]interface{} {
m := map[string]interface{}{}
v := reflect.ValueOf(x)
t := reflect.TypeOf(x)
for i := 0; i < v.NumField(); i++ {
m[t.Field(i).Name] = v.Field(i).Interface()
}
return m
}
You may use it like this:
msg := Message{
SomeKey: "v1",
SomeOtherKey: "v2",
}
m := struct2Map(msg)
fmt.Println(m)
Which outputs:
map[SomeKey:v1 SomeOtherKey:v2]
The backward conversion is even simpler, but know that for a function to modify a struct, you must pass a pointer to it. Again, without checking for possible errors, the essence of it is:
func map2Struct(m map[string]interface{}, d interface{}) {
s := reflect.ValueOf(d).Elem()
for k, v := range m {
s.FieldByName(k).Set(reflect.ValueOf(v))
}
}
Using it:
var msg2 Message
map2Struct(m, &msg2)
fmt.Printf("%+v\n", msg2)
Which outputs:
{SomeKey:v1 SomeOtherKey:v2}
Try the examples on the Go Playground.
You may build the validation into these conversion functions, and return an error or use default / zero values when an invalid value is found.
Here's my code:
func test(v map[string]string) {
v["foo"] = "bar"
}
func main() {
v := make(map[string]string)
test(v)
fmt.Printf("%v\n", v) // prints map[foo:bar]
}
I'm pretty new to Go, but as far as I was aware, since I'm passing the map value to test() and not a pointer to the map, the test() function should modify a different variable of the map, and thus, not affect the value of the variable in main(). I would have expected it to print map[]. I tested a different scenario:
type myStruct struct {
foo int
}
func test2(v myStruct) {
v.foo = 5
}
func main() {
v := myStruct{1}
test2(v)
fmt.Printf("%v\n", v) // prints {1}
}
In this scenario, the code behaves as I would expect. The v variable in the main() function is not affected by the changes to the variable in test2(). So why is map different?
You are right in that when you pass something to a function, a copy will be made. But maps are some kind of descriptors to an underlying data structure. So when you pass a map value to a function, only the descriptor will be copied which will denote / point to the same data structures where the map data (entries) are stored.
This means if the function does any modification to the entries of the map (add, delete, modify entries), that is observed from the caller.
Read The Go Blog: Go maps in action for details.
Note that the same applies to slices and channels too; generally speaking the types that you can create using the built-in make() function. That's why the zero value of these types is nil, because a value of these types need some extra initialization which is done when calling make().
In your other example you are using a struct value, they are not descriptors. When you pass a struct value to another function, that creates a complete copy of the struct value (copying values of all its fields), which –when modified inside the function– will not have any effect on the original, as the memory of the copy will be modified – which is distinct.
So let's say that we have a function of the following form:
func WorkMagic(obj interface{}) interface{} {
switch t := obj.(type) {
case string:
// Do string magic
default:
// Do slice magic
}
...
}
I am expecting obj to be either a string or a slice, which I can ascertain via the switch. In the case of a slice, I want to be able to do ordering work on any arbitrary slice, regardless of type. Seems like the best way to accomplish this is using the unsafe package in a similar fashion to that discussed in this article.
Here however, the function accepts a specific type of slice ([]string), whereas I would like to be able to work on any slice. So the question is, given that I am accepting an empty interface as input, how might I access the underlying slice / array using unsafe.Pointer so as to be able to loop through and modify which value is associate with which index?
You'll want to use reflection. It enables you to work generically without giving up type and memory safety like unsafe would. Read the Go blog's Laws of Reflection.
func actOnSlices(i interface{}) {
v := reflect.ValueOf(i)
for v.Kind() == reflect.Ptr { // dereference pointers
v = v.Elem()
}
if v.Kind() != reflect.Slice { // ensure you actually got a slice
panic("given argument is not a slice")
}
// do slice stuff
}
Edit to answer your second question:
Yes – this can be done: elements of a slice are adressable and hence settable. See the following working example:
package main
import (
"fmt"
"reflect"
)
func main() {
s := []string{"foo", "bar"}
fmt.Println(swapIndexes(s, 0, 1)) // prints [bar foo]
}
func swapIndexes(i interface{}, x, y int) interface{} {
v := reflect.ValueOf(i)
for v.Kind() == reflect.Ptr { // dereference pointers
v = v.Elem()
}
if v.Kind() != reflect.Slice { // ensure you actually got a slice
panic("given argument is not a slice")
}
t := v.Index(x).Interface()
v.Index(x).Set(v.Index(y))
v.Index(y).Set(reflect.ValueOf(t))
return v.Interface()
}
Edit to answer your third question:
The unsafe package is not something you'll encounter much in user-land code. It exists to implement certain features (e.g. reflection, C interaction) that need to circumvent Go's safety guarantees to work. Using unsafe is unsafe, as the name suggests, because you can mess up big time without even realizing. By using unsafe, you're incurring in a big trade-off, so it better be worth it. Quoting #twotwotwo:
The downside of unsafe is that if you mess up you're in the old days of segfaults, memory corruption, and buffer-overflow security holes.
Also, as #twotwotwo suggested; it's more "Go-like" to repeat code than using reflection to achieve genericity.
To Go's type-system, []string and []int are two completely separate and unrelated types. just as int and string would be. The relation (both are slices) is obvious only to the programmer. There is no way of expressing "a slice" without saying a slice of what.
I have an issue using this statement
m.Post(Model, binding.Form(Wish), func(wish Wish, r render.Render, db *mgo.Database) {
This worked fine if I use the struct define inside the prog like
m.Post(Model, binding.Form(Wish1{}) , func(wish Wish1, r render.Render, db *mgo.Database) {
but I need this to be an independent package.
I get "Wish is not a type" wish is the return of the binding function.
This worked with a primary Type struct. I am passing the strut as a interface{}
I am using GO with Martini.Classic() It is really complicated for me to change Martini or Binding package. Any suggestions.
This is the all code
package chlistpkg
import (
"github.com/codegangsta/martini"
"github.com/codegangsta/martini-contrib/binding"
"github.com/codegangsta/martini-contrib/render"
"labix.org/v2/mgo"
"time"
"fmt"
"html/template"
"reflect"
"adminStruct"
)
just to show the struct that I need to pass as to routine Doall
type Wish1 struct {
Name string `form:"name"`
Description string `form:"description"`
AnyDate time.Time `form:"anydate"`
Active bool `form:"active"`
Number int `form:"number"`
NumDec float32 `form:"numDec"`
}
DB Returns a martini.Handler
func DB() martini.Handler {
session, err := mgo.Dial("mongodb://localhost")
if err != nil {
panic(err)
}
return func(c martini.Context) {
s := session.Clone()
c.Map(s.DB("advent2"))
defer s.Close()
c.Next()
}
}
GetAll returns all Wishes in the database
func GetAll(db *mgo.Database, entList interface{}) interface{} {
db.C("wishes").Find(nil).All(entList)
fmt.Println("GettAll entList =", entList)
return entList
}
func Doall(Model string, Wish interface{}, Wish2 interface{}, Wishlist interface{} ) {
m := martini.Classic()
fmt.Println ("martini.Classic =", m)
m.Use(martini.Static("images")) // serve from the "images" directory as well
m.Use(render.Renderer(render.Options{
Directory: "templates",
Layout: "layout",
}))
m.Use(DB())
m.Get(Model, func(r render.Render, db *mgo.Database) {
r.HTML(200, "lista4", GetAll(db, Wishlist))
})
binding does not take a pointer. I have to pass the struct by reference on "Wish"
the issue is the return on "wish Wish" I got an error Wish is not a type
at compilation time
m.Post(Model, binding.Form(Wish), func(wish Wish, r render.Render, db *mgo.Database) {
fmt.Println("Input wish =", wish)
db.C("wishes").Insert(wish)
r.HTML(200, "lista4", GetAll(db, Wishlist))
})
m.Run()
Thanks in advance
Luis
The reason you are getting an error is that you have called your type Wish1 (with a numerical 1) but you are referring to the Wish type (which does not exist!) in your code.
Change your struct to be:
// Note: "Wish", not "Wish1"
type Wish struct {
Name string `form:"name"`
Description string `form:"description"`
AnyDate time.Time `form:"anydate"`
Active bool `form:"active"`
Number int `form:"number"`
NumDec float32 `form:"numDec"`
}
If you want to put your type into another package (tip: don't overdo the sub-packages), then it will need to become a pkgname.Wish as names are fully qualified.
Added
After a second look, you're also messing things up here:
func Doall(Model string, Wish interface{}, Wish2 interface{}, Wishlist interface{} ) {
m := martini.Classic()
fmt.Println ("martini.Classic =", m)
m.Use(martini.Static("images")) // serve from the "images" directory as well
Your parameter list needs to provide a name for each type; you can't pass Wish interface{} as a parameter as Wish is a type, not a variable name.
You should either:
func DoAll(model string, wish interface{}, wish2 interface{}, wishList interface{}) { ... }
Or, better still, stop using interface{} like this and write:
func DoAll(model string, wishList []Wish, wishes... Wish) { ... }
However, your DoAll function does not seem to be referenced elsewhere, and is creating its own Martini instance. I highly suggest thinking about why things are "split out" like this if you're just starting out. Keep it simple - e.g.
func main() {
m := martini.Classic()
m.Use(martini.Static("images"))
m.Use(DB())
m.Use(render.Renderer(render.Options{...}))
// No need for an anonymous function, which just adds clutter
m.Get("/wishes/all", GetAllWishes)
// Same goes for here
m.Post("/wishes/new", PostWish)
m.Run()
}
PS: I've fixed the formatting of your code, as it has a lot of unnecessary spacing before/after parenthesis. Make sure to use gofmt, which is included with the Go install and can be hooked into most popular editors.
I have some trouble building a function that can dynamically use parametrized structs. For that reason my code has 20+ functions that are similar except basically for one type that gets used. Most of my experience is with Java, and I'd just develop basic generic functions, or use plain Object as parameter to function (and reflection from that point on). I would need something similar, using Go.
I have several types like:
// The List structs are mostly needed for json marshalling
type OrangeList struct {
Oranges []Orange
}
type BananaList struct {
Bananas []Banana
}
type Orange struct {
Orange_id string
Field_1 int
// The fields are different for different types, I am simplifying the code example
}
type Banana struct {
Banana_id string
Field_1 int
// The fields are different for different types, I am simplifying the code example
}
Then I have function, basically for each list type:
// In the end there are 20+ of these, the only difference is basically in two types!
// This is very un-DRY!
func buildOranges(rows *sqlx.Rows) ([]byte, error) {
oranges := OrangeList{} // This type changes
for rows.Next() {
orange := Orange{} // This type changes
err := rows.StructScan(&orange) // This can handle each case already, could also use reflect myself too
checkError(err, "rows.Scan")
oranges.Oranges = append(oranges.Oranges,orange)
}
checkError(rows.Err(), "rows.Err")
jsontext, err := json.Marshal(oranges)
return jsontext, err
}
Yes, I could change the sql library to use more intelligent ORM or framework, but that's besides the point. I want to learn on how to build generic function that can handle similar function for all my different types.
I got this far, but it still doesn't work properly (target isn't expected struct I think):
func buildWhatever(rows *sqlx.Rows, tgt interface{}) ([]byte, error) {
tgtValueOf := reflect.ValueOf(tgt)
tgtType := tgtValueOf.Type()
targets := reflect.SliceOf(tgtValueOf.Type())
for rows.Next() {
target := reflect.New(tgtType)
err := rows.StructScan(&target) // At this stage target still isn't 1:1 smilar struct so the StructScan fails... It's some perverted "Value" object instead. Meh.
// Removed appending to the list because the solutions for that would be similar
checkError(err, "rows.Scan")
}
checkError(rows.Err(), "rows.Err")
jsontext, err := json.Marshal(targets)
return jsontext, err
}
So umm, I would need to give the list type, and the vanilla type as parameters, then build one of each, and the rest of my logic would be probably fixable quite easily.
Turns out there's an sqlx.StructScan(rows, &destSlice) function that will do your inner loop, given a slice of the appropriate type. The sqlx docs refer to caching results of reflection operations, so it may have some additional optimizations compared to writing one.
Sounds like the immediate question you're actually asking is "how do I get something out of my reflect.Value that rows.StructScan will accept?" And the direct answer is reflect.Interface(target); it should return an interface{} representing an *Orange you can pass directly to StructScan (no additional & operation needed). Then, I think targets = reflect.Append(targets, target.Indirect()) will turn your target into a reflect.Value representing an Orange and append it to the slice. targets.Interface() should get you an interface{} representing an []Orange that json.Marshal understands. I say all these 'should's and 'I think's because I haven't tried that route.
Reflection, in general, is verbose and slow. Sometimes it's the best or only way to get something done, but it's often worth looking for a way to get your task done without it when you can.
So, if it works in your app, you can also convert Rows straight to JSON, without going through intermediate structs. Here's a sample program (requires sqlite3 of course) that turns sql.Rows into map[string]string and then into JSON. (Note it doesn't try to handle NULL, represent numbers as JSON numbers, or generally handle anything that won't fit in a map[string]string.)
package main
import (
_ "code.google.com/p/go-sqlite/go1/sqlite3"
"database/sql"
"encoding/json"
"os"
)
func main() {
db, err := sql.Open("sqlite3", "foo")
if err != nil {
panic(err)
}
tryQuery := func(query string, args ...interface{}) *sql.Rows {
rows, err := db.Query(query, args...)
if err != nil {
panic(err)
}
return rows
}
tryQuery("drop table if exists t")
tryQuery("create table t(i integer, j integer)")
tryQuery("insert into t values(?, ?)", 1, 2)
tryQuery("insert into t values(?, ?)", 3, 1)
// now query and serialize
rows := tryQuery("select * from t")
names, err := rows.Columns()
if err != nil {
panic(err)
}
// vals stores the values from one row
vals := make([]interface{}, 0, len(names))
for _, _ = range names {
vals = append(vals, new(string))
}
// rowMaps stores all rows
rowMaps := make([]map[string]string, 0)
for rows.Next() {
rows.Scan(vals...)
// now make value list into name=>value map
currRow := make(map[string]string)
for i, name := range names {
currRow[name] = *(vals[i].(*string))
}
// accumulating rowMaps is the easy way out
rowMaps = append(rowMaps, currRow)
}
json, err := json.Marshal(rowMaps)
if err != nil {
panic(err)
}
os.Stdout.Write(json)
}
In theory, you could build this to do fewer allocations by not reusing the same rowMap each time and using a json.Encoder to append each row's JSON to a buffer. You could go a step further and not use a rowMap at all, just the lists of names and values. I should say I haven't compared the speed against a reflect-based approach, though I know reflect is slow enough it might be worth comparing them if you can put up with either strategy.