Say I have this struct:
type Foo struct {
Bar *string `json:"bar"`
Baz *int64 `json:"baz,omitempty"`
Qux *string `json:"qux"`
Quux string `json:"quux"`
}
After unmarshalling the json, I check for nil like so:
switch {
case f.Bar == nil:
return errors.New("Missing 'bar'")
case f.Baz == nil:
f.Baz = 42
case f.Qux == nil:
return errors.New("Missing 'qux'")
}
(or through a series of if statements, etc...)
I understand that I can put all the nil comparisons in one comma separated case, but each nil check will have differing returns.
My question: is there a less verbose way of doing the nil checks?
A question to you: how less verbose you want to get? Because you want to do different things on different conditions (different fields being nil). Your code contains these different things and the different conditions. Beyond that what's "redundant" in your code are just the switch and case keywords. You want to leave them out? Because the rest is not "redundant", they are required.
Also note that in Go cases do not fall through even without a break (unlike in other languages), so in your above example if f.Baz is nil, you will set it to 42 and f.Qux will not be checked (so no error will be returned), but if f.Baz is non-nil and f.Qux is nil, an error will be returned. I know it's just an example, but this is something to keep in mind. You should handle errors first if you use a switch! Or use if statements and then the error will be detected and returned regardless of the order of field checks.
Your code with switch is clean and efficient. If you want to make it less verbose, readability (and performance) will suffer.
You may use a helper function which checks if a pointer value is nil:
func n(i interface{}) bool {
v := reflect.ValueOf(i)
return v.Kind() == reflect.Ptr && v.IsNil()
}
And using it:
func check(f *Foo) error {
switch {
case n(f.Bar):
return errors.New("Missing 'bar'")
case n(f.Qux):
return errors.New("Missing 'qux'")
case n(f.Baz):
x := int64(42)
f.Baz = &x
}
return nil
}
Or using if statements:
func check2(f *Foo) error {
if n(f.Bar) {
return errors.New("Missing 'bar'")
}
if n(f.Qux) {
return errors.New("Missing 'qux'")
}
if n(f.Baz) {
x := int64(42)
f.Baz = &x
}
return nil
}
Try these on the Go Playground.
Related
I am studying reflect in Go and trying to implement function which get map and return another map, where keys will be values and values will be keys.
Example:
m := map[string]int{"one": 1, "two": 2}
fmt.Println(ReverseMap(m)) // {1: "one", 2: "two"}
Here is my code:
func ReverseMap(in interface{}) interface{} {
var out reflect.Value
v := reflect.ValueOf(in)
if v.Kind() == reflect.Map {
for idx, key := range v.MapKeys() {
value := v.MapIndex(key)
if idx == 0 {
mapType := reflect.MapOf(reflect.TypeOf(value), reflect.TypeOf(key))
out = reflect.MakeMap(mapType)
}
out.SetMapIndex(value, key)
}
}
return out
}
This code panic with error:
panic: reflect.Value.SetMapIndex: value of type int is not assignable to type reflect.Value
I think the reason of this error is the declaration of out variable, but I don't know how to declare it correctly, if I don't know the type of this variable.
How can I fix this error?
The key and value are of type reflect.Value, so passing them to reflect.TypeOf() will not return the type descriptors of the key and value types of the map (string and int), but instead the type descriptor of the reflect.Value type itself.
Instead simply call their Value.Type() method:
mapType := reflect.MapOf(value.Type(), key.Type())
With this it'll (almost) work and print (try it on the Go Playground):
map[1:one 2:two]
I wrote "almost" because you're returning a reflect.Value, not a map. But if a reflect.Value is passed to the fmt package, it prints the value wrapped inside it:
If the operand is a reflect.Value, the operand is replaced by the concrete value that it holds, and printing continues with the next rule.
So you should call Value.Interface() on out before returning it.
It's easier to return early if the kind is not map, so you can create the map right after that:
func ReverseMap(in interface{}) interface{} {
v := reflect.ValueOf(in)
if v.Kind() != reflect.Map {
return nil
}
mapType := reflect.MapOf(v.Type().Elem(), v.Type().Key())
out := reflect.MakeMap(mapType)
for _, key := range v.MapKeys() {
out.SetMapIndex(v.MapIndex(key), key)
}
return out.Interface()
}
Try this variant on the Go Playground.
Another approach may be using Value.MapRange():
for iter := v.MapRange(); iter.Next(); {
out.SetMapIndex(iter.Value(), iter.Key())
}
Try this variant on the Go Playground.
Now i have a map with only one write/delete goroutine and many read goroutines, there are some solutions upon Map with concurrent access, such as RWMutex, sync.map, concurrent-map, sync.atomic, sync.Value, what's the best choice for me?
RWMutex's read lock is a little redundant
sync.map and concurrent-map focus on many write goroutines
Your question is a little vague - so I'll break it down.
What form of concurrent access should I use for a map?
The choice depends on the performance you require from the map. I would opt for a simple mutex (or a RWMutex) based approach.
Granted, you can get better performance from a concurrent map. sync.Mutex locks all of a maps buckets, whereas in a concurrent map, each bucket has it's own sync.Mutex.
Again - it all depends on the scale of your program and the performance you require.
How would I use a mutex for concurrent access?
To ensure the map is being used correctly, you can wrap this in a struct.
type Store struct {
Data map[T]T
}
This a more object-oriented solution, but it allows us to make sure any read/writes are performed concurrently. As well as this, we can easily store other information that may be useful for debugging or security, such as author.
Now, we would implement this with a set of methods like so:
mux sync.Mutex
// New initialises a Store type with an empty map
func New(t, h uint) *Store {
return &Store{
Data: map[T]T{},
}
}
// Insert adds a new key i to the store and places the value of x at this location
// If there is an error, this is returned - if not, this is nil
func (s *Store) Insert(i, x T) error {
mux.Lock()
defer mux.Unlock()
_, ok := s.Data[i]
if ok {
return fmt.Errorf("index %s already exists; use update", i)
}
s.Data[i] = x
return nil
}
// Update changes the value found at key i to x
// If there is an error, this is returned - if not, this is nil
func (s *Store) Update(i, x T) error {
mux.Lock()
defer mux.Unlock()
_, ok := s.Data[i]
if !ok {
return fmt.Errorf("value at index %s does not exist; use insert", i)
}
s.Data[i] = x
return nil
}
// Fetch returns the value found at index i in the store
// If there is an error, this is returned - if not, this is nil
func (s *Store) Fetch(i T) (T, error) {
mux.Lock()
defer mux.Unlock()
v, ok := s.Data[i]
if !ok {
return "", fmt.Errorf("no value for key %s exists", i)
}
return v, nil
}
// Delete removes the index i from store
// If there is an error, this is returned - if not, this is nil
func (s *Store) Delete(i T) (T, error) {
mux.Lock()
defer mux.Unlock()
v, ok := s.Data[i]
if !ok {
return "", fmt.Errorf("index %s already empty", i)
}
delete(s.Data, i)
return v, nil
}
In my solution, I've used a simple sync.Mutex - but you can simply change this code to accommodate RWMutex.
I recommend you take a look at How to use RWMutex in Golang?.
My scan is not updating its destination variable. I sort of got it working with:
ValueName := reflect.New(reflect.ValueOf(value).Elem().Type())
But I don't think it is working the way I want.
func (self LightweightQuery) Execute(incrementedValue interface{}) {
existingObj := reflect.New(reflect.ValueOf(incrementedValue).Elem().Type())
if session, err := connection.GetRandomSession(); err != nil {
panic(err)
} else {
// buildSelect just generates a select query, I have test the query and it comes back with results.
query := session.Query(self.buildSelect(incrementedValue))
bindQuery := cqlr.BindQuery(query)
logger.Error("Existing obj ", existingObj)
for bindQuery.Scan(&existingObj) {
logger.Error("Existing obj ", existingObj)
....
}
}
}
Both log messages are the exact same Existing obj &{ 0 0 0 0 0 0 0 0 0 0 0 0} (Spaces are string fields.) Is this because of the heavy use of reflection to generate a new object? In their docs it says I should use var ValueName type to define my destination but I cannot seem to do that with reflection. I realize this may be silly, but maybe even just pointing me in the direction for further debugging this would be great. My skills with Go are quite lacking!
What is it you want exactly? Do you want to update a variable you pass to Execute()?
If so, you have to pass a pointer to Execute(). And then you only need to pass reflect.ValueOf(incrementedValue).Interface() to Scan(). This works because reflect.ValueOf(incrementedValue) is a reflect.Value holding an interface{} (the type of your parameter) which holds a pointer (the pointer you pass to Execute()), and Value.Interface() will return a value of type interface{} holding the pointer, the exact thing you have to pass Scan().
See this example (which uses fmt.Sscanf(), but concept is the same):
func main() {
i := 0
Execute(&i)
fmt.Println(i)
}
func Execute(i interface{}) {
fmt.Sscanf("1", "%d", reflect.ValueOf(i).Interface())
}
It will print 1 from main(), as the value 1 is set inside Execute().
If you don't want to update the variable passed to Execute(), just create a new value with identical type, since you're using reflect.New() which returns the Value of a pointer, you have to pass existingObj.Interface() which returns an interface{} holding the pointer, the thing you want to pass to Scan(). (What you did is you passed a pointer to a reflect.Value to Scan() which is not something Scan() expects.)
Demonstration with fmt.Sscanf():
func main() {
i := 0
Execute2(&i)
}
func Execute2(i interface{}) {
o := reflect.New(reflect.ValueOf(i).Elem().Type())
fmt.Sscanf("2", "%d", o.Interface())
fmt.Println(o.Elem().Interface())
}
This will print 2.
Another variant of Execute2() is that if you call Interface() right on the value returned by reflect.New():
func Execute3(i interface{}) {
o := reflect.New(reflect.ValueOf(i).Elem().Type()).Interface()
fmt.Sscanf("3", "%d", o)
fmt.Println(*(o.(*int))) // type assertion to extract pointer for printing purposes
}
This Execute3() will print 3 as expected.
Try all examples on the Go Playground.
I fail to understand how to correctly assure that something is not nil in this case:
package main
type shower interface {
getWater() []shower
}
type display struct {
SubDisplay *display
}
func (d display) getWater() []shower {
return []shower{display{}, d.SubDisplay}
}
func main() {
// SubDisplay will be initialized with null
s := display{}
// water := []shower{nil}
water := s.getWater()
for _, x := range water {
if x == nil {
panic("everything ok, nil found")
}
// First iteration display{} is not nil and will
// therefore work, on the second iteration
// x is nil, and getWater panics.
x.getWater()
}
}
The only way I found to check if that value is actually nil is by using reflection.
Is this really wanted behaviour? Or do I fail to see some major mistake in my code?
Play link here
The problem here is that shower is an interface type. Interface types in Go hold the actual value and its dynamic type. More details about this: The Laws of Reflection #The representation of an interface.
The slice you return contains 2 non-nil values. The 2nd value is an interface value, a (value;type) pair holding a nil pointer value and a *display concrete type. Quoting from the Go Language Specification: Comparison operators:
Interface values are comparable. Two interface values are equal if they have identical dynamic types and equal dynamic values or if both have value nil.
So if you compare it to nil, it will be false. If you compare it to an interface value representing the pair (nil;*display), it will be true:
if x == (*display)(nil) {
panic("everything ok, nil found")
}
This seems unfeasible as you'd have to know the actual type the interface holds. But note that you can use reflection to tell if a non-nil interface value wraps a nil value using Value.IsNil(). You can see an example of this on the Go Playground.
Why is it implemented this way?
Interfaces unlike other concrete types (non-interfaces) can hold values of different concrete types (different static types). The runtime needs to know the dynamic or runtime-type of the value stored in a variable of interface type.
An interface is just a method set, any type implements it if the same methods are part of the method set of the type. There are types which cannot be nil, for example a struct or a custom type with int as its underlying type. In these cases you would not need to be able to store a nil value of that specific type.
But any type also includes concrete types where nil is a valid value (e.g. slices, maps, channels, all pointer types), so in order to store the value at runtime that satisfies the interface it is reasonable to support storing nil inside the interface. But besides the nil inside the interface we must store its dynamic type as the nil value does not carry such information. The alternate option would be to use nil as the interface value itself when the value to be stored in it is nil, but this solution is insufficient as it would lose the dynamic type information.
Some people say that Go's interfaces are dynamically typed, but that is misleading. They are statically typed: a variable of interface type always has the same static type, and even though at run time the value stored in the interface variable may change type, that value will always satisfy the interface.
In general if you want to indicate nil for a value of interface type, use explicit nil value and then you can test for nil equality. The most common example is the built-in error type which is an interface with one method. Whenever there is no error, you explicitly set or return the value nil and not the value of some concrete (non-interface) type error variable (which would be really bad practice, see demonstration below).
In your example the confusion arises from the facts that:
you want to have a value as an interface type (shower)
but the value you want to store in the slice is not of type shower but a concrete type
So when you put a *display type into the shower slice, an interface value will be created, which is a pair of (value;type) where value is nil and type is *display. The value inside the pair will be nil, not the interface value itself. If you would put a nil value into the slice, then the interface value itself would be nil and a condition x == nil would be true.
Demonstration
See this example: Playground
type MyErr string
func (m MyErr) Error() string {
return "big fail"
}
func doSomething(i int) error {
switch i {
default:
return nil // == nil
case 1:
var p *MyErr
return p // != nil
case 2:
return (*MyErr)(nil) // != nil
case 3:
var p *MyErr
return error(p) // != nil because the interface points to a
// nil item but is not nil itself.
case 4:
var err error // == nil: zero value is nil for the interface
return err // This will be true because err is already interface type
}
}
func main() {
for i := 0; i <= 4; i++ {
err := doSomething(i)
fmt.Println(i, err, err == nil)
}
}
Output:
0 <nil> true
1 <nil> false
2 <nil> false
3 <nil> false
4 <nil> true
In case 2 a nil pointer is returned but first it is converted to an interface type (error) so an interface value is created which holds a nil value and the type *MyErr, so the interface value is not nil.
Let's think of an interface as a pointer.
Say you have a pointer a and it's nil, pointing to nothing.
var a *int // nil
Then you have a pointer b and it's pointing to a.
var b **int
b = &a // not nil
See what happened? b points to a pointer that points to nothing. So even if it's a nil pointer at the end of the chain, b does point to something - it isn't nil.
If you'd peek at the process' memory, it might look like this:
address | name | value
1000000 | a | 0
2000000 | b | 1000000
See? a is pointing to address 0 (which means it's nil), and b is pointing to the address of a (1000000).
The same applies to interfaces (except that they look a bit different in memory).
Like a pointer, an interface pointing to a nil pointer would not be nil itself.
Here, see for yourself how this works with pointers and how it works with interfaces.
I'll take an alternative route to answer your concrete question, by providing the exact answer you were looking for:
Replace the check:
if x == nil {
panic("everything is ok. nil found")
}
with:
if _, ok := x.(display); !ok {
panic("everything is ok. nil found")
}
The idea here is that we are trying to convert the interface type (shower) to the concrete type display. Obviously the second slice item (d.SubDisplay) is not.
I have some trouble building a function that can dynamically use parametrized structs. For that reason my code has 20+ functions that are similar except basically for one type that gets used. Most of my experience is with Java, and I'd just develop basic generic functions, or use plain Object as parameter to function (and reflection from that point on). I would need something similar, using Go.
I have several types like:
// The List structs are mostly needed for json marshalling
type OrangeList struct {
Oranges []Orange
}
type BananaList struct {
Bananas []Banana
}
type Orange struct {
Orange_id string
Field_1 int
// The fields are different for different types, I am simplifying the code example
}
type Banana struct {
Banana_id string
Field_1 int
// The fields are different for different types, I am simplifying the code example
}
Then I have function, basically for each list type:
// In the end there are 20+ of these, the only difference is basically in two types!
// This is very un-DRY!
func buildOranges(rows *sqlx.Rows) ([]byte, error) {
oranges := OrangeList{} // This type changes
for rows.Next() {
orange := Orange{} // This type changes
err := rows.StructScan(&orange) // This can handle each case already, could also use reflect myself too
checkError(err, "rows.Scan")
oranges.Oranges = append(oranges.Oranges,orange)
}
checkError(rows.Err(), "rows.Err")
jsontext, err := json.Marshal(oranges)
return jsontext, err
}
Yes, I could change the sql library to use more intelligent ORM or framework, but that's besides the point. I want to learn on how to build generic function that can handle similar function for all my different types.
I got this far, but it still doesn't work properly (target isn't expected struct I think):
func buildWhatever(rows *sqlx.Rows, tgt interface{}) ([]byte, error) {
tgtValueOf := reflect.ValueOf(tgt)
tgtType := tgtValueOf.Type()
targets := reflect.SliceOf(tgtValueOf.Type())
for rows.Next() {
target := reflect.New(tgtType)
err := rows.StructScan(&target) // At this stage target still isn't 1:1 smilar struct so the StructScan fails... It's some perverted "Value" object instead. Meh.
// Removed appending to the list because the solutions for that would be similar
checkError(err, "rows.Scan")
}
checkError(rows.Err(), "rows.Err")
jsontext, err := json.Marshal(targets)
return jsontext, err
}
So umm, I would need to give the list type, and the vanilla type as parameters, then build one of each, and the rest of my logic would be probably fixable quite easily.
Turns out there's an sqlx.StructScan(rows, &destSlice) function that will do your inner loop, given a slice of the appropriate type. The sqlx docs refer to caching results of reflection operations, so it may have some additional optimizations compared to writing one.
Sounds like the immediate question you're actually asking is "how do I get something out of my reflect.Value that rows.StructScan will accept?" And the direct answer is reflect.Interface(target); it should return an interface{} representing an *Orange you can pass directly to StructScan (no additional & operation needed). Then, I think targets = reflect.Append(targets, target.Indirect()) will turn your target into a reflect.Value representing an Orange and append it to the slice. targets.Interface() should get you an interface{} representing an []Orange that json.Marshal understands. I say all these 'should's and 'I think's because I haven't tried that route.
Reflection, in general, is verbose and slow. Sometimes it's the best or only way to get something done, but it's often worth looking for a way to get your task done without it when you can.
So, if it works in your app, you can also convert Rows straight to JSON, without going through intermediate structs. Here's a sample program (requires sqlite3 of course) that turns sql.Rows into map[string]string and then into JSON. (Note it doesn't try to handle NULL, represent numbers as JSON numbers, or generally handle anything that won't fit in a map[string]string.)
package main
import (
_ "code.google.com/p/go-sqlite/go1/sqlite3"
"database/sql"
"encoding/json"
"os"
)
func main() {
db, err := sql.Open("sqlite3", "foo")
if err != nil {
panic(err)
}
tryQuery := func(query string, args ...interface{}) *sql.Rows {
rows, err := db.Query(query, args...)
if err != nil {
panic(err)
}
return rows
}
tryQuery("drop table if exists t")
tryQuery("create table t(i integer, j integer)")
tryQuery("insert into t values(?, ?)", 1, 2)
tryQuery("insert into t values(?, ?)", 3, 1)
// now query and serialize
rows := tryQuery("select * from t")
names, err := rows.Columns()
if err != nil {
panic(err)
}
// vals stores the values from one row
vals := make([]interface{}, 0, len(names))
for _, _ = range names {
vals = append(vals, new(string))
}
// rowMaps stores all rows
rowMaps := make([]map[string]string, 0)
for rows.Next() {
rows.Scan(vals...)
// now make value list into name=>value map
currRow := make(map[string]string)
for i, name := range names {
currRow[name] = *(vals[i].(*string))
}
// accumulating rowMaps is the easy way out
rowMaps = append(rowMaps, currRow)
}
json, err := json.Marshal(rowMaps)
if err != nil {
panic(err)
}
os.Stdout.Write(json)
}
In theory, you could build this to do fewer allocations by not reusing the same rowMap each time and using a json.Encoder to append each row's JSON to a buffer. You could go a step further and not use a rowMap at all, just the lists of names and values. I should say I haven't compared the speed against a reflect-based approach, though I know reflect is slow enough it might be worth comparing them if you can put up with either strategy.