In short: How do I traverse a map in sorted key order, regardless of the map's type?
I found a few related questions, the closest one suggesting that it can't be done without relying on the reflect module. Is this understanding correct?
Consider this Go code, which traverses two maps of different types, in sorted order of their keys:
mapOne := map[int]string {
1: "a",
2: "b",
3: "c",
}
keysOne := make([]int, 0, len(mapOne))
for key, _ := range mapOne {
keysOne = append(keysOne, key)
}
sort.Ints(keysOne)
for _, key := range keysOne {
value := mapOne[key]
fmt.Println(key, value)
}
mapTwo := map[string]int {
"a": 1,
"b": 2,
"c": 3,
}
keysTwo := make([]string, 0, len(mapTwo))
for key, _ := range mapTwo {
keysTwo = append(keysTwo, key)
}
sort.Strings(keysTwo)
for _, key := range keysTwo {
value := mapTwo[key]
fmt.Println(key, value)
}
The logic to extract the keys and then sort them is duplicated for the two
different map types. Is there any way to factor out this logic and avoid
duplication?
I got stuck trying to write an interface to provide a SortedKeys method. In
particular, the return type of SortedKeys depends on the type of the map,
and I can't figure out how to express that in Go.
I think whoever told you you'd need reflect was correct; that's probably overkill though. I think the duplication is acceptable here.
(alternatively, you could implement your own map that uses some kind of interface for keys, but you'd still end up needing to make a type that satisfies the interface for each underlying key type)
Related
First, my definition of composite key - two ore more values combine to make the key. Not to confuse with composite keys in databases.
My goal is to save computed values of pow(x, y) in a hash table, where x and y are integers. This is where I need ideas on how to make a key, so that given x and y, I can look it up in the hash table, to find pow(x,y).
For example:
pow(2, 3) => {key(2,3):8}
What I want to figure out is how to get the map key for the pair (2,3), i.e. the best way to generate a key which is a combination of multiple values, and use it in hash table.
The easiest and most flexible way is to use a struct as the key type, including all the data you want to be part of the key, so in your case:
type Key struct {
X, Y int
}
And that's all. Using it:
m := map[Key]int{}
m[Key{2, 2}] = 4
m[Key{2, 3}] = 8
fmt.Println("2^2 = ", m[Key{2, 2}])
fmt.Println("2^3 = ", m[Key{2, 3}])
Output (try it on the Go Playground):
2^2 = 4
2^3 = 8
Spec: Map types: You may use any types as the key where the comparison operators == and != are fully defined, and the above Key struct type fulfills this.
Spec: Comparison operators: Struct values are comparable if all their fields are comparable. Two struct values are equal if their corresponding non-blank fields are equal.
One important thing: you should not use a pointer as the key type (e.g. *Key), because comparing pointers only compares the memory address, and not the pointed values.
Also note that you could also use arrays (not slices) as key type, but arrays are not as flexible as structs. You can read more about this here: Why have arrays in Go?
This is how it would look like with arrays:
type Key [2]int
m := map[Key]int{}
m[Key{2, 2}] = 4
m[Key{2, 3}] = 8
fmt.Println("2^2 = ", m[Key{2, 2}])
fmt.Println("2^3 = ", m[Key{2, 3}])
Output is the same. Try it on the Go Playground.
Go can't make a hash of a slice of ints.
Therefore the way I would approach this is mapping a struct to a number.
Here is an example of how that could be done:
package main
import (
"fmt"
)
type Nums struct {
num1 int
num2 int
}
func main() {
powers := make(map[Nums]int)
numbers := Nums{num1: 2, num2: 4}
powers[numbers] = 6
fmt.Printf("%v", powers[input])
}
I hope that helps
Your specific problem is nicely solved by the other answers. I want to add an additional trick that may be useful in some corner cases.
Given that map keys must be comparable, you can also use interfaces. Interfaces are comparable if their dynamic values are comparable.
This allows you to essentially partition the map, i.e. to use multiple types of keys within the same data structure. For example if you want to store in your map n-tuples (it wouldn't work with arrays, because the array length is part of the type).
The idea is to define an interface with a dummy method (but it can surely be not dummy at all), and use that as map key:
type CompKey interface {
isCompositeKey() bool
}
var m map[CompKey]string
At this point you can have arbitrary types implementing the interface, either explicitly or by just embedding it.
In this example, the idea is to make the interface method unexported so that other structs may just embed the interface without having to provide an actual implementation — the method can't be called from outside its package. It will just signal that the struct is usable as a composite map key.
type AbsoluteCoords struct {
CompKey
x, y int
}
type RelativeCoords struct {
CompKey
x, y int
}
func foo() {
p := AbsoluteCoords{x: 1, y: 2}
r := RelativeCoords{x: 10, y: 20}
m[p] = "foo"
m[r] = "bar"
fmt.Println(m[AbsoluteCoords{x: 10, y: 20}]) // "" (empty, types don't match)
fmt.Println(m[RelativeCoords{x: 10, y: 20}]) // "bar" (matches, key present)
}
Of course nothing stops you from declaring actual methods on the interface, that may be useful when ranging over the map keys.
The disadvantage of this interface key is that it is now your responsibility to make sure the implementing types are actually comparable. E.g. this map key will panic:
type BadKey struct {
CompKey
nonComparableSliceField []int
}
b := BadKey{nil, []int{1,2}}
m[b] = "bad!" // panic: runtime error: hash of unhashable type main.BadKey
All in all, this might be an interesting approach when you need to keep two sets of K/V pairs in the same map, e.g. to keep some sanity in function signatures or to avoid defining structs with N very similar map fields.
Playground https://play.golang.org/p/0t7fcvSWdy7
What is the shortest (and idiomatic) way to create an array from the keys and values of a map w/o compromising on time complexity too much?
For instance, from the following map:
map[string]string { "1":"a", "2":"b" }
I need to create the following array:
[]string{"1","a", "2","b"}
I can do this in Scala with following:
val myMap = Map("1" -> "a", "2" -> "b")
myMap.keySet ++ myMap.values
Thank you.
Simplest way would be to just iterate the map, since in Go the syntax would allow direct access to keys and values and dump them into the array.
m := map[string]string { "1":"a", "2":"b" }
arr := []string{}
for k, v := range m {
arr = append(arr, k, v)
}
One caveat here: In Go, map iteration order is randomized, as you can see here, under "Iteration Order":
https://blog.golang.org/go-maps-in-action
So if you want your resulting array to have a particular ordering, you should first dump the keys and order (as shown in that same blog entry).
Playground (without the sorting part):
https://play.golang.org/p/mCe6eEy25A
I have a map with complex keys - for example, 2D arrays:
m := make(map[[2][3]int]int)
When I insert a new key into the map, does Go make a deep copy of the key?
a := [2][3]int{{1, 2, 3}, {4, 5, 6}}
m[a] = 1
In other words, if I change the array a after using it as a map key, does the map still contain the old value of a?
Short answer, it is copied.
By specification, Arrays are value types.
Go's arrays are values. An array variable denotes the entire array; it is not a pointer to the first array element (as would be the case in C). This means that when you assign or pass around an array value you will make a copy of its contents. (To avoid the copy you could pass a pointer to the array, but then that's a pointer to an array, not an array.)
https://blog.golang.org/go-slices-usage-and-internals
See for yourself:
https://play.golang.org/p/fEUYWwN-pm
package main
import (
"fmt"
)
func main() {
m := make(map[[2][3]int]int)
a := [2][3]int{{1, 2, 3}, {4, 5, 6}}
fmt.Printf("Pointer to a: %p\n", &a)
m[a] = 1
for k, _ := range m {
fmt.Printf("Pointer to k: %p\n", &k)
}
}
The pointers do not match.
EDIT: The real reason is when inserting into a map, the key value is copied. Or, you can continue to just remember the rule above: arrays are value types and their reuse denotes a copy. Either works here. :)
Arrays are always passed by value, so, yes in this case Go will make a deep copy of the key.
From the language spec
The comparison operators == and != must be fully defined for operands of the key type; thus the key type must not be a function, map, or slice. If the key type is an interface type, these comparison operators must be defined for the dynamic key values; failure will cause a run-time panic.
The keys are copied into the map. Excluding map and slice as valid keys means that the keys can't change. Note that go doesn't follow pointers if you define a map type with a pointer as a key (eg map[*int]int) it compares the pointers directly.
I have a recursive function that creates objects representing file paths (the keys are paths and the values are info about the file). It's recursive as it's only meant to handle files, so if a directory is encountered, the function is recursively called on the directory.
All that being said, I'd like to do the equivalent of a set union on two maps (i.e. the "main" map updated with the values from the recursive call). Is there an idiomatic way to do this aside from iterating over one map and assigning each key, value in it to the same thing in the other map?
That is: given a,b are of type map [string] *SomeObject, and a and b are eventually populated, is there any way to update a with all the values in b?
There is no built in way, nor any method in the standard packages to do such a merge.
The idomatic way is to simply iterate:
for k, v := range b {
a[k] = v
}
Since Go 1.18, you can simply use the Copy function from the golang.org/x/exp/maps package:
package main
import (
"fmt"
"golang.org/x/exp/maps"
)
func main() {
src := map[string]int{
"one": 1,
"two": 2,
}
dst := map[string]int{
"two": 2,
"three": 3,
}
maps.Copy(dst, src)
fmt.Println("src:", src)
fmt.Println("dst:", dst)
}
(Playground)
Output:
src: map[one:1 two:2]
dst: map[one:1 three:3 two:2]
One caveat of this approach is that, in Go versions 1.18.x to 1.19.x, your map's key type must be concrete, i.e. not an interface type. For instance, the compiler won't allow you to pass values of type map[io.Reader]int to the Copy function:
package main
import (
"fmt"
"io"
"golang.org/x/exp/maps"
)
func main() {
var src, dst map[io.Reader]int
maps.Copy(dst, src)
fmt.Println("src:", src)
fmt.Println("dst:", dst)
}
(Playground)
Compiler output:
go: finding module for package golang.org/x/exp/maps
go: downloading golang.org/x/exp v0.0.0-20220328175248-053ad81199eb
./prog.go:12:11: io.Reader does not implement comparable
Go build failed.
This limitation was lifted in Go 1.20.
(Playground)
Starting at go 1.18, thanks to the release of the Generics feature, there are now generic functions that union maps!
You can use a package like https://github.com/samber/lo in order to do so.
Note that the key can be of any "comparable" type, while the value can be of any type.
Example:
package main
import (
"fmt"
"github.com/samber/lo"
)
func main() {
map1 := map[string]interface{}{"k1": "v1", "k2": 2}
map2 := map[string]interface{}{"k2": "v2new", "k3": true}
map1 = lo.Assign(map1, map2)
fmt.Printf("%v", map1)
}
The result is:
map[k1:v1 k2:v2new k3:true]
If you have a couple of nested maps, left and right, then this function will recursively add the items from right into left. If the key is already in left then we recurse deeper into the structure and attempt only add keys to left (e.g. never replace them).
type m = map[string]interface{}
// Given two maps, recursively merge right into left, NEVER replacing any key that already exists in left
func mergeKeys(left, right m) m {
for key, rightVal := range right {
if leftVal, present := left[key]; present {
//then we don't want to replace it - recurse
left[key] = mergeKeys(leftVal.(m), rightVal.(m))
} else {
// key not in left so we can just shove it in
left[key] = rightVal
}
}
return left
}
NOTE: I do not handle the case in which the value is not itself a map[string]interface{}. So if you have left["x"] = 1 and right["x"] = 2 then the above code will panic when attempting leftVal.(m).
Here is another option,
in case you are trying to limit the number of third-party dependencies such github.com/samber/lo, OR
you are not comfortable with the experimental nature of golang.org/x/exp (read the warning), OR
you would rather the convenience of an append()-like API instead of exp.Copy() from golang.org/x/exp (append accepts any number of lists, whereas Copy() accepts only 2).
However it requires Go 1.18+ as it uses go generics.
Save the following in one of your modules/packages:
func MergeMaps[M ~map[K]V, K comparable, V any](src ...M) M {
merged := make(M)
for _, m := range src {
for k, v := range m {
merged[k] = v
}
}
return merged
}
Then you can use it very similarly to append():
func main() {
mergedMaps := MergeMaps(
map[string]int{"a": 1, "b": 2},
map[string]int{"b": 3, "c": 4},
map[string]int{"c": 3, "d": 4},
)
fmt.Println(mergedMaps)
}
Go is limited by what type of map it is. I'd suspect that there isn't built in functions because of the infinite number of type declarations that could exist for a map. So you have to build your own Merge functions depending on what type of map you are using:
func MergeJSONMaps(maps ...map[string]interface{}) (result map[string]interface{}) {
result = make(map[string]interface{})
for _, m := range maps {
for k, v := range m {
result[k] = v
}
}
return result
}
The Go Programming Language Specification says:
3. The iteration order over maps is not specified. [...]
That's to be expected since a map type can be implemented as a hash table, or as a search tree, or as some other data structure. But how is map actually implemented in Go?
Put differently, what determines the iteration order of the keys in
for k, _ := range m { fmt.Println(k) }
I started wondering about this after I saw that a map with string keys apparently do have a certain iteration order. A program like
package main
import ("fmt"; "time"; "rand")
func main() {
rand.Seed(time.Seconds())
words := [...]string{"foo", "bar", "a", "b", "c", "hello", "world",
"0", "1", "10", "100", "123"}
stringMap := make(map[string]byte)
for i := range rand.Perm(len(words)) {
stringMap[words[i]] = byte(rand.Int())
}
fmt.Print("stringMap keys:")
for k, _ := range stringMap { fmt.Print(" ", k) }
fmt.Println()
}
prints the following on my machine:
stringMap keys: a c b 100 hello foo bar 10 world 123 1 0
regardless of the insertion order.
The equivalent program with a map[byte]byte map also prints the keys in a shuffled order, but here the key order depends on the insertion order.
How is all this implemented? Is the map specialized for integers and for strings?
Map is implemented in Go as a hashmap.
The Go run-time uses a common hashmap implementation which is implemented in C. The only implementation differences between map[string]T and map[byte]T are: hash function, equivalence function and copy function.
Unlike (some) C++ maps, Go maps aren't fully specialized for integers and for strings.
In Go release.r60, the iteration order is independent from insertion order as long as there are no key collisions. If there are collisions, iteration order is affected by insertion order. This holds true regardless of key type. There is no difference between keys of type string and keys of type byte in this respect, so it is only a coincidence that your program always printed the string keys in the same order. The iteration order is always the same unless the map is modified.
However, in the newest Go weekly release (and in Go1 which may be expected to be released this month), the iteration order is randomized (it starts at a pseudo-randomly chosen key, and the hashcode computation is seeded with a pseudo-random number). If you compile your program with the weekly release (and with Go1), the iteration order will be different each time you run your program. That said, running your program an infinite number of times probably wouldn't print all possible permutations of the key set. Example outputs:
stringMap keys: b 0 hello c world 10 1 123 bar foo 100 a
stringMap keys: hello world c 1 10 bar foo 123 100 a b 0
stringMap keys: bar foo 123 100 world c 1 10 b 0 hello a
...
If the specs say the iteration order is not specified then a specific order in specific cases is not ruled out.
The point is one cannot rely on that order in any case, not even in some special case. The implementation is free to change this behavior at any given moment, run time included.
Note that it is not that odd for order to be stable regardless of insertion order if there is a total order over the keys (as there frequently may be if they are of a homogenous type); if nothing else, it can allow efficient searching over keys which generate the same hash.
This may well also reflect a different underlying implementation - in particular, this is something people might want for strings, but for integers you could use a sparse array instead.
To extend #user811773 answer. A semi-random range clause iteration does not mean that the chance of returning each element in a map is also equal. See https://medium.com/i0exception/map-iteration-in-go-275abb76f721 and https://play.golang.org/p/GpSd8i7XZoG.
package main
import "fmt"
type intSet map[int]struct{}
func (s intSet) put(v int) { s[v] = struct{}{} }
func (s intSet) get() (int, bool) {
for k := range s {
return k, true
}
return 0, false
}
func main() {
s := make(intSet)
for i := 0; i < 4; i++ {
s.put(i)
}
counts := make(map[int]int)
for i := 0; i < 1024*1024; i++ {
v, ok := s.get()
if !ok {return}
counts[v]++
}
for k, v := range counts {
fmt.Printf("Value: %v, Count: %v\n", k, v)
}
}
/*
Value: 1, Count: 130752
Value: 2, Count: 130833
Value: 0, Count: 655840
Value: 3, Count: 131151
*/