What is the shortest (and idiomatic) way to create an array from the keys and values of a map w/o compromising on time complexity too much?
For instance, from the following map:
map[string]string { "1":"a", "2":"b" }
I need to create the following array:
[]string{"1","a", "2","b"}
I can do this in Scala with following:
val myMap = Map("1" -> "a", "2" -> "b")
myMap.keySet ++ myMap.values
Thank you.
Simplest way would be to just iterate the map, since in Go the syntax would allow direct access to keys and values and dump them into the array.
m := map[string]string { "1":"a", "2":"b" }
arr := []string{}
for k, v := range m {
arr = append(arr, k, v)
}
One caveat here: In Go, map iteration order is randomized, as you can see here, under "Iteration Order":
https://blog.golang.org/go-maps-in-action
So if you want your resulting array to have a particular ordering, you should first dump the keys and order (as shown in that same blog entry).
Playground (without the sorting part):
https://play.golang.org/p/mCe6eEy25A
Related
First, my definition of composite key - two ore more values combine to make the key. Not to confuse with composite keys in databases.
My goal is to save computed values of pow(x, y) in a hash table, where x and y are integers. This is where I need ideas on how to make a key, so that given x and y, I can look it up in the hash table, to find pow(x,y).
For example:
pow(2, 3) => {key(2,3):8}
What I want to figure out is how to get the map key for the pair (2,3), i.e. the best way to generate a key which is a combination of multiple values, and use it in hash table.
The easiest and most flexible way is to use a struct as the key type, including all the data you want to be part of the key, so in your case:
type Key struct {
X, Y int
}
And that's all. Using it:
m := map[Key]int{}
m[Key{2, 2}] = 4
m[Key{2, 3}] = 8
fmt.Println("2^2 = ", m[Key{2, 2}])
fmt.Println("2^3 = ", m[Key{2, 3}])
Output (try it on the Go Playground):
2^2 = 4
2^3 = 8
Spec: Map types: You may use any types as the key where the comparison operators == and != are fully defined, and the above Key struct type fulfills this.
Spec: Comparison operators: Struct values are comparable if all their fields are comparable. Two struct values are equal if their corresponding non-blank fields are equal.
One important thing: you should not use a pointer as the key type (e.g. *Key), because comparing pointers only compares the memory address, and not the pointed values.
Also note that you could also use arrays (not slices) as key type, but arrays are not as flexible as structs. You can read more about this here: Why have arrays in Go?
This is how it would look like with arrays:
type Key [2]int
m := map[Key]int{}
m[Key{2, 2}] = 4
m[Key{2, 3}] = 8
fmt.Println("2^2 = ", m[Key{2, 2}])
fmt.Println("2^3 = ", m[Key{2, 3}])
Output is the same. Try it on the Go Playground.
Go can't make a hash of a slice of ints.
Therefore the way I would approach this is mapping a struct to a number.
Here is an example of how that could be done:
package main
import (
"fmt"
)
type Nums struct {
num1 int
num2 int
}
func main() {
powers := make(map[Nums]int)
numbers := Nums{num1: 2, num2: 4}
powers[numbers] = 6
fmt.Printf("%v", powers[input])
}
I hope that helps
Your specific problem is nicely solved by the other answers. I want to add an additional trick that may be useful in some corner cases.
Given that map keys must be comparable, you can also use interfaces. Interfaces are comparable if their dynamic values are comparable.
This allows you to essentially partition the map, i.e. to use multiple types of keys within the same data structure. For example if you want to store in your map n-tuples (it wouldn't work with arrays, because the array length is part of the type).
The idea is to define an interface with a dummy method (but it can surely be not dummy at all), and use that as map key:
type CompKey interface {
isCompositeKey() bool
}
var m map[CompKey]string
At this point you can have arbitrary types implementing the interface, either explicitly or by just embedding it.
In this example, the idea is to make the interface method unexported so that other structs may just embed the interface without having to provide an actual implementation — the method can't be called from outside its package. It will just signal that the struct is usable as a composite map key.
type AbsoluteCoords struct {
CompKey
x, y int
}
type RelativeCoords struct {
CompKey
x, y int
}
func foo() {
p := AbsoluteCoords{x: 1, y: 2}
r := RelativeCoords{x: 10, y: 20}
m[p] = "foo"
m[r] = "bar"
fmt.Println(m[AbsoluteCoords{x: 10, y: 20}]) // "" (empty, types don't match)
fmt.Println(m[RelativeCoords{x: 10, y: 20}]) // "bar" (matches, key present)
}
Of course nothing stops you from declaring actual methods on the interface, that may be useful when ranging over the map keys.
The disadvantage of this interface key is that it is now your responsibility to make sure the implementing types are actually comparable. E.g. this map key will panic:
type BadKey struct {
CompKey
nonComparableSliceField []int
}
b := BadKey{nil, []int{1,2}}
m[b] = "bad!" // panic: runtime error: hash of unhashable type main.BadKey
All in all, this might be an interesting approach when you need to keep two sets of K/V pairs in the same map, e.g. to keep some sanity in function signatures or to avoid defining structs with N very similar map fields.
Playground https://play.golang.org/p/0t7fcvSWdy7
I have a struct v with members A, B, C string. Using reflection, I can get the names of the fields and their values:
typ := v.Type()
for i := 0; i < v.NumField(); i++ {
// gets us a StructField
fi := typ.Field(i)
fieldname := fi.Name
fmt.Println(fieldname)
val := fmt.Sprintf("%v", v.Field(i).Interface())
}
since I have the name, and can get the value OUT, can I assign new values to the fields? I would like to do essentially:
v.Field(fieldname).Interface() = "new value"
but that obviously doesn't work. Is it possible to assign a value into a struct if you only know the name of the field?
In practice, I'm trying to assign values from a map[string]string to corresponding fields in a struct, where the struct and map definitions may expand of change over time, and the map may contain more, or less, values than the struct. I've considered doing it w/JSON, but that approach leaves me a little cold, seeing as how easy it was to use reflection to get "almost" there!
Thanks,
Ken
Yes, it is possible.
Introduction
Since you want to access and modify the value of a variable (or field), you need to use the reflect.Value type instead of reflect.Type. You can acquire it with reflect.ValueOf(). Also in order to modify it with reflection, you need to pass the address (a pointer) of the struct or value you want to modify (else you could only read it but not modify it).
But you don't want to modify the address/pointer but the pointed value, so you have to "navigate" from the Value of the pointer to the Value of the pointed variable (struct), this is what Value.Elem() is for. It looks like this: reflect.ValueOf(&s).Elem()
You can get the Value of a struct field with the Value.FieldByName() method, which since we passed the address of the pointer to the ValueOf() function will be settable.
The Code
The code is much simpler than the introduction once you understand it. You can also try it on the Go Playground:
var s struct {
A, B, C string
}
s.A, s.B, s.C = "a1", "b2", "c3"
fmt.Println("Before: ", s)
v := reflect.ValueOf(&s).Elem()
v.FieldByName("A").SetString("2a")
v.FieldByName("B").SetString("2b")
v.FieldByName("C").SetString("2c")
fmt.Println("After: ", s)
// Using a map:
m := map[string]string{"A": "ma", "B": "mb", "C": "mc"}
for mk, mv := range m {
v.FieldByName(mk).SetString(mv)
}
fmt.Println("From Map:", s)
Output:
Before: {a1 b2 c3}
After: {2a 2b 2c}
From Map: {ma mb mc}
I recommend to read this blog post to learn the basics of the reflection in Go:
The Laws of Reflection
How can one remove selected keys from a map?
Is it safe to combine delete() with range, as in the code below?
package main
import "fmt"
type Info struct {
value string
}
func main() {
table := make(map[string]*Info)
for i := 0; i < 10; i++ {
str := fmt.Sprintf("%v", i)
table[str] = &Info{str}
}
for key, value := range table {
fmt.Printf("deleting %v=>%v\n", key, value.value)
delete(table, key)
}
}
https://play.golang.org/p/u1vufvEjSw
This is safe! You can also find a similar sample in Effective Go:
for key := range m {
if key.expired() {
delete(m, key)
}
}
And the language specification:
The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next. If map entries that have not yet been reached are removed during iteration, the corresponding iteration values will not be produced. If map entries are created during iteration, that entry may be produced during the iteration or may be skipped. The choice may vary for each entry created and from one iteration to the next. If the map is nil, the number of iterations is 0.
Sebastian's answer is accurate, but I wanted to know why it was safe, so I did some digging into the Map source code. It looks like on a call to delete(k, v), it basically just sets a flag (as well as changing the count value) instead of actually deleting the value:
b->tophash[i] = Empty;
(Empty is a constant for the value 0)
What the map appears to actually be doing is allocating a set number of buckets depending on the size of the map, which grows as you perform inserts at the rate of 2^B (from this source code):
byte *buckets; // array of 2^B Buckets. may be nil if count==0.
So there are almost always more buckets allocated than you're using, and when you do a range over the map, it checks that tophash value of each bucket in that 2^B to see if it can skip over it.
To summarize, the delete within a range is safe because the data is technically still there, but when it checks the tophash it sees that it can just skip over it and not include it in whatever range operation you're performing. The source code even includes a TODO:
// TODO: consolidate buckets if they are mostly empty
// can only consolidate if there are no live iterators at this size.
This explains why using the delete(k,v) function doesn't actually free up memory, just removes it from the list of buckets you're allowed to access. If you want to free up the actual memory you'll need to make the entire map unreachable so that garbage collection will step in. You can do this using a line like
map = nil
I was wondering if a memory leak could happen. So I wrote a test program:
package main
import (
log "github.com/Sirupsen/logrus"
"os/signal"
"os"
"math/rand"
"time"
)
func main() {
log.Info("=== START ===")
defer func() { log.Info("=== DONE ===") }()
go func() {
m := make(map[string]string)
for {
k := GenerateRandStr(1024)
m[k] = GenerateRandStr(1024*1024)
for k2, _ := range m {
delete(m, k2)
break
}
}
}()
osSignals := make(chan os.Signal, 1)
signal.Notify(osSignals, os.Interrupt)
for {
select {
case <-osSignals:
log.Info("Recieved ^C command. Exit")
return
}
}
}
func GenerateRandStr(n int) string {
rand.Seed(time.Now().UnixNano())
const letterBytes = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
b := make([]byte, n)
for i := range b {
b[i] = letterBytes[rand.Int63() % int64(len(letterBytes))]
}
return string(b)
}
Looks like GC do frees the memory. So it's okay.
In short, yes. See previous answers.
And also this, from here:
ianlancetaylor commented on Feb 18, 2015
I think the key to understanding this is to realize that while executing the body of a for/range statement, there is no current iteration. There is a set of values that have been seen, and a set of values that have not been seen. While executing the body, one of the key/value pairs that has been seen--the most recent pair--was assigned to the variable(s) of the range statement. There is nothing special about that key/value pair, it's just one of the ones that has already been seen during the iteration.
The question he's answering is about modifying map elements in place during a range operation, which is why he mentions the "current iteration". But it's also relevant here: you can delete keys during a range, and that just means that you won't see them later on in the range (and if you already saw them, that's okay).
I have a recursive function that creates objects representing file paths (the keys are paths and the values are info about the file). It's recursive as it's only meant to handle files, so if a directory is encountered, the function is recursively called on the directory.
All that being said, I'd like to do the equivalent of a set union on two maps (i.e. the "main" map updated with the values from the recursive call). Is there an idiomatic way to do this aside from iterating over one map and assigning each key, value in it to the same thing in the other map?
That is: given a,b are of type map [string] *SomeObject, and a and b are eventually populated, is there any way to update a with all the values in b?
There is no built in way, nor any method in the standard packages to do such a merge.
The idomatic way is to simply iterate:
for k, v := range b {
a[k] = v
}
Since Go 1.18, you can simply use the Copy function from the golang.org/x/exp/maps package:
package main
import (
"fmt"
"golang.org/x/exp/maps"
)
func main() {
src := map[string]int{
"one": 1,
"two": 2,
}
dst := map[string]int{
"two": 2,
"three": 3,
}
maps.Copy(dst, src)
fmt.Println("src:", src)
fmt.Println("dst:", dst)
}
(Playground)
Output:
src: map[one:1 two:2]
dst: map[one:1 three:3 two:2]
One caveat of this approach is that, in Go versions 1.18.x to 1.19.x, your map's key type must be concrete, i.e. not an interface type. For instance, the compiler won't allow you to pass values of type map[io.Reader]int to the Copy function:
package main
import (
"fmt"
"io"
"golang.org/x/exp/maps"
)
func main() {
var src, dst map[io.Reader]int
maps.Copy(dst, src)
fmt.Println("src:", src)
fmt.Println("dst:", dst)
}
(Playground)
Compiler output:
go: finding module for package golang.org/x/exp/maps
go: downloading golang.org/x/exp v0.0.0-20220328175248-053ad81199eb
./prog.go:12:11: io.Reader does not implement comparable
Go build failed.
This limitation was lifted in Go 1.20.
(Playground)
Starting at go 1.18, thanks to the release of the Generics feature, there are now generic functions that union maps!
You can use a package like https://github.com/samber/lo in order to do so.
Note that the key can be of any "comparable" type, while the value can be of any type.
Example:
package main
import (
"fmt"
"github.com/samber/lo"
)
func main() {
map1 := map[string]interface{}{"k1": "v1", "k2": 2}
map2 := map[string]interface{}{"k2": "v2new", "k3": true}
map1 = lo.Assign(map1, map2)
fmt.Printf("%v", map1)
}
The result is:
map[k1:v1 k2:v2new k3:true]
If you have a couple of nested maps, left and right, then this function will recursively add the items from right into left. If the key is already in left then we recurse deeper into the structure and attempt only add keys to left (e.g. never replace them).
type m = map[string]interface{}
// Given two maps, recursively merge right into left, NEVER replacing any key that already exists in left
func mergeKeys(left, right m) m {
for key, rightVal := range right {
if leftVal, present := left[key]; present {
//then we don't want to replace it - recurse
left[key] = mergeKeys(leftVal.(m), rightVal.(m))
} else {
// key not in left so we can just shove it in
left[key] = rightVal
}
}
return left
}
NOTE: I do not handle the case in which the value is not itself a map[string]interface{}. So if you have left["x"] = 1 and right["x"] = 2 then the above code will panic when attempting leftVal.(m).
Here is another option,
in case you are trying to limit the number of third-party dependencies such github.com/samber/lo, OR
you are not comfortable with the experimental nature of golang.org/x/exp (read the warning), OR
you would rather the convenience of an append()-like API instead of exp.Copy() from golang.org/x/exp (append accepts any number of lists, whereas Copy() accepts only 2).
However it requires Go 1.18+ as it uses go generics.
Save the following in one of your modules/packages:
func MergeMaps[M ~map[K]V, K comparable, V any](src ...M) M {
merged := make(M)
for _, m := range src {
for k, v := range m {
merged[k] = v
}
}
return merged
}
Then you can use it very similarly to append():
func main() {
mergedMaps := MergeMaps(
map[string]int{"a": 1, "b": 2},
map[string]int{"b": 3, "c": 4},
map[string]int{"c": 3, "d": 4},
)
fmt.Println(mergedMaps)
}
Go is limited by what type of map it is. I'd suspect that there isn't built in functions because of the infinite number of type declarations that could exist for a map. So you have to build your own Merge functions depending on what type of map you are using:
func MergeJSONMaps(maps ...map[string]interface{}) (result map[string]interface{}) {
result = make(map[string]interface{})
for _, m := range maps {
for k, v := range m {
result[k] = v
}
}
return result
}
How does one delete data from a map in Go? For example, having
m := map[string]string{ "key1":"val1", "key2":"val2" };
I want to make m drop the "key1" without copying the entire map by iterating over its keys (which could get big in some uses). Is it enough to assign a nil value to "key1", or will that still keep the key in the map structure with an assigned value of nil? That is, if I later iterate over the keys of the map, will "key1" appear?
Deletion of map elements
The built-in function delete removes the element with key k from a map
m.
delete(m, k) // remove element m[k] from map m
For example,
package main
import "fmt"
func main() {
m := map[string]string{"key1": "val1", "key2": "val2"}
fmt.Println(m)
delete(m, "key1")
fmt.Println(m)
}
Output:
map[key1:val1 key2:val2]
map[key2:val2]