Given
var dst, src map[K]V
I can copy all entries from src into dst by doing
for k, v := range src {
dst[k] = v
}
Is there a more idiomatic way to do this?
copy only works on slices (and string as a source).
That looks like a perfectly fine way to do this to me. I don't think copying one map into another is common enough to have a one-liner solution.
Using a simple for range loop is the most efficient solution.
Note that a builtin copy could not just copy the memory of src to the address of dst because they may have entirely different memory layout. Maps grow to accommodate the number of items stored in them. So for example if you have a map with a million elements, it occupies a lot more memory than a freshly created new map, and so a builtin copy could not just copy memory without allocating new.
If your map is big, you can speed up copying elements if you may create the destination map having a big-enough capacity to avoid rehashing and reallocation (the initial capacity does not bound its size), e.g.:
dst := make(map[K]V, len(src))
for k, v := range src {
dst[k] = v
}
If performance is not an issue (e.g. you're working with small maps), a general solution may be created using the reflect package:
func MapCopy(dst, src interface{}) {
dv, sv := reflect.ValueOf(dst), reflect.ValueOf(src)
for _, k := range sv.MapKeys() {
dv.SetMapIndex(k, sv.MapIndex(k))
}
}
This solution does not check if the arguments are really maps and if the destination is not nil. Testing it:
m1 := map[int]string{1: "one", 2: "two"}
m2 := map[int]string{}
MapCopy(m2, m1)
fmt.Println(m2)
m3 := map[string]int{"one": 1, "two": 2}
m4 := map[string]int{}
MapCopy(m4, m3)
fmt.Println(m4)
Output (try it on the Go Playground):
map[1:one 2:two]
map[one:1 two:2]
You could use github.com/linkosmos/mapop
input := map[string]interface{}{
"Key1": 2,
"key3": nil,
"val": 2,
"val2": "str",
"val3": 4,
}
input2 := map[string]interface{}{
"a2": "str",
"a3": 4,
}
input = mapop.Merge(input, input2)
input{"Key1": 2, "key3": nil, "val": 2, "val2": "str", "val3": 4, "a2": "str", "a3": 4}
Go 1.18
Use the generic function maps.Copy
Copy copies all key/value pairs in src adding them to dst. When a key in src is already present in dst, the value in dst will be overwritten by the value associated with the key in src.
You use it as such:
package main
import (
"fmt"
"golang.org/x/exp/maps"
)
func main() {
src := map[int]string{200: "foo", 300: "bar"}
dest := map[int]string{}
maps.Copy(dest, src)
fmt.Println(dest) // map[200:foo 300:bar]
dest2 := map[int]string{200: "will be overwritten"}
maps.Copy(dest2, src)
fmt.Println(dest2) // map[200:foo 300:bar]
}
Playground: https://go.dev/play/p/of_H-YaEtir
Note that the maps package is in golang.org/x/exp/maps, which is still experimental — i.e. outside of Go compatibility guarantee. Hopefully it will be moved into the standard lib at some point in the near future.
If you don't want to import exp packages, the function can be trivially written with Go 1.18 type parameters. The following code is identical to the maps source:
func Copy[M ~map[K]V, K comparable, V any](dst, src M) {
for k, v := range src {
dst[k] = v
}
}
At the time of writing, there is ongoing discussion to add two map type parameters to the Copy function, so it would become like this:
func Copy[M1, M2 ~map[K]V, K comparable, V any](dst M1, src M2) {
for k, v := range src {
dst[k] = v
}
}
This is functionally the same thing, as the type parameters K and V are the same for both M1 and M2 but brings the signature in line with the other functions in the maps package.
Related
I'm trying to convert key-value map to slice of pairs, for example given a map like:
m := make(map[int64]int64)
m[521] = 4
m[528] = 8
How do I convert that into a slice of its entries, like: [[521, 4], [528, 8]]
I'm thinking about ranging over all those key-values then create slice for that, but is there any simple code to do that?
package main
import "fmt"
func main() {
//create a map
m := map[int64]int64{512: 8, 513: 9, 234: 9, 392: 0}
//create a slice to hold required values
s := make([][]int64, 0)
//range over map `m` to append to slice `s`
for k, v := range m {
// append each element, with a new slice []int64{k, v}
s = append(s, []int64{k, v})
}
fmt.Println(s)
}
Go 1.18
It is now possible to write a generic function to extract all key-value pairs, i.e. the map entries, with any key and value types.
Notes:
the map iterations are still unordered — using generics doesn't change that.
the constraint for the map key must be comparable
type Pair[K, V any] struct {
First K
Second V
}
func Entries[M ~map[K]V, K comparable, V any](m M) []Pair[K, V] {
entries := make([]Pair[K, V], 0)
for k, v := range m {
entries = append(entries, Pair[K, V]{k, v})
}
return entries
}
The type Pair here is used to preserve type safety in the return value. If you really must return a slice of slices, then it can only be [][]any (or [][2]any) in order to hold different types.
If the map key and value have the same type, of course you can still use Pair but you can also use a type-safe variation of the above:
func Entries[T comparable](m map[T]T) [][2]T {
entries := make([][2]T, 0)
for k, v := range m {
entries = append(entries, [2]T{k, v})
}
return entries
}
Again, T must be comparable or stricter in order to work as a map key.
Playground: https://go.dev/play/p/RwCGmp7MHKW
Here is the code:
type field struct {
name string
}
func print(p *field) {
fmt.Println(p.name)
}
func fix1() {
data := []*field{{name: "one"}, {name: "two"}, {name: "three"}}
for _, v := range data {
go print(v)
}
time.Sleep(time.Millisecond * 200)
}
func wrong1() {
data := []*field{{name: "one"}, {name: "two"}, {name: "three"}}
for _, v := range data {
go func() {
print(v)
}()
}
time.Sleep(time.Millisecond * 200)
}
func main() {
wrong1()
}
As far as I understand, all goroutines in function wrong1 share the same local variable v. At the moment of a goroutine execution, the value of v may be equal to any value in data, therefore the function prints random data three times.
However, I am failing to understand why function fix1 behaves differently (it prints each value in data exactly once).
wrong1(): go func() { print(v) }()
Go: Frequently Asked Questions (FAQ)
What happens with closures running as goroutines?
Some confusion may arise when using closures with concurrency.
Consider the following program:
func main() {
done := make(chan bool)
values := []string{"a", "b", "c"}
for _, v := range values {
go func() {
fmt.Println(v)
done <- true
}()
}
// wait for all goroutines to complete before exiting
for _ = range values {
<-done
}
}
One might mistakenly expect to see a, b, c as the output. What you'll
probably see instead is c, c, c. This is because each iteration of the
loop uses the same instance of the variable v, so each closure shares
that single variable. When the closure runs, it prints the value of v
at the time fmt.Println is executed, but v may have been modified
since the goroutine was launched.
To bind the current value of v to each closure as it is launched, one
must modify the inner loop to create a new variable each iteration.
One way is to pass the variable as an argument to the closure:
for _, v := range values {
go func(u string) {
fmt.Println(u)
done <- true
}(v)
}
In this example, the value of v is passed as an argument to the
anonymous function. That value is then accessible inside the function
as the variable u.
Even easier is just to create a new variable, using a declaration
style that may seem odd but works fine in Go:
for _, v := range values {
v := v // create a new 'v'.
go func() {
fmt.Println(v)
done <- true
}()
}
Your wrong1 example,
for _, v := range data {
go func() {
print(v)
}()
}
Playground: https://play.golang.org/p/0w86nvVMt1g
Output:
three
three
three
Your wrong1 example, creating a new variable,
for _, v := range data {
v := v
go func() {
print(v)
}()
}
Playground: https://play.golang.org/p/z5RCI0ZZU8Z
Output:
one
two
three
Your wrong1 example, passing the variable as an argument,
for _, v := range data {
go func(v *field) {
print(v)
}(v)
}
Playground: https://play.golang.org/p/1JVI7XYSqvv
Output:
one
two
three
fix1(): go print(v)
The Go Programming Language Specification
Calls
Given an expression f of function type F,
f(a1, a2, … an)
calls f with arguments a1, a2, … an. Except for one special case,
arguments must be single-valued expressions assignable to the
parameter types of F and are evaluated before the function is called.
Go statements
The function value and parameters are evaluated as usual in the
calling goroutine.
Your fix1 example, evaluating the value of v before the function is called,
for _, v := range data {
go print(v)
}
Playground: https://play.golang.org/p/rN3UNaGi-ge
Output:
one
two
three
http://play.golang.org/p/Vd3meom5VF
I have this code for some context free grammar in Go
And I am looking at this code so many times and still don't see any reason for the struct values to be changed. Could anybody see why the change like the following happens?
Rules:
S -> . [DP VP]
VP -> . [V DP]
VP -> . [V DP AdvP]
After I run some functions as in the line
or2 = append(or2, OstarCF([]QRS{q}, []string{"sees"}, g2.Nullables(), g2.ChainsTo(g2.Nullables()))...)
Somehow my struct value is changed... I don't know why...
Rules:
S -> . [VP VP]
VP -> . [DP DP]
VP -> . [AdvP AdvP AdvP]
This should have been same as above.
Rules:
S -> DP,VP
VP -> V,DP
VP -> V,DP,AdvP
or2 := []QRS{}
g2 := ToGrammar(cfg2)
fmt.Printf("%s\n", g2)
for _, rule := range g2.Rules {
q := QRS{
one: rule.Src,
two: []string{},
three: rule.Right,
}
or2 = append(or2, OstarCF([]QRS{q}, []string{"sees"}, g2.Nullables(), g2.ChainsTo(g2.Nullables()))...)
}
fmt.Printf("%s\n", g2)
As you see, I do not use any pointer the variable rule, and they are only used to instantiate another struct value, but how come the original struct field rule has changed? The function OstarCF does not do anything about this field rule
func OstarCF(Qs []QRS, R []string, nD map[string]bool, cD map[string][]string) []QRS {
symbols := []string{}
for _, r := range R {
symbols = append(symbols, cD[r]...)
}
product := []QRS{}
for _, Q := range Qs {
a := Q.one
b := Q.two
c := Q.three
if len(c) > 0 && CheckStr(c[0], symbols) {
b = append(b, c[0])
np := QRS{
one: a,
two: b,
three: c[1:],
}
product = append(product, np)
for len(np.three) > 0 && nD[np.three[0]] == true {
np.two = append(np.two, np.three[0])
np = QRS{
one: np.one,
two: np.two,
three: np.three[1:],
}
product = append(product, np)
}
}
}
return product
}
The original Rules field changes because pointers and slices (which are references as well) are used.
Before calling OstarCF, the ChainsTo method is called. It uses the grammar object by value, so a copy is done, but the Rules field is a slice of pointers on Rules. So when this field is copied, it still points to the data of the original object.
Then, in method ChainsTo, there is a loop on the Rules field. It copies the Right field which is a slice of strings (so it still points to data of the original object):
rhs := rule.Right
Finally, a ns variable is declared by slicing rhs:
ns := rhs[:i]
ns = append(ns, rhs[i+1:]...)
At this stage, the ns variable still points to the buffer containing the slice of strings of the original object. Initially, i=0, so ns is an empty slice reusing the buffer. When items are appended, they replace the original data.
That's why your data are changed.
You can fix this problem by explicitly making a copy, for instance by replacing the above lines by:
ns := make( []string, 0, len(rhs) )
ns = append( ns, rhs[:i]...)
ns = append( ns, rhs[i+1:]...)
Go slices have replaced C pointer arithmetic, but they can be almost as dangerous/misleading in some cases.
Is there any simpler/nicer way of getting a slice of keys from a map in Go?
Currently I am iterating over the map and copying the keys to a slice:
i := 0
keys := make([]int, len(mymap))
for k := range mymap {
keys[i] = k
i++
}
This is an old question, but here's my two cents. PeterSO's answer is slightly more concise, but slightly less efficient. You already know how big it's going to be so you don't even need to use append:
keys := make([]int, len(mymap))
i := 0
for k := range mymap {
keys[i] = k
i++
}
In most situations it probably won't make much of a difference, but it's not much more work, and in my tests (using a map with 1,000,000 random int64 keys and then generating the array of keys ten times with each method), it was about 20% faster to assign members of the array directly than to use append.
Although setting the capacity eliminates reallocations, append still has to do extra work to check if you've reached capacity on each append.
For example,
package main
func main() {
mymap := make(map[int]string)
keys := make([]int, 0, len(mymap))
for k := range mymap {
keys = append(keys, k)
}
}
To be efficient in Go, it's important to minimize memory allocations.
You also can take an array of keys with type []Value by method MapKeys of struct Value from package "reflect":
package main
import (
"fmt"
"reflect"
)
func main() {
abc := map[string]int{
"a": 1,
"b": 2,
"c": 3,
}
keys := reflect.ValueOf(abc).MapKeys()
fmt.Println(keys) // [a b c]
}
Go now has generics. You can get the keys of any map with maps.Keys.
Example usage:
intMap := map[int]int{1: 1, 2: 2}
intKeys := maps.Keys(intMap)
// intKeys is []int
fmt.Println(intKeys)
strMap := map[string]int{"alpha": 1, "bravo": 2}
strKeys := maps.Keys(strMap)
// strKeys is []string
fmt.Println(strKeys)
maps package is found in golang.org/x/exp/maps. This is experimental and outside of Go compatibility guarantee. They aim to move it into the std lib in Go 1.19 the future.
Playground: https://go.dev/play/p/fkm9PrJYTly
For those who don't like to import exp packages, you can copy the source code:
// Keys returns the keys of the map m.
// The keys will be an indeterminate order.
func Keys[M ~map[K]V, K comparable, V any](m M) []K {
r := make([]K, 0, len(m))
for k := range m {
r = append(r, k)
}
return r
}
I made a sketchy benchmark on the three methods described in other responses.
Obviously pre-allocating the slice before pulling the keys is faster than appending, but surprisingly, the reflect.ValueOf(m).MapKeys() method is significantly slower than the latter:
❯ go run scratch.go
populating
filling 100000000 slots
done in 56.630774791s
running prealloc
took: 9.989049786s
running append
took: 18.948676741s
running reflect
took: 25.50070649s
Here's the code: https://play.golang.org/p/Z8O6a2jyfTH
(running it in the playground aborts claiming that it takes too long, so, well, run it locally.)
A nicer way to do this would be to use append:
keys = []int{}
for k := range mymap {
keys = append(keys, k)
}
Other than that, you’re out of luck—Go isn’t a very expressive language.
A generic version (go 1.18+) of Vinay Pai's answer.
// MapKeysToSlice extract keys of map as slice,
func MapKeysToSlice[K comparable, V any](m map[K]V) []K {
keys := make([]K, len(m))
i := 0
for k := range m {
keys[i] = k
i++
}
return keys
}
Visit https://play.golang.org/p/dx6PTtuBXQW
package main
import (
"fmt"
"sort"
)
func main() {
mapEg := map[string]string{"c":"a","a":"c","b":"b"}
keys := make([]string, 0, len(mapEg))
for k := range mapEg {
keys = append(keys, k)
}
sort.Strings(keys)
fmt.Println(keys)
}
There is a cool lib called lo
A Lodash-style Go library based on Go 1.18+ Generics (map, filter, contains, find...)
With this lib you could do many convinient operations like map, filter, reduce and more. Also there are some helpers for map type
Keys
Creates an array of the map keys.
keys := lo.Keys[string, int](map[string]int{"foo": 1, "bar": 2})
// []string{"bar", "foo"}
Values
Creates an array of the map values.
values := lo.Values[string, int](map[string]int{"foo": 1, "bar": 2})
// []int{1, 2}
If I have a map m is there a better way of getting a slice of the values v than this?
package main
import (
"fmt"
)
func main() {
m := make(map[int]string)
m[1] = "a"
m[2] = "b"
m[3] = "c"
m[4] = "d"
// Can this be done better?
v := make([]string, len(m), len(m))
idx := 0
for _, value := range m {
v[idx] = value
idx++
}
fmt.Println(v)
}
Is there a built-in feature of a map? Is there a function in a Go package, or is this the only way to do this?
As an addition to jimt's post:
You may also use append rather than explicitly assigning the values to their indices:
m := make(map[int]string)
m[1] = "a"
m[2] = "b"
m[3] = "c"
m[4] = "d"
v := make([]string, 0, len(m))
for _, value := range m {
v = append(v, value)
}
Note that the length is zero (no elements present yet) but the capacity (allocated space) is initialized with the number of elements of m. This is done so append does not need to allocate memory each time the capacity of the slice v runs out.
You could also make the slice without the capacity value and let append allocate the memory for itself.
Unfortunately, no. There is no builtin way to do this.
As a side note, you can omit the capacity argument in your slice creation:
v := make([]string, len(m))
The capacity is implied to be the same as the length here.
Go 1.18
You can use maps.Values from the golang.org/x/exp package.
Values returns the values of the map m. The values will be in an indeterminate order.
func main() {
m := map[int]string{1: "a", 2: "b", 3: "c", 4: "d"}
v := maps.Values(m)
fmt.Println(v)
}
The package exp includes experimental code. The signatures may or may not change in the future, and may or may not be promoted to the standard library.
If you don't want to depend on an experimental package, you can easily implement it yourself. In fact, this code is a copy-paste from the exp package:
func Values[M ~map[K]V, K comparable, V any](m M) []V {
r := make([]V, 0, len(m))
for _, v := range m {
r = append(r, v)
}
return r
}
Not necessarily better, but the cleaner way to do this is by defining both the Slice LENGTH and CAPACITY like txs := make([]Tx, 0, len(txMap))
// Defines the Slice capacity to match the Map elements count
txs := make([]Tx, 0, len(txMap))
for _, tx := range txMap {
txs = append(txs, tx)
}
Full example:
package main
import (
"github.com/davecgh/go-spew/spew"
)
type Tx struct {
from string
to string
value uint64
}
func main() {
// Extra touch pre-defining the Map length to avoid reallocation
txMap := make(map[string]Tx, 3)
txMap["tx1"] = Tx{"andrej", "babayaga", 10}
txMap["tx2"] = Tx{"andrej", "babayaga", 20}
txMap["tx3"] = Tx{"andrej", "babayaga", 30}
txSlice := getTXsAsSlice(txMap)
spew.Dump(txSlice)
}
func getTXsAsSlice(txMap map[string]Tx) []Tx {
// Defines the Slice capacity to match the Map elements count
txs := make([]Tx, 0, len(txMap))
for _, tx := range txMap {
txs = append(txs, tx)
}
return txs
}
Simple solution but a lot of gotchas. Read this blog post for more details: https://web3.coach/golang-how-to-convert-map-to-slice-three-gotchas
As far as I'm currently aware, go doesn't have a way method for concatenation of strings/bytes in to a resulting string without making at least /two/ copies.
You currently have to grow a []byte since all string values are const, THEN you have to use the string builtin to have the language create a 'blessed' string object, which it will copy the buffer for since something somewhere could have a reference to the address backing the []byte.
If a []byte is suitable then you can gain a very slight lead over the bytes.Join function by making one allocation and doing the copy calls your self.
package main
import (
"fmt"
)
func main() {
m := make(map[int]string)
m[1] = "a" ; m[2] = "b" ; m[3] = "c" ; m[4] = "d"
ip := 0
/* If the elements of m are not all of fixed length you must use a method like this;
* in that case also consider:
* bytes.Join() and/or
* strings.Join()
* They are likely preferable for maintainability over small performance change.
for _, v := range m {
ip += len(v)
}
*/
ip = len(m) * 1 // length of elements in m
r := make([]byte, ip, ip)
ip = 0
for _, v := range m {
ip += copy(r[ip:], v)
}
// r (return value) is currently a []byte, it mostly differs from 'string'
// in that it can be grown and has a different default fmt method.
fmt.Printf("%s\n", r)
}
As of 1.18, this is the best way:
https://stackoverflow.com/a/71635953/130427
Pre 1.18
You can use this maps package:
go get https://github.com/drgrib/maps
Then all you have to call is
values := maps.GetValuesIntString(m)
It's type-safe for that common map combination. You can generate other type-safe functions for any other type of map using the mapper tool in the same package.
Full disclosure: I am the creator of this package. I created it because I found myself rewriting these functions for map repeatedly.