How does one delete data from a map in Go? For example, having
m := map[string]string{ "key1":"val1", "key2":"val2" };
I want to make m drop the "key1" without copying the entire map by iterating over its keys (which could get big in some uses). Is it enough to assign a nil value to "key1", or will that still keep the key in the map structure with an assigned value of nil? That is, if I later iterate over the keys of the map, will "key1" appear?
Deletion of map elements
The built-in function delete removes the element with key k from a map
m.
delete(m, k) // remove element m[k] from map m
For example,
package main
import "fmt"
func main() {
m := map[string]string{"key1": "val1", "key2": "val2"}
fmt.Println(m)
delete(m, "key1")
fmt.Println(m)
}
Output:
map[key1:val1 key2:val2]
map[key2:val2]
Related
Here is the simplified code of a cache. Suppose Container placed in a package, so it don't know about Member.
While I wanna store instances of Member in Container, So I store an empty instance of Member in Container as outerType.
In the Container->GetMysql, I fill a new variable by test values (but, in real world, It fill by data of database, dynamically).
then in the function Put, I store data in items as Cache for next uses. In the Get I get the data stored in the items.
Before this every thing is fine. My problem is where i want to convert result of Get to type of Member m = res.(Member) . How Can I convert it to an instance of Member
I found many question about this subject, but none of them solved my problem
For more detail: I want the Get return data with its pointer of where it stored in items. So if I get some variable of same member, an change in one are shown in others
package main
import (
"fmt"
"reflect"
)
type Member struct {
Id int
Name string
Credit int
Age int
}
type Container struct {
outerType interface{}
items map[string]*interface{}
}
func (cls *Container)GetMysql(s string, a int64) interface{}{
obj := reflect.New(reflect.TypeOf(cls.outerType))
elem := obj.Elem()
//elem := reflect.ValueOf(o).Elem()
if elem.Kind() == reflect.Struct {
f := elem.FieldByName("Name")
f.SetString(s)
f = elem.FieldByName("Credit")
f.SetInt(a)
}
return obj.Interface()
}
func (cls *Container)Get(value string) *interface{}{
return cls.items[value]
}
func (cls *Container)Put(value string, a int64) {
res := cls.GetMysql(value, a)
cls.items[value] = &res
}
func main() {
c := Container{outerType:Member{}}
c.items = make(map[string]*interface{})
c.Put("Jack", 500)
res := c.Get("Jack")
fmt.Println(*res)
m := &Member{}
m = res.(Member) // Here is the problem. How to convert ?
fmt.Println(m)
}
You should hardly ever use pointer to interface. My advice is to never use it, when you'll need it, you'll know.
Instead if you need a pointer to something (so you can have the same pointer at multiple places, and so modifying the pointed value somewhere, it will have effect on the others), "wrap the pointer" in the interface value.
So first modify the items field so that it stores interface{} values instead of pointers:
items map[string]interface{}
This means no restriction: you can pass and store pointers, that's not a problem.
Next modify Get() to return interface{}:
func (cls *Container) Get(value string) interface{}{
return cls.items[value]
}
And also in Put(), don't take the address of an interface{}:
func (cls *Container) Put(value string, a int64) {
res := cls.GetMysql(value, a)
cls.items[value] = res
}
And you have to type-assert *Member from the values returned by Get().
And now testing it:
c := Container{outerType: Member{}}
c.items = make(map[string]interface{})
c.Put("Jack", 500)
res := c.Get("Jack")
fmt.Println(res)
m := res.(*Member) // Here is the problem. How to convert ?
fmt.Println(m)
Output (try it on the Go Playground):
&{0 Jack 500 0}
&{0 Jack 500 0}
Now if you would modify a field of m:
m.Credit = 11
And then get the value form the cache:
fmt.Println(c.Get("Jack"))
We'll see the modified value, even though we did not call Put() (try it on the Go Playground):
&{0 Jack 11 0}
What is the shortest (and idiomatic) way to create an array from the keys and values of a map w/o compromising on time complexity too much?
For instance, from the following map:
map[string]string { "1":"a", "2":"b" }
I need to create the following array:
[]string{"1","a", "2","b"}
I can do this in Scala with following:
val myMap = Map("1" -> "a", "2" -> "b")
myMap.keySet ++ myMap.values
Thank you.
Simplest way would be to just iterate the map, since in Go the syntax would allow direct access to keys and values and dump them into the array.
m := map[string]string { "1":"a", "2":"b" }
arr := []string{}
for k, v := range m {
arr = append(arr, k, v)
}
One caveat here: In Go, map iteration order is randomized, as you can see here, under "Iteration Order":
https://blog.golang.org/go-maps-in-action
So if you want your resulting array to have a particular ordering, you should first dump the keys and order (as shown in that same blog entry).
Playground (without the sorting part):
https://play.golang.org/p/mCe6eEy25A
Here's my code:
func test(v map[string]string) {
v["foo"] = "bar"
}
func main() {
v := make(map[string]string)
test(v)
fmt.Printf("%v\n", v) // prints map[foo:bar]
}
I'm pretty new to Go, but as far as I was aware, since I'm passing the map value to test() and not a pointer to the map, the test() function should modify a different variable of the map, and thus, not affect the value of the variable in main(). I would have expected it to print map[]. I tested a different scenario:
type myStruct struct {
foo int
}
func test2(v myStruct) {
v.foo = 5
}
func main() {
v := myStruct{1}
test2(v)
fmt.Printf("%v\n", v) // prints {1}
}
In this scenario, the code behaves as I would expect. The v variable in the main() function is not affected by the changes to the variable in test2(). So why is map different?
You are right in that when you pass something to a function, a copy will be made. But maps are some kind of descriptors to an underlying data structure. So when you pass a map value to a function, only the descriptor will be copied which will denote / point to the same data structures where the map data (entries) are stored.
This means if the function does any modification to the entries of the map (add, delete, modify entries), that is observed from the caller.
Read The Go Blog: Go maps in action for details.
Note that the same applies to slices and channels too; generally speaking the types that you can create using the built-in make() function. That's why the zero value of these types is nil, because a value of these types need some extra initialization which is done when calling make().
In your other example you are using a struct value, they are not descriptors. When you pass a struct value to another function, that creates a complete copy of the struct value (copying values of all its fields), which –when modified inside the function– will not have any effect on the original, as the memory of the copy will be modified – which is distinct.
How can one remove selected keys from a map?
Is it safe to combine delete() with range, as in the code below?
package main
import "fmt"
type Info struct {
value string
}
func main() {
table := make(map[string]*Info)
for i := 0; i < 10; i++ {
str := fmt.Sprintf("%v", i)
table[str] = &Info{str}
}
for key, value := range table {
fmt.Printf("deleting %v=>%v\n", key, value.value)
delete(table, key)
}
}
https://play.golang.org/p/u1vufvEjSw
This is safe! You can also find a similar sample in Effective Go:
for key := range m {
if key.expired() {
delete(m, key)
}
}
And the language specification:
The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next. If map entries that have not yet been reached are removed during iteration, the corresponding iteration values will not be produced. If map entries are created during iteration, that entry may be produced during the iteration or may be skipped. The choice may vary for each entry created and from one iteration to the next. If the map is nil, the number of iterations is 0.
Sebastian's answer is accurate, but I wanted to know why it was safe, so I did some digging into the Map source code. It looks like on a call to delete(k, v), it basically just sets a flag (as well as changing the count value) instead of actually deleting the value:
b->tophash[i] = Empty;
(Empty is a constant for the value 0)
What the map appears to actually be doing is allocating a set number of buckets depending on the size of the map, which grows as you perform inserts at the rate of 2^B (from this source code):
byte *buckets; // array of 2^B Buckets. may be nil if count==0.
So there are almost always more buckets allocated than you're using, and when you do a range over the map, it checks that tophash value of each bucket in that 2^B to see if it can skip over it.
To summarize, the delete within a range is safe because the data is technically still there, but when it checks the tophash it sees that it can just skip over it and not include it in whatever range operation you're performing. The source code even includes a TODO:
// TODO: consolidate buckets if they are mostly empty
// can only consolidate if there are no live iterators at this size.
This explains why using the delete(k,v) function doesn't actually free up memory, just removes it from the list of buckets you're allowed to access. If you want to free up the actual memory you'll need to make the entire map unreachable so that garbage collection will step in. You can do this using a line like
map = nil
I was wondering if a memory leak could happen. So I wrote a test program:
package main
import (
log "github.com/Sirupsen/logrus"
"os/signal"
"os"
"math/rand"
"time"
)
func main() {
log.Info("=== START ===")
defer func() { log.Info("=== DONE ===") }()
go func() {
m := make(map[string]string)
for {
k := GenerateRandStr(1024)
m[k] = GenerateRandStr(1024*1024)
for k2, _ := range m {
delete(m, k2)
break
}
}
}()
osSignals := make(chan os.Signal, 1)
signal.Notify(osSignals, os.Interrupt)
for {
select {
case <-osSignals:
log.Info("Recieved ^C command. Exit")
return
}
}
}
func GenerateRandStr(n int) string {
rand.Seed(time.Now().UnixNano())
const letterBytes = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
b := make([]byte, n)
for i := range b {
b[i] = letterBytes[rand.Int63() % int64(len(letterBytes))]
}
return string(b)
}
Looks like GC do frees the memory. So it's okay.
In short, yes. See previous answers.
And also this, from here:
ianlancetaylor commented on Feb 18, 2015
I think the key to understanding this is to realize that while executing the body of a for/range statement, there is no current iteration. There is a set of values that have been seen, and a set of values that have not been seen. While executing the body, one of the key/value pairs that has been seen--the most recent pair--was assigned to the variable(s) of the range statement. There is nothing special about that key/value pair, it's just one of the ones that has already been seen during the iteration.
The question he's answering is about modifying map elements in place during a range operation, which is why he mentions the "current iteration". But it's also relevant here: you can delete keys during a range, and that just means that you won't see them later on in the range (and if you already saw them, that's okay).
I have a recursive function that creates objects representing file paths (the keys are paths and the values are info about the file). It's recursive as it's only meant to handle files, so if a directory is encountered, the function is recursively called on the directory.
All that being said, I'd like to do the equivalent of a set union on two maps (i.e. the "main" map updated with the values from the recursive call). Is there an idiomatic way to do this aside from iterating over one map and assigning each key, value in it to the same thing in the other map?
That is: given a,b are of type map [string] *SomeObject, and a and b are eventually populated, is there any way to update a with all the values in b?
There is no built in way, nor any method in the standard packages to do such a merge.
The idomatic way is to simply iterate:
for k, v := range b {
a[k] = v
}
Since Go 1.18, you can simply use the Copy function from the golang.org/x/exp/maps package:
package main
import (
"fmt"
"golang.org/x/exp/maps"
)
func main() {
src := map[string]int{
"one": 1,
"two": 2,
}
dst := map[string]int{
"two": 2,
"three": 3,
}
maps.Copy(dst, src)
fmt.Println("src:", src)
fmt.Println("dst:", dst)
}
(Playground)
Output:
src: map[one:1 two:2]
dst: map[one:1 three:3 two:2]
One caveat of this approach is that, in Go versions 1.18.x to 1.19.x, your map's key type must be concrete, i.e. not an interface type. For instance, the compiler won't allow you to pass values of type map[io.Reader]int to the Copy function:
package main
import (
"fmt"
"io"
"golang.org/x/exp/maps"
)
func main() {
var src, dst map[io.Reader]int
maps.Copy(dst, src)
fmt.Println("src:", src)
fmt.Println("dst:", dst)
}
(Playground)
Compiler output:
go: finding module for package golang.org/x/exp/maps
go: downloading golang.org/x/exp v0.0.0-20220328175248-053ad81199eb
./prog.go:12:11: io.Reader does not implement comparable
Go build failed.
This limitation was lifted in Go 1.20.
(Playground)
Starting at go 1.18, thanks to the release of the Generics feature, there are now generic functions that union maps!
You can use a package like https://github.com/samber/lo in order to do so.
Note that the key can be of any "comparable" type, while the value can be of any type.
Example:
package main
import (
"fmt"
"github.com/samber/lo"
)
func main() {
map1 := map[string]interface{}{"k1": "v1", "k2": 2}
map2 := map[string]interface{}{"k2": "v2new", "k3": true}
map1 = lo.Assign(map1, map2)
fmt.Printf("%v", map1)
}
The result is:
map[k1:v1 k2:v2new k3:true]
If you have a couple of nested maps, left and right, then this function will recursively add the items from right into left. If the key is already in left then we recurse deeper into the structure and attempt only add keys to left (e.g. never replace them).
type m = map[string]interface{}
// Given two maps, recursively merge right into left, NEVER replacing any key that already exists in left
func mergeKeys(left, right m) m {
for key, rightVal := range right {
if leftVal, present := left[key]; present {
//then we don't want to replace it - recurse
left[key] = mergeKeys(leftVal.(m), rightVal.(m))
} else {
// key not in left so we can just shove it in
left[key] = rightVal
}
}
return left
}
NOTE: I do not handle the case in which the value is not itself a map[string]interface{}. So if you have left["x"] = 1 and right["x"] = 2 then the above code will panic when attempting leftVal.(m).
Here is another option,
in case you are trying to limit the number of third-party dependencies such github.com/samber/lo, OR
you are not comfortable with the experimental nature of golang.org/x/exp (read the warning), OR
you would rather the convenience of an append()-like API instead of exp.Copy() from golang.org/x/exp (append accepts any number of lists, whereas Copy() accepts only 2).
However it requires Go 1.18+ as it uses go generics.
Save the following in one of your modules/packages:
func MergeMaps[M ~map[K]V, K comparable, V any](src ...M) M {
merged := make(M)
for _, m := range src {
for k, v := range m {
merged[k] = v
}
}
return merged
}
Then you can use it very similarly to append():
func main() {
mergedMaps := MergeMaps(
map[string]int{"a": 1, "b": 2},
map[string]int{"b": 3, "c": 4},
map[string]int{"c": 3, "d": 4},
)
fmt.Println(mergedMaps)
}
Go is limited by what type of map it is. I'd suspect that there isn't built in functions because of the infinite number of type declarations that could exist for a map. So you have to build your own Merge functions depending on what type of map you are using:
func MergeJSONMaps(maps ...map[string]interface{}) (result map[string]interface{}) {
result = make(map[string]interface{})
for _, m := range maps {
for k, v := range m {
result[k] = v
}
}
return result
}