type A struct {
x1 []int
x2 []string
}
func (this *A) Test() {
fmt.Printf("this is: %p, %+v\n", this, *this)
}
func main() {
var fn func()
{
a := &A{}
a.x1 = []int{1, 2, 3}
a.x2 = []string{"one", "two", "three"}
fn = a.Test
}
fn()
}
Please see: https://play.golang.org/p/YiwHG0b1hW-
My question is:
Will 'a' be released out of {} local scope?
Is the lifetime of 'a' equal to the one of 'fn'?
Go is a garbage collected language. As long as you don't touch package unsafe (or similar like Value.UnsafeAddr() in package reflect), all values remain in memory as long as they are reachable. You don't have to worry about memory management.
That's why it's also safe to return addresses (pointers) of local variables created inside functions. And also it's safe to refer to local variables from function values (closures) that will be out of scope when the function value is executed some time in the future, like this one:
func counter() func() int {
i := 0
return func() int {
i++
return i
}
}
This counter() returns a function (closure) which when called, returns increasing values:
c := counter()
fmt.Println(c())
fmt.Println(c())
fmt.Println(c())
This outputs (try it on the Go Playground):
1
2
3
counter() creates a local variable i which is not returned, but is accessed from the function value returned by it. As long as the returned function value is accessible, the local variable i is not freed. Also if you call counter() again, that creates a new i variable distinct from the previous one.
See related questions:
How to delete struct object in go?
Cannot free memory once occupied by bytes.Buffer
Related
I've run into an issue in my current project where I have two modules, one implementing an interface for testing purposes, and one just a concrete struct, which each depend on a method from the other.
In order to resolve this tension, I've attempted to create a top-level "container" struct that holds a reference to the dependent struct and interface, and then with a method on the container struct, assign as a member of each component struct that top level container's pointer to the other struct. I am doing this instead of using globals in order to be able to better encapsulate my code for testing purposes.
However, it seems that whichever struct is initialized first does not see the change in the other struct's address when the second struct is initialized. I do not understand why, and I don't seem to be able to make this function as expected.
Since there are many extraneous details in the actual code I've created this toy example to illustrate what I'm talking about.
type container struct {
r requestor
a *A
}
type requestor interface {
Request()
}
type A struct {
r requestor
}
type R struct {
a *A
}
func (r R) Request() {
log.Info("I requested")
return
}
func (container *container) NewA() *A {
log.Info("New A received container.r: ", container.r)
a := &A{
r: container.r,
}
container.a = a
return a
}
func (container *container) NewR() *R {
r := &R{
a: container.a,
}
container.r = r
return r
}
func TestDepResolution(t *testing.T) {
top := container{}
top.NewR()
top.NewA()
// top.a.r = r
log.Infof("top: %+v", top)
log.Infof("R: %+v", top.r)
log.Infof("A: %+v", top.a)
}
It's setup as a test so I can easily execute it within my project. The output is as such:
=== RUN TestDepResolution
INFO[0000] New A received container.r: <nil>
INFO[0000] top: {r:0xc000010028 a:0xc00006abc0}
INFO[0000] R: &{a:0xc00006abc0}
INFO[0000] A: &{r:<nil>}
I expected that A's r variable would become equal to top's r variable after NewR() was called, but it doesn't seem to change. The same issue occurs the other way around if I switch the order of NewA() and NewR().
I expected since I am using pointers and interfaces here that the values would be connected when top's values changed, but it's apparent I must be misunderstanding something. I've tried playing around with the pointers quite a bit to no avail.
So why doesn't this work as I expected? Is there a way to make this work as I've proposed? Or am I thinking about this issue in an entirely wrongheaded way? I have tried to think about extracting functionality from the modules so that they are not mutually dependent and I could avoid this issue entirely, but I have not been able to come up with a good way to do so.
To be able to utilize pointers the way you seem to want to, you first need actual pointers (i.e. not nil pointers) and you also need to use pointer indirection to be able to "share" the updates to the pointed values.
For example:
type T struct { F string }
a := &T{"foo"} // non-nil pointer
b := a
fmt.Println(b) // output: {"foo"}
*a = T{"bar"} // pointer indirection
fmt.Println(b) // output: {"bar"}
For comparison, here's what your code is attempting to do:
type T struct { F string }
a := (*T)(nil) // nil pointer
b := a
fmt.Println(b) // output: <nil>
a = &T{"bar"} // plain assignment
fmt.Println(b) // output: <nil>
And note that even if you used pointer indirection, it is illegal to do so on a nil pointer and the runtime, if it encounters such an operation, will panic.
a := (*T)(nil) // nil pointer
b := a
fmt.Println(b) // output: <nil>
*a = T{"bar"} // pointer indirection on nil, will crash the program
fmt.Println(b)
So, your example doesn't work because it does not properly initialize the pointers and it does not use pointer indirection, rather, it uses simple assignment which just updates the target variable's pointer and not the pointed-to value.
To initialize the container properly you should do it in one step:
func NewContainer() *container {
c := &container{a: &A{}}
c.r = &R{a: c.a}
c.a.r = c.r
return c
}
https://play.golang.com/p/hfbqJEVyAHZ
Or, if you want to do it in two, you can do something like this:
func (c *container) NewA() *A {
log.Println("New A received c.r: ", c.r)
a := &A{
r: c.r,
}
if c.a != nil {
*c.a = *a
} else {
c.a = a
}
return a
}
func (c *container) NewR() *R {
if c.a == nil {
c.a = new(A)
}
r := &R{
a: c.a,
}
c.r = r
c.a.r = r
return r
}
https://play.golang.com/p/krmUQOsACdU
but, as you can see, the multi step approach to initializing so tightly coupled dependencies can get unnecessarily convoluted and ugly, i.e. complex, i.e. very much error prone. Avoid it if you can.
All that said, personally, I would consider this kind of circular dependency a smell and would start thinking about redesign, but maybe that's just me.
Given the following types :
type A struct {
...
}
func (a *A) Process() {
...
}
I would like to pass the method Process of the type A to another function and be able to access the content of the underlying instance of A.
How should I pass the method to another function? Via a pointer? And how should it be called ?
The Process() method won't modify the instance of A, I am using a pointer on the method receiver because the struct is quite large. The idea behind my question is to avoid declaring the function Process() outside the struct and pass a ton of arguments to it (instead it access to the members of the struct).
You can even do it directly, without an interface:
package main
import "fmt"
type A struct {
Name string
}
func (a *A) Go() {
fmt.Printf("GO: %v\n", a)
}
func Test(fn func()) {
fn()
}
func main() {
aa := &A{Name: "FOO"}
bb := (*A)(nil)
cc := &A{}
Test(aa.Go)
Test(bb.Go)
Test(cc.Go)
}
Output:
GO: &{FOO}
GO: <nil>
GO: &{}
On the playground: https://play.golang.org/p/V-q2_zwX8h
Another option would be to define the func as a type:
type Process func(a *A)
Then use it as a parameter when calling your other func:
func Test(p Process)
One thing to remember is that these definitions are exactly the same thing:
func (a *A) Process() { a.MyVar }
func Process(a *A) { a.MyVar }
A pointer receiver is just a func with a local variable of a pointer to the struct. And conversely, a value receiver is a func with a local variable to a value copy of the struct.
Why would you not use a method on the struct, like option 1? There are many reasons. It does seem intuitive to group related methods onto the struct itself like option 1 (especially if coming from other OOP languages like Java or .NET where you normally stick upteen-thousand methods on a single struct). But, since you stated that the struct itself is quite large, this smells of SoC (that it is too large) and may need to be broken up.
Personally, the rule I follow when using option 2 above is:
If the func is not using the entire struct's properties (e.g. it is only operating on a sub-set of data, or even none at all), I instead use option 2 with a pointer. (or, use an interface itself with a zero-byte struct)
This allows for much easier unit testing by breaking up my struct that is "quite large" as you say, allowing me to mock up only the interface of functionality I need to support that method.
Now, func definitions are, by definitions, types themselves. So far we have this type:
func(a *A)
This can be used as an input into another method, as you asked for, like so:
func AnotherFunc(fn func(a *A)) {
a := &A{}
fn(a)
}
But to me, this makes things a bit hard to read not to mention brittle - someone could change the func definition there and break other things elsewhere.
This is where I prefer to define a type:
type Process func(a *A)
That way, I can consume it like:
func AnotherFunc(p Process) {
a := &A{}
p(a)
}
This allows you to access p as your pointer to the func, to pass around as you like. (Note though, you don't have to access the actual pointer of p. IOW, don't do this &p because func types are passed by reference in Golang anyways, just like slices and maps.)
Overall, you typically follow this kind of pattern when you want to break up your logic into smaller manageable (and more testable) pieces - by using smaller, more manageable AnotherFunc() methods to export and unit test an API contract for, while hiding the internals.
Working Example
http://play.golang.org/p/RAJ2t0nWEc
package main
import "fmt"
type Process func(a *A)
type A struct {
MyVar string
}
func processA(a *A) {
fmt.Println(a.MyVar)
}
func AnotherFunc(a *A, p Process) {
p(a)
}
func main() {
a := &A{
MyVar: "I'm here!",
}
AnotherFunc(a, processA)
}
Unit Testing
Taking the concept of func types to another level, would be to ease unit testing.
You can define global variables for your Process() function:
var Process = func(a *A)
It would continue to be used the exact same way:
func Test(p Process)
The difference now is during unit testing, you can override the function:
package mypackage_test
import "testing"
func TestProcessHasError(t *testing.T) {
// keep a backup copy of original definition
originalFunctionality := Process
// now, override it
Process = func(a *A) error {
// do something different here, like return error
return errors.New("force it to error")
}
// call your Test func normally, using it
err := Test(Process)
// check for error
assert.Error(err)
// be a good tester and restore Process back to normal
Process = originalFunctionality
}
When I get my hands on an existing codebase, these are some of the tricks I start implementing to help decouple the application from itself - and allow for more testing.
You can achieve this with an interface:
type Processor interface {
Process()
}
func anotherFunction(p Processor) {
p.Process()
}
...
var a A
anotherFunction(a)
We have written one program by which we try to find an address of a constant. Is it possible to do it like that?
package main
func main() {
const k = 5
address := &k
}
It gives an error, can anyone tell how can we find the address of a constant?
In short: you can't.
The error message says:
cannot take the address of k
There are limitations on the operand of the address operator &. Spec: Address operators:
For an operand x of type T, the address operation &x generates a pointer of type *T to x. The operand must be addressable, that is, either a variable, pointer indirection, or slice indexing operation; or a field selector of an addressable struct operand; or an array indexing operation of an addressable array. As an exception to the addressability requirement, x may also be a (possibly parenthesized) composite literal. If the evaluation of x would cause a run-time panic, then the evaluation of &x does too.
Constants are not listed as addressable, and things that are not listed in the spec as addressable (quoted above) cannot be the operand of the address operator & (you can't take the address of them).
It is not allowed to take the address of a constant. This is for 2 reasons:
A constant may not have an address at all.
And even if a constant value is stored in memory at runtime, this is to help the runtime to keep constants that: constant. If you could take the address of a constant value, you could assign the address (pointer) to a variable and you could change that (the pointed value, the value of the constant). Robert Griesemer (one of Go's authors) wrote why it's not allowed to take a string literal's address: "If you could take the address of a string constant, you could call a function [that assigns to the pointed value resulting in] possibly strange effects - you certainly wouldn't want the literal string constant to change." (source)
If you need a pointer to a value being equal to that constant, assign it to a variable of which is addressable so you can take its address, e.g.
func main() {
const k = 5
v := k
address := &v // This is allowed
}
But know that in Go numeric constants represent values of arbitrary precision and do not overflow. When you assign the value of a constant to a variable, it may not be possible (e.g. the constant may be greater than the max value of the variable's type you're assigning it to - resulting in compile-time error), or it may not be the same (e.g. in case of floating point constants, it may lose precision).
I often hit this problem when creating large, nested JSON objects during unit tests. I might have a structure where all the fields are pointers to strings/ints:
type Obj struct {
Prop1 *string
Prop2 *int
Status *string
}
and want to write something like:
obj := Obj{
Prop1: &"a string property",
Prop2: &5,
Status: &statuses.Awesome,
}
When I initialise it, but the language doesn't allow this directly. A quick way to bypass this is to define a function that takes a constant and returns its address:
s := func(s string) *string { return &s }
i := func(i int) *int { return &i }
obj := Obj{
Prop1: s("a string property"),
Prop2: i(5),
Status: s(statuses.Awesome)
}
This works due to the fact that when the constant is passed as a parameter to the function, a copy of the constant is made which means the pointer created in the function does not point to the address of the constant, but to the address of its copy, in the same way as when a constant value is assigned to a var. However, using a function to do this makes it more readable/less cumbersome IMO than having to forward declare large blocks of variables.
The AWS SDK uses this technique. I now find myself regularly adding a package to my projects that looks something like:
package ref
import "time"
func Bool(i bool) *bool {
return &i
}
func Int(i int) *int {
return &i
}
func Int64(i int64) *int64 {
return &i
}
func String(i string) *string {
return &i
}
func Duration(i time.Duration) *time.Duration {
return &i
}
func Strings(ss []string) []*string {
r := make([]*string, len(ss))
for i := range ss {
r[i] = &ss[i]
}
return r
}
Which I call in the following way:
func (t: Target) assignString(to string, value string) {
if to == tags.AuthorityId {
t.authorityId = ref.String(value)
}
// ...
}
You can also add a deref package, though I have generally found this to be less useful:
package deref
func String(s *string, d string) string {
if s != nil { return *s }
return d
}
// more derefs here.
EDIT April 2022:
With the release of go 1.18, it's now possible to define a single method to handle all conversions from constants into pointers:
package ref
func Of[E any](e E) *E {
return &e
}
I found another way to deal with this, which is using AWS API:
import "github.com/aws/aws-sdk-go/aws"
type Obj struct {
*int
}
x := aws.Int(16) // return address
obj := Obj{x} // work fine
this method is literally same as the answer above, but you dont have to write the whole functions on your own.
See: https://docs.aws.amazon.com/sdk-for-go/api/aws/
These 3 options could be helpful:
Using a helper function with generics. (Works for both primitive and custom types)
package main
import "fmt"
type Role string
const (
Engineer Role = "ENGINEER"
Architect Role = "ARCHITECT"
)
const (
EngineerStr string = "ENGINEER"
ArchitectStr string = "ARCHITECT"
)
func main() {
fmt.Println(PointerTo(Engineer)) // works for custom types
fmt.Println(PointerTo(EngineerStr)) // works for primitive types
}
func PointerTo[T any](v T) *T {
return &v
}
Try it on playground
Using pointy. (Works only for primitive types)
Using a ToPointer() method. (Works only for custom types)
package main
import "fmt"
type Role string
const (
Engineer Role = "ENGINEER"
Architect Role = "ARCHITECT"
)
func (r Role) ToPointer() *Role {
return &r
}
func main() {
fmt.Println(Engineer.ToPointer())
}
Try it on playground
What the constants section does not make very clear: Constants are, unlike variables, not present in the compiled code or running program. They are untyped and will only be in memory once they are assigned to a variable.
As a result, they seem1 to have infinite precision. If you look at this example, you can see that I can assign the constant to a variable without casting it, and the variable will hold as much of the constants precision as it can.
1 As the spec also points out, integers have at least 256 bits, floats at least 256 bits mantissa and 32 bits exponent, and the compiler will throw an error if its internal constructs cannot accurately store a constant.
I understand that Go doesn't have any constructors and a New func is used in its place, but according to this example.
func NewFile(fd int, name string) *File {
if fd < 0 {
return nil
}
f := File{fd, name, nil, 0}
return &f
}
They always return &f. Why just simply returning File isn't suffice?
Update
I've tried returning the created object for a simple struct and it's fine. So, I wonder if returning an address is a standard way of constructor or something.
Thanks.
As mentioned, yes, the spec allows you to return either values (as non-pointers) or pointers. It's just a decision you have to make.
When to return pointer?
Usually if the value you return is "more useful" as a pointer. When is it more useful?
For example if it has many methods with pointer receiver. Yes, you could store the return value in a variable and so it will be addressable and you can still call its methods that have pointer receivers. But if a pointer is returned right away, you can "chain" method calls. See this example:
type My int
func (m *My) Str() string { return strconv.Itoa(int(*m)) }
func createMy(i int) My { return My(i) }
Now writing:
fmt.Println(createMy(12).Str())
Will result in error: cannot call pointer method on createMy(12)
But if works if you return a pointer:
func createMy(i int) *My { return (*My)(&i) }
Also if you store the returned value in a data structure which is not addressable (map for example), you cannot call methods on values by indexing a map because values of a map are not addressable.
See this example: My.Str() has pointer receiver. So if you try to do this:
m := map[int]My{0: My(12)}
m[0].Str() // Error!
You can't because "cannot take the address of m[0]". But the following works:
m := map[int]*My{}
my := My(12)
m[0] = &my // Store a pointer in the map
m[0].Str() // You can call it, no need to take the address of m[0]
// as it is already a pointer
And another example for pointers being useful is if it is a "big" struct which will be passed around a lot. http.Request is a shining example. It is big, it is usually passed around a lot to other handlers, and it has methods with pointer receiver.
If you return a pointer, that usually suggests that the returned value is better if stored and passed around as a pointer.
Pointer receiver accepts both pointer and value types, as long as it matches the data type.
type User struct {
name string
email string
age int
}
// NewUserV returns value ... ideally for a User we should not be
// returning value
func NewUserV(name, email string, age int) User {
return User{name, email, age}
}
// NewUserP returns pointer ...
func NewUserP(name, email string, age int) *User {
return &User{name, email, age}
}
// ChangeEmail ...
func (u *User) ChangeEmail(newEmail string) {
u.email = newEmail
}
func main() {
// with value type
usr1 := NewUserV("frank", "frank#camero.com", 22)
fmt.Println("Before change: ", usr1)
usr1.ChangeEmail("frank#gmail.com")
fmt.Println("After change: ", usr1)
// with pointer type
usr2 := NewUserP("john", "john#liliput.com", 22)
fmt.Println("Before change: ", usr2)
usr2.ChangeEmail("john#macabre.com")
fmt.Println("After change: ", usr2)
}
In addition to what icza mentioned about the big struct being passed around. Pointer values are a way of saying that pointer semantics are at play and who ever uses the particular type should not make copy of the value which is being shared by the pointer.
If you look at the struct of File or http type, it maintains channels or some other pointer types which is unique to that value. Make a copy of the value (given to you by the pointer) would lead to hard to find bugs since the copied value might end up writing or reading to the pointer types of the original value.
I am new to GoLang, coming from the Delphi, C++ world - admittedly very excited about this language, which I think is destined to become "the next big thing".
I am trying to get a handle around how the Go parser and compiler handle pointers and references - can't seem to find any place where some clear rules are laid out.
In the below code sample for example, the return type *list.List and the local variable l are pointer types and require the pointer symbol * in their declarations, but they don't have to be dereferenced in use: l.PushBack(i). But in this same code the input parameter value *int64 is declared as a pointer and has to be dereferenced to be used properly: var i int64 = *value / 2
I assume that this is because list.List is a reference type, so the dereferencing is implicit when used, while int64 is a value type and must be handled just as any pointer to a value type, as in C++ for example: It must be dereferenced.
What is confusing to me is that even though *list.List has to be declared as a pointer type using *, when using the list instance, dereferencing is not required. This had me quite confused initially. Is that "just the way it is", or am I missing something?
Sample:
func GetFactors(value *int64) *list.List {
l := list.New()
l.PushBack(*value)
var i int64 = *value / 2
for ; i > 1; i-- {
if *value%i == 0 {
l.PushBack(i)
}
}
return l
}
All of the methods for a List have *List receivers: (http://golang.org/pkg/container/list/)
func (l *List) Back() *Element
func (l *List) Front() *Element
func (l *List) Init() *List
...
func (l *List) Remove(e *Element) interface{}
In your example l is of type *List, so there's no need to dereference them.
Suppose, instead, that you had something like this:
type A struct{}
func (a A) X() {
fmt.Println("X")
}
func (a *A) Y() {
fmt.Println("Y")
}
You are allowed to write:
a := A{}
a.X()
a.Y() // == (&a).Y()
Or you can do the following:
a := &A{}
a.X() // same like == (*a).X()
a.Y()
But it only works for method receivers. Go will not automatically convert function arguments. Given these functions:
func A(x *int) {
fmt.Println(*x)
}
func B(y int) {
fmt.Println(y)
}
This is invalid:
A(5)
You have to do this:
var x int
A(&x)
This is also invalid:
var y *int
B(y)
You have to do this:
B(*y)
Unlike C# or Java, when it comes to structs, Go does not make a distinction between reference and value types. A *List is a pointer, a List is not. Modifying a field on a List only modifies the local copy. Modifying a field on a *List modifies all "copies". (cause they aren't copies... they all point to the same thing in memory)
There are types which seem to hide the underlying pointer (like a slice contains a pointer to an array), but Go is always pass by value.