I Go, I assumed slices were passed by reference, but this seems to work for values
but not for the array itself. For example, If I have this struct:
l := Line{
Points: []Point{
Point{3, 4},
},
}
I can define a variable, which gets passed a reference to the struct's slice
slice := l.Points
And then if I modify it, the original struct referenced by the variable
is going to reflect those modifications.
slice[0].X = 1000
fmt.Printf(
"This value %d is the same as this %d",
slice[0].X,
l.Points[0].X,
)
This differs from the behavior of arrays which, I assume, are passed by value.
So, for example, if I had defined the previous code using an array:
l := Line{
Points: [1]Point{
Point{3, 4},
},
}
arr := l.Points
arr[0].X = 1000
fmt.Println(arr.[0].X != s.Points[0].X) // equals true, original struct is untouched
Then, the l struct wouldn't have been modified.
Now, if I want to modify the slice itself I obviously cannot do this:
slice = append(slice, Point{99, 100})
Since that would only redefine the slice variable, losing the original reference.
I know I can simply do this:
l.Points = append(l.Points, Point{99, 100})
But, in some cases, it is more convenient to have another variable instead of having
to type the whole thing.
I tried this:
*slice = append(*slice, Point{99, 100})
But it doesn't work as I am trying to dereference something that apparently is not a pointer.
I finally tried this:
slice := &l.Points
*slice = append(l.Points, Point{99, 100})
And it works, but I am not sure what is happening. Why is the value of slice not overwritten? How does append works here?
Let's dispense first with a terminology issue. The Go language specification does not use the word reference the way you are using it. Go does however have pointers, and pointers are a form of reference. In addition, slices and maps are kind of special as there's some underlying data—the array underneath a slice, or the storage for a map—that may or may not already exist or be created by declaring or defining a variable whose type is slice of T or map[T1]T2 for some type T or type-pair T1 and T2.1
We can take your usage of the word reference to mean explicit pointer when talking about, e.g.:
func f1(p *int) {
// code ...
}
and the implied pointer when talking about:
func f2(m map[T1]T2) { ... }
func f3(s []T) { ... }
In f1, p really is a pointer: it thus refers to some actual int, or is nil. In f2, m refers to some underlying map, or is nil. In f3, s refers to some underlying array, or is nil.
But if you write:
l := Line{
Points: []Point{
Point{3, 4},
},
}
then you must have written:
type Line struct {
// ... maybe some fields here ...
Points []Point
// ... maybe more fields here ...
}
This Line is a struct type. It is not a slice type; it is not a map type. It contains a slice type but it is not itself one.
You now talk about passing these slices. If you pass l, you're passing the entire struct by value. It's pretty important to distinguish between that, and passing the value of l.Points. The function that receives one of these arguments must declare it with the right type.
For the most part, then, talking about references is just a red herring—a distraction from what's really going on. What we need to know is: What variables are you assigning what values, using what source code?
With all of that out of the way, let's talk about your actual code samples:
l.Points = append(l.Points, Point{99, 100})
This does just what it says:
Pass l.Points to append, which is a built-in as it is somewhat magically type-flexible (vs the rest of Go, where types are pretty rigid). It takes any value of type []T (slice of T, for any valid type T) plus one or more values of type T, and produces a new value of the same type, []T.
Assigns the result to l.Points.
When append does its work, it may:
receive nil (of the given type): in this case, it creates the underlying array, or
receive a non-nil slice: in this case, it writes into the underlying array or discards that array in favor of a new larger-capacity array as needed.2
So in all cases, the underlying array may have, in effect, just been created or replaced. It's therefore important that any other use of the same underlying array be updated appropriately. Assigning the result back to l.Points updates the—presumably one-and-only—slice variable that refers to the underlying array.
We can, however, break these assumptions:
s2 := l.Points
Now l.Points and s2 both refer to the (single) underlying array. Operations that modify that underlying array will, at least potentially, affect both s2 and l.Points.
Your second example is itself OK:
*slice = append(*slice, Point{99, 100})
but you haven't shown how slice itself was declared and/or assigned-to.
Your third example is fine as well:
slice := &l.Points
*slice = append(l.Points, Point{99, 100})
The first of these lines declares-and-initializes slice to point to l.Points. The variable slice therefore has type *[]Point. Its value—the value in slice, that is, rather than that in *slice—is the address of l.Points, which has type []Point.
The value in *slice is the value in l.Points. So you could write:
*slice = append(*slice, Point{99, 100})
here. Since *slice is just another name for l.Points, you can also write:
l.Points = append(*slice, Point{99, 100})
You only need to use *slice if there's some reason that l.Points is not available,3 but you may use *slice if that's more convenient. Reading *slice reads l.Points and updating *slice updates l.Points.
1To see what I mean by may or may not be created here, consider:
var s []int
vs:
var s = []int{42}
The first leaves s == nil while the second creates an underlying array with the capacity to hold the one int value 42, holding the one int value 42, so that s != nil.
2It's not clear to me whether there is a promise never to write on an existing slice-array whose capacity is greater than its current length, but not sufficient to hold the final result. That is, can append first append 10 objects to the existing underlying array, then discover that it needs a bigger array and expand the underlying array? The difference is observable if there are other slice values referring to the existing underlying array.
3Here, a classic example would occur if you have reason to pass l.Points or &l.Points to some existing (pre-written) function:
If you need pass l.Points—the slice value—to some existing function, that existing function cannot change the slice value, but could change the underlying array. That's probably a bad plan, so if it does do this, make sure that this is OK! If it only reads the slice and underlying array, that's a lot safer.
If you need to pass &l.Points—a value that points to the slice value—to some existing function, that existing function can change both the slice, and the underlying array.
If you're writing a new function, it's up to you to write it in whatever manner is most appropriate. If you're only going to read the slice and underlying array, you can take a value of type []Point. If you intend to update the slice in place, you should take a value of type *[]Point—pointer to slice of Point.
Append returns a new slice that may modify the original backing array of the initial slice. The original slice will still point to the original backing array, not the new one (which may or may not be in the same place in memory)
For example (playground)
slice := []int{1,2,3}
fmt.Println(len(slice))
// Output: 3
newSlice := append(slice, 4)
fmt.Println(len(newSlice))
// Output: 4
fmt.Println(len(slice))
// Output: 3
While a slice can be described as a "fat pointer to an array", it is not a pointer and therefore you can't dereference it, which is why you get an error.
By creating a pointer to a slice, and using append as you did above, you are setting the slice the pointer points to to the "new" slice returned by append.
For more information, check out Go Slice Usage And Internals
Your first attempt didn't work because slices are not pointers, they can be considered reference types. Append will modify the underlying array if it has enough capacity, otherwise it returns a new slice.
You can achieve what you want with a combination of your two attempts.
playground
l := Line{
Points: []Point{
Point{3, 4},
},
}
slice := &l.Points
for i := 0; i < 100; i++ {
*slice = append(*slice, Point{99 + i, 100 + i})
}
fmt.Println(l.Points)
I know that this might be sacrilegious, but, for me, it is useful to think of slices
as structs.
type Slice struct {
len int
cap int
Array *[n]T // Pointer to array of type T
}
Since in languages like C, the [] operator is also a dereferencing operator, we can think that every time we are accessing a slice, we are actually dereferencing the underlying array and assigning some value to it. That is:
var s []int
s[0] = 1
Might be thought of as equivalent to (in pseudo-code):
var s Slice
*s.Array[0] = 1
That is why we can say that slices are "pointers". For that reason, it can modify its underlying array like this:
myArray := [3]int{1,1,1}
mySlice := myArray[0:1]
mySlice = append(mySlice, 2, 3) // myArray == mySlice
Modifying mySlice also modifies myArray, since the slice stores a pointer to the array and, on appending, we are dereferencing that pointer.
This behavior, nonetheless, is not always like this. If we exceed the capacity of the original array, a new array is created and the original array is left untouched.
myArray := [3]int{1,1,1}
mySlice := myArray[0:1]
mySlice = append(mySlice, 2, 3, 4, 5) // myArray != mySlice
The confusion arises when we try to treat the slice itself as an actual pointer. Since we can modify an underlying array by appending to it, we are led to believe that in this case:
sliceCopy := mySlice
sliceCopy = append(sliceCopy, 6)
both slices, slice and sliceCopy are the same, but they are not. We have to explicitly pass a reference to the memory address of the slice (using the & operator) in order to modify it. That is:
sliceAddress := &mySlice
*sliceAddress = append(mySlice, 6) // or append(*sliceAddress, 6)
See also
https://forum.golangbridge.org/t/slice-pass-as-value-or-pointer/2866/4
https://blog.golang.org/go-slices-usage-and-internals
https://appliedgo.net/slices/
As I understand it, I cannot define equality for user-defined types in Go. So what would be the idiomatic way of computing the number of distinct objects of some custom type (possibly recursively defined). Here is an example of the kind of thing I am trying to do.
package main
import "fmt"
type tree struct {
left *tree
right *tree
}
func shapeOf(a tree) string {
temp := "{"
if a.left != nil {
temp += shapeOf(*(a.left))
}
temp += "}{"
if a.right != nil {
temp += shapeOf(*(a.right))
}
temp += "}"
return temp;
}
func main() {
a := tree{nil, nil}
b := tree{nil, &a}
c := tree{nil, nil}
d := tree{nil, &c}
e := tree{nil, nil}
f := tree{&e, nil}
s := make(map[string]bool)
s[shapeOf(b)] = true
s[shapeOf(d)] = true
s[shapeOf(f)] = true
fmt.Println(len(s)) // As required, prints 2 because the first two trees have the same shape
}
It works, but the use of strings is extremely ugly, and probably inefficient too. Obviously I could easily write a recursive method to tell if two trees are equal - something like
func areEqual(a, b tree) bool
but this wouldn't enable me to use trees as map keys. What is the idiomatic Go way to do something like this?
You cannot define equality for user-defined type because it is already defined by go. Basically, all there is to know about it is explained in the comparable section.
Short story: two struct values can be compared if their fields can be compared (no slice, map or function). And same thing for equality: two structs are equal if their fields are equal. In your case, the problem is that for comparing pointers, Golang compares the memory addresses, not the struct they point to.
So, is this possible to count distinct values of a certain struct ? Yes, if the struct contain no nested slice, map, function or pointer. For recursive types, that's not possible because you cannot define something like this:
type tree struct {
left tree
right tree
}
The idiomatic way of testing the equality of recursive types is to use reflect.DeepEqual(t1, t2 interface{}) as it follows indirections. However, this method is inefficient because uses heavy reflection. In your case, I do not think there is any clean and elegant solution to get what you want.
I am trying to write a csv parser using the example provided here. It works great for all native types but I am having trouble with any structs that contain a timestamp of type time.Time. It exits with an error of "cannot convert this type".
This is the code.
//For each field in a given struct...
//Get a field
val := sv.Field(i)
// this is necessary because Kind can't tell
// distinguish between a primitive type
// and a type derived from it. We're looking
// for a Value interface defined on
// the pointer to this value
_, ok := val.Addr().Interface().(Value)
if ok {
val = val.Addr()
kind = value_k
} else {
switch Kind {
case reflect.Int, reflect.Int16, reflect.Int8,
reflect.Int32, reflect.Int64:
kind = int_k
case reflect.Uint, reflect.Uint16, reflect.Uint8,
reflect.Uint32, reflect.Uint64:
kind = uint_k
case reflect.Float32, reflect.Float64:
kind = float_k
case reflect.String:
kind = string_k
default:
// Kind is Struct here
kind = value_k
_, ok := val.Interface().(Value)
if !ok {
err = os.NewError("cannot convert this type ")
this = nil
return
}
}
}
What this code does is take an interface and a reader. It attempts to match the field headers in the reader (csv file) with field names in the interface. It also reflects on the interface (struct) and collects positional a type information for later setting the fields in the iterator. It is this step that is failing for non-native types.
I've tried a few methods to work around this but the only thing that seems to work is changing the timestamp to a string. I am undoubtedly missing something and would greatly appreciate some guidance.
I was hitting an issue in a project I'm working on. I found a way around it, but I wasn't sure why my solution worked. I'm hoping that someone more experience with how Go pointers work could help me.
I have a Model interface and a Region struct that implements the interface. The Model interface is implemented on the pointer of the Region struct. I also have a Regions collection which is a slice of Region objects. I have a method that can turn a Regions object into a []Model:
// Regions is the collection of the Region model
type Regions []Region
// Returns the model collection as a list of models
func (coll *Regions) ToModelList() []Model {
output := make([]Model, len(*coll))
for idx, item := range *coll {
output[idx] = &item
}
return output
}
When I run this code, I end up with the first pointer to the Region outputted multiple times. So, if the Regions collection has two distinct items, I will get the same address duplicated twice. When I print the variables before I set them in the slice, they have the proper data.
I messed with it a little bit, thinking Go might be reusing the memory address between loops. This solution is currently working for me in my tests:
// Returns the model collection as a list of models
func (coll *Regions) ToModelList() []Model {
output := make([]Model, len(*coll))
for idx, _ := range *coll {
i := (*coll)[idx]
output[idx] = &i
}
return output
}
This gives the expected output of two distinct addresses in the output slice.
This honestly seems like a bug with the range function reusing the same memory address between runs, but I always assume I'm missing something in cases like this.
I hope I explained this well enough for you. I'm surprised that the original solution did not work.
Thanks!
In your first (non working) example item is the loop variable. Its address is not changing, only its value. That's why you get the same address in output idx times.
Run this code to see the mechanics in action;
func main() {
coll := []int{5, 10, 15}
for i, v := range coll {
fmt.Printf("This one is always the same; %v\n", &v)
fmt.Println("This one is 4 bytes larger each iteration; %v\n", &coll[i])
}
}
There is just one item variable for the entire loop, which is assigned the corresponding value during each iteration of the loop. You do not get a new item variable in each iteration. So you are just repeatedly taking the address of the same variable, which will of course be the same.
On the other hand, if you declared a local variable inside the loop, it will be a new variable in each iteration, and the addresses will be different:
for idx, item := range *coll {
temp := item
output[idx] = &temp
}
I'm just starting to evaluate Rust. Using Rust and the sqlite3 repo on Github, I'm attempting to determine EOF for a Cursor. I'm not sure how to do that "correctly", I think it may be via the "match" statement.
The 2nd line in the following 2 lines is how I'm currently determining EOF, but this is obviously not the "correct" way:
let oNextResult:sqlite::types::ResultCode = oDbCursor.step();
tDone = (fmt!("%?", oNextResult) == ~"SQLITE_DONE");
The following is the unfinished function containing the above 2 lines. Please excuse the lack of Rust naming-convention, but I will look at implementing that.
/********************
**** Update Data ****
*********************/
fn fUpdateData(oDb1:&sqlite::database::Database, iUpdateMax:int) -> bool {
println(fmt!("Updating %d Rows .......", iUpdateMax));
let sSql:~str = fmt!("Select ikey, sname, iborn, dbal from test LIMIT %d",
iUpdateMax);
let oDbExec = oDb1.exec(sSql);
if oDbExec.is_err() {
println(fmt!("Select Failed! : %?, sql=%s", oDbExec, sSql));
return false;
}
println("Select succeeded. Processing select list .....");
let mut iUpdateCount: int = 0;
let oDbCursor:sqlite::cursor::Cursor = oDb1.prepare(sSql, &None).unwrap();
let mut tDone:bool = false;
while !tDone {
let oNextResult:sqlite::types::ResultCode = oDbCursor.step();
tDone = (fmt!("%?", oNextResult) == ~"SQLITE_DONE");
if !tDone {
let sKey = oDbCursor.get_text(0);
let sName = oDbCursor.get_text(1);
let sBorn = oDbCursor.get_text(2);
let sBal = oDbCursor.get_text(3);
println(fmt!("skey = %s, sname = %s, sBorn = %s, sBal = %s", sKey,
sName, sBorn, sBal));
iUpdateCount += 1;
}
}
println(fmt!("Update succeeded, items updated = %d", iUpdateCount));
return true;
}
I don't know if there is a correct way at the moment but you can also the result codes from the types module:
use sqlite::types::ResultCode;
and then do something like this so there's no need for using fmt!
while cursor.step() == SQLITE_ROW {...}
or this:
while cursor.get_column_count() != 0 {...; cursor.step()}
Function get_column_count returns an int. If there's no data it will return 0. It calls int sqlite3_data_count(sqlite3_stmt *pStmt); under the hood and here's what sqlite docs say about it:
The sqlite3_data_count(P) interface returns the number of columns in
the current row of the result set of prepared statement P. If prepared
statement P does not have results ready to return (via calls to the
sqlite3_column_*() of interfaces) then sqlite3_data_count(P) returns
0. The sqlite3_data_count(P) routine also returns 0 if P is a NULL pointer. The sqlite3_data_count(P) routine returns 0 if the previous
call to sqlite3_step(P) returned SQLITE_DONE. The
sqlite3_data_count(P) will return non-zero if previous call to
sqlite3_step(P) returned SQLITE_ROW, except in the case of the PRAGMA
incremental_vacuum where it always returns zero since each step of
that multi-step pragma returns 0 columns of data.
As it's mentioned on the readme file rustsqlite interface is not finalized, watch out for changes.