I have an object like os.Stdout and I want to know if it supports io.WriteCloser on my platform. I can get the type of my object, but it doesn't tell me anything about interfaces.
package main
import ("fmt"; "reflect"; "os")
func main() {
fmt.Println(reflect.TypeOf(os.Stdout))
}
This code prints *os.File to console.
I can manually lookup if os.File matches io.WriteCloser methods, but I am curious to get all interfaces that this object supports.
It's not an exactly answer on the question, because it is not for runtime. Anyway I think it maybe useful
Take a look on https://golang.org/lib/godoc/analysis/help.html
godoc has static analysis features. And it can display your type implements relations.
For example you can run godoc -http=:8081 -analysis=type and get all your packages documentation with type analysis.
To expand on the comment from #Volker regarding type assertions, that would look like this:
_, implements := interface{}(os.Stdout).(io.Reader)
It casts os.Stdout to an interface{} type and then attempts to assert that it is an io.Reader. Type assertions return two values; the first is the asserted value (or nil if assertion fails) and the second is a boolean indicating if the assertion was successful or not. If you omit capturing the second return value then a failed assertion will cause a panic.
For alternative, possibly more generic or runtime requirements the types package may have some useful functions based on reflection: https://godoc.org/golang.org/x/tools/go/types
Related
I need to test an app which uses Google Cloud Pubsub, and so must wrap its types pubsub.Client and pubsub.Subscriber for testing purposes. However, despite several attempts I can't get an interface around them which compiles.
The definitions of the methods I'm trying to wrap are:
func (s *Subscription) Receive(
ctx context.Context, f func(context.Context, *Message)) error
func (c *Client) Subscription(id string) *Subscription
Here is the current code. The Receiver interface (wrapper around Subscriber) seems to work, but I suspect it may need to change in order to fix SubscriptionMaker, so I've include both.
Note: I've tried several variations of where to reference and dereference pointers, so please don't tell me to change that unless you have an explanation of why your suggested configuration is the correct one or you've personally verified it compiles.
import (
"context"
"cloud.google.com/go/pubsub"
)
type Receiver interface {
Receive(context.Context, func(ctx context.Context, msg *pubsub.Message)) (err error)
}
// Pubsub subscriptions implement Receiver
var _ Receiver = &pubsub.Subscription{}
type SubscriptionMaker interface {
Subscription(name string) (s Receiver)
}
// Pubsub clients implement SubscriptionMaker
var _ SubscriptionMaker = pubsub.Client{}
Current error message:
common_types.go:21:5: cannot use "cloud.google.com/go/pubsub".Client literal (type "cloud.google.com/go/pubsub".Client) as type SubscriptionMaker in assignment:
"cloud.google.com/go/pubsub".Client does not implement SubscriptionMaker (wrong type for Subscription method)
have Subscription(string) *"cloud.google.com/go/pubsub".Subscription
want Subscription(string) Receiver
First, for most uses, using the ptest package is probably a much easier approach for testing pubsub. But of course, your specific question can apply to any library, and the below approach can be useful for many things, not just mocking pubsub.
Your broader goal of using interfaces to mock a library like this, is doable. But it is complicated when the library you wish to mock out returns concrete types that you cannot mock (probably due to unreported fields). The approach to be taken is much more involved than is often worth it, as there may be easier ways to test your code.
But if you're intent on doing this, the approach you must take is to not wrap the entire package in interfaces, not just the specific methods you wish to mock.
You would need to wrap any types that you wish to mock which are returned by or accepted by your interface, too. This usually means you also need to modify your production code (not just your test code), so this can sometimes be a deal-breaker for existing code bases.
Where I have usually done this before is when mocking something like the standard library's sql driver, but the same approach can be applied here. In essence, you would need to create a wrapper package for your pubsub library, which you use even in your production code. Again, this can be quite intrusive on existing codebases, but for the sake of illustration. Using your defined interfaces:
package mypubsub
import "cloud.google.com/go/pubsub"
type Receiver interface {
Recieve(context.Context, func(context.Context, *pubsub.Message) error)
}
type SubscriptionMaker interface {
Subscription(string) Receiver
}
You can then wrap the default implementation, for use in production code:
// defaultClient wraps the default pubsub Client functionality.
type defaultClient struct {
*pubsub.Client
}
func (d defaultImplementation) Subscription(name string) Receiver {
return d.Client.Subscription()
}
Naturally, you'd need to expand this package to wrap most or all of the pubsub package you're using. This can be a bit daunting.
But once you've done that, then use your mypubsub package everywhere in your code, instead of directly depending on the pubsub package. And now you can easily swap out a mock implementation anywhere you need for testing.
It can't be done.
When defining the type signature of a method on an interface, it must match exactly. func (c *Client) Subscription(id string) *Subscription returns a *Subscription, and a *Subscription is a valid Receiver, but it does not count as conforming to the interface method Subscription(string) Receiver. Go requires precise matching for function signatures, not the duck-typing style that it usually uses for interfaces.
Hey guys this code is part of a mock client, mock server interaction. I am having trouble understanding context.
Here I explicitly "associate" my tracker interface with context using 'WithValue' and then inject it into my request using WithContext. But when I check if my request's context contains the tracker interface I am returned the error "This context should contain a tracker" . What is it about context and WithValue that I am just not understanding?
var tracker Tracker
ctx := context.WithValue(context.Background(), contextKey, tracker)
req := httptest.NewRequest("GET", "localhost:12345/test", nil)
req.Header.Add(HEADER)
req = req.WithContext(ctx)
_, ok := ctx.Value(contextKey).(Tracker)
if !ok {
log.Fatal("1: This context should contain a tracker")
}
Tracker is an interface, and it's not set to anything, so it's nil. So, it can't change nil to a Tracker, so it fails.
https://play.golang.org/p/4-KQXlCR8vD
The problem isn't that tracker is nil, the problem is that it doesn't contain a value of a concrete type that implements Tracker. The value of tracker could be nil, and this would work, but it has to be a "typed" nil.
The type assertion fails, because type assertions and type switches only work when the instance being asserted has a concrete type. A value can be of a concrete type by declaration , assignment, type assertion , or a type switch case.
When you do the type assertion ctx.Value(contextKey).(Tracker)
There's (conceptually) a 2-step process to this:
Determine the ctx.Value(contextKey)'s concrete type.
Determine whether that concrete type implements Tracker.
Here's a playground example that hopefully illustrates this better: https://play.golang.org/p/ojYzLObkisd
The DoAny() function in there is emulating what is happening when you put your tracker into context and then try to retrieve it with the type assertion.
I know this is just a basic example, but it's not very good practice to use var something SomeInterfaceType, even if you do assign it a value. You should use concrete types for that. Or even better, just use type inference, and you won't have to worry about it. For example:
type Foo interface {
DoFoo() string
}
type MyFoo struct {}
func (f *MyFoo) DoFoo() string {
return "some foo value"
}
// not so good
var f Foo = new(MyFoo)
// good
var f *MyFoo = new(MyFoo)
// better
f := new(MyFoo)
This leverages the fact that interfaces are implicit in Go, and leads to much more obvious and maintainable code, especially in larger projects. Declaring a variable with an interface type is essentially making your variable poloymorphic, but the real power of interfaces isn't polymorphism. The real power is in using them as "functional requirements" for your package's exposed functions/methods. One rule that illustrates this really well is "Accept interfaces, return structs".
EDIT:
I've edited my original answer to correct some mistakes and make some improvements. I also would like to answer a follow-up question from the OP:
what I did not understand is that unless you have an object that utilizes that tracker's methods your interface is nil. Is this correct thinking?
I think what you're trying to say is correct, but the words used aren't quite right. First, there are no objects in Go, as it is not object-oriented. Where in OOP languages, objects hold instances of types, Go uses variables and constants to hold instances of types. So the concept exists, but not by the same name. So your tracker variable will be an instance of a type that satisfies your Tracker interface, but its value with be nil unless you assign it a non-nil instance of a type that satisfies the Tracker interface.
Values that can be converted to a JSON string via json.dumps are:
Scalars: Numbers and strings
Containers: Mapping and Iterable
Union[str, int, float, Mapping, Iterable]
Do you have a better suggestion?
Long story short, you have the following options:
If you have zero idea how your JSON is structured and must support arbitrary JSON blobs, you can:
Wait for mypy to support recursive types.
If you can't wait, just use object or Dict[str, object]. It ends up being nearly identical to using recursive types in practice.
If you don't want to constantly have to type-check your code, use Any or Dict[str, Any]. Doing this lets you avoid needing to sprinkle in a bunch of isinstance checks or casts at the expense of type safety.
If you know precisely what your JSON data looks like, you can:
Use a TypedDict
Use a library like Pydantic to deserialize your JSON into an object
More discussion follows below.
Case 1: You do not know how your JSON is structured
Properly typing arbitrary JSON blobs is unfortunately awkward to do with PEP 484 types. This is partly because mypy (currently) lacks recursive types: this means that the best we can do is use types similar to the one you constructed.
(We can, however, make a few refinements to your type. In particular, json.Dumps(...) actually does not accept arbitrary iterables. A generator is a subtype of Iterable, for example, but json.dumps(...) will refuse to serialize generators. You probably want to use something like Sequence instead.)
That said, having access to recursive types may not end up helping that much either: in order to use such a type, you would need to start sprinkling in isinstance checks or casts into your code. For example:
JsonType = Union[None, int, str, bool, List[JsonType], Dict[JsonType]]
def load_config() -> JsonType:
# ...snip...
config = load_config()
assert isinstance(config, dict)
name = config["name"]
assert isinstance(name, str)
So if that's the case, do we really need the full precision of recursive types? In most cases, we can just use object or Dict[str, object] instead: the code we write at runtime is going to be nearly the same in either case.
For example, if we changed the example above to use JsonType = object, we would still end up needing both asserts.
Alternatively, if you find sprinkling in assert/isinstance checks to be unnecessary for your use case, a third option is to use Any or Dict[str, Any] and have your JSON be dynamically typed.
It's obviously less precise than the options presented above, but asking mypy to not type check uses of your JSON dict and relying on runtime exceptions instead can sometimes be more ergonomic in practice.
Case 2: You know how your JSON data will be structured
If you do not need to support arbitrary JSON blobs and can assume it forms a particular shape, we have a few more options.
The first option is to use TypedDicts instead. Basically, you construct a type explicitly specifying what a particular JSON blob is expected to look like and use that instead. This is more work to do, but can let you gain more type-safety.
The main disadvantage of using TypedDicts is that it's basically the equivalent of a giant cast in the end. For example, if you do:
from typing import TypedDict
import json
class Config(TypedDict):
name: str
env: str
with open("my-config.txt") as f:
config: Config = json.load(f)
...how do we know that my-config.txt actually matches this TypedDict?
Well, we don't, not for certain.
This can be fine if you have full control over where the JSON is coming from. In this case, it might be fine to not bother validating the incoming data: just having mypy check uses of your dict is good enough.
But if having runtime validation is important to you, your options are to either implement that validation logic yourself or use a 3rd party library that can do it on your behalf, such as Pydantic:
from pydantic import BaseModel
import json
class Config(BaseModel):
name: str
env: str
with open("my-config.txt") as f:
# The constructor will raise an exception at runtime
# if the input data does not match the schema
config = Config(**json.load(f))
The main advantage of using these types of libraries is that you get full type safety. You can also use object attribute syntax instead of dict lookups (e.g. do config.name instead of config["name"]), which is arguably more ergonomic.
The main disadvantage is doing this validation does add some runtime cost, since you're now scanning over the entire JSON blob. This might end up introducing some non-trivial slowdowns to your code if your JSON happens to contain a large quantity of data.
Converting your data into an object can also sometimes be a bit inconvenient, especially if you plan on converting it back into a dict later on.
There has been a lengthy discussion (https://github.com/python/typing/issues/182) about the possibility of introducing a JSONType; however, no definitive conclusion has yet been reached.
The current suggestion is to just define JSONType = t.Union[str, int, float, bool, None, t.Dict[str, t.Any], t.List[t.Any]] or something similar in your own code.
Reading about value receivers vs pointer receivers across the web and stackoverflow, I understand the basic rule to be: If you don't plan to modify the receiver, and the receiver is relatively small, there is no need for pointers.
Then, reading about implementing the error interface (eg. https://blog.golang.org/error-handling-and-go), I see that examples of the Error() function all use pointer receiver.
Yet, we are not modifying the receiver, and the struct is very small.
I feel like the code is much nicer without pointers (return &appError{} vs return appError{}).
Is there a reason why the examples are using pointers?
First, the blog post you linked and took your example from, appError is not an error. It's a wrapper that carries an error value and other related info used by the implementation of the examples, they are not exposed, and not appError nor *appError is ever used as an error value.
So the example you quoted has nothing to do with your actual question. But to answer the question in title:
In general, consistency may be the reason. If a type has many methods and some need pointer receiver (e.g. because they modify the value), often it's useful to declare all methods with pointer receiver, so there's no confusion about the method sets of the type and the pointer type.
Answering regarding error implementations: when you use a struct value to implement an error value, it's dangerous to use a non-pointer to implement the error interface. Why is it so?
Because error is an interface. And interface values are comparable. And they are compared by comparing the values they wrap. And you get different comparison result based what values / types are wrapped inside them! Because if you store pointers in them, the error values will be equal if they store the same pointer. And if you store non-pointers (structs) in them, they are equal if the struct values are equal.
To elaborate on this and show an example:
The standard library has an errors package. You can create error values from string values using the errors.New() function. If you look at its implementation (errors/errors.go), it's simple:
// Package errors implements functions to manipulate errors.
package errors
// New returns an error that formats as the given text.
func New(text string) error {
return &errorString{text}
}
// errorString is a trivial implementation of error.
type errorString struct {
s string
}
func (e *errorString) Error() string {
return e.s
}
The implementation returns a pointer to a very simple struct value. This is so that if you create 2 error values with the same string value, they won't be equal:
e1 := errors.New("hey")
e2 := errors.New("hey")
fmt.Println(e1, e2, e1 == e2)
Output:
hey hey false
This is intentional.
Now if you would return a non-pointer:
func New(text string) error {
return errorString{text}
}
type errorString struct {
s string
}
func (e errorString) Error() string {
return e.s
}
2 error values with the same string would be equal:
e1 = New("hey")
e2 = New("hey")
fmt.Println(e1, e2, e1 == e2)
Output:
hey hey true
Try the examples on the Go Playground.
A shining example why this is important: Look at the error value stored in the variable io.EOF:
var EOF = errors.New("EOF")
It is expected that io.Reader implementations return this specific error value to signal end of input. So you can peacefully compare the error returned by Reader.Read() to io.EOF to tell if end of input is reached. You can be sure that if they occasionally return custom errors, they will never be equal to io.EOF, this is what errors.New() guarantees (because it returns a pointer to an unexported struct value).
Errors in go only satisfy the error interface, i.e. provide a .Error() method. Creating custom errors, or digging through Go source code, you will find errors to be much more behind the scenes. If a struct is being populated in your application, to avoid making copies in memory it is more efficient to pass it as a pointer. Furthermore, as illustrated in The Go Programming Language book:
The fmt.Errorf function formats an error message using fmt.Sprintf and returns a new error value. We use it to build descriptive errors by successively prefixing additional context information to the original error message. When the error is ultimately handled by the program’s main function, it should provide a clear causal chain from the root problem to the overall failure, reminiscent of a NASA accident investigation:
genesis: crashed: no parachute: G-switch failed: bad relay orientation
Because error messages are frequently chained together, message strings should not be capitalized and newlines should be avoided. The resulting errors may be long, but they will be self-contained when found by tools like grep.
From this we can see that if a single 'error type' holds a wealth of information, and on top of this we are 'chaining' them together to create a detailed message, using pointers will be the best way to achieve this.
We can look at this from the error handling's perspective, instead of the error creation.
Error Definiton Side's Story
type ErrType1 struct {}
func (e *ErrType1) Error() string {
return "ErrType1"
}
type ErrType2 struct {}
func (e ErrType2) Error() string {
return "ErrType1"
}
Error Handler Side's Story
err := someFunc()
switch err.(type) {
case *ErrType1
...
case ErrType2, *ErrType2
...
default
...
}
As you can see, if you implements a error type on a value receiver, then when you are doing the type assertion, you need to worry about both cases.
For ErrType2, both &ErrType2{} and ErrType2{} satisfy the interface.
Because someFunc returns an error interface, you never know if it returns a struct value or a struct pointer, especially when someFunc isn't written by you.
Therefore, by using a pointer receiver doesn't stop a user from returning a pointer as an error.
That been said, all other aspects such as
Stack vs. Heap (memory allocation, GC pressure) still apply.
Choose your implementation according to your use cases.
In general, I prefer to a pointer receiver for the reason I demonstrated above. I prefer to Friendly API over performance and sometimes, when error type contains huge information, it's more performant.
No :)
https://blog.golang.org/error-handling-and-go#TOC_2.
Go interfaces allow for anything that complies with the error interface to be handled by code expecting error
type error interface {
Error() string
}
Like you mentioned, If you don't plan to modify state there is little incentive to pass around pointers:
allocating to heap
GC pressure
Mutable state and concurrency, etc
On a random rant , Anecdotally, I personally think that seeing examples like this one are why new go programers favor pointer receivers by default.
The tour of go explains the general reasons for pointer receivers pretty well:
https://tour.golang.org/methods/8
There are two reasons to use a pointer receiver.
The first is so that the method can modify the value that its receiver points to.
In general, all methods on a given type should have either value or pointer receivers, but not a mixture of both.
I have following code in main():
msgs, err := ch.Consume(
q.Name, // queue
//..
)
cache := ttlru.New(100, ttlru.WithTTL(5 * time.Minute)) //Cache type
//log.Println(reflect.TypeOf(msgs)) 'chan amqp.Delivery'
go func() {
//here I use `cache` and `msgs` as closures. And it works fine.
}
I decided to create separate function for instead of anonymous.
I declared it as func hitCache(cache *ttlru.Cache, msgs *chan amqp.Delivery) {
I get compile exception:
./go_server.go:61: cannot use cache (type ttlru.Cache) as type *ttlru.Cache in argument to hitCache:
*ttlru.Cache is pointer to interface, not interface
./go_server.go:61: cannot use msgs (type <-chan amqp.Delivery) as type *chan amqp.Delivery in argument to hitCache
Question: How should I pass msg and cache into the new function?
Well, if the receiving variable or a function parameter expects a value
of type *T — that is, "a pointer to T",
and you have a variable of type T, to get a pointer to it,
you have to get the address of that variable.
That's because "a pointer" is a value holding an address.
The address-taking operator in Go is &, so you need something like
hitCache(&cache, &msgs)
But note that some types have so-called "reference semantics".
That is, values of them keep references to some "hidden" data structure.
That means when you copy such values, you're copying references which all reference the same data structure.
In Go, the built-in types maps, slices and channels have reference semantics,
and hence you almost never need to pass around pointers to the values of such types (well, sometimes it can be useful but not now).
Interfaces can be thought of to have reference semantics, too (let's not for now digress into discussing this) because each value of any interface type contains two pointers.
So, in your case it's better to merely not declare the formal parameters of your function as pointers — declare them as "plain" types and be done with it.
All in all, you should definitely complete some basic resource on Go which explains these basic matters in more detail and more extensively.
You're using pointers in the function signature but not passing pointers - which is fine; as noted in the comments, there is no reason to use pointers for interface or channel values. Just change the function signature to:
hitCache(cache ttlru.Cache, msgs chan amqp.Delivery)
And it should work fine.
Pointers to interfaces are nearly never used. You may simplify things and use interfaces of pass by value.