My Go gRPC server is instrumented with
Google Tracing span exporter:
import texporter "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace"
...
gcp, err := texporter.NewExporter()
...
tracer := trace.NewTracerProvider(trace.WithSyncer(traceExporter),
trace.WithSampler(trace.AlwaysSample()))
otel.SetTracerProvider(tracer)
otelgrpc interceptors registered on the gRPC server.
unaryInterceptors := grpc_middleware.WithUnaryServerChain(
otelgrpc.UnaryServerInterceptor(),
)
streamInterceptors := grpc_middleware.WithStreamServerChain(
otelgrpc.StreamServerInterceptor(),
)
Now I'm trying to create a trace span inside the RPC implementation so I can have child spans for the method e.g.:
func (s *srv) Test(ctx context.Context, req *pb.Request) (*pb.TestResponse, error) {
// create a span here
span1 := [??????].Start()
doWork1()
span1.End()
span2 := [??????].Start()
doWork2()
span2.End()
...
}
However it is wildly unclear from the OpenTelemetry docs how does one do that.
Closest I've gotten is otel.GetTracerProvider().Tracer("some string here???") providing a Start(ctx)(ctx,Span). But it is not clear to me what string to provide here (my exporter doesn't have a url like the docs indicate) and this seems quite inconvenient.
I'm thinking there's something like a otelgrpc.SpanFromCtx(ctx) method somewhere that pulls the tracer+creates a span with the rpc ctx that I'm not finding. Sadly the docs are quite lacking on OT+gRPC+Go.
Since you are using github.com/GoogleCloudPlatform/opentelemetry-operations-go, you should use itself to create span rather than opentelemetry.
You can create a new span like this:
// Create custom span.
tracer := otel.TraceProvider().Tracer("example.com/trace")
err = func(ctx context.Context) error {
ctx, span := tracer.Start(ctx, "foo")
defer span.End()
// Do some work.
return nil
}(ctx)
For more detail, you can reference OpenTelemetry Google Cloud Trace Exporter
Related
I'm trying to create a document in firestore using golang sdk
How can I check the error code returned by the call to DocumentRef.Get() ?
Right now I'm checking if the error contains "NotFound"... Definitely not ideal but can't seem to find anything else
Current Code:
doc := firestoreClient.Collection("users").Doc(id)
userSnapshot, err := doc.Get(ctx)
var u *User
if err != nil {
isNotFound := strings.Contains(err.Error(), "NotFound")
if isNotFound {
// create the document
}
}
Ideally I would like to do something like if err.Code == NotFound // handle appropriately
Or even better, if there's a right pattern for how to GetOrCreate an item in Firestore (using golang)
I just found this in the official docs(https://pkg.go.dev/cloud.google.com/go/firestore#DocumentRef.Get)
import (
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
status.Code(err) == codes.NotFound
Thanks for the pointer here! What is the idomatic way to get a single document from Firestore using Go?
Another thread: https://github.com/googleapis/google-cloud-go/issues/861
Hello fellow developers.
I am trying to learn GO while constructing a simple web API using sqlite3. I got stuck at somepoint where i am unable to delete rows from my table by sending a DELETE request from postman. I am trying to use the code below to delete a row. I have already verified that I have access to db and I can also delete rows by using command tool of sqlite3. I do not understand what is wrong!
func deleteArticle(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
params := mux.Vars(r) // get any params
db := connectToDB(dbName)
defer db.Close()
_, err := db.Query("DELETE FROM article WHERE id=" + params["id"])
if err != nil {
fmt.Fprintf(w, "article couldn't be found in db")
}
}
Here is the navigation part:
myRouter.HandleFunc("/articles/{id}", deleteArticle).Methods("DELETE")
No mather what I do I cannot delete an article from db using postman.
Thanks bunches.
Thanks to #mkopriva 's comments I have learned that
1.
It is very important that you do not use Query nor QueryRow for SQL
queries that do not return any rows, for these cases use the Exec
method. When you use Query you always have to assign the result to a
non-blank identifier, i.e. anything but _, and then invoke the Close
method on that once you're done with the result. If you do not do that
then your application will leak db connections and very soon will
start crashing.
2.
when you want to pass user input (including record ids) to your
queries you have to utilize, at all times, the parameter-reference
syntax supported by the sql dialect and/or dirver you are using, and
then pass the input separately. That means that you should never do
Exec("DELETE FROM article WHERE id=" + params["id"]),
instead you should always do
Exec("DELETE FROM article WHERE id= ?",params["id"])
If you do not do it the proper way and instead continue
using plain string concatenation your app will be vulnerable to SQL
injection attacks.
Regarding this information I have changed my code into:
func deleteArticle(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
params := mux.Vars(r) // get any params
db := connectToDB(dbName)
defer db.Close()
fmt.Printf("%q\n", params["id"])
statement, err := db.Prepare("DELETE FROM article WHERE id= ?")
if err != nil {
fmt.Fprintf(w, "article couldn't be found in db")
}
statement.Exec(params["id"])
}
Which has solved my problem. So thank you #mkopriva
So I'm receiving a request to my server that looks a little something like this
http://localhost:8080/#access_token=tokenhere&scope=scopeshere
and I can't seem to find a way to parse the token from the url.
If the # were a ? I could just parse it a standard query param.
I tried to just getting everything after the / and even the full URL, but with no luck.
Any help is greatly appreciated.
edit:
So I've solved the issue now, and the correct answer is you can't really do it in GO. So I made a simple package that will do it on the browser side and then send the token back to the server.
Check it out if you're trying to do local twitch API stuff in GO:
https://github.com/SimplySerenity/twitchOAuth
Anchor part is not even (generally) sent by a client to the server.
Eg, browsers don't send it.
For parse urls use the golang net/url package: https://golang.org/pkg/net/url/
OBS: You should use the Authorization header for send auth tokens.
Example code with extracted data from your example url:
package main
import (
"fmt"
"net"
"net/url"
)
func main() {
// Your url with hash
s := "http://localhost:8080/#access_token=tokenhere&scope=scopeshere"
// Parse the URL and ensure there are no errors.
u, err := url.Parse(s)
if err != nil {
panic(err)
}
// ---> here is where you will get the url hash #
fmt.Println(u.Fragment)
fragments, _ := url.ParseQuery(u.Fragment)
fmt.Println("Fragments:", fragments)
if fragments["access_token"] != nil {
fmt.Println("Access token:", fragments["access_token"][0])
} else {
fmt.Println("Access token not found")
}
// ---> Others data get from URL:
fmt.Println("\n\nOther data:\n")
// Accessing the scheme is straightforward.
fmt.Println("Scheme:", u.Scheme)
// The `Host` contains both the hostname and the port,
// if present. Use `SplitHostPort` to extract them.
fmt.Println("Host:", u.Host)
host, port, _ := net.SplitHostPort(u.Host)
fmt.Println("Host without port:", host)
fmt.Println("Port:",port)
// To get query params in a string of `k=v` format,
// use `RawQuery`. You can also parse query params
// into a map. The parsed query param maps are from
// strings to slices of strings, so index into `[0]`
// if you only want the first value.
fmt.Println("Raw query:", u.RawQuery)
m, _ := url.ParseQuery(u.RawQuery)
fmt.Println(m)
}
// part of this code was get from: https://gobyexample.com/url-parsing
Say I have this http handler:
func SomeHandler(w http.ResponseWriter, r *http.Request) {
data := GetSomeData()
_, err := w.Write(data)
}
Should I check the error returned by w.Write? Examples I've seen just ignore it and do nothing. Also, functions like http.Error() do not return an error to be handled.
It's up to you. My advice is that unless the documentation of some method / function explicitly states that it never returns a non-nil error (such as bytes.Buffer.Write()), always check the error and the least you can do is log it, so if an error occurs, it will leave some mark which you can investigate should it become a problem later.
This is also true for writing to http.ResponseWriter.
You might think ResponseWriter.Write() may only return errors if sending the data fails (e.g. connection closed), but that is not true. The concrete type that implements http.ResponseWriter is the unexported http.response type, and if you check the unexported response.write() method, you'll see it might return a non-nil error for a bunch of other reasons.
Reasons why ResponseWriter.Write() may return a non-nil error:
If the connection was hijacked (see http.Hijacker): http.ErrHijacked
If content length was specified, and you attempt to write more than that: http.ErrContentLength
If the HTTP method and / or HTTP status does not allow a response body at all, and you attempt to write more than 0 bytes: http.ErrBodyNotAllowed
If writing data to the actual connection fails.
Even if you can't do anything with the error, logging it may be of great help debugging the error later on. E.g. you (or someone else in the handler chain) hijacked the connection, and you attempt to write to it later; you get an error (http.ErrHijacked), logging it will reveal the cause immediately.
Tip for "easy" logging errors
If you can't do anything with the occasional error and it's not a "showstopper", you may create and use a simple function that does the check and logging, something like this:
func logerr(n int, err error) {
if err != nil {
log.Printf("Write failed: %v", err)
}
}
Using it:
logerr(w.Write(data))
Tip for "auto-logging" errors
If you don't even want to use the logerr() function all the time, you may create a wrapper for http.ResponseWriter which does this "automatically":
type LogWriter struct {
http.ResponseWriter
}
func (w LogWriter) Write(p []byte) (n int, err error) {
n, err = w.ResponseWriter.Write(p)
if err != nil {
log.Printf("Write failed: %v", err)
}
return
}
Using it:
func SomeHandler(w http.ResponseWriter, r *http.Request) {
w = LogWriter{w}
w.Write([]byte("hi"))
}
Using LogWriter as a wrapper around http.ResponseWriter, should writes to the original http.ResponseWriter fail, it will be logged automatically.
This also has the great benefit of not expecting a logger function to be called, so you can pass a value of your LogWriter "down" the chain, and everyone who attempts to write to it will be monitored and logged, they don't have to worry or even know about this.
But care must be taken when passing LogWriter down the chain, as there's also a downside to this: a value of LogWriter will not implement other interfaces the original http.ResponseWriter might also do, e.g. http.Hijacker or http.Pusher.
Here's an example on the Go Playground that shows this in action, and also shows that LogWriter will not implement other interfaces; and also shows a way (using 2 "nested" type assertions) how to still get out what we want from LogWriter (an http.Pusher in the example).
I want to add to #icza solution. You don't need to create logging structure, you can use simple function:
func logWrite(write func([]byte) (int, error), body []byte) {
_, err := write(body)
if err != nil {
log.Printf("Write failed: %v", err)
}
}
Take a look on that approach based on the #icza code: https://play.golang.org/p/PAetVixCgv4
I am writing some Go web services (also implementing the webserver in Go with http.ListenAndServe).
I have a map of structs which I would like to keep in memory (with an approximate data size of 100Kb) to be used by different HTTP requests.
How can this be achieved in Go? I am thinking to use global package variables or caching systems (like memcache/groupcache).
In addition to the answers you've already received, consider making use of receiver-curried method values and http.HandlerFunc.
If your data is data that is loaded before the process starts, you could go with something like this:
type Common struct {
Data map[string]*Data
}
func NewCommon() (*Common, error) {
// load data
return c, err
}
func (c *Common) Root(w http.ResponseWriter, r *http.Request) {
// handler
}
func (c *Common) Page(w http.ResponseWriter, r *http.Request) {
// handler
}
func main() {
common, err := NewCommon()
if err != nil { ... }
http.HandleFunc("/", common.Root)
http.HandleFunc("/page", common.Page)
http.ListenAndServe(...)
}
This works nicely if all of the Common data is read-only. If the Common data is read/write, then you'll want to have something more like:
type Common struct {
lock sync.RWMutex
data map[string]Data // Data should probably not have any reference fields
}
func (c *Common) Get(key string) (*Data, bool) {
c.lock.RLock()
defer c.lock.RUnlock()
d, ok := c.data[key]
return &d, ok
}
func (c *Common) Set(key string, d *Data) {
c.lock.Lock()
defer c.lock.Unlock()
c.data[key] = *d
}
The rest is basically the same, except instead of accessing the data through the receiver's fields directly, you'd access them through the getters and setters. In a webserver where most of the data is being read, you will probably want an RWMutex, so that reads can be executed concurrently with one another. Another advantage of the second approach is that you've encapsulated the data, so you can add in transparent writes to and/or reads from a memcache or a groupcache or something of that nature in the future if your application grows such a need.
One thing that I really like about defining my handlers as methods on an object is that it makes it much easier to unit test them: you can easily define a table driven test that includes the values you want and the output you expect without having to muck around with global variables.
Don't indulge in premature optimization. Define a Go package API to encapsulate the data and then you can change the implementation at any time. For example, just scribbling,
package data
type Key struct {
// . . .
}
type Data struct {
// . . .
}
var dataMap map[Key]Data
func init() {
dataMap = make(map[Key]Data)
}
func GetData(key Key) (*Data, error) {
data := dataMap[key]
return &data, nil
}