Golang: returning pointers and derefrencing - pointers

So I have a function getToken()
func getToken() jwt.MapClaims {
tokenString := "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6ImFkYW0iLCJwYXNzd29yZCI6InRlc3QiLCJpYXQiOjE0ODcyMDY2OTIsImV4cCI6MTUxODc2NDI5Mn0.6LQo_gRwXiFBvNIJOwtf9UuxoQMZZ3XNILTnU-46-Zg"
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"])
}
hmacSampleSecret := []byte("supersecretkittysecret")
return hmacSampleSecret, nil
})
if err != nil {
println("error")
}
if claims, ok := token.Claims.(jwt.MapClaims); ok && token.Valid {
return claims
} else {
return nil
}
}
Then the corresponding call:
res := getToken()
println(res["username"])
Why is res["username"] equal to two memory addresses (0x2b3c20,0xc420075420)? This should just be a string like adam. I have also tried func getToken() *jwt.MapClaims and return &claims, but this still did not help.

You should try using fmt.Println instead of println. Here's an example of printing a map using println vs fmt.Println
package main
import (
"fmt"
)
func foo() map[string]string {
return map[string]string{
"k": "value",
}
}
func main() {
res := foo()
println("Output from println:", res) // prints pointer address
fmt.Println("Output from fmt.Println: ", res) // prints the map
}
https://play.golang.org/p/gCNqng3KEE
Output:
Output from println: 0x10432200
Output from fmt.Println: map[k:value]

Related

Error handling on a call stack - http request handler

In the below code:
func (p *ProductHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// handle the request for a list of products
if r.Method == http.MethodGet {
p.getProductHandler(w, r)
return
}
if r.Method == http.MethodPost {
p.addProductHandler(w, r)
return
}
if r.Method == http.MethodPut {
id := findID(w, r)
p.updateProductHandler(id, w, r)
return
}
// catch all
// if no method is satisfied return an error
w.WriteHeader(http.StatusMethodNotAllowed)
}
// getProducts returns the products from the data store
func (p *ProductHandler) getProductHandler(w http.ResponseWriter, r *http.Request) {
p.l.Println("Handle GET Products")
// fetch the products from the datastore
productList := data.GetProducts()
// serialize the list to JSON
err := productList.WriteJSON(w)
if err != nil {
http.Error(w, "Unable to marshal json", http.StatusInternalServerError)
return
}
}
func (p *ProductHandler) addProductHandler(w http.ResponseWriter, r *http.Request) {
p.l.Println("Handle POST products")
// Read the item from the incoming request
productItem := &data.Product{}
err := productItem.ReadJSON(r.Body)
if err != nil {
http.Error(w, "Unable to unmarshal JSON", http.StatusBadRequest)
return
}
p.l.Printf("Product item: %#v\n", productItem)
data.AddProductItem(productItem)
}
func (p *ProductHandler) updateProductHandler(id int, w http.ResponseWriter, r *http.Request) {
// whatever
return
}
func findID(w http.ResponseWriter, r *http.Request) int {
// expect the id in the URI
dfa := regexp.MustCompile(`/([0-9]+)`)
matches := dfa.FindAllStringSubmatch(r.URL.Path, -1) // returns [][]string
if len(matches) != 1 {
http.Error(w, "Invalid URI", http.StatusBadRequest)
return
}
if len(matches[0]) != 2 {
http.Error(w, "Invlaid URI", http.StatusBadRequest)
return
}
idString := matches[0][1]
id, err := strconv.Atoi(idString)
if err != nil {
http.Error(w, "Invlaid URI", http.StatusBadRequest)
return
}
return id
}
On error, we use http.Error() within handlers that handle POST & GET and then return
But for PUT request, call stack stack is ServeHTTP -> findID() and then ServeHTTP() -> updateProductHandler(). Error handling is required in findID() but cannot return immediately, because findID() returns int.
What is the design pattern to perform error handling for a call stack? Error wrapping using github.com/pkg/errors....
func (p *ProductHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// handle the request for a list of products
if r.Method == http.MethodGet {
p.getProductHandler(w, r)
return
}
if r.Method == http.MethodPost {
p.addProductHandler(w, r)
return
}
if r.Method == http.MethodPut {
id, err := findID(r.URL.Path)
if err == nil {
p.updateProductHandler(id, w, r)
} else {
http.Error(w, err.Error(), http.StatusBadRequest)
}
return
}
// catch all
// if no method is satisfied return an error
w.WriteHeader(http.StatusMethodNotAllowed)
w.Header().Add("Allow", "GET, POST, PUT")
}
func findID(path string) (int, error) {
// expect the id in the URI
dfa := regexp.MustCompile(`/([0-9]+)`)
matches := dfa.FindAllStringSubmatch(path, -1) // returns [][]string
if len(matches) != 1 {
return 0, fmt.Errorf("Invalid URI %s", path)
}
if len(matches[0]) != 2 {
return 0, fmt.Errorf("Invalid URI %s", path)
}
idString := matches[0][1]
id, err := strconv.Atoi(idString)
if err != nil {
return 0, fmt.Errorf("Invalid URI %s", path)
}
return id, nil
}

How to return Document Index Name Value

This is a GoLang, Firebase AdminSDK question.
This example works to iterate through all of the documents in a FireStore DB.
How can I get the Document Name?
To put another way: If the collection name is JohnyCollection, and JohnyCollection has 20 Documents called (Document1, Document2.... Document20), how do I get the document name in golang Code?
//========================================
package main
import (
"context"
"fmt"
"log"
"firebase.google.com/go"
"google.golang.org/api/iterator"
"google.golang.org/api/option"
)
func check(e error) {
if e != nil {
panic(e)
}
}
func main() {
ctx := context.Background()
sa := option.WithCredentialsFile("./scai-qit-fb-adminsdk.json")
app, err := firebase.NewApp(ctx, nil, sa)
if err != nil {
log.Fatalf("error initializing app: %v\n", err)
}
client, err := app.Firestore(ctx)
if err != nil {
log.Fatal(err)
}
defer client.Close()
iter := client.Collection("COMPLEX_NONACS").Documents(ctx)
for {
doc, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
log.Fatalf("Failed to iterate: %v", err)
}
//This part works. WIll return a Map of each Document
fmt.Println("--------------------------/n")
fmt.Println(doc.Data())
// This is the question. How do I get the INDEX name of the Document?
// something like...
fmt.Println(doc.Index_value_or_something_that_returns_IndexName())
// for example...
// {
// "ABC":{"line1":"yabba dabba","line2":"dingo dong"},
// "DEF":{"line1":"hooty tooty","line2":"blah blah"}
// }
// How to just get the "ABC" and "DEF"
}
}
You can get the document ID from the DocumentSnapshot, by first looking up the DocumentRef:
fmt.Println(doc.Ref.ID)
See the reference docs for DocumentSnapshot and DocumentRef.

How to reference multiple libs with same functions and switch between them inline in Go

I am wondering how I can do something similar to this. I currently have multiple packages with that same structure and functions but they actually retrieve values from multiple APIs. I am also loading in a config that has an array with parameters to use one of those packages per array item.
I am wondering how I can create a variable that uses one of those packages based on the config value. Hopefully this is clear enough. Here is pseudo code that I have written up to explain. Thanks in advance
package main
import (
"errors"
"flag"
"os"
"project/lib"
"project/morelib"
"project/extralib"
"fmt"
"math"
"math/rand"
"time"
)
func stuff(info RunInfo) (err error) {
apiKey:= "stuff1" // actually in the config
apiSecret := "stuff2" // actually in the config
variable := lib.New(apiKey, apiSecret) //returns *Lib struct
//this is where I have to comment out the other libs and uncomment them as needed
// variable := morelib.New(apiKey, apiSecret)
// variable := extralib.New(apiKey, apiSecret)
//trying to switch between libraries like this or in a switch statement
if info.libname == "lib"{
variable = lib.New(apiKey, apiSecret) //.New("", "") returns *Lib struct
}else if info.libname == "morelib"{
variable = morelib.New(apiKey, apiSecret) //.New("", "") returns *MoreLib struct
}else if info.libname == "extralib"{
variable = extralib.New(apiKey, apiSecret) //.New("", "") returns *ExtraLib struct
}else{
err = errors.New("there was an error with the value.....")
return err
}
mystuffs, err := variable.GetBalance("USD")
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("mystuff value: %v", mystuffs.value)
return
}
type RunInfo struct{
libname string
//other stuff
}
func main() {
//load from config with array
config := config.Load()
for i:=0; i<compare; i++{
var runInfo RunInfo
runInfo.libname = config.libname
stuff(runInfo)
}
}
pseudo lib code:
func New(apiKey, apiSecret string) *Lib {
client := NewClient(apiKey, apiSecret)
return &Lib{client}
}
func NewClient(apiKey, apiSecret string) (c *client) {
return &client{apiKey, apiSecret, &http.Client{}, false}
}
type Lib struct {
client *client
}
type client struct {
apiKey string
apiSecret string
httpClient *http.Client
debug bool
}
func (b *Lib) GetBalance(currency string) (balance Balance, err error) {
payload, err := json.Marshal(BalanceParams{Currency: currency})
if err != nil {
return
}
r, err := b.client.do("POST", "GetBalance", string(payload), true)
if err != nil {
return
}
var response jsonResponse
if err = json.Unmarshal(r, &response); err != nil {
return
}
if err = handleErr(response); err != nil {
return
}
err = json.Unmarshal(response.Result, &balance)
return
}
Use and if statement as in the question, a switch statement or a map.
I assume that the type returned by the New function is the following interface:
type GetStuffer interface
GetStuff(string) (Stuff, error)
}
The switch statement is:
var variable GetStuffer
switch info.CompareVal {
case "lib":
variable = lib.New(string1, string2)
case "morelib":
variable = morelib.New(string1, string2)
case "extralib":
variable = extralib.New(string1, string2)
default:
return errors.New("there was an error with the value.....")
}
mystuffs, err := variable.GetMyStuff()
if err != nil {
fmt.Println(err)
return
}
For the map, initialize a package level variable with the map:
var m = map[string]func(string,string) GetStuffer {
"lib": lib.New,
"morelib": morelib.New,
"extralib": extralib.New,
}
and use it like this:
fn := m[info.CompareValue]
if m == nil {
return errors.New("there was an error with the value.....")
}
variable := fn(string1, string2)
mystuffs, err := variable.GetMyStuff()
if err != nil {
fmt.Println(err)
return
}
If the assumption above about the return type is not correct, then there are two options. The first and likely the simplest is to modify New functions to return type GetStuffer. If that's not possible, then write little adaptor functions:
var m = map[string]func(string,string) GetStuffer {
"lib":func(s1, s2 string) GetStuffer { return lib.New(s1, s2) }
"morelib":func(s1, s2 string) GetStuffer { return morelib.New(s1, s2) }
"extralib":func(s1, s2 string) GetStuffer { return extralib.New(s1, s2) }
}
Why don't you define an interface that's only one function? In your example would be
type Stuffer interface {
GetMyStuff(string) (Stuff, error)
}
Then you declare your variable as type Stuffer.

implement tls.Config.GetCertificate with self signed certificates

I m trying to figure out how i can implement a function to feed to tls.Config.GetCertificate with self signed certificates.
I used this bin source as a base, https://golang.org/src/crypto/tls/generate_cert.go
Also read this,
https://ericchiang.github.io/tls/go/https/2015/06/21/go-tls.html
Unfortunately, so far i m stuck with this error
2016/11/03 23:18:20 http2: server: error reading preface from client 127.0.0.1:34346: remote error: tls: unknown certificate authority
I think i need to generate a CA cert and then sign the key with it, but i m not sure how to proceed (....).
Here is my code, can someone help with that ?
package gssc
import (
"crypto/rand"
"crypto/rsa"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"github.com/pkg/errors"
"math/big"
"net"
"strings"
"time"
)
func GetCertificate(arg interface{}) func(clientHello *tls.ClientHelloInfo) (*tls.Certificate, error) {
var opts Certopts
var err error
if host, ok := arg.(string); ok {
opts = Certopts{
RsaBits: 2048,
Host: host,
ValidFrom: time.Now(),
}
} else if o, ok := arg.(Certopts); ok {
opts = o
} else {
err = errors.New("Invalid arg type, must be string(hostname) or Certopt{...}")
}
return func(clientHello *tls.ClientHelloInfo) (*tls.Certificate, error) {
if err != nil {
return nil, err
}
return generate(opts)
}
}
type Certopts struct {
RsaBits int
Host string
IsCA bool
ValidFrom time.Time
ValidFor time.Duration
}
func generate(opts Certopts) (*tls.Certificate, error) {
priv, err := rsa.GenerateKey(rand.Reader, opts.RsaBits)
if err != nil {
return nil, errors.Wrap(err, "failed to generate private key")
}
notAfter := opts.ValidFrom.Add(opts.ValidFor)
serialNumberLimit := new(big.Int).Lsh(big.NewInt(1), 128)
serialNumber, err := rand.Int(rand.Reader, serialNumberLimit)
if err != nil {
return nil, errors.Wrap(err, "Failed to generate serial number\n")
}
template := x509.Certificate{
SerialNumber: serialNumber,
Subject: pkix.Name{
Organization: []string{"Acme Co"},
},
NotBefore: opts.ValidFrom,
NotAfter: notAfter,
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
BasicConstraintsValid: true,
}
hosts := strings.Split(opts.Host, ",")
for _, h := range hosts {
if ip := net.ParseIP(h); ip != nil {
template.IPAddresses = append(template.IPAddresses, ip)
} else {
template.DNSNames = append(template.DNSNames, h)
}
}
if opts.IsCA {
template.IsCA = true
template.KeyUsage |= x509.KeyUsageCertSign
}
derBytes, err := x509.CreateCertificate(rand.Reader, &template, &template, &priv.PublicKey, priv)
if err != nil {
return nil, errors.Wrap(err, "Failed to create certificate")
}
return &tls.Certificate{
Certificate: [][]byte{derBytes},
PrivateKey: priv,
}, nil
}
This is the test code i use
package main
import (
"crypto/tls"
"github.com/mh-cbon/gssc"
"net/http"
)
type ww struct{}
func (s *ww) ServeHTTP(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "text/plain")
w.Write([]byte("This is an example server.\n"))
}
func main() {
s := &http.Server{
Handler: &ww{},
Addr: ":8080",
TLSConfig: &tls.Config{
InsecureSkipVerify: true,
GetCertificate: gssc.GetCertificate("example.org"),
},
}
s.ListenAndServeTLS("", "")
}
Thanks a lot!
Your implementation of tls.Config.GetCertificate is causing the problem.
You are generating a certificate each time tls.Config.GetCertificate is called. You need to generate the certificate once and then return it in the anonymous function.
In gssc.GetCertificate :
cert, err := generate(opts)
return func(clientHello *tls.ClientHelloInfo) (*tls.Certificate, error) {
if err != nil {
return nil, err
}
return cert, err
}

Getting "127.0.0.1 can't assign requested address" - http.Client

What I'm doing is fairly straight-forward. I need to create a "proxy" server that is very minimal and fast. Currently I have a baseline server that is proxied to (nodejs) and a proxy-service (go). Please excuse the lack of actual "proxy'ing" - just testing for now.
Baseline Service
var http = require('http');
http.createServer(function (req, res) {
// console.log("received request");
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(8080, '127.0.0.1');
console.log('Server running at http://127.0.0.1:8080/');
Proxy Service
package main
import (
"flag"
"log"
"net/http"
"net/url"
)
var (
listen = flag.String("listen", "0.0.0.0:9000", "listen on address")
logp = flag.Bool("log", false, "enable logging")
)
func main() {
flag.Parse()
proxyHandler := http.HandlerFunc(proxyHandlerFunc)
log.Fatal(http.ListenAndServe(*listen, proxyHandler))
log.Println("Started router-server on 0.0.0.0:9000")
}
func proxyHandlerFunc(w http.ResponseWriter, r *http.Request) {
// Log if requested
if *logp {
log.Println(r.URL)
}
/*
* Tweak the request as appropriate:
* - RequestURI may not be sent to client
* - Set new URL
*/
r.RequestURI = ""
u, err := url.Parse("http://localhost:8080/")
if err != nil {
log.Fatal(err)
}
r.URL = u
// And proxy
// resp, err := client.Do(r)
c := make(chan *http.Response)
go doRequest(c)
resp := <-c
if resp != nil {
err := resp.Write(w)
if err != nil {
log.Println("Error writing response")
} else {
resp.Body.Close()
}
}
}
func doRequest(c chan *http.Response) {
// new client for every request.
client := &http.Client{}
resp, err := client.Get("http://127.0.0.1:8080/test")
if err != nil {
log.Println(err)
c <- nil
} else {
c <- resp
}
}
My issue, as mentioned within the title, is that I am getting errors stating 2013/10/28 21:22:30 Get http://127.0.0.1:8080/test: dial tcp 127.0.0.1:8080: can't assign requested address from the doRequest function, and I have no clue why. Googling this particular error yields seemingly irrelevant results.
There are 2 major problems with this code.
You are not handling the client stalling or using keep alives (handled below by getTimeoutServer)
You are not handling the server (what your http.Client is talking to) timing out (handled below by TimeoutConn).
This is probably why you are exhausting your local ports. I know from past experience node.js will keep-alive you very aggressively.
There are lots of little issues, creating objects every-time when you don't need to. Creating unneeded goroutines (each incoming request is in its own goroutine before you handle it).
Here is a quick stab (that I don't have time to test well). Hopefully it will put you on the right track: (You will want to upgrade this to not buffer the responses locally)
package main
import (
"bytes"
"errors"
"flag"
"fmt"
"log"
"net"
"net/http"
"net/url"
"runtime"
"strconv"
"time"
)
const DEFAULT_IDLE_TIMEOUT = 5 * time.Second
var (
listen string
logOn bool
localhost, _ = url.Parse("http://localhost:8080/")
client = &http.Client{
Transport: &http.Transport{
Proxy: NoProxyAllowed,
Dial: func(network, addr string) (net.Conn, error) {
return NewTimeoutConnDial(network, addr, DEFAULT_IDLE_TIMEOUT)
},
},
}
)
func main() {
runtime.GOMAXPROCS(runtime.NumCPU())
flag.StringVar(&listen, "listen", "0.0.0.0:9000", "listen on address")
flag.BoolVar(&logOn, "log", true, "enable logging")
flag.Parse()
server := getTimeoutServer(listen, http.HandlerFunc(proxyHandlerFunc))
log.Printf("Starting router-server on %s\n", listen)
log.Fatal(server.ListenAndServe())
}
func proxyHandlerFunc(w http.ResponseWriter, req *http.Request) {
if logOn {
log.Printf("%+v\n", req)
}
// Setup request URL
origURL := req.URL
req.URL = new(url.URL)
*req.URL = *localhost
req.URL.Path, req.URL.RawQuery, req.URL.Fragment = origURL.Path, origURL.RawQuery, origURL.Fragment
req.RequestURI, req.Host = "", req.URL.Host
// Perform request
resp, err := client.Do(req)
if err != nil {
w.WriteHeader(http.StatusBadGateway)
w.Write([]byte(fmt.Sprintf("%d - StatusBadGateway: %s", http.StatusBadGateway, err)))
return
}
defer resp.Body.Close()
var respBuffer *bytes.Buffer
if resp.ContentLength != -1 {
respBuffer = bytes.NewBuffer(make([]byte, 0, resp.ContentLength))
} else {
respBuffer = new(bytes.Buffer)
}
if _, err = respBuffer.ReadFrom(resp.Body); err != nil {
w.WriteHeader(http.StatusBadGateway)
w.Write([]byte(fmt.Sprintf("%d - StatusBadGateway: %s", http.StatusBadGateway, err)))
return
}
// Write result of request
headers := w.Header()
var key string
var val []string
for key, val = range resp.Header {
headers[key] = val
}
headers.Set("Content-Length", strconv.Itoa(respBuffer.Len()))
w.WriteHeader(resp.StatusCode)
w.Write(respBuffer.Bytes())
}
func getTimeoutServer(addr string, handler http.Handler) *http.Server {
//keeps people who are slow or are sending keep-alives from eating all our sockets
const (
HTTP_READ_TO = DEFAULT_IDLE_TIMEOUT
HTTP_WRITE_TO = DEFAULT_IDLE_TIMEOUT
)
return &http.Server{
Addr: addr,
Handler: handler,
ReadTimeout: HTTP_READ_TO,
WriteTimeout: HTTP_WRITE_TO,
}
}
func NoProxyAllowed(request *http.Request) (*url.URL, error) {
return nil, nil
}
//TimeoutConn-------------------------
//Put me in my own TimeoutConn.go ?
type TimeoutConn struct {
net.Conn
readTimeout, writeTimeout time.Duration
}
var invalidOperationError = errors.New("TimeoutConn does not support or allow .SetDeadline operations")
func NewTimeoutConn(conn net.Conn, ioTimeout time.Duration) (*TimeoutConn, error) {
return NewTimeoutConnReadWriteTO(conn, ioTimeout, ioTimeout)
}
func NewTimeoutConnReadWriteTO(conn net.Conn, readTimeout, writeTimeout time.Duration) (*TimeoutConn, error) {
this := &TimeoutConn{
Conn: conn,
readTimeout: readTimeout,
writeTimeout: writeTimeout,
}
now := time.Now()
err := this.Conn.SetReadDeadline(now.Add(this.readTimeout))
if err != nil {
return nil, err
}
err = this.Conn.SetWriteDeadline(now.Add(this.writeTimeout))
if err != nil {
return nil, err
}
return this, nil
}
func NewTimeoutConnDial(network, addr string, ioTimeout time.Duration) (net.Conn, error) {
conn, err := net.DialTimeout(network, addr, ioTimeout)
if err != nil {
return nil, err
}
if conn, err = NewTimeoutConn(conn, ioTimeout); err != nil {
return nil, err
}
return conn, nil
}
func (this *TimeoutConn) Read(data []byte) (int, error) {
this.Conn.SetReadDeadline(time.Now().Add(this.readTimeout))
return this.Conn.Read(data)
}
func (this *TimeoutConn) Write(data []byte) (int, error) {
this.Conn.SetWriteDeadline(time.Now().Add(this.writeTimeout))
return this.Conn.Write(data)
}
func (this *TimeoutConn) SetDeadline(time time.Time) error {
return invalidOperationError
}
func (this *TimeoutConn) SetReadDeadline(time time.Time) error {
return invalidOperationError
}
func (this *TimeoutConn) SetWriteDeadline(time time.Time) error {
return invalidOperationError
}
We ran into this and after a lot of time trying to debug, I came across this: https://code.google.com/p/go/source/detail?r=d4e1ec84876c
This shifts the burden onto clients to read their whole response
bodies if they want the advantage of reusing TCP connections.
So be sure you read the entire body before closing, there are a couple of ways to do it. This function can come in handy to close to let you see whether you have this issue by logging the extra bytes that haven't been read and cleaning the stream out for you so it can reuse the connection:
func closeResponse(response *http.Response) error {
// ensure we read the entire body
bs, err2 := ioutil.ReadAll(response.Body)
if err2 != nil {
log.Println("Error during ReadAll!!", err2)
}
if len(bs) > 0 {
log.Println("Had to read some bytes, not good!", bs, string(bs))
}
return response.Body.Close()
}
Or if you really don't care about the body, you can just discard it with this:
io.Copy(ioutil.Discard, response.Body)
I have encountered this problem too, and i add an option {DisableKeepAlives: true} to http.Transport fixed this issue, you can have a try.
I came here when running a massive amount of SQL queries per second on a system without limiting the number of idle connections over a long period of time. As pointed out in this issue comment on github explicitly setting db.SetMaxIdleConns(5) completely solved my problem.

Resources