I am trying my hand on my first go program which is supposed to be a very simple IRC bot.
I have the part of the connections etc done but I am confused with structs and pointers such. The structs are new to me coming from languages that use classes.
I have this struct and the constructor for it:
type Bot struct {
server string
port string
nick string
channel string
pass string
conn net.Conn
}
// NewBot main config
func NewBot() *Bot {
return &Bot{
server: "irc.twitch.tv",
port: "6667",
nick: "username",
channel: "#channel",
pass: "password123",
conn: nil,
}
}
My connect() method looks like this
func (bot *Bot) Connect() (conn net.Conn, err error) {
ircbot := NewBot()
conn, err = net.Dial("tcp", bot.server+":"+bot.port)
// irc connection...
return bot.conn, nil
}
Everything of that works fine the problem I am having is with another method to my struct named Message. It's just supposed to send a message. Looks like this:
// Message to send a message
func (bot *Bot) Message(message string) {
if message == "" {
return
}
fmt.Printf("Bot: " + message + "\n")
fmt.Fprintf(bot.conn, "PRIVMSG "+bot.channel+" :"+message+"\r\n")
}
everytime when I then try to use this function I get this error and the program crashes
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x20 pc=0x463d73]
I'm unsure about how to use the & and * signs correctly to achieve what I want to do.
I thought a goroutine is something that is for multithreading and is done by saying "go [do something]" but I don't use that anywhere.
Edit:
Solution
Thanks I found the solution! For those interested:
I created a new instance of Bot in the place where I called the Message() function which resulted in an empty conn.
This was the important bit, I stupidly didn't post here. handle() wasn't even a method of Bot which is even more stupid of me.
func handle(line string) {
ircbot := NewBot();
// get username, message etc...
ircbot .CmdInterpreter(username[1], usermessage)
}
and this is the correct way:
func (bot *Bot) handle(line string) {
// get username, message etc...
bot.CmdInterpreter(username[1], usermessage)
}
The issue here seems to be with the Connect method, and not really with pointers in particular:
func (bot *Bot) Connect() (conn net.Conn, err error) {
ircbot := NewBot()
conn, err = net.Dial("tcp", bot.server+":"+bot.port)
// irc connection...
return bot.conn, nil
}
The method is defined to be on (a pointer to the) Bot struct, but it creates a new Bot, on this line:
ircbot := NewBot()
and then proceeds to use the one the method is defined on (called bot, not ircbot). If you choose to keep it so that there are separate NewBot and Connect functions (which is fine), then you should change it so that Connect actually uses an instantiated *Bot:
func (bot *Bot) Connect() (conn net.Conn, err error) {
conn, err = net.Dial("tcp", bot.server+":"+bot.port)
// irc connection...
return bot.conn, nil
}
and call it with something like this:
bot := NewBot()
conn, err := bot.Connect()
The error,
panic: runtime error: invalid memory address or nil pointer dereference
is probably because you are accessing bot in the Connect method (e.g. in bot.server) but it has not been defined, and the pointer is nil.
Related
I am writing an HTTP server in Go, which uses the following pattern to handle API output:
func handler(w http.ResponsWriter, r *http.Request) {
defer reply(w, r, L)() //L is a Logger
//do things...
}
func reply(w http.ResponseWriter, r *http.Request, log Logger) func() {
cid := []byte{0, 0, 0, 0}
if log != nil {
rand.Read(cid)
log.Debug("[%x] %s %s", cid, r.Method, r.URL.String())
}
entry := time.Now()
return func() {
if log != nil {
defer log.Debug("[%x] elapsed %d millis", cid, time.Since(entry).Milliseconds())
}
_, err := w.Write(nil)
if err == http.ErrHijacked {
return //API is a WEBSOCKET entry point, do nothing
}
//handle common output logic for normal HTTP APIs...
}
}
The reason I do this, is that I found this comment in the standard library:
// ErrHijacked is returned by ResponseWriter.Write calls when
// the underlying connection has been hijacked using the
// Hijacker interface. A zero-byte write on a hijacked
// connection will return ErrHijacked without any other side
// effects.
ErrHijacked = errors.New("http: connection has been hijacked")
However following the Write() method, I got this comment:
// Write writes the data to the connection as part of an HTTP reply.
//
// If WriteHeader has not yet been called, Write calls
// WriteHeader(http.StatusOK) before writing the data. If the Header
// does not contain a Content-Type line, Write adds a Content-Type set
// to the result of passing the initial 512 bytes of written data to
// ...
Write([]byte) (int, error)
My questions are:
Is it OK to use my code to safely detect if a HTTP connection is hijacked? I only want to check the connection is hijacked or not, but do NOT want it to add headers for me!
Since the ResponseWriter is an interface, I cannot click through the source code to find out how the standard library implements that method. In general, how can I drill down to the standard library (or any open source code) to find out the implementation of an interface?
Thanks to Cerise, I found the source code of the standard response.Writer:
func (w *response) write(lenData int, dataB []byte, dataS string) (n int, err error) {
if w.conn.hijacked() {
if lenData > 0 {
caller := relevantCaller()
w.conn.server.logf("http: response.Write on hijacked connection from %s (%s:%d)", caller.Function, path.Base(caller.File), caller.Line)
}
return 0, ErrHijacked
}
... ....
So, as said in the document, there is NO side effect.
I have an HTTP server that when it recieves a request calls on an underlying gRPC server.
I have chosen to abstract away the gRPC call with an interface, to make testing of the http server easier.
The problem is that I am constantly getting the errors:
rpc error: code = Canceled desc = grpc: the client connection is closing
or
rpc error: code = Canceled desc = context canceled
And as I understand both of these are related to the context getting passed into the grpc call. And that I want the context to be alive throughout both the HTTP and gRPC calls.
type SetterGetter interface {
Getter(key string) (val string)
}
type Service struct {
sg SetterGetter
ctx context.Context
}
func (s *Service) getHandler(rw http.ResponseWriter, r *http.Request) {
key := r.URL.Query()["key"][0]
res := s.sg.Getter(key)
fmt.Fprintf(rw, "Successfully got value: %s\n", res)
}
func main() {
s := new(Service)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
s.sg = gc.NewClientwrapper(ctx)
http.HandleFunc("/get", s.getHandler)
log.Fatal(http.ListenAndServe(port, nil))
}
And my Getter implementation looks like this:
type clientwrapper struct {
sc pb.ServicesClient
ctx context.Context
}
func NewClientwrapper(ctx context.Context) *clientwrapper {
cw := new(clientwrapper)
conn, err := grpc.Dial(address, grpc.WithInsecure(), grpc.WithBlock())
if err != nil {
err = fmt.Errorf("Error could not dial address: %v", err)
}
defer conn.Close()
cw.ctx = ctx
cw.sc = pb.NewServicesClient(conn)
return cw
}
func (cw *clientwrapper) Getter(key string) (val string) {
// Make the GRPC request
res, err := cw.sc.Get(cw.ctx, &pb.GetRequest{Key: key})
if err != nil {
return ""
}
getVal := res.GetValue()
return getVal
}
So here I am creating a context in my http servers main menu, and passing it onwards. I do it like this because it worked if I removed my interface and put everything in the main file.
I have also tried to create the context both in the http handler and passing it to the Getter and I have also tried creating it in the Getter itself.
I think the correct approach is to create the context in the http request using the context that gets created by the request and then passing it to the grpc Getter. Like such:
func (s *Service) getHandler(rw http.ResponseWriter, r *http.Request) {
// Create it like such
ctx, cancel := context.WithTimeout(r.Context(), 100*time.Second)
key := r.URL.Query()["key"][0]
// And pass it onwards (of course we need to change function signature for this to work)
res := s.sg.Getter(ctx, key)
fmt.Fprintf(rw, "Successfully got value: %s\n", res)
}
So how should I create my context here, to not get these errors?
If your goal is to keep a long-running task running in the background, that doesn't cancel when the request is finalized, then don't use the request's context. Use context.Background() instead.
For example:
func (s *Service) getHandler(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Second)
// ...
I'm a nodejs developer and I generally use a structure for my apps which holds a configuration package/object that holds references to my often used libs and configuration options. Generally this configuration object holds my database connections as well and it's reachable trough my application.
I tried to build similar to this in go and failed horribly.
My plan was to build a public variable which holds a reference for my config struct. But when I try to call my Config.Database I get this panic:
2017/02/19 14:05:44 http: panic serving 127.0.0.1:53554: runtime error: invalid memory address or nil pointer dereference
goroutine 50 [running]:
net/http.(*conn).serve.func1(0xc42027c000)
/usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0x9f45c0, 0xc42000c100)
/usr/local/go/src/runtime/panic.go:458 +0x243
main.SignUp(0xc4202780e0)
/home/attila/dev/gopath/src/github.com/attilasatan/helloiris/handlers.go:31 +0x258
github.com/kataras/iris.HandlerFunc.Serve(0xafaf00, 0xc4202780e0)
/home/attila/dev/gopath/src/github.com/kataras/iris/http.go:211 +0x30
github.com/kataras/iris.(*Context).Do(0xc4202780e0)
/home/attila/dev/gopath/src/github.com/kataras/iris/context.go:152 +0x4d
github.com/kataras/iris.(*serveMux).BuildHandler.func1(0xc4202780e0)
/home/attila/dev/gopath/src/github.com/kataras/iris/http.go:1059 +0x6ea
github.com/kataras/iris.(*Framework).Build.func1.1(0xd87e20, 0xc4202701a0, 0xc420284000)
/home/attila/dev/gopath/src/github.com/kataras/iris/iris.go:411 +0x72
net/http.HandlerFunc.ServeHTTP(0xc420235080, 0xd87e20, 0xc4202701a0, 0xc420284000)
/usr/local/go/src/net/http/server.go:1726 +0x44
net/http.serverHandler.ServeHTTP(0xc420089f80, 0xd87e20, 0xc4202701a0, 0xc420284000)
/usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc42027c000, 0xd88820, 0xc42015c200)
/usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2293 +0x44d
2017/02/19 14:05:44 http: panic serving 127.0.0.1:53560: runtime error: invalid memory address or nil pointer dereference
goroutine 51 [running]:
net/http.(*conn).serve.func1(0xc42027c180)
/usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0x9f45c0, 0xc42000c100)
/usr/local/go/src/runtime/panic.go:458 +0x243
main.SignUp(0xc4202ac070)
/home/attila/dev/gopath/src/github.com/attilasatan/helloiris/handlers.go:31 +0x258
github.com/kataras/iris.HandlerFunc.Serve(0xafaf00, 0xc4202ac070)
/home/attila/dev/gopath/src/github.com/kataras/iris/http.go:211 +0x30
github.com/kataras/iris.(*Context).Do(0xc4202ac070)
/home/attila/dev/gopath/src/github.com/kataras/iris/context.go:152 +0x4d
github.com/kataras/iris.(*serveMux).BuildHandler.func1(0xc4202ac070)
/home/attila/dev/gopath/src/github.com/kataras/iris/http.go:1059 +0x6ea
github.com/kataras/iris.(*Framework).Build.func1.1(0xd87e20, 0xc4202a60d0, 0xc4202840f0)
/home/attila/dev/gopath/src/github.com/kataras/iris/iris.go:411 +0x72
net/http.HandlerFunc.ServeHTTP(0xc420235080, 0xd87e20, 0xc4202a60d0, 0xc4202840f0)
/usr/local/go/src/net/http/server.go:1726 +0x44
net/http.serverHandler.ServeHTTP(0xc420089f80, 0xd87e20, 0xc4202a60d0, 0xc4202840f0)
/usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc42027c180, 0xd88820, 0xc42015c480)
/usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
Here is my configure file. As you can see I'm using tideland/golib/redis for redis connection.
configure.go
package main
import (
"fmt"
"strconv"
"time"
"github.com/tideland/golib/redis"
)
/*Configuration is the main type of app configuration */
type Configuration struct {
Database *redis.Connection
}
/*Config is app configuration holder */
var Config *Configuration
/*Configure handles database connection */
func Configure() (*Configuration, error) {
db, err := redis.Open(redis.TcpConnection("127.0.0.1:6379", 30*time.Second))
if err != nil {
fmt.Printf("Database connection error")
return nil, err
}
conn, err := db.Connection()
n, _ := conn.DoInt("INCR", "IDIDID")
fmt.Printf(strconv.Itoa(n))
if err != nil {
fmt.Printf("Database connection error")
return nil, err
}
/*Config is the main configuration object*/
Config := &Configuration{conn}
return Config, err
}
And here is where I use Config.Database.
handlers.go
func SignUp(ctx *iris.Context) {
mail := ctx.FormValue("email")
password := ctx.FormValue("password")
passwordConfirm := ctx.FormValue("password-confirm")
if password != passwordConfirm {
ctx.RenderWithStatus(iris.StatusBadRequest, "400.html", ErrorPageData{"passwords dont match"})
} else {
user := User{mail, password, 0}
db := Config.Database
userID, err := db.DoInt("INCR", "HELLOIRIS:ID")
if err != nil {
ctx.RenderWithStatus(iris.StatusBadRequest, "400.html", ErrorPageData{"passwords dont match"})
} else {
user.ID = userID
fmt.Printf("SAVED")
ctx.Render("signup-success.html", nil)
}
ctx.JSON(200, user)
}
}
After this fail I changed Configure function like this:
configure.go
func Configure() (Config *Configuration, err error) {
if Config != nil {
return
}
}
and I changed the usage at handler
handlers.go
config, err := Configure()
if err != nil {
ctx.RenderWithStatus(iris.StatusBadRequest, "400.html", ErrorPageData{"try again later"})
return
}
user := User{mail, password, 0}
db := config.Database
... and everything start to work perfectly.
My problem is I simply don't understand why... Why did I get this panic when I use a public pointer and why didn't I get when I returned the same pointer from a function?
Config := &Configuration{conn}
This line creates a new local variable Config so the global Config is never set to anything. Drop the : to not create a new variable and use the global Config instead.
You're getting this problem because your Config global variable is not set or still hold nil value. as the error said :
runtime error: invalid memory address or nil pointer dereference
Make sure you called
func Configure() before the handler called in your main function
and as tkausl answer remove the : when you assign the conn redis to global variable to be like this in your configure.go file:
Config = &Configuration{conn}
I'm tying to create an util function that will read body of the Request/Response and return it.
Here's what I've done for the moment:
func GetBody(in interface{}) []byte {
var body io.Reader
var statusCode int
switch v := in.(type) {
case *http.Request, *http.Response:
body = v.Body
statusCode = v.StatusCode
default:
log.Fatal("Only http.Request and http.Response parameters can be accepted to parse body")
}
if statusCode != 200 {
log.Fatalf("Received status code [%d] instead of [200]", statusCode)
}
body, err := ioutil.ReadAll(body)
if err != nil {
log.Fatal(err)
}
return body
}
But I'm receiving an error of compiler: v.Body undefined (type interface {} is interface with no methods)
Am I missing something or it's impossible to make a generic function that will server both for *http.Request and *http.Response
It is because of the double case.
v is still an interface{} because it could be either an *http.Request or an *http.Response
switch v := in.(type) {
case *http.Request
body = v.Body
statusCode = v.StatusCode
case *http.Response:
body = v.Body
statusCode = v.StatusCode
default:
log.Fatal("Only http.Request and http.Response parameters can be accepted to parse body")
}
This should work
In addition to the other answers which directly address the issue with your type switch, I'd like to point out an alternative solution.
Note, the interface{} solution is perfectly fine and may easily be considered preferable to this. This is presented for edification.
First, as a slight aside, if the thing you were interested in was a common method (e.g. Write or Cookies) rather than a common field (Body), it would be easy and better to access it via a custom interface.
Either by defining a type:
type cookier interface { // Should probably use a better name
Cookies() []*http.Cookie
}
func ShowCookies1(r cookier) {
log.Println("Got cookies:", r.Cookies())
}
Or by using an anonymous type in the function definition:
func ShowCookies2(r interface {
Cookies() []*http.Cookie
}) {
log.Println("Got cookies:", r.Cookies())
}
These functions can accept anything that has a Cookies method, this includes *http.Request and *http.Response.
Unfortunately, in your particular case you wish to access a common field rather than a common method so you can't directly just use a matching interface.
You could make a small wrapper type that adds a GetBody method (one could argue such a function should have been defined in the standard package).
type reqbody struct{ *http.Request }
type respbody struct{ *http.Response }
type getbody interface {
GetBody() io.ReadCloser
}
func (r reqbody) GetBody() io.ReadCloser { return r.Body }
func (r respbody) GetBody() io.ReadCloser { return r.Body }
func GetBody2(r getbody) ([]byte, error) {
body := r.GetBody()
defer body.Close()
return ioutil.ReadAll(body)
}
The caller knows what type they have and does one of:
buf, err = GetBody2(reqbody{req})
buf, err = GetBody2(respbody{resp})
In some sense this is uglier than just using interface{}.
But it has the benefit that instead of having a function that takes absolutely any type and panics/errors at run-time if a programmer mistakenly calls it with something not of the appropriate type,
this instead forces the caller to safely pass something you known is of the correct type at compile time.
Looking at this further, you're just reading everything from an io.ReadCloser and then closing it so it could further be simplified to the following
(which probably is better than your interface{} solution):
func GetReqBody(r *http.Request) io.ReadCloser { return r.Body }
// Could add checking r.StatusCode to the following one as well:
func GetRespBody(r *http.Response) io.ReadCloser { return r.Body }
func ReadAndClose(rc io.ReadCloser) ([]byte, error) {
defer rc.Close()
return ioutil.ReadAll(rc)
}
Again, the caller knows what type they have and does one of:
buf, err = ReadAndClose(GetReqBody(req))
buf, err = ReadAndClose(GetRespBody(resp))
Or just:
buf, err = ReadAndClose(req.Body)
buf, err = ReadAndClose(resp.Body)
You can see an example of all these options on the Go Playground.
Finally, be careful using ioutil.ReadAll.
Often it is better to avoid pre-reading an entire file or network steam into a buffer and instead process it as a stream as it is being read.
In particular, it's trivial to make an HTTP request with an arbitrary large body as a denial of service attack or to waste server resources
(http.MaxBytesReader can also help).
What you are trying to do is more something like that:
switch in.(type) {
case *http.Request:
body = v.(*http.Request).Body
case *http.Response:
body = v.(*http.Response).Body
default:
log.Fatal(...)
}
EDIT: I removed the wrong part of my answer, see HectorJ's answer for the more syntaxic please way of doing this.
i'm new to socket and trying to create a connection pooling over tcp socket. my implementation send 32bit length then binary message for each call. But i'm having problem with sometimes the reader receiving previous response from server (could happened when client close and re-establish socket on send error). how do i flush socket (remaining bytes from previous call) before a new request. any suggestion?
Edit: i learned that tcp always stream 0s, what if i send byte(1) before message so i can have a flush function to check if socket not empty before a new call.
Your post actually asks several questions:
How to manage a connection pool?
How to handle communication over the sockets?
These are really two different things. A connection pool is just a way to manage a set of connections. A simple way to implement this is with a class such as:
package netpool
import (
"net"
)
const MaxConnections = 3
type Error string
func (e Error) Error() string {
return string(e)
}
var ErrMaxConn = Error("Maximum connections reached")
type Netpool struct {
name string
conns int
free []net.Conn
}
func NewNetpool(name string) *Netpool {
return &Netpool{
name: name,
}
}
func (n *Netpool) Open() (conn net.Conn, err error) {
if n.conns >= MaxConnections && len(n.free) == 0 {
return nil, ErrMaxConn
}
if len(n.free) > 0 {
// return the first free connection in the pool
conn = n.free[0]
n.free = n.free[1:]
} else {
addr, err := net.ResolveTCPAddr("tcp", n.name)
if err != nil {
return nil, err
}
conn, err = net.DialTCP("tcp", nil, addr)
if err != nil {
return nil, err
}
n.conns += 1
}
return conn, err
}
func (n *Netpool) Close(conn net.Conn) error {
n.free = append(n.free, conn)
return nil
}
I have created a stand-alone class here. It would typically be implemented as part of a higher-level class such as MyHTTPHost, or MyDatabase.
In this simple implementation, connections that are returned via netpool.Open() are not tracked. It's possible to leak connections by calling Open(), then closing the connections outside of netpool.Close(). It's possible to track them if you want to hold an active and inactive pool, for example, which would solve this problem.
A couple of other things you might want to add to a pooling implementation:
Threading protection (using sync.Mutex, for example)
Closing of connections in the freepool after some length of inactivity
Error checking to be sure that closed connections are still valid
Once you have a connection, you can call Read and Write on it normally. To flush all oustanding data on the socket, you can simply use the ioutil.ReadAll() helper function. By default, this will block indefinitely if there is no data available. To avoid that, add a read timeout using:
conn.SetReadDeadline(time.Now().Add(500 * time.Millisecond))
_, err = ioutil.ReadAll(conn)
neterr, ok := err.(net.Error)
if ok && neterr.Timeout() {
err = nil // timeout isn't an error in this case
}
if err != nil {
// handle the error case.
}
This will read all the data from the given connection if any is pending, or will return after 500ms with an I/O Timeout error if no data was pending.
The type assertion is required because ioutil.ReadAll() returns an Error interface, rather than a net.Error interface, and we need the latter to be able to easily find out if the call returned due to a timeout.