Let's suppose client uploads heavy image by http with slow internet connection, he opens connection to server, starts to write data and suddenly he loses this connection because of network problems. How can I detect this on server if handler function was not yet called. The solution I found is to check the connection state. But the problem is that it's not scalable, because a lot of goroutines will interact with global variable. Are there any more elegant solutions?
package main
import (
"fmt"
"log"
"net"
"net/http"
)
// current state of connections
var connStates = make(map[string]http.ConnState)
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Println("got new request")
fmt.Println(connStates)
})
srv := &http.Server{
Addr: ":8080",
ConnState: func(conn net.Conn, event http.ConnState) {
if event != http.StateClosed {
connStates[conn.RemoteAddr().String()] = event
} else {
if connStates[conn.RemoteAddr().String()] == http.StateActive {
fmt.Println("client cancelled request")
}
delete(connStates, conn.RemoteAddr().String())
}
},
}
log.Fatal(srv.ListenAndServe())
}
You could use context within your handler, for example, this would detect when the client disconnects and return and http.StatusPartialContent besides calling someCleanup() in where you could have your logging logic.
https://play.golang.org/p/5Yr_HBuyiZW
func helloWorld(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
ch := make(chan struct{})
go func() {
time.Sleep(5 * time.Second)
fmt.Fprintln(w, "Hello World!")
ch <- struct{}{}
}()
select {
case <-ch:
case <-ctx.Done():
http.Error(w, ctx.Err().Error(), http.StatusPartialContent)
someCleanUP()
}
}
If you only need to have logs you can even simplify the code:
srv := &http.Server{
Addr: ":8080",
ConnState: func(conn net.Conn, event http.ConnState) {
log.Printf("addr: %s, changed state to: %s", conn.RemoteAddr(), event.String())
},
}
That callback will be triggered on each change of the conn
Related
I'm aiming for curl client sends http req to Go API and Go API does (1. run a background shell, 2. returns response to client instantly but 3. keeps on running the point 1 server-side command in the background). Problem is point 2 is not being returned to client instantly, client is only getting a response after point 3 finishes
I tried:
import (
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"os/exec"
"log"
"strings"
"flag"
"strconv"
"crypto/tls"
"crypto/x509"
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
"github.com/go-ldap/ldap"
"regexp"
"errors"
"encoding/base64"
"time"
)
func insert(w http.ResponseWriter, r *http.Request) (error) {
fullcmd := fmt.Sprintf("/home/ec2-user/spark_home/bin/spark-submit %s", "dgs")
cmd := exec.Command("/bin/sh", "-c", fullcmd)
err4 := cmd.Start()
if err4 != nil {
e1 := fmt.Sprintf("Error")
l.Printf(e1)
http.Error(w, e1, http.StatusInternalServerError)
return err4
} else {
l.Printf("The data is being ingested asynchronously in the background \n")
fmt.Fprintf(w, "request received. The data is being ingested asynchronously in the background \n")
w.Header().Set("Content-Type", "text/plain")
w.Write([]byte(fmt.Sprintf("request received. The data is being ingested asynchronously in the background \n")))
}
//wait for the spark command to finish, need to Wait() otherwise zombie/orphan pid is created
cmd.Wait()
//do bunch of other commands here that take 30+ seconds
l.Printf("success")
return nil
}
r := mux.NewRouter()
r.HandleFunc("/test", insert).Methods(http.MethodPost)
http.Handle("/", r)
server := &http.Server{
Addr: ":" + strconv.Itoa(*port),
Handler: handlers.LoggingHandler(os.Stdout, http.DefaultServeMux),
TLSConfig: tlsConfig,
}
server.ListenAndServeTLS(TLS_SERVER_CERTFILE, TLS_SERVER_KEYFILE)
The response will be completed when the HTTP handler returns, so if you want to start a job that will continue, you have to do that in a separate goroutine. You can start the goroutine as soon as the shell process starts, using something like this:
func insert(w http.ResponseWriter, r *http.Request) (error) {
...
err4 := cmd.Start()
if err4 != nil {
...
}
w.Header().Set("Content-Type", "text/plain")
fmt.Fprintf(w, "request received. The data is being ingested asynchronously in the background \n")
go func() {
cmd.Wait()
// Do other stuff
}()
return nil
}
Is there any way to print something when the http server starts? For instance "Server is started at port 8080"
In Node (using Express), it would be like:
app.listen(8080, function() { console.log('Server started at port 8080') });
This is my code:
func main() {
http.HandleFunc("/", MyHandler)
http.ListenAndServe(":8080", nil)
}
Thanks.
Use Go's log package:
package main
import (
"net/http"
"log"
)
func main() {
addr := ":8080"
http.HandleFunc("/", MyHandler)
log.Println("listen on", addr)
log.Fatal( http.ListenAndServe(addr, nil) )
}
http.ListenAndServe opens the server port, and blocks forever waiting for clients. If it fails to open the port, the log.Fatal call will report the problem and exit the program.
You can't print a log message after ListenAndServe since it blocks and never returns, so basically you have two main options:
Print "Starting server on port...." and that's it - BUT if ListenAndServe could not start it returns an error, so unless there's some error or panic printed because of that, you can assume the server started.
Call ListenAndServe in a separate goroutine, and make sure there was no error returned and print "Server started..." etc.
I personally prefer the first approach.
To run ListenAndServe in a goroutine as mentioned by Not_a_Golfer, you can use an unbuffered, blocking channel to run it in a goroutine and also keep the server alive.
The following example creates a channel called done where <-done will keep the server alive as it waits for the goroutine to finish, which it won't in this case. Typically, the goroutine will tell the main function it is finished by executing done <- true.
package main
import (
"log"
"net/http"
)
func MyHandler(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello World"))
}
func main() {
port := "8080"
http.HandleFunc("/", MyHandler)
done := make(chan bool)
go http.ListenAndServe(":"+port, nil)
log.Printf("Server started at port %v", port)
<-done
}
Here's a larger example that has the server verify it is operational, using Listen and Serve separately. The nice thing about doing it this way is you can capture an incorrect port easily.
package main
import (
"log"
"net"
"net/http"
"os"
)
func MyHandler(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello World"))
}
func main() {
port := "8080"
http.HandleFunc("/", MyHandler)
listener, err := net.Listen("tcp", ":"+port)
if err != nil {
log.Fatal(err)
}
done := make(chan bool)
go http.Serve(listener, nil)
// Log server started
log.Printf("Server started at port %v", port)
// Attempt to connect
log.Printf("Fetching...")
res, err := http.Get("http://" + listener.Addr().String())
log.Printf("Received: %v, %v", res, err)
if err != nil {
log.Fatal(err)
}
res.Write(os.Stdout)
<-done
}
In Go, how can I start the browser AFTER the server started listening?
Preferably the simplest way possible.
My code so far, super dumbed down to the point:
package main
import (
// Standard library packages
"fmt"
"net/http"
"github.com/skratchdot/open-golang/open"
// Third party packages
"github.com/julienschmidt/httprouter"
)
// go get github.com/toqueteos/webbrowser
func main() {
// Instantiate a new router
r := httprouter.New()
// Add a handler on /test
r.GET("/test", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
// Simply write some test data for now
fmt.Fprint(w, "Welcome!\n")
})
//open.Run("https://google.com/")
// open.Start("https://google.com")
// http://127.0.0.1:3000/test
// Fire up the server
http.ListenAndServe("localhost:3000", r)
fmt.Println("ListenAndServe is blocking")
open.RunWith("http://localhost:3000/test", "firefox")
fmt.Println("Done")
}
Open the listener, start the browser and then enter the server loop:
l, err := net.Listen("tcp", "localhost:3000")
if err != nil {
log.Fatal(err)
}
// The browser can connect now because the listening socket is open.
err := open.Start("http://localhost:3000/test")
if err != nil {
log.Println(err)
}
// Start the blocking server loop.
log.Fatal(http.Serve(l, r))
There's no need to poll as shown in another answer. The browser will connect if the listening socket is open before the browser is started.
ListenAndServe is a convenience function that opens a socket and calls Serve. The code in this answer splits out these steps so the browser can be opened after listening starts but before the blocking call to Serve.
If there is no error, http.ListenAndServe() will never return. So you shouldn't add code after that except code that handles failure.
You have to start a new goroutine, so ListenAndServe() is called in one goroutine, and code checking if it is up should run on the other goroutine.
And you can check if your server is up by making a simple HTTP GET call to it, for example using http.Get().
The following example delays startup for 7 seconds on purpose. The new goroutine starts an endless for loop that checks if server is up, sleeping 1 second between attempts.
Example:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hi!"))
})
go func() {
for {
time.Sleep(time.Second)
log.Println("Checking if started...")
resp, err := http.Get("http://localhost:8081")
if err != nil {
log.Println("Failed:", err)
continue
}
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
log.Println("Not OK:", resp.StatusCode)
continue
}
// Reached this point: server is up and running!
break
}
log.Println("SERVER UP AND RUNNING!")
}()
log.Println("Starting server...")
time.Sleep(time.Second * 7)
log.Fatal(http.ListenAndServe(":8081", nil))
Example output:
2015/09/23 13:53:03 Starting server...
2015/09/23 13:53:04 Checking if started...
2015/09/23 13:53:06 Failed: Get http://localhost:8081: dial tcp [::1]:8081: connectex: No connection could be made because the target machine actively refused it.
2015/09/23 13:53:07 Checking if started...
2015/09/23 13:53:09 Failed: Get http://localhost:8081: dial tcp [::1]:8081: connectex: No connection could be made because the target machine actively refused it.
2015/09/23 13:53:10 Checking if started...
2015/09/23 13:53:10 SERVER UP AND RUNNING!
The API is not absolutely terrible, but let's just say "It takes some getting used to". Here is how you use custom attributes on the Server struct:
s := &http.Server{
Addr: cnf.API_SERVER_ADDRESS,
Handler: h,
ReadTimeout: 0, // 1 * time.Minute,
WriteTimeout: 30 * time.Minute,
MaxHeaderBytes: 1 << 20,
}
go func() {
l, err := net.Listen("tcp", cnf.API_SERVER_ADDRESS)
if err != nil {
log.Fatal(err)
}
fmt.Println(`{"server_state":"listening"}`)
log.Fatal(s.Serve(l));
}()
because if you instead use:
http.Serve(l, handler)
then you can't define custom properties on the server
Why can't I run both HTTP and HTTPS from the same golang program?
Here is the code where the two servers are initiated.. The server which is initiated first will run - the second won't.. If they are switched arround the other will run and the other won't..
No errors are returned when running the program, but the requests http://www.localhost or https://secure.localhost times out
// Start HTTP
err_http := http.ListenAndServe(fmt.Sprintf(":%d", port), http_r)
if err_http != nil {
log.Fatal("Web server (HTTP): ", err_http)
}
// Start HTTPS
err_https := http.ListenAndServeTLS(fmt.Sprintf(":%d", ssl_port), "D:/Go/src/www/ssl/public.crt", "D:/Go/src/www/ssl/private.key", https_r)
if err_https != nil {
log.Fatal("Web server (HTTPS): ", err_https)
}
Here is the complete code
package main
import (
"net/http"
"fmt"
"log"
"os"
"io"
"runtime"
// go get github.com/gorilla/mux
"github.com/gorilla/mux"
)
const (
HOST = "localhost"
)
func Handler_404(w http.ResponseWriter, r *http.Request){
fmt.Fprint(w, "Oops, something went wrong!")
}
func Handler_www(w http.ResponseWriter, r *http.Request){
fmt.Fprint(w, "Hello world :)")
}
func Handler_api(w http.ResponseWriter, r *http.Request){
fmt.Fprint(w, "This is the API")
}
func Handler_secure(w http.ResponseWriter, r *http.Request){
fmt.Fprint(w, "This is Secure")
}
func redirect(r *mux.Router, from string, to string){
r.Host(from).Subrouter().HandleFunc("/", func (w http.ResponseWriter, r *http.Request){
http.Redirect(w, r, to, 301)
})
}
func main(){
port := 9000
ssl_port := 443
runtime.GOMAXPROCS(runtime.NumCPU())
http_r := mux.NewRouter()
https_r := mux.NewRouter()
// HTTP 404
http_r.NotFoundHandler = http.HandlerFunc(Handler_404)
// Redirect "http://HOST" => "http://www.HOST"
redirect(http_r, HOST, fmt.Sprintf("http://www.%s:%d", HOST, port))
// Redirect "http://secure.HOST" => "https://secure.HOST"
redirect(http_r, "secure."+HOST, fmt.Sprintf("https://secure.%s", HOST))
www := http_r.Host("www."+HOST).Subrouter()
www.HandleFunc("/", Handler_www)
api := http_r.Host("api."+HOST).Subrouter()
api.HandleFunc("/", Handler_api)
secure := https_r.Host("secure."+HOST).Subrouter()
secure.HandleFunc("/", Handler_secure)
// Start HTTP
err_http := http.ListenAndServe(fmt.Sprintf(":%d", port), http_r)
if err_http != nil {
log.Fatal("Web server (HTTP): ", err_http)
}
// Start HTTPS
err_https := http.ListenAndServeTLS(fmt.Sprintf(":%d", ssl_port), "D:/Go/src/www/ssl/public.crt", "D:/Go/src/www/ssl/private.key", https_r)
if err_https != nil {
log.Fatal("Web server (HTTPS): ", err_https)
}
}
ListenAndServe and ListenAndServeTLS open the listening socket and then loop forever serving client connections. These functions only return on an error.
The main goroutine never gets to the starting the TLS server because the main goroutine is busy waiting for HTTP connections in ListenAndServe.
To fix the problem, start the HTTP server in a new goroutine:
// Start HTTP
go func() {
err_http := http.ListenAndServe(fmt.Sprintf(":%d", port), http_r)
if err_http != nil {
log.Fatal("Web server (HTTP): ", err_http)
}
}()
// Start HTTPS
err_https := http.ListenAndServeTLS(fmt.Sprintf(":%d", ssl_port), "D:/Go/src/www/ssl/public.crt", "D:/Go/src/www/ssl/private.key", https_r)
if err_https != nil {
log.Fatal("Web server (HTTPS): ", err_https)
}
As previously said, both ListenAndServe and ListenAndServeTLS are blocking. That being said, I would agree that examples above are in fact resolving your issue as the point is to be in goroutine BUT same examples are not quite following go idioms.
You should be using error channels here as you want to capture ALL errors that are sent to you instead of having just one error returned back. Here's fully working sample that starts HTTP as HTTPS servers and return errors as channel that's later on used just to display errors.
package main
import (
"log"
"net/http"
)
func Run(addr string, sslAddr string, ssl map[string]string) chan error {
errs := make(chan error)
// Starting HTTP server
go func() {
log.Printf("Staring HTTP service on %s ...", addr)
if err := http.ListenAndServe(addr, nil); err != nil {
errs <- err
}
}()
// Starting HTTPS server
go func() {
log.Printf("Staring HTTPS service on %s ...", addr)
if err := http.ListenAndServeTLS(sslAddr, ssl["cert"], ssl["key"], nil); err != nil {
errs <- err
}
}()
return errs
}
func sampleHandler(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "text/plain")
w.Write([]byte("This is an example server.\n"))
}
func main() {
http.HandleFunc("/", sampleHandler)
errs := Run(":8080", ":10443", map[string]string{
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem",
})
// This will run forever until channel receives error
select {
case err := <-errs:
log.Printf("Could not start serving service due to (error: %s)", err)
}
}
Hope this helps! :)
func serveHTTP(mux *http.ServeMux, errs chan<- error) {
errs <- http.ListenAndServe(":80", mux)
}
func serveHTTPS(mux *http.ServeMux, errs chan<- error) {
errs <- http.ListenAndServeTLS(":443", "fullchain.pem", "privkey.pem", mux)
}
func main() {
mux := http.NewServeMux()
// setup routes for mux // define your endpoints
errs := make(chan error, 1) // a channel for errors
go serveHTTP(mux, errs) // start the http server in a thread
go serveHTTPS(mux, errs) // start the https server in a thread
log.Fatal(<-errs) // block until one of the servers writes an error
}
The ListenAndServe (and ListenAndServeTLS) functions do not return to their caller (unless an error is encountered). You can test this by trying to print something in between the two calls.
I want to stop the http server remotely (which I can do), but I also want to receive a message indicating that it has been stopped after it has been stopped. That is causing me some problems. The only solution that I could find is as shown below, which I do not consider ideal.
Can anyone provide a better solution. The problem is that the final message sent to the client is not getting through unless I use the goroutine as shown at the end "go func() {".
Code is as follows:
//*************
func stopServer(ohtWriter http.ResponseWriter, phtRequest *http.Request) {// Stop The Server
//*************
var iBytesSent int
var oOsError os.Error
var sErmes string
println("Stopping Server")
iBytesSent,oOsError = ohtWriter.Write([]byte("Message from server - server now stopped."))
if oOsError != nil {
sErmes = ". Error = " +oOsError.String()
} else {
sErmes = ". No error on write"
}
println("stopServer: Bytes sent = " +strconv.Itoa(iBytesSent) +sErmes)
ohtFlusher, tCanFlush := ohtWriter.(http.Flusher)
if tCanFlush {
ohtFlusher.Flush()
}
go func() {
time.Sleep(3e9)
os.Exit(0)
}()
}
Yeah I think without support from the http package, a graceful shutdown is not really possible. This is maybe a little bit less cringe-worthy, but will still slam closed any other concurrent requests in flight at the time of this request. Maybe try filing a feature request on the Go issue tracker. Better yet, open up the http package, and add a graceful shutdown method, and submit it.
Edit: I guess if you control all of the http.Handlers in your app, you could keep a count of in-flight requests (using appropriate thread synchronization), and modify the code below to a) refuse new connections once "shutdown" is called and b) wait for all in-flight requests to complete before shutting down...
package main
import (
"http"
"os"
"io"
"log"
"strconv"
)
func main() {
http.HandleFunc("/", ServeHTTP)
http.ListenAndServe(":8081", nil)
}
const responseString = "Shutting down\n"
func ServeHTTP(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("Content-Length", strconv.Itoa(len(responseString)))
io.WriteString(w, responseString)
f, canFlush := w.(http.Flusher)
if canFlush {
f.Flush()
}
conn, _, err := w.(http.Hijacker).Hijack()
if err != nil {
log.Fatalf("error while shutting down: %v", err)
}
conn.Close()
log.Println("Shutting down")
os.Exit(0)
}
Haven't tried it yet, but using http.ServerConn directly might work.
Here's a simple way that's good enough for local development.
http://www.sergiotapia.me/how-to-stop-your-go-http-server/
package main
import (
"net/http"
"os"
"github.com/bmizerany/pat"
)
var mux = pat.New()
func main() {
mux.Get("/kill", http.HandlerFunc(kill))
http.Handle("/", mux)
http.ListenAndServe(":8080", nil)
}
func kill(w http.ResponseWriter, r *http.Request) {
os.Exit(0)
}