As i learned from golang docs, if i set runtime.GOMAXPROCS(8) with a cpu of 8 cores (intel i7), then start a goroutine of infinite-loop, other gorutines should not be blocked because there are engough threads and goprocs. But this is not true when using net/http package, an infinite-loop goroutine will block http server after a few invocations.
Can anyone help to explain why ?
If i comment the line of "go infinite loop", start client after server, client will output 1000 asterisks; but if i enable the goroutine, client will block after print a few asterisks
I have tried add runtime.LockOSThread() in the goroutine, it seems that doesn't work
My Environment: osx 10.10, go version go1.3.1 darwin/amd64
Server code:
package main
import (
"fmt"
"log"
"net/http"
"runtime"
)
func myHandler(w http.ResponseWriter, req *http.Request) {
w.Write([]byte("hello"))
}
func infiniteloop() {
for {
}
}
func main() {
// set max procs for multi-thread executing
runtime.GOMAXPROCS(runtime.NumCPU())
// print GOMAXPROCS=8 on my computer
fmt.Println("GOMAXPROCS=", runtime.GOMAXPROCS(-1))
http.Handle("/", http.HandlerFunc(myHandler))
// uncomment below line cause server block after some requests
// go infiniteloop()
if err := http.ListenAndServe(":8280", nil); err != nil {
log.Fatal(err)
}
}
Client code:
package main
import (
"fmt"
"net/http"
)
func getOnce() {
if resp, err := http.Get("http://localhost:8280"); err != nil {
fmt.Println(err)
return
} else {
defer func() {
if err := resp.Body.Close(); err != nil {
fmt.Println(err)
}
}()
if resp.StatusCode != 200 {
fmt.Println("error codde:", resp.StatusCode)
return
} else {
fmt.Print("*")
}
}
}
func main() {
for i := 1; i < 1000; i++ {
getOnce()
if i%50 == 0 {
fmt.Println()
}
}
}
Now i know why such emtpy loop block other goroutines, but why runtime.LockOSThread() doesn't help either?
func infiniteloop() {
// add LockOSThread will not help
runtime.LockOSThread()
for {
}
}
As http://golang.org/pkg/runtime/#LockOSThread mentioned, the empty loop should be executed in an standalone thread, and other goroutines should not be impacted by the busy loop. What's wrong in my understanding?
The Go runtime's scheduler is not fully pre-emptive at this time. Go 1.2 improved matters by occasionally calling into the scheduler on function calls, but the infinite loops in your example have no function calls so this doesn't help.
With an actual body to your infinite loop handlers, you may see better behaviour. Alternatively, a manual call to runtime.Gosched may help in cases like this.
The scheduler might not be able to preempt such an empty "infinite" loop. The scheduler got better and better during the last release, maybe he should be good enough for such code; he definitely is good enough for real code. Just don't do such nonsense.
Related
I am finding a way to implement the permission check functionality using http
The idea is there are APIs that should be used only by login sessions.
func CheckPermissionFilter(w http.ResponseWriter, r *http.Response){
sid, err := r.Cookie("sid")
// check the permission with sid, if permission is granted then just let the
// process go on, otherwise, just break the filter chain and return Http Error Code.
}
func SomeHttpHandler(w http.ResponseWriter, r *http.Response){
CheckPermissionFilter(w, r)
// if not breaked by above filter function, process the request...
}
I have no problem with the permission checking, but I can't find a way to break the HTTP Request processing.
The call to CheckPermissionFilter within your SomeHttpHandler handler cannot terminate the latter early. Instead, you should define CheckPermissionFilter as a middleware (see also decorator pattern):
package main
import (
"net/http"
)
func main() {
http.Handle("/foo", CheckPermissionFilter(SomeHttpHandler))
// ...
}
func CheckPermissionFilter(h http.HandlerFunc) http.HandlerFunc {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
sid, err := r.Cookie("sid")
// handle err
if !Validate(sid) {
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
h(w, r)
})
}
func SomeHttpHandler(w http.ResponseWriter, r *http.Request) {
// ...
}
func Validate(sid string) bool {
return true // simplistic implementation for this example
}
I would like to know how to properly implement/use context.Done() method when using it within an HTTP server and implementing middleware, my goal is to cancel subsequent events when a client disconnects across nested middleware.
For testing I created the following code, I don't know if is the correct way of doing it since I had to create a channel within the HandleFunc and a goroutine to handle the requests, putting all this together within a select wait statement.
package main
import (
"fmt"
"log"
"net/http"
"time"
)
func hello(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
log.Println("handler started")
defer log.Println("hander ended")
ch := make(chan struct{})
go func() {
time.Sleep(5 * time.Second)
fmt.Fprintln(w, "Hello")
ch <- struct{}{}
}()
select {
case <-ch:
case <-ctx.Done():
err := ctx.Err()
log.Println(err)
http.Error(w, err.Error(), http.StatusPartialContent)
}
}
func main() {
http.HandleFunc("/", hello)
log.Fatal(http.ListenAndServe(":8080", nil))
}
Basically here the request simulates load by sleeping 5 seconds, and then prints Hello, but if the client cancels the request, for example:
$ curl 0:8080
And then pressing ctl + c, this will be loged:
2017/07/07 22:22:40 handler started
2017/07/07 22:22:42 context canceled
2017/07/07 22:22:42 hander ended
This works but wondering if this pattern (the goroutine and select) should be used in every nested handler or if there is a better way of implementing this.:
ch := make(chan struct{})
go func() {
// some logic
ch <- struct{}{}
}()
select {
case <-ch:
case <-ctx.Done():
err := ctx.Err()
log.Println(err)
http.Error(w, err.Error(), http.StatusPartialContent)
}
At Google, we require that Go programmers pass a Context parameter as the first argument to every function on the call path between incoming and outgoing requests.
-- Go Concurrency Patterns: Context
I want to create a simple script that checks if a certain hostname:port is running. I only want to get a bool response if that URL is live, but I'm not sure if there's a straightforward way of doing it.
If you only want see if a URL is reachable you could use net.DialTimeout. Like this:
timeout := 1 * time.Second
conn, err := net.DialTimeout("tcp","mysyte:myport", timeout)
if err != nil {
log.Println("Site unreachable, error: ", err)
}
If you want to check if a Web server answers on a certain URL, you can invoke an HTTP GET request using net/http.
You will get a timeout if the server doesn't response at all. You might also check the response status.
resp, err := http.Get("http://google.com/")
if err != nil {
print(err.Error())
} else {
print(string(resp.StatusCode) + resp.Status)
}
You can change the default timeout by initializing a http.Client.
timeout := time.Duration(1 * time.Second)
client := http.Client{
Timeout: timeout,
}
resp, err := client.Get("http://google.com")
Bonus:
Go generally does not rely on exceptions and the built in libraries generally do not panic, but return an error as a second value.
See Why does Go not have exceptions?.
You can assume that something very bad happened if your call to a native function panics.
You can make a HEAD request:
package main
import "net/http"
func head(s string) bool {
r, e := http.Head(s)
return e == nil && r.StatusCode == 200
}
func main() {
b := head("https://stackoverflow.com")
println(b)
}
https://golang.org/pkg/net/http#Head
If you don't mind the port, use http.Get(web):
package main
import (
"fmt"
"net/http"
"os"
)
func main() {
web := os.Args[1]
fmt.Println(webIsReachable(web))
}
func webIsReachable(web string) bool {
response, errors := http.Get(web)
if errors != nil {
_, netErrors := http.Get("https://www.google.com")
if netErrors != nil {
fmt.Fprintf(os.Stderr, "no internet\n")
os.Exit(1)
}
return false
}
if response.StatusCode == 200 {
return true
}
return false
}
In Go, how can I start the browser AFTER the server started listening?
Preferably the simplest way possible.
My code so far, super dumbed down to the point:
package main
import (
// Standard library packages
"fmt"
"net/http"
"github.com/skratchdot/open-golang/open"
// Third party packages
"github.com/julienschmidt/httprouter"
)
// go get github.com/toqueteos/webbrowser
func main() {
// Instantiate a new router
r := httprouter.New()
// Add a handler on /test
r.GET("/test", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
// Simply write some test data for now
fmt.Fprint(w, "Welcome!\n")
})
//open.Run("https://google.com/")
// open.Start("https://google.com")
// http://127.0.0.1:3000/test
// Fire up the server
http.ListenAndServe("localhost:3000", r)
fmt.Println("ListenAndServe is blocking")
open.RunWith("http://localhost:3000/test", "firefox")
fmt.Println("Done")
}
Open the listener, start the browser and then enter the server loop:
l, err := net.Listen("tcp", "localhost:3000")
if err != nil {
log.Fatal(err)
}
// The browser can connect now because the listening socket is open.
err := open.Start("http://localhost:3000/test")
if err != nil {
log.Println(err)
}
// Start the blocking server loop.
log.Fatal(http.Serve(l, r))
There's no need to poll as shown in another answer. The browser will connect if the listening socket is open before the browser is started.
ListenAndServe is a convenience function that opens a socket and calls Serve. The code in this answer splits out these steps so the browser can be opened after listening starts but before the blocking call to Serve.
If there is no error, http.ListenAndServe() will never return. So you shouldn't add code after that except code that handles failure.
You have to start a new goroutine, so ListenAndServe() is called in one goroutine, and code checking if it is up should run on the other goroutine.
And you can check if your server is up by making a simple HTTP GET call to it, for example using http.Get().
The following example delays startup for 7 seconds on purpose. The new goroutine starts an endless for loop that checks if server is up, sleeping 1 second between attempts.
Example:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hi!"))
})
go func() {
for {
time.Sleep(time.Second)
log.Println("Checking if started...")
resp, err := http.Get("http://localhost:8081")
if err != nil {
log.Println("Failed:", err)
continue
}
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
log.Println("Not OK:", resp.StatusCode)
continue
}
// Reached this point: server is up and running!
break
}
log.Println("SERVER UP AND RUNNING!")
}()
log.Println("Starting server...")
time.Sleep(time.Second * 7)
log.Fatal(http.ListenAndServe(":8081", nil))
Example output:
2015/09/23 13:53:03 Starting server...
2015/09/23 13:53:04 Checking if started...
2015/09/23 13:53:06 Failed: Get http://localhost:8081: dial tcp [::1]:8081: connectex: No connection could be made because the target machine actively refused it.
2015/09/23 13:53:07 Checking if started...
2015/09/23 13:53:09 Failed: Get http://localhost:8081: dial tcp [::1]:8081: connectex: No connection could be made because the target machine actively refused it.
2015/09/23 13:53:10 Checking if started...
2015/09/23 13:53:10 SERVER UP AND RUNNING!
The API is not absolutely terrible, but let's just say "It takes some getting used to". Here is how you use custom attributes on the Server struct:
s := &http.Server{
Addr: cnf.API_SERVER_ADDRESS,
Handler: h,
ReadTimeout: 0, // 1 * time.Minute,
WriteTimeout: 30 * time.Minute,
MaxHeaderBytes: 1 << 20,
}
go func() {
l, err := net.Listen("tcp", cnf.API_SERVER_ADDRESS)
if err != nil {
log.Fatal(err)
}
fmt.Println(`{"server_state":"listening"}`)
log.Fatal(s.Serve(l));
}()
because if you instead use:
http.Serve(l, handler)
then you can't define custom properties on the server
I want to stop the http server remotely (which I can do), but I also want to receive a message indicating that it has been stopped after it has been stopped. That is causing me some problems. The only solution that I could find is as shown below, which I do not consider ideal.
Can anyone provide a better solution. The problem is that the final message sent to the client is not getting through unless I use the goroutine as shown at the end "go func() {".
Code is as follows:
//*************
func stopServer(ohtWriter http.ResponseWriter, phtRequest *http.Request) {// Stop The Server
//*************
var iBytesSent int
var oOsError os.Error
var sErmes string
println("Stopping Server")
iBytesSent,oOsError = ohtWriter.Write([]byte("Message from server - server now stopped."))
if oOsError != nil {
sErmes = ". Error = " +oOsError.String()
} else {
sErmes = ". No error on write"
}
println("stopServer: Bytes sent = " +strconv.Itoa(iBytesSent) +sErmes)
ohtFlusher, tCanFlush := ohtWriter.(http.Flusher)
if tCanFlush {
ohtFlusher.Flush()
}
go func() {
time.Sleep(3e9)
os.Exit(0)
}()
}
Yeah I think without support from the http package, a graceful shutdown is not really possible. This is maybe a little bit less cringe-worthy, but will still slam closed any other concurrent requests in flight at the time of this request. Maybe try filing a feature request on the Go issue tracker. Better yet, open up the http package, and add a graceful shutdown method, and submit it.
Edit: I guess if you control all of the http.Handlers in your app, you could keep a count of in-flight requests (using appropriate thread synchronization), and modify the code below to a) refuse new connections once "shutdown" is called and b) wait for all in-flight requests to complete before shutting down...
package main
import (
"http"
"os"
"io"
"log"
"strconv"
)
func main() {
http.HandleFunc("/", ServeHTTP)
http.ListenAndServe(":8081", nil)
}
const responseString = "Shutting down\n"
func ServeHTTP(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("Content-Length", strconv.Itoa(len(responseString)))
io.WriteString(w, responseString)
f, canFlush := w.(http.Flusher)
if canFlush {
f.Flush()
}
conn, _, err := w.(http.Hijacker).Hijack()
if err != nil {
log.Fatalf("error while shutting down: %v", err)
}
conn.Close()
log.Println("Shutting down")
os.Exit(0)
}
Haven't tried it yet, but using http.ServerConn directly might work.
Here's a simple way that's good enough for local development.
http://www.sergiotapia.me/how-to-stop-your-go-http-server/
package main
import (
"net/http"
"os"
"github.com/bmizerany/pat"
)
var mux = pat.New()
func main() {
mux.Get("/kill", http.HandlerFunc(kill))
http.Handle("/", mux)
http.ListenAndServe(":8080", nil)
}
func kill(w http.ResponseWriter, r *http.Request) {
os.Exit(0)
}