I want to run some slow routine in another goroutine, is it safe to do it like this:
func someHandler(w http.ResponseWriter, r *http.Request) {
go someReallySlowFunction() // sending mail or something slow
fmt.Fprintf(w,"Mail will be delivered shortly..")
}
func otherHandler(w http.ResponseWriter, r *http.Request) {
foo := int64(0)
bar := func() {
// do slow things with foo
}
go bar()
fmt.Fprintf(w,"Mail will be delivered shortly..")
}
Is there any gotchas by doing this?
Serving each http request runs in its own goroutine (more details on this). You are allowed to start new goroutines from your handler, and they will run concurrently, independently from the goroutine executing the handler.
Some things to look out for:
The new goroutine runs independently from the handler goroutine. This means it may complete before or after the handler goroutine, you cannot (should not) assume anything regarding to this without explicit synchronization.
The http.ResponseWriter and http.Request arguments of the handler are only valid and safe to use until the handler returns! These values (or "parts" of them) may be reused - this is an implementation detail of which you should also not assume anything. Once the handler returns, you should not touch (not even read) these values.
Once the handler returns, the response is committed (or may be committed at any moment). Which means your new goroutine should not attempt to send back any data using the http.ResponseWriter after this. This is true to the extent that even if you don't touch the http.ResponseWriter in your handler, not panicing from the handler is taken as a successful handling of the request and thus HTTP 200 status is sent back (see an example of this).
You are allowed to pass the http.Request and http.ResponseWriter values to other functions and to new goroutines, but care must be taken: you should use explicit synchronization (e.g. locks, channels) if you intend to read / modify these values from multiple goroutines (or you want to send back data from multiple goroutines).
Note that seemingly if both your handler goroutine and your new goroutine just reads / inspects the http.Request, that still may be problematic. Yes, multiple goroutines can read the same variable without synchronization (if nobody modifies it). But calling certain methods of http.Request also modify the http.Request, and without synchronization there is no guarantee what other goroutines would see from this change. For example Request.FormValue() returns a form value associated with the given key. But this method calls ParseMultiPartForm() and ParseForm() if necessary which modify the http.Request (e.g. they set the Request.PostForm and Request.Form struct fields).
So unless you synchronize your goroutines, you should not pass Request and ResponseWriter to the new goroutine, but acquire data needed from the Request in the handler goroutine, and pass only e.g. a struct holding the needed data.
Your second example:
foo := int64(0)
bar := func() {
// do slow things with foo
}
go bar()
This is perfectly fine. This is a closure, and local variables referred by it will survive as long as they are accessible.
Note that alternatively you could pass the value of the local variable to the anonymous function call as an argument like this:
foo := int64(0)
bar := func(foo int64) {
// do slow things with param foo (not the local foo var)
}
go bar(foo)
In this example the anonymous function will see and use its parameter foo and not the local variable foo. This may or may not be what you want (depending on whether the handler also uses the foo and whether changes made by any of the goroutines need to be visible to the other - but that would require synchronization anyway, which would be superseded by a channel solution).
If you care for acknowledgement for the mail, then the posted code won't help. Running the code in separate goroutine makes it independent and the server reply will be success even if the mail is not sent due to some error in the goroutine function.
Related
In Node.js, to finish writing to a stream (and in theory with HTTP, tell the client there is no more data), we use response.end(). Using Go, the ResponseWriter interface is like:
type ResponseWriter interface {
Header() Header
Write([]byte) (int, error)
WriteHeader(statusCode int)
}
so my question is twofold:
How can we get the HTTP status code from the ResponseWriter?
more importantly: How does Go (and routers like Mux) know when the programmer is done writing to the ResponseWriter? Is it when the goroutine ends? What if you wanted to finish the response before the goroutine stack is empty? Seems like an implementation flaw to not have an End() method in the ResponseWriter interface.
This is not possible with the standard http.ResponseWriter implementation. But this is an interface, so it's easy to write your own implementation that records the status. The beginning of a simple implementation might be:
type statusRecorder struct {
http.ResponseWriter
status int
}
func (r *statusRecorder) WriteHeader(status int) {
r.status = status
r.ResponseWriter.WriteHeader(status)
}
While this may seem like a limitation of the API, it's actually the opposite. By using an interface, it is possible to create an implementation that does anything, or records any information you want, rather than being limited to whatever functionality the standard library authors may have decided to expose.
When the handler returns, it is done. If you wish to do additional work after sending a response, you can spawn a goroutine to continue operating after the main handler returns.
For the past few days, I've been reading about Go and one concept that I keep returning to are contexts.
I think I understand the motivation behind creating such a structure. The thing that I don't understand is a particular use case when using a context in the incoming HTTP request.
Let's say we have a following httpHandlerFunc. Inside that handler, we call a function that requires a context to be passed. I often saw this solution
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(context.Background(), "request", r)
otherFunc(ctx)
}
My question is, why don't we just pass a context from the request, like so
func myHandler(w http.ResponseWriter, r *http.Request) {
otherFunc(r.Context())
}
Doesn't it make more sense to pass the context of the request since we want the context to flow through our program? I thought that creating a background context is something we want to do only in the root parent, like init() function.
You might be missing the main point of contexts — supposedly due to poor HOWTOs you're dealing with.
The possiblility of carrying around arbitrary values in contexts is actually a misfeature of this type, regretted by its designers because it creates an anti-pattern (a proper way to deal with context-as-some-state is to have a set of values explicitly passed around).
The chief reason contexts exist is because they provide tree-like propagation of a signal (cancellation or "done" in the case of contexts).
So the original idea behind contexts is like follows:
The "root" context object is created for an incoming request.
Each "task" which is needed to be executed on behalf of the request is associated with its own context, derived from that of the request¹.
Those tasks may produce other tasks and so on.
As you can see, a hierarchy of "units of works" is formed, — linked to the object which is the reason for these units to exist and execute.
When the incoming request is cancelled (the client's socket got disconnected, for example), the context object associated with it is cancelled as well, and then all the linked tasks receive it as it's propagated from the root of the resulting context tree down to its leaves — making sure all the tasks being executed for the request are (eventually) cancelled.
Of course, in order for this to work, each "task" — which is usually a goroutine doing something — is required to "listen" from the context passed to it for that "done" signal.
Contexts also support timeout out of the box, so you might create a context which cancels itself after some fixed time interval passes.
So, back to the examples in your question.
The first example ignores the request's context completely and creates a from-scratch context ostensibly with the sole reason to carry stuff in it (bad).
The second example might use the context for its intended purpose (but we do not know as we cannot see that otherFunc).
I would advise you to read https://blog.golang.org/context, and the articles on concurrency patters in Go linked there.
¹ Actually, a new context need not be created if the task to be controlled by it has no other policy to "add" to the existing, parent, context.
The idea of derivation is here to implment additional ways to cancel work in this particular task as well as honoring the cancellation of the parent context.
For instance, a context derived for a particular task could have its own deadline or have a way to cancel only this particular context.
Of course, a complex—nested—context can be derived for a task: for example, a context with a deadline can be derived from the parent context, and then a cancellable context can be derived form the former. The result would be a context which is cancelled either explicitly by the code or when the deadline expires or when the parent context signals its cancellation.
Your two examples do entirely different things.
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(context.Background(), "request", r)
otherFunc(ctx)
}
This creates a new context, and stores the request as a value. There is rarely, if ever, any reason to do exactly this. A far more idiomatic solution would be just to pass the request to otherFunc like so:
func myHandler(w http.ResponseWriter, r *http.Request) {
otherFunc(r)
}
If you really do need to pass the request as a context value, you should probably do it with the current request's context, like so:
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(r.Context(), "request", r)
otherFunc(ctx)
}
I have handlers that respond to https requests. In the handlers I call a function F1() which does some application logic and connects to a mysql db and does a query. I want to know how I can use the golang context package to cancel the Db query if the client cancels the request. Do I need to pass the ctx to F1()? Also the code I have now will take 4 seconds even if F1() returns in less then 4. How can I return as soon as F1() returns?
func handler(w http.ResponseWriter, r *http.Request) {
ctx:= r.context()
F1()
select {
case <-ctx.Done():
case <- time.After( 4*time.Second):
}
w.WriteHeader(http.statusOk)
return
}
To begin, I highly recommend taking a look at the Context blog post to familiarize yourself with contexts, in addition to reading over the context documentation itself.
To address your specific questions:
How can you cancel the database query if the user cancels their quest?
To make this work, there are a few things you want to check:
Ensure that your database driver (if you are using database/sql) supports context cancellation.
Ensure you are using the Context variants of all available methods (e.g. db.QueryContext instead of db.Query).
Ensure that you are passing the context (or a derivative of the context) through the stack from your HTTP request through to the database calls.
Do I need to pass the ctx to F1()?
Per #3 above, yes: you will need to pass the context through all intermediate calls to "connect" the database call with the request context.
How can I return as soon as F1() returns?
The code that you have in your question calls F1 in series, rather than concurrently, with your cancellation/timeout select.
If you want to apply a specific deadline to your database call, use context.WithTimeout to limit how long it can take. Otherwise, you do not need to do anything special: just call F1 with your context, and the rest will happen for you, no select is needed.
I was wondering if there is already a library to do that or maybe a suggestion which way to go for the following problem:
Client A makes request for resource A, this is a long running request since resource A is expensive and it results in a cache miss. In the meantime client B makes request for resource A, now it's still a cache miss since client A's request hasn't returned and populated the cache yet. so instead of making a new request to generate resource A, client B should block and be notified when client A's request is complete and has populated the cache.
I think the group cache library has something along those lines, but I haven't been able to browse through the code to figure out how they do it, I also don't wanna tie the implementation to it and use it as a dependency.
The only solution I had so far is a pub-sub type of thing, where we have a global map of the current in-flight requests with the reqID as a key. When req1 comes it sets its ID in the map, req2 comes and checks if its id is in the map, since its requesting the same resource it is, so we block on a notifier channel. When req1 finishes it does 3 things:
evicts its ID from the map
saves the entry in the cache
sends a broadcast with its ID to the notifier channel
req2 receives the notification, unblocks and fetches from the cache.
Since go doesn't have built in support for broadcasts, theres probably 1 grouting listening on the broadcast channel and then keeping a list of subscribers to broadcast to for each request, or maybe we change the map to reqId => list(broadcastChannelSubscribers). Something along those lines.
If you think there is a better way to do it with Go's primitives, any input would be appreciated. The only piece of this solution that bothers me is this global map, surrounded by locks, I assume it quickly is going to become a bottleneck. IF you have some non-locking ideas, even if they are probabilistic Im happy to hear them.
It reminds me of one question where someone was implementing a similar thing:
Coalescing items in channel
I gave an answer with an example of implementing such a middle layer. I think this is in line with your ideas: have a routine keeping track of requests for the same resource and prevent them from being recalculating in parallel.
If you have a separate routine responsible for taking requests and managing access to cache, you don't need an explicit lock (there is one buried in a channel though). Anyhow, I don't know specifics of your application, but considering you need to check cache (probably locked) and (occasionally) perform an expensive calculation of missing entry – lock on map lookups doesn't seem like a massive problem to me. You can also always span more such middle layer routines if you think this would help, but you would need a deterministic way of routing the requests (so each cache entry is managed by a single routine).
Sorry for not bringing you a silver bullet solution, but it sounds like you're on a good way of solving your problem anyway.
Caching and perfomance problems are always tricky and you should always make a basic solution to benchmark against to ensure that your assumptions are correct. But if we know that the bottleneck is fetching the resource and that caching will give significant returns you could use Go's channels to implement queuing. Assuming that response is the type of your resource.
type request struct {
back chan *response
}
func main() {
c := make(chan request,10) // non-blocking
go func(input chan request){
var cached *response
for _,i := range input {
if cached == nil { // only make request once
cached = makeLongRunningRequest()
}
i.back <- cached
}
}(c)
resp := make(chan *response)
c <- request{resp} // cache miss
c <- request{resp} // will get queued
c <- request{resp} // will get queued
for _,r := range resp {
// do something with response
}
}
Here we're only fetching one resource but you could start one goroutine for each resource you want to fetch. Goroutines are cheap so unless you need millions of resources cached at the same time you should be ok. You could of course also kill your goroutines after a while.
To keep track of which resource id belongs to which channel, I'd use a map
map[resourceId]chan request
with a mutex. Again, if fetching the resource is the bottle neck then the cost of locking the map should be negligible. If locking the map turns out to be a problem, consider using a sharded map.
In general you seem to be well on your way. I'd advise to try to keep your design as simple as possible and use channels instead of locks when possible. They do protect from terrible concurrency bugs.
One solution is a concurrent non-blocking cache as discussed in detail in The Go Programming Language, chapter 9.
The code samples are well worth a look because the authors take you through several versions (memo1, memo2, etc), illustrating problems of race conditions, using mutexes to protect maps, and a version using just channels.
Also consider https://blog.golang.org/context as it has similar concepts and deals with cancellation of in flight requests.
It's impractical to copy the content into this answer, so hopefully the links are of use.
This is already provided by Golang as a feature single flight.
For your use case just use some extra logic on top of single flight. Consider the code snippet below:
func main() {
http.HandleFunc("/github", func(w http.ResponseWriter, r *http.Request) {
var key = "facebook"
var requestGroup singleflight.Group
// Search The Cache, if found in cache return from cache, else make single flight request
if res, err := searchCache(); err != nil{
return res
}
// Cache Miss-> Make Single Flight Request, and Cache it
v, err, shared := requestGroup.Do(key, func() (interface{}, error) {
// companyStatus() returns string, error, which statifies interface{}, error, so we can return the result directly.
if err != nil {
return interface{}, err
}
return companyStatus(), nil
})
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
//Set the Cache Here
setCache(key, v)
status := v.(string)
log.Printf("/Company handler requst: status %q, shared result %t", status, shared)
fmt.Fprintf(w, "Company Status: %q", status)
})
http.ListenAndServe("127.0.0.1:8080", nil)
}
// companyStatus retrieves Comapny's API status
func getCompanyStatus() (string, error) {
log.Println("Making request to Some API")
defer log.Println("Request to Some API Complete")
time.Sleep(1 * time.Second)
resp, err := http.Get("Get URL")
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return "", fmt.Errorf("Upstream response: %s", resp.Status)
}
r := struct{ Status string }{}
err = json.NewDecoder(resp.Body).Decode(&r)
return r.Status, err
}
I hope the code snippet is self explanatory and you can refer to Single Flight Official Docs to delve deep into single flight.
Im creating a Networking API and want people to be able to route requests to specific endpoints using a ServeMux. Instead of using a Server instance, I need to use my own low level ServerConn. This is because I am receiving both incoming HTTP requests and plain text data from the same port.
The problem, however, is that if I want to forward a request using my ServeMux, I would use it's ServeHTTP method. For this, I need to provide a ResponseWriter, which I don't know how to create an instance of since it is an interface, not a struct. Should a I create my own ResponseWriter struct? Is there one given by the Golang Standard Library? Or is there an alternate solution to this altogether?
I would avoid doing this altogether if at all possible. Mixing protocols on the same connection is bound to lead to hard-to-trace bugs, and unexpected behavior. If you really want to do it, and have all the http/1.1 mechanisms work correctly, leave as much as possible to the http package.
Since ResponseWriter is an interface, you would implement your own type to satisfy it. Look at the unexported response type in the http package for a full example. There's a lot to get right, and using it in combination with a ServerConn (which is documented as "do no use") is probably not a good idea.
The place to do this at a lower level would be in Accept inside the Server's net.Listener. Since you're going to have to parse the start of every request twice, you would need a net.Conn that can be "rewound" partly.
Make yourself a net.Listener that checks the start of the stream on a new connection, and if it looks like an http request, return a net.Conn that replays the first chunk you read off the wire on its first Reads. Something like:
type replayConn struct {
net.Conn
buf []byte
pos int
}
func (c *replayConn) Read(b []byte) (int, error) {
if c.pos < len(c.buf) {
n := copy(b, c.buf[c.pos:])
c.pos += n
return n, nil
}
return c.Conn.Read(b)
}
If the connection isn't http, then send the connection off to your other type of handler, and continue blocking on Accept.