I'm looking to pull some repetitive logic out of my handlers and put it into some per-handler middleware: specifically things like CSRF checks, checking for an existing session value (i.e. for auth, or for preview pages), etc.
I've read a few articles on this, but many examples focus on a per-server middleware (wrapping http.Handler): I have a smaller set of handlers that need the middleware. Most of my other pages do not, and therefore if I can avoid checking sessions/etc. for those requests the better.
My middleware, so far, typically looks something like this:
func checkCSRF(h http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// get the session, check/validate/create the token based on HTTP method, etc.
// return HTTP 403 on a failed check
// else invoke the wrapped handler h(w, r)
}
}
However, in many cases I want to pass a variable to the wrapped handler: a generated CSRF token to pass to the template, or a struct that contains form data—one piece of middleware checks the session for the presence of some saved form data before the user hits a /preview/ URL, else it redirects them away (since they have nothing to preview!).
I'd like to pass that struct along to the wrapped handler to save having to duplicate the session.Get/type assertion/error checking logic I just wrote in the middleware.
I could write something like:
type CSRFHandlerFunc func(w http.ResponseWriter, r *http.Request, t string)
... and then write the middleware like so:
func csrfCheck(h CSRFHandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// get the session, check/validate/create the/a token based on HTTP method, etc.
// return HTTP 403 on a failed check
// else invoke the wrapped handler and pass the token h(w, r, token)
}
... but that raises a few questions:
Is this a sensible way to implement per-handler middleware and pass per-request variables?
Prior to testing this (don't have access to my dev machine!), if I need to wrap a handler with multiple pieces of middleware, I assume I can just r.HandleFunc("/path/preview/", checkCSRF(checkExisting(previewHandler)))? The issue I'm seeing here is that the middleware is now tightly coupled: the wrapped middleware now needs to receive and then pass on the variable from the outer middleware. This makes extending http.HandlerFunc trickier/more convoluted.
Would gorilla/context fit better here and allow me to avoid writing 2-3 custom handler types (or a generic handler type) — and if so, how would I make use of it? Or could I implement my own "context" map (and run into issues with concurrent access?).
Where possible I'm trying to avoid falling for the "don't get caught writing a library" trap, but middleware is something that I'm likely to add/build on later in the project's life, and I'd like to "get it right" the first time around.
Some guidance on this would be much appreciated. Go's been great so far for writing a web application, but there's not a ton of examples around at this stage in its life and I'm therefore leaning on SO a little.
If I understood your question correctly, you're looking for a convenient way to pass additional parameters to your middleware, right?
Now, it's important to define what those parameters are. They could be some configuration values for your middleware – those can be set when the Handler type is being constructed). Instead of NewMyMiddleware(MyHandler), you do NewMyMiddleware(MyHandler, "parameter"), no problem here.
But in your case it seems like you want to pass per-request parameters, like a CSRF token. Passing those into the handler function would modify its signature and it would deviate from the standard Handler[Func] interface. You're right about middleware being more tightly coupled in this case.
You kind of mentioned the solution yourself – a context map is, in my opinion, a viable tool for this. It's not that hard to write one yourself – you basically need a map[*http.Request]interface{} and an RWMutex for safe concurrent access. Still, simply using gorilla/context should suffice – it seems like a (relatively) mature, well-written package with a nice API.
Shameless plug: if you're dealing with CSRF checks, why not try out my nosurf package?
Related
For the past few days, I've been reading about Go and one concept that I keep returning to are contexts.
I think I understand the motivation behind creating such a structure. The thing that I don't understand is a particular use case when using a context in the incoming HTTP request.
Let's say we have a following httpHandlerFunc. Inside that handler, we call a function that requires a context to be passed. I often saw this solution
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(context.Background(), "request", r)
otherFunc(ctx)
}
My question is, why don't we just pass a context from the request, like so
func myHandler(w http.ResponseWriter, r *http.Request) {
otherFunc(r.Context())
}
Doesn't it make more sense to pass the context of the request since we want the context to flow through our program? I thought that creating a background context is something we want to do only in the root parent, like init() function.
You might be missing the main point of contexts — supposedly due to poor HOWTOs you're dealing with.
The possiblility of carrying around arbitrary values in contexts is actually a misfeature of this type, regretted by its designers because it creates an anti-pattern (a proper way to deal with context-as-some-state is to have a set of values explicitly passed around).
The chief reason contexts exist is because they provide tree-like propagation of a signal (cancellation or "done" in the case of contexts).
So the original idea behind contexts is like follows:
The "root" context object is created for an incoming request.
Each "task" which is needed to be executed on behalf of the request is associated with its own context, derived from that of the request¹.
Those tasks may produce other tasks and so on.
As you can see, a hierarchy of "units of works" is formed, — linked to the object which is the reason for these units to exist and execute.
When the incoming request is cancelled (the client's socket got disconnected, for example), the context object associated with it is cancelled as well, and then all the linked tasks receive it as it's propagated from the root of the resulting context tree down to its leaves — making sure all the tasks being executed for the request are (eventually) cancelled.
Of course, in order for this to work, each "task" — which is usually a goroutine doing something — is required to "listen" from the context passed to it for that "done" signal.
Contexts also support timeout out of the box, so you might create a context which cancels itself after some fixed time interval passes.
So, back to the examples in your question.
The first example ignores the request's context completely and creates a from-scratch context ostensibly with the sole reason to carry stuff in it (bad).
The second example might use the context for its intended purpose (but we do not know as we cannot see that otherFunc).
I would advise you to read https://blog.golang.org/context, and the articles on concurrency patters in Go linked there.
¹ Actually, a new context need not be created if the task to be controlled by it has no other policy to "add" to the existing, parent, context.
The idea of derivation is here to implment additional ways to cancel work in this particular task as well as honoring the cancellation of the parent context.
For instance, a context derived for a particular task could have its own deadline or have a way to cancel only this particular context.
Of course, a complex—nested—context can be derived for a task: for example, a context with a deadline can be derived from the parent context, and then a cancellable context can be derived form the former. The result would be a context which is cancelled either explicitly by the code or when the deadline expires or when the parent context signals its cancellation.
Your two examples do entirely different things.
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(context.Background(), "request", r)
otherFunc(ctx)
}
This creates a new context, and stores the request as a value. There is rarely, if ever, any reason to do exactly this. A far more idiomatic solution would be just to pass the request to otherFunc like so:
func myHandler(w http.ResponseWriter, r *http.Request) {
otherFunc(r)
}
If you really do need to pass the request as a context value, you should probably do it with the current request's context, like so:
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(r.Context(), "request", r)
otherFunc(ctx)
}
I have handlers that respond to https requests. In the handlers I call a function F1() which does some application logic and connects to a mysql db and does a query. I want to know how I can use the golang context package to cancel the Db query if the client cancels the request. Do I need to pass the ctx to F1()? Also the code I have now will take 4 seconds even if F1() returns in less then 4. How can I return as soon as F1() returns?
func handler(w http.ResponseWriter, r *http.Request) {
ctx:= r.context()
F1()
select {
case <-ctx.Done():
case <- time.After( 4*time.Second):
}
w.WriteHeader(http.statusOk)
return
}
To begin, I highly recommend taking a look at the Context blog post to familiarize yourself with contexts, in addition to reading over the context documentation itself.
To address your specific questions:
How can you cancel the database query if the user cancels their quest?
To make this work, there are a few things you want to check:
Ensure that your database driver (if you are using database/sql) supports context cancellation.
Ensure you are using the Context variants of all available methods (e.g. db.QueryContext instead of db.Query).
Ensure that you are passing the context (or a derivative of the context) through the stack from your HTTP request through to the database calls.
Do I need to pass the ctx to F1()?
Per #3 above, yes: you will need to pass the context through all intermediate calls to "connect" the database call with the request context.
How can I return as soon as F1() returns?
The code that you have in your question calls F1 in series, rather than concurrently, with your cancellation/timeout select.
If you want to apply a specific deadline to your database call, use context.WithTimeout to limit how long it can take. Otherwise, you do not need to do anything special: just call F1 with your context, and the rest will happen for you, no select is needed.
We are using MediatR to implement a "Pipeline" for our dotnet core WebAPI backend, trying to follow the CQRS principle.
I can't decide if I should try to implement a IPipelineBehavior chain, or if it is better to construct a new Request and call MediatR.Send from within my Handler method (for the request).
The scenario is essentially this:
User requests an action to be executed, i.e. Delete something
We have to check if that something is being used by someone else
We have to mark that something as deleted in the database
We have to actually delete the files from the file system.
Option 1 is what we have now: A DeleteRequest which is handled by one class, wherein the Handler checks if it is being used, marks it as deleted, and then sends a new TaskStartRequest with the parameters to Delete.
Option 2 is what I'm considering: A DeleteRequest which implements the marker interfaces IRequireCheck, IStartTask, with a pipeline which runs:
IPipelineBehavior<IRequireCheck> first to check if the something is being used,
IPipelineBehavior<DeleteRequest> to mark the something as deleted in database and
IPipelineBehavior<IStartTask> to start the Task.
I haven't fully figured out what Option 2 would look like, but this is the general idea.
I guess I'm mainly wondering if it is code smell to call MediatR.Send(TRequest2) within a Handler for a TRequest1.
If those are the options you're set on going with - I say Option 2. Sending requests from inside existing Mediatr handlers can be seen as a code smell. You're hiding side effects and breaking the Single Responsibility Principle. You're also coupling your requests together and you should try to avoid situations where you can't send one type of request before another.
However, I think there might be an alternative. If a delete request can't happen without the validation and marking beforehand you may be able to leverage a preprocessor (example here) for your TaskStartRequest. That way you can have a single request that does everything you need. This even mirrors your pipeline example by simply leveraging the existing Mediatr patterns.
Is there any need to break the tasks into multiple Handlers? Maybe I am missing the point in mediatr. Wouldn't this suffice?
public async Task<Result<IFailure,ISuccess>> Handle(DeleteRequest request)
{
var thing = await this.repo.GetById(request.Id);
if (thing.IsBeignUsed())
{
return Failure.BeignUsed();
}
var deleted = await this.repo.Delete(request.Id);
return deleted ? new Success(request.Id) : Failure.DbError();
}
Im creating a Networking API and want people to be able to route requests to specific endpoints using a ServeMux. Instead of using a Server instance, I need to use my own low level ServerConn. This is because I am receiving both incoming HTTP requests and plain text data from the same port.
The problem, however, is that if I want to forward a request using my ServeMux, I would use it's ServeHTTP method. For this, I need to provide a ResponseWriter, which I don't know how to create an instance of since it is an interface, not a struct. Should a I create my own ResponseWriter struct? Is there one given by the Golang Standard Library? Or is there an alternate solution to this altogether?
I would avoid doing this altogether if at all possible. Mixing protocols on the same connection is bound to lead to hard-to-trace bugs, and unexpected behavior. If you really want to do it, and have all the http/1.1 mechanisms work correctly, leave as much as possible to the http package.
Since ResponseWriter is an interface, you would implement your own type to satisfy it. Look at the unexported response type in the http package for a full example. There's a lot to get right, and using it in combination with a ServerConn (which is documented as "do no use") is probably not a good idea.
The place to do this at a lower level would be in Accept inside the Server's net.Listener. Since you're going to have to parse the start of every request twice, you would need a net.Conn that can be "rewound" partly.
Make yourself a net.Listener that checks the start of the stream on a new connection, and if it looks like an http request, return a net.Conn that replays the first chunk you read off the wire on its first Reads. Something like:
type replayConn struct {
net.Conn
buf []byte
pos int
}
func (c *replayConn) Read(b []byte) (int, error) {
if c.pos < len(c.buf) {
n := copy(b, c.buf[c.pos:])
c.pos += n
return n, nil
}
return c.Conn.Read(b)
}
If the connection isn't http, then send the connection off to your other type of handler, and continue blocking on Accept.
I'm attempting to incorporate cloud enpoints into my app, I'm currently using the Python Quickstart for proof of concept. I'm having an issue when I attempt to call a method to send a card to my glass. Below is my code, please ignore the indentation is missing.
#endpoints.api(name='tasks', version='v1',
description='API for TaskAlert Management',
allowed_client_ids=[CLIENT_ID, endpoints.API_EXPLORER_CLIENT_ID])
class TaskAlertApi(remote.Service):
#endpoints.method(Task, Task,
name='task.insert',
path='tasker',
http_method='POST')
def insert_task(self, request):
TaskModel(author=request.author, content=request.content, date=request.date).put()
themirror = MainHandler()
themirror._insert_map_with_distance_from_home()
return request
So when "themirror._insert_map_with_distance_from_home()" is called I am getting the following error. Does anyone have any suggestions? I am trying to call this from myappspot.com/_ah/api/explorer.
in _insert_map_with_distance_from_home
self.mirror_service.timeline().insert(body=body).execute()
AttributeError: 'MainHandler' object has no attribute 'mirror_service'
I'm afraid you will have to rethink your code quite a bit for this but I'll try to get the basics explained here.
The main problem is that the MainHandler does quite a bit when actually receiving a HTTP request. The most important thing is what happens in the #util.auth_required decorator of the MainHandler's get method, which actually creates the mirror_service, authenticated for the current user. When you access the MainHandler directly from your code, none of this actually happens, so there is no mirror_service available (which results in the error you get).
Since the way endpoints are called is quite different from the way normal RequestHandlers are called, you also can't rely on stored session credentials or similar to match an Endpoints User to the Mirror User.
Basically what you would have to do is to create a new mirror_service inside of your endpoint method.
For this you will have to call your API authenticated (adding the Mirror API scopes to the authentication scopes). You can then extract the used access_token from the request header and use this access_token to create OAuth2Credentials to create the mirror_service.
Some code snippets without promises of completeness since it's hard to tell without knowing your actual code:
import os
from oauth2client.client import AccessTokenCredentials
# extract the token from request
if "HTTP_AUTHORIZATION" in os.environ:
(tokentype, token) = os.environ["HTTP_AUTHORIZATION"].split(" ")
# create simple OAuth2Credentials using the token
credentials = AccessTokenCredentials(token, 'my-user-agent/1.0')
# create mirror_service (using the method from util.py from the quickstart(
mirror_service = create_service('mirror', 'v1', credentials)
Of course you would then also have to change the _insert_map_with_distance_from_home to use this mirror_service object, but moving this method away from your MainHandler would make more sense in this context anyway.