Fastly allows to customise vcl_error subroutine. However, If I override it (e.g.
sub vcl_error {
#FASTLY error
set obj.http.Custom-Header = "foo-bar";
return(deliver);
}
), original response body with error cause (e.g. "first byte timout") is lost.
Is it possible to obtain a cause, so I can add it to additional some header or syntetic body
The vcl_error subroutine is triggered either implicitly by Fastly (see the documentation for examples of when it does this) or explicitly using the error statement.
Within vcl_error the obj.status and obj.response variables provide information about the nature of the error.
If you're explicitly triggering vcl_error, then from within the subroutine you're invoking the error statement you should set a custom HTTP header on an object that you can read back from within vcl_error.
For example, the req object is available to all subroutines so you could use req.http.{NAME}) to store off any contextual error information you want to use as part of your synthetic error response.
One caveat with persisting data in this fashion is that you can't persist data across certain boundaries, such as the move from a subroutine on a 'fetching node' to a 'delivery node' (see clustering for details of what the difference between fetching/delivery nodes).
Off the top of my head (see also: https://www.integralist.co.uk/posts/fastly-varnish/#breadcrumb-trail) I believe if you're invoking error from vcl_fetch you'll need to persist the data to the beresp object and not req. Varnish will copy beresp to the obj object that is exposed to the vcl_error subroutine.
If you have any other questions or concerns, then please reach out to support#fastly.com who will be happy to help.
Please also refer to the Fastly 'Developer Hub' which has a bunch of resources on Varnish and VCL that might be useful to you.
Related
For the past few days, I've been reading about Go and one concept that I keep returning to are contexts.
I think I understand the motivation behind creating such a structure. The thing that I don't understand is a particular use case when using a context in the incoming HTTP request.
Let's say we have a following httpHandlerFunc. Inside that handler, we call a function that requires a context to be passed. I often saw this solution
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(context.Background(), "request", r)
otherFunc(ctx)
}
My question is, why don't we just pass a context from the request, like so
func myHandler(w http.ResponseWriter, r *http.Request) {
otherFunc(r.Context())
}
Doesn't it make more sense to pass the context of the request since we want the context to flow through our program? I thought that creating a background context is something we want to do only in the root parent, like init() function.
You might be missing the main point of contexts — supposedly due to poor HOWTOs you're dealing with.
The possiblility of carrying around arbitrary values in contexts is actually a misfeature of this type, regretted by its designers because it creates an anti-pattern (a proper way to deal with context-as-some-state is to have a set of values explicitly passed around).
The chief reason contexts exist is because they provide tree-like propagation of a signal (cancellation or "done" in the case of contexts).
So the original idea behind contexts is like follows:
The "root" context object is created for an incoming request.
Each "task" which is needed to be executed on behalf of the request is associated with its own context, derived from that of the request¹.
Those tasks may produce other tasks and so on.
As you can see, a hierarchy of "units of works" is formed, — linked to the object which is the reason for these units to exist and execute.
When the incoming request is cancelled (the client's socket got disconnected, for example), the context object associated with it is cancelled as well, and then all the linked tasks receive it as it's propagated from the root of the resulting context tree down to its leaves — making sure all the tasks being executed for the request are (eventually) cancelled.
Of course, in order for this to work, each "task" — which is usually a goroutine doing something — is required to "listen" from the context passed to it for that "done" signal.
Contexts also support timeout out of the box, so you might create a context which cancels itself after some fixed time interval passes.
So, back to the examples in your question.
The first example ignores the request's context completely and creates a from-scratch context ostensibly with the sole reason to carry stuff in it (bad).
The second example might use the context for its intended purpose (but we do not know as we cannot see that otherFunc).
I would advise you to read https://blog.golang.org/context, and the articles on concurrency patters in Go linked there.
¹ Actually, a new context need not be created if the task to be controlled by it has no other policy to "add" to the existing, parent, context.
The idea of derivation is here to implment additional ways to cancel work in this particular task as well as honoring the cancellation of the parent context.
For instance, a context derived for a particular task could have its own deadline or have a way to cancel only this particular context.
Of course, a complex—nested—context can be derived for a task: for example, a context with a deadline can be derived from the parent context, and then a cancellable context can be derived form the former. The result would be a context which is cancelled either explicitly by the code or when the deadline expires or when the parent context signals its cancellation.
Your two examples do entirely different things.
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(context.Background(), "request", r)
otherFunc(ctx)
}
This creates a new context, and stores the request as a value. There is rarely, if ever, any reason to do exactly this. A far more idiomatic solution would be just to pass the request to otherFunc like so:
func myHandler(w http.ResponseWriter, r *http.Request) {
otherFunc(r)
}
If you really do need to pass the request as a context value, you should probably do it with the current request's context, like so:
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(r.Context(), "request", r)
otherFunc(ctx)
}
I am experimenting with firebase-queue. I saw the option for sanitizing. It's described in the doc as
sanitize - specifies whether the data object passed to the processing
function is sanitized of internal keys reserved for use by the queue.
Defaults to true.
What does it mean?
I am getting an error for not specifying { sanitize : false }
When the sanitize option is set, the queue sanitizes (or cleans) the input provided to the processing function so that it resembles that which the original client placed onto the queue, and doesn't contain any of the keys added by the implementation of the queue itself.
If, however, you rely on a key (usually the keys starting with an underscore, e.g. _id) that is added by the queue, and not the original client, you need to set sanitize: false so those keys are returned to your function and they're not undefined.
You can clearly see the difference with a simple processing function that just performs a console.log(data).
A quick note about why these keys are removed by default: Reading or writing directly to the location (as it looks like you're perhaps doing, by passing undefined into the client SDK child() method instead of data._id) is generally a bad idea from within the worker itself as writes performed directly are not guarded by the extensive transaction logic in the queue to prevent race conditions. If you can isolate the work to taking input from the provided data field, and returning outputs to the resolve() function, you'll likely have a better time scaling up your queue.
I am new to gRPC and trying to use it in my existing system. However, I get this unused parameter error while compiling it.
server_grpc.cc:100:39: error: unused parameter ‘context’[-Werror=unused-parameter]
Status MyFunc(ServerContext* context, const QueryRequest* request,
Probably the context parameter is used in some other cases. But, in simple hello world type of example it is not used. Is there a way to compile the protocol buffer without generating the ServerContext parameter ?
I know I can make the compiler ignore warning messages. But, just wondering if it can be done without affecting the way my system is being compiled right now.
I would like to know how the context is used ? It would be great if anybody can give pointers to how to use this context. I might find a use of it in my work.
The ServerContext is provided to, well, add context for every RPC you get. It'll allow you to tweak certain aspects of the RPC, such as deal with authentication, or add metadata to your response back to the client. You may or may not need that parameter, obviously, depending on your needs.
We didn't want to add an option for this specifically, because that'd complexify the code and tool for little benefit, so the code generator and the function signature force you to have that parameter at all times. Now this isn't really a big deal, because in C++, you can specifically ask your compiler to ignore a parameter in a specific instance, for example with the following:
Status SayHello(ServerContext* context, const HelloRequest* request,
HelloReply* reply) override {
(void) context; // ignore that variable without causing warnings
std::string prefix("Hello ");
reply->set_message(prefix + request->name());
return Status::OK;
}
And that's how I'd suggest you to take care of that warning in that specific instance, without causing your whole project to not have warnings enabled.
I'm trying to set a header in the response object after I call
chain.doFilter()
However, the header does not get set. Does control ever come back to the doFilter() method after a call to chain.doFilter()?
However, the header does not get set.
That can happen if the response is already committed (read: when the first part of the response, including all headers, are already been sent to the client; this is a point of no return). A bit sane servletcontainer would throw an IllegalStateException on any attempt to set a header on an already committed response. This should be visible in the server logs. Have you read them?
Does control ever come back to the doFilter() method after a call to chain.doFilter()?
You could easily check it yourself by placing a breakpoint or a sysout/logger line. But yes, surely the control comes back the usual Java way and the remaining lines will be executed, provided that there's no uncaught exception coming from the doFilter() call.
Coming back to your concrete functional requirement, you'd need to rewrite your code in such way that the header is been set before the response is ever committed. As the concrete functional requirement is not elaborated in any way, it's not possible to give some hints or kickoff examples in this answer. The most straightforward way would be to just set the header before calling doFilter().
Request filters can:
perform security checks
reformat request headers or bodies
audit or log requests
Response filters can:
compress the response stream
append or alter the response stream
create a different response altogether
I was trying to do a little data access optimization, and I ran into a situation where a dictionary appeared to get out of sync in a way that should be impossible, unless I'm somehow getting into a multithreaded situation without knowing it.
One column of GridLabels binds to a property that does data access -- which is a tad expensive. However, multiple rows end up making the same call, so I should be able to head any problems off at the pass by doing a little caching.
However, elsewhere in the app, this same code is called in ways where caching would not be appropriate, I needed a way to enable caching on demand. So my databinding code looks like this:
OrderLabelAPI.MultiSyringeCacheEnabled = True
Me.GridLabels.DataBind()
OrderLabelAPI.MultiSyringeCacheEnabled = False
And the expensive call where the caching happens looks like this:
Private Shared MultiSyringeCache As New Dictionary(Of Integer, Boolean)
Private Shared m_MultiSyringeCacheEnabled As Boolean = False
Public Shared Function IsMultiSyringe(orderLabelID As Integer) As Boolean
If m_MultiSyringeCacheEnabled Then
'Since this can get hit a lot, we cache the values into a dictionary. Obviously,
'it goes away after each request. And the cache is disabled by default.
If Not MultiSyringeCache.ContainsKey(orderLabelID) Then
MultiSyringeCache.Add(orderLabelID, DoIsMultiSyringe(orderLabelID))
End If
Return MultiSyringeCache(orderLabelID)
Else
Return DoIsMultiSyringe(orderLabelID)
End If
End Function
And here is the MultiSyringeCacheEnabled property:
Public Shared Property MultiSyringeCacheEnabled As Boolean
Get
Return m_MultiSyringeCacheEnabled
End Get
Set(value As Boolean)
ClearMultiSyringeCache()
m_MultiSyringeCacheEnabled = value
End Set
End Property
Very, very rarely (unreproducably rare...) I will get the following exception: The given key was not present in the dictionary.
If you look closely at the caching code, that's impossible since the first thing it does is ensure that the key exists. If DoIsMultiSyringe tampered with the dictionary (either explicitly or by setting MultiSyringeCacheEnabled), that could also cause problems, and for awhile I assumed this had to be the culprit. But it isn't. I've been over the code very carefully several times. I would post it here but it gets into a deeper object graph than would be appropriate.
So. My question is, does datagridview databinding actually get into some kind of zany multithreaded situation that is causing the dictionary to seize? Am I missing some aspect of shared members?
I've actually gone ahead and yanked this code from the project, but I want to understand what I'm missing. Thanks!
Since this is ASP.NET, you have an implicit multithreaded scenario. You are using a shared variable (see What is the use of a shared variable in VB.NET?), which is (as the keyword implies) "shared" across multiple threads (from different people visiting the site).
You can very easily have a scenario where one visitor's thread gets to here:
'Since this can get hit a lot, we cache the values into a dictionary. Obviously,
'it goes away after each request. And the cache is disabled by default.
If Not MultiSyringeCache.ContainsKey(orderLabelID) Then
MultiSyringeCache.Add(orderLabelID, DoIsMultiSyringe(orderLabelID))
End If
' My thread is right here, when you visit the site
Return MultiSyringeCache(orderLabelID)
and then your thread comes in here and supercedes my thread:
Set(value As Boolean)
ClearMultiSyringeCache()
m_MultiSyringeCacheEnabled = value
End Set
Then my thread is going to try to read a value from the dictionary after you've cleared it.
That said, I am not sure what performance benefit you expect from a "cache" that you clear with every request. It looks like you should simply not make this variable shared- make it an instance variable- and any user request accessing it will have their own copy.