I am using Envoy's LocalRatelimit.
Version Info:
istio 1.11.4
Envoy v1.19.1
Kubenretes 1.12 (bare metal)
In order to make the request Burst resistant, the Token Bucket is specified as follows.
name: envoy.filters.http.local_ratelimit
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
stat_prefix: http_local_rate_limiter
token_bucket:
max_tokens: 200
tokens_per_fill: 20
fill_interval: 1s
filter_enabled:
runtime_key: local_rate_limit_enabled
default_value:
numerator: 100
denominator: HUNDRED
filter_enforced:
runtime_key: local_rate_limit_enforced
default_value:
numerator: 100
denominator: HUNDRED
response_headers_to_add:
- append: false
header:
key: x-local-rate-limit
value: "true"
https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/local_rate_limit_filter
This means that a maximum of 200 rps can be accepted for one second, and then every second, the Token will recover by 20 to prepare for the next Burst.
However, what I want to achieve now is not to have these 200rps reach the upstream server immediately, but to allow some delay and send the request to the upstream server. Specifically, I'm hoping for something equivalent to the delay option in nginx's limit_req directive.
http://nginx.org/en/docs/http/ngx_http_limit_req_module.html
Is there any richness in Envoy to specify this delay feature? If not, will this be implemented in the future?
This post has been multi-posted to the EnvoyProxy user group (here).
Related
I am using wordpress. I deploy Varnish Using Docker. This Is My default.vcl. What's Wrong With This Config?? Sometimes, Get Random 503 Error. I Exclude Wordpress Search Page Using RegEx. Also Get Random 503 Error On wordpress Search Page Too!!
varnishlog
https://www.dropbox.com/s/ruczg2i3h/log.txt
I am Using NGINX Backened..
Help Appreciated
Thanks
Your log output contains the following lines:
- Error out of workspace (bo)
- LostHeader Date:
- BerespHeader Server: Varnish
- VCL_call BACKEND_ERROR
- LostHeader Content-Type:
- LostHeader Retry-After:
Apparently you ran out of workspace memory because the size of the response.
The following parameters in your Docker config might cause that:
-p http_resp_hdr_len=65536 \
-p http_resp_size=98304 \
While increasing the size of individual headers and the total response size, the total memory consumption exceeds the workspace_backend value, which defaults to 64k.
Here's the documentation for http_resp_size:
$ varnishadm param.show http_resp_size
http_resp_size
Value is: 32k [bytes] (default)
Minimum is: 0.25k
Maximum number of bytes of HTTP backend response we will deal
with. This is a limit on all bytes up to the double blank line
which ends the HTTP response.
The memory for the response is allocated from the backend
workspace (param: workspace_backend) and this parameter limits
how much of that the response is allowed to take up.
As you can see, it affects workspace_backend. So here's the documentation for that:
$ varnishadm param.show workspace_backend
workspace_backend
Value is: 64k [bytes] (default)
Minimum is: 1k
Bytes of HTTP protocol workspace for backend HTTP req/resp. If
larger than 4k, use a multiple of 4k for VM efficiency.
NB: This parameter may take quite some time to take (full)
effect.
The solution is to increase the workspace_backend through a -p runtime parameter.
I want to block excessive requests on a per-IP basis, allowing at maxium 12 requests per second. For this sake, I have the following in /etc/nginx/nginx.conf:
http {
##
# Basic Settings
##
...
limit_req_zone $binary_remote_addr zone=perip:10m rate=12r/s;
server {
listen 80;
location / {
limit_req zone=perip nodelay;
}
}
Now, when I'm running the command ab -k -c 100 -n 900 'https://www.mywebsite.com/' in the terminal, I get the output with only 179 non-2xx responses out of 900.
Why isn't Nginx blocking most requests?
Taken from here:
with nodelay nginx process all burst requests instantly and without
this option nginx makes excessive requests to wait so that overall
rate would be no more than 1 request per second and last successful
request took 5 seconds to complete.
rate=6r/s actually means one request in 1/6th of a second. So if
you send 6 request simultaneously you'll get 5 of them with 503
The problem might be in your testing method (or your expectations from it).
I am currently try to load data from Prometheus Pagespeed Exporter (https://github.com/foomo/pagespeed_exporter) directly into ELK using Metricbeat. It seems so, that the Call of Pagespeed Exporter requires more time than Metricbeats offers to scrape the required data. A Client Timeout occures:
unable to decode response from prometheus endpoint: error making http request: Get http://pagespeed-exporter-monitoring:9271/metrics: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
Currently the Request is cancelled after about 10s while the Timeout in metricbeat should be defined like Timeout = Period (https://www.elastic.co/guide/en/beats/devguide/current/metricset-details.html). The Period in my case is configured to 3600s.
metricbeat:
deployment:
metricbeatConfig:
metricbeat.yml: |
metricbeat.modules:
- module: prometheus
enabled: true
period: 3600s
hosts: ["pagespeed-exporter-monitoring:9271"]
metrics_path: /metrics
Is there any option to increase the Timeout or is there any other issue?
You can override the timeout on the module config,
https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-metricbeat.html#_timeout
For example:
metricbeat:
deployment:
metricbeatConfig:
metricbeat.yml: |
metricbeat.modules:
- module: prometheus
enabled: true
period: 3600s
hosts: ["pagespeed-exporter-monitoring:9271"]
metrics_path: /metrics
timeout: 60s # adjust here
I am running an nginx-ingress controller in a kubernetes cluster and one of my log statements for the request looks like this:
upstream_response_length: 0, 840
upstream_response_time: 60.000, 0.760
upstream_status: 504, 200
I cannot quite understand what does that mean? Nginx has a response timeout equal to 60 seconds, and tries to request one more time after that (successfully) and logs both requests?
P.S. Config for log format:
log-format-upstream: >-
{
...
"upstream_status": "$upstream_status",
"upstream_response_length": "$upstream_response_length",
"upstream_response_time": "$upstream_response_time",
...
}
According to split_upstream_var method of ingress-nginx, it splits results of nginx health checks.
Since nginx can have several upstreams, your log could be interpreted this way:
First upstream is dead (504)
upstream_response_length: 0 // responce from dead upstream has zero length
upstream_response_time: 60.000 // nginx dropped connection after 60sec
upstream_status: 504 // responce code, upstream doesn't answer
Second upstream works (200)
upstream_response_length: 840 // healthy upstream returned 840b
upstream_response_time: 0.760 // healthy upstream responced in 0.760
upstream_status: 200 // responce code, upstream is ok
P.S. JFYI, here's a cool HTTP headers state diagram
Nginx supports proxy response buffering, according to the mailing list (Nov 2013):
The proxy_buffering directive disables response buffering, not request
buffering.
As of now, there is no way to prevent request body buffering in nginx.
It's always fully read by nginx before a request is passed to an
upstream server. It's basically a part of nginx being a web
accelerator - it handles slow communication with clients by itself and
only asks a backend to process a request when everything is ready.
Since version 1.7.11 Nginx (Mar 2015) directive proxy_request_buffering see details:
Update: in nginx 1.7.11 the proxy_request_buffering directive is
available, which allows to disable buffering of a request body. It
should be used with care though, see docs.
See docs for more details:
Syntax: proxy_request_buffering on | off;
Default: proxy_request_buffering on;
Context: http, server, location
When buffering is enabled, the entire request body is read from the
client before sending the request to a proxied server.
When buffering is disabled, the request body is sent to the proxied
server immediately as it is received. In this case, the request cannot
be passed to the next server if nginx already started sending the
request body.
When HTTP/1.1 chunked transfer encoding is used to send the original
request body, the request body will be buffered regardless of the
directive value unless HTTP/1.1 is enabled for proxying.
The question is about buffering for Nginx (Load Balancer). For instance, we have the following scheme:
Nginx (LB) -> Nginx (APP) -> Backend
Nginx (APP) buffers the request from the load balancer and also buffers response from Backend. But does it make sense to buffer request and response on the Nginx (LB) side if both Nginx nodes are physically located close to each other (less than 2ms ping) with pretty fast network connection in between?
I measured the following benchmarks - Please note that this is for illustration only - Production loads would be significantly higher :
siege --concurrent=50 --reps=50 https://domain.com
proxy_request_buffering on;
Transactions: 2500 hits
Availability: 100.00 %
Elapsed time: 58.57 secs
Data transferred: 577.31 MB
Response time: 0.55 secs
Transaction rate: 42.68 trans/sec
Throughput: 9.86 MB/sec
Concurrency: 23.47
Successful transactions: 2500
Failed transactions: 0
Longest transaction: 2.12
Shortest transaction: 0.10
proxy_request_buffering off;
Transactions: 2500 hits
Availability: 100.00 %
Elapsed time: 57.80 secs
Data transferred: 577.31 MB
Response time: 0.53 secs
Transaction rate: 43.25 trans/sec
Throughput: 9.99 MB/sec
Concurrency: 22.75
Successful transactions: 2500
Failed transactions: 0
Longest transaction: 2.01
Shortest transaction: 0.09