We are already using nginx default ngx_http_limit_req_module for rate limiting for single nginx plus instance. We are planning to add more than 1 nginx for horizontal scaling so we need centralise place to store key count. We have implemented rate limiting based on API key rate limit tiers. Some API key required 100 RPS rate limit and some required 50 RPS and so on.
Since we want to scale instance horizontally we need to store API key at some centralised caching store like Redis or memcached. We explored several external modules on GitHub but all works on the IP address based rate limiting and not the API key based tiers.
Any solution which let me store API key on redis and multiple nginx can write/read value from it?
Tried several modules like https://github.com/limithit/ngx_dynamic_limit_req_module but they allow only 1 rate limiting rule per server block which is not working on our use case.
Related
To overcome latency, On "Startup.cs" of asp.net core 2.1, I am creating 2 static connections to Azure Redis & reuse those same connection instances during the application entire life cycle.
Is it is good practice to create multiple connections to one Azure redis instance? what is max. no. of connections? will multiple instance have billing implications? Is Azure redis usage charges based number of connections or as per the amount of data transfer? please confirm.
First, it is not a good practice to create two Azure Redis static connections in the application.
In general projects, Redis is not frequently used, but is instantiated and created when the business needs it, and released after use. If you need to use it frequently, you can instantiate it in Startup.cs when the project starts, and define an instance globally, so that there will be no frequent creation and deletion of instances.
For Azure Redis billing methods, you can refer to the official documentation. It is not based on the number of connections nor the amount of transmission. It is billed according to time.
It is actually recommend to use different connections to reflect the varying data packet sizes ie you could setup a connection with higher timeout for data that is bigger in size, as opposed to data that is small in size. This is recommended only when you have data packets being stored in redis of varying sizes ex: 1kb to 100 kb and you cannot reduce their size of the packet.
Having different connection ensures that pipelining that usually happens when fetching data does not result in cascading timeouts. Multi connection is also recommended in Microsoft docs, have a look here by scrolling to the bottom and see point 3
https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-troubleshoot-client#large-request-or-response-size
Hi I am looking for using envoy proxy for a multi tenant saas application, where we would like to let our customers have their own domains (kind of bring your own certificate model).
I am currently looking at using nginx,envoy etc as our front proxy/reverse proxy.Assuming we may have 1000's of customer , i was wondering how many certificates, routes (there by customers) can each envoy proxy /nginx can support with reasonable qos(of course not defined here).
Googling around this did not yield much results, hence this post here.
Have any body tested reverse proxies for their capability of supporting multiple certificates, rule sets etc, and if there is a degradation in performance as and when we add more listeners, certificates, routes?
Interestingly most cloud providers have some kind of limits e.g AWS one can upload only 25 certifcates per ALB, have only finite set of target groups (2k or something) and 100 odd routing rules.
Likewise Azure application gateway too has restrictions with rules, listeners etc
so does google application load balancer (100 certificates i guess).
My fundamental question is these limits on rules/ certificate etc defined for maintaining basic QOS or is it purely a commercial in nature?
Any insights/suggestions/blogs/posts/performance bench mark in this area will be of immense help for making right decision.
There's an IIB HTTP SOAP service exposed to multiple channels - the service has 4 operations and one of them is being consumed very frequently by a particular channel (less than 1 transaction per second).
Is there any way within IBM Integration Bus (broker or service level) to limit number of HTTP requests per channel (IP address) to 1 or n transactions per second?
You could implement it manually using the standard facilities of IIB, but rate-limiting is an API management feature and best implemented using out-of-the-box features of IBM API Connect. It works well with IIB, btw.
As already suggested above, this kind of logic should be done outside of IIB if you need it globaly.
On IIB level, you can configure many things, like the maximum amount of connection, but there's no logic to have this kind of pool for each users.
The best solution, in my opinion, is to use a network component specialized in this kind of logic. On my side, I've decided to implement this rule on the load balancer I have in front of my IIB server. A proxy could probably also do it.
For your specific case, if it is the only case where you need this logic, you can also consider creating different entry point for each application. If this is SOAP, and that the users currently calls /kimbertService/, you can consider having multiple SOAP Input node, with the following routes instead : /kimbertService/App1, /kimbertService/App2, /kimbertService/App3, and then you'll be sure that App1 will never block App2 ...
IIB has a feature of throttling by limiting the number of messages processed through a given message flow per second.
For example, to set the maximumRateMsgsPerSec property for a message flow included in an application, you can use the following sample code:
mqsiapplybaroverride –b BARfile -k applicationName -m sampleFlow#maximumRateMsgsPerSec=100
You can also do it through workload management policies by using the IIB web user interface.
Below is the link:
https://www.ibm.com/support/knowledgecenter/en/SSMKHH_9.0.0/com.ibm.etools.mft.doc/bj58270_.htm
A WORK-A-ROUND SOLUTION
The ideal solution, as others have mentioned, would be to have a API management gateway sitting in front of IIB to manage your API.
Now, a work-around solution could be following:
1) Have your main service flow duplicated, making them two different message flows. These two are your back-end flows performing the same thing but on one of them you can enable throttling.
2) Build a new router IIB flow which takes HTTP requests from consumers. This flow identifies the requester and routes it to the back-end flows accordingly.
Hope this helps.
Because a required pipeline components seems to have trouble hitting a database to details of messages, I am planning to use Host Throttling to limit the amount of files BizTalk is processing at the receive location. I want to be able to indicate that X number of messages should be processed within Y seconds (or any other feasible timespan). Does anyone know which throttling settings can be used to force this behavior?
I know how to set values, however I cannot find the best configuration.
(note: one of the solutions might also adjust the pipeline, but it contains third party components which cannot be adjusted.)
From How BizTalk Server Implements Host Throttling BizTalk looks at
Amount of memory in use (both systemwide and host process memory).
Number of in-process messages being delivered or processed (threshold for outbound throttling).
Number of threads in use. Database size, measured by the number of items in the queue tables for all hosts and the number of items in the spool and tracking tables.
Number of concurrent database connections.
Rate of message publishing (inbound)
and delivery or processing (outbound).
The only one that throttles inbound is the Rate of message publishing, but that is possible after the pipeline/port has processed the message so may not be of any use in this scenario, but you would have to test that.
You will probably want to set up that process under it's own host so if it hits throttling thresholds it does not throttle everything else as well.
If possible you should move the component to a send port pipeline as throttling send ports is much more controllable. One way is to set the send port to ordered delivery, although that can cause a backlog especially if you get a suspended message.
I think your most straightforward approach here would be to write a custom adapter. Unfortunately, the out of the box File Adapter does not directly support throttling/polling intervals, and I don't think the suggestions given already would not directly impact the custom pipeline processing if it's directly hitting the DB through ADO.NET (but it couldn't hurt to try). You can set the BatchSize property on the file adapter settings, but even then there's nothing stopping it from submitting that batchsize as fast as it possibly can over and over again.
A custom adapter could be created to wait for a period before submitting additional files for processing. You could base it on the SDK File Adapter sample.
I need to set up a tracking server that will only serve 1x1 pixels and log all requests.
I initially thought of using Amazon's S3 or CloudFront but their costs are prohibitively high for me. I need to serve 500M pixels a day, and S3 charges $0.4 per 1M GET requests, so even without the data transfer costs I'm at $6,000/month.
I am considering setting up nginx or lighttpd on an EC2 instance. What performance should I expect with those two (e.g. per one large EC2 instance)? Are there better free products for this task?
Nginx is indeed a good candidate for this and already has built in support for empty GIFs (see http://wiki.nginx.org/HttpEmptyGifModule).
Disk I/O will probably be the biggest issue for this server because of the access logging. The only way to figure out the performance of the different EC2 instances is to test them.
If one EC2 instance does not offer the performance you need, or if you need any redundancy for this service, you should also look into using a load balancer (either an AWS Elastic Load Balancer or your own custom one).
You could also set up multiple smaller servers in different geographical regions and use DNS latency based routing to route requests to them (use either AWS Route 53 latency based routing or another DNS solution). This would significantly reduce the connection time to your server and would distribute the load across several data centers.