When working with the PageSpeed product from Huckabuy which uses Cloudflare workers to implement some page speed boosters I want to be able to bypass the behavior of the boosters without having to reconfigure the settings. Are there any ways to accomplish this that are exposed by Huckabuy or perhaps a generic way using a URL query string parameter or an HTTP header to bypass any given workers in Cloudflare?
Below is an example of what I'd like to be able to do.
https://www.example.com?huckabuy_pagespeed=false
If that's not possible then perhaps something specific to Cloudflare like the example below.
https://www.example.com?disable_cf_workers=true
Or potentially the following example HTTP header.
DISABLE_CF_WORKER: true
I don't know anything about Huckabuy specifically, but there is no general way to bypass Cloudflare Workers with special request properties. Workers are often used to implement security policies, so it's important that they cannot be bypassed.
Of course, any particular Worker is free to implement its own bypass mechanism if it so chooses.
Related
I am looking for as reliable and accurate / quick means possible to add in some htaccess code to block visits to a website from countries / IPs which are not in the white listed list of countries I want to allow access for. I have looked at https://www.ip2location.com/free/visitor-blocker which seems to offer a solution - for the 4 allowed countries I want to allow access - it has created a 4.1MB htaccess file! Will this mean slow access when someone attempts to view the site? I guess using a free service like this means the data is likely nowhere near comprehensive?
Does anyone have any suggestions on a good way to allow just visitors from a few countries access to a website?
It sounds like the service you used basically tried tried to brute force the blacklist. If you look into the htaccess file I'm sure you will be a long list of hard coded IP blocks.
In my opinion this is a terrible way to handle a geographic blacklist. To your original question - there is no "most reliable, most accurate, and quickest" method. Those are separate categories and you will need to preference one over the next.
For performance you could consider blacklisting at the routing level / dns server / proxy. This obviously isn't going to be the quickest way in terms of performance. There are Apache Modules that exist that allow you to use a local database to compare the incoming IP address with a list of known IP blocks from the blacklisted country. One of the main issues with this is that you need to constantly update your database to take in new IP blocks.
In my opinion the "best" method to do this is a simple redirect at the application layer using server side code. There exists several geographic API's where you can send in the IP or Hostname and get back a country of origin. An example:
$xml= new SimpleXMLElement(file_get_contents('http://www.freegeoip.net/xml/{IP_or_hostname}'));
if($xml->CountryCode == "US") {
header('Location: http://www.google.com');
}
There are two ways to block a visitor in web server. One is using firewall (.htaccess etc) and another one is using server-side scripting (PHP etc).
If you are concern of the performance of the firewall option, then you can download the IP2Location LITE database from http://lite.ip2location.com and implement the database in your local server. For every connection, you query the visitor IP address and find their country. You can redirect or block them using the PHP codes. Please find the complete steps in https://www.ip2location.com/tutorials/redirect-web-visitors-by-country-using-php-and-mysql-database
There is also another option to use remote geolocation API. However, we do not suggest this method because of network latency. It will slow down all user experience due to API queries.
I've been looking around but I couldn't find anything useful. What would be the best practice of securing a Symfony app from brute force attacks? I looked into the SecurityBundle but I couldn't find anything.
Something that I do for this is that I keep a log using event subscribers based on IP addresses and/or usernames attempting to log in. Then, if after an x amount of time an IP/User has tried to log in with a failure then I move that IP address/User to a ban list.. and after that anytime that IP/User tries to log in I deny it right away based on that ban list.
You can also play with the time between attempts and all those goodies inside the event subscriber
Let me know if it makes sense.
Use cloudflare for DDOS attacks. However it may be expensive.
You can prevent dictionary attacks using https://github.com/codeconsortium/CCDNUserSecurityBundle
Honestly I do that with my web/cache server when I need to. I recently used varnish cache to do that with a plugin called vsthrottle. (which is probably one of many things you can use on the server level) the advantage of doing it on the webserver level instead of symfony is that you are not even hitting the php level and compiling all the vendors just to end up rejecting a request, and you are not using a separate data storage (be it mysql or something fast like memcached) to log every request and compare on the next one... If the request reaches the php layer, then it already cost you some performance, and a DDOS of that type will still hurt you even if you are returning a rejection from symfony because it is causing the server to compile php and part of the symfony code.
If you insist on doing it in symfony, register a listener that listens on all requests, parse request headers for either IP addresses or X_forwarded_for (in case you are behind a load balancer in which case only the load balancer ip will keep showing with regular ip check) and then find a suitable way to keep track of all requests up to a minute old (you could probably use memcached for fast storage, with a smart way to increment counts for each ips) and if an ip hits you more than lets say 100 times per the last 1 minute, you return a forbidden response or a too many requests response instead of the usual one... But I do not recommend this as usually built solutions (like the varnish I used) are better, in my case I could throttle for specific routes and not others for example.
I am looking for some effective ways to bypass the cache whenever necessary. In the process of searching for that I have found this link
From the referenced post I found that the query string approach may not work when the squid like proxies are used. I did not test this.
I see that stackoverflow in itself is using the query string approach, below is the screenshot for the same captured before login to the site.
Would like to know if the query string approach is a reliable solution to push the css and js updates whenever a new software build is released.
It's reliable browser side, meaning that since the URL is different (because there's a different query parameter), it will fetch a new copy.
Server side it depends on your server. Some caching proxies may ignore query parameters for the purpose of determining URL equality. AWS CloudFront for example does so by default. That's always a configurable setting. Since, presumably, you are in charge of the server, you can configure it as needed.
I want to let my clients speak directly with ElasticSearch REST API, obviously preventing them from performing any data or configuration change.
I had a look at ElasticSearch REST interface and I noticed the pattern: HTTP GET requests are pretty safe (harmless queries and status of cluster).
So I thought I can use Cloudfront as a CDN/Proxy that only allows GET/HEAD methods (you can impose such restrict it in the main configuration).
So far so good, all is set up. But things don't work because I would need to open my EC2 security group to the world in order to be reachable from Cloudfront! I don't want this, really!
When I use EC2 with RDS, I can simply allow access to my EC2 security group in RDS security groups. Why can't I do this with CloudFront? Or can I?
Ideas?
edit: It's not documented, but ES accepts facets query, which involve a (JSON) body, not only with POST, but also with GET. This simply breaks HTTP recommendation (as for RFC3616) by not ignoring the body for GET request (source).
This relates because, as pointed out, exposing ES REST interface directly can lead to easy DOS attacks using complex queries. I'm still convinced though, having one less proxy is still worth it.
edit: Other option for me would be to skip CloudFront and adding a security layer as an ElasticSearch plugin as shown here
I ended coding with my own plugin. Surprisingly there was nothing quite like this around.
No proxies, no Jetty, no Tomcat.
Just a the original ES rest module and my RestFilter. Using a minimum of reflection to obtain the remote address of the requests.
enjoy:
https://github.com/sscarduzio/elasticsearch-readonlyrest-plugin
Note that even a GET request can be harmful in Elasticsearch. A query which simply takes up too much resources to compute will bring down your cluster. Facets are a good way to do this.
I'd recommend writing a simple REST API you place in front of ES so you get much more control over what hits your search cluster. If that's not an option you could consider running Nginx on your ES boxes to act as a local reverse proxy, which will give you the same control (and a whole lot more) as CloudFront does. Then you'd only have to open up Nginx to the world, instead of ES.
A way to do this in AWS would be:
Set up an Application Load Balancer in front of your ES cluster. Create a TLS cert for the ALB and serve https. Open the ES security group to the ALB.
Set up CloudFront and use the ALB as origin. Pass a custom header with a secret value (for WAF, see next point).
Set up WAF on your ALB to only allow requests that contain the custom header with the secret value. Now all requests have to go through CloudFront.
Set up a Lambda#Edge function on your CloudFront distribution to either remove the body from GET requests, or DENY such requests.
It’s quite some work, but there’s advantages over the plugin, e.g.:
CloudFront comes with free network DDOS protection
CloudFront gives your users lower latency to ES because of the fast CloudFront network and global PoP’s.
Opens many options to use CloudFront, WAF and Lamba#Edge to further protect your ES cluster.
I’m working on sample code in CDK to set all of this up. Will report back when that’s ready.
My ASP.NET application will have to handle HTTP GET requests that will have the following URL format:
http://mySite/getStuff?id="actualIdHere"
currently the requirement is to validate that there're no parameters in the query string except id and indicate an error like "unknown parameter P passed".
Is such requirement a good idea? Will it interfere with some obviously valid cases of using the application I haven't thought of?
It would be better to just validate the presence of id.
Validating unknown parameters doesn't serve much of a purpose, they will just be ignored.
Just edited my answer here:
There are also tracking solutions out there that will add to your query string.
One that comes to mind is web analytics.
If your application is going to be a public web site, you will want to implement some tracking of your traffic (e.g. google analytics).
If you want to implement a marketing campaign to draw traffic to your site, you will likely need to add a few parameters (specific to the tracking system you're using) to your querystring to check the effectiveness of your campaign.
It depends on your target audience.
It is not a good practice for public websites where you are aware of SEO, for example if you implement Google Analytics then a user come to your site from Search Results may have a parameter in URL like googleclid.
However in more protected websites it is fine.
It might affect forward compatiblity. For example, if you have separate client applications/websites that actually call this URLs, and future versions of these clients might provide additional parameters to getStuff (like a sort ordering, backlink, etc), making hard requirements on the parameters might make it harder to roll out new versions smoothly (i.e. cannot roll out new clients until the server is updated).
This in addition to the traffic forwarding parameters public websites might get as additional input, like the other answers mention.