I'm having to learn Squid on the fly for a project and have tried to Google this, but not sure how to ask the question so Google can find it. I have a customer that wants to distribute larger amounts of video to multiple clients, but they have a slow WAN. The idea is to have Squid as a reverse proxy and cache to hold the videos on the local networks for faster transfer to the end users and to avoid saturating the WAN.
My question is this. If multiple users try to download a video that Squid has not yet cached, or fully cached, at the same time, what happens? In my particular scenario (insert unnecessary explanation here), this will happen frequently. Is a separate download of the video from the host opened for each request to Squid, or is Squid smart enough to manage it and only have one connection back to the host actually downloading the video?
This problem is called thundering herd or cache miss storm. It happens with new resources and also on higly used resources's cache invalidation.
For squid used as a reverse Proxy there are some options available to handle this, like the Collapsed Forwarding. The idea is to temporize parallel queries to the same resources, and use only one query fetching the resource on the backend.
Check the link for details, because depending on the squid version you may have to try different configuration tokens and values.
Related
Intro
Hi! First, let me say that I am a networking n00b. With that in mind :
We have a small video conferencing app, dedicated to eLearning, which does live streaming through janus (webRTC). We offer the possibility to our clients to record their videos. The live streaming part we got running pretty well now, but we are having a small issue with video playback.
When we are playing back the videos, as long as there aren't many large video files to stream, everything plays back correctly. However, we had fringe cases of trouble for a while and I was sent to investigate them.
Problem encountered
When playing back multiple large video files, I noticed that I can't get more than 4-5 large files to stream simultaneously on a single client.
When the limit is hit, there seems to be some kind of race condition lock happening : a few videos are stuck, and a few are playing (usually 2 or 3). Then one of the playing videos will get stuck, and one of the stuck videos will start playing.
However, it doesn't affect the playback on other clients. When it get stuck, I can't even connect to the MinIO web interface from the same client, but I can from another client (i.e. another browser, or from another machine). I can also stream as much from the other client as I can from the one that is stuck.
I've been testing different configurations on a test minio server by loading the same file many times from different tabs in Chrome, which seems to recreate the problem.
Server Description
The files are hosted on a cloud storage that offers > 7 Gpbs bandwidth. The storage is mounted on a MinIO instance in a kubernetes cluster, behind a NGINX Ingress Controller that serves as the single point of entry to the cluster, and so it also controls traffic to the other micro-services on the k8s cluster.
Each k8s node has a guaranteed bandwitdth of > 250 Mbps, if that matters in this case.
The MinIO instance is mainly used to create transient sharing rights. We call the videos simply by pointing to their location using the DNS we set up for the service.
What has been investigated and tried
At first, I thought it might be a MinIO misconfiguration. However, I looked at the config files and the documentation and couldn't find anything that seemed to limit the number of connections / requests per client.
While reading, I stumbled upon the fact that HTML/1.1 didn't allow for more than 6 connections on Chrome and thought I hit the jackpot. But then I went and looked and the protocol used to get the files is already HTTP2 (h2).
Then I went one level higher and looked through the configuration of the NGINX Ingress Controller. Here again, everything seems ok :
events {
multi_accept on;
worker_connections 16384;
use epoll;
}
[...]
http2_max_field_size 4k;
http2_max_header_size 16k;
http2_max_requests 1000;
http2_max_concurrent_streams 128;
[...]
So I've been scouring the net for a good while now and I'm getting more and more confused by what I could investigate next, so I thought I'd come here and ask my very first StackOverflow question.
So, is there anything I could do with the current setup to make it so we can stream more large files simultaneously? If not, what are your thoughts and recommendations?
Edit :
I've found a workaround by searching hard enough : Increase Concurrent HTTP calls
At first I was not a fan - HTTP2 is supposed, from my understanding, to support a lot of parallel requests. However, I think I found the crux of the problem here : https://en.wikipedia.org/wiki/Head-of-line_blocking
Further research led me to find these mitigations to that problem : https://tools.ietf.org/id/draft-scharf-tcpm-reordering-00.html#rfc.section.3.2
I'll have to look into SCTP and see if it is something I'd like to implement, however. At first glance, that seems rather complicated and might not be worth the time investment.
I have built a distributed HTTP scraper solution that uses different "exit addresses" addresses by design in order to balance the network load.
The solution supports IPv4, IPv6 and HTTP proxy to route the traffic.
Each processor was responsible to define the most efficient route to balance the traffic and it was temporarily implemented manually for prototyping. Currently, the solution grows and with the number of processors as the complexity of the load balancing task get higher, that's why I need a way to create a component dedicated to it.
I did some rather extensive research, but seem to have failed in finding a solution for load balancing traffic between IPv6, IPv4 (thousands of local addresses) and public HTTP proxies. The solution needs to support weights, app-level response checks and cool-down periods.
Does anyone know a solution that already solves this problem? Before I start developing a custom one.
Thanks for your help!
If you search for load balancing proxy you'll discover the Cache Array Routing Protocol (CARP). This CARP might not be what you're searching for and there exists servers only for the proxy-cache what I never knew till now.
Nevertheless those servers have own load balancers too, and perhaps that's a detail where it's worth it to search more.
I found a presentation mentioning CARP as outstanding solution too: https://cs.nyu.edu/artg/internet/Spring2004/lectures/lec_8b.pdf
Example: for proxy-arrays in Netra Proxy Cache Server: https://docs.oracle.com/cd/E19957-01/805-3512-10/6j3bg665f/index.html
Also there exist several concepts for load-balancing (https://link.springer.com/article/10.1023/A:1020943021842):
The three proposed methods can broadly be divided into centralized and decentralized
approaches. The centralized history (CH) method makes use of the transfer rate of each
request to decide which proxy can provide the fastest turnaround time for the next job.
The route transfer pattern (RTP) method learns from the past history to build a virtual
map of traffic flow conditions of the major routes on the Internet at different times of the
day. The map information is then used to predict the best path for a request at a particular time of the day. The two methods require a central executive to collate information
and route requests to proxies. Experimental results show that self-organization can be
achieved (Tsui et al., 2001). The drawback of the centralized approach is that a bottleneck and a single point of failure is created by the central executive. The decentralized
approach—the decentralized history (DH) method—attempts to overcome this problem
by removing the central executive and put a decision maker in every proxy (Kaiser et al.,
2000b) regarding whether it should fetch a requested object or forward the request to another
proxy.
As you use public proxy-servers probably you won't use decentralized history (DH) but centralized history (CH) OR the route transfer pattern (RTP).
Perhaps it would be even useful to replace your own solution completely, i.e. by this: https://github.blog/2018-08-08-glb-director-open-source-load-balancer/. I've no reason for this special example, it's just random by search results I found.
As I'm not working with proxy-servers this post is just a collection of findings, but perhaps there is a usable detail for you. If not, don't mind - probably you know most or all already and it's never adding anything new for you. Also I never mention any concrete solution.
Have you checked this project? https://Traefik.io which supports http/2 and tcp load balancing. The project is open source and available on github. It is build using Go. I'm using it now as my reverse proxy with load balancing for almost everything.
I also wrote a small blog post on docker and Go where I showcase the usage of Traefik. That also might help you in your search. https://marcofranssen.nl/docker-tips-and-tricks-for-your-go-projects/
In the traefik code base you might find your answer, or you might decide to utilize traefik to achieve your goal instead of home grown solution.
See here for a nice explanation on the soon to be arriving Traefik 2.0 with TCP support.
https://blog.containo.us/back-to-traefik-2-0-2f9aa17be305
UPDATE: See the answer I've provided below for the solution I eventually got set up on AWS.
I'm currently experimenting with methods to implement a global load-balancing layer for my app servers on Digital Ocean and there's a few pieces I've yet to put together.
The Goal
Offer highly-available service to my users by routing all connections to the closest 'cluster' of servers in SFO, NYC, LON, and eventually Singapore.
Additionally, I would eventually like to automate the maintenance of this by writing a daemon that can monitor, scale, and heal any of the servers on the system. Or I'll combine various services to achieve the same automation goals. First I need to figure out how to do it manually.
The Stack
Ubuntu 14.04
Nginx 1.4.6
node.js
MongoDB from Compose.io (formerly MongoHQ)
Global Domain Breakdown
Once I rig everything up, my domain would look something like this:
**GLOBAL**
global-balancing-1.myapp.com
global-balancing-2.myapp.com
global-balancing-3.myapp.com
**NYC**
nyc-load-balancing-1.myapp.com
nyc-load-balancing-2.myapp.com
nyc-load-balancing-3.myapp.com
nyc-app-1.myapp.com
nyc-app-2.myapp.com
nyc-app-3.myapp.com
nyc-api-1.myapp.com
nyc-api-2.myapp.com
nyc-api-3.myapp.com
**SFO**
sfo-load-balancing-1.myapp.com
sfo-load-balancing-2.myapp.com
sfo-load-balancing-3.myapp.com
sfo-app-1.myapp.com
sfo-app-2.myapp.com
sfo-app-3.myapp.com
sfo-api-1.myapp.com
sfo-api-2.myapp.com
sfo-api-3.myapp.com
**LON**
lon-load-balancing-1.myapp.com
lon-load-balancing-2.myapp.com
lon-load-balancing-3.myapp.com
lon-app-1.myapp.com
lon-app-2.myapp.com
lon-app-3.myapp.com
lon-api-1.myapp.com
lon-api-2.myapp.com
lon-api-3.myapp.com
And then if there's any strain on any given layer, in any given region, I can just spin up a new droplet to help out: nyc-app-4.myapp.com, lon-load-balancing-5.myapp.com, etc…
Current Working Methodology
A (minimum) trio of global-balancing servers receive all traffic.
These servers are "DNS Round-Robin" balanced as illustrated in this
(frankly confusing) article: How To Configure DNS Round-Robin Load
Balancing.
Using the Nginx GeoIP
Module and
MaxMind GeoIP Data
the origin of any given request is determined down to the
$geoip_city_continent_code.
The global-balancing layer then routes the request to the least
connected server on the load-balancing layer of the appropriate
cluster: nyc-load-balancing-1, sfo-load-balancing-3,
lon-load-balancing-2, etc.. This layer is also a (minimum) trio of
droplets.
The regional load-balancing layer then routes the request to the
least connected server in the app or api layer: nyc-app-2,
sfo-api-1, lon-api-3, etc…
The details of the Nginx kung fu can be found in this tutorial:
Villiage Idiot: Setting up Nginx with GSLB/Reverse Proxy on
AWS. More general info about Nginx load-balancing is available
here
and
here.
Questions
Where do I put the global-balancing servers?
It strikes me as odd that I would put them either all in one place, or spread that layer out around the globe either. Say, for instance, I put them all in NYC. Then someone from France hits my domain. The request would go from France, to NYC, and then be routed back to LON. Or if I put one of each in SFO, NYC, and LON then isn't it still possible that a user from Toronto (Parkdale, represent) could send a request that ends up going to LON only to be routed back to NYC?
Do subsequent requests get routed to the same IP?
As in, if a user from Toronto sends a request that the global-balancing layer determines should be going to NYC, does the next request from that origin go directly to NYC, or is it still luck of the draw that it will hit the nearest global-balancing server (NYC in this case).
What about sessions?
I've configured Nginx to use the ip_hash; directive so it will direct the user to the same app or api endpoint (a node process, in my case) but how will global balancing affect this, if at all?
Any DNS Examples?
I'm not exactly a DNS expert (I'm currently trying to figure out why my CNAME records aren't resolving) but I'm a quick study when provided with a solid example. Has anyone gone through this process before and can provide a sample of what the DNS records look like for a successful setup?
What about SSL/TLS?
Would I need a certificate for every server, or just for the three global-balancing servers since that's the only public-facing gateway?
If you read this whole thing then reward yourself with a cupcake. Thanks in advance for any help.
The Goal: Offer highly-available service to my users by routing all connections to the closest 'cluster' of servers in SFO, NYC, LON, and eventually Singapore.
The global-balancing layer then routes the request to theleast
connected server...
If I'm reading your configuration correctly, you're actually proxying from your global balancers to the balancers at each region. This does not meet your goal of routing users to the nearest region.
There are three ways that I know of to get what you're looking for:
30x Redirect Your global balancers receive the HTTP request and then redirect it to a server group in or near the region it thinks the request is coming from, based on IP address. This sounds like what you were trying to set up. This method has side effects for some applications, and also increases the time it takes for a user to get data since you're adding a ton of overhead. This only makes sense if the resources you're redirecting to are very large, and the local regional cluster will be able to serve much more efficiently.
Anycast (taking advantage of BGP routing) This is what the big players like Akamai use for their CDN. Basically, there are multiple servers out on the internet with the exact same routable IP address. Suppose I have servers in several regions, and they have the IP address of 192.0.2.1. If I'm in the US and try to connect to 192.0.2.1, and someone is in Europe that tries to connect to 192.0.2.1, it's likely that we'll be routed to the nearest server. This uses the internet's own routing to find the best path (based on network conditions) for the traffic. Unfortunately, you can't just use this method. You need your own AS number, and physical hardware. If you find a VPS provider that lets you have a chunk of their Anycast block, let me know!
Geo-DNS There are some DNS providers that provide a service often marketed as "Geo-DNS". They have a bunch of DNS servers hosted on anycast addresses which can route traffic to your nearest servers. If a client queries a European DNS server, it should return the address for your European region servers, vs. some in other regions. There are many variations on the Geo DNS services. Others simply maintain a geo-IP database and return the server for the region they think is closer, just like the redirect method but for DNS before the HTTP request is ever made. This is usually the good option, for price and ease of use.
Do subsequent requests get routed to the same IP?
Many load balancers have a "stickiness" option that says requests from the same network address should be routed to the same end server (provided that end server is still up and running).
What about sessions?
This is exactly why you would want that stickiness. When it comes to session data, you are going to have to find a way to keep all your servers up-to-date. Realistically, this isn't always guaranteed. How you handle it depends on your application. Can you keep a Redis instance or whatever out there for all your servers to reliably hit from around the world? Do you really need that session data in every region? Or can you have your main application servers dealing with session data in one location?
Any DNS Examples?
Post separate questions for these. Everyone's "successful setup" looks differently.
What about SSL/TLS?
If you're proxying data, only your global balancers need to handle HTTPS. If you're redirecting, then all the servers need to handle it.
A Working Solution
I've had a wild ride over the past few months figuring out the whole Global-HA setup. Tonnes of fun and I've finally settled with a rig that works very well, and is nothing like the one outlined in the above question.
I still plan on writing this up in tutorial form, but time is scarce as I head into the final sprint to get my app launched early next year, so here's a quick outline of the working rig I ended up with.
Overview
I ended up moving my entire deployment to AWS. I love Digital Ocean, but the frank reality is that AWS is light years ahead of them (and everyone, really) when it comes to the services offered under one roof. My monthly expenses went up slightly, but once I was done tweaking and streamlining I ended up with a solution that costs about $75/month per region for the most basic deployment (2 instances behind an ELB). And a new region can be spun up and deployed within about 30 minutes.
Global Balancing
I quickly found out (thanks to #Brad's answer above) that trying to spin up my own global balancing DNS layer is insane. It was a hell of a lot of fun figuring out how a layer like this works, but short of getting on a plane and scraping my knuckles installing millions of dollars worth of equipment around the world, it was not going to be possible to roll my own.
When I finally figured out what I was looking for, I found my new best friend: AWS Route 53. It offers a robust DNS network with about 50-odd nodes globally and the ability to do some really cool routing tricks like location-based routing, latency-based routing (which is kinda awesome), and AWS Alias records that 'automagically' route traffic to other AWS Services you'll be using (Like ELB for load balancing).
I ended up using latency-based routing that directs the global traffic to the closest regional Elastic Load Balancer, which has an Auto-Scaling Group attached to it in any given region.
I'll leave it up to you to do your homework on the other providers: www.f5.com, www.dyn.com, www.akamai.com, www.dnsmadeeasy.com. Depending on your needs, there may be a better solution for you, but this works very well for me.
Content Delivery Network
Route 53 integrates with AWS Cloudfront very nicely. I setup an S3 bucket that I'm using to store all the static media files that my users will upload, and I've configured a Cloudfront distribution to source from my media.myapp.com S3 bucket. There are other CDN providers, so do your shopping. But Cloudfront gets pretty good reviews and it's a snap to setup.
Load Balancing & SSL Termination
I'm currently using AWS Elastic Load Balancer to balance the load across my application instances, which live in an Auto-Scaling Group. The request is first received by ELB, at which point SSL is terminated and the request is passed through to an instance in the Auto-Scaling Group.
NOTE: One giant caveat for ELB is that, somewhat ironically, it doesn't handle massive spikes very well. It can take up to 15 minutes for an ELB to trigger a scale-up event for itself, creating 500/timeouts in the meantime. A steady, constant increase in traffic is supposedly handled quite well, but if you get hit with a spike it can fail you. If you know you're going to get hit, you can 'call ahead' and AWS will warm up your ELB for you, which is pretty ridiculous and anti-pattern to the essence of AWS, but I imaging they're either working on it, or ignoring it because it's not really that big of a problem. You can always spin up your own HAProxy or Nginx load-balancing layer if ELB doesn't work for you.
Auto-Scaling Group
Each region has an ASG which is programmed to scale when the load passes a certain metric:
IF CPU > 90% FOR 5 MINUTES: SCALEUP
IF CPU < 70% FOR 5 MINUTES: SCALEDN
I haven't yet put the ELB/ASG combo through its paces. That's a little way down my To-Do list, but I do know that there are many others using this setup and it doesn't seem to have any major performance issues.
The config for an Auto-Scaling Group is a little convoluted in my opinion. It's actually a three-step process:
Create an AMI configured to your liking.
Create a Launch Configuration that uses the AMI you've created.
Create an Auto-Scaling Group that uses the Launch Configuration you've created to determine what AMI and instance type to launch for any given SCALEUP event.
To handle config and app deployment when any instance launches, you use the "User Data" field to input a script that will run once any given instance launches. This is possibly the worst nomenclature in the history of time. How "User Data" describes a startup script only the author knows. Anyhow, that's where you stick the script that handles all your apt-gets, mkdirs, git clones, etc.
Instances & Internal Balancing
I've also added an additional 'internal balancing layer' using Nginx that allows me to 'flat-pack' all my Node.js apps (app.myapp.com, api.myapp.com, mobile.myapp.com, www.myapp.com, etc.myapp.com) on every instance. When an instance receives a request passed to it from ELB, Nginx handles routing the request to the correct Node.js port for any given application. Sort of like a poor-mans containerization. This has the added benefit that any time one of my apps needs to talk to the other (like when app. needs to send a request to api.) it's done via localhost:XXXX rather than having to go out across the AWS network, or the internet itself.
This setup also maximizes usage of my resources by eliminating any idle infrastructure if the app layer it hosts happens to be receiving light traffic. It also obviates the need to have and ELB/ASG combo for every app, saving more cash.
There's no gotchas or caveats that I've run into using this sort of setup, but there is one work-around that needs to be in place with regard to health-checking (see below).
There's also a nice benefit in that all instances have an IAM role which means that your AWS creds are 'baked in' to each instance upon birth and accessible via your ENV vars. And AWS 'automagically' rotates your creds for you. Very secure, very cool.
Health Checks
If you go the route of the above setup, flat-packing all your apps on one box and running an internal load-balancer, then you need to create a little utility to handle the ELB Health Checks. What I did was create an additional app called ping.myapp.com. And then I configured my ELB Health Checks to send any health checks to the port that my ping app is running on, like so:
Ping Protocol: HTTP
Ping Port: XXXX
Ping Path: /ping
This sends all health checks to my little ping helper, which in turn hits localhost:XXXX/ping on all the apps residing on the instance. If they all return a 200 response, my ping app then returns a 200 response to the ELB health check and the instances gets to live for another 30 seconds.
NOTE: Do not use Auto-Scaling Health Checks if you're using an ELB. Use the ELB health checks. It's kinda confusing, I thought they were the same thing, they're not. You have the option to enable one or the other. Go with ELB.
The Data Layer
One thing that is glaringly absent from my setup is the data layer. I use Compose.io as my managed data-layer provider and I deploy on AWS so I get very low latency between my app layers and my data layer. I've done some prelim investigation on how I would roll my data layer out globally and found that it's very complex — and very expensive — so I've kicked it down my list as a problem that doesn't yet need to be solved. Worst case is that I'll be running my data layer in US-East only and beefing up the hardware. This isn't the worst thing in the world since my API is strictly JSON data on the wire so the average response is relatively tiny. But I can see this becoming a bottleneck at very large, global scale — if I ever get there. If anyone has any input on this layer I'd love to hear what you have to say.
Ta-Da!
Global High Availability On A Beer Budget. Only took me 6 months to figure it out.
Love to hear any input or ideas from anyone that happens to read this.
You can use Anycast for your webservice for free if using Cloudflare free plan.
Digital Ocean now supports Load Balancing of servers itself. It is extremely easy to set up and works great! Saves you having to add in unnecessary components such as nginx (if you only want to use for load balancing).
We were having issues using SSL file uploads with nginx on a digital ocean server, however since the Digital Ocean update, we have removed nginx and now use Digital Ocean's load balancing feature and it works just as we need it to!
I am developing a browser extension. The extension works on external websites we have no control over.
I would like to be able to test the extension. One of the major problems I'm facing is displaying a website 'as-is' locally.
Is it possible to display a website 'as-is' locally?
I want to be able to serve the website exactly as-is locally for testing. This means I want to simulate the exact same HTTP data, including iframe ads, etc.
Is there an easy way to do this?
More info:
I'd like my system to act as closely to the remote website as possible. I'd like to run command fetch for example which would allow me to go to the site in my browser (without the internet on) and get the exact same thing I would otherwise (including information that is not from a single domain, google ads, etc).
I don't mind using a virtual machine if this helps.
I figured this was quite a useful thing in testing. Especially when I have a bug I need to reliably reproduce in sites that have many random factors (what ads show, etc).
As was already mentioned, caching proxies should do the trick for you (BTW, this is the simplest solution). There are quite a lot of different implementations, so you just need to spend some time selecting a proper one (according to my experience squid is a good solution). Anyway, I would like to highlight two other interesting options:
Option 1: Betamax
Betamax is a tool for mocking external HTTP resources such as web services and REST APIs in your tests. The project was inspired by the VCR library for Ruby. Betamax aims to solve these problems by intercepting HTTP connections initiated by your application and replaying previously recorded responses.
Betamax comes in two flavors. The first is an HTTP and HTTPS proxy that can intercept traffic made in any way that respects Java’s http.proxyHost and http.proxyPort system properties. The second is a simple wrapper for Apache HttpClient.
BTW, Betamax has a very interesting feature for you:
Betamax is a testing tool and not a spec-compliant HTTP proxy. It ignores any and all headers that would normally be used to prevent a proxy caching or storing HTTP traffic.
Option 2: Wireshark and replay proxy
Grab all traffic you are interested in using Wireshark and replay it. This I would say it is not that hard to implement required replaying tool, but you can use available solution called replayproxy
Replayproxy parses HTTP streams from .pcap files
opens a TCP socket on port 3128 and listens as a HTTP proxy using the extracted HTTP responses as a cache while refusing all requests for unknown URLs.
Such approach provide you with the full control and bit-to-bit precise simulation.
I don't know if there is an easy way, but there is a way.
You can set up a local webserver, something like IIS, Apache, or minihttpd.
Then you can grab the website contents using wget. (It has an option for mirroring). And many browsers have an option for "save whole web page" that will grab everything, like images.
Ads will most likely come from remote sites, so you may have to manually edit those lines in the HTML to either not reference the actual ad-servers, or set up a mock ad yourself (like a banner image).
Then you can navigate your browser to http://localhost to visit your local website, assuming port 80 which is the default.
Hope this helps!
I assume you want to serve a remote site that's not under your control. In that case you can use a proxy server and have that server cache every response aggressively. However, this has it's limits. First of all you will have to visit every site you intend to use through this proxy (with a browser for example), second you will not be able to emulate form processing.
Alternatively you could use a spider to download all content of a certain website. Depending on the spider software, it may even be able to download JavaScript-built links. You then can use a webserver to serve that content.
This service http://www.json-gen.com provides mock for html, json and xml via rest. By this way, you can test your frontend separately from backend.
I have a slowly evolving dynamic website served from J2EE. The response time and load capacity of the server are inadequate for client needs. Moreover, ad hoc requests can unexpectedly affect other services running on the same application server/database. I know the reasons and can't address them in the short term. I understand HTTP caching hints (expiry, etags....) and for the purpose of this question, please assume that I have maxed out the opportunities to reduce load.
I am thinking of doing a brute force traversal of all URLs in the system to prime a cache and then copying the cache contents to geodispersed cache servers near the clients. I'm thinking of Squid or Apache HTTPD mod_disk_cache. I want to prime one copy and (manually) replicate the cache contents. I don't need a federation or intelligence amongst the slaves. When the data changes, invalidating the cache, I will refresh my master cache and update the slave versions, probably once a night.
Has anyone done this? Is it a good idea? Are there other technologies that I should investigate? I can program this, but I would prefer a configuration of open source technologies solution
Thanks
I've used Squid before to reduce load on dynamically-created RSS feeds, and it worked quite well. It just takes some careful configuration and tuning to get it working the way you want.
Using a primed cache server is an excellent idea (I've done the same thing using wget and Squid). However, it is probably unnecessary in this scenario.
It sounds like your data is fairly static and the problem is server load, not network bandwidth. Generally, the problem exists in one of two areas:
Database query load on your DB server.
Business logic load on your web/application server.
Here is a JSP-specific overview of caching options.
I have seen huge performance increases by simply caching query results. Even adding a cache with a duration of 60 seconds can dramatically reduce load on a database server. JSP has several options for in-memory cache.
Another area available to you is output caching. This means that the content of a page is created once, but the output is used multiple times. This reduces the CPU load of a web server dramatically.
My experience is with ASP, but the exact same mechanisms are available on JSP pages. In my experience, with even a small amount of caching you can expect a 5-10x increase in max requests per sec.
I would use tiered caching here; deploy Squid as a reverse proxy server in front of your app server as you suggest, but then deploy a Squid at each client site that points to your origin cache.
If geographic latency isn't a big deal, then you can probably get away with just priming the origin cache like you were planning to do and then letting the remote caches prime themselves off that one based on client requests. In other words, just deploying caches out at the clients might be all you need to do beyond priming the origin cache.