/* I'm new here and I've done days of research. */
What is the best practice to share files with multiple autoscaling Google Compute Engine VMs?
I'm planning to set up an Instance group of VMs with NGINX for serving static files for multiple domains. These VMs would autoscale to n (multiply itself) and the files would change a lot. I need a storage for the files these VMs will serve.
So far I've found these solutions:
1) Persistent disk + rsync -> This should have the smallest latency, but when I reach GBs of files, autoscaled VMs would be syncing for a long time after they spawn, thus throwing 404s.
2) Master VM without web server + nfs/smb -> Small latency, but no redundancy.
3) Cloud Storage + FUSE -> Big latency, great redundancy, no SLA.
4) Shared Persistent disk -> Small latency, but read-only.
5) NGINX + Cloud SQL/Bigtable/Spanner/Datastore -> Mehish latency and I don't feel well about connecting webserver to a DB.
Are there any other better solutions?
Thanks
EDIT: The static files are multiple index.html files -> homepages of multiple domains.
There is also:
6) Firebase Hosting - https://firebase.google.com/docs/hosting
or
7) another way - I would personally go with Cloud Storage but not FUSE. Or at least not for serving. You can still use FUSE for writing to Bucket(s). Of course the best way would be to just use Cloud Storage API from within the application.
For serving files:
I would create a Load balancer with a Backend bucket the same bucket where the application writes. Also be careful to enable Cloud CDN on that Load balancer.
More details at:
Load balancer - https://cloud.google.com/load-balancing/
Cloud CDN - https://cloud.google.com/cdn/
or just be bold and create now a load balancer at https://console.cloud.google.com/networking/loadbalancing/loadBalancers/list?project=
For serving static files the best is definitely to use a load balancer and backend buckets with Cloud CDN enabled.
The load balancer have rules to forward traffic. For example it can intercept all requests by hosts, subdomain, or path.
*.mysite1.com => bucket1
demo.mysite2.net => bucket1
test.mysite3.com => bucket2
Because files are served with Cloud CDN, the latency becomes minimal.
In order to write your files to a bucket you could use FUSE, or create files locally and use gsutil cp.
Persistent disks can only be shared within several Compute Engines in read mode. If you need write mode it won't work.
The last option Cloud SQL + Nginx is actually pretty good. Cloud SQL is way more fast than Mysql servers. And connection between cloud sql and gce is easy and reliable.
But it is more a matter of preferences here... if you feel comfortable writing the scripts that will read and write to it.
Related
I mounted the bucket to compute the engine VM via gcsfuse and then ran the Nginx program in the VM. Normally the Nginx program writes the cache to the directory, i.e. the mounted bucket, using the proxy_cache directive.
However, I have a problem that the Nginx can create cache files in the filesystem under the bucket directory, but the size of the cache file is always 0B. client requests keep getting "Miss cache" status.
So it seems that after mounting with gcsfuse, the Nginx application can only create cache files, but cannot write to them.
My VM environment is.
Machine type: n2-standard-2
CPU platform: Intel Cascade Lake
Architecture: x86/64
System: ubuntu-1804
In addition, gcsfuse has specified a service account with owner privileges via the --key-file directive, and the Nginx program has been run with the root user.
For example, the following debug log is an empty file (8/2b/0a7ed71cddee71b9276a0d72a030f2b8) created in the bucket after a client request and not written to the cache. What could be the cause of this possibility?
https://storage.googleapis.com/cloud_cdn_cache_bucket/debug_log.txt
Here is the debug log obtained by the command --debug_fuse --debug_fs --debug_gcs --debug_http -foreground.
You can't use Cloud Storage Fuse for cache.
There is a technical reason to that: GCSFuse is a Cloud Storage API call wrapper that transforms system calls to API calls. However, all the system calls aren't supported, especially those related to "database" format with stream write, seek, and rewrite partial content of the file. All common operations for a database (or cache) but not compliant with Cloud Storage: you can only write/read/delete a file. Update/partial write aren't supported. It's not a file system!
In addition, because you now know that GCSFuse is a wrapper (of system calls to API calls), you should feel that using that file system type is not a good idea: the latency is terrible!! It's API calls! Absolutely not recommended for cache and low latency operations.
The best solution is to use a local file system dedicated to cache. But if you scale out (more servers in parallel) you could have issues (cache is not shared between instances):
Use sticky session mechanism to always route the same user session on the same NGINX server and therefore always use the same cache context
Use the Filestore service that offers a NFS (Network File Share) system to mount the same storage space on different servers, with an interesting latency (not as good as a local file system)
You talked about a key file in your question also. I recommend you to avoid as much as you can to use the service account key files; especially if your app runs on Google Cloud. Let me know what's your key file usage in detail if you want more guidance
I want to share a persistence volumen from my Google Container Engine instance with my FTP Server instance in other to access to the file that upload to FTP server.
If I've use a Computer Engine Disk I can't do that because the disk became readonly.
I see that I can use a Cloud Storage but I don't know If I can used as persistence volumen in Container Engine.
Anybody can help me?
Regards
First of all you need a FTP server container image for that. I'm not sure if anything off-the-shelf exists, you may need to build it yourself.
Then you can simply use GCE Persistent Disks as you said (however I'm not sure what you mean by "because the disk became readonly", you're supposed to explain your problem in detail at StackOverflow, people here simply can't guess it correctly all the time).
So you can follow this tutorial to learn about how to use GCE Persistent Disks on GKE clusters: https://cloud.google.com/kubernetes-engine/docs/tutorials/persistent-disk then you can modify the mysql part with your FTP server docker image.
Google Cloud Storage won't give you the filesystem semantics (such as permission bits, user/group on Linux) so it probably won't suit for FTP. Also you can't mount GCS to your compute instances.
Consider the following cluster running on Google Container Engine:
tier: g1-small
cpu cores: 2
memory: 1,7GB per cpu core (3,4GB total in this case)
autoscaling: enabled, min=2, max=5
On this cluster, I have the following Deployments running via Kubernetes:
Load Balancer using NGINX
Web App using Node.js (communicating with WordPress via REST calls)
example.com
CMS using WordPress on Apache (wp.example.com)
wp.example.com
For clarity, every request goes through the NGINX Load Balancer first, then, depending on the subdomain, to either the CMS or the Web App.
I'd like to have more control over how much resources each Deployment consumes in order to consume resources more efficiently by applying Kubernetes Limit Ranges to my Pods/Containers resources.
Limits can be set on CPU and Memory. These are well explained in the docs. So far, so good.
The problem I'm having is to figure out what limits to apply per Deployment.
Example
The WordPress Deployment contains two containers in the deployment.yaml file. One for the WordPress image itself, one for the Cloud SQL Proxy Container that is needed for WordPress to connect to my Cloud SQL Database. How would I know what each container needs with respect to CPU/Memory resources?
Furthermore, considering that all HTTP/HTTPS traffic hits my NGINX Load Balancer first, an educated guess would be apply relatively more resources to the NGINX Deployment than to my CMS and Web App Deployment, right?!.
So is there a way to better estimate how many resources each Deployment would need?
Any help is greatly appreciated!
k8s' default value to pods is 100m CPU request and no CPU limit, and no memory request/limit. If you don't set limitation pods/containers will consume as much as it need. Which is pretty convenient since usually you don't specify limitation one by one.
Nginx as a load balancer is pretty light-weighted. So it's hard to say which one needs more resources. I would follow the default at the beginning then use kubectl top pod to check CPU/memory pressure for tuning reference.
Idea
Gradually use a few small-scale dedicated servers in combination with an expensive cloud platform, where - on little traffic - the dedicated servers should first filled up before the cloud kicks in. Hedging against occasional traffic spikes.
nginx
Is there an easy way (without nginx plus) to achieve a "waterfall like" set-up, where small servers should first be served up to a maximum number of concurrent connections, or better, current bandwidth before the cloud platform sees any traffic?
Nginx Config, Libraries, Tools?
Thanks
You will use nginx upstream module.
If you want total control, set your cloud servers with backup parameter, so that they won't be used until your primary servers fail. Then use custom monitoring scripts to determine when those could servers should kick-in, change nginx config and remove the backup keyword from them. Also monitor conditions when you want to stop using the cloud servers and alter the nginx config.
More simple solution (but without fine tuning like avoiding spikes) is to use the max_conns=number parameter. Nginx should start to use the backup server if all other already have max number of connections (I didn't test it).
NOTE: max_conns parameter was only available in paid nginx between v1.5.9 and v1.11.5, so the only solution with these versions is own monitoring + reloading of nginx config when needed to change the upstream servers. Thanks Mickaël Le Baillif's comment to point out this parameter is now available to all.
I am building a website that will be handling many persistent connections - either through long polling or WebSockets (if I get a host that has IIS8). My question is, are either of these viable on a shared host? Or do I need a true dedicated server?
I understand that long-polling will require me to do some custom configuration on the maximum number of requests handled by the server.. so it seems like this is not an option on a shared host. What about WebSockets? And, will either of these work with "virtual" or "cloud" hosting?
If you want or need to use a shared host then you could offload the handling of the persistent connections to a realtime web hosted service. I work for one such company, Pusher, but there are a few others out there too (all linked to from the guide above).
These techniques will work with any kind of hosting, but you will exceed the resource limit of shared hosting quite fast. So I'd recommend starting with a VPS at least.