I'm trying to implement Vertical autoscaling by using heat template of autoscaling by increasing the load on instance and gathering the metrics via ceilometer.
My requirement is to change the flavor of the instance from m1.tiny to m1.small if load increases above 80% in CPU via autoscaling method of heat template.
can someone please help me out in implementing the template of vertical scaling.
Im a newbie to openstack.
Related
We are currently looking at setting up multiple Airflow stacks on the same GKE cluster but in different namespaces (to save on costs) to run DBT jobs.
Each Airflow stack within its namespace would use RBAC authentication to auth end users to run or observe jobs.
I understand this isn't a typical use case but the other alternative would be to have a separate Cloud Composer for each service line which would be quite costly.
Any help is much appreciated :)
You can run multiple Airflow DAGs on the same GKE cluster by using KubernetesPodOperator in the way given here 1. Affinity is used to constrain which nodes your pod is eligible to be scheduled on, based on labels. The code is inside affinity{...} in the same link. Also, it is mentioned that you can use a custom namespace.
The system I'm working on has multiple environments, each running in separate Azure regions. Our CosmosDB is replicated to these regions and multi-region writes are enabled. We're using the default consistency model (Session).
We have azure functions that use the CosmosDb trigger deployed in all three regions. Currently these use the same lease prefix which means that only one function processes changes at any given time. I know that we can set each
region to have different lease prefixes to enable concurrent processing but I'd like to solidify my understanding before taking this step.
My question is around the behaviour of the change feed with regards to replication in this scenario? According to this link https://github.com/MicrosoftDocs/azure-docs/issues/42248#issuecomment-552207409 data is first converged on the primary region and then the change feed is updated.
Other resources I've read seem to suggest that each region has it's own change feed which will update upon replication. Also, the previous link recommends only running a change feed processor in the primary region in multi-master.
In an ideal world, I'd like change feed processors in each region to handle local writes quickly. These functions will make updates to CosmosDB and I also want to avoid issues with replication. My question is - what is the actual behavior in a multi master configuration (and by extension the correct architecture)?. Is it "safe" to use per-region change feed processors, or should we use a single processor in the primary region?
You cannot have per-region Change Feed Processor's that only process the local changes, because the Change Feed in each region contains the local writes plus the replicated writes from each other region.
Technically you can use a single Change Feed Processor deployment connecting to one of the regions to process events on all the regions.
I am trying to implement dynamodb autoscaling using terraform but I am having a bit of difficulty in understanding the difference between aws_appautoscaling_target and aws_appautoscaling_policy.
Do we need both specified for the autoscaling group? Can some one kidly explain what each is meant for?
Thanks a ton!!
The aws_appautoscaling_target ties your policy to the DynamoDB table. You can define a policy once and use it over and over (i.e. build standard set of scaling policies for your organization to use), the target allows you to bind a policy to a resource.
An auto scaling group doesn't have to have either a target or a resource. An ASG can scale EC2 instances in/out based other triggers such as instance health (defined by EC2 health checks or LB health checks) or desired capacity. This allows a load balanced application to replace bad instances when they are unable to respond to instance traffic and also recover from failures to keep your cluster at the right size. You could add additional scaling policies to better react to demand. For example, your cluster has 2 instances but they're at max capacity, a scaling policy can watch those instances and add more when needed and then remove them when demand falls.
I have the following setup:
K8 Ingress --> K8 Service --> K8 Pods x n
(GCP LB for SSL) (NodePort) (containing the application)
I can setup a stackdriver HTTPS uptime alert to notify when the site is down, but this will only alert when all n pods are out of action.
Is it possible to create a stackdriver alert that will alert when any of the n application pods are down?
firstly, you should consider putting correct scaling in place so that you may altogether avoid the need to alert when n application pods are down. Additionally, it's best to alert on what symptoms your users experience (increased latency or errors) rather than on the underlying infrastructure. Since it might be okay if n pods are down temporarily, as long as user requests still get served.
That being said, if you're running in GKE you can alert on container uptime. From your question I'm assuming that's not the case, so you could either:
* Log your own uptime checks, create a logs metric and alert when it's under a certain threshold.
* Similarly, create a custom uptime metric and alert on that.
Note that I would avoid creating a custom metric and using metric absence as the alerting policy condition, so I didn't list that as an option.
HTH and thanks for using Stackdriver.
I want to calculate and show node specific cpu usage in percent in my own web application using Kubernetes API.
I need the same information as Kube UI and Cadvisor displays but I want to use the Kubernetes API.
I have found some cpu metrics under node-ip:10255/stats which contains timestamp, cpu usage: total, user and system in big weird numbers which I do not understand. Also the CPU-Limit is reported as 1024.
How does Kube UI calculate cpu usage and is it possible to do the same via the API?
If you use Kubernetes v1.2, there is a new, cleaner metrics summary API. From the release note:
Kubelet exposes a new Alpha metrics API - /stats/summary in a user friendly format with reduced system overhead.
You can access the endpoint through <node-ip>:10255/stats/summary and detailed API objects is here.
So the way CPU usage metrics are usually collected in Kubernetes is using cAdvisor https://github.com/google/cadvisor which looks at the cgroups to get metircs, so mostly CPU and some memory metrics. cAdvisor then can put its data into a metrics DB like heapster, influxDB or prometheus. Kubernetes does not directly deal with metrics, so therefore does not expose it through the API, however you can use the metrics DB instead. Additionally you can use an additional container in your pod to collect metrics and place that into your metrics DB. Additionally, you can get resource quotas through the API, but not usage. This proposal may be of some interest for you as well https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/proposals/metrics-plumbing.md.