I am wondering what is a stack in openslack.
It seems to be components (network, containers and stuff)
Can i mix thoses items with docker containers for example ?
Thanks
OpenStack version of AWS cloudformation (infrastructure as a code) is performed trough HEAT (OpenStack Component). A "stack" is a set of resources that can be deployed in the cloud. What are those resources ?: Everything that can be created withing the cloud environment:
- A network port...
- A set of instances (virtual machines).
- A load balancer.
- A storage device (block or object).
Basically, you use a template (or a set of templates) in order to create a complete infrastructure inside the cloud (database servers, web servers, a load balancer for the web servers, maybe the storage too... etc.).
The template uses a common language to define those components and how are they created in the cloud.
Note that, many "AWS Cloudformation" objects can be defined in OpenStack HEAT templates, but, HEAT also has it's very own set of objects definitions for OpenStack.
At the end of my site you will find a sample HEAT template for a stack that include servers, loadbalancer and autoscalling:
https://tigerlinux.github.io/recipes/openstack/index.html
Also, more examples directly from the OpenStack community:
https://git.openstack.org/cgit/openstack/heat-templates/tree/hot
Related
I'm learning Kubernetes, and trying to setup a cluster that could handle a single Wordpress site with high traffic. From reading multiple examples online from both Google Cloud and Kubernetes.io - they all set the "accessMode" - "readWriteOnce" when creating the PVCs.
Does this mean if I scaled the Wordpress Deployment to use multiple replicas, they all use the same single PVC to store persistent data - read/write data. (Just like they use the single DB instance?)
The google example here only uses a single-replica, single-db instance - https://cloud.google.com/kubernetes-engine/docs/tutorials/persistent-disk
My question is how do you handle persistent storage on a multiple-replica instance?
ReadWriteOnce means all replicas will use the same volume and therefore they will all run on one node. This can be suboptimal.
You can set up ReadWriteMany volume (NFS, GlusterFS, CephFS and others) storage class that will allow multiple nodes to mount one volume.
Alternatively you can run your application as StatefulSet with volumeClaimTemplate which ensures that each replica will mount its own ReadWriteOnce volume.
If on AWS (and therefore blocked by the limitation of EBS volumes only mountable on a single instance at a time), another option here is setting up the Pod Affinity to schedule on the same node. Not ideal from an HA standpoint, but it is an option.
If after setting that up you start running into any wonky issues (e.g. unable to login to the admin, redirect loops, disappearing media), I wrote a guide on some of the more common issues people run into when running Wordpress on Kubernetes, might be worth having a look!
I am trying to understand clustering concept of WSO2. My basic understanding of cluster is that there are 2 or more server with same function using VIP or load balance in front. So I would like to know which of the WSO2 components can be clustered. I am trying to achieve configuration mentioned in this diagram.
Image of Config I am trying to achieve:
Can this configuration is achievable or not?
Can we cluster 2 Publisher nodes and 2 store nodes or not?
And how do we cluster Key Manager use same setting as Identity Manager?
Should we use port offset when running 2 components on the same server? And if yes how we make sure that components are using the ports as mentioned in port offset?
Should we create separate external database for each CarnonDB datasource entry in master_datasource.xml file or we can keep using local H2 database for this. I have created following databases Let me know if I am correct in doing this or not. wso2 databases I created:
I made several copies of wso2 binary files as shown in Image and copied them to the servers where I want to run 2 components on same server. Is this correct way of running 2 components on same server?
For Load balancing which components should we load balance and what ports should be use for load balancing?
That configuration is achievable. But Analytics servers are best to run on separate servers as they utilize a lot of resources.
Yes, you can.
Yes, you need port-offset. If you're on Linux, you can use netstat -pln command and filter by server PID.
Every server needs a local database and other databases are shared as mentioned in https://docs.wso2.com/display/CLUSTER44x/Clustering+API+Manager+2.0.0
Having copies is one way of doing that. Another way is letting a single server act as multiple components. For example, you can run publisher and store components together. You can see the recommended patterns in https://docs.wso2.com/display/AM210/Deployment+Patterns.
Except for Traffic manager, you can load balance every other component. For traffic manager, you can use fail-over. Here are the ports you need to load balance.
Servlet port - 9443(https)/9763 (For admin console and admin services)
NIO port - 8243(https)/8280 (For API calls at gateway)
I'm working on a project to run a Kubernetes cluster on GCE. My goal is to run a cluster containing a WordPress site in multiple zones. I've been reading a lot of documentation, but I can't seem to find anything that is direct and to the point on persistent volumes and statefulsets in a multiple zone scenario. Is this not a supported configuration? I can get the cluster up and the statefulsets deployed, but I'm not getting the state replicated throughout the cluster. Any suggestions?
Thanks,
Darryl
Reading the docs, I see that the recommended configuration would be to create a MySQL cluster with replication: https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/. This way, you would have the data properly replicated between the instances of your cluster (if you are in a multi-zone deployment you may have to create an external endpoint).
Regarding the Wordpress data, my advice would be to go for an immutable deployment: https://engineering.bitnami.com/articles/why-your-next-web-service-should-be-immutable.html . This way, if you need to add a plugin or perform upgrades, you would create a new container image and re-deploy it. Regarding the media library assets and immutability, I think the best option would be to use an external storage service like S3 https://wordpress.org/plugins/amazon-s3-and-cloudfront/
So, to answer the original question: I think that statefulset synchronization is not available in K8s (at the moment). Maybe using a volume provider that allows ReadWriteMany access mode could fit your needs (https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes), though I am quite unsure about the stability of it.
My thinking is that people use Docker to be sure that local environment is the same as production and that I they can stop thinking about where are their apps running physically and balancing mechanisms should just allocate apps in best places for that moment.
I'm 100% web based and I'm going to move to cloud together with our databases, and what cannot be moved will be seamlessly bridged so the corporate stuff and the cloud will become one subnetwork.
And so I'm wondering, maybe Service Fabric already does the same thing that Docker does plus it gives as address translation service (fabric:// that acts a bit like DNS for the processes in fabric space) plus (important for some) encourages on demand worker allocation - huge scalability perk.
Can Service Fabric successfully replace Docker?
Is it gaining audience and acceptance? Because otherwise even the greatest invention can fail.
It's confusing since Docker (the company) is trying to stake claims in everything cloud.
Docker Engine (what most people call "Docker") is a containerization technology. It can give you
Process isolation
Network isolation
Consistent application environment
Docker Hub is an image registry. It stores Docker images so you can download them as part of your deployment.
Docker Cloud is an orchestration system for Docker. It can give you
Scale your applications up and down
Connect your applications to each other
CI testing, integrated with Docker Hub (this isn't part of orchestration, just another thing it does)
Service Fabric is an orchestration system. It can orchestrate Docker containers, but it can also integrate more tightly with your services if you build specifically for Fabric. (Docker is completely agnostic about what runs inside a container.)
So Service Fabric is mostly comparable to Docker Cloud, though it's not an exact match. There are some other Docker-based orchestration solutions (Kubernetes is probably the biggest) and there are other cloud-based micro-service solutions (Heroku is probably the best-known).
The primary disadvantage of Service Fabric is that it's a Microsoft technology and so you're going to be tied to Azure to a greater degree than if you were running Docker. The other is that Docker has a broader range of choices for building your stack: all three Docker-things I listed above have at least one open-source alternative (this is also a big disadvantage of Docker, since nobody's laying out a single Best Practices For You document).
If you love Microsoft and if cobbling systems together is not something that's important to you, then Service Fabric should be a fine alternative to the Docker ecosystem. (And you can still run Docker containers under it.)
The key similarities between the Service Fabric and Docker containerization:
Both Dockers and SF, are capable of creating an immutable image out of your micro-service implementation, on both the platforms - Linux and Windows.
Both Dockers and SF, are capable of orchestrating your containerized application within a cluster of VMs. These VMs can be anywhere - public cloud, private cloud or your own data center. Please note that both of them are cloud platform agnostic, that means, they don't have strong affinity on any of the cloud service. So as long as you are not using any cloud specific feature within your micro-service, this should be fine.
Both Dockers and SF, are capable exhibiting essential capabilities of an orchestrating platform: Service Discovery, Service level load balancing, Network level isolation among services, fail-over handling and replication control etc.
The key differences between the Service Fabric and Docker containerization:
Docker container is essentially a deployment / packaging construct. That said, docker doesn't dictate on what you are packaging within a container as part of your service implementation. Neither it provides any programming construct to implement your kind of service. Whereas, Service Fabric provides programming constructs in the form of base types / interfaces from which your service implementation can start with a certain kind of service declared - stateful service, stateless service, virtual actor.
In the Docker world, everything is a container, i.e. your minimum deployment / orchestration unit is a container. Hence, it doesn't recognize or support an individual process. Whereas, in SF, we have a provision wherein, your micro-service derived from stateless / stateful service can be orchestrated and governed as a process. However, SF also supports container orchestration the way Docker does. Also, the latest version of SF allows packaging your stateful / stateless service within a container.
With above facts in mind, please note that SF doesn't have any strong affinity on any cloud provider. It can run equally on any public cloud - Azure, AWS or GCP, as long as you are able to create the VMs with desired platform.
It is not comparable at all. With service fabric, you get health monitoring, code integration with the fabric, logging, monitoring, load-balancing, and other intelligent features. Your application can even execute shutdown code. Service Fabric is not just for Microsoft technologies and even docker can reside inside SF so is rkt or Unix OS. Security and networking features(in-line with web apps) is another plus. Reliable collections is simply brilliant. And a roadmap to better application building and performance is guaranteed for the companies adopting it (history says so).
This question is highly favoring 'greatest invention' Docker. This comparison can do good for Docker marketing but no one will replace SF for Docker. Docker is just a tiny OS copy (nothing to do with services, applications or intelligence). Docker even has nothing to do with application development, it wasn't the intention. Just that people have started to find the need for isolation and sharing. And that is what Docker is all about.
To handle high traffic, I'm planning to scale out, run my web application (WordPress based) on some EC2 instances (I'm very new to AWS). The instances need to work on the same data (images, videos...).
I am thinking about using S3 as the storage for this shared data.
My questions are:
If I use S3, do I need to write extra codes for my application to upload and get data to/from S3? Or there is a magic way to mount EC2 instances to S3, and after that EC2 instances can access S3 as accessing the local storage?
I've heard that S3 is a bit slow since it is accessed through web services (if users upload files and it takes time to upload the files to S3). So is there any better way for storing shared data?
I've read some documents about the ability of scaling of Amazon EC2. But none of them mentions about how to handle shared data. Any help is highly appreciated. Thanks.
There is no native facility to 'mount' an S3 bucket as storage to an EC2 instance, although there are several 3rd-party apps which offer mechanisms to make S3 storage available as a virtual drive or repository. Most of them offer a preset amount of free storage and then a tiered charging mechanism for larger amounts - Google for 'S3 storage interface' and take a look.
Whether you write code to use S3 through the API or use an interface layer, there will always be some latency between your app and the storage. That's a fact of physics and there's nothing you can do to eliminate the delay, because the S3 repository is not local to the EC2 cluster - so you will never achieve 'local' storage access speeds.
An alternative might be to use EBS which is local to your EC2 instance - it has different properties to S3 (for example, it does not offer edge locations for regionally-localised access) but is much faster for application use because it is inside the EC2 cluster and mounted as local storage.
You could mount S3 bucket onto all your EC2 instances. It's a 2-way mount so all your files will be synchronized. You could use s3f3 to do the mounting.
I used this guide and set up mine pretty quick: Mount S3 onto EC2
If you are then concerned about speed, you can use Amazon ElastiCache or even use EBS as a cache drive.
For starts you question lacks some details about your application architecture, but there are some possibilities.
First, if you project is medium-sized you could use GlusterFS on your main nodes as servers and clients at the same time (using native or NFS protocol), RDS *Multi-AZ* MySQL instance for DataBase. CloudFront as CDN with CDN linker or W3TC plugins. Also, put an ELB in front.
In this particular case I would recommend a couple c3.large instances at least.
Second, when your project would grow - you should make instance AMI and created an auto-scaling group that would just connect to your main storage and compute instances. (Also consider lifting compute load from these rather small instances).
Things to consider additionally:
Great WordPress article about WordPress clusters for is http://harish11g.blogspot.ru/2012/01/scaling-wordpress-aws-amazon-ec2-high.html
Alternative to GlusterFS solution could be Ceph File System
You also could (or maybe should) you SSD cache (for example flashcache) for that GlusterFS volume.