I'm trying to set up a general architecture for a system that I'm moving to Kubernetes (self-hosted, probably on VSphere).
I'm not very well versed in networking and I have the following problem that I cannot seem to be able to conceptually solve:
I have many microservices which were split out of a monolith, but the monolith is still significant. All of it is moving to K8s. It's a clustered application and does a lot of all-to-all networking under high load, which I would like to separate from all the other services in the Kubernetes cluster.
Before moving to K8s we provided a way to specify a network device that is used only for the cluster communication, and as such could be strictly separated from other traffic, and alas, even use separate networking hardware for clustering.
So this is where I would request your input: is it possible to have completely separate networking for this application-level cluster inside the Kubernetes cluster? The ideal solution would allow me to continue using our existing logic, i.e. to have a separate network (and network adapter) for the chatty bits but it's not a hard requirement to keep it that way. I have looked at Calico, Flannel, and Istio, but haven't been able to come up with a sound concept.
Use k8s NetworkPolicies, by applying these policing, you can allow/deny traffic for pods base on label selector. you can try WeaveNet and Calico, both are good and support NetworkPolicies.
It is good to have Calico network plugin. Because Flannel doesn't support network policies. You can create NetworkPolicies resources for allow/deny the traffics.
On OpenShift, you can have an isolated network per project (Kubernetes namespace). See https://docs.openshift.com/container-platform/3.5/admin_guide/managing_networking.html#isolating-project-networks
Related
While trying to understand Kubernetes networking, one point has confused me. Why Kubernetes doesn't handle pod-to-pod communication inbuilt/itself?
As per the docs - https://kubernetes.io/docs/concepts/cluster-administration/networking/
There are 4 distinct networking problems to address:
Highly-coupled container-to-container communications: this is solved
by pods and localhost communications.
Pod-to-Pod communications: this is the primary focus of this
document.
Pod-to-Service communications: this is covered by services.
External-to-Service communications: this is covered by services.
When Kubernetes can handle all the other problems(mentioned above) of networking, why does pod-to-pod communication needs to handled by other plugins like ACI, Cilium, Flannel, Jaguar and so on..
Would like to know is there any specific reason for such architecture?
The short answer is that networks are complex and highly customized. It's not easy to provide an efficient built-in that works everywhere. All of the cloud provider networks are different than bare-metal networks. Rather than pick a bad default we require that the end user, who really is the only person who could possibly comprehend their network, makes a decision.
Doing a built-in VXLAN or something might be possible but would be far from ideal for many users, and defaults tend to stick...
Agree with Tim above. Kubernetes in general is mostly an abstraction and orchestration layer of the Compute, storage and networking for the developers so that they don't have to be aware of the implementation. The implementation itself will be tied to the underlying infrastructure and kubernetes just defines the interface for them (CRI for containers -compute, CSI for storage and CNI for networking).
By just defining the interface the implementations can evolve independently without breaking the contract. For example, in future it might become possible to offload pod-to-pod networking to the nic-card and expecting kubernetes to evolve to such a technology change might be a big ask. By not being intimately tied to the implementation it allows development of technology to be accelerated in each layer.
I'm in a high availability project which includes deployment of 2-node high availability cluster for hot replacement of services (applications) running on the cluster nodes. The applications have inbound and outbound tcp connections as well as process udp traffic (mainly for communicating with ntp server).
The problem is pretty standard until one needs to provide a hot migration of services to backup node with all the data stored in RAM. Applications are agnostic of backup mechanisms and it is highly undesirable to modify them.
As only approach to this problem, I've come off with a duplication approach assuming that both cluster nodes will run the same applications repeating calculations of each other. In case of failure the primary server the backup server will become a primary.
However, I have not found any ready solution for proxy which will have synchronous port mirroring. No existing proxy servers (haproxy, dante, 3proxy etc.) support such feature as far as I know. Have I missed something, or I should write a new one from scratch?
A rough sketch of the functionality can be found here:
p.s. I assume that it is possible to compare traffic from the two clones of the same application...
I have been experimenting with k8s/kops/aws suite.
So far things have been going pretty well (except an issue when updating cluster via kops)
I want to be able to make use of my existing resources/cluster and deploy 2 flavors of my app (i.e. production and testing) in the same cluster.
I would like to be on the safe side and maximize as much as possible the isolation between the k8s resources of those two deployments.
Definately there going on different namespaces.
From some investigation I have found out that I need to also apply NetworkPolicy to prevent inter-namespace communication; however applying NetworkPolicy resources requires a supporting networking solution (currently using kubenet, the default of kops which doesn't).
What is the solution/plugin to go for?
Just want (at least for the time being) the level of isolation described above which I assume can be achieved via NetworkPolicy even if there is a common CIDR for all pods (just saying that to emphasise that there is a need for just the simplest possible networking solution that achieves that, nothing more fancy with multiple CIDRs etc).
Ideally I would like to be able just to use NetworkPolicy resource for some namespace-based (namespaceSelector) and pod-based (podSelector) ingress rules and that's it (?)
On my kops clusters I use weave networking (also I provision them as private topology, which excludes kubenet anyway). So my first suggestion would be to go with a different networking, weave and calico being first ones coming to my mind.
Other then that, you might want to look into service mesh solution like Istio which can leverage NetworkPolicies as well (some istio policy reading)
Long story short - I need to use networking between projects to have separate billing for them.
I'd like to reach all the VMs in different projects from a single point that I will use for provisioning systems (let's call it coordinator node).
It looks like VPC network peering is a perfect solution to this. But unfortunately one of the existing networks is "legacy". Here's what google docs state about legacy networks.
About legacy networks
Note: Legacy networks are not recommended. Many newer GCP features are not supported in legacy networks.
OK, naturally the question arises: how do you migrate out of legacy network? Documentation does not address this topic. Is it not possible?
I have a bunch of VMs, and I'd be able to shutdown them one by one:
shutdown
change something
restart
unfortunately it does not seem possible to change network even when VM is down?
EDIT:
it has been suggested to recreate VMs keeping the same disks. I would still need a way to bridge legacy network with new VPC network to make migration fluent. Any thoughts on how to do that using GCE toolset?
One possible solution - for each VM in the legacy network:
Get VM parameters (API get method)
Delete VM without deleting PD (persistent disk)
Create VM in the new VPC network using parameters from step 1 (and existing persistent disk)
This way stop-change-start is not so different from delete-recreate-with-changes. It's possible to write a script to fully automate this (migration of a whole network). I wouldn't be surprised if someone already did that.
UDPATE
https://github.com/googleinterns/vm-network-migration tool automates the above process, plus it supports migration of a whole Instance Group or Load Balancer, etc. Check it out.
How to make two VMs communicate with each other? I have to split a task between two VMs, so I think MPI has to be used, If so are there any useful resources that I can use to get started? Any help would be appreciated.
P.S : I have instaled devstack juno
Your question is not really clear.
Openstack is just a virtualization technology. There's almost no difference between having two hardware servers and two VMs. E.g. normally if two servers belong to the same network segment they will have access to each other's open ports. Openstack works just in the same way - if you assign the same network to VMs then this will also work.
However if you wish to install two VMs that will consume from a list of tasks and do them in parallel I would recommend you to read about Enterprise Integration Patterns (e.g. here). Technically this is implemented by using one or several messaging middleware servers such as ActiveMQ or ZeroMQ.