Checklist: pros and cons of using multiple Kubernetes clusters, and how to distribute workloads between them
Here is a list of pros and cons I found for using multiple clusters vs. a single one.
Reasons to have multiple clusters
- Scalability limits, for example a Kubernetes cluster has a limit of 150,000 pods. An OpenShift cluster has a limit of 10,000 services.
- Separation of production/development/test
especially for testing a new version of Kubernetes, of a service mesh, of other cluster software. - Compliance
according to some regulations some applications must run in separate clusters/separate VPNs. - Multi-vendor
to prevent vendor lock-in running clusters of multiple providers. - Cloud/on-prem
to split the load between on-premise services. - Regionality for latency
run clusters in different geographical regions to reduce latency in those regions. - Regionality for availability
run in clusters in different regions/availability zones to reduce damage of a failing datacenter/region. - Better isolation for security
- Isolation for easier billing/resource allocation
Reasons to have a single cluster
- Reduce setup, maintenance and administration overhead
- Improve utilization
- Reduce latency between applications in multiple clusters
- Cost reduction
How to allocate workloads to clusters
- Compliance
some applications must run on separate clusters. - Locality for latency
allocate the applications according to the regions, to reduce latency. - Billing/Quotas
allocate applications together per billing account, to facilitate billing/quota enforcement. - Maintainability
put the applications in the same cluster when it makes sense to perform maintenance of the cluster for all them (upgrading Kubernetes version, etc.). - Hardware requirements
allocate high-performance applications to clusters with hardware for high performance. - Dependencies
reduce the need in intra-cluster service registries by allocating dependent applications together. - Identity and Access management
allocate applications in such a way that in-cluster identity and access management would suffice - Monitoring, tracing, logging
allocate applications to reduce the need for distributed monitoring, tracing, logging.
Sources:
- https://kubernetes.io/docs/setup/cluster-large/
- https://docs.openshift.com/container-platform/3.11/scaling_performance/cluster_limits.html#scaling-performance-current-cluster-limits
- https://kubernetes.io/docs/concepts/cluster-administration/federation/
- https://cloud.google.com/solutions/scope-and-size-kubernetes-engine-clusters
- https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-organizing-with-namespaces
No comments:
Post a Comment