Tuesday, March 26, 2019

Checklist: pros and cons of using multiple Kubernetes clusters, and how to distribute workloads between them


Here is a list of pros and cons I found for using multiple clusters vs. a single one.

Reasons to have multiple clusters

  • Scalability limits, for example a Kubernetes cluster has a limit of 150,000 pods. An OpenShift cluster has a limit of 10,000 services.
  • Separation of production/development/test
    especially for testing a new version of Kubernetes, of a service mesh, of other cluster software.
  • Compliance
    according to some regulations some applications must run in separate clusters/separate VPNs.
  • Multi-vendor
    to prevent vendor lock-in running clusters of multiple providers.
  • Cloud/on-prem
    to split the load between on-premise services.
  • Regionality for latency
    run clusters in different geographical regions to reduce latency in those regions.
  • Regionality for availability
    run in clusters in different regions/availability zones to reduce damage of a failing datacenter/region.
  • Better isolation for security
  • Isolation for easier billing/resource allocation

Reasons to have a single cluster

  • Reduce setup, maintenance and administration overhead
  • Improve utilization
  • Reduce latency between applications in multiple clusters
  • Cost reduction

How to allocate workloads to clusters

  • Compliance
    some applications must run on separate clusters.
  • Locality for latency
    allocate the applications according to the regions, to reduce latency.
  • Billing/Quotas
    allocate applications together per billing account, to facilitate billing/quota enforcement.
  • Maintainability
    put the applications in the same cluster when it makes sense to perform maintenance of the cluster for all them (upgrading Kubernetes version, etc.).
  • Hardware requirements
    allocate high-performance applications to clusters with hardware for high performance.
  • Dependencies
    reduce the need in intra-cluster service registries by allocating dependent applications together.
  • Identity and Access management
    allocate applications in such a way that in-cluster identity and access management would suffice
  • Monitoring, tracing, logging
    allocate applications to reduce the need for distributed monitoring, tracing, logging.

Sources:

No comments:

Post a Comment