Kubernetes Architecture - Orchestrating Containerized Applications at Scale

Posted on:March 20, 2026 at 12:00 AM

Introduction

Kubernetes (K8s) has become the de facto standard for container orchestration, managing the deployment, scaling, and networking of containerized applications across clusters of machines. But how does this powerful platform actually work? This deep dive explores the architecture and components that make Kubernetes the foundation of modern cloud-native computing.

Why Kubernetes?

Before Kubernetes, manual container management was the norm. As the number of containers grew, this became increasingly complex:

Kubernetes addresses all these challenges through intelligent orchestration, automation, and declarative configuration.

The Journey of Kubernetes

Kubernetes Architecture: Master-Slave Design

Kubernetes uses a master-slave architecture (now called control plane - worker nodes architecture) [18]:

Figure 5: Kubernetes Architecture

Figure 5 (from thesis): Kubernetes architecture: Master node (API Server, etcd, Controller Manager, Scheduler) orchestrates workloads on worker nodes (Kubelet, container runtimes, pods, containers).

Master Node (Control Plane)

The master node is the central control point of a Kubernetes cluster. It:

Worker Nodes

Worker nodes are compute machines that:

Core Kubernetes Components

1. API Server

The API Server is the gateway to the Kubernetes cluster.

Key Responsibilities:

How it works:

kubectl → (REST over HTTPS) → API Server
                               ↓
                            Validate
                            Authenticate
                            Authorize
                               ↓
                            Process Request
                            Update etcd

2. etcd - Cluster State Storage

etcd is a consistent, distributed key-value store that serves as Kubernetes’ source of truth.

What it stores:

Critical characteristics:

API Server → etcd (store/retrieve state)
            ↓
      Cluster State
      (authoritative source)

3. Scheduler

The Scheduler is responsible for pod placement optimization.

How the Scheduler works:

1. Watch for unscheduled pods
   ↓
2. Evaluate pod requirements
   - Resource requests (CPU, memory)
   - Node selectors
   - Affinity/anti-affinity rules
   - Taints and tolerations
   ↓
3. Filter suitable nodes
   ↓
4. Score nodes based on optimization strategies
   ↓
5. Assign pod to best-scoring node
   ↓
6. Update etcd with pod-to-node binding

Scheduling algorithm:

4. Controller Manager

The Controller Manager runs multiple controllers that maintain cluster state.

Key Controllers:

ControllerResponsibility
Deployment ControllerManages Deployment replicas and updates
StatefulSet ControllerManages stateful application pods
DaemonSet ControllerEnsures pod runs on every node
Job ControllerManages batch job completion
Service ControllerManages load balancing and networking
Endpoints ControllerManages service endpoints
Replication ControllerMaintains desired pod count

How controllers work:

Controller watches for state changes
    ↓
Detects difference from desired state
    ↓
Takes action to reconcile state
    ↓
Updates cluster state

Example - Deployment Controller Reconciliation:

Desired: 3 replicas of nginx
Current: 2 replicas running

Controller detects mismatch
    ↓
Creates 1 new pod
    ↓
Scheduler assigns pod to node
    ↓
Kubelet creates pod on assigned node
    ↓
State: 3 replicas running ✓

5. Kubelet - Node Agent

The Kubelet is the Kubernetes agent running on every worker node.

Key Responsibilities:

Kubelet Workflow:

API Server sends pod spec
    ↓
Kubelet receives spec
    ↓
Kubelet communicates with container runtime
    ↓
Container runtime creates/manages containers
    ↓
Kubelet monitors container health
    ↓
Kubelet reports status to API Server

Kubernetes Core Objects

Pods

Pods are the smallest deployable units in Kubernetes.

apiVersion: v1
kind: Pod
metadata:
  name: web-server
spec:
  containers:
  - name: nginx
    image: nginx:1.21
    ports:
    - containerPort: 80

Services

Services expose pods and provide networking.

apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  type: LoadBalancer

ConfigMaps and Secrets

ConfigMaps store non-sensitive configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  database_url: "postgres://db:5432"
  log_level: "info"

Secrets store sensitive data (credentials, tokens):

apiVersion: v1
kind: Secret
metadata:
  name: app-credentials
type: Opaque
data:
  username: dXNlcm5hbWU=  # base64 encoded
  password: cGFzc3dvcmQ=  # base64 encoded

Deployments

Deployments manage ReplicaSets and provide rolling updates:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx:1.21

Communication Flow

Developer runs kubectl command
    ↓
kubectl → HTTPS → API Server
    ↓
API Server validates request
    ↓
API Server writes to etcd
    ↓
Controllers watch etcd for changes
    ↓
Scheduler assigns pods to nodes
    ↓
Kubelet on nodes receives assignments
    ↓
Kubelet communicates with container runtime
    ↓
Containers are created and started
    ↓
Kubelet reports status back to API Server

Kubernetes Networking

Pod-to-Pod Communication

Every pod has unique IP
    ↓
Pods can communicate directly
    ↓
No port mapping needed
    ↓
Flat networking model

Service-to-Pod Mapping

Service (stable IP) → EndpointSlices → Pods (ephemeral IPs)
    ↓
Traffic load balanced across pods
    ↓
DNS enables service discovery

Key Kubernetes Features

Declarative Configuration

# You specify DESIRED state
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 5
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: web
        image: myapp:v1.0

# Kubernetes controllers make it happen

Self-Healing

Automatic Scaling

# Horizontal Pod Autoscaler
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 80

Rolling Updates and Rollbacks

Kubernetes automatically manages version updates:

kubectl set image deployment/web-app web=myapp:v2.0

Kubernetes gradually:
- Starts new v2.0 pods
- Drains old v1.0 pods
- Monitors health
- Completes zero-downtime update

Extensibility and Modularity

One of Kubernetes’ greatest strengths is its modularity. Various components can be swapped:

Container Runtimes:  Docker, containerd, CRI-O, Kata, gVisor
Networking:          Flannel, Calico, Weave, Cilium
Storage:             NFS, Ceph, AWS EBS, GCP Persistent Disk
Monitoring:          Prometheus, Datadog, New Relic
Logging:             ELK, Splunk, Datadog

Conclusion

Kubernetes architecture elegantly solves the complexity of container orchestration through:

  1. Master-worker separation: Clear division of control and computation
  2. Declarative configuration: Specify desired state, not steps
  3. Controllers: Automatic reconciliation of actual vs. desired state
  4. Modularity: Swap components as needs evolve
  5. Extensibility: Rich ecosystem of add-ons and tools

Understanding this architecture is fundamental for anyone deploying containerized applications at scale. Whether you’re managing dozens or millions of containers, Kubernetes provides the abstraction and automation that makes it manageable.

Next: Explore how Kubernetes’ modular architecture enables running alternative workloads like WebAssembly alongside traditional containers!

References

Based on “Exploring WebAssembly-Based Microservices Implementation & Deployment Methods in Kubernetes” by Micheal Choudhary (2024):

[2] “Kubernetes Documentation,” [Online]. Available: https://kubernetes.io/docs/concepts/overview/. [Accessed 26 October 2023].

[3] “9 INSIGHTS ON REAL-WORLD CONTAINER USE,” DataDog, November 2022. [Online]. Available: https://www.datadoghq.com/container-report/. [Accessed 26 October 2023].

[18] “Kubernetes: Kubernetes Components,” [Online]. Available: https://kubernetes.io/docs/concepts/overview/components/. [Accessed 29 October 2023].

[19] “Scaleway: Understanding Kubernetes Autoscaling,” [Online]. Available: https://www.scaleway.com/en/blog/understanding-kubernetes-autoscaling/. [Accessed 29 April 2023].

[20] “Horizontal Pod Autoscaling,” [Online]. Available: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/. [Accessed 01 May 2023].

[21] “Kubernetes Cluster Autoscaler,” [Online]. Available: https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler. [Accessed 2 May 2023].

[22] “Pods,” [Online]. Available: https://kubernetes.io/docs/concepts/workloads/pods/. [Accessed 03 August 2023].

[23] “The Kubernetes API,” [Online]. Available: https://kubernetes.io/docs/concepts/overview/kubernetes-api/. [Accessed 15 October 2023].

[24] “The Kubernetes API Server,” oreilly, [Online]. Available: https://www.oreilly.com/library/view/managing-kubernetes/9781492033905/ch04.html. [Accessed 15 October 2023].

[25] “Objects In Kubernetes,” [Online]. Available: https://kubernetes.io/docs/concepts/overview/working-with-objects/. [Accessed 14 October 2023].

[26] “What is etcd?,” IBM, [Online]. Available: https://www.ibm.com/topics/etcd. [Accessed 20 October 2023].

[27] “Kubernetes Scheduler,” [Online]. Available: https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/. [Accessed 20 October 2023].