december 2024

🚀 From Localhost to the Cloud: The Ultimate Node.js Scaling Journey

A comprehensive guide on taking a basic, single-threaded Node.js application and transforming it into a highly available, auto-scaling, enterprise-grade architecture on AWS Elastic Kubernetes Service (EKS).

📖 The Problem: Node.js is Single-Threaded

By default, Node.js runs on a single thread. If you run a heavy computation loop, it blocks the entire process, preventing any other users from accessing your API.

🛡️ Stage 1: Vertical Scaling with PM2

To fix the single-thread limit on a single server, we use PM2. PM2 automatically spawns multiple instances of your Node.js app across all available CPU cores, distributing the load.

# Install PM2 and run the application utilizing max available cores
npm i pm2 -g
pm2 start index.js -i max

The Limit: PM2 is fantastic, but it is bounded by the physical limits of the single machine it runs on. If traffic exceeds the machine's maximum capacity, the server crashes.

🐳 Stage 2: Containerization (Docker)

To deploy horizontally across hundreds of machines, we must package the code and PM2 into a universal format: a Docker Container.

Dockerfile

FROM node:18-alpine
WORKDIR /app

RUN npm install pm2 -g
COPY package*.json ./
RUN npm install
COPY . .

EXPOSE 8000

# 'pm2-runtime' keeps the container running in the foreground
# '-i max' dynamically scales workers to the container's allowed CPU limits
CMD ["pm2-runtime", "01-single-node/index.js", "-i", "max"]

☸️ Stage 3: Horizontal Kubernetes Scaling (K8s)

Kubernetes (K8s) allows us to run multiple copies of our Docker container simultaneously across a cluster of servers.

We use a Horizontal Pod Autoscaler (HPA). It watches CPU metrics and spins up more temporary containers when traffic spikes, deleting them when traffic dies down.

k8s.yaml (The Infrastructure Blueprint)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: node-pm2-deployment
  labels:
    app: node-pm2-app
spec:
  replicas: 3 # Minimum baseline replicas
  selector:
    matchLabels:
      app: node-pm2-app
  template:
    metadata:
      labels:
        app: node-pm2-app
    spec:
      containers:
        - name: pm2-container
          image: <YOUR_AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/pm2-node-app:latest
          imagePullPolicy: Always
          ports:
            - containerPort: 8000
          resources:
            requests:
              cpu: "250m"
            limits:
              cpu: "2" # Kubernetes gives 2 cores; PM2 automatically spawns 2 workers!
---
apiVersion: v1
kind: Service
metadata:
  name: node-pm2-service
spec:
  type: LoadBalancer
  selector:
    app: node-pm2-app
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8000
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: node-pm2-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: node-pm2-deployment
  minReplicas: 3
  maxReplicas: 20 # Scale up to 20 containers if needed!
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50 # Trigger scaling if avg CPU > 50%

☁️ Stage 4: Deploying to AWS EKS

1. Prerequisites (For Mac)

brew install awscli
brew tap weaveworks/tap
brew install weaveworks/tap/eksctl
aws configure

2. Spinning up the AWS EKS Cluster

We start by spinning up the Kubernetes control plane and some standard AWS virtual machines (t3.medium).

eksctl create cluster \
  --name production-node-cluster \
  --region us-east-1 \
  --nodegroup-name standard-workers \
  --node-type t3.medium \
  --nodes 2 \
  --managed

3. Resolving the ARM64 (Mac) to AMD64 (AWS) Architecture Conflict

🚨 THE GOTCHA: If you build a Docker image on an Apple Silicon Mac (M1/M2/M3), it creates an ARM64 image. AWS t3/c5 instances use Intel AMD64 CPUs. This results in AWS logging an ImagePullBackOff and no match for platform in manifest error.

The Fix: Force Docker to cross-compile for Intel processors locally!

# 1. Build for Intel (AMD64)
docker build --platform linux/amd64 -t pm2-node-app:latest .

# 2. Create AWS Remote Registry
aws ecr create-repository --repository-name pm2-node-app --region us-east-1

# 3. Authenticate your terminal
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <YOUR_AWS_ID>.dkr.ecr.us-east-1.amazonaws.com

# 4. Tag and Push
docker tag pm2-node-app:latest <YOUR_AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/pm2-node-app:latest
docker push <YOUR_AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/pm2-node-app:latest

4. Deploying the Architecture to AWS

# Install Metrics Server (Crucial for HPA CPU scaling)
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# Apply the Deployment, Service, and HPA
kubectl apply -f k8s.yaml

⛈️ Stage 5: The "Full Rumble" Load Test

We hit the AWS public Load Balancer using autocannon to simulate 5,000 rapid web requests:

autocannon -c 200 -a 5000 http://<YOUR_AWS_LOADBALANCER_URL>:8080/heavy

🚨 THE SECOND GOTCHA: Pods stuck in Pending During the test, our HPA successfully demanded 20 pods. However, 8 pods got stuck in a Pending state.

  • Why? We set limits: cpu: "2". 20 pods x 2 cores = 40 cores needed. Our baseline t3.medium instances only had 4 cores total! Furthermore, AWS bursted down our t3 CPU credits due to the extreme load, resulting in 4,000 timeouts.

The Fix: Provisioning Compute-Monsters

We ditched the t3.medium servers for heavy-duty, compute-optimized c5.xlarge nodes (4 servers, giving us 16 unthrottled CPUs).

# Provision the monsters
eksctl create nodegroup --cluster production-node-cluster --region us-east-1 --name compute-monsters --node-type c5.xlarge --nodes 4 --managed

# Delete the weak, standard workers
eksctl delete nodegroup --cluster production-node-cluster --region us-east-1 --name standard-workers

The Results

Under the exact same 5,000-request load test:

  • Creation Speed: All 20 Pods moved from Pending -> Running in under 3 seconds.
  • Test Duration: Slashed in half (from 250s to 126s).
  • Timeouts: Crashed by 85%.
  • Autoscaling: K8s dynamically scaled pods from 3 ➡️ 6 ➡️ 12 ➡️ 20, completely mitigating the traffic spike, then gracefully stepped them back down to 3 when traffic subsided.

🏰 The Final Enterprise Architecture

🧹 Stage 6: Teardown (CRITICAL)

Running massive c5 instances costs money ($0.17/hour each). ALWAYS destroy your cluster once testing is finished.

eksctl delete cluster --name production-node-cluster --region us-east-1