Skip to content

Instantly share code, notes, and snippets.

@roninjin10
Created December 24, 2025 07:11
Show Gist options
  • Select an option

  • Save roninjin10/e0d46ca241af3aac403089477d49fce2 to your computer and use it in GitHub Desktop.

Select an option

Save roninjin10/e0d46ca241af3aac403089477d49fce2 to your computer and use it in GitHub Desktop.
Great prompt

Task: Route All Plue Traffic Through Cloudflare Edge

Objective

Refactor Plue's architecture so ALL traffic flows through Cloudflare:

  1. HTTP/API → Cloudflare Edge Worker (already exists, add SIWE auth)
  2. Git SSH → Cloudflare Spectrum (new)
  3. Origin Protection → mTLS with custom certificates (new)

After this change, the origin server will ONLY accept connections from Cloudflare.

Current State

infra/terraform/modules/ ├── cloudflare-workers/ # Edge Worker deployment (exists) ├── cloudflare-tunnel/ # Tunnel to origin (exists) ├── gke/ # Kubernetes cluster (exists) └── cloudsql/ # PostgreSQL (exists)

edge/ ├── index.ts # Caching proxy (exists) └── types.ts # Type definitions (exists)

server/src/ ├── ssh/server.zig # SSH server on port 2222 (exists) ├── lib/siwe.zig # SIWE verification (exists, will be simplified) └── routes/auth.zig # Auth routes (exists, will be simplified)

Target Architecture

┌─────────────────────────────────────────────────────────────────────────────────────┐ │ ALL CLIENTS │ │ │ │ Browser Git SSH API Client │ │ │ │ │ │ │ │ HTTPS │ SSH (port 22 or 443) │ HTTPS │ │ │ │ │ │ └────────────┼──────────────────────────┼──────────────────────────────┼──────────────┘ │ │ │ ▼ ▼ ▼ ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ CLOUDFLARE EDGE │ │ │ │ ┌────────────────────────────────────────────────────────────────────────────────┐ │ │ │ Edge Worker (TypeScript) │ │ │ │ │ │ │ │ NEW: SIWE Authentication │ │ │ │ - GET /api/auth/nonce → Generate nonce, store in KV │ │ │ │ - POST /api/auth/verify → Verify signature, create session JWT │ │ │ │ - POST /api/auth/logout → Clear session │ │ │ │ │ │ │ │ NEW: Auth Middleware │ │ │ │ - Check session JWT on protected routes │ │ │ │ - Add X-Plue-User-Address header to origin requests │ │ │ │ - Bypass auth for public routes (landing, public repos) │ │ │ │ │ │ │ │ EXISTING: Caching │ │ │ │ - Session-aware (bypass cache for authenticated users) │ │ │ │ - Version-based invalidation │ │ │ │ │ │ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌────────────────────────────────────────────────────────────────────────────────┐ │ │ │ Cloudflare Spectrum (TCP Proxy) │ │ │ │ │ │ │ │ - Protocol: SSH (TCP/22) │ │ │ │ - Edge Port: 443 (bypass restrictive firewalls) │ │ │ │ - PROXY Protocol v1 enabled (preserve client IPs) │ │ │ │ - Argo Smart Routing enabled │ │ │ │ - DDoS protection │ │ │ │ │ │ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌────────────────────────────────────────────────────────────────────────────────┐ │ │ │ Authenticated Origin Pulls (mTLS) │ │ │ │ │ │ │ │ - Custom CA certificate (NOT Cloudflare's shared cert) │ │ │ │ - Per-zone configuration for plue.dev │ │ │ │ - Origin rejects ALL connections without valid client cert │ │ │ │ │ │ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │ │ └───────────────────────────────────────────────────────────────────────────────────┬──┘ │ mTLS + PROXY proto│ ▼ ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ ZIG ORIGIN SERVER │ │ │ │ SIMPLIFIED: │ │ - Trust X-Plue-User-Address header from Cloudflare │ │ - Remove SIWE signature verification (moved to edge) │ │ - Remove nonce management (moved to edge KV) │ │ - SSH server parses PROXY protocol for real client IP │ │ - mTLS required (reject non-Cloudflare connections) │ │ │ └─────────────────────────────────────────────────────────────────────────────────────┘


Phase 1: Move SIWE Authentication to Edge Worker

Context

Current SIWE implementation:

  • server/src/lib/siwe.zig - Signature verification
  • server/src/routes/auth.zig - Auth endpoints
  • db/schema.sql - siwe_nonces table for replay protection

The official SIWE implementation uses the siwe npm package from SpruceID. Reference: https://docs.login.xyz/ and https://github.com/spruceid/siwe-quickstart

Task

  1. Install dependencies in edge/:
    npm install siwe ethers
    
  2. Create edge/lib/siwe.ts - SIWE message creation and verification: import { SiweMessage, generateNonce } from 'siwe';

// Follow the official flow from docs.login.xyz: // 1. generateNonce() - create random nonce // 2. new SiweMessage({...}) - create message // 3. message.verify({ signature }) - verify signed message 3. Create edge/lib/session.ts - JWT session management: - Sign JWTs with a secret stored in Worker secrets - Include wallet address and expiry in JWT payload - Use jose or @tsndr/cloudflare-worker-jwt for JWT operations 4. Create edge/routes/auth.ts - Auth route handlers: // GET /api/auth/nonce // - Generate nonce with generateNonce() // - Store in KV with 5-minute TTL: await env.AUTH_KV.put(nonce, '', { expirationTtl: 300 }) // - Return nonce

// POST /api/auth/verify // - Parse { message, signature } from body // - Verify nonce exists in KV (replay protection) // - Verify signature with SiweMessage.verify() // - Delete nonce from KV // - Create signed JWT with wallet address // - Set session cookie // - Return success

// POST /api/auth/logout // - Clear session cookie 5. Update edge/index.ts - Add auth middleware: // Before proxying to origin: // 1. Check for session cookie // 2. If present, verify JWT signature // 3. If valid, add header: X-Plue-User-Address: 0x... // 4. If /api/auth/* route, handle in worker (don't proxy) 6. Create edge/lib/public-routes.ts - Routes that don't require auth: export const PUBLIC_ROUTES = [ '/', // Landing page '/api/auth/*', // Auth endpoints /^/[^/]+/[^/]+$/, // Public repo pages: /owner/repo /^/[^/]+/[^/]+/tree//, // Public tree view /^/[^/]+/[^/]+/blob//, // Public blob view ]; 7. Add KV namespace for auth in infra/terraform/modules/cloudflare-workers/main.tf: resource "cloudflare_workers_kv_namespace" "auth" { account_id = var.account_id title = "plue-auth-${var.environment}" } 8. Simplify Zig server: - server/src/routes/auth.zig - Remove SIWE verification, trust X-Plue-User-Address header - server/src/lib/siwe.zig - Remove or keep only for reference - server/src/middleware/auth.zig - Read user from X-Plue-User-Address header

Files to Create/Modify

edge/ ├── lib/ │ ├── siwe.ts (create) - SIWE message handling │ ├── session.ts (create) - JWT session management │ └── public-routes.ts (create) - Public route patterns ├── routes/ │ └── auth.ts (create) - Auth route handlers ├── index.ts (modify) - Add auth middleware ├── types.ts (modify) - Add AUTH_KV binding └── package.json (modify) - Add siwe, ethers deps

infra/terraform/modules/cloudflare-workers/ ├── main.tf (modify) - Add AUTH_KV namespace └── variables.tf (modify) - Add jwt_secret variable

server/src/ ├── routes/auth.zig (modify) - Simplify, trust edge ├── lib/siwe.zig (modify) - Remove verification logic └── middleware/auth.zig (modify) - Read X-Plue-User-Address


Phase 2: Add Cloudflare Spectrum for SSH

Context

Current SSH server:

  • server/src/ssh/server.zig - Listens on port 2222
  • Uses SSH public key authentication via ssh_keys table
  • Direct connection from clients (bypasses Cloudflare)

Cloudflare Spectrum:

  • Proxies TCP traffic with DDoS protection
  • SSH supported on Pro/Business plans (port 22)
  • Can run on port 443 to bypass restrictive firewalls
  • PROXY protocol preserves real client IPs

Reference: https://developers.cloudflare.com/spectrum/

Task

  1. Create infra/terraform/modules/cloudflare-spectrum/main.tf: terraform { required_providers { cloudflare = { source = "cloudflare/cloudflare" version = "~> 4.0" } } }

SSH on standard port 22

resource "cloudflare_spectrum_application" "ssh" { zone_id = var.zone_id protocol = "ssh" traffic_type = "direct"

dns {
  type = "CNAME"
  name = "ssh.${var.domain}"
}

origin_direct = ["tcp://${var.origin_ip}:22"]

# Enable PROXY protocol to preserve client IPs
proxy_protocol = "v1"

# Smart routing for performance
argo_smart_routing = true

# IP firewall integration
ip_firewall = true

}

SSH on port 443 (bypass restrictive firewalls)

resource "cloudflare_spectrum_application" "ssh_443" { zone_id = var.zone_id protocol = "tcp/443" traffic_type = "direct"

dns {
  type = "CNAME"
  name = "git.${var.domain}"
}

origin_direct = ["tcp://${var.origin_ip}:22"]

proxy_protocol = "v1"
argo_smart_routing = true
ip_firewall = true

} 2. Create infra/terraform/modules/cloudflare-spectrum/variables.tf: variable "zone_id" { description = "Cloudflare zone ID" type = string }

variable "domain" { description = "Base domain (e.g., plue.dev)" type = string }

variable "origin_ip" { description = "Origin server IP or hostname" type = string } 3. Create server/src/ssh/proxy_protocol.zig - Parse PROXY protocol v1: /// Parse PROXY protocol v1 header /// Format: "PROXY TCP4 <src_ip> <dst_ip> <src_port> <dst_port>\r\n" /// Reference: https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt pub fn parseProxyProtocolV1(data: []const u8) !?ProxyInfo { // Check for "PROXY " prefix // Parse protocol (TCP4/TCP6) // Extract source IP and port // Return ProxyInfo struct with real client IP }

pub const ProxyInfo = struct { client_ip: std.net.Address, client_port: u16, server_ip: std.net.Address, server_port: u16, protocol: Protocol, }; 4. Update server/src/ssh/server.zig: - On new connection, check for PROXY protocol header - Extract real client IP from PROXY header - Use real IP for logging, rate limiting, and auth checks - Fall back to connection IP if no PROXY header (local dev) 5. Update SSH port to 22 in config (Spectrum expects standard port) 6. Update GKE service to expose port 22:

infra/k8s/ or terraform

spec: ports: - name: ssh port: 22 targetPort: 22

Files to Create/Modify

infra/terraform/modules/cloudflare-spectrum/ ├── main.tf (create) - Spectrum applications ├── variables.tf (create) - Module variables └── outputs.tf (create) - DNS names output

infra/terraform/environments/production/ └── main.tf (modify) - Add spectrum module

server/src/ssh/ ├── proxy_protocol.zig (create) - PROXY protocol parser └── server.zig (modify) - Use PROXY protocol

server/src/config.zig (modify) - Change SSH port to 22


Phase 3: Enable Authenticated Origin Pulls (mTLS)

Context

Currently, origin accepts connections from any IP. We need to ensure ONLY Cloudflare can reach it.

Using a custom certificate (not Cloudflare's shared cert) prevents other Cloudflare users from pointing their domains at our origin.

Reference: https://developers.cloudflare.com/ssl/origin-configuration/authenticated-origin-pull/

Task

  1. Create infra/scripts/generate-mtls-certs.sh: #!/bin/bash

Generate custom CA and client certificate for mTLS

set -euo pipefail

OUTPUT_DIR="${1:-./certs}" mkdir -p "$OUTPUT_DIR"

Generate CA private key

openssl genrsa -out "$OUTPUT_DIR/ca.key" 4096

Generate CA certificate (10 year validity)

openssl req -new -x509 -days 3650 -key "$OUTPUT_DIR/ca.key"
-subj "/CN=Plue Origin CA/O=Plue/C=US"
-out "$OUTPUT_DIR/ca.crt"

Generate client private key

openssl genrsa -out "$OUTPUT_DIR/client.key" 4096

Generate client CSR

openssl req -new -key "$OUTPUT_DIR/client.key"
-subj "/CN=Cloudflare Client/O=Plue/C=US"
-out "$OUTPUT_DIR/client.csr"

Sign client certificate with CA (1 year validity)

openssl x509 -req -days 365 -in "$OUTPUT_DIR/client.csr"
-CA "$OUTPUT_DIR/ca.crt" -CAkey "$OUTPUT_DIR/ca.key"
-CAcreateserial -out "$OUTPUT_DIR/client.crt"

Combine client cert and key for Cloudflare upload

cat "$OUTPUT_DIR/client.crt" "$OUTPUT_DIR/client.key" > "$OUTPUT_DIR/client.pem"

echo "Certificates generated in $OUTPUT_DIR" echo "Upload client.pem to Cloudflare for Authenticated Origin Pulls" echo "Configure origin server with ca.crt to verify client certificates" 2. Create infra/terraform/modules/cloudflare-mtls/main.tf: terraform { required_providers { cloudflare = { source = "cloudflare/cloudflare" version = "~> 4.0" } } }

Upload custom client certificate

resource "cloudflare_authenticated_origin_pulls_certificate" "origin" { zone_id = var.zone_id certificate = var.client_certificate private_key = var.client_private_key type = "per-zone" }

Enable Authenticated Origin Pulls

resource "cloudflare_authenticated_origin_pulls" "origin" { zone_id = var.zone_id authenticated_origin_pulls_certificate = cloudflare_authenticated_origin_pulls_certificate.origin.id enabled = true } 3. Create infra/terraform/modules/cloudflare-mtls/variables.tf: variable "zone_id" { description = "Cloudflare zone ID" type = string }

variable "client_certificate" { description = "Client certificate PEM content" type = string sensitive = true }

variable "client_private_key" { description = "Client private key PEM content" type = string sensitive = true } 4. Store CA cert in Kubernetes secret:

In infra/terraform/kubernetes/secrets.tf

resource "kubernetes_secret" "mtls_ca" { metadata { name = "mtls-ca" namespace = "production" }

data = {
  "ca.crt" = var.mtls_ca_certificate
}

} 5. Update Zig server for mTLS - Modify TLS configuration to require client certificates: // In server/src/main.zig or config // When TLS is enabled: // - Load CA certificate from /etc/plue/certs/ca.crt // - Set client_ca to require client certificate verification // - Reject connections without valid client cert 6. Add environment flag for local development (skip mTLS in dev mode) 7. Update GKE deployment to mount CA certificate: volumeMounts: - name: mtls-ca mountPath: /etc/plue/certs readOnly: true volumes: - name: mtls-ca secret: secretName: mtls-ca

Files to Create/Modify

infra/scripts/ └── generate-mtls-certs.sh (create) - Certificate generation

infra/terraform/modules/cloudflare-mtls/ ├── main.tf (create) - AOP configuration ├── variables.tf (create) - Module variables └── outputs.tf (create) - Status outputs

infra/terraform/kubernetes/ └── secrets.tf (modify) - Add mtls-ca secret

infra/terraform/environments/production/ └── main.tf (modify) - Add mtls module

server/src/ ├── main.zig (modify) - Add mTLS config └── config.zig (modify) - Add mTLS options


Phase 4: Update Production Environment

Task

  1. Update infra/terraform/environments/production/main.tf:

Existing modules...

module "cloudflare_spectrum" { source = "../../modules/cloudflare-spectrum"

zone_id   = var.cloudflare_zone_id
domain    = var.domain
origin_ip = module.gke.external_ip

}

module "cloudflare_mtls" { source = "../../modules/cloudflare-mtls"

zone_id            = var.cloudflare_zone_id
client_certificate = var.mtls_client_cert
client_private_key = var.mtls_client_key

} 2. Add new variables to variables.tf: variable "mtls_client_cert" { description = "mTLS client certificate for Cloudflare" type = string sensitive = true }

variable "mtls_client_key" { description = "mTLS client private key for Cloudflare" type = string sensitive = true }

variable "mtls_ca_cert" { description = "mTLS CA certificate for origin verification" type = string sensitive = true } 3. Update GKE LoadBalancer - restrict to Cloudflare IPs only (defense in depth):

Add loadBalancerSourceRanges with Cloudflare IP ranges


Testing Strategy

  1. Local Development: - PLUE_DEV_MODE=true skips mTLS requirement - Mock X-Plue-User-Address header for testing auth - SSH server accepts connections without PROXY protocol
  2. Staging: - Deploy full stack to staging namespace - Configure Spectrum for staging.plue.dev - Test SIWE flow end-to-end - Test SSH clone/push through Spectrum
  3. Production Migration: - Deploy with feature flag: accept both old sessions AND new JWTs - Monitor error rates - Switch DNS to use Spectrum - Remove old auth code after 1 week

Success Criteria

  • All HTTP traffic authenticated at Cloudflare edge
  • SIWE verification happens in Edge Worker (no origin round-trip)
  • Git SSH traffic flows through Cloudflare Spectrum
  • SSH server correctly parses PROXY protocol for client IPs
  • Origin rejects non-Cloudflare connections (mTLS)
  • Terraform applies cleanly with no manual steps
  • No user-facing changes to auth flow
  • DDoS protection covers all protocols
  • Local development still works without Cloudflare

Rollback Plan

  1. SIWE at edge: Toggle feature flag to use origin SIWE verification
  2. Spectrum: Point DNS directly to origin IP (bypass Spectrum)
  3. mTLS: Disable client cert requirement in Zig server config
  4. All changes are infrastructure-level and can be reverted via Terraform

Sources:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment