Refactor Plue's architecture so ALL traffic flows through Cloudflare:
- HTTP/API → Cloudflare Edge Worker (already exists, add SIWE auth)
- Git SSH → Cloudflare Spectrum (new)
- Origin Protection → mTLS with custom certificates (new)
After this change, the origin server will ONLY accept connections from Cloudflare.
infra/terraform/modules/ ├── cloudflare-workers/ # Edge Worker deployment (exists) ├── cloudflare-tunnel/ # Tunnel to origin (exists) ├── gke/ # Kubernetes cluster (exists) └── cloudsql/ # PostgreSQL (exists)
edge/ ├── index.ts # Caching proxy (exists) └── types.ts # Type definitions (exists)
server/src/ ├── ssh/server.zig # SSH server on port 2222 (exists) ├── lib/siwe.zig # SIWE verification (exists, will be simplified) └── routes/auth.zig # Auth routes (exists, will be simplified)
┌─────────────────────────────────────────────────────────────────────────────────────┐ │ ALL CLIENTS │ │ │ │ Browser Git SSH API Client │ │ │ │ │ │ │ │ HTTPS │ SSH (port 22 or 443) │ HTTPS │ │ │ │ │ │ └────────────┼──────────────────────────┼──────────────────────────────┼──────────────┘ │ │ │ ▼ ▼ ▼ ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ CLOUDFLARE EDGE │ │ │ │ ┌────────────────────────────────────────────────────────────────────────────────┐ │ │ │ Edge Worker (TypeScript) │ │ │ │ │ │ │ │ NEW: SIWE Authentication │ │ │ │ - GET /api/auth/nonce → Generate nonce, store in KV │ │ │ │ - POST /api/auth/verify → Verify signature, create session JWT │ │ │ │ - POST /api/auth/logout → Clear session │ │ │ │ │ │ │ │ NEW: Auth Middleware │ │ │ │ - Check session JWT on protected routes │ │ │ │ - Add X-Plue-User-Address header to origin requests │ │ │ │ - Bypass auth for public routes (landing, public repos) │ │ │ │ │ │ │ │ EXISTING: Caching │ │ │ │ - Session-aware (bypass cache for authenticated users) │ │ │ │ - Version-based invalidation │ │ │ │ │ │ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌────────────────────────────────────────────────────────────────────────────────┐ │ │ │ Cloudflare Spectrum (TCP Proxy) │ │ │ │ │ │ │ │ - Protocol: SSH (TCP/22) │ │ │ │ - Edge Port: 443 (bypass restrictive firewalls) │ │ │ │ - PROXY Protocol v1 enabled (preserve client IPs) │ │ │ │ - Argo Smart Routing enabled │ │ │ │ - DDoS protection │ │ │ │ │ │ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌────────────────────────────────────────────────────────────────────────────────┐ │ │ │ Authenticated Origin Pulls (mTLS) │ │ │ │ │ │ │ │ - Custom CA certificate (NOT Cloudflare's shared cert) │ │ │ │ - Per-zone configuration for plue.dev │ │ │ │ - Origin rejects ALL connections without valid client cert │ │ │ │ │ │ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │ │ └───────────────────────────────────────────────────────────────────────────────────┬──┘ │ mTLS + PROXY proto│ ▼ ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ ZIG ORIGIN SERVER │ │ │ │ SIMPLIFIED: │ │ - Trust X-Plue-User-Address header from Cloudflare │ │ - Remove SIWE signature verification (moved to edge) │ │ - Remove nonce management (moved to edge KV) │ │ - SSH server parses PROXY protocol for real client IP │ │ - mTLS required (reject non-Cloudflare connections) │ │ │ └─────────────────────────────────────────────────────────────────────────────────────┘
Current SIWE implementation:
server/src/lib/siwe.zig- Signature verificationserver/src/routes/auth.zig- Auth endpointsdb/schema.sql-siwe_noncestable for replay protection
The official SIWE implementation uses the siwe npm package from SpruceID.
Reference: https://docs.login.xyz/ and https://github.com/spruceid/siwe-quickstart
- Install dependencies in
edge/:npm install siwe ethers
- Create edge/lib/siwe.ts - SIWE message creation and verification: import { SiweMessage, generateNonce } from 'siwe';
// Follow the official flow from docs.login.xyz: // 1. generateNonce() - create random nonce // 2. new SiweMessage({...}) - create message // 3. message.verify({ signature }) - verify signed message 3. Create edge/lib/session.ts - JWT session management: - Sign JWTs with a secret stored in Worker secrets - Include wallet address and expiry in JWT payload - Use jose or @tsndr/cloudflare-worker-jwt for JWT operations 4. Create edge/routes/auth.ts - Auth route handlers: // GET /api/auth/nonce // - Generate nonce with generateNonce() // - Store in KV with 5-minute TTL: await env.AUTH_KV.put(nonce, '', { expirationTtl: 300 }) // - Return nonce
// POST /api/auth/verify // - Parse { message, signature } from body // - Verify nonce exists in KV (replay protection) // - Verify signature with SiweMessage.verify() // - Delete nonce from KV // - Create signed JWT with wallet address // - Set session cookie // - Return success
// POST /api/auth/logout // - Clear session cookie 5. Update edge/index.ts - Add auth middleware: // Before proxying to origin: // 1. Check for session cookie // 2. If present, verify JWT signature // 3. If valid, add header: X-Plue-User-Address: 0x... // 4. If /api/auth/* route, handle in worker (don't proxy) 6. Create edge/lib/public-routes.ts - Routes that don't require auth: export const PUBLIC_ROUTES = [ '/', // Landing page '/api/auth/*', // Auth endpoints /^/[^/]+/[^/]+$/, // Public repo pages: /owner/repo /^/[^/]+/[^/]+/tree//, // Public tree view /^/[^/]+/[^/]+/blob//, // Public blob view ]; 7. Add KV namespace for auth in infra/terraform/modules/cloudflare-workers/main.tf: resource "cloudflare_workers_kv_namespace" "auth" { account_id = var.account_id title = "plue-auth-${var.environment}" } 8. Simplify Zig server: - server/src/routes/auth.zig - Remove SIWE verification, trust X-Plue-User-Address header - server/src/lib/siwe.zig - Remove or keep only for reference - server/src/middleware/auth.zig - Read user from X-Plue-User-Address header
Files to Create/Modify
edge/ ├── lib/ │ ├── siwe.ts (create) - SIWE message handling │ ├── session.ts (create) - JWT session management │ └── public-routes.ts (create) - Public route patterns ├── routes/ │ └── auth.ts (create) - Auth route handlers ├── index.ts (modify) - Add auth middleware ├── types.ts (modify) - Add AUTH_KV binding └── package.json (modify) - Add siwe, ethers deps
infra/terraform/modules/cloudflare-workers/ ├── main.tf (modify) - Add AUTH_KV namespace └── variables.tf (modify) - Add jwt_secret variable
server/src/ ├── routes/auth.zig (modify) - Simplify, trust edge ├── lib/siwe.zig (modify) - Remove verification logic └── middleware/auth.zig (modify) - Read X-Plue-User-Address
Phase 2: Add Cloudflare Spectrum for SSH
Context
Current SSH server:
- server/src/ssh/server.zig - Listens on port 2222
- Uses SSH public key authentication via ssh_keys table
- Direct connection from clients (bypasses Cloudflare)
Cloudflare Spectrum:
- Proxies TCP traffic with DDoS protection
- SSH supported on Pro/Business plans (port 22)
- Can run on port 443 to bypass restrictive firewalls
- PROXY protocol preserves real client IPs
Reference: https://developers.cloudflare.com/spectrum/
Task
- Create infra/terraform/modules/cloudflare-spectrum/main.tf: terraform { required_providers { cloudflare = { source = "cloudflare/cloudflare" version = "~> 4.0" } } }
resource "cloudflare_spectrum_application" "ssh" { zone_id = var.zone_id protocol = "ssh" traffic_type = "direct"
dns {
type = "CNAME"
name = "ssh.${var.domain}"
}
origin_direct = ["tcp://${var.origin_ip}:22"]
# Enable PROXY protocol to preserve client IPs
proxy_protocol = "v1"
# Smart routing for performance
argo_smart_routing = true
# IP firewall integration
ip_firewall = true
}
resource "cloudflare_spectrum_application" "ssh_443" { zone_id = var.zone_id protocol = "tcp/443" traffic_type = "direct"
dns {
type = "CNAME"
name = "git.${var.domain}"
}
origin_direct = ["tcp://${var.origin_ip}:22"]
proxy_protocol = "v1"
argo_smart_routing = true
ip_firewall = true
} 2. Create infra/terraform/modules/cloudflare-spectrum/variables.tf: variable "zone_id" { description = "Cloudflare zone ID" type = string }
variable "domain" { description = "Base domain (e.g., plue.dev)" type = string }
variable "origin_ip" { description = "Origin server IP or hostname" type = string } 3. Create server/src/ssh/proxy_protocol.zig - Parse PROXY protocol v1: /// Parse PROXY protocol v1 header /// Format: "PROXY TCP4 <src_ip> <dst_ip> <src_port> <dst_port>\r\n" /// Reference: https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt pub fn parseProxyProtocolV1(data: []const u8) !?ProxyInfo { // Check for "PROXY " prefix // Parse protocol (TCP4/TCP6) // Extract source IP and port // Return ProxyInfo struct with real client IP }
pub const ProxyInfo = struct { client_ip: std.net.Address, client_port: u16, server_ip: std.net.Address, server_port: u16, protocol: Protocol, }; 4. Update server/src/ssh/server.zig: - On new connection, check for PROXY protocol header - Extract real client IP from PROXY header - Use real IP for logging, rate limiting, and auth checks - Fall back to connection IP if no PROXY header (local dev) 5. Update SSH port to 22 in config (Spectrum expects standard port) 6. Update GKE service to expose port 22:
spec: ports: - name: ssh port: 22 targetPort: 22
Files to Create/Modify
infra/terraform/modules/cloudflare-spectrum/ ├── main.tf (create) - Spectrum applications ├── variables.tf (create) - Module variables └── outputs.tf (create) - DNS names output
infra/terraform/environments/production/ └── main.tf (modify) - Add spectrum module
server/src/ssh/ ├── proxy_protocol.zig (create) - PROXY protocol parser └── server.zig (modify) - Use PROXY protocol
server/src/config.zig (modify) - Change SSH port to 22
Phase 3: Enable Authenticated Origin Pulls (mTLS)
Context
Currently, origin accepts connections from any IP. We need to ensure ONLY Cloudflare can reach it.
Using a custom certificate (not Cloudflare's shared cert) prevents other Cloudflare users from pointing their domains at our origin.
Reference: https://developers.cloudflare.com/ssl/origin-configuration/authenticated-origin-pull/
Task
- Create infra/scripts/generate-mtls-certs.sh: #!/bin/bash
set -euo pipefail
OUTPUT_DIR="${1:-./certs}" mkdir -p "$OUTPUT_DIR"
openssl genrsa -out "$OUTPUT_DIR/ca.key" 4096
openssl req -new -x509 -days 3650 -key "$OUTPUT_DIR/ca.key"
-subj "/CN=Plue Origin CA/O=Plue/C=US"
-out "$OUTPUT_DIR/ca.crt"
openssl genrsa -out "$OUTPUT_DIR/client.key" 4096
openssl req -new -key "$OUTPUT_DIR/client.key"
-subj "/CN=Cloudflare Client/O=Plue/C=US"
-out "$OUTPUT_DIR/client.csr"
openssl x509 -req -days 365 -in "$OUTPUT_DIR/client.csr"
-CA "$OUTPUT_DIR/ca.crt" -CAkey "$OUTPUT_DIR/ca.key"
-CAcreateserial -out "$OUTPUT_DIR/client.crt"
cat "$OUTPUT_DIR/client.crt" "$OUTPUT_DIR/client.key" > "$OUTPUT_DIR/client.pem"
echo "Certificates generated in $OUTPUT_DIR" echo "Upload client.pem to Cloudflare for Authenticated Origin Pulls" echo "Configure origin server with ca.crt to verify client certificates" 2. Create infra/terraform/modules/cloudflare-mtls/main.tf: terraform { required_providers { cloudflare = { source = "cloudflare/cloudflare" version = "~> 4.0" } } }
resource "cloudflare_authenticated_origin_pulls_certificate" "origin" { zone_id = var.zone_id certificate = var.client_certificate private_key = var.client_private_key type = "per-zone" }
resource "cloudflare_authenticated_origin_pulls" "origin" { zone_id = var.zone_id authenticated_origin_pulls_certificate = cloudflare_authenticated_origin_pulls_certificate.origin.id enabled = true } 3. Create infra/terraform/modules/cloudflare-mtls/variables.tf: variable "zone_id" { description = "Cloudflare zone ID" type = string }
variable "client_certificate" { description = "Client certificate PEM content" type = string sensitive = true }
variable "client_private_key" { description = "Client private key PEM content" type = string sensitive = true } 4. Store CA cert in Kubernetes secret:
resource "kubernetes_secret" "mtls_ca" { metadata { name = "mtls-ca" namespace = "production" }
data = {
"ca.crt" = var.mtls_ca_certificate
}
} 5. Update Zig server for mTLS - Modify TLS configuration to require client certificates: // In server/src/main.zig or config // When TLS is enabled: // - Load CA certificate from /etc/plue/certs/ca.crt // - Set client_ca to require client certificate verification // - Reject connections without valid client cert 6. Add environment flag for local development (skip mTLS in dev mode) 7. Update GKE deployment to mount CA certificate: volumeMounts: - name: mtls-ca mountPath: /etc/plue/certs readOnly: true volumes: - name: mtls-ca secret: secretName: mtls-ca
Files to Create/Modify
infra/scripts/ └── generate-mtls-certs.sh (create) - Certificate generation
infra/terraform/modules/cloudflare-mtls/ ├── main.tf (create) - AOP configuration ├── variables.tf (create) - Module variables └── outputs.tf (create) - Status outputs
infra/terraform/kubernetes/ └── secrets.tf (modify) - Add mtls-ca secret
infra/terraform/environments/production/ └── main.tf (modify) - Add mtls module
server/src/ ├── main.zig (modify) - Add mTLS config └── config.zig (modify) - Add mTLS options
Phase 4: Update Production Environment
Task
- Update infra/terraform/environments/production/main.tf:
module "cloudflare_spectrum" { source = "../../modules/cloudflare-spectrum"
zone_id = var.cloudflare_zone_id
domain = var.domain
origin_ip = module.gke.external_ip
}
module "cloudflare_mtls" { source = "../../modules/cloudflare-mtls"
zone_id = var.cloudflare_zone_id
client_certificate = var.mtls_client_cert
client_private_key = var.mtls_client_key
} 2. Add new variables to variables.tf: variable "mtls_client_cert" { description = "mTLS client certificate for Cloudflare" type = string sensitive = true }
variable "mtls_client_key" { description = "mTLS client private key for Cloudflare" type = string sensitive = true }
variable "mtls_ca_cert" { description = "mTLS CA certificate for origin verification" type = string sensitive = true } 3. Update GKE LoadBalancer - restrict to Cloudflare IPs only (defense in depth):
Testing Strategy
- Local Development: - PLUE_DEV_MODE=true skips mTLS requirement - Mock X-Plue-User-Address header for testing auth - SSH server accepts connections without PROXY protocol
- Staging: - Deploy full stack to staging namespace - Configure Spectrum for staging.plue.dev - Test SIWE flow end-to-end - Test SSH clone/push through Spectrum
- Production Migration: - Deploy with feature flag: accept both old sessions AND new JWTs - Monitor error rates - Switch DNS to use Spectrum - Remove old auth code after 1 week
Success Criteria
- All HTTP traffic authenticated at Cloudflare edge
- SIWE verification happens in Edge Worker (no origin round-trip)
- Git SSH traffic flows through Cloudflare Spectrum
- SSH server correctly parses PROXY protocol for client IPs
- Origin rejects non-Cloudflare connections (mTLS)
- Terraform applies cleanly with no manual steps
- No user-facing changes to auth flow
- DDoS protection covers all protocols
- Local development still works without Cloudflare
Rollback Plan
- SIWE at edge: Toggle feature flag to use origin SIWE verification
- Spectrum: Point DNS directly to origin IP (bypass Spectrum)
- mTLS: Disable client cert requirement in Zig server config
- All changes are infrastructure-level and can be reverted via Terraform
Sources: