Static Asset Caching Plan for Deployed Apps (Issue #2459)

static-asset-caching-plan.md

Static Asset Caching for Deployed Apps (FastAPI + HTML Template)

Problem Statement

User-deployed apps using the data-app-html-python template serve all static assets (CSS, JS, images, fonts) through the FastAPI/Uvicorn process running on Modal. Every request -- including repeat visits for the same unchanged styles.css -- travels from browser to Cloudflare edge to Modal container to Python to filesystem and back. This creates:

1. Unnecessary latency: Static files go through the full Python middleware stack (~10x slower than native file serving)
2. Cold start exposure: After 15 min idle, Modal scales to zero. The next visitor waits 1-3+ seconds for container boot, even for a CSS file
3. Wasted compute cost: Modal bills per-second for container time. Serving static files through Python wastes expensive compute on file I/O
4. Poor repeat-visit performance: No browser-side Cache-Control headers means browsers revalidate every request

Current Architecture

Browser → slug.memex.build
            ↓
  Cloudflare Worker (auth-proxy)
            ↓
     isStaticAsset()?
        ↓ yes          ↓ no
   Check edge cache    Proxy to Modal
        ↓ miss              ↓
   Proxy to Modal     Modal container
        ↓              (FastAPI app)
   Modal container
   (FastAPI app)
        ↓
   Cache response
   (max-age=3600)

What Already Exists

The auth-proxy worker (cloudflare/auth-proxy/src/index.ts:502-613) already has partial caching:

• isStaticAsset() in utils.ts detects static files by extension (.js, .css, .png, .jpg, .jpeg, .gif, .svg, .ico, .woff, .woff2, .ttf, .eot, .map) and path prefix (/static/, /assets/, /media/)
• For cache hits, returns from caches.default (Cloudflare edge cache)
• For cache misses, fetches from Modal, stores response with Cache-Control: public, max-age=3600 (1 hour)
• Cache key uses the full Modal URL: https://memexapps-dev--slug.modal.run/static/styles.css?v=abc123

What the Template Already Does

The data-app-html-python template (routes.py) generates content-based hashes:

def get_file_hash(filepath: str) -> str:
    with open(filepath, "rb") as f:
        return hashlib.md5(f.read()).hexdigest()[:8]

And uses them in index.html:

<link rel="stylesheet" href="/static/styles.css?v={{ css_hash }}">
<script src="/static/app.js?v={{ js_hash }}"></script>

When code changes → hash changes → new URL → automatic cache miss.

Gaps

Gap	Detail
Edge cache too short for hashed assets	max-age=3600 (1 hour) even though ?v=hash guarantees freshness. Should be 1 year.
No browser-side Cache-Control	The 1-hour cache only lives on Cloudflare edge. Browser gets no caching directive -- revalidates every request.
HTML responses not explicitly controlled	index.html has <meta http-equiv="Cache-Control"> tags but browsers ignore these for HTTP caching. No actual HTTP Cache-Control header is set.
API responses not marked no-store	/api/* responses pass through without cache directives, relying on default behavior.
Unhashed user assets have no strategy	Users can add images/fonts to /static/ without going through the Jinja hash mechanism. These get the same 1-hour edge cache with no browser cache.
No cache purge on redeploy	Unhashed assets stay cached up to 1 hour after a new deploy.
Template doesn't set Cache-Control	FastAPI's StaticFiles only sets ETag/Last-Modified, not Cache-Control.

Plan

Phase 1: Cloudflare Worker -- Smarter Cache Headers (auth-proxy)

Files: cloudflare/auth-proxy/src/index.ts, cloudflare/auth-proxy/src/utils.ts

1.1 Add URL classification helpers to utils.ts

Add functions to classify requests for cache policy:

/**
 * Check if a URL has a cache-busting version parameter (?v=...)
 */
export function hasVersionParam(url: URL): boolean {
  return url.searchParams.has("v");
}

/**
 * Check if a path is an API route
 */
export function isApiRoute(pathname: string): boolean {
  return pathname.startsWith("/api/") || pathname === "/api";
}

/**
 * Check if a path is an HTML page request (root or .html extension)
 */
export function isHtmlRequest(pathname: string): boolean {
  return pathname === "/" || pathname.endsWith(".html") || pathname.endsWith("/");
}

1.2 Update proxyToModalUrl cache policy in index.ts

Replace the current flat max-age=3600 with differentiated headers:

Request Pattern	Cache-Control Header	Edge Cache TTL	Rationale
Static asset with ?v= param	public, max-age=31536000, immutable	1 year	Content hash guarantees freshness
Static asset without version param	public, max-age=300, stale-while-revalidate=86400	5 minutes	Short fresh window, 1-day background revalidation
HTML / root path (/, *.html)	no-cache	Do not cache	Always revalidate to pick up new deploys
API routes (/api/*)	private, no-store	Do not cache	Dynamic, personalized content

Implementation sketch for the caching block (~line 597-613):

if (shouldCache && response.status === 200) {
  const cache = caches.default;
  const cacheKey = new Request(fullModalUrl.toString(), request);
  const responseToCache = response.clone();
  const responseHeaders = new Headers(responseToCache.headers);

  if (hasVersionParam(url)) {
    // Hashed asset: cache aggressively (1 year)
    responseHeaders.set("Cache-Control", "public, max-age=31536000, immutable");
  } else {
    // Unhashed static asset: short cache with background revalidation
    responseHeaders.set("Cache-Control", "public, max-age=300, stale-while-revalidate=86400");
  }

  const cachedResponse = new Response(responseToCache.body, {
    status: responseToCache.status,
    statusText: responseToCache.statusText,
    headers: responseHeaders,
  });

  ctx.waitUntil(cache.put(cacheKey, cachedResponse));
}

Also add Cache-Control to non-cached responses:

// For HTML responses, set no-cache
if (isHtmlRequest(url.pathname)) {
  const responseHeaders = new Headers(response.headers);
  responseHeaders.set("Cache-Control", "no-cache");
  return new Response(response.body, {
    status: response.status,
    statusText: response.statusText,
    headers: responseHeaders,
  });
}

// For API responses, set no-store
if (isApiRoute(url.pathname)) {
  const responseHeaders = new Headers(response.headers);
  responseHeaders.set("Cache-Control", "private, no-store");
  return new Response(response.body, {
    status: response.status,
    statusText: response.statusText,
    headers: responseHeaders,
  });
}

1.3 Add tests

Add unit tests for the new helper functions in utils, and integration-style tests verifying the correct Cache-Control headers are set for each request type.

Phase 2: Template -- Add Cache-Control to FastAPI StaticFiles

Repo: memextech/data-app-html-python Files: routes.py

This ensures proper browser caching even when apps are accessed directly (dev, or if Cloudflare is bypassed).

2.1 Subclass StaticFiles with Cache-Control

from starlette.responses import FileResponse
from starlette.staticfiles import StaticFiles


class CachedStaticFiles(StaticFiles):
    """StaticFiles with Cache-Control headers based on version parameter."""

    async def get_response(self, path: str, scope):
        response = await super().get_response(path, scope)
        if isinstance(response, FileResponse):
            query = scope.get("query_string", b"").decode()
            if "v=" in query:
                response.headers["Cache-Control"] = "public, max-age=31536000, immutable"
            else:
                response.headers["Cache-Control"] = "public, max-age=300"
        return response

2.2 Update create_app to use CachedStaticFiles

def create_app(static_dir: str) -> FastAPI:
    # ... existing code ...
    app.mount("/static", CachedStaticFiles(directory=static_dir), name="ui")
    return app

2.3 Set Cache-Control on index.html response

@app.get("/", response_class=HTMLResponse)
def index(request: Request):
    css_hash = get_file_hash(os.path.join(static_dir, "styles.css"))
    js_hash = get_file_hash(os.path.join(static_dir, "app.js"))
    response = templates.TemplateResponse(
        request, "index.html", {"css_hash": css_hash, "js_hash": js_hash}
    )
    response.headers["Cache-Control"] = "no-cache"
    return response

Phase 3: Cache Purge on Redeploy (Optional)

Files: backend/memex-backend-all/modal_backend/deployment_manager.py, backend/memex-backend-core/memex_backend_core/cloudflare/kv_manager.py

After a successful deploy, purge Cloudflare's edge cache for the slug's static assets so unhashed resources update immediately.

3.1 Add cache purge to _update_slug_mapping()

After the KV write in deployment_manager.py, call the Cloudflare cache purge API:

# Purge cached static assets for this slug after deploy
domain_base = self._get_domain_base()
slug_url = f"https://{slug}.{domain_base}"
await self._purge_cloudflare_cache(slug_url)

3.2 Implement _purge_cloudflare_cache()

Use the Cloudflare API to purge by prefix:

async def _purge_cloudflare_cache(self, slug_base_url: str) -> None:
    """Purge Cloudflare edge cache for a deployed app's static assets."""
    try:
        # Cloudflare Zone Cache Purge API
        # https://developers.cloudflare.com/api/resources/cache/methods/purge/
        zone_id = os.environ.get("CLOUDFLARE_ZONE_ID")
        api_token = os.environ.get("CLOUDFLARE_API_TOKEN")
        if not zone_id or not api_token:
            logger.warning("Cloudflare cache purge skipped: missing credentials")
            return

        async with httpx.AsyncClient() as client:
            resp = await client.post(
                f"https://api.cloudflare.com/client/v4/zones/{zone_id}/purge_cache",
                headers={"Authorization": f"Bearer {api_token}"},
                json={"prefixes": [f"{slug_base_url}/static/"]},
            )
            if resp.status_code == 200:
                logger.info(f"Cloudflare cache purged for {slug_base_url}/static/")
            else:
                logger.warning(f"Cloudflare cache purge failed: {resp.status_code} {resp.text}")
    except Exception:
        logger.warning("Cloudflare cache purge failed", exc_info=True)

Note: Prefix-based purge requires a Cloudflare Enterprise plan. On the free/pro plan, alternatives are:

• Purge individual URLs (up to 30 per API call) -- list known static files
• Purge everything for the zone (affects all slugs -- not ideal)
• Rely on the max-age=300 TTL for unhashed assets (5 min staleness is acceptable)

If not on Enterprise, skip Phase 3 and rely on the short TTL from Phase 1.

Expected Impact

Metric	Before	After Phase 1	After Phase 1+2
Hashed asset repeat load	Full round-trip to Modal every hour	Served from browser cache for 1 year	Same + works without Cloudflare
Unhashed asset repeat load	Full round-trip to Modal every hour	5 min browser cache + 1 day background revalidation	Same + works without Cloudflare
HTML page load	No cache directive (browser default)	Explicit no-cache -- revalidates with 304	Same + FastAPI sets header too
Cold start exposure	All requests hit Modal	Only first visit per edge PoP + unhashed assets every 5 min	Same
Modal compute cost	Every request = container time	Hashed assets: ~0 after first load. Unhashed: ~1/300th of requests	Same

Rollout Plan

1. Phase 1 (auth-proxy): Deploy to dev (*.memex.host) first. Test with existing deployed apps. Monitor Cloudflare analytics for cache hit ratio. Deploy to prod (*.memex.build).
2. Phase 2 (template): Update memextech/data-app-html-python template. New deploys pick it up automatically. Existing deployed apps benefit from Phase 1 alone.
3. Phase 3 (cache purge): Implement only if unhashed asset staleness is a user-reported problem after Phase 1+2.

Open Questions

1. Cloudflare plan tier: Is the Cloudflare zone on free, pro, or enterprise? This determines whether prefix-based cache purge is available.
2. Existing deployed apps: Phase 1 benefits all existing apps immediately (Cloudflare worker change). Phase 2 only benefits new deploys from the updated template. Should we backfill existing apps?
3. Other templates: Does the Streamlit template have similar static asset caching needs? Streamlit serves its own React SPA, which has its own caching characteristics.