E2E Shard 2/4 flaky failure: drizzle v1 'decoder' bug in grant-feature-flag.ts

gistfile0.txt

# E2E Shard 2/4 Flaky Failure: Drizzle v1 "decoder" Bug

## Problem
E2E Shard 2/4 fails intermittently across multiple PRs (confirmed on #6498 and #6548).
The root cause is the drizzle-orm v1 (beta.15) "decoder" bug affecting E2E setup scripts.

## Error
```
❌ Failed to grant feature flag: Unknown relational filter field: "decoder"
❌ Failed to set user context: Unknown relational filter field: "decoder"
```

This causes `auth.setup.ts` to timeout (38.8s) because the user context and feature flags
can't be set via scripts, forcing a slower browser-based onboarding fallback.

## Root Cause
In drizzle-orm v1 (beta.15), `SQL` objects have an enumerable `decoder` property.
When old-style `where: eq(column, value)` is passed to `db.query.X.findFirst()`,
the `relationsFilterToSQL` function iterates `Object.entries(filter)`, encounters
`decoder` as a key, and throws because it's not a recognized column or relation name.

Source: `node_modules/drizzle-orm/relations.js` lines 372-406:
```javascript
function relationsFilterToSQL(table, filter, ...) {
  for (const [target, value] of Object.entries(filter)) {
    // ...
    default: {
      if (table[TableColumns][target]) { /* column filter */ continue; }
      const relation = tableRelations[target];
      if (!relation) throw new DrizzleError({ message: `Unknown relational filter field: "${target}"` });
    }
  }
}
```

## What's Been Fixed (PR #6498)
- `set-user-context.ts`: Replaced `db.query.users.findFirst({ where: eq(...) })` with
  `db.select().from(users).where(eq(...)).limit(1)` workaround
- `grant-feature-flag.ts`: Added stack trace logging to catch block for debugging

## What Still Fails
`grant-feature-flag.ts` ALREADY uses `db.select()` (not relational queries) via a `findFirst`
helper — yet it still fails with the "decoder" error on branches that add new schema exports.

### Mystery: Why does db.select() trigger a relational error?
- The script only uses `db.select().from(table).where(where).limit(1)` and `db.insert()`
- It does NOT use `db.query.X.findFirst()` anywhere
- The `findFirst` helper was specifically written to avoid the relational API
- Yet the error `Unknown relational filter field: "decoder"` still occurs
- This ONLY happens on branches that add new tables/enums to schema.ts
  (works fine on PR #6548 which has no schema changes)

### Hypothesis
When `drizzle({ client, schema, relations })` is constructed with schema containing new
table exports, something in the initialization or import chain triggers `relationsFilterToSQL`.
The constructor calls `V1.extractTablesRelationalConfig(config.schema, ...)` from
`_relations.js` but that function only filters for Table/Relations instances and shouldn't
call `relationsFilterToSQL`.

### Files to Investigate
- `apps/api/scripts/grant-feature-flag.ts` - The failing script (uses db.select() only)
- `apps/api/src/db/index.ts` - Creates drizzle instance with schema + relations
- `apps/api/src/db/relations.ts` - defineRelations(schema, callback)
- `apps/api/src/db/schema.ts` - All table/enum exports
- `node_modules/drizzle-orm/relations.js` - relationsFilterToSQL (line 372)
- `node_modules/drizzle-orm/_relations.js` - V1 compat layer
- `node_modules/drizzle-orm/postgres-js/driver.js` - drizzle() constructor

### Reproduction
The issue is flaky but consistently appears on branches with new schema additions.
Adding stack trace logging (already done in grant-feature-flag.ts catch block) should
reveal the exact call path on the next CI failure.

## CI Links
- PR #6498 Shard 2/4 failure: https://github.com/FlatFilers/obvious/actions/runs/21808510949/job/62916061804
- PR #6548 Shard 2/4 failure (same issue, different PR): https://github.com/FlatFilers/obvious/actions/runs/21808119822