Created
February 10, 2026 21:04
-
-
Save denniswon/bf1ababf29d5669fee9ced42736185b9 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- | |
| Repository Changes Audit: Impact on Newton Privacy Layer | |
| ★ Insight ───────────────────────────────────── | |
| This analysis compares the current codebase (as of Feb 10, 2026) against the Linear issues and Notion docs created on January 17, 2026. | |
| Since then, ~70 non-merge commits landed, including major architectural features like Two-Phase Consensus, Two-Digest System, direct | |
| attestation validation, and significant gateway refactoring. These changes strengthen the privacy layer integration rather than invalidate | |
| it, but several Linear issue descriptions reference outdated struct names, missing file paths, and don't account for new architecture. | |
| ───────────────────────────────────────────────── | |
| --- | |
| 1. Major Architectural Changes Since January 17 | |
| ┌──────────────────────────────┬───────────────────┬──────────────────────────────────────────────────────────────────────────────────────┐ | |
| │ Change │ Commits │ Impact on Privacy Layer │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ Two-Phase Consensus fully │ 1794b22d, │ Privacy integration point now exists in concrete code (not theoretical). NEWT-188 │ | |
| │ implemented │ 6fdee404, │ can reference actual methods. │ | |
| │ │ 1db1f6f0 │ │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ Two-Digest System │ 1794b22d, │ Privacy data flows through consensus/full digest split. Linear issues should │ | |
| │ │ 8356d1db │ reference this. │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ PolicyDetails → PolicyState │ 33fd5c10 │ Any docs/issues referencing PolicyDetails must be updated. │ | |
| │ rename │ │ │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ signature_data on │ 55756247 │ New field for validateAttestationDirect. Privacy mode may need to interact with │ | |
| │ CreateTaskResponse │ │ this. │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ Direct attestation │ e2517b2b, │ New validateAttestationDirect path. Privacy/ephemeral tasks that skip on-chain │ | |
| │ validation │ fdf0a04d, │ submission need to consider whether this path is used. │ | |
| │ │ aef0206e │ │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ Gateway operator worker │ 1db1f6f0, │ GatewayOperatorWorker with Prepare/FetchPolicyData/Commit task variants. Privacy │ | |
| │ pattern │ 06a1d2e0 │ decryption must integrate with this worker pattern too. │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ Unified quorum tracking │ 06a1d2e0 │ broadcast_with_unified_quorum() replaces old broadcast methods. Privacy integration │ | |
| │ │ │ must use new APIs. │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ Multiple policy data │ 361edeb4 │ Each policy can now have multiple PolicyData entries. Privacy decryption must handle │ | |
| │ addresses │ │ per-PolicyData encryption. │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ Configurable │ 52c091e9 │ New DataProviderConfig.max_policy_data_limit field. │ | |
| │ max_policy_data_limit │ │ │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ BlockCache hardening │ 5726024d │ Gateway block number caching with retry. Privacy operations use this for │ | |
| │ │ │ taskCreatedBlock. │ | |
| ├──────────────────────────────┼───────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ | |
| │ Transaction submission │ 9cefd445 │ Background worker for on-chain submission. Privacy "ephemeral" mode skips this. │ | |
| │ worker │ │ │ | |
| └──────────────────────────────┴───────────────────┴──────────────────────────────────────────────────────────────────────────────────────┘ | |
| --- | |
| 2. Specific Linear Issue Discrepancies | |
| NEWT-431: encrypted_user_data DB Table (Urgent) | |
| Status: Still valid, but should reference existing patterns. | |
| Updates needed: | |
| - Reference existing migration pattern in crates/gateway/migrations/ (two existing migrations: | |
| 20251213100000_api_keys_user_pk_address_api_key.sql and 20251213100500_create_policy_client_secret.sql) | |
| - Reference existing repository pattern in crates/core/src/database/ (ApiKeyRepository, WasmSecretRepository, PolicyClientOwnerRepository) | |
| - Reference DatabaseManager singleton with OnceLock pattern (crates/core/src/database/mod.rs) | |
| - The existing policy_client_secret table shows the pattern for scoped-to-policy-client encrypted storage — the new encrypted_user_data | |
| table should follow the same FK relationship to policy_client_owner | |
| NEWT-432: HPKE Decryption in Operator fetch_policy_data (Urgent) | |
| Status: Needs significant update. | |
| Updates needed: | |
| - The handler is now in crates/operator/src/core.rs method fetch_policy_data() (lines ~450-496), not a standalone handler file | |
| - DataProviderGenerator wraps DataProvider and implements PolicyTaskDataGenerator trait | |
| - The actual data fetching is DataProvider::get_policy_task_data_for_client() in crates/data-provider/src/lib.rs | |
| - HPKE decryption should happen inside process_single_policy_data() (after WASM execution fetches encrypted data) or as a pre-step before | |
| WASM execution | |
| - The existing SecretsProvider pattern (crates/data-provider/src/wasm/secrets.rs) shows how secrets are injected into WASM execution — HPKE | |
| decryption could follow a similar provider pattern | |
| - ConsensusPrepareResponse struct is already defined with policy_task_data, data_hash, timestamp | |
| NEWT-433: Auth Signature Validation in Gateway (Urgent) | |
| Status: Still valid. Authorization validation happens in create_task() in crates/gateway/src/rpc/api/sync.rs. | |
| Updates needed: | |
| - Current auth is via API keys (validate_api_key_permissions at line ~77-81 of sync.rs) | |
| - Ed25519 signature validation would be a new layer on top of API key auth | |
| - Reference the existing GatewayServiceImpl struct in crates/gateway/src/handler/mod.rs where the auth module would be injected | |
| - Note that both Two-Phase and Centralized paths go through create_task(), so auth validation at the top covers both | |
| NEWT-434: Privacy Fields in CreateTaskRequest (High) | |
| Status: Needs update for current struct definition. | |
| Updates needed: | |
| - Current CreateTaskRequest fields (from crates/gateway/src/rpc/types/mod.rs): | |
| policy_client, intent, intent_signature, quorum_number, | |
| quorum_threshold_percentage, wasm_args, timeout, use_two_phase | |
| - New privacy fields (privacy_mode, encrypted_data_ref, auth_signatures, skip_onchain) would be siblings to use_two_phase | |
| - The use_two_phase: Option<bool> pattern shows exactly how to add optional privacy mode toggles | |
| - CreateTaskResponse now has signature_data, task, task_response, reference_block, expiration fields — privacy response may need a | |
| privacy_proof or similar | |
| NEWT-435: Secure Envelope Storage API (High) | |
| Status: Still valid. Should reference existing secrets upload pattern. | |
| Updates needed: | |
| - Reference the existing newt_storeEncryptedSecrets RPC endpoint in crates/gateway/src/processor/secrets_upload.rs as the pattern to follow | |
| - The SecretsUploadValidator trait pattern (decrypt, validate, store) is exactly what the Secure Envelope storage API needs | |
| - Existing ownership enforcement via policy_client_owner table applies to privacy data too | |
| - The PublishSecretsEncryptedRequest struct shows the RPC pattern | |
| NEWT-438: Encrypted Data Ref in PolicyTaskData (High) | |
| Status: Needs update for current contract types. | |
| Updates needed: | |
| - PolicyDetails has been renamed to PolicyState (commit 33fd5c10) | |
| - PolicyTaskData is defined in Solidity bindings as NewtonMessage::PolicyTaskData with fields: policyId, policyAddress, policy, | |
| policyData[] | |
| - PolicyData has: data, attestation, policyDataAddress, expireBlock | |
| - Adding an encryptedDataRef field to PolicyData or PolicyTaskData would require a contract change and binding regeneration | |
| - Consider whether the encrypted ref goes in the Solidity struct (requires contract upgrade) or only in the Rust-side | |
| ConsensusPrepareRequest/ConsensusCommitRequest structs | |
| NEWT-184: Offchain Authorization Module (High) | |
| Status: Still valid. No existing auth module beyond API keys. | |
| NEWT-185: Secure Envelope Data Structure with AAD (High) | |
| Status: Still valid. No existing HPKE types in codebase. | |
| Updates needed: | |
| - Should be placed in crates/core/src/ (new module, e.g., privacy/) | |
| - Reference the existing type pattern: domain types in crates/core/src/common/, config in crates/core/src/config/ | |
| NEWT-187: HPKE Encryption Module (High) | |
| Status: Still valid. No HPKE code exists. | |
| NEWT-188: Privacy Extension for Two-Phase Consensus (High, Epic) | |
| Status: Needs significant update — this is the most affected issue. | |
| Updates needed: | |
| - Two-Phase is now fully implemented. The integration points are concrete: | |
| - Prepare Phase: broadcast_two_phase_and_aggregate() in sync.rs lines 703-1127 | |
| - Consensus: compute_consensus_from_unsigned() in consensus.rs lines 400-558 | |
| - Commit Phase: build_evaluate_and_sign_request() in sync.rs lines 885-1024 | |
| - Gateway operator worker (GatewayOperatorWorker in worker.rs) needs privacy-aware task variants | |
| - The OperatorPool trait methods (broadcast_fetch_policy_data_unified, broadcast_with_unified_quorum) need encrypted data forwarding | |
| - ConsensusCommitRequest struct is the vehicle for passing consensus data — privacy fields would extend this | |
| - Two-Digest System means encrypted data ref needs to be in the consensus digest (or excluded like attestations — design decision) | |
| NEWT-436: Privacy Metrics (Medium) | |
| Status: Still valid. | |
| Updates needed: | |
| - Reference existing Two-Phase metrics pattern in crates/metrics/src/lib.rs lines 175-218 | |
| - Privacy metrics would follow the same describe_histogram!/describe_counter! pattern | |
| - Recording functions pattern: record_* and inc_* functions | |
| NEWT-437: E2E Tests (Medium) | |
| Status: Needs update for current test infrastructure. | |
| Updates needed: | |
| - Reference unified E2E test pattern in integration-tests/src/gateway.rs | |
| - GatewayTestSetup, TaskRequestConfig, TaskRequestResult structs | |
| - AttestationValidationMode enum (Skip, Regular, Direct) | |
| - Test modes: GATEWAY_AS_OPERATOR, TWO_PHASE_CONSENSUS, VALIDATE_ATTESTATION env vars | |
| - Privacy E2E tests would add a PRIVACY_MODE env var following existing patterns | |
| --- | |
| 3. Notion Document Accuracy | |
| Privacy Litepaper | |
| - Mostly accurate. The Feb 2026 update correctly describes Two-Phase integration. | |
| - Minor update: The doc references "policy evaluation" happening in Prepare Phase — in current code, Prepare Phase only fetches data | |
| (newt_fetchPolicyData); evaluation happens in Commit Phase (newt_evaluateAndSign). The privacy litepaper's description of "decrypt in | |
| Prepare, evaluate in Commit" is correct. | |
| Signature-Gated Decryption Whitepaper | |
| - Still accurate. Crypto primitives (HPKE X25519/HKDF/ChaCha20, Ed25519) are implementation choices that don't depend on the codebase | |
| changes. | |
| - Update recommended: Add a section on how SGD interacts with the Two-Digest System (should the encrypted data ref be in the consensus | |
| digest or the full digest?). | |
| MPC-backed Private-Data Registry & NCDH | |
| - Phase 3 documents — not immediately affected by Phase 1A code changes. Still accurate for future work. | |
| --- | |
| 4. Missing Documentation | |
| The PRIVACY_LAYER_AUDIT.md file referenced in CONSENSUS.md (line 524, 532) does not exist. This should be created as part of Phase 1A work | |
| to document the privacy architecture decisions. | |
| --- | |
| 5. Assessment: Is the Plan Still Correct? | |
| Yes, the implementation plan is fundamentally correct. The dependency ordering is still valid: | |
| Core types (NEWT-185, NEWT-187) | |
| → DB layer (NEWT-431) | |
| → Gateway auth + RPC (NEWT-433, NEWT-434, NEWT-435) | |
| → Operator decryption (NEWT-432, NEWT-438) | |
| → Two-Phase integration (NEWT-188) | |
| → Metrics + E2E (NEWT-436, NEWT-437) | |
| However, the issue descriptions need updates to reference current struct names, file paths, and architectural patterns. The biggest impact | |
| is on NEWT-188 (Two-Phase privacy extension) which can now reference concrete code instead of theoretical architecture. | |
| --- | |
| 6. Recommended Updates Summary | |
| Issue: NEWT-431 | |
| Update Priority: Low | |
| What to Update: Add reference to existing migration/repository patterns | |
| ──────────────────────────────────────── | |
| Issue: NEWT-432 | |
| Update Priority: High | |
| What to Update: Update handler path, reference OperatorCore::fetch_policy_data(), DataProvider::process_single_policy_data() | |
| ──────────────────────────────────────── | |
| Issue: NEWT-433 | |
| Update Priority: Low | |
| What to Update: Reference GatewayServiceImpl and current auth flow | |
| ──────────────────────────────────────── | |
| Issue: NEWT-434 | |
| Update Priority: Medium | |
| What to Update: Update CreateTaskRequest field list, note use_two_phase pattern | |
| ──────────────────────────────────────── | |
| Issue: NEWT-435 | |
| Update Priority: Medium | |
| What to Update: Reference secrets_upload.rs as pattern, existing ownership model | |
| ──────────────────────────────────────── | |
| Issue: NEWT-438 | |
| Update Priority: High | |
| What to Update: Fix PolicyDetails → PolicyState, note contract upgrade implications | |
| ──────────────────────────────────────── | |
| Issue: NEWT-184 | |
| Update Priority: Low | |
| What to Update: Still accurate | |
| ──────────────────────────────────────── | |
| Issue: NEWT-185 | |
| Update Priority: Low | |
| What to Update: Still accurate, add module placement guidance | |
| ──────────────────────────────────────── | |
| Issue: NEWT-187 | |
| Update Priority: Low | |
| What to Update: Still accurate | |
| ──────────────────────────────────────── | |
| Issue: NEWT-188 | |
| Update Priority: Critical | |
| What to Update: Rewrite with concrete code references to Two-Phase implementation | |
| ──────────────────────────────────────── | |
| Issue: NEWT-436 | |
| Update Priority: Low | |
| What to Update: Reference current metrics patterns | |
| ──────────────────────────────────────── | |
| Issue: NEWT-437 | |
| Update Priority: Medium | |
| What to Update: Reference unified E2E test infrastructure | |
| ──────────────────────────────────────── | |
| Issue: CONSENSUS.md | |
| Update Priority: Medium | |
| What to Update: Create the referenced PRIVACY_LAYER_AUDIT.md | |
| ★ Insight ───────────────────────────────────── | |
| Key takeaway: The Jan 17 design was prescient — it correctly anticipated integrating privacy into the Two-Phase Consensus flow. The | |
| codebase has since implemented that flow, which means the privacy layer now has concrete hooks to attach to rather than needing to build | |
| the consensus infrastructure from scratch. The main risk is stale references in issue descriptions leading to confusion during | |
| implementation. Updating the 3 high/critical issues (NEWT-432, NEWT-438, NEWT-188) before starting implementation will save significant | |
| debugging time. | |
| ───────────────────────────────────────────────── | |
| Would you like me to proceed with updating the Linear issues and creating the missing PRIVACY_LAYER_AUDIT.md, or shall we start | |
| implementing the Phase 1A issues directly (updating references as we go)? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment