Claude Code PostToolUse Typecheck Hook - Parallel Agent Experiment Report

2025-12-13-posttooluse-typecheck-hook-experiment.md

date	project	tags	agent	task
2025-12-13	poke-decomp	typecheck hooks parallel-agents effect-ts haiku sonnet claude-code	opus-orchestrator	Comprehensive experiment testing PostToolUse typecheck hooks with parallel agents

PostToolUse Typecheck Hook Experiment Report

Executive Summary

Successfully implemented and tested a PostToolUse typecheck hook that provides immediate type error feedback after every Edit/Write operation. Tested with parallel agents (both Sonnet and Haiku models) and validated that the hook feedback is clear, actionable, and not confusing even during concurrent multi-agent activity.

Key Finding: The hook works excellently for parallel agent coordination. Haiku models are fully capable of self-correcting based on hook feedback, making them viable for delegated Effect-TS work.

---

Setup

Hook Configuration

Added to /Users/ryanhunter/poke-decomp/.claude/settings.local.json:

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Edit|Write",
        "hooks": [
          {
            "type": "command",
            "command": "cd \"$CLAUDE_PROJECT_DIR\" && pnpm exec tsx /Users/ryanhunter/artimath/lever/packages/tools/typecheck-runner/src/bin.ts",
            "timeout": 60
          }
        ]
      }
    ]
  }
}

Typecheck Runner Features

The lever project's typecheck-runner provides:

• Debouncing - 2s window to avoid redundant checks on rapid edits
• Package resolution - finds nearest package.json for monorepos
• Structured output - "Edited file" vs "Neighbour files" distinction
• Error distribution - summary table showing errors per file
• Effect-TS awareness - catches missingEffectContext errors via language service

---

Experiment 1: Basic Hook Verification

Test

1. Added intentional type error: const broken: string = 123
2. Observed hook feedback
3. Fixed error
4. Observed hook feedback

Results

• ✅ Hook caught error immediately: TS2322: Type 'number' is not assignable to type 'string'
• ✅ Showed exact file:line location
• ✅ Confirmed "TypeScript check passed" after fix

---

Experiment 2: Multi-File Error Landscape

Test

Added errors across 4 files in reverse order:

1. src/platform/memory.ts (different directory) - 2 errors
2. src/ported/task.ts (same directory) - 2 errors
3. src/ported/sprite.ts (same directory) - 2 errors
4. src/ported/random.ts (original file) - 4 errors including Effect-specific

Results

Hook output structure:

▶ Edited file (random.ts - 4 of 10 errors)
  [detailed errors for just-edited file]

▶ Neighbour files
  [truncated view of nearby errors]

▶ Error distribution (4 files)
  src/ported/random.ts: 4 errors ← edited
  src/platform/memory.ts: 2 errors
  src/ported/sprite.ts: 2 errors
  src/ported/task.ts: 2 errors

Key Observations

• Effect errors surfaced: missingEffectContext detected for missing RandomService requirement
• Clear hierarchy: edited file shown first with full detail, neighbours summarized
• Distribution table: provides quick overview of error landscape

Example: Full Hook Output with 10 Errors

Here's what the actual hook output looked like after adding errors including Effect-specific ones:

🔴 TypeScript: 10 errors detected in poke-decomp

▶ Edited file (random.ts - 4 of 10 errors)
src/ported/random.ts:25:7 - error TS2322: Type 'string' is not assignable to type 'number'.

25 const randomError1: number = "oops"
         ~~~~~~~~~~~~

src/ported/random.ts:28:7 - error TS2375: Type 'Effect<number, never, never>' is not assignable
to type 'Effect<string, never, never>' with 'exactOptionalPropertyTypes: true'.
Consider adding 'undefined' to the types of the target's properties.

28 const badEffect: Effect.Effect<string> = Effect.succeed(123)
         ~~~~~~~~~

src/ported/random.ts:36:7 - error TS2375: Type 'Effect<Effect<number, never, never>, never,
RandomService>' is not assignable to type 'Effect<number, never, never>'.

36 const willFail: Effect.Effect<number, never, never> = needsService
         ~~~~~~~~

src/ported/random.ts:36:7 - error TS1: Missing 'RandomService' in the expected Effect context.
    effect(missingEffectContext)

▶ Neighbour files
src/ported/sprite.ts:18:7 - error TS2322: Type 'string' is not assignable to type '() => void'.

src/ported/sprite.ts:19:7 - error TS2322: Type '[number, number, number]' is not assignable
to type '[number, number]'.

…and 2 more errors in neighbouring files

▶ Error distribution (4 files)
  src/ported/random.ts: 4 errors ← edited
  src/platform/memory.ts: 2 errors
  src/ported/sprite.ts: 2 errors
  src/ported/task.ts: 2 errors

Key features visible:

• Effect-specific error (missingEffectContext) detected via Effect language service
• Edited file section shows full detail for just-modified file
• Neighbour files shows truncated view with "…and N more" indicator
• Distribution table gives birds-eye view with arrow marking the edited file

---

Experiment 3: Parallel Agent Chaos (Sonnet)

Setup

Spawned 2 Sonnet agents in parallel:

• AGENT-ALPHA: Add 5 errors to palette.ts, text.ts, bg.ts, gpu-regs.ts, window.ts
• AGENT-BETA: Add 5 errors to window.ts, wild-encounter.ts, trig.ts, string-util.ts, scanline-effect.ts

Both used comment prefixes (// AGENT-ALPHA ERROR N:) for attribution.

Results

Total errors introduced: 11 across 9 files (window.ts had both agents' errors)

Agent Reports:

AGENT-ALPHA observed:

• Error count progression: 2 → 4 → 7 → 9 → 11
• Saw BETA's errors appearing in "Neighbour files" section
• Got file conflict detection when BETA edited window.ts first (correct behavior!)
• Not confusing - "Edited file" vs "Neighbour files" made attribution clear

AGENT-BETA observed:

• Same error progression visibility
• Could see ALPHA's errors accumulating in real-time
• Picked end-of-alphabet files to minimize collision (worked!)
• Not confusing - parallel visibility "worked great"

File Conflict Detection

When ALPHA tried to edit window.ts after BETA had modified it:

• Got "File has been modified since read" error
• Correct behavior - prevents race conditions
• Agent recovered by re-reading and retrying

---

Experiment 4: Agent Continuation Test

Setup

Tested whether resumed agents retain their original prompt.

Part 1: Agent given secret phrase "BANANA ROCKET SHIP", wrote to exo_cortex, returned. Part 2: Resumed same agent, asked to recall secret phrase.

Results

• Original prompt NOT preserved on resume
• Resumed agent had zero memory of "BANANA ROCKET SHIP"
• Only saw Part 2 instructions
• Confirms issue #11712 is still present

Implication: Must use explicit state handoff (exo_cortex files, exo-coord mail/actions) for multi-turn agent workflows. Resume is NOT stateful continuation.

---

Experiment 5: Haiku Cleanup Test

Setup

Spawned 2 Haiku agents to fix the 11 errors from Experiment 3:

• HAIKU-CLEANER-1: Fix ALPHA's errors (palette.ts, text.ts, bg.ts)
• HAIKU-CLEANER-2: Fix BETA's errors + remaining ALPHA errors (6 files)

Results

HAIKU-CLEANER-1:

• Fixed 3/3 files successfully
• Hook showed error count: 9 → 7 → 5 as fixes applied
• Time: ~2 minutes
• 100% success rate, zero rework

HAIKU-CLEANER-2:

• Fixed 6/6 files successfully
• Hook showed error count: 5 → 4 → 3 → 2 → 1 → 0
• Time: ~1 minute
• 100% success rate, zero rework

Final typecheck: PASSED ✅

Key Finding

Haiku models are fully capable of:

• Reading hook feedback correctly
• Understanding error locations and types
• Making surgical fixes without collateral damage
• Working in parallel without confusion

This validates delegating mechanical cleanup tasks to Haiku for cost/speed optimization.

---

Hook Feedback Analysis

What Works Well

Feature	Benefit
Immediate feedback	No waiting until session end to discover broken types
"Edited file" section	Always clear which errors you just introduced
"Neighbour files" section	Spatial awareness of nearby issues
Error distribution table	Quick landscape view across codebase
Effect error detection	missingEffectContext surfaces immediately
File conflict detection	Prevents race conditions in parallel work
Debouncing	Avoids redundant checks on rapid multi-file edits

What Could Be Improved

Issue	Potential Fix
No agent attribution in output	Could add metadata tracking who introduced each error
"Blocking error" framing even on success	Minor UX confusion (says "blocking error: TypeScript check passed")
No explicit parallel activity warning	Could add "⚠️ Detected errors in files not recently edited by you"
1-2s delay per edit	Acceptable but noticeable; could explore caching

---

Recommendations

For Parallel Agent Workflows

1. Use comment prefixes for attribution (// AGENT-X ERROR:)
2. Partition files explicitly to minimize collision
3. Trust file conflict detection - it's correct behavior
4. Review "Neighbour files" section to see other agents' work
5. Monitor error count progression as implicit parallel activity signal

For Agent Delegation

1. Haiku is viable for mechanical cleanup, simple ports, error fixing
2. Sonnet for complex work - state machines, intricate Effect patterns
3. Opus for orchestration - planning, architecture decisions
4. Always persist to exo_cortex - agents are ephemeral

For Resume/Continuation

1. Don't rely on resume for context preservation
2. Use exo_cortex files for state handoff
3. Use exo-coord mail/actions for richer coordination
4. Include "boot sequence" in prompts for stateful agents

---

Files Created

File	Purpose
.claude/settings.local.json	Hook configuration
package.json	Added check script alias
~/exo_cortex/logs/2025-12-13-agent-alpha-parallel-typecheck-test.md	ALPHA's report
~/exo_cortex/logs/2025-12-13-haiku-cleaner-1-report.md	Haiku 1's report
~/exo_cortex/logs/2025-12-13-haiku-cleaner-2-report.md	Haiku 2's report
~/exo_cortex/logs/2025-12-13-continuation-test-part1.md	Continuation test part 1
~/exo_cortex/logs/2025-12-13-continuation-test-part2.md	Continuation test part 2
This file	Comprehensive experiment report

---

Conclusion

The PostToolUse typecheck hook is a significant improvement for Effect-TS development workflows:

1. Immediate feedback prevents error accumulation
2. Parallel agents work smoothly - no confusion with proper output structure
3. Haiku models can self-correct - enables cost-effective delegation
4. Effect-specific errors surface - critical for service/layer pattern compliance

The setup is production-ready for the poke-decomp project and provides a template for other Effect-TS codebases.

---

Open Questions for Future Exploration

1. Should there be a max error threshold where agents auto-pause?
2. Would agent-specific error filtering be useful? (show only my errors vs all errors)
3. Can we integrate Effect docs MCP for automated pattern lookup when agents hit Effect-specific errors?
4. Would tspc (Effect's TypeScript plugin CLI) provide better formatting than tsc?
5. How to best coordinate cleanup when multiple agents finish with overlapping errors?