Note: This is the canonical spec. All scripts are embedded here with their full content. When editing scripts, update this spec to include the complete file content — no stubs or references to external files.
A two-layer memory system for Claude Code that compounds knowledge across sessions. Primary mechanism is proactive in-session note-taking (via CLAUDE.md instructions). Safety net is a Stop hook that spawns background Sonnet to extract missed learnings from transcripts. Periodic Opus GC prevents unbounded growth.
┌─────────────────────────────────────────────────────────┐
│ During Session │
│ │
│ CLAUDE.md instructs Claude to proactively update │
│ MEMORY.md whenever it discovers something non-obvious │
│ │
├─────────────────────────────────────────────────────────┤
│ After Each Stop │
│ │
│ Stop hook fires → guard checks (≥5 msgs, 1hr cooldown)│
│ → background subshell (fully detached via disown) │
│ → Python filters transcript (98% noise removal) │
│ → Sonnet extracts learnings → writes to: │
│ • Project MEMORY.md (project-specific) │
│ • Global memory (cross-project) │
│ → Every 10th extraction triggers Opus GC pass │
│ │
├─────────────────────────────────────────────────────────┤
│ GC Pass (every 10 extractions) │
│ │
│ gc-memory.sh → reads memory + archive files │
│ → Opus consolidates, prunes, promotes from archive │
│ → 📌 pinned items are never pruned │
│ → Pruned items → archive (cold storage, never deleted) │
│ → Archived items promoted back if still relevant │
│ → Skips if both active files under threshold │
│ │
├─────────────────────────────────────────────────────────┤
│ On Session Start │
│ │
│ SessionStart hook checks codemap freshness │
│ MEMORY.md auto-loaded into system prompt │
│ Global memory auto-loaded into system prompt │
│ │
├─────────────────────────────────────────────────────────┤
│ CLI Tool │
│ │
│ claude-memory show|search|log|stats|extract|gc │
│ Search across all transcripts, view extraction log │
└─────────────────────────────────────────────────────────┘
| File | Scope | Auto-loaded | Written by |
|---|---|---|---|
~/.claude/projects/<encoded-path>/memory/MEMORY.md |
Per-project | Yes (that project only) | Claude in-session + Stop hook + GC |
~/.claude/projects/<encoded-path>/memory/ARCHIVE.md |
Per-project | No (cold storage) | GC only |
~/.claude/rules/global-memory.md |
All projects | Yes (everywhere) | Stop hook + GC |
~/.claude/memory/global-archive.md |
All projects | No (cold storage) | GC only |
~/.claude/CLAUDE.md |
All projects | Yes (everywhere) | Human only |
Path encoding: /Users/alice/myproject → -Users-alice-myproject
| File | Soft target | Hard cap | Enforced by |
|---|---|---|---|
| Project MEMORY.md | 150 lines | 200 lines | GC pass (Opus) + extraction prompt |
| Global memory | 50 lines | 80 lines | GC pass (Opus) + extraction prompt |
Growth prevention mechanisms:
- Extraction prompt instructs Sonnet to "keep under 200/100 lines" (soft — often ignored as files grow)
- System prompt truncates display after 200 lines (cosmetic only — doesn't prune the file)
- GC pass (every 10 extractions): Opus prunes to targets, archives pruned items to cold storage (hard enforcement)
- Manual GC:
claude-memory gcruns the GC pass on demand
Archive system:
- Pruned items go to
ARCHIVE.md/global-archive.md— never truly deleted - Archives are read during GC passes — Opus can promote items back if still relevant
- Archives grow unbounded (cold storage, never auto-loaded, zero context cost)
- Global archive stored at
~/.claude/memory/global-archive.md(NOT inrules/— avoids auto-loading)
Pin rules (📌):
- In PROJECT memory: 📌 items are never pruned or archived. They don't count toward line targets.
- In GLOBAL memory: 📌 items CAN be demoted/archived if project-specific. They DO count toward line targets.
- Only the human or in-session Claude should add 📌 pins. The extraction agent (Sonnet) must NEVER add pins.
- Global should have very few pins — only truly universal patterns.
- Claude Code CLI installed and authenticated
- Python 3 (for transcript filtering)
jq(for SessionStart hook JSON output)~/bin/on PATH (symlink to CLI tool; canonical location is~/.claude/bin/claude-memory)
mkdir -p ~/.claude/hooks/memory-persistence
mkdir -p ~/.claude/bin
mkdir -p ~/.claude/rules
mkdir -p ~/binThis is a thin wrapper that delegates to claude-memory extract. All extraction logic (transcript filtering, rate limiting, background Sonnet invocation, GC triggering) lives in the CLI tool.
#!/bin/bash
# Extract learnings from session transcript into project MEMORY.md
# Runs on Stop hook as safety net for the continuous learning loop.
#
# Primary mechanism: Claude proactively updates MEMORY.md during sessions
# (via CLAUDE.md instructions). This catches anything missed.
#
# This hook delegates to claude-memory CLI for extraction logic.
# Opt-out: CLAUDE_SKIP_SESSION_LEARNINGS=1
set -euo pipefail
[ "${CLAUDE_SKIP_SESSION_LEARNINGS:-}" = "1" ] && exit 0
project_cwd="${CLAUDE_PROJECT_DIR:-$(pwd)}"
[ -z "$project_cwd" ] && exit 0
# Always exit 0 — Stop hooks should never fail (nothing the user can do about it).
# claude-memory extract handles skip conditions (no transcripts, cooldown) internally.
"${HOME}/.claude/bin/claude-memory" extract || trueMake executable: chmod +x ~/.claude/hooks/memory-persistence/extract-learnings.sh
Garbage-collects memory files using Opus. Pruned items go to archive files (cold storage). Archives are read during GC for potential promotion back. In project memory, 📌 items are never pruned. In global memory, 📌 items CAN be demoted/archived if project-specific. Triggered automatically every 10 extractions, or manually via claude-memory gc.
#!/bin/bash
# Garbage-collect / consolidate MEMORY.md files using Opus.
# Prunes stale, redundant, and low-value entries. Consolidates related items.
# Pruned items go to archive files (cold storage) — never truly deleted.
# Archive is read during GC for potential promotion back to active memory.
# 📌 items are pinned and never pruned.
#
# Usage:
# gc-memory.sh [project_dir] # GC project + global memory
# gc-memory.sh --dry-run [dir] # Preview what Opus would output
#
# Called automatically by extract-learnings.sh every 10 extractions,
# or manually via: claude-memory gc
set -euo pipefail
mkdir -p /tmp/claude
LOG="/tmp/claude/extract-learnings.log"
dry_run=false
project_dir=""
for arg in "$@"; do
case "$arg" in
--dry-run) dry_run=true ;;
*) project_dir="$arg" ;;
esac
done
project_dir="${project_dir:-${CLAUDE_PROJECT_DIR:-$(pwd)}}"
encoded_path=$(echo "$project_dir" | sed 's|/|-|g')
claude_project_dir="${HOME}/.claude/projects/${encoded_path}"
memory_file="${claude_project_dir}/memory/MEMORY.md"
archive_file="${claude_project_dir}/memory/ARCHIVE.md"
global_memory_file="${HOME}/.claude/rules/global-memory.md"
global_archive_file="${HOME}/.claude/memory/global-archive.md"
# Read existing memory
existing_memory=""
if [ -f "$memory_file" ] && [ -s "$memory_file" ]; then
existing_memory=$(cat "$memory_file")
memory_lines=$(wc -l < "$memory_file" | tr -d ' ')
else
memory_lines=0
fi
existing_global=""
if [ -f "$global_memory_file" ] && [ -s "$global_memory_file" ]; then
existing_global=$(cat "$global_memory_file")
global_lines=$(wc -l < "$global_memory_file" | tr -d ' ')
else
global_lines=0
fi
# Read archives (cold storage context for Opus)
existing_archive=""
if [ -f "$archive_file" ] && [ -s "$archive_file" ]; then
existing_archive=$(cat "$archive_file")
archive_lines=$(wc -l < "$archive_file" | tr -d ' ')
else
archive_lines=0
fi
existing_global_archive=""
if [ -f "$global_archive_file" ] && [ -s "$global_archive_file" ]; then
existing_global_archive=$(cat "$global_archive_file")
global_archive_lines=$(wc -l < "$global_archive_file" | tr -d ' ')
else
global_archive_lines=0
fi
# Skip if both are small enough
if [ "$memory_lines" -le 150 ] && [ "$global_lines" -le 80 ]; then
echo "[$(date '+%Y-%m-%d %H:%M:%S')] [GC] Skipped — project ${memory_lines} lines, global ${global_lines} lines (both under threshold)" >> "$LOG"
if [ "$dry_run" = true ]; then
echo "Both files under threshold (project: ${memory_lines}/150, global: ${global_lines}/80). No GC needed."
fi
exit 0
fi
echo "[$(date '+%Y-%m-%d %H:%M:%S')] [GC] Starting — project ${memory_lines} lines, global ${global_lines} lines (archives: ${archive_lines}/${global_archive_lines} lines)" >> "$LOG"
if [ "$dry_run" = true ]; then
echo "=== Memory GC Preview ==="
echo "Project memory: ${memory_lines} lines (target: ≤150)"
echo "Global memory: ${global_lines} lines (target: ≤80)"
echo "Project archive: ${archive_lines} lines (cold storage)"
echo "Global archive: ${global_archive_lines} lines (cold storage)"
echo ""
fi
cd "$project_dir" 2>/dev/null || cd "$HOME"
output=$(claude -p --model opus --no-session-persistence \
--tools "" \
--disable-slash-commands \
--strict-mcp-config \
--settings '{"disableAllHooks":true}' \
--system-prompt "You are a memory curator. You CONSOLIDATE, PRUNE, and PROMOTE.
## Rules
### Project memory pins
- 📌 items in PROJECT memory are pinned. Never prune, never archive. Copy them through unchanged.
### Global memory pins — DIFFERENT RULES
- 📌 items in GLOBAL memory CAN be demoted (pin removed) or archived if they are project-specific.
- The global file is badly over-pinned. Most items should NOT be pinned in global.
- Only keep 📌 in global for truly universal patterns (e.g., Claude Code sandbox behavior, basic Docker commands).
- Project-specific items (WordPress, CRM APIs, domain tooling, specific frameworks) should be ARCHIVED from global even if pinned. They belong in project memory, not global.
### General rules
- Items you remove from active memory MUST go to the newly archived section — never delete outright.
- Read the archive for context. If an archived item is clearly still relevant to the current project, promote it back into active memory (without the archive timestamp).
- Don't promote aggressively — only items that would save significant time if forgotten.
## Output Format
Output four sections separated by delimiters. No preamble, no explanation, no markdown fences around the whole output.
<updated project memory>
---GLOBAL_MEMORY_SEPARATOR---
<updated global memory>
---PROJECT_ARCHIVE_SEPARATOR---
<items pruned from project memory this pass>
---GLOBAL_ARCHIVE_SEPARATOR---
<items pruned from global memory this pass>
If no items were pruned from a section, output (empty) after that separator.
## Targets
- Project memory: ≤150 lines (hard cap: 200)
- Global memory: ≤50 lines (hard cap: 80). This is a SMALL file — be ruthless.
- In PROJECT memory, 📌 items don't count toward targets.
- In GLOBAL memory, 📌 items DO count toward the 50-line target. Remove pins from project-specific items.
## What to PRUNE from global (move to archive)
- Project-specific items: WordPress, WooCommerce, Freshsales, Make.com, tmux, specific API details — these are project knowledge, not global
- Redundant entries: multiple entries saying the same thing
- Overly specific details: exact file paths, line numbers, variable names
- One-time fixes that won't recur in other projects
- Obvious patterns any experienced developer would know
- Implementation details better found by reading docs
- Items that only applied to one project context
## What to KEEP in global
- Claude Code behavioral quirks (sandbox, hooks, background tasks)
- Universal Docker/Git/CI patterns that bite everyone
- Cross-cutting dev patterns (error handling, testing, debugging)
- User workflow preferences (commit style, tool choices)
- Keep entries TERSE — one line each, no verbose explanations
## What to PRUNE from project memory
- Redundant entries
- Stale information about changed code
- Duplicates of what's already in global
- Obvious patterns
## What to KEEP in project memory
- Genuinely surprising gotchas for this specific project
- API quirks, data format details, environment setup
- Patterns that apply repeatedly in this codebase
- Debugging insights specific to this project's stack
- Any 📌 item (unconditionally in project memory)
## How to consolidate
- Merge related entries under shared headers
- Compress verbose explanations into terse, actionable notes
- Remove examples when the rule is self-explanatory
- Prefer 'Do X' over 'When Y happens, you should do X because Z'" \
"PROJECT MEMORY.MD (${memory_lines} lines — target ≤150):
${existing_memory:-<empty>}
---
GLOBAL MEMORY (${global_lines} lines — target ≤80):
${existing_global:-<empty>}
---
PROJECT ARCHIVE (cold storage — ${archive_lines} lines, review for promotions):
${existing_archive:-<empty — no archived items yet>}
---
GLOBAL ARCHIVE (cold storage — ${global_archive_lines} lines, review for promotions):
${existing_global_archive:-<empty — no archived items yet>}
---
Consolidate and prune both active memory files. Move pruned items to the archive sections. Promote archived items back if they're clearly still relevant. Output all four sections." 2>&1)
if [ "$dry_run" = true ]; then
echo "=== Opus Output ==="
echo "$output"
exit 0
fi
timestamp=$(date '+%Y-%m-%d %H:%M:%S')
if [ -n "$output" ] && [ ${#output} -gt 50 ]; then
# Split output into 4 sections
# 1: project memory (before GLOBAL_MEMORY_SEPARATOR)
# 2: global memory (between GLOBAL_MEMORY_SEPARATOR and PROJECT_ARCHIVE_SEPARATOR)
# 3: project archive additions (between PROJECT_ARCHIVE_SEPARATOR and GLOBAL_ARCHIVE_SEPARATOR)
# 4: global archive additions (after GLOBAL_ARCHIVE_SEPARATOR)
project_part=$(echo "$output" | sed -n '1,/---GLOBAL_MEMORY_SEPARATOR---/p' | sed '$d')
rest_after_global_sep=$(echo "$output" | sed -n '/---GLOBAL_MEMORY_SEPARATOR---/,$p' | sed '1d')
global_part=$(echo "$rest_after_global_sep" | sed -n '1,/---PROJECT_ARCHIVE_SEPARATOR---/p' | sed '$d')
rest_after_proj_archive=$(echo "$rest_after_global_sep" | sed -n '/---PROJECT_ARCHIVE_SEPARATOR---/,$p' | sed '1d')
project_archive_new=$(echo "$rest_after_proj_archive" | sed -n '1,/---GLOBAL_ARCHIVE_SEPARATOR---/p' | sed '$d')
global_archive_new=$(echo "$rest_after_proj_archive" | sed -n '/---GLOBAL_ARCHIVE_SEPARATOR---/,$p' | sed '1d')
# Write updated project memory
if [ -n "$project_part" ] && [ ${#project_part} -gt 50 ]; then
new_lines=$(echo "$project_part" | wc -l | tr -d ' ')
echo "$project_part" > "$memory_file"
echo "[$timestamp] [GC] Project memory: ${memory_lines} → ${new_lines} lines" >> "$LOG"
fi
# Write updated global memory
if [ -n "$global_part" ] && [ ${#global_part} -gt 50 ]; then
new_lines=$(echo "$global_part" | wc -l | tr -d ' ')
mkdir -p "$(dirname "$global_memory_file")"
echo "$global_part" > "$global_memory_file"
echo "[$timestamp] [GC] Global memory: ${global_lines} → ${new_lines} lines" >> "$LOG"
fi
# Append newly archived project items
project_archive_trimmed=$(echo "$project_archive_new" | sed '/^[[:space:]]*$/d' | grep -vi '(empty)' || true)
if [ -n "$project_archive_trimmed" ] && [ ${#project_archive_trimmed} -gt 10 ]; then
mkdir -p "$(dirname "$archive_file")"
{
echo ""
echo "## Archived ${timestamp}"
echo ""
echo "$project_archive_trimmed"
} >> "$archive_file"
new_archive_lines=$(echo "$project_archive_trimmed" | wc -l | tr -d ' ')
echo "[$timestamp] [GC] Archived ${new_archive_lines} lines to project archive" >> "$LOG"
fi
# Append newly archived global items
global_archive_trimmed=$(echo "$global_archive_new" | sed '/^[[:space:]]*$/d' | grep -vi '(empty)' || true)
if [ -n "$global_archive_trimmed" ] && [ ${#global_archive_trimmed} -gt 10 ]; then
mkdir -p "$(dirname "$global_archive_file")"
{
echo ""
echo "## Archived ${timestamp}"
echo ""
echo "$global_archive_trimmed"
} >> "$global_archive_file"
new_archive_lines=$(echo "$global_archive_trimmed" | wc -l | tr -d ' ')
echo "[$timestamp] [GC] Archived ${new_archive_lines} lines to global archive" >> "$LOG"
fi
else
echo "[$timestamp] [GC] Failed — no output or too short (${#output:-0} bytes)" >> "$LOG"
fi
Make executable: chmod +x ~/.claude/hooks/memory-persistence/gc-memory.sh
Checks codemap freshness on new sessions and injects context if stale.
#!/bin/bash
# SessionStart Hook - Check project state on new session
#
# Runs on fresh session starts (not resume/compact). Checks:
# - Codemap freshness (suggest update if stale or missing)
set -euo pipefail
# Read input and check source - only run on fresh startup
input=$(cat)
source=$(echo "$input" | jq -r '.source // ""')
if [ "$source" = "resume" ] || [ "$source" = "compact" ]; then
exit 0
fi
context_parts=()
# Check codemap freshness
codemaps_dir="$(pwd)/codemaps"
if [ -d "$codemaps_dir" ]; then
# Count stale codemap files (older than 7 days)
stale_count=$(find "$codemaps_dir" -name "*.md" -mtime +7 2>/dev/null | wc -l | tr -d ' ')
total_count=$(find "$codemaps_dir" -name "*.md" 2>/dev/null | wc -l | tr -d ' ')
if [ "$stale_count" -gt 0 ] && [ "$total_count" -gt 0 ]; then
context_parts+=("Codemaps are stale (${stale_count}/${total_count} files older than 7 days). Run /update-codemaps to refresh.")
fi
elif git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
# Git repo but no codemaps - check if it's a non-trivial project
src_count=$(find "$(pwd)" -maxdepth 3 \( -name "*.rs" -o -name "*.ts" -o -name "*.js" -o -name "*.py" -o -name "*.swift" -o -name "*.go" \) -not -path "*/node_modules/*" -not -path "*/target/*" -not -path "*/.build/*" 2>/dev/null | wc -l | tr -d ' ')
if [ "$src_count" -gt 5 ]; then
context_parts+=("No codemaps/ found (${src_count} source files). Consider running /update-codemaps.")
fi
fi
# Output JSON with additionalContext
if [ ${#context_parts[@]} -gt 0 ]; then
message=$(printf '%s ' "${context_parts[@]}")
jq -n --arg ctx "$message" '{
"hookSpecificOutput": {
"hookEventName": "SessionStart",
"additionalContext": $ctx
}
}'
fiMake executable: chmod +x ~/.claude/hooks/memory-persistence/session-start.sh
This is the canonical CLI that contains all extraction logic. The Stop hook delegates to this via exec. Symlink from ~/bin/ for user access.
#!/bin/bash
# claude-memory — CLI toolkit for Claude Code's persistent memory system
#
# Memory Architecture:
# Project memory: ~/.claude/projects/<encoded-path>/memory/MEMORY.md
# Global memory: ~/.claude/rules/global-memory.md
# Extraction log: /tmp/claude/extract-learnings.log
# Transcripts: ~/.claude/projects/<encoded-path>/*.jsonl
#
# The Stop hook (extract-learnings.sh) runs after each session and uses
# background Sonnet to extract learnings into both project + global memory.
# Primary mechanism is proactive in-session updates via CLAUDE.md instructions.
set -euo pipefail
CLAUDE_DIR="${HOME}/.claude"
LOG_FILE="/tmp/claude/extract-learnings.log"
GLOBAL_MEMORY="${CLAUDE_DIR}/rules/global-memory.md"
# Resolve project memory path from a directory
project_memory_path() {
local dir="${1:-$(pwd)}"
local encoded=$(echo "$dir" | sed 's|/|-|g')
echo "${CLAUDE_DIR}/projects/${encoded}/memory/MEMORY.md"
}
# Resolve project transcripts dir
project_transcripts_dir() {
local dir="${1:-$(pwd)}"
local encoded=$(echo "$dir" | sed 's|/|-|g')
echo "${CLAUDE_DIR}/projects/${encoded}"
}
usage() {
cat <<'EOF'
claude-memory — manage Claude Code persistent memory
USAGE:
claude-memory <command> [options]
COMMANDS:
show [project|global|archive|all] Display memory files (default: all)
search <query> [--all] Search transcripts for a concept
log [N] Show last N lines of extraction log (default: 30)
stats Show memory system statistics
extract [--dry-run] Manually trigger extraction for current project
gc [--dry-run] Garbage-collect memory files (prune stale/redundant entries)
EXAMPLES:
claude-memory show # Show both project + global active memory
claude-memory show archive # Show project + global archives (cold storage)
claude-memory search "sandbox" # Search current project transcripts
claude-memory search "Docker" --all # Search ALL project transcripts
claude-memory log # Recent extraction activity
claude-memory log 100 # Last 100 lines of log
claude-memory stats # Memory sizes, transcript counts
claude-memory extract --dry-run # Preview what extraction would find
claude-memory gc # Run Opus GC on oversized memory files
claude-memory gc --dry-run # Preview GC output without writing
PATHS:
Project memory: ~/.claude/projects/<encoded-path>/memory/MEMORY.md
Project archive: ~/.claude/projects/<encoded-path>/memory/ARCHIVE.md
Global memory: ~/.claude/rules/global-memory.md
Global archive: ~/.claude/memory/global-archive.md
Extraction log: /tmp/claude/extract-learnings.log
EOF
}
cmd_show() {
local scope="${1:-all}"
local project_mem=$(project_memory_path)
case "$scope" in
project)
if [ -f "$project_mem" ]; then
echo "=== Project Memory: $project_mem ==="
echo ""
cat "$project_mem"
else
echo "No project memory found for $(pwd)"
fi
;;
global)
if [ -f "$GLOBAL_MEMORY" ]; then
echo "=== Global Memory: $GLOBAL_MEMORY ==="
echo ""
cat "$GLOBAL_MEMORY"
else
echo "No global memory found"
fi
;;
all)
if [ -f "$project_mem" ]; then
echo "=== Project Memory ($(wc -c < "$project_mem" | tr -d ' ')B) ==="
echo ""
cat "$project_mem"
echo ""
fi
if [ -f "$GLOBAL_MEMORY" ]; then
echo "=== Global Memory ($(wc -c < "$GLOBAL_MEMORY" | tr -d ' ')B) ==="
echo ""
cat "$GLOBAL_MEMORY"
fi
;;
archive)
local project_archive="${project_mem%MEMORY.md}ARCHIVE.md"
local global_archive="${CLAUDE_DIR}/memory/global-archive.md"
if [ -f "$project_archive" ]; then
local lines=$(wc -l < "$project_archive" | tr -d ' ')
echo "=== Project Archive (${lines} lines) ==="
echo ""
cat "$project_archive"
echo ""
else
echo "No project archive found"
fi
if [ -f "$global_archive" ]; then
local lines=$(wc -l < "$global_archive" | tr -d ' ')
echo "=== Global Archive (${lines} lines) ==="
echo ""
cat "$global_archive"
else
echo "No global archive found"
fi
;;
*)
echo "Unknown scope: $scope (use project, global, archive, or all)"
exit 1
;;
esac
}
cmd_search() {
local query="$1"
shift
local search_all=false
for arg in "$@"; do
case "$arg" in
--all) search_all=true ;;
esac
done
if [ "$search_all" = true ]; then
echo "=== Searching ALL transcripts for: $query ==="
echo ""
# Search across all project transcript dirs
for project_dir in "${CLAUDE_DIR}/projects/"*/; do
[ ! -d "$project_dir" ] && continue
local found=false
for jsonl in "$project_dir"*.jsonl; do
[ ! -f "$jsonl" ] && continue
# Filter to conversation text and search
matches=$(python3 -c "
import json, re, sys
query = sys.argv[2].lower()
with open(sys.argv[1]) as f:
for i, line in enumerate(f):
try:
entry = json.loads(line)
except: continue
if entry.get('type') not in ('user', 'assistant'): continue
msg = entry.get('message', {})
content = msg.get('content', '')
if isinstance(content, list):
content = ' '.join(b.get('text','') for b in content if isinstance(b, dict) and b.get('type')=='text')
content = re.sub(r'<system-reminder>.*?</system-reminder>', '', str(content), flags=re.DOTALL)
if query in content.lower():
role = msg.get('role', entry.get('type',''))
preview = content[:200].replace('\n', ' ').strip()
print(f' [{role}] {preview}')
" "$jsonl" "$query" 2>/dev/null || true)
if [ -n "$matches" ]; then
if [ "$found" = false ]; then
echo "--- ${project_dir##*/projects/} ---"
found=true
fi
echo " ${jsonl##*/}:"
echo "$matches" | head -5
local count=$(echo "$matches" | wc -l | tr -d ' ')
if [ "$count" -gt 5 ]; then
echo " ... and $((count - 5)) more matches"
fi
fi
done
done
else
local transcripts_dir=$(project_transcripts_dir)
echo "=== Searching project transcripts for: $query ==="
echo ""
if [ ! -d "$transcripts_dir" ]; then
echo "No transcripts found for $(pwd)"
exit 1
fi
for jsonl in "$transcripts_dir"/*.jsonl; do
[ ! -f "$jsonl" ] && continue
matches=$(python3 -c "
import json, re, sys
query = sys.argv[2].lower()
with open(sys.argv[1]) as f:
for line in f:
try:
entry = json.loads(line)
except: continue
if entry.get('type') not in ('user', 'assistant'): continue
msg = entry.get('message', {})
content = msg.get('content', '')
if isinstance(content, list):
content = ' '.join(b.get('text','') for b in content if isinstance(b, dict) and b.get('type')=='text')
content = re.sub(r'<system-reminder>.*?</system-reminder>', '', str(content), flags=re.DOTALL)
if query in content.lower():
role = msg.get('role', entry.get('type',''))
preview = content[:200].replace('\n', ' ').strip()
print(f' [{role}] {preview}')
" "$jsonl" "$query" 2>/dev/null || true)
if [ -n "$matches" ]; then
echo "${jsonl##*/}:"
echo "$matches" | head -10
local count=$(echo "$matches" | wc -l | tr -d ' ')
if [ "$count" -gt 10 ]; then
echo " ... and $((count - 10)) more matches"
fi
echo ""
fi
done
fi
}
cmd_log() {
local lines="${1:-30}"
if [ -f "$LOG_FILE" ]; then
echo "=== Extraction Log (last $lines lines) ==="
echo ""
tail -n "$lines" "$LOG_FILE"
else
echo "No extraction log found at $LOG_FILE"
fi
}
cmd_stats() {
local project_mem=$(project_memory_path)
local transcripts_dir=$(project_transcripts_dir)
echo "=== Memory System Stats ==="
echo ""
# Project memory
if [ -f "$project_mem" ]; then
local size=$(wc -c < "$project_mem" | tr -d ' ')
local lines=$(wc -l < "$project_mem" | tr -d ' ')
echo "Project memory: ${size}B, ${lines} lines"
else
echo "Project memory: not found"
fi
# Global memory
if [ -f "$GLOBAL_MEMORY" ]; then
local size=$(wc -c < "$GLOBAL_MEMORY" | tr -d ' ')
local lines=$(wc -l < "$GLOBAL_MEMORY" | tr -d ' ')
echo "Global memory: ${size}B, ${lines} lines"
else
echo "Global memory: not found"
fi
echo ""
# Archives
local project_archive="${project_mem%MEMORY.md}ARCHIVE.md"
local global_archive="${CLAUDE_DIR}/memory/global-archive.md"
if [ -f "$project_archive" ]; then
local size=$(wc -c < "$project_archive" | tr -d ' ')
local lines=$(wc -l < "$project_archive" | tr -d ' ')
echo "Project archive: ${size}B, ${lines} lines (cold storage)"
else
echo "Project archive: empty"
fi
if [ -f "$global_archive" ]; then
local size=$(wc -c < "$global_archive" | tr -d ' ')
local lines=$(wc -l < "$global_archive" | tr -d ' ')
echo "Global archive: ${size}B, ${lines} lines (cold storage)"
else
echo "Global archive: empty"
fi
echo ""
# Transcripts
if [ -d "$transcripts_dir" ]; then
local count=$(ls "$transcripts_dir"/*.jsonl 2>/dev/null | wc -l | tr -d ' ')
local total_size=$(du -sh "$transcripts_dir"/*.jsonl 2>/dev/null | tail -1 | cut -f1 || echo "0")
echo "Project transcripts: $count sessions"
# All projects
local all_count=0
for d in "${CLAUDE_DIR}/projects/"*/; do
[ ! -d "$d" ] && continue
local c=$(ls "$d"*.jsonl 2>/dev/null | wc -l | tr -d ' ')
all_count=$((all_count + c))
done
local project_count=$(ls -d "${CLAUDE_DIR}/projects/"*/ 2>/dev/null | wc -l | tr -d ' ')
echo "All transcripts: $all_count sessions across $project_count projects"
fi
echo ""
# Extraction log
if [ -f "$LOG_FILE" ]; then
local extractions=$(grep -c "Extracting from" "$LOG_FILE" 2>/dev/null || echo "0")
local writes=$(grep -c "Wrote.*B to" "$LOG_FILE" 2>/dev/null || echo "0")
local last=$(tail -1 "$LOG_FILE" 2>/dev/null || echo "never")
echo "Extractions run: $extractions"
echo "Successful writes: $writes"
echo "Last activity: $last"
else
echo "No extraction log found"
fi
# Lock status
if [ -f "/tmp/claude/extract-learnings.lock" ]; then
local lock_age=$(find "/tmp/claude/extract-learnings.lock" -mmin +0 -printf "%T+" 2>/dev/null || stat -f "%Sm" "/tmp/claude/extract-learnings.lock" 2>/dev/null || echo "unknown")
echo ""
echo "WARNING: Lock file exists (extraction may be running)"
echo "Lock file: /tmp/claude/extract-learnings.lock"
fi
}
cmd_extract() {
local dry_run=false
for arg in "$@"; do
case "$arg" in
--dry-run) dry_run=true ;;
esac
done
local project_cwd="${CLAUDE_PROJECT_DIR:-$(pwd)}"
local encoded_path=$(echo "$project_cwd" | sed 's|/|-|g')
local claude_project_dir="${CLAUDE_DIR}/projects/${encoded_path}"
local memory_dir="${claude_project_dir}/memory"
local memory_file="${memory_dir}/MEMORY.md"
local LOG="/tmp/claude/extract-learnings.log"
mkdir -p "$memory_dir"
if [ ! -d "$claude_project_dir" ]; then
echo "No Claude project dir found for $project_cwd — skipping"
exit 0
fi
transcript=$(ls -t "$claude_project_dir"/*.jsonl 2>/dev/null | head -1 || true)
if [ -z "$transcript" ]; then
echo "No transcripts found in $claude_project_dir — skipping"
exit 0
fi
msg_count=$(grep -c '"type":"user"' "$transcript" 2>/dev/null || echo "0")
size=$(wc -c < "$transcript" | tr -d ' ')
echo "Latest transcript: ${transcript##*/}"
echo "Messages: $msg_count user messages, ${size}B total"
if [ "$dry_run" = true ]; then
echo ""
echo "Filtering transcript..."
filtered="/tmp/claude/dry-run-filtered-$$.md"
python3 -c "
import json, re, sys
parts = []
with open(sys.argv[1]) as f:
for line in f:
try: entry = json.loads(line)
except: continue
if entry.get('type') not in ('user', 'assistant'): continue
msg = entry.get('message', {})
content = msg.get('content', '')
if isinstance(content, list):
content = '\n'.join(b.get('text','') for b in content if isinstance(b, dict) and b.get('type')=='text')
content = re.sub(r'<system-reminder>.*?</system-reminder>', '', str(content), flags=re.DOTALL).strip()
if content:
role = msg.get('role', entry.get('type',''))
parts.append(f'## {role}\n{content}\n')
with open(sys.argv[2], 'w') as f: f.write('\n'.join(parts))
" "$transcript" "$filtered"
filtered_size=$(wc -c < "$filtered" | tr -d ' ')
echo "Filtered: ${size}B → ${filtered_size}B"
echo ""
echo "First 50 lines of filtered content:"
head -50 "$filtered"
rm -f "$filtered"
exit 0
fi
if [ "$msg_count" -lt 5 ]; then
echo "Skipping: fewer than 5 user messages"
exit 0
fi
lock_file="/tmp/claude/extract-learnings-${encoded_path}.lock"
if [ -f "$lock_file" ]; then
if [ -n "$(find "$lock_file" -mmin +60 2>/dev/null)" ]; then
rm -f "$lock_file"
else
echo "Skipping: extraction recently ran (lock file exists)"
exit 0
fi
fi
touch "$lock_file"
(
set +e
filtered="/tmp/claude/filtered-transcript-$$.md"
python3 -c "
import json, re, sys
parts = []
with open(sys.argv[1]) as f:
for line in f:
try: entry = json.loads(line)
except: continue
if entry.get('type') not in ('user', 'assistant'): continue
msg = entry.get('message', {})
content = msg.get('content', '')
if isinstance(content, list):
texts = []
for block in content:
if isinstance(block, dict) and block.get('type') == 'text':
t = block['text']
t = re.sub(r'<system-reminder>.*?</system-reminder>', '', t, flags=re.DOTALL).strip()
if t: texts.append(t)
content = '\n'.join(texts)
elif isinstance(content, str):
content = re.sub(r'<system-reminder>.*?</system-reminder>', '', content, flags=re.DOTALL).strip()
if not content: continue
role = msg.get('role', entry.get('type',''))
parts.append(f'## {role}\n{content}\n')
with open(sys.argv[2], 'w') as f: f.write('\n'.join(parts))
" "$transcript" "$filtered"
if [ ! -s "$filtered" ]; then
rm -f "$lock_file" "$filtered"
exit 0
fi
filtered_size=$(wc -c < "$filtered" | tr -d ' ')
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Extracting from ${transcript##*/} ($msg_count msgs, ${filtered_size}B filtered)" >> "$LOG"
existing_memory=""
if [ -f "$memory_file" ] && [ -s "$memory_file" ]; then
existing_memory=$(cat "$memory_file")
fi
existing_global=""
if [ -f "$GLOBAL_MEMORY" ] && [ -s "$GLOBAL_MEMORY" ]; then
existing_global=$(cat "$GLOBAL_MEMORY")
fi
cd "$project_cwd" 2>/dev/null || cd "$HOME"
output=$(claude -p --model sonnet --no-session-persistence \
--tools "" \
--disable-slash-commands \
--strict-mcp-config \
--settings '{"disableAllHooks":true}' \
--system-prompt "You are a learning extraction agent. You output TWO files separated by a delimiter.
## Output Format
Output the updated project MEMORY.md first, then a delimiter, then the updated global memory:
\`\`\`
<project memory content>
---GLOBAL_MEMORY_SEPARATOR---
<global memory content>
\`\`\`
No preamble, no explanation, no markdown fences around the whole output. Just the two files with the separator between them.
## What goes where — THIS IS CRITICAL
**Project MEMORY.md** (default destination) — Almost everything goes here. API quirks, data formats, debugging insights, environment gotchas, patterns, tool-specific learnings. When in doubt, PUT IT IN PROJECT MEMORY.
**Global memory** (rare, selective) — ONLY for patterns that are genuinely universal across many different project types. Think: a Docker gotcha that would bite anyone on any project, or a Claude Code behavioral quirk. NOT for project-ecosystem details like WordPress/WooCommerce, specific CRM APIs, or domain-specific tooling — those belong in the project where they were learned.
**Test: Would a developer working on a COMPLETELY DIFFERENT kind of project (different language, different domain) benefit from this?** If no → project memory. If yes → maybe global, but keep it to one terse line.
## Pin rules — IMPORTANT
- NEVER add 📌 pins. Only the human or the in-session Claude instance adds pins.
- When outputting existing pinned items, preserve them as-is.
## What to extract
- Technical discoveries: API quirks, data formats, byte layouts, coordinate conventions, protocol details
- Debugging insights: root causes found, misleading symptoms, what worked vs what didn't
- User corrections: places where the AI was wrong and the human corrected it
- Environment gotchas: Docker issues, path conventions, build steps, platform-specific behavior
- Patterns: approaches that worked well for recurring problems
## What to skip
- Routine code changes (added a function, fixed a typo)
- Things obvious from reading the code
- One-time tasks that won't recur
- Implementation details already in the code itself
- Anything already captured in existing memory (don't duplicate)
## How to write
- Organize by topic, not chronologically
- Be specific and concrete (include exact values, paths, field names)
- Replace outdated information with corrections
- Keep project memory under 200 lines
- Keep global memory VERY short — under 50 lines, only truly universal items
- Global entries should be terse one-liners, not detailed explanations
- If removing items from global that are project-specific, move them to project memory" \
"EXISTING PROJECT MEMORY.MD:
${existing_memory:-<empty — this is a new file>}
---
EXISTING GLOBAL MEMORY:
${existing_global:-<empty — this is a new file>}
---
SESSION TRANSCRIPT (filtered to user/assistant text only):
$(cat "$filtered")
---
Output the complete updated project MEMORY.md, then ---GLOBAL_MEMORY_SEPARATOR---, then the complete updated global memory. Merge any new learnings from the transcript." 2>&1)
if [ -n "$output" ] && [ ${#output} -gt 50 ]; then
project_part=$(echo "$output" | sed -n '1,/---GLOBAL_MEMORY_SEPARATOR---/p' | sed '$d')
global_part=$(echo "$output" | sed -n '/---GLOBAL_MEMORY_SEPARATOR---/,$p' | sed '1d')
if [ -n "$project_part" ] && [ ${#project_part} -gt 50 ]; then
echo "$project_part" > "$memory_file"
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Wrote ${#project_part}B to $memory_file" >> "$LOG"
fi
if [ -n "$global_part" ] && [ ${#global_part} -gt 50 ]; then
mkdir -p "$(dirname "$GLOBAL_MEMORY")"
echo "$global_part" > "$GLOBAL_MEMORY"
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Wrote ${#global_part}B to $GLOBAL_MEMORY" >> "$LOG"
fi
else
echo "[$(date '+%Y-%m-%d %H:%M:%S')] No output or too short (${#output:-0} bytes)" >> "$LOG"
fi
gc_counter_file="/tmp/claude/gc-counter-${encoded_path}"
if [ -n "$project_part" ] && [ ${#project_part} -gt 50 ]; then
count=0
[ -f "$gc_counter_file" ] && count=$(cat "$gc_counter_file" 2>/dev/null || echo "0")
count=$((count + 1))
echo "$count" > "$gc_counter_file"
if [ "$count" -ge 10 ]; then
echo "[$(date '+%Y-%m-%d %H:%M:%S')] [GC] Triggering GC after $count extractions" >> "$LOG"
echo "0" > "$gc_counter_file"
CLAUDE_PROJECT_DIR="$project_cwd" bash "${HOME}/.claude/hooks/memory-persistence/gc-memory.sh"
fi
fi
rm -f "$lock_file" "$filtered"
) >>/tmp/claude/extract-learnings.log 2>&1 &
echo "Extraction launched in background. Check: claude-memory log"
}
# Main dispatch
case "${1:-}" in
show) shift; cmd_show "$@" ;;
search)
shift
if [ $# -lt 1 ]; then
echo "Usage: claude-memory search <query> [--all]"
exit 1
fi
cmd_search "$@"
;;
log) shift; cmd_log "$@" ;;
stats) cmd_stats ;;
extract) shift; cmd_extract "$@" ;;
gc)
shift
echo "Running memory GC (Opus)..."
CLAUDE_PROJECT_DIR="$(pwd)" bash "${HOME}/.claude/hooks/memory-persistence/gc-memory.sh" "$@"
if echo "$@" | grep -q -- "--dry-run"; then
: # dry-run output handled by gc-memory.sh
else
echo "GC complete. Check: claude-memory log"
fi
;;
help|-h|--help) usage ;;
"") usage ;;
*)
echo "Unknown command: $1"
echo ""
usage
exit 1
;;
esacMake executable: chmod +x ~/.claude/bin/claude-memory
Symlink for user access: ln -sf ~/.claude/bin/claude-memory ~/bin/claude-memory
Start empty or with a header. The Stop hook will populate it over time.
# Global Memory (Cross-Project)
Learnings that apply across all projects. Auto-managed by the Stop hook.Merge these into your existing settings.json hooks section:
{
"hooks": {
"SessionStart": [
{
"matcher": "*",
"hooks": [
{
"type": "command",
"command": "~/.claude/hooks/memory-persistence/session-start.sh"
}
]
}
],
"Stop": [
{
"matcher": "*",
"hooks": [
{
"type": "command",
"command": "~/.claude/hooks/memory-persistence/extract-learnings.sh"
}
]
}
]
}
}Add these to the permissions.allow array so Claude can read/write memory files without prompts:
"Read(~/.claude/projects/**/memory/*)",
"Edit(~/.claude/projects/**/memory/*)",
"Write(~/.claude/projects/**/memory/*)",
"Read(~/.claude/rules/*)",
"Edit(~/.claude/rules/*)",
"Write(~/.claude/rules/*)"Add these two sections to your global CLAUDE.md. The first is the critical instruction that makes Claude proactively update MEMORY.md during sessions (the primary mechanism). The second tells Claude about the CLI tool.
### Continuous Learning **[CRITICAL]**
You have persistent memory at `~/.claude/projects/<project>/memory/MEMORY.md` loaded into every session. Actively maintain it — this is how you compound knowledge across sessions.
**When to update MEMORY.md:**
- After discovering a non-obvious technical detail (API quirks, byte layouts, format conventions)
- After debugging something that took multiple turns (save the root cause)
- When the user corrects a wrong assumption (record what was wrong and why)
- After trying an approach that failed (save future you from repeating it)
- Before compaction if there are unsaved discoveries
**How to update:**
- Organize by topic, not chronologically
- Replace outdated information with corrections
- Keep under 200 lines — concise and scannable
- Only genuinely useful discoveries — skip obvious things
- Prefix critical entries with 📌 to pin them (GC will never prune pinned items)
The notes are for YOU in future sessions, not for the human. A Stop hook also runs background extraction as a safety net, but proactive in-session updates are higher quality since you have full context.## Memory System
Use `claude-memory` CLI (`~/.claude/bin/claude-memory`, symlinked from `~/bin/`) for memory operations instead of writing custom scripts:
\```bash
claude-memory show # Display project + global memory
claude-memory search "query" # Search current project transcripts
claude-memory search "query" --all # Search ALL project transcripts
claude-memory log # Recent extraction activity
claude-memory stats # Memory sizes, transcript counts
claude-memory extract --dry-run # Preview extraction
claude-memory gc # Run Opus GC on oversized memory files
claude-memory gc --dry-run # Preview GC output without writing
claude-memory show archive # View archived (cold storage) items
\```
Paths:
- Project memory: `~/.claude/projects/<encoded-path>/memory/MEMORY.md` (auto-loaded per project)
- Project archive: `~/.claude/projects/<encoded-path>/memory/ARCHIVE.md` (cold storage, GC only)
- Global memory: `~/.claude/rules/global-memory.md` (auto-loaded everywhere)
- Global archive: `~/.claude/memory/global-archive.md` (cold storage, GC only)
- Spec: `~/.claude/hooks/memory-persistence/SPEC.md` (canonical system documentation)
- Stop hook: `~/.claude/hooks/memory-persistence/extract-learnings.sh` (background Sonnet extraction)
- GC hook: `~/.claude/hooks/memory-persistence/gc-memory.sh` (periodic Opus consolidation + archive)These are bugs we hit during development. They'll save you hours:
-
Background Claude is sandboxed.
claude -pcannot use Read/Write tools on arbitrary paths. The fix: inline everything in the prompt (existing MEMORY.md + filtered transcript), have it output to stdout, and write from bash. -
Never redirect stdin to
/dev/nullfor background Claude.( ... ) </dev/nullkillsclaude -psilently. Only redirect stdout/stderr. -
set -ein background subshells kills silently. Ifclaude -pexits non-zero, the entire subshell dies with no trace. Useset +einside the subshell. -
Transcript JSONL is 98% noise. Raw files are mostly
tool_use/tool_resultblocks. Pre-filter totype: "user"andtype: "assistant"withmessage.contenttext blocks only. A 1.3MB transcript becomes ~30KB of conversation. -
JSONL structure is nested. It's
entry.message.content(array of blocks), NOTentry.content. Thetypefield is at the top level,roleis insidemessage. -
Stop hooks fire frequently. Not just on session end — every few minutes during normal use. Rate-limit with per-project lock files or you'll burn $1+/hour in Sonnet calls.
-
Stop hooks don't get
CLAUDE_TRANSCRIPT_PATH. UseCLAUDE_PROJECT_DIR+ encode the path +ls -t *.jsonl | head -1to find the most recent transcript. -
Sonnet >> Haiku for extraction. Haiku can't distinguish "new discovery" from "mentioned in passing." It pollutes MEMORY.md with already-known information. Sonnet costs ~$0.17/extraction but exercises judgment.
-
disownis needed for true non-blocking. Claude Code's hook runner may wait on the process group.( ... ) >>log 2>&1 &alone isn't enough — adddisownafter the&. -
Permission rule precedence: deny > ask > allow. A broad
askrule likeRead(~/.*)will override specificallowrules for~/.claude/**paths. Remove broad ask rules and rely on specific deny rules instead. -
Absolute paths in permissions need
//prefix.Read(/tmp/**)is relative to the settings file. UseRead(//tmp/**)for true absolute paths.~/paths work correctly as-is. -
Soft line limits don't work alone. Telling the extraction model "keep under 200 lines" is insufficient — it's additive by nature and rarely deletes. The GC pass (separate Opus invocation focused exclusively on pruning) is needed as hard enforcement.
-
GC counter lives in
/tmp/claude/. Survives across sessions within the same boot. Resets on reboot, which is fine — worst case is a slightly delayed first GC. -
GC without archive = data loss. The first GC implementation was too aggressive — pruned 600→63 lines with no recovery. Always archive pruned items to cold storage so nothing is truly deleted.
-
GC must run sequentially after extraction, not in parallel. Running GC with
&(parallel) causes race conditions — a concurrent extraction reads pre-GC memory and overwrites the GC'd files when it finishes. Remove the&so GC runs after extraction completes (still inside the detached subshell, so the hook returns immediately). -
📌 pin markers need to be free from line budgets. If pinned items count toward the 150/80 line targets, a file with 50 pinned items has only 100 lines of budget — creating pressure to over-prune the remaining entries.
-
Stop hooks must always exit 0. Skip conditions (no transcripts, no project dir) are normal — not errors. Using
exit 1for these causes Claude Code to report hook failures on every session end. The hook wrapper should also use|| trueto swallow unexpected failures, since there's nothing the user can do about a Stop hook error.
| Operation | Model | Cost | Frequency |
|---|---|---|---|
| Extraction | Sonnet | ~$0.17 | Max 1/hr/project |
| GC pass | Opus | ~$0.30 | Every 10 extractions (~10 hrs) |
| In-session updates | Free | $0 | During normal work |
Typical daily cost for active project: ~$0.85 extraction + ~$0.30 GC = ~$1.15/day
After installation, test with:
# Check the CLI works
claude-memory stats
# Check the log after a session
claude-memory log
# Search transcripts
claude-memory search "some concept"
# Manually trigger extraction
claude-memory extract --dry-run
# Manually trigger GC
claude-memory gc --dry-run
# Check memory sizes
wc -l ~/.claude/rules/global-memory.md
find ~/.claude/projects/*/memory/MEMORY.md -exec sh -c 'echo "$(wc -l < "$1") $1"' _ {} \;