oaustegard avatar

knowledge-and-memory

Research methodology, corpus search, persistent memory, iterative workflows, and project knowledge management.

version
1.3.0
GitHub stars
404

Install

/plugin install knowledge-and-memory@oaustegard-claude-skills

Run it inside Claude Code after adding the marketplace with /plugin marketplace add oaustegard/claude-skills.

Contents

Discovered from source

Skills 9

Asking Questions

Purpose

Ask clarifying questions when the answer materially changes what you'll build. This skill helps identify when to ask, how to structure questions effectively, and when to proceed autonomously.

When to Use

Ask questions for:

  1. Ambiguous implementation choices - Multiple valid technical approaches (middleware vs wrapper functions, library selection, architectural patterns)
  2. Missing critical context - Specific information needed (database type, deployment platform, credential management)
  3. Potentially destructive actions - Requests that could be interpreted dangerously ("clean up files" = delete vs archive)
  4. Scope clarification - Vague terms like "refactor," "optimize," or "improve"
  5. Conflicting requirements - Goals that may work against each other ("make it faster" + "add extensive logging")
  6. Technical trade-offs - Solutions with different costs/benefits depending on priorities

When NOT to Use

Don't ask when:

  • Request is clear and unambiguous - One obvious implementation path
  • You can determine the answer from context - Codebase patterns, project structure, existing conventions
  • Over-clarification - Questions that don't materially affect implementation
  • Standard engineering practices - Established patterns already in the codebase

Question Structure

Template

[Context: What you found/analyzed]

[Present 2-5 specific options with brief trade-offs]

[Direct question asking for preference]

[Optional: Offer to make reasonable default choice]

Guidelines

  1. Acknowledge understanding first - Show you've analyzed the situation

    • "I found your API endpoints and see you're using Express..."
  2. Present clear options - Offer 2-5 specific choices with brief context

    I can implement this in several ways:
    1. **Global middleware** - Catches all errors centrally (simplest)
    2. **Wrapper functions** - More granular control per endpoint
    3. **Custom error classes** - Typed errors with status codes
    
  3. Ask directly - Clear question that guides decision

    • "Which approach fits your needs?"
  4. Offer autonomy (optional) - For less critical decisions

    • "I can implement option 2 as it's the safest, or let me know if you prefer something else."

Progressive Disclosure

Layer questions instead of asking everything upfront:

Good ✓

  • First: "For real-time updates, should I use WebSockets, Server-Sent Events, or polling?"
  • Then: "For WebSockets, do you want Socket.io (easier) or native (lighter)?"

Bad ✗

  • "For real-time updates should I use WebSockets or SSE or polling and if WebSockets should I use Socket.io or native and should I implement reconnection and what about authentication and..."

Examples

Good ✓

"I see you're using JWT authentication. To add refresh tokens, I can:

  1. httpOnly cookies - More secure, harder to XSS
  2. localStorage - Simpler, works with mobile apps
  3. In-memory only - Most secure, lost on refresh

What works best for your use case?"

Bad ✗

"How should I implement the authentication refresh token storage mechanism considering security implications, XSS vulnerabilities, mobile compatibility, UX impacts, and compliance considerations?"

Too verbose, no clear options, asks everything at once


Good ✓

"You mentioned 'clean up migrations.' Do you want me to archive them to /old-migrations or delete them entirely? (Note: deletion can break databases that haven't run them yet)"

Bad ✗

"What do you mean by clean up?"

Too vague, doesn't guide the decision

After Receiving an Answer

  1. Acknowledge - Confirm you understood their choice
  2. Proceed immediately - Don't ask if they want you to continue
  3. Apply preferences - If they chose "security first" once, lean that direction for similar future choices

Key Principle

Ask only when the answer materially changes what you'll build. Avoid building the wrong thing, not asking questions for the sake of asking.

bm25

Ranked content search over any text corpus. One CLI, in-memory BM25 index per process, with a session-local disk cache so repeat invocations against the same corpus load in tens of milliseconds instead of rebuilding.

Setup

uv pip install --system --break-system-packages bm25s

Install is sub-second on a warm uv cache. That's the entire dependency.

Usage

BM25=/mnt/skills/user/bm25/scripts/bm25.py

# Local directory
python3 $BM25 ./repo 'csrf middleware'

# Multiple queries against the same in-memory index (build once, query many)
python3 $BM25 ./repo 'csrf middleware' 'session backend' 'queryset filter'

# Cloned GitHub repo via tarball (one HTTP call)
python3 $BM25 'github.com/django/django' 'atomic transaction'
python3 $BM25 'github.com/django/django@stable/5.0.x' 'atomic transaction'

# Project knowledge or uploads
python3 $BM25 project 'RAG scaling laws'
python3 $BM25 uploads 'tax loss harvesting'

# Filters
python3 $BM25 ./repo 'auth flow' --exclude 'tests/*' --exclude '*/tests/*'
python3 $BM25 ./repo 'config' --include '*.py' --include '*.toml'

# Interactive (REPL — single corpus, many queries)
python3 $BM25 ./repo --interactive

# JSON output for piping
python3 $BM25 ./repo 'auth flow' --json

Corpus types

SpecMeaning
./path or /abs/pathLocal directory
uploads/mnt/user-data/uploads/
project/mnt/project/
github.com/owner/repo[@ref]Tarball fetch via GitHub API (GH_TOKEN used if set)

Options

OptionDefaultDescription
--top-k N10Results per query
--include GLOB(auto)Repeatable. If set, only files matching one of these globs are indexed
--exclude GLOBRepeatable. Skip files matching these globs
--snippet-lines N3Lines of snippet context per hit (0 = none)
--max-file-bytes N2,000,000Skip files larger than this
--jsonMachine-readable output
--interactive / -iREPL mode for ad-hoc querying within one session
--statsPrint discover + index timings as JSON
--no-cacheBypass the session-local index cache; build in-memory only

With no --include, a default set of text/code extensions is indexed (Python, JS/TS, Go, Rust, Markdown, JSON, YAML, etc.). Standard noise dirs are skipped unconditionally: .git, node_modules, __pycache__, .venv, dist, etc.

When to use bm25

Question shapeTool
"Find lines matching class.*Error"grep / ripgrep
"Show me where parse_input is defined"tree-sitting (find:/source:)
"Which files are about CSRF handling?"bm25
"Rank these docs by relevance to 'rate limiting strategies'"bm25
"What's the implementation of the atomic transaction context manager?"bm25, then tree-sitting source:
"Find code by natural-language concept (in a code repo)"searching-codebases (which has its own TF-IDF mode)

The boundary with searching-codebases: that skill is code-specific (routes between regex and TF-IDF, expands via tree-sitting AST). bm25 is the simpler general-purpose tool — any corpus, no AST awareness, no routing. Prefer searching-codebases for code; reach for bm25 when the corpus is mixed (docs + code), non-code (notes, transcripts, PDFs converted to text), or when you specifically want BM25's length-normalized scoring.

Design notes

  • Session-local disk cache at /home/claude/.bm25-cache/<key>/. The key is a hash of (resolved_corpus_path, include_globs, exclude_globs, max_file_bytes) plus every discovered file's path and content hash, so editing, adding or deleting a file invalidates it. A hit still reads the corpus to hash it but skips tokenizing and indexing (1,649 files: 0.35s hit vs ~1s rebuild). The cache lives in /home/claude, which is ephemeral, so it expires at the session boundary — same lifetime as the corpus state itself, no cross-session staleness. ~5–35MB per cached index, depending on corpus size.
  • --no-cache bypasses both load and save — useful to confirm a rebuild matches.
  • A corpus with no indexable terms (empty, or only one-character tokens) returns zero-score hits instead of crashing bm25s.
  • Reuse within a single invocation. The retriever stays in memory between queries in one process. Passing multiple queries positionally, or using --interactive, amortizes any rebuild cost across queries.
  • No AST awareness. Chunking is per-file. For symbol-level results in code, combine with tree-sitting queries on the same paths.
  • Tokenizer. Default bm25s.tokenize with stopwords disabled — over a small Django sample, AST-derived token streams (identifiers/strings/ comments only) gave near-identical rankings, so we don't bother.

Output format

Default (human-readable):

QUERY: csrf middleware
----------------------------------------------------------------------
  1.   5.51  django/core/checks/security/csrf.py
    def _csrf_middleware():
        return "django.middleware.csrf.CsrfViewMiddleware" in settings.MIDDLEWARE
  2.   5.34  docs/howto/csrf.txt
    ...

--json produces {"query": ..., "results": [{"path", "score", "snippet"}, ...]}.

Architecture

bm25.py CLI
  ├── resolve_corpus(spec)         → local Path (downloads tarball if github.com/...)
  ├── cache_key(...)               → 16-hex sha256 of inputs that determine the index
  ├── CorpusIndex.load(cache_dir)  → returns cached index if present, else None
  ├── CorpusIndex.build(...)       → walks files, tokenizes, indexes with bm25s
  ├── CorpusIndex.save(cache_dir)  → persists to /home/claude/.bm25-cache/<key>/
  ├── query(q, k)                  → ranked (doc_idx, score) pairs
  └── best_snippet(doc, q, lines)  → pick line w/ most query-term hits + context

Cache contents per directory:

  • bm25/ — bm25s.BM25.save() output (NumPy arrays + vocab)
  • corpus.pkl — pickled {paths, docs} so we can render snippets without re-reading the source files
  • manifest.json — corpus root, files count, built_at timestamp

No network beyond optional tarball fetch on github.com/... corpora. No state outside /home/claude/, which is ephemeral.

Cloning Project

When users request to clone, copy, export, or backup their current project:

When NOT to use this skill

"Project" here is a Claude.ai workspace, never a code repository. The word collides badly, so route by what is actually being copied:

SituationUse
Clone or fetch a git repoaccessing-github-repos
"I just cloned this, what is it?"exploring-codebases
What a codebase contains or exposestree-sitting
Back up a repo's filesgit, not this

This skill reads the Claude.ai workspace you are talking in. It touches no checkout and knows nothing about code.

Step 1: Verify Project Context

Check if the user is in a project by looking for project-specific indicators:

  • Project instructions in the context window
  • Knowledge files mentioned in <available_skills> or similar sections
  • User explicitly stating they're in a project

If not in a project, inform the user that this skill requires being inside a Claude project.

Step 2: Gather Project Instructions

Project instructions are typically found in the context window as XML or markdown content describing the project's purpose and configuration. Extract and save these instructions to a file.

Create the instructions file:

cat > /home/claude/project-instructions.md << 'INSTRUCTIONS'
[Extracted project instructions content]
INSTRUCTIONS

Step 3: Locate Knowledge Files

Knowledge files are user-uploaded documents stored in /mnt/user-data/uploads/. List all files:

ls -lh /mnt/user-data/uploads/

If knowledge files exist, copy them to the working directory for bundling:

cp /mnt/user-data/uploads/* /home/claude/project-export/

Step 4: Create Export Bundle

If only project instructions exist (no knowledge files):

  • Copy instructions directly to outputs:
    cp /home/claude/project-instructions.md /mnt/user-data/outputs/
    

If project has both instructions and knowledge files:

  • Create an organized export directory:
    mkdir -p /home/claude/project-export
    cp /home/claude/project-instructions.md /home/claude/project-export/
    cp /mnt/user-data/uploads/* /home/claude/project-export/ 2>/dev/null || true
    
  • Create zip bundle:
    cd /home/claude
    zip -r /mnt/user-data/outputs/project-export.zip project-export/
    
  • Show bundle contents:
    unzip -l /mnt/user-data/outputs/project-export.zip
    

Step 5: Provide Files and Instructions to User

Link the exported files:

For single instructions file:

[Download project-instructions.md](computer:///mnt/user-data/outputs/project-instructions.md)

For bundled export:

[Download project-export.zip](computer:///mnt/user-data/outputs/project-export.zip)

Then provide clear setup instructions:

## How to Import Into a New Project

1. **Create a new Claude project** (or open an existing one where you want to clone this configuration)

2. **Add project instructions:**
   - Open the new project's settings
   - Navigate to the "Instructions" section
   - Copy and paste the content from `project-instructions.md`
   - Save the instructions

3. **Upload knowledge files** (if applicable):
   - In the new project, go to the "Knowledge" section
   - Upload all files from the export (except project-instructions.md)
   - Files will become available as project knowledge

Your project is now cloned with the same configuration and knowledge base.

Edge Cases

Empty project (no custom instructions or knowledge):

  • Inform the user that the project has no custom configuration to export
  • Explain that projects by default only have access to Claude's base capabilities

Large knowledge bases:

  • If the zip file exceeds reasonable size (~50MB), warn the user and suggest selective export
  • Provide individual file links as fallback

Corrupted or inaccessible files:

  • Skip files that cannot be read
  • Report which files were successfully exported vs. skipped
  • Provide export anyway with available content

creating-kb

Turn a pile of files into a portable, deployable knowledgebase. The output is a .skill bundle — an ordinary zip — containing a BM25 inverted index, the chunk text, a pure-Node searcher, and a query protocol. It has no embedding model and no semantic search: retrieval is lexical, and the consuming agent supplies the semantic layer by expanding the query at search time. That is what makes the bundle portable — any agent that can run node can query it with no npm install, no model download, and no network.

The whole toolchain is JavaScript so one implementation serves both this builder and the in-browser packer. Build with the bundled script; do not hand-roll the index.

SCRIPTS=/mnt/skills/user/creating-kb/scripts
node $SCRIPTS/build_lexkb.js CORPUS_DIR --out /tmp/kb --name my-kb --zip

Workflow

1. Gather the sources

Collect the files into one directory. In a Claude.ai chat, uploads land in /mnt/user-data/uploads/ — point the builder there. Otherwise use any path the user names. Supported extensions default to txt,md,html,htm; pass --ext to change them.

This MVP interface is bounded by how many files a chat can accept. For a large corpus, stage the files in a directory first, or use the browser packer (built from the same scripts) that runs entirely client-side.

2. Build the bundle

SCRIPTS=/mnt/skills/user/creating-kb/scripts
node $SCRIPTS/build_lexkb.js /mnt/user-data/uploads \
  --out /tmp/kb --name my-kb --zip \
  --source "human description of the corpus"

The script chunks each file, builds the BM25 index, writes the bundle dir (SKILL.md + search.js + index.json + chunks.jsonl), and — with --zip — emits my-kb.skill next to --out.

3. Deliver

Move the .skill to the outputs directory and give the user a download link:

cp /tmp/my-kb.skill /mnt/user-data/outputs/
[Download my-kb.skill](computer:///mnt/user-data/outputs/my-kb.skill)

Tell the user how to deploy it: unzip into an agent's skill directory (or upload it as a skill). The bundle's own SKILL.md then drives querying — the consuming agent reads it, expands each question into search terms, and runs the bundled search.js. No further setup.

Choosing chunk size

The retrieval unit and the reasoning unit are decoupled, which makes chunk size a low-stakes choice. search.py/search.js rank on the whole chunk (best recall) but return only the query-densest passage of it by default (--snippet, ~1200 chars), so a big chunk does not flood the consuming agent's context with surrounding noise. Index for recall; the searcher handles signal.

--target-chars controls chunk size (whole paragraphs are packed up to the target; --target-chars 0 makes each file one chunk). Lexical BM25 tolerates — and on a real-corpus sweep slightly preferred — larger chunks than embedding-based retrieval, because there is no vector to dilute: BM25 scores individual term presence with length normalization, so a big chunk still ranks on the exact terms it contains.

  • Default: --target-chars 0 (whole document). Best recall, fewest chunks; the snippet return keeps reasoning context focused.
  • Long, multi-topic files where you want tighter citation units: 1500–4000.
  • 500 only if you need very fine-grained chunk ids and accept more chunks.

Verifying the bundle

Test before delivering. Run a query against the freshly built bundle and confirm it returns sensible hits:

node /tmp/kb/search.js --query "a representative question" \
  --core "key term" --expand "synonym" --k 3

Each hit's text is the query-focused passage by default; add --snippet 0 to inspect a full chunk.

search.js prints JSON {"hits": [...]}. Confirm the right chunks surface.

What ships in the bundle

FileRole
SKILL.mdthe query protocol the consuming agent follows (expand → search → cite)
search.js / search.pyequivalent BM25 + RM3 + metadata-filter searchers; return query-focused passages (matched sentences kept in neighbour context, merged); the agent runs whichever runtime it has
index.jsonprecomputed inverted index (postings, df, doc lengths, BM25 params)
chunks.jsonlchunk text + structured metadata

Both searchers are thin readers of the same neutral JSON index, so the bundle runs in a Node-only or a Python-only consumer. Metadata stays structured (not folded into the indexed text), which lets the consuming agent filter on it (--filter section=blog, --filter date>=2025).

Scripts

  • scripts/build_lexkb.js — chunker + BM25 index builder + .skill writer.
  • scripts/search.js — the JS runtime searcher, copied verbatim into every bundle. It owns the tokenizer; the builder imports it so index and queries tokenize identically.
  • scripts/search.py — the Python runtime searcher, copied verbatim into every bundle; a thin reader of the same neutral JSON index, parity-pinned to search.js (identical results on a shared index).
  • scripts/zipstore.js — pure-JS ZIP-STORED writer (used by the builder; shared with the in-browser packer).
  • scripts/bundle_SKILL.md — the query-side SKILL.md template written into each bundle.

Iterating

Maintain context across multiple sessions by persisting state in Work Logs.

Environment Detection

Detect environment and load appropriate reference:

if [ "$CLAUDE_CODE_REMOTE" = "true" ]; then
  # Claude Code on the Web (CCotw) - writes to GitHub
  # Read: references/ccotw-environment.md
elif [ -n "$CLAUDE_CODE_REMOTE" ]; then
  # Claude Code CLI - writes to local filesystem
  # Read: references/codecli-environment.md
elif [ -x "$(command -v osascript)" ] && [ -d "/Applications/Claude.app" ]; then
  # Claude Desktop - may write to disk OR output for download
  # Read: references/desktop-environment.md
else
  # Claude.ai (web/chat/native app) - outputs for download
  # Read: references/chat-environment.md
fi

See environment-specific reference for persistence and retrieval details.

Checkpoint Pattern

The iterating skill enforces a checkpoint-and-save pattern to prevent work loss:

  1. Create/Update WorkLog → Output for user → STOP
  2. User saves WorkLog to project knowledge (survives conversation limits)
  3. User says "continue" (same or new conversation)
  4. Make incremental progress on ONE item → Update WorkLog → STOP
  5. Repeat

This matters because separate conversations don't share context automatically: skipping the STOP step ("full waterfall") produces work the user hasn't seen or been able to redirect, and risks losing everything mid-task if the conversation ends before a checkpoint lands.

WorkLog Format

---
version: v1
status: in_progress
---

# [Project Name] Work Log

## v1 | YYYY-MM-DD HH:MM | Title

**Prev:** [previous context OR "Starting new work"]
**Now:** [current goal]

**Progress:** [X% complete OR milestone status]

**Files:**
- `path/to/file.ext` (Why this file matters)
  - L45-67: [What to examine/change here]
  - L123-145: [Another area, specific issue]

**Work:**
+: [additions with file:line]
~: [changes with file:line]
!: [fixes with file:line]

**Decisions:**
- [what]: [why] (vs [alternatives])

**Works:** [effective approaches]
**Fails:** [ineffective approaches, why]

**Blockers:** [None OR specific blocker with owner/ETA]

**Next:**
- [HIGH] [Critical action item]
- [MED] [Important but not urgent]
- [LOW] [Nice to have]

**Open:** [questions needing answers]

Core Workflow

Starting new work:

  1. Detect environment
  2. Create WorkLog v1 with task objective, decisions, file references, next steps
  3. Persist using environment-specific method
  4. STOP - Present WorkLog to user
    • Explain what's planned
    • Tell user to save WorkLog to project knowledge
    • Wait for user to say "continue" before starting work

Continuing work:

  1. Detect environment
  2. Retrieve WorkLog using environment-specific method OR recognize pasted WorkLog
  3. Parse latest version and status
  4. Acknowledge: "From WorkLog vN, status: [status]. Progress: [X%]. Working on: [specific HIGH item]"
  5. Execute ONE HIGH priority item.
  6. Update WorkLog, increment version
  7. Persist using environment-specific method
  8. STOP - Present updated WorkLog to user
    • Summarize what was completed
    • Tell user to save updated WorkLog
    • Wait for user to say "continue" before next item

Recognizing pasted WorkLog: If user pastes content with WorkLog frontmatter at conversation start:

  1. Parse version and status from YAML
  2. Acknowledge: "From WorkLog vN, status: [status]. Task: [objective]. Next: [HIGH item]"
  3. Continue workflow from step 5 above

Version Management

  • Simple incremental: v1 → v2 → v3
  • Frontmatter: version: vN (required)
  • Filename: WorkLog vN.md (optional)
  • Multiple files: Use highest version number

Status States

  • in_progress: Active work continuing
  • blocked: Waiting on external dependency/decision
  • needs_review: Ready for human inspection
  • completed: Task finished

Priority System

Next steps must be prioritized:

  • [HIGH]: Critical items blocking other work
  • [MED]: Important but not urgent
  • [LOW]: Nice-to-have improvements

Claude works on one HIGH priority item per iteration (see Core Workflow above) unless told otherwise — this keeps each checkpoint reviewable instead of bundling several changes into one WorkLog update.

File References

Use relative paths from project root with line ranges:

**Files:**
- `src/auth/oauth.ts` (OAuth implementation needs refactoring)
  - L45-67: Current token validation logic
  - L123-145: Refresh token handling (race condition on L134)

Critical: Use relative paths, NOT absolute /home/claude/ paths (fresh compute each session).

Progress Tracking

Always include progress indicators (token/quota constraints may prevent completing full plan):

**Progress:** 60% complete

Or for longer projects:

**Progress:** Phase 2/3 | Auth ✅ | Payments 50% 🔄 | UI ⏳

What to Document

Include:

  • Key decisions with rationale and alternatives
  • Effective and ineffective approaches
  • Important discoveries
  • File references with line ranges
  • Next steps with priorities
  • Progress indicators
  • Blockers with owner/ETA

Don't include:

  • Minor code changes (use git)
  • Obvious information
  • Raw data dumps
  • Implementation details (use code comments)

User Communication

After creating WorkLog:

  • "Created WorkLog v1. Please save this to project knowledge."
  • "Ready to start when you say 'continue'."

After completing an item:

  • "Completed [item]. Updated WorkLog to vN."
  • "Please save updated WorkLog to project knowledge."
  • "Ready for next item when you say 'continue'."

When continuing:

  • "From WorkLog vN, status: [status]. Progress: [X%]."
  • "Working on: [specific HIGH item]"

Status changes:

  • "Updated WorkLog status to [new_status]: [reason]"
  • "Please save updated WorkLog."

Advanced Patterns

Read references/advanced-patterns.md when:

  • Working on projects spanning 5+ sessions
  • User mentions "debugging strategy", "hypothesis tracking", or "decision evolution"
  • Managing multiple concurrent workstreams
  • User asks about long-running project patterns
  • Blocked on complex issues requiring systematic approach

Otherwise skip - basic workflow above is sufficient for most cases.

Reference Documentation

Mapping Documents

Generate _MAP.md files providing hierarchical document structure with semantic annotations. Maps show section summaries, typed claims (result/definition/method/caveat/open-question), symbol definitions, and cross-section dependencies — all anchored to page numbers.

The structural analog to tree-sitting: tree-sitter parses code via grammar, docmap parses documents via font analysis + LLM extraction.

Installation

pip install pdfplumber anthropic --break-system-packages -q

Generate Maps

# Full run (structure + semantic extraction via Claude API)
python /mnt/skills/user/mapping-documents/scripts/docmap.py paper.pdf \
  --out docs/ --genre paper --workers 4

# Structure only (no API calls, no cost)
python /mnt/skills/user/mapping-documents/scripts/docmap.py paper.pdf \
  --out docs/ --structure-only

API key resolution: --api-key flag > ANTHROPIC_API_KEY env > API_KEY env.

Output Artifacts

Four files, forming a three-layer progressive-disclosure stack:

CLAUDE.md / project instructions     ← curated invariants (you write this)
    ↕ (_USAGE.md bridges the gap)
_MAP.md + JSON indexes               ← navigable document map (docmap generates)
    ↕
raw PDF                              ← the source document
FilePurposeWhen to read
{stem}_USAGE.mdSnippet for pasting into CLAUDE.md / AGENTS.md / project knowledge. Describes the reading order and JSON query patterns.Once, at setup
{stem}_MAP.mdSection map: TOC with summaries, typed claims, defined symbols, dependencies. All page-anchored.Any question about what the document says
{stem}.symbols.jsonFlat symbol index: where defined, where used, what it means."Where is X defined?"
{stem}.anchors.jsonEvery claim: section ID, type, text, page number."What caveats exist?" / "What does §3 claim?"

After Generating: Wire It Up

Generating the map is step 1. Step 2 is telling the agent the map exists.

For a code repo (CLAUDE.md / AGENTS.md):

# Paste the generated usage snippet into your agent instructions
cat docs/paper_USAGE.md >> CLAUDE.md

For Claude.ai project knowledge: Upload _MAP.md as a project knowledge file, or paste the _USAGE.md content into project instructions.

The _USAGE.md snippet includes copy-pasteable query commands for the JSON indexes. Replace QUERY and SECTION_ID placeholders with actual values.

Navigate Via Maps

After generating and wiring up, use the map for navigation — read _MAP.md, not the raw PDF.

Workflow:

  1. Read _USAGE.md block in CLAUDE.md for orientation
  2. Read top-level TOC in _MAP.md for structure and section summaries
  3. Drill into relevant sections for typed claims and symbol definitions
  4. Query .symbols.json for "where is X defined?" lookups
  5. Query .anchors.json for claim filtering by type or section
  6. Read the raw PDF only when exact wording or figures are needed

Querying the JSON indexes:

# Symbol lookup
python3 -c "import json; [print(f'§{s[\"defined_in\"]} p.{s[\"defined_at_page\"]}') \
  for s in json.load(open('docs/paper.symbols.json')) if 'edl' in s['symbol']]"

# All caveats in the document
python3 -c "import json; [print(f'p.{c[\"page\"]} {c[\"text\"]}') \
  for c in json.load(open('docs/paper.anchors.json')) if c['type'] == 'caveat']"

# All claims in a section
python3 -c "import json; [print(f'[{c[\"type\"]}] {c[\"text\"]}') \
  for c in json.load(open('docs/paper.anchors.json')) if c['section'] == '4.3']"

Genre Support

Genre controls the claim taxonomy used in semantic extraction.

GenreClaim typesBest for
paper (default)definition, result, method, claim, caveat, open-questionAcademic papers, arXiv preprints
specrequirement, definition, constraint, example, noteRFCs, API specs, technical standards
legaldefinition, obligation, right, exception, condition, referenceContracts, policy documents, regulations

Limitations (v0.1.x)

  • PDF-only. No DOCX, HTML, or plain text input yet.
  • Single-column layout assumed. Two-column papers may mis-order text within sections.
  • No caching. Re-running re-extracts everything.
  • No citation cross-referencing.
  • Genre must be specified manually.
  • Semantic extraction can hallucinate. Every claim is page-anchored, but the page number comes from the LLM. Verify critical claims against the source.

CLI Reference

python docmap.py paper.pdf [options]

Options:
  --genre {paper,spec,legal}   Claim taxonomy (default: paper)
  --structure-only             Skip LLM pass (free, fast)
  --out DIR                    Output directory (default: .)
  --api-key KEY                Anthropic API key
  --model MODEL                Model (default: claude-sonnet-5-5)
  --workers N                  Parallel workers (default: 4)
  --no-usage-snippet           Skip _USAGE.md generation
  -v                           Verbose structural parsing

Session Memory - Running Notes

Maintain a single structured markdown document that tracks what is happening right now in the current work session. This is within-session continuity, distinct from remember() (cross-session semantic memory) and stash (coarse checkpoint).

When to Use

Invoke on any of these triggers:

  • User says "session notes", "update notes", "start session notes", "show session notes", "note the session", "write it down"
  • User indicates they want continuity across context compression: "before you forget", "write that down", "for when context gets compressed"
  • You have independently completed a meaningful block of work (file edits, a bugfix, a design choice, a reversal after correction) and no note has been updated for a while

Do NOT invoke for:

  • One-off answers with no follow-up
  • Cross-session persistence — that's remember()
  • Coarse checkpoints intended for another session to resume — that's stash

Template (Fixed Sections)

Every session note uses exactly these sections, in this order, with these headers. Preserve the structure on every update. Never rename or reorder sections. Update the content within sections; leave empty sections present with a single _(nothing yet)_ placeholder.

# Session: {short title, 3-8 words, derived from the initial task}

## Current State
_What is actively being worked on right now. Explicit next step._

## Task Specification
_What the user originally asked for. Constraints, acceptance criteria,
design decisions that framed the work._

## Files and Functions
_Important paths touched or referenced, one per line, with a one-line
note on what they contain and why they matter._

## Errors & Corrections
_Errors encountered and how they were fixed. User corrections to my
approach — these have priority over routine progress entries._

## Key Decisions
_Choices made during the session with their rationale. One bullet per
decision. Lead with the choice, then the why._

## Worklog
_Terse, append-only, chronological. One line per attempt or step.
Prefix with ✓ (done), ✗ (failed), → (in progress), or ↺ (reverted)._

Storage

Persist as a procedure memory via the remembering skill, so notes survive container death within the session thread.

from remembering.scripts import remember, recall, supersede

# First write in a session: create the memory
note_id = remember(
    note_markdown,
    "procedure",
    tags=["session-memory", "active"],
    priority=1,
)

# Subsequent updates: find the active note and supersede it
existing = recall(tags_all=["session-memory", "active"], n=1)
if existing:
    note_id = supersede(existing[0].id, updated_markdown, "procedure",
                        tags=["session-memory", "active"], priority=1)

Session boundary: when the user says "session done", "wrap up", "end session", or the conversation is clearly winding down, retag the active note from active to archived by superseding it with the same body and tags=["session-memory", "archived"].

One note is active at a time. If recall(tags_all=["session-memory", "active"]) returns more than one, the oldest is stale — archive it before updating the current one.

Update Discipline

On each invocation:

  1. Read the existing note (if any) via recall(tags_all=["session-memory", "active"], n=1). Work from its current body — do not regenerate from scratch.
  2. Update in place. Edit the relevant sections; append to ## Worklog. Do not drop prior content unless it is now wrong (then move the correction to ## Errors & Corrections).
  3. Prioritize user corrections. When the user pushes back or corrects an approach, that goes into ## Errors & Corrections before other updates.
  4. Deduplicate against stash and explicit memories. If an item is already captured in a stash or a remember() call, reference it by ID rather than restating. Notes are supplementary, not duplicative.
  5. Supersede, don't append. Write the full updated document back via supersede(). This keeps exactly one active note per session.

Budget

Target ~12K tokens for the note document. Prefer concise phrasing, but do not truncate substantive content to hit the target — trigger ## Key Decisions consolidation (collapse related bullets) before cutting. If the document exceeds ~20K tokens, compress ## Worklog first (merge consecutive ✓ entries into a single summary line, keep corrections and decisions intact).

Rationale: we run on 200K–1M context models, so the original 2K budget from the issue spec was over-constrained. 12K matches Claude Code's upstream design and leaves ample room for the surrounding conversation.

Surface to User

When the user asks to "show session notes", print the current note body verbatim in a fenced code block. Do not paraphrase.

When updating silently (you triggered it yourself), confirm with a single line: Updated session notes (note id: <short-id>). Do not dump the full body unsolicited.

Invariants

  • Section headers and order are fixed. Updates change content, not structure.
  • User corrections take priority over routine progress in ordering and detail.
  • Notes do not duplicate what is already in stash or explicit memories.
  • Manual invocation always works. Automatic triggers are a convenience, not a requirement.
  • Exactly one session-memory + active memory exists per session. Stale actives are archived, not left dangling.

Related

  • remembering — cross-session memory store (where these notes persist)
  • stash-resume-protocol (ops) — coarse session checkpoints
  • context-hygiene (ops) — when and what to offload from context

Tracking Todos

Maintain a structured, persistent task list while working through multi-step requests. Track progress explicitly, mark completion honestly, and make the plan visible to Oskar.

When to Use

Use proactively when:

  • Request requires 3+ distinct steps or actions
  • User provides multiple tasks (numbered, comma-separated, or a list)
  • Work spans multiple tool calls, file edits, or research phases
  • Non-trivial planning would otherwise happen implicitly in your head
  • User explicitly asks for a todo list

Skip when:

  • Single, straightforward task
  • Purely conversational or informational exchange
  • Task completes in <3 trivial steps
  • Quick fact lookup or recall

When in doubt, use it. Being explicit about multi-step plans makes failure modes visible.

Schema

Each todo has exactly three string fields:

  • content — imperative form ("Run tests", "Fix auth bug")
  • status — one of: pending, in_progress, completed
  • activeForm — present continuous ("Running tests", "Fixing auth bug")

Both content and activeForm are required for every todo. The activeForm is what gets shown while a task is executing.

API

from todos import get_todos, write_todos, abandon, render

# Read current list
todos = get_todos()

# Replace entire list (Claude Code semantics — one call, whole list)
write_todos([
    {"content": "Fetch issue body", "status": "in_progress", "activeForm": "Fetching issue body"},
    {"content": "Draft fix plan", "status": "pending", "activeForm": "Drafting fix plan"},
    {"content": "Implement and test", "status": "pending", "activeForm": "Implementing and testing"},
])

# Display to Oskar
print(render())

# Discard remaining todos when starting fresh unrelated work
abandon()

write_todos() validates schema and that at most one item is in_progress. When every item is completed, it auto-clears the list (matches Claude Code behavior — done-state is empty-state).

Behavioral Rules

One in_progress at a time, strictly. Before starting work on a task, set it to in_progress. Never have two in_progress items simultaneously. The validator enforces this.

Mark complete IMMEDIATELY. Update to completed the moment the work is actually done. Don't batch completions at the end — that defeats the tracking purpose and erases information about when things were finished.

Only mark complete if FULLY done. If tests fail, if the implementation is partial, if you hit an unresolved error — keep the task in_progress and add a new task describing the blocker. Lying about completion status is the failure mode this tool exists to prevent.

Remove irrelevant tasks. If a task becomes obsolete (scope change, wrong approach), remove it entirely from the list rather than marking it done. The list should reflect actual remaining work.

Break complex tasks down. Vague items like "implement the feature" defeat the point. Prefer specific actions: "Add validator to route handler", "Write 3 test cases for edge inputs", "Run existing test suite".

Storage Model

Todos live in Muninn's config store under key active-todos (category=ops). This means:

  • Survives container death within the same conversation
  • Persists across conversations until explicitly cleared or auto-cleared when all done
  • Cross-conversation persistence is a feature: unfinished work shows up next session as a nudge

If you start a conversation and find stale todos that don't apply, call abandon() to clear them.

Known Limitations

Single writer assumed. The storage uses read-modify-write without locking. If a scheduled CCotw task and a live conversation both call write_todos() concurrently, the second write wins. Muninn runs single-threaded within a conversation, and scheduled tasks (perch, zeitgeist) don't use this skill, so the assumption holds in practice. Don't use this skill from concurrent agents without adding a lock.

Global scope is intentional, not per-conversation. Unfinished work leaks into the next conversation by design — it's a nudge that something was left open. If stale todos from prior work become noise, call abandon() to clear.

Replace-whole-list API vs delta updates. Matches Claude Code's TodoWrite exactly. The LLM failure mode (forgetting items when rewriting the full list) is mitigated by the prompt, not by the API shape. If omissions become a pattern, re-render the current list before composing the new one.

Display Pattern

When the plan is substantive, show it to Oskar once after creating it, then rely on inline mentions rather than re-rendering the full list on every update. Noise defeats the signal.

▶ Fetching issue body
☐ Drafting fix plan
☐ Implementing and testing

Examples

Good use case — multi-file refactor:

write_todos([
    {"content": "Find all call sites of getCwd", "status": "in_progress", "activeForm": "Finding all call sites of getCwd"},
    {"content": "Rename occurrences in src/utils/", "status": "pending", "activeForm": "Renaming occurrences in src/utils/"},
    {"content": "Rename occurrences in src/agents/", "status": "pending", "activeForm": "Renaming occurrences in src/agents/"},
    {"content": "Run test suite", "status": "pending", "activeForm": "Running test suite"},
])

Bad use case — should not use:

User: "What's the current time in Tokyo?"
→ Single fact lookup. Skip todos.

Mid-work update (after finishing step 1, hitting a blocker in step 2):

write_todos([
    {"content": "Find all call sites of getCwd", "status": "completed", "activeForm": "Finding all call sites of getCwd"},
    {"content": "Rename occurrences in src/utils/", "status": "completed", "activeForm": "Renaming occurrences in src/utils/"},
    {"content": "Resolve type error in AgentContext after rename", "status": "in_progress", "activeForm": "Resolving type error in AgentContext after rename"},
    {"content": "Rename occurrences in src/agents/", "status": "pending", "activeForm": "Renaming occurrences in src/agents/"},
    {"content": "Run test suite", "status": "pending", "activeForm": "Running test suite"},
])

Note: the blocker became a new task rather than marking step 3 complete.

Updating Knowledge

Core Principles

  • Systematic > ad-hoc - Follow research methodology, don't scatter searches randomly
  • Cross-validate - Verify claims across multiple independent sources
  • Confidence tracking - Explicitly flag single-source claims and gaps
  • Concise output - Present findings efficiently, introspection in thinking blocks

Preflight Check

This skill requires web_search tool access.

Verify it's available before proceeding. If it isn't, tell the user immediately — "I need web search enabled for research tasks. Please toggle on 'Web search' in the feature menu." — and stop there rather than researching from memory alone or spending context on workarounds.

Trigger Disambiguation

Avoid triggering on generic action verbs like "update" (which could mean file/code changes instead) — trigger on the explicit knowledge-gathering requests named in the description above.

Research Workflow

1. Scope Definition (internal)

Identify in thinking block:

  • What specifically needs investigation
  • Current knowledge baseline
  • Known gaps
  • Required depth (overview vs comprehensive)

2. Source Gathering (2-10 tool calls)

Source priority:

  1. Official documentation, project repos, company announcements
  2. Academic papers, technical blogs with clear expertise
  3. Community discussions (for ecosystem context only)

Tool selection:

  • web_search - Find authoritative sources
  • web_fetch - Extract complete content from any URL (no prior search required)
  • Internal tools (GitHub, Drive, etc.) - Company/personal context
  • Multiple tool types as needed

Gather 3-5 diverse sources minimum for cross-validation.

3. Synthesis (internal)

In thinking block:

  • Cross-validate major claims across sources
  • Note contradictions explicitly
  • Track publication dates
  • Assess source credibility
  • Identify remaining gaps

4. Output (concise)

Present findings concisely:

## [Topic]

**Key findings:**
[2-3 sentence summary with most important updates]

**Current state:**
[Concise description of what's true now]

**Changes since cutoff:**
[Only if relevant - what's different]

**Gaps:**
[Only if user needs to know - what wasn't found]

Sources: [URLs]

Confidence indicators:

  • State explicitly when claims are single-source
  • Note contradictions between sources
  • Flag unverified information
  • Quantify confidence (high/medium/low) when uncertain

Quality Standards

Minimum requirements:

  • 3+ independent sources consulted
  • Major claims cross-validated
  • Explicit confidence assessment
  • Remaining gaps identified when relevant

Avoid:

  • Single-source claims presented as definitive
  • Ignoring publication dates
  • Marketing language as fact
  • Verbose deliberation in final output (use thinking blocks)
  • Over-explaining research process to user

Tool Usage Notes

web_fetch capabilities:

  • Can fetch ANY URL directly - doesn't require prior search results
  • Use for complete documentation pages, technical articles
  • Effective for deep content extraction after search identifies sources

web_search patterns:

  • Start specific, broaden if needed
  • Never repeat similar queries - make each unique
  • Use the actual current date when relevant — check it rather than assuming

Internal tools:

  • Prioritize for company/personal data
  • Combine with web tools for comparative analysis

Example Outputs

Technical documentation inquiry:

TypeScript 5.3 (Nov 2023) added import attributes for JSON modules. The syntax uses `with` instead of deprecated `assert`. Runtime performance unchanged—validation happens at parse time.

Gap: No official benchmarks comparing validation overhead across bundlers.

Sources: [TS release notes, GitHub issue #54242, esbuild docs]

Conflicting information:

React Server Components ship in Next.js 13.4+ (stable) and Remix 2.0 (experimental). 

Contradiction: Meta's blog claims "production-ready" while Remix docs flag "breaking changes expected." Cross-validation shows Meta refers to Next.js implementation only.

Confidence: High on Next.js status, medium on Remix timeline.

Sources: [Next.js changelog, Remix docs, React.dev]

Avoid narrating process:

❌ "After conducting extensive research and cross-validating multiple authoritative sources..."
✓ Just present findings with source attribution

Sponsors

Details

Marketplace
oaustegard/claude-skills
Source
plugins/knowledge-and-memory
Category
Knowledge And Memory
License
MIT
Keywords
asking-questions, bm25, cloning-project, creating-kb, iterating, mapping-documents, session-memory, tracking-todos, updating-knowledge

More from oaustegard/claude-skills

All 14