
knowledge-and-memory
Research methodology, corpus search, persistent memory, iterative workflows, and project knowledge management.
- version
- 1.3.0
- GitHub stars
- 404
Install
/plugin install knowledge-and-memory@oaustegard-claude-skillsRun it inside Claude Code after adding the marketplace with /plugin marketplace add oaustegard/claude-skills.
Contents
Discovered from sourceSkills 9
Asking Questions
Purpose
Ask clarifying questions when the answer materially changes what you'll build. This skill helps identify when to ask, how to structure questions effectively, and when to proceed autonomously.
When to Use
Ask questions for:
- Ambiguous implementation choices - Multiple valid technical approaches (middleware vs wrapper functions, library selection, architectural patterns)
- Missing critical context - Specific information needed (database type, deployment platform, credential management)
- Potentially destructive actions - Requests that could be interpreted dangerously ("clean up files" = delete vs archive)
- Scope clarification - Vague terms like "refactor," "optimize," or "improve"
- Conflicting requirements - Goals that may work against each other ("make it faster" + "add extensive logging")
- Technical trade-offs - Solutions with different costs/benefits depending on priorities
When NOT to Use
Don't ask when:
- Request is clear and unambiguous - One obvious implementation path
- You can determine the answer from context - Codebase patterns, project structure, existing conventions
- Over-clarification - Questions that don't materially affect implementation
- Standard engineering practices - Established patterns already in the codebase
Question Structure
Template
[Context: What you found/analyzed]
[Present 2-5 specific options with brief trade-offs]
[Direct question asking for preference]
[Optional: Offer to make reasonable default choice]
Guidelines
-
Acknowledge understanding first - Show you've analyzed the situation
- "I found your API endpoints and see you're using Express..."
-
Present clear options - Offer 2-5 specific choices with brief context
I can implement this in several ways: 1. **Global middleware** - Catches all errors centrally (simplest) 2. **Wrapper functions** - More granular control per endpoint 3. **Custom error classes** - Typed errors with status codes -
Ask directly - Clear question that guides decision
- "Which approach fits your needs?"
-
Offer autonomy (optional) - For less critical decisions
- "I can implement option 2 as it's the safest, or let me know if you prefer something else."
Progressive Disclosure
Layer questions instead of asking everything upfront:
Good ✓
- First: "For real-time updates, should I use WebSockets, Server-Sent Events, or polling?"
- Then: "For WebSockets, do you want Socket.io (easier) or native (lighter)?"
Bad ✗
- "For real-time updates should I use WebSockets or SSE or polling and if WebSockets should I use Socket.io or native and should I implement reconnection and what about authentication and..."
Examples
Good ✓
"I see you're using JWT authentication. To add refresh tokens, I can:
- httpOnly cookies - More secure, harder to XSS
- localStorage - Simpler, works with mobile apps
- In-memory only - Most secure, lost on refresh
What works best for your use case?"
Bad ✗
"How should I implement the authentication refresh token storage mechanism considering security implications, XSS vulnerabilities, mobile compatibility, UX impacts, and compliance considerations?"
Too verbose, no clear options, asks everything at once
Good ✓
"You mentioned 'clean up migrations.' Do you want me to archive them to /old-migrations or delete them entirely? (Note: deletion can break databases that haven't run them yet)"
Bad ✗
"What do you mean by clean up?"
Too vague, doesn't guide the decision
After Receiving an Answer
- Acknowledge - Confirm you understood their choice
- Proceed immediately - Don't ask if they want you to continue
- Apply preferences - If they chose "security first" once, lean that direction for similar future choices
Key Principle
Ask only when the answer materially changes what you'll build. Avoid building the wrong thing, not asking questions for the sake of asking.
bm25
Ranked content search over any text corpus. One CLI, in-memory BM25 index per process, with a session-local disk cache so repeat invocations against the same corpus load in tens of milliseconds instead of rebuilding.
Setup
uv pip install --system --break-system-packages bm25s
Install is sub-second on a warm uv cache. That's the entire dependency.
Usage
BM25=/mnt/skills/user/bm25/scripts/bm25.py
# Local directory
python3 $BM25 ./repo 'csrf middleware'
# Multiple queries against the same in-memory index (build once, query many)
python3 $BM25 ./repo 'csrf middleware' 'session backend' 'queryset filter'
# Cloned GitHub repo via tarball (one HTTP call)
python3 $BM25 'github.com/django/django' 'atomic transaction'
python3 $BM25 'github.com/django/django@stable/5.0.x' 'atomic transaction'
# Project knowledge or uploads
python3 $BM25 project 'RAG scaling laws'
python3 $BM25 uploads 'tax loss harvesting'
# Filters
python3 $BM25 ./repo 'auth flow' --exclude 'tests/*' --exclude '*/tests/*'
python3 $BM25 ./repo 'config' --include '*.py' --include '*.toml'
# Interactive (REPL — single corpus, many queries)
python3 $BM25 ./repo --interactive
# JSON output for piping
python3 $BM25 ./repo 'auth flow' --json
Corpus types
| Spec | Meaning |
|---|---|
./path or /abs/path | Local directory |
uploads | /mnt/user-data/uploads/ |
project | /mnt/project/ |
github.com/owner/repo[@ref] | Tarball fetch via GitHub API (GH_TOKEN used if set) |
Options
| Option | Default | Description |
|---|---|---|
--top-k N | 10 | Results per query |
--include GLOB | (auto) | Repeatable. If set, only files matching one of these globs are indexed |
--exclude GLOB | Repeatable. Skip files matching these globs | |
--snippet-lines N | 3 | Lines of snippet context per hit (0 = none) |
--max-file-bytes N | 2,000,000 | Skip files larger than this |
--json | Machine-readable output | |
--interactive / -i | REPL mode for ad-hoc querying within one session | |
--stats | Print discover + index timings as JSON | |
--no-cache | Bypass the session-local index cache; build in-memory only |
With no --include, a default set of text/code extensions is indexed (Python,
JS/TS, Go, Rust, Markdown, JSON, YAML, etc.). Standard noise dirs are skipped
unconditionally: .git, node_modules, __pycache__, .venv, dist, etc.
When to use bm25
| Question shape | Tool |
|---|---|
"Find lines matching class.*Error" | grep / ripgrep |
"Show me where parse_input is defined" | tree-sitting (find:/source:) |
| "Which files are about CSRF handling?" | bm25 |
| "Rank these docs by relevance to 'rate limiting strategies'" | bm25 |
| "What's the implementation of the atomic transaction context manager?" | bm25, then tree-sitting source: |
| "Find code by natural-language concept (in a code repo)" | searching-codebases (which has its own TF-IDF mode) |
The boundary with searching-codebases: that skill is code-specific (routes
between regex and TF-IDF, expands via tree-sitting AST). bm25 is the simpler
general-purpose tool — any corpus, no AST awareness, no routing. Prefer
searching-codebases for code; reach for bm25 when the corpus is mixed
(docs + code), non-code (notes, transcripts, PDFs converted to text), or when
you specifically want BM25's length-normalized scoring.
Design notes
- Session-local disk cache at
/home/claude/.bm25-cache/<key>/. The key is a hash of(resolved_corpus_path, include_globs, exclude_globs, max_file_bytes)plus every discovered file's path and content hash, so editing, adding or deleting a file invalidates it. A hit still reads the corpus to hash it but skips tokenizing and indexing (1,649 files: 0.35s hit vs ~1s rebuild). The cache lives in/home/claude, which is ephemeral, so it expires at the session boundary — same lifetime as the corpus state itself, no cross-session staleness. ~5–35MB per cached index, depending on corpus size. --no-cachebypasses both load and save — useful to confirm a rebuild matches.- A corpus with no indexable terms (empty, or only one-character tokens) returns zero-score hits instead of crashing bm25s.
- Reuse within a single invocation. The retriever stays in memory
between queries in one process. Passing multiple queries positionally,
or using
--interactive, amortizes any rebuild cost across queries. - No AST awareness. Chunking is per-file. For symbol-level results in
code, combine with
tree-sittingqueries on the same paths. - Tokenizer. Default
bm25s.tokenizewith stopwords disabled — over a small Django sample, AST-derived token streams (identifiers/strings/ comments only) gave near-identical rankings, so we don't bother.
Output format
Default (human-readable):
QUERY: csrf middleware
----------------------------------------------------------------------
1. 5.51 django/core/checks/security/csrf.py
def _csrf_middleware():
return "django.middleware.csrf.CsrfViewMiddleware" in settings.MIDDLEWARE
2. 5.34 docs/howto/csrf.txt
...
--json produces {"query": ..., "results": [{"path", "score", "snippet"}, ...]}.
Architecture
bm25.py CLI
├── resolve_corpus(spec) → local Path (downloads tarball if github.com/...)
├── cache_key(...) → 16-hex sha256 of inputs that determine the index
├── CorpusIndex.load(cache_dir) → returns cached index if present, else None
├── CorpusIndex.build(...) → walks files, tokenizes, indexes with bm25s
├── CorpusIndex.save(cache_dir) → persists to /home/claude/.bm25-cache/<key>/
├── query(q, k) → ranked (doc_idx, score) pairs
└── best_snippet(doc, q, lines) → pick line w/ most query-term hits + context
Cache contents per directory:
bm25/— bm25s.BM25.save() output (NumPy arrays + vocab)corpus.pkl— pickled{paths, docs}so we can render snippets without re-reading the source filesmanifest.json— corpus root, files count, built_at timestamp
No network beyond optional tarball fetch on github.com/... corpora. No
state outside /home/claude/, which is ephemeral.
Cloning Project
When users request to clone, copy, export, or backup their current project:
When NOT to use this skill
"Project" here is a Claude.ai workspace, never a code repository. The word collides badly, so route by what is actually being copied:
| Situation | Use |
|---|---|
| Clone or fetch a git repo | accessing-github-repos |
| "I just cloned this, what is it?" | exploring-codebases |
| What a codebase contains or exposes | tree-sitting |
| Back up a repo's files | git, not this |
This skill reads the Claude.ai workspace you are talking in. It touches no checkout and knows nothing about code.
Step 1: Verify Project Context
Check if the user is in a project by looking for project-specific indicators:
- Project instructions in the context window
- Knowledge files mentioned in
<available_skills>or similar sections - User explicitly stating they're in a project
If not in a project, inform the user that this skill requires being inside a Claude project.
Step 2: Gather Project Instructions
Project instructions are typically found in the context window as XML or markdown content describing the project's purpose and configuration. Extract and save these instructions to a file.
Create the instructions file:
cat > /home/claude/project-instructions.md << 'INSTRUCTIONS'
[Extracted project instructions content]
INSTRUCTIONS
Step 3: Locate Knowledge Files
Knowledge files are user-uploaded documents stored in /mnt/user-data/uploads/. List all files:
ls -lh /mnt/user-data/uploads/
If knowledge files exist, copy them to the working directory for bundling:
cp /mnt/user-data/uploads/* /home/claude/project-export/
Step 4: Create Export Bundle
If only project instructions exist (no knowledge files):
- Copy instructions directly to outputs:
cp /home/claude/project-instructions.md /mnt/user-data/outputs/
If project has both instructions and knowledge files:
- Create an organized export directory:
mkdir -p /home/claude/project-export cp /home/claude/project-instructions.md /home/claude/project-export/ cp /mnt/user-data/uploads/* /home/claude/project-export/ 2>/dev/null || true - Create zip bundle:
cd /home/claude zip -r /mnt/user-data/outputs/project-export.zip project-export/ - Show bundle contents:
unzip -l /mnt/user-data/outputs/project-export.zip
Step 5: Provide Files and Instructions to User
Link the exported files:
For single instructions file:
[Download project-instructions.md](computer:///mnt/user-data/outputs/project-instructions.md)
For bundled export:
[Download project-export.zip](computer:///mnt/user-data/outputs/project-export.zip)
Then provide clear setup instructions:
## How to Import Into a New Project
1. **Create a new Claude project** (or open an existing one where you want to clone this configuration)
2. **Add project instructions:**
- Open the new project's settings
- Navigate to the "Instructions" section
- Copy and paste the content from `project-instructions.md`
- Save the instructions
3. **Upload knowledge files** (if applicable):
- In the new project, go to the "Knowledge" section
- Upload all files from the export (except project-instructions.md)
- Files will become available as project knowledge
Your project is now cloned with the same configuration and knowledge base.
Edge Cases
Empty project (no custom instructions or knowledge):
- Inform the user that the project has no custom configuration to export
- Explain that projects by default only have access to Claude's base capabilities
Large knowledge bases:
- If the zip file exceeds reasonable size (~50MB), warn the user and suggest selective export
- Provide individual file links as fallback
Corrupted or inaccessible files:
- Skip files that cannot be read
- Report which files were successfully exported vs. skipped
- Provide export anyway with available content
creating-kb
Turn a pile of files into a portable, deployable knowledgebase. The output
is a .skill bundle — an ordinary zip — containing a BM25 inverted index, the
chunk text, a pure-Node searcher, and a query protocol. It has no embedding
model and no semantic search: retrieval is lexical, and the consuming agent
supplies the semantic layer by expanding the query at search time. That is what
makes the bundle portable — any agent that can run node can query it with no
npm install, no model download, and no network.
The whole toolchain is JavaScript so one implementation serves both this builder and the in-browser packer. Build with the bundled script; do not hand-roll the index.
SCRIPTS=/mnt/skills/user/creating-kb/scripts
node $SCRIPTS/build_lexkb.js CORPUS_DIR --out /tmp/kb --name my-kb --zip
Workflow
1. Gather the sources
Collect the files into one directory. In a Claude.ai chat, uploads land in
/mnt/user-data/uploads/ — point the builder there. Otherwise use any path the
user names. Supported extensions default to txt,md,html,htm; pass --ext to
change them.
This MVP interface is bounded by how many files a chat can accept. For a large corpus, stage the files in a directory first, or use the browser packer (built from the same scripts) that runs entirely client-side.
2. Build the bundle
SCRIPTS=/mnt/skills/user/creating-kb/scripts
node $SCRIPTS/build_lexkb.js /mnt/user-data/uploads \
--out /tmp/kb --name my-kb --zip \
--source "human description of the corpus"
The script chunks each file, builds the BM25 index, writes the bundle dir
(SKILL.md + search.js + index.json + chunks.jsonl), and — with --zip —
emits my-kb.skill next to --out.
3. Deliver
Move the .skill to the outputs directory and give the user a download link:
cp /tmp/my-kb.skill /mnt/user-data/outputs/
[Download my-kb.skill](computer:///mnt/user-data/outputs/my-kb.skill)
Tell the user how to deploy it: unzip into an agent's skill directory (or upload
it as a skill). The bundle's own SKILL.md then drives querying — the consuming
agent reads it, expands each question into search terms, and runs the bundled
search.js. No further setup.
Choosing chunk size
The retrieval unit and the reasoning unit are decoupled, which makes chunk size a
low-stakes choice. search.py/search.js rank on the whole chunk (best
recall) but return only the query-densest passage of it by default
(--snippet, ~1200 chars), so a big chunk does not flood the consuming agent's
context with surrounding noise. Index for recall; the searcher handles signal.
--target-chars controls chunk size (whole paragraphs are packed up to the
target; --target-chars 0 makes each file one chunk). Lexical BM25 tolerates —
and on a real-corpus sweep slightly preferred — larger chunks than
embedding-based retrieval, because there is no vector to dilute: BM25 scores
individual term presence with length normalization, so a big chunk still ranks on
the exact terms it contains.
- Default:
--target-chars 0(whole document). Best recall, fewest chunks; the snippet return keeps reasoning context focused. - Long, multi-topic files where you want tighter citation units:
1500–4000. 500only if you need very fine-grained chunk ids and accept more chunks.
Verifying the bundle
Test before delivering. Run a query against the freshly built bundle and confirm it returns sensible hits:
node /tmp/kb/search.js --query "a representative question" \
--core "key term" --expand "synonym" --k 3
Each hit's text is the query-focused passage by default; add --snippet 0 to
inspect a full chunk.
search.js prints JSON {"hits": [...]}. Confirm the right chunks surface.
What ships in the bundle
| File | Role |
|---|---|
SKILL.md | the query protocol the consuming agent follows (expand → search → cite) |
search.js / search.py | equivalent BM25 + RM3 + metadata-filter searchers; return query-focused passages (matched sentences kept in neighbour context, merged); the agent runs whichever runtime it has |
index.json | precomputed inverted index (postings, df, doc lengths, BM25 params) |
chunks.jsonl | chunk text + structured metadata |
Both searchers are thin readers of the same neutral JSON index, so the bundle
runs in a Node-only or a Python-only consumer. Metadata stays structured (not
folded into the indexed text), which lets the consuming agent filter on it
(--filter section=blog, --filter date>=2025).
Scripts
scripts/build_lexkb.js— chunker + BM25 index builder +.skillwriter.scripts/search.js— the JS runtime searcher, copied verbatim into every bundle. It owns the tokenizer; the builder imports it so index and queries tokenize identically.scripts/search.py— the Python runtime searcher, copied verbatim into every bundle; a thin reader of the same neutral JSON index, parity-pinned tosearch.js(identical results on a shared index).scripts/zipstore.js— pure-JS ZIP-STORED writer (used by the builder; shared with the in-browser packer).scripts/bundle_SKILL.md— the query-side SKILL.md template written into each bundle.
Iterating
Maintain context across multiple sessions by persisting state in Work Logs.
Environment Detection
Detect environment and load appropriate reference:
if [ "$CLAUDE_CODE_REMOTE" = "true" ]; then
# Claude Code on the Web (CCotw) - writes to GitHub
# Read: references/ccotw-environment.md
elif [ -n "$CLAUDE_CODE_REMOTE" ]; then
# Claude Code CLI - writes to local filesystem
# Read: references/codecli-environment.md
elif [ -x "$(command -v osascript)" ] && [ -d "/Applications/Claude.app" ]; then
# Claude Desktop - may write to disk OR output for download
# Read: references/desktop-environment.md
else
# Claude.ai (web/chat/native app) - outputs for download
# Read: references/chat-environment.md
fi
See environment-specific reference for persistence and retrieval details.
Checkpoint Pattern
The iterating skill enforces a checkpoint-and-save pattern to prevent work loss:
- Create/Update WorkLog → Output for user → STOP
- User saves WorkLog to project knowledge (survives conversation limits)
- User says "continue" (same or new conversation)
- Make incremental progress on ONE item → Update WorkLog → STOP
- Repeat
This matters because separate conversations don't share context automatically: skipping the STOP step ("full waterfall") produces work the user hasn't seen or been able to redirect, and risks losing everything mid-task if the conversation ends before a checkpoint lands.
WorkLog Format
---
version: v1
status: in_progress
---
# [Project Name] Work Log
## v1 | YYYY-MM-DD HH:MM | Title
**Prev:** [previous context OR "Starting new work"]
**Now:** [current goal]
**Progress:** [X% complete OR milestone status]
**Files:**
- `path/to/file.ext` (Why this file matters)
- L45-67: [What to examine/change here]
- L123-145: [Another area, specific issue]
**Work:**
+: [additions with file:line]
~: [changes with file:line]
!: [fixes with file:line]
**Decisions:**
- [what]: [why] (vs [alternatives])
**Works:** [effective approaches]
**Fails:** [ineffective approaches, why]
**Blockers:** [None OR specific blocker with owner/ETA]
**Next:**
- [HIGH] [Critical action item]
- [MED] [Important but not urgent]
- [LOW] [Nice to have]
**Open:** [questions needing answers]
Core Workflow
Starting new work:
- Detect environment
- Create WorkLog v1 with task objective, decisions, file references, next steps
- Persist using environment-specific method
- STOP - Present WorkLog to user
- Explain what's planned
- Tell user to save WorkLog to project knowledge
- Wait for user to say "continue" before starting work
Continuing work:
- Detect environment
- Retrieve WorkLog using environment-specific method OR recognize pasted WorkLog
- Parse latest version and status
- Acknowledge: "From WorkLog vN, status: [status]. Progress: [X%]. Working on: [specific HIGH item]"
- Execute ONE HIGH priority item.
- Update WorkLog, increment version
- Persist using environment-specific method
- STOP - Present updated WorkLog to user
- Summarize what was completed
- Tell user to save updated WorkLog
- Wait for user to say "continue" before next item
Recognizing pasted WorkLog: If user pastes content with WorkLog frontmatter at conversation start:
- Parse version and status from YAML
- Acknowledge: "From WorkLog vN, status: [status]. Task: [objective]. Next: [HIGH item]"
- Continue workflow from step 5 above
Version Management
- Simple incremental: v1 → v2 → v3
- Frontmatter:
version: vN(required) - Filename:
WorkLog vN.md(optional) - Multiple files: Use highest version number
Status States
- in_progress: Active work continuing
- blocked: Waiting on external dependency/decision
- needs_review: Ready for human inspection
- completed: Task finished
Priority System
Next steps must be prioritized:
- [HIGH]: Critical items blocking other work
- [MED]: Important but not urgent
- [LOW]: Nice-to-have improvements
Claude works on one HIGH priority item per iteration (see Core Workflow above) unless told otherwise — this keeps each checkpoint reviewable instead of bundling several changes into one WorkLog update.
File References
Use relative paths from project root with line ranges:
**Files:**
- `src/auth/oauth.ts` (OAuth implementation needs refactoring)
- L45-67: Current token validation logic
- L123-145: Refresh token handling (race condition on L134)
Critical: Use relative paths, NOT absolute /home/claude/ paths (fresh compute each session).
Progress Tracking
Always include progress indicators (token/quota constraints may prevent completing full plan):
**Progress:** 60% complete
Or for longer projects:
**Progress:** Phase 2/3 | Auth ✅ | Payments 50% 🔄 | UI ⏳
What to Document
Include:
- Key decisions with rationale and alternatives
- Effective and ineffective approaches
- Important discoveries
- File references with line ranges
- Next steps with priorities
- Progress indicators
- Blockers with owner/ETA
Don't include:
- Minor code changes (use git)
- Obvious information
- Raw data dumps
- Implementation details (use code comments)
User Communication
After creating WorkLog:
- "Created WorkLog v1. Please save this to project knowledge."
- "Ready to start when you say 'continue'."
After completing an item:
- "Completed [item]. Updated WorkLog to vN."
- "Please save updated WorkLog to project knowledge."
- "Ready for next item when you say 'continue'."
When continuing:
- "From WorkLog vN, status: [status]. Progress: [X%]."
- "Working on: [specific HIGH item]"
Status changes:
- "Updated WorkLog status to [new_status]: [reason]"
- "Please save updated WorkLog."
Advanced Patterns
Read references/advanced-patterns.md when:
- Working on projects spanning 5+ sessions
- User mentions "debugging strategy", "hypothesis tracking", or "decision evolution"
- Managing multiple concurrent workstreams
- User asks about long-running project patterns
- Blocked on complex issues requiring systematic approach
Otherwise skip - basic workflow above is sufficient for most cases.
Reference Documentation
- references/chat-environment.md - Claude.ai (web/chat/native app)
- references/desktop-environment.md - Claude Desktop
- references/codecli-environment.md - Claude Code CLI
- references/ccotw-environment.md - Claude Code on the Web
Mapping Documents
Generate _MAP.md files providing hierarchical document structure with semantic annotations. Maps show section summaries, typed claims (result/definition/method/caveat/open-question), symbol definitions, and cross-section dependencies — all anchored to page numbers.
The structural analog to tree-sitting: tree-sitter parses code via grammar, docmap parses documents via font analysis + LLM extraction.
Installation
pip install pdfplumber anthropic --break-system-packages -q
Generate Maps
# Full run (structure + semantic extraction via Claude API)
python /mnt/skills/user/mapping-documents/scripts/docmap.py paper.pdf \
--out docs/ --genre paper --workers 4
# Structure only (no API calls, no cost)
python /mnt/skills/user/mapping-documents/scripts/docmap.py paper.pdf \
--out docs/ --structure-only
API key resolution: --api-key flag > ANTHROPIC_API_KEY env > API_KEY env.
Output Artifacts
Four files, forming a three-layer progressive-disclosure stack:
CLAUDE.md / project instructions ← curated invariants (you write this)
↕ (_USAGE.md bridges the gap)
_MAP.md + JSON indexes ← navigable document map (docmap generates)
↕
raw PDF ← the source document
| File | Purpose | When to read |
|---|---|---|
{stem}_USAGE.md | Snippet for pasting into CLAUDE.md / AGENTS.md / project knowledge. Describes the reading order and JSON query patterns. | Once, at setup |
{stem}_MAP.md | Section map: TOC with summaries, typed claims, defined symbols, dependencies. All page-anchored. | Any question about what the document says |
{stem}.symbols.json | Flat symbol index: where defined, where used, what it means. | "Where is X defined?" |
{stem}.anchors.json | Every claim: section ID, type, text, page number. | "What caveats exist?" / "What does §3 claim?" |
After Generating: Wire It Up
Generating the map is step 1. Step 2 is telling the agent the map exists.
For a code repo (CLAUDE.md / AGENTS.md):
# Paste the generated usage snippet into your agent instructions
cat docs/paper_USAGE.md >> CLAUDE.md
For Claude.ai project knowledge:
Upload _MAP.md as a project knowledge file, or paste the _USAGE.md content into project instructions.
The _USAGE.md snippet includes copy-pasteable query commands for the JSON indexes. Replace QUERY and SECTION_ID placeholders with actual values.
Navigate Via Maps
After generating and wiring up, use the map for navigation — read _MAP.md, not the raw PDF.
Workflow:
- Read
_USAGE.mdblock in CLAUDE.md for orientation - Read top-level TOC in
_MAP.mdfor structure and section summaries - Drill into relevant sections for typed claims and symbol definitions
- Query
.symbols.jsonfor "where is X defined?" lookups - Query
.anchors.jsonfor claim filtering by type or section - Read the raw PDF only when exact wording or figures are needed
Querying the JSON indexes:
# Symbol lookup
python3 -c "import json; [print(f'§{s[\"defined_in\"]} p.{s[\"defined_at_page\"]}') \
for s in json.load(open('docs/paper.symbols.json')) if 'edl' in s['symbol']]"
# All caveats in the document
python3 -c "import json; [print(f'p.{c[\"page\"]} {c[\"text\"]}') \
for c in json.load(open('docs/paper.anchors.json')) if c['type'] == 'caveat']"
# All claims in a section
python3 -c "import json; [print(f'[{c[\"type\"]}] {c[\"text\"]}') \
for c in json.load(open('docs/paper.anchors.json')) if c['section'] == '4.3']"
Genre Support
Genre controls the claim taxonomy used in semantic extraction.
| Genre | Claim types | Best for |
|---|---|---|
paper (default) | definition, result, method, claim, caveat, open-question | Academic papers, arXiv preprints |
spec | requirement, definition, constraint, example, note | RFCs, API specs, technical standards |
legal | definition, obligation, right, exception, condition, reference | Contracts, policy documents, regulations |
Limitations (v0.1.x)
- PDF-only. No DOCX, HTML, or plain text input yet.
- Single-column layout assumed. Two-column papers may mis-order text within sections.
- No caching. Re-running re-extracts everything.
- No citation cross-referencing.
- Genre must be specified manually.
- Semantic extraction can hallucinate. Every claim is page-anchored, but the page number comes from the LLM. Verify critical claims against the source.
CLI Reference
python docmap.py paper.pdf [options]
Options:
--genre {paper,spec,legal} Claim taxonomy (default: paper)
--structure-only Skip LLM pass (free, fast)
--out DIR Output directory (default: .)
--api-key KEY Anthropic API key
--model MODEL Model (default: claude-sonnet-5-5)
--workers N Parallel workers (default: 4)
--no-usage-snippet Skip _USAGE.md generation
-v Verbose structural parsing
Session Memory - Running Notes
Maintain a single structured markdown document that tracks what is happening
right now in the current work session. This is within-session continuity,
distinct from remember() (cross-session semantic memory) and stash
(coarse checkpoint).
When to Use
Invoke on any of these triggers:
- User says "session notes", "update notes", "start session notes", "show session notes", "note the session", "write it down"
- User indicates they want continuity across context compression: "before you forget", "write that down", "for when context gets compressed"
- You have independently completed a meaningful block of work (file edits, a bugfix, a design choice, a reversal after correction) and no note has been updated for a while
Do NOT invoke for:
- One-off answers with no follow-up
- Cross-session persistence — that's
remember() - Coarse checkpoints intended for another session to resume — that's
stash
Template (Fixed Sections)
Every session note uses exactly these sections, in this order, with these
headers. Preserve the structure on every update. Never rename or reorder
sections. Update the content within sections; leave empty sections present
with a single _(nothing yet)_ placeholder.
# Session: {short title, 3-8 words, derived from the initial task}
## Current State
_What is actively being worked on right now. Explicit next step._
## Task Specification
_What the user originally asked for. Constraints, acceptance criteria,
design decisions that framed the work._
## Files and Functions
_Important paths touched or referenced, one per line, with a one-line
note on what they contain and why they matter._
## Errors & Corrections
_Errors encountered and how they were fixed. User corrections to my
approach — these have priority over routine progress entries._
## Key Decisions
_Choices made during the session with their rationale. One bullet per
decision. Lead with the choice, then the why._
## Worklog
_Terse, append-only, chronological. One line per attempt or step.
Prefix with ✓ (done), ✗ (failed), → (in progress), or ↺ (reverted)._
Storage
Persist as a procedure memory via the remembering skill, so notes
survive container death within the session thread.
from remembering.scripts import remember, recall, supersede
# First write in a session: create the memory
note_id = remember(
note_markdown,
"procedure",
tags=["session-memory", "active"],
priority=1,
)
# Subsequent updates: find the active note and supersede it
existing = recall(tags_all=["session-memory", "active"], n=1)
if existing:
note_id = supersede(existing[0].id, updated_markdown, "procedure",
tags=["session-memory", "active"], priority=1)
Session boundary: when the user says "session done", "wrap up",
"end session", or the conversation is clearly winding down, retag the
active note from active to archived by superseding it with the same
body and tags=["session-memory", "archived"].
One note is active at a time. If recall(tags_all=["session-memory", "active"]) returns more than one, the oldest is stale — archive it before
updating the current one.
Update Discipline
On each invocation:
- Read the existing note (if any) via
recall(tags_all=["session-memory", "active"], n=1). Work from its current body — do not regenerate from scratch. - Update in place. Edit the relevant sections; append to
## Worklog. Do not drop prior content unless it is now wrong (then move the correction to## Errors & Corrections). - Prioritize user corrections. When the user pushes back or corrects an
approach, that goes into
## Errors & Correctionsbefore other updates. - Deduplicate against stash and explicit memories. If an item is already
captured in a stash or a
remember()call, reference it by ID rather than restating. Notes are supplementary, not duplicative. - Supersede, don't append. Write the full updated document back via
supersede(). This keeps exactly one active note per session.
Budget
Target ~12K tokens for the note document. Prefer concise phrasing, but do
not truncate substantive content to hit the target — trigger
## Key Decisions consolidation (collapse related bullets) before cutting.
If the document exceeds ~20K tokens, compress ## Worklog first (merge
consecutive ✓ entries into a single summary line, keep corrections and
decisions intact).
Rationale: we run on 200K–1M context models, so the original 2K budget from the issue spec was over-constrained. 12K matches Claude Code's upstream design and leaves ample room for the surrounding conversation.
Surface to User
When the user asks to "show session notes", print the current note body verbatim in a fenced code block. Do not paraphrase.
When updating silently (you triggered it yourself), confirm with a single
line: Updated session notes (note id: <short-id>). Do not dump the full
body unsolicited.
Invariants
- Section headers and order are fixed. Updates change content, not structure.
- User corrections take priority over routine progress in ordering and detail.
- Notes do not duplicate what is already in stash or explicit memories.
- Manual invocation always works. Automatic triggers are a convenience, not a requirement.
- Exactly one
session-memory + activememory exists per session. Stale actives are archived, not left dangling.
Related
remembering— cross-session memory store (where these notes persist)- stash-resume-protocol (ops) — coarse session checkpoints
- context-hygiene (ops) — when and what to offload from context
Tracking Todos
Maintain a structured, persistent task list while working through multi-step requests. Track progress explicitly, mark completion honestly, and make the plan visible to Oskar.
When to Use
Use proactively when:
- Request requires 3+ distinct steps or actions
- User provides multiple tasks (numbered, comma-separated, or a list)
- Work spans multiple tool calls, file edits, or research phases
- Non-trivial planning would otherwise happen implicitly in your head
- User explicitly asks for a todo list
Skip when:
- Single, straightforward task
- Purely conversational or informational exchange
- Task completes in <3 trivial steps
- Quick fact lookup or recall
When in doubt, use it. Being explicit about multi-step plans makes failure modes visible.
Schema
Each todo has exactly three string fields:
content— imperative form ("Run tests", "Fix auth bug")status— one of:pending,in_progress,completedactiveForm— present continuous ("Running tests", "Fixing auth bug")
Both content and activeForm are required for every todo. The activeForm is what gets shown while a task is executing.
API
from todos import get_todos, write_todos, abandon, render
# Read current list
todos = get_todos()
# Replace entire list (Claude Code semantics — one call, whole list)
write_todos([
{"content": "Fetch issue body", "status": "in_progress", "activeForm": "Fetching issue body"},
{"content": "Draft fix plan", "status": "pending", "activeForm": "Drafting fix plan"},
{"content": "Implement and test", "status": "pending", "activeForm": "Implementing and testing"},
])
# Display to Oskar
print(render())
# Discard remaining todos when starting fresh unrelated work
abandon()
write_todos() validates schema and that at most one item is in_progress. When every item is completed, it auto-clears the list (matches Claude Code behavior — done-state is empty-state).
Behavioral Rules
One in_progress at a time, strictly. Before starting work on a task, set it to in_progress. Never have two in_progress items simultaneously. The validator enforces this.
Mark complete IMMEDIATELY. Update to completed the moment the work is actually done. Don't batch completions at the end — that defeats the tracking purpose and erases information about when things were finished.
Only mark complete if FULLY done. If tests fail, if the implementation is partial, if you hit an unresolved error — keep the task in_progress and add a new task describing the blocker. Lying about completion status is the failure mode this tool exists to prevent.
Remove irrelevant tasks. If a task becomes obsolete (scope change, wrong approach), remove it entirely from the list rather than marking it done. The list should reflect actual remaining work.
Break complex tasks down. Vague items like "implement the feature" defeat the point. Prefer specific actions: "Add validator to route handler", "Write 3 test cases for edge inputs", "Run existing test suite".
Storage Model
Todos live in Muninn's config store under key active-todos (category=ops). This means:
- Survives container death within the same conversation
- Persists across conversations until explicitly cleared or auto-cleared when all done
- Cross-conversation persistence is a feature: unfinished work shows up next session as a nudge
If you start a conversation and find stale todos that don't apply, call abandon() to clear them.
Known Limitations
Single writer assumed. The storage uses read-modify-write without locking. If a scheduled CCotw task and a live conversation both call write_todos() concurrently, the second write wins. Muninn runs single-threaded within a conversation, and scheduled tasks (perch, zeitgeist) don't use this skill, so the assumption holds in practice. Don't use this skill from concurrent agents without adding a lock.
Global scope is intentional, not per-conversation. Unfinished work leaks into the next conversation by design — it's a nudge that something was left open. If stale todos from prior work become noise, call abandon() to clear.
Replace-whole-list API vs delta updates. Matches Claude Code's TodoWrite exactly. The LLM failure mode (forgetting items when rewriting the full list) is mitigated by the prompt, not by the API shape. If omissions become a pattern, re-render the current list before composing the new one.
Display Pattern
When the plan is substantive, show it to Oskar once after creating it, then rely on inline mentions rather than re-rendering the full list on every update. Noise defeats the signal.
▶ Fetching issue body
☐ Drafting fix plan
☐ Implementing and testing
Examples
Good use case — multi-file refactor:
write_todos([
{"content": "Find all call sites of getCwd", "status": "in_progress", "activeForm": "Finding all call sites of getCwd"},
{"content": "Rename occurrences in src/utils/", "status": "pending", "activeForm": "Renaming occurrences in src/utils/"},
{"content": "Rename occurrences in src/agents/", "status": "pending", "activeForm": "Renaming occurrences in src/agents/"},
{"content": "Run test suite", "status": "pending", "activeForm": "Running test suite"},
])
Bad use case — should not use:
User: "What's the current time in Tokyo?"
→ Single fact lookup. Skip todos.
Mid-work update (after finishing step 1, hitting a blocker in step 2):
write_todos([
{"content": "Find all call sites of getCwd", "status": "completed", "activeForm": "Finding all call sites of getCwd"},
{"content": "Rename occurrences in src/utils/", "status": "completed", "activeForm": "Renaming occurrences in src/utils/"},
{"content": "Resolve type error in AgentContext after rename", "status": "in_progress", "activeForm": "Resolving type error in AgentContext after rename"},
{"content": "Rename occurrences in src/agents/", "status": "pending", "activeForm": "Renaming occurrences in src/agents/"},
{"content": "Run test suite", "status": "pending", "activeForm": "Running test suite"},
])
Note: the blocker became a new task rather than marking step 3 complete.
Updating Knowledge
Core Principles
- Systematic > ad-hoc - Follow research methodology, don't scatter searches randomly
- Cross-validate - Verify claims across multiple independent sources
- Confidence tracking - Explicitly flag single-source claims and gaps
- Concise output - Present findings efficiently, introspection in thinking blocks
Preflight Check
This skill requires web_search tool access.
Verify it's available before proceeding. If it isn't, tell the user immediately — "I need web search enabled for research tasks. Please toggle on 'Web search' in the feature menu." — and stop there rather than researching from memory alone or spending context on workarounds.
Trigger Disambiguation
Avoid triggering on generic action verbs like "update" (which could mean file/code changes instead) — trigger on the explicit knowledge-gathering requests named in the description above.
Research Workflow
1. Scope Definition (internal)
Identify in thinking block:
- What specifically needs investigation
- Current knowledge baseline
- Known gaps
- Required depth (overview vs comprehensive)
2. Source Gathering (2-10 tool calls)
Source priority:
- Official documentation, project repos, company announcements
- Academic papers, technical blogs with clear expertise
- Community discussions (for ecosystem context only)
Tool selection:
web_search- Find authoritative sourcesweb_fetch- Extract complete content from any URL (no prior search required)- Internal tools (GitHub, Drive, etc.) - Company/personal context
- Multiple tool types as needed
Gather 3-5 diverse sources minimum for cross-validation.
3. Synthesis (internal)
In thinking block:
- Cross-validate major claims across sources
- Note contradictions explicitly
- Track publication dates
- Assess source credibility
- Identify remaining gaps
4. Output (concise)
Present findings concisely:
## [Topic]
**Key findings:**
[2-3 sentence summary with most important updates]
**Current state:**
[Concise description of what's true now]
**Changes since cutoff:**
[Only if relevant - what's different]
**Gaps:**
[Only if user needs to know - what wasn't found]
Sources: [URLs]
Confidence indicators:
- State explicitly when claims are single-source
- Note contradictions between sources
- Flag unverified information
- Quantify confidence (high/medium/low) when uncertain
Quality Standards
Minimum requirements:
- 3+ independent sources consulted
- Major claims cross-validated
- Explicit confidence assessment
- Remaining gaps identified when relevant
Avoid:
- Single-source claims presented as definitive
- Ignoring publication dates
- Marketing language as fact
- Verbose deliberation in final output (use thinking blocks)
- Over-explaining research process to user
Tool Usage Notes
web_fetch capabilities:
- Can fetch ANY URL directly - doesn't require prior search results
- Use for complete documentation pages, technical articles
- Effective for deep content extraction after search identifies sources
web_search patterns:
- Start specific, broaden if needed
- Never repeat similar queries - make each unique
- Use the actual current date when relevant — check it rather than assuming
Internal tools:
- Prioritize for company/personal data
- Combine with web tools for comparative analysis
Example Outputs
Technical documentation inquiry:
TypeScript 5.3 (Nov 2023) added import attributes for JSON modules. The syntax uses `with` instead of deprecated `assert`. Runtime performance unchanged—validation happens at parse time.
Gap: No official benchmarks comparing validation overhead across bundlers.
Sources: [TS release notes, GitHub issue #54242, esbuild docs]
Conflicting information:
React Server Components ship in Next.js 13.4+ (stable) and Remix 2.0 (experimental).
Contradiction: Meta's blog claims "production-ready" while Remix docs flag "breaking changes expected." Cross-validation shows Meta refers to Next.js implementation only.
Confidence: High on Next.js status, medium on Remix timeline.
Sources: [Next.js changelog, Remix docs, React.dev]
Avoid narrating process:
❌ "After conducting extensive research and cross-validating multiple authoritative sources..."
✓ Just present findings with source attribution
Sponsors
Details
- Marketplace
- oaustegard/claude-skills
- Source
- plugins/knowledge-and-memory
- Category
- Knowledge And Memory
- License
- MIT
- Keywords
- asking-questions, bm25, cloning-project, creating-kb, iterating, mapping-documents, session-memory, tracking-todos, updating-knowledge
More from oaustegard/claude-skills
All 14- media-processingv2.3.0
- skill-developmentv2.6.0
- social-and-lifestylev1.0.0
- web-and-retrievalv1.0.4
- ai-and-reasoningv2.9.1
- code-intelligencev2.6.0
- data-and-visualizationv1.0.0
- deciding-with-confidencev0.4.0
- declaudingv0.14.0
- delegating-with-contextv0.3.0
- development-toolsv1.2.1
- environment-and-configv2.0.0
- github-and-gitv2.0.1