Summary
This is a local memory layer for multi-agent setups that catches drift before agents act. It embeds raw prompts with BGE-m3 instead of extracting entities through an LLM, which keeps everything on your machine and lets it score the next action against past failure patterns. You get seven MCP tools including add_memory, recall, and add_worker for registering deterministic worker specs. The trade is real: it scores 56.6% on LongMemEval compared to white-box leaders in the 80s, but runs 14x cheaper and works across any LLM without sending data out. Built for Claude Desktop, Cline, Cursor, and Continue. Reach for it when you need cross-session coordination without an orchestrator or when agents keep forgetting rules you already set.
Plug Mailtrap into your AI workflow and let it handle the email.
Connect Mailtrap MCP →
Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →
Make coding agent sessions - Searchable, Shareable, Vendor-neutral & Scored.
Try For Free →
Your agent targets a perfect 10 Code Health score. Deterministic. Every commit.
Try For Free →
Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →
belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →nautilus-compass
Open-source memory & reliability layer for AI agents. Long-term memory that now beats mem0 on all three LongMemEval-S metrics while staying fully local & 14× cheaper — plus drift detection and cross-agent contracts that no other memory layer ships.
Plugin for Claude Code / Desktop · Cline · Cursor · Continue.dev · Zed · any MCP client.
Built by Nautilus Platform · open agent ecosystem · join as agent →
🇬🇧 English (this file) · 🇨🇳 中文
What this is (2026-08 state)
Three pillars, one plugin:
1 · Black-box long-term memory — now with SOTA retrieval.
Raw text embedded locally with BGE-m3. No extraction LLM at ingest, no graph,
no data leaving your machine. In Aug 2026 we added utterance-routed chunk
retrieval: single-session and knowledge-update questions route to
turn-window chunks (the answer usually lives in ONE user turn; whole-session
embedding dilutes it), everything else uses session-level hybrid
(BM25 + dense RRF). Result on LongMemEval-S full 500 questions,
same-question head-to-head vs mem0 2.0.19 (both sides infer=False,
our reproduction):
| LongMemEval-S · n=500 | P@1 | P@5 | MRR |
|---|---|---|---|
| compass | 0.890 | 0.978 | 0.929 |
| mem0 2.0.19 | 0.774 | 0.916 | 0.834 |
One-command reproduction (retrieval-only, no LLM calls, CPU works / GPU recommended for the full 500):
bash scripts/reproduce_lmes_retrieval.sh # full 500
SUBSET=12 bash scripts/reproduce_lmes_retrieval.sh # smoke
The same utterance ammo overtakes mem0 on its own home benchmark
(LOCOMO-10, n=1986: 0.644 / 0.890 vs 0.592 / 0.802) and fixes the
single-session collapse on LongMemEval-M (0.20 → 1.00). Full evidence chain
with per-type breakdowns and every config flag:
docs/evidence/headhead_mem0_full500_20260826.json
— including the experiments that failed (cross-encoder reranking hurts on
this corpus; candidate-pool K is a no-op; Qwen3-0.6B swap is a wash).
2 · Drift detection — the half nobody else solves. Memory recalled doesn't stop the AI from breaking the rule this time. compass scores every prompt against an anchor set of real failure patterns (25 positive + 35 negative) before the agent acts. AUC 0.83 held-out, p95 latency <50 ms, fire rate 0.5% in production traffic. White-box layers abstract prompts into facts before drift becomes checkable — structurally out of their reach.
3 · Cross-agent contracts + governance.
When you run multiple agents (or multiple Claude dialogs) on shared files,
compass derives implicit contracts from handoff files, tracks closure, and
audits for fake-closure / red drift. A 4-dialog 28-hour field study lives in
docs/case_study_4dialog_compass.md.
The trade that flipped: earlier versions traded −30 points on LongMemEval-S for local deployment and cost. As of 2026-08 there is no trade — full sweep at 1/14 the reproduction cost (~$3.50 per 500 questions vs $50+ for GPT-4o-judged stacks). Full argument: paper/BLACKBOX_VS_WHITEBOX.md.
Quickstart
30 seconds (Claude Code / Desktop · local daemon)
git clone https://github.com/chunxiaoxx/nautilus-compass ~/.claude/plugins/nautilus-compass
bash ~/.claude/plugins/nautilus-compass/install.sh
# start the BGE-m3 daemon (one-time per boot)
bash ~/.claude/plugins/nautilus-compass/daemon_start.sh
Deploy notes (field-verified pitfalls, 2026-08-28):
COMPASS_USE_INOTIFY=0disables new-file discovery — recalls won't see fresh writes, silently. Only set it if you know why; the daemon logs a WARNING when it's off.driftfails loudly now: ifanchors.jsonis missing from the plugin dir, responses carrydrift.anchors_error+should_alert: true(it used to silently return "no risk" — that was a security hole).- Token changes to
tokens.jsonhot-reload via mtime check (no systemd restart needed since v3.1.0).- First recall after daemon idle may take up to 90 s (model cold-load); the MCP client auto-retries once with the extended timeout.
The installer wires three hooks into ~/.claude/settings.json:
UserPromptSubmit→ time-bucketed memory recall + drift checkPostToolUse→ mid-session writerStop→ end-of-session summary (writes a session battle-report to~/.claude/.cache/compass-last-session.txt)
Slash commands: /compass-verify · /compass-drift · /compass-recall ·
/compass-search · /compass-status.
Any other MCP client
python ~/.claude/plugins/nautilus-compass/scripts/install_to_agent.py
Auto-detects Claude Desktop, Cursor, Cline, Continue.dev, Zed and patches
their MCP config. Per-agent copy-paste configs:
docs/AGENT_ONBOARDING.md · raw protocol:
docs/mcp-usage.md.
Cloud-hosted (invitation-only during review period)
The hosted gateway — MCP over HTTPS with scoped tokens and per-user memory
isolation — is in invitation-only beta while self-serve signup is being built
(design locked: docs/plans/2026-08-30-multi-tenant-memory-design.md).
To request a token during the beta, open a GitHub issue on this repo.
MCP endpoint: https://compass.nautilus.social/mcp/ (Bearer token ·
streamable-http). A2A discovery:
curl https://compass.nautilus.social/.well-known/agent.json
Nautilus platform agents (cloud ssh quickstart)
For agents on machines with ssh access to your Nautilus cloud box —
generates a scoped token, wires the cloud MCP bridge, writes .mcp.json,
and runs an end-to-end self-check. Add --hud to install the fused status
line (live recall hit-counter 🧠, drift state, 5-min traffic).
bash ~/.claude/plugins/nautilus-compass/ops/agent_quickstart.sh my-agent
Headline numbers
| Benchmark | Score | Honest compare |
|---|---|---|
| LongMemEval-S 500q full (utt-routed + hybrid, n=500) | P@1 0.890 · P@5 0.978 · MRR 0.929 | sweeps mem0 2.0.19 (0.774/0.916/0.834, our reproduction): +11.6/+6.2/+9.5pt. Largest flip: single-session-user P@1 0.90 vs 0.49 |
| LOCOMO-10 (n=1986 · mem0's home benchmark) | P@1 0.644 · P@5 0.890 · MRR 0.740 | overtakes mem0 (0.592/0.802/0.677, our reproduction) +5.2/+8.8pt |
| LongMemEval-M 500q full (~501 sessions/question) | P@5 0.888 | 12x larger session pools cost only 9pt vs S500; ssu collapse fixed at n=500 (0.20 → 0.93); ssp 0.53 newly exposed; no mem0 M head-to-head yet |
| EverMemBench-Dynamic (n=500) | 44.4% (Run 1) / 47.3% (Run 2) | tops the four published Table 4 baselines (Mem0 37.09, Zep 39.97, MemOS 42.55, MemoBase 34.27). Not claiming "industry SOTA" — OMEGA / Mem0g haven't reported publicly |
| LongMemEval-S e2e 500q full (doubao subject × glm judge, 2026-08-29) | 42.6% (213/500) by type: ssu 95.7% · ssp 80.0% · ku 73.1% · ssa 25.0% · ms 22.6% · tr 15.8% (was 0% — date-anchor fix) | honest two-tier: single-session near ceiling; cross-session aggregation (ms/ssa/tr) is the public next lever (answer spread over 3+ sessions exceeds 3-utterance context). 30q paired run: 56.7% (+30pt context fix, small-sample). Full JSON: docs/evidence/e2e_500_full_20260829.json |
| LongMemEval-V2 small tier (451q, tuned v2 · 2026-08-30) | web 36.7% / enterprise 40.3% (first untuned baseline same-day: 19.6% / 12.8%) | brand-new multi-session benchmark (webarena agent trajectories); published untuned baseline AND tuned v2. Two levers: abstention judging alignment (bare-UNKNOWN 121→0; unanswerable-from-snapshot questions 2.8% → 45.8% via the rubric's two legitimate routes) + retrieval unit upgrade (a11y-structure pruning, per-trajectory dense rerank, budget 12k→24k; procedure +16.6pt). Honest caveats: abstention route relies on judge discretion (sampling showed both 1s and 0s on similar wording); web dynamic-type dipped -3.9pt. Follow-up 2026-08-31: LoRA retrieval-augmentation full validation closed at parity (ent 40.8% / web 36.7% after judge-retry correction, both < d12+5pt gate) — not adopted, tuned-v2 stack stays. Evidence: vtf/_compass_lmev2_out/ |
| Drift detector AUC | 0.83 held-out / 0.92 in-set | only public memory layer doing drift detection at all |
| Reproduction cost | ~$3.50 / 500 questions | ~14× cheaper than GPT-4o-judged stacks |
| p95 hook latency | <50 ms | safe for every-prompt invocation |
We deliberately report Run 1 (44.4%) as the EverMemBench headline to avoid
cherry-picking; cross-run mean 45.84% clears MemOS by +3.3pt. Dual-run +
Gemini cross-judge sensitivity analysis:
paper/sections/paper2_06_5_evermembench.tex.
Try it without installing: live drift-detection + Merkle-integrity demo at huggingface.co/spaces/chunxiaox/nautilus-compass (CPU only · metadata-mode jaccard fallback · no signup).
Reproduce the numbers — eval dataset (behavioral anchors + labeled traces + LongMemEval-S / EverMemBench scoring) on the Hub: huggingface.co/datasets/chunxiaox/nautilus-compass-test-data
from datasets import load_dataset
ds = load_dataset("chunxiaox/nautilus-compass-test-data")
Benchmark entrypoint: bash ops/bench_all.sh l0 (fast layer, no GPU) ·
bash ops/bench_all.sh l1 30 (LongMemEval subset). Retrieval levers are
env-switched in tests/eval_longmemeval_accuracy.py
(ZMM_UTTERANCE_RETRIEVE / ZMM_UTTERANCE_TYPES / ZMM_HYBRID /
ZMM_RETRIEVE_K / ZMM_DATE_ANCHOR / ZMM_EMBED_CACHE).
How it works
User prompt: "Fix bug X for me"
│
▼
┌─────────────────────────────────────┐
│ UserPromptSubmit Hook (this plugin)│
└─────────────────────────────────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
┌────────┐ ┌─────────┐ ┌──────────┐
│ recall │ │ drift │ │ profile │
│ memory │ │ check │ │ aggregate│
└────────┘ └─────────┘ └──────────┘
│
▼
Hooks inject results into Claude's system prompt:
- Time-bucketed past memory (BGE-m3 semantic + keyword hybrid)
- Drift score + nearest negative anchor (if score < threshold)
- Profile facts ("you have 3 unfinished tasks in this repo")
│
▼
Claude answers — with full context loaded
Drift detector: each prompt vs anchor set (real failure transcripts), BGE-m3 cosine. AUC 0.83 held-out.
What's exposed (MCP tools)
17 tools — core seven:
| Tool | Purpose | Latency (local daemon) |
|---|---|---|
ingest_obs(name, body, agent_id?) | Write observation with auto-anchor + drift signal | ~150 ms |
recall(query, project?, top_k?) | BGE-m3 semantic + keyword hybrid search | ~200 ms |
session_search(query, since?) | Time-bucketed session-log search | ~80 ms |
profile(user_id?) | Work-profile aggregate (topics, agents, drift trend) | ~100 ms |
drift_check(prompt, project?) | Black-box drift score against anchors | <50 ms |
drift_history(since?, agent_id?) | Drift score timeline for trend audit | ~30 ms |
feedback_log(direction, reason) | Log positive/negative anchor signal | <20 ms |
Latencies are local-daemon figures. Over the public HTTPS MCP endpoint (
https://compass.nautilus.social/mcp/) add TLS + WAN round-trip: measured p50 ≈ 0.9–1.7 s per call (2026-08-28 field test).
Plus: thread_recall · proof_of_impact · long_task · platform bridge
(submit_platform_task / ingest_platform_task_result) · governance
(governance_dispatch / governance_audit / governance_lock_check ·
governance_plan) · add_worker. JSON-RPC 2.0 over stdio / TCP / TLS / mTLS;
notifications/*, logging/setLevel, resources/* spec-complete.
Full guide: docs/mcp-usage.md.
Token scopes (v2.3.1)
Tokens are scoped, not global. ops/compass_token_admin.py grant <agent> --scopes read:<project>,write:<project> issues a least-privilege token;
read:* (all-project recall, incl. scope=user) requires an explicit
--yes-i-want-star. The HTTP server enforces scopes per call (fail-closed);
legacy list-format tokens map to full access for backward compatibility.
The quickstart script signs read-only, current-project tokens by default.
Comparison
| Capability | this | mem0 | Letta | Zep | claude-mem | MemOS | Smriti |
|---|---|---|---|---|---|---|---|
| Cross-agent memory | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | archive-only |
| MCP A2A protocol native | ✅ TLS+mTLS+RBAC | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Drift detection | ✅ AUC 0.83 | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Merkle integrity audit log | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| LongMemEval-S retrieval (500q head-to-head) | ✅ 0.890 / 0.978 / 0.929 | 0.774 / 0.916 / 0.834 (our reproduction) | n/r | n/r | n/r | ❌ | ❌ |
| LOCOMO-10 retrieval (n=1986) | ✅ 0.644 / 0.890 / 0.740 | 0.592 / 0.802 / 0.677 (our reproduction) | n/r | n/r | n/r | n/r | n/r |
| EverMemBench verified | ✅ 44.4-47.3% | 37.09 | n/r | 39.97 | n/r | 42.55 | ❌ |
| LongMemEval-S e2e (their own harness) | 500q full 42.6% — by type: ssu 95.7 / ssp 80.0 / ku 73.1 / ssa 25.0 / ms 22.6 / tr 15.8 | 94.4% (self-reported) | n/r | n/r | n/r | n/r | n/r |
2026 newcomers not yet same-machine reproduced by us: Hindsight, Supermemory (self-reports LongMemEval SOTA), Cognee, LangMem, Membase — rows pending; their published numbers use their own harnesses and are not directly comparable to our head-to-head protocol. | Self-host + hosted both | ✅ | ☁ only | ✅ | ☁ only | ✅ | OSS only | OSS only | | License | MIT | Apache | Apache | proprietary | MIT | Apache | MIT |
n/r = not reported in their published evaluations. Smriti is a team
conversation archive — different scope, listed for completeness.
Case study · 4-dialog OSS multi-agent reliability
28 hours, four Claude Code dialogs on shared filesystem protocols:
drift fired 314×/7d (act-on rate instrumented), contract
cnt_compass_soul_sub_a1 closed in 17.92h vs 6d21h budget, 13 plan-dup
audits saved ~40-50h, first cross-dialog L4 fire settled 50 NAU. Field log
- 7 generalizable patterns:
docs/case_study_4dialog_compass.md.
Advanced (opt-in surface)
Drift loop closure · act-on rate
Every fired alert gets a stable alert_id in
.cache/drift_mitigation_log.jsonl. Acknowledge via
feedback.py log <alert_id> fp|tp; audit_kpi.py reports
act_on_rate(window_hours) (target ≥0.70; <0.30 = cry-wolf → raise
threshold or retrain anchors).
from audit_kpi import act_on_rate
m = act_on_rate(window_hours=168)
assert m["rate"] >= 0.70
v3 opt-in LLM switches (all default-off, byte-equal promise)
With no opt-in env set, daemon behavior is byte-equal to v2.0.1 — gated by
tests/test_llm_opt_in.py on every PR.
| env var | tier | feature |
|---|---|---|
COMPASS_USE_LLM_RESOLVE | 1 (session-end) | LLM contradiction resolution |
COMPASS_USE_LLM_VERIFY | 4 (runtime) | anti-confabulation cite-or-refuse |
COMPASS_USE_LLM_DRIFT_PAY | 4 (runtime) | drift × outcome anchor feedback |
COMPASS_USE_LLM_REFLECT | 3 (periodic) | self-reflection semantic emit |
COMPASS_USE_LLM_ECON | 4 (runtime) | memory-as-economy NAU budget |
Deterministic v3 surface (always on): typed knowledge graph layer (NO-OP
until built), confidence scoring + contradiction hook, MEMORY_REPORT.md
auto-gen, implementation_notes frontmatter. Registry: llm_opt_in.py.
Platform integration · BP1/BP3 + V7 governance
OSS↔platform bridge without a new HTTP server:
submit_platform_task (compass → platform queue, file-based or HTTP when
COMPASS_PLATFORM_QUEUE_URL is set) · ingest_platform_task_result
(platform → compass, searchable via recall). Round-trip demo:
python examples/platform_flywheel_demo.py.
V7 governance (multi-executor deployments): governance_dispatch
(decompose 1 task → N routed sub-tasks) · governance_audit (fake-closure /
red-drift scan) · governance_lock_check (SHA256 lock on the L0 core).
Demo: python examples/v7_governance_demo.py. Contract details:
docs/PLATFORM_HANDSHAKE.md.
Release history · v3.0.0 / v2.1.0 / v2.0.0
v3.0.0 · "from memory library to evolution engine" — same system closing the loop: memories feed a extract fuel → external verdict → distill cycle. Semantic-recall revival (Windows torch long-path fix), GOAL-SSOT ledger + hourly heartbeat, cloud capacity root-cause fixes (load 10-14 → 1.x), daemon atomic pkl + per-project locks, paired-control evidence (tribal-fact retrieval 0/3 → 3/3), fused HUD, 30-second quickstart.
v2.1.0 · drift v2 + line reconciliation — cry-wolf fix (fire rate 64.5% → 0.5% via rule-hit OR drift_score < −0.07), cross-agent contract scanner (L4 substrate), L3 tier promotion + PoI, daemon hardening (bounded pools, in-flight semaphore, BM25+vector RRF opt-in).
v2.0.0 · Opinionated EvoMap — deterministic lifecycle layer on the
black-box base. No LLM at ingest / tier promotion / forgetting; no vendoring
of GBrain/OpenViking; no graph rerank for closed haystacks (cost −6.2pt in
v0.8 — paper/RESULTS_v0.8.md).
Full notes: CHANGELOG.md · release:
v3.0.0
Documentation
docs/AGENT_ONBOARDING.md— per-agent install configs (6 platforms + 3 frameworks)docs/mcp-usage.md— raw MCP protocol guide, TLS setup, RBACdocs/PLATFORM_HANDSHAKE.md— OSS↔SaaS coordination contractdocs/evidence/— raw benchmark evidence files (JSON, per-question rows)paper/— two papers (drift detection + memory pipeline) and eval scriptsops/GPU_EVAL_RECIPE_4090.md— 12-minute rented-GPU benchmark recipeCHANGELOG.md·CONTRIBUTING.md
Citation
Paper 1 · drift detection:
@misc{nautiluscompass-drift-2026,
title = {Nautilus Compass: Black-box Persona Drift Detection
for Production LLM Agents},
author = {Chunxiao Wang},
year = {2026},
note = {Yiluo Technology Co., Ltd.},
howpublished = {\url{https://github.com/chunxiaoxx/nautilus-compass}}
}
Paper 2 · memory pipeline + EverMemBench cross-bench:
@misc{nautiluscompass-memrecall-2026,
title = {Closing the Memory Recall Gap with Chinese LLMs:
A Multi-Stage Retrieval Pipeline Achieving Zep-SOTA Performance
on LongMemEval-S at 1/15 Cost},
author = {Chunxiao Wang},
year = {2026},
note = {Yiluo Technology Co., Ltd.},
howpublished = {\url{https://github.com/chunxiaoxx/nautilus-compass}}
}
Prior work we build on (cite as appropriate): BGE-m3 / BGE-Reranker (BAAI 2024) · Persona Vectors (Anthropic, arXiv:2507.21509, complementary white-box) · DPT-Agent (arXiv:2502.11882) · A-MEM (arXiv:2502.12110) · LongMemEval (Wu et al., NeurIPS 2024) · EverMemBench (Hu et al., 2026).
License
- Code, plugin, MCP wrapper, papers, scripts — MIT (
LICENSE) - Behavioral anchor files (
anchors*.json) — CC0 1.0 Universal (LICENSE-ANCHORS)
Star history
Contributors
PRs welcome — see CONTRIBUTING.md.
Contact
- Author: Chunxiao Wang · Yiluo Technology Co., Ltd. ·
chunxiaoxx@gmail.com - Issues: github.com/chunxiaoxx/nautilus-compass/issues
- Hosted gateway: compass.nautilus.social
- 中文文档: README.zh-CN.md
Featured
Plug Mailtrap into your AI workflow and let it handle the email.
Connect Mailtrap MCP →Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →Make coding agent sessions - Searchable, Shareable, Vendor-neutral & Scored.
Try For Free →Your agent targets a perfect 10 Code Health score. Deterministic. Every commit.
Try For Free →Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →Categories
Registryactive
Packagenautilus-compass
TransportSTDIO
UpdatedMay 18, 2026
