
This server wraps the GoldenMatch entity resolution toolkit, which finds duplicate records across messy datasets with a 97.2% F1 score out of the box. You get access to deduplication, clustering, and the broader Golden Suite pipeline (InferMap for schema alignment, GoldenCheck for profiling, GoldenFlow for standardization). The MCP layer exposes 36+ tools including auto_configure for adaptive tuning and controller_telemetry for inspecting clustering decisions. Useful when you need to clean customer lists, merge data sources, or resolve entities across organizations without writing custom fuzzy matching logic. The zero-config defaults work immediately, and a learning memory system stops asking for the same correction twice across runs.
claude mcp add --transport http goldenmatch https://goldenmatch-mcp-production.up.railway.app/mcp/Run in your terminal. Add --scope user to make it available in every project.
Review the command, arguments, and environment values before installing — MCP servers run with your local permissions.
Verified live against the running server on Jun 10, 2026.
analyze_dataProfile data, detect domain, recommend ER strategy1 paramsProfile data, detect domain, recommend ER strategy
file_path*stringauto_configureRun AutoConfigController on a CSV; return the committed GoldenMatchConfig (incl. negative_evidence / Path Y when chosen) plus telemetry — stop_reason, health, decision trace, indicator column priors. Programmatic equivalent of `goldenmatch autoconfig`.2 paramsRun AutoConfigController on a CSV; return the committed GoldenMatchConfig (incl. negative_evidence / Path Y when chosen) plus telemetry — stop_reason, health, decision trace, indicator column priors. Programmatic equivalent of `goldenmatch autoconfig`.
constraintsobjectfile_path*stringcontroller_telemetryReturn the AutoConfigController telemetry from the most recent `auto_configure` or `agent_deduplicate` call in this MCP session. Same JSON shape as the web /api/v1/controller/telemetry endpoint.Return the AutoConfigController telemetry from the most recent `auto_configure` or `agent_deduplicate` call in this MCP session. Same JSON shape as the web /api/v1/controller/telemetry endpoint.
No parameters — call it with no arguments.
agent_deduplicateRun full ER pipeline with confidence gating and reasoning2 paramsRun full ER pipeline with confidence gating and reasoning
configobjectfile_path*stringagent_match_sourcesMatch two files with intelligent strategy selection3 paramsMatch two files with intelligent strategy selection
configobjectfile_a*stringfile_b*stringagent_explain_pairNatural language explanation for a record pair4 paramsNatural language explanation for a record pair
exactarrayfuzzyobjectrecord_a*objectrecord_b*objectagent_explain_clusterExplain why records are in the same cluster1 paramsExplain why records are in the same cluster
cluster_id*integeragent_review_queueGet borderline pairs awaiting approval1 paramsGet borderline pairs awaiting approval
job_name*stringagent_approve_rejectApprove or reject a review queue pair6 paramsApprove or reject a review queue pair
decided_by*stringdecision*stringid_a*integerid_b*integerjob_name*stringreasonstringagent_compare_strategiesCompare ER strategies on your data2 paramsCompare ER strategies on your data
file_path*stringground_truthstringsuggest_pprlCheck if data needs privacy-preserving matching1 paramsCheck if data needs privacy-preserving matching
file_path*stringscan_qualityRun GoldenCheck data quality scan on a CSV file. Returns issues found (encoding errors, Unicode problems, format violations) without applying fixes. Requires goldencheck: pip install goldenmatch[quality]2 paramsRun GoldenCheck data quality scan on a CSV file. Returns issues found (encoding errors, Unicode problems, format violations) without applying fixes. Requires goldencheck: pip install goldenmatch[quality]
domainstringfile_path*stringfix_qualityRun GoldenCheck scan and apply fixes to a CSV file. Returns the fixed data summary and a manifest of all fixes applied. Requires goldencheck: pip install goldenmatch[quality]4 paramsRun GoldenCheck scan and apply fixes to a CSV file. Returns the fixed data summary and a manifest of all fixes applied. Requires goldencheck: pip install goldenmatch[quality]
domainstringfile_path*stringfix_modestringsafe · moderatedefault: safeoutput_pathstringrun_transformsRun GoldenFlow data transforms on a CSV file. Normalizes phone numbers (E.164), dates (ISO), categorical spelling, and Unicode issues. Returns a manifest of transforms applied. Requires goldenflow: pip install goldenmatch[transform]2 paramsRun GoldenFlow data transforms on a CSV file. Normalizes phone numbers (E.164), dates (ISO), categorical spelling, and Unicode issues. Returns a manifest of transforms applied. Requires goldenflow: pip install goldenmatch[transform]
file_path*stringoutput_pathstringlist_correctionsList stored Learning Memory corrections, optionally filtered by dataset. Returns id_a, id_b, decision, source, trust, reason, matchkey_name, dataset, original_score, created_at.2 paramsList stored Learning Memory corrections, optionally filtered by dataset. Returns id_a, id_b, decision, source, trust, reason, matchkey_name, dataset, original_score, created_at.
datasetstringpathstringadd_correctionAdd a pair correction to Learning Memory. Source is set to 'agent' with trust=0.5 (lower than human steward decisions which are 1.0). Pair (id_a, id_b) is canonicalized to (min, max) before storage.7 paramsAdd a pair correction to Learning Memory. Source is set to 'agent' with trust=0.5 (lower than human steward decisions which are 1.0). Pair (id_a, id_b) is canonicalized to (min, max) before storage.
dataset*stringdecision*stringapprove · rejectid_a*integerid_b*integermatchkey_namestringpathstringreasonstringlearn_thresholdsForce a MemoryLearner pass over accumulated corrections. Returns the list of LearnedAdjustments produced (matchkey_name, threshold, sample_size, learned_at). Requires >= 10 corrections per matchkey before threshold tuning fires; otherwise returns an empty list.2 paramsForce a MemoryLearner pass over accumulated corrections. Returns the list of LearnedAdjustments produced (matchkey_name, threshold, sample_size, learned_at). Requires >= 10 corrections per matchkey before threshold tuning fires; otherwise returns an empty list.
matchkey_namestringpathstringmemory_statsReturn Learning Memory status: total correction count, last learn time, and current learned adjustments. Cheap; safe for status checks.1 paramsReturn Learning Memory status: total correction count, last learn time, and current learned adjustments. Cheap; safe for status checks.
pathstringmemory_exportReturn all corrections as a list of dicts (CSV-shaped). Caller is responsible for writing the file. Optionally filter by dataset.2 paramsReturn all corrections as a list of dicts (CSV-shaped). Caller is responsible for writing the file. Optionally filter by dataset.
datasetstringpathstringidentity_resolveResolve a record_id to its durable identity. Returns the full identity view (members, evidence edges, recent events) or null when no identity exists for that record.2 paramsResolve a record_id to its durable identity. Returns the full identity view (members, evidence edges, recent events) or null when no identity exists for that record.
pathstringrecord_id*stringidentity_listList identities, optionally filtered by dataset/status.5 paramsList identities, optionally filtered by dataset/status.
datasetstringlimitintegeroffsetintegerpathstringstatusstringidentity_historyReturn the temporal event log for an identity.3 paramsReturn the temporal event log for an identity.
entity_id*stringlimitintegerpathstringidentity_conflictsList evidence edges marked `conflicts_with`.2 paramsList evidence edges marked `conflicts_with`.
datasetstringpathstringidentity_mergeManually merge two identities. All records from `absorb_entity_id` are reassigned to `keep_entity_id`.4 paramsManually merge two identities. All records from `absorb_entity_id` are reassigned to `keep_entity_id`.
absorb_entity_id*stringkeep_entity_id*stringpathstringreasonstringidentity_splitSplit a subset of records off an identity into a brand-new identity. The original keeps the remaining records.4 paramsSplit a subset of records off an identity into a brand-new identity. The original keeps the remaining records.
entity_id*stringpathstringreasonstringrecord_ids*arrayget_statsGet dataset statistics: record count, cluster count, match rate, cluster sizes.Get dataset statistics: record count, cluster count, match rate, cluster sizes.
No parameters — call it with no arguments.
find_duplicatesFind duplicate matches for a record. Provide field values to search against the loaded dataset.2 paramsFind duplicate matches for a record. Provide field values to search against the loaded dataset.
record*objecttop_kintegerexplain_matchExplain why two records match or don't match. Shows per-field score breakdown.2 paramsExplain why two records match or don't match. Shows per-field score breakdown.
record_a*objectrecord_b*objectlist_clustersList duplicate clusters found in the dataset. Returns cluster IDs, sizes, and member counts.2 paramsList duplicate clusters found in the dataset. Returns cluster IDs, sizes, and member counts.
limitintegermin_sizeintegerget_clusterGet details of a specific cluster: all member records and their field values.1 paramsGet details of a specific cluster: all member records and their field values.
cluster_id*integerget_golden_recordGet the merged golden (canonical) record for a cluster.1 paramsGet the merged golden (canonical) record for a cluster.
cluster_id*integermatch_recordMatch a single record against the loaded dataset in real-time. Paste a record's fields and instantly see if it matches any existing record. Uses the configured matchkeys, scorers, and thresholds. Example: {"name": "John Smith", "email": "john@test.com", "zip": "10001"}3 paramsMatch a single record against the loaded dataset in real-time. Paste a record's fields and instantly see if it matches any existing record. Uses the configured matchkeys, scorers, and thresholds. Example: {"name": "John Smith", "email": "john@test.com", "zip": "10001"}
record*objectthresholdnumbertop_kintegerunmerge_recordRemove a record from its cluster. The record becomes a singleton. Remaining cluster members are re-clustered using stored pair scores. Use this to fix bad merges.1 paramsRemove a record from its cluster. The record becomes a singleton. Remaining cluster members are re-clustered using stored pair scores. Use this to fix bad merges.
record_id*integershatter_clusterBreak an entire cluster into individual records. All members become singletons. Use when a cluster is completely wrong.1 paramsBreak an entire cluster into individual records. All members become singletons. Use when a cluster is completely wrong.
cluster_id*integersuggest_configAnalyze bad merges and suggest config changes. Provide examples of incorrect merges (pairs that should NOT have matched) and GoldenMatch will identify which fields/thresholds to tighten. Example: [{"record_a": {...}, "record_b": {...}, "reason": "different people"}]1 paramsAnalyze bad merges and suggest config changes. Provide examples of incorrect merges (pairs that should NOT have matched) and GoldenMatch will identify which fields/thresholds to tighten. Example: [{"record_a": {...}, "record_b": {...}, "reason": "different people"}]
bad_merges*arrayprofile_dataGet data quality profile: column types, null rates, unique counts, sample values.Get data quality profile: column types, null rates, unique counts, sample values.
No parameters — call it with no arguments.
export_resultsExport matching results to a file (CSV or JSON).2 paramsExport matching results to a file (CSV or JSON).
formatstringcsv · jsondefault: csvoutput_path*stringlist_domainsList available domain extraction rulebooks (built-in + user-defined).List available domain extraction rulebooks (built-in + user-defined).
No parameters — call it with no arguments.
create_domainCreate a custom domain extraction rulebook. Define patterns for a specific data domain (medical devices, automotive parts, real estate, etc.).7 paramsCreate a custom domain extraction rulebook. Define patterns for a specific data domain (medical devices, automotive parts, real estate, etc.).
attribute_patternsobjectbrand_patternsarrayidentifier_patternsobjectname*stringscopestringlocal · globaldefault: localsignals*arraystop_wordsarraytest_domainTest a domain extraction rulebook against sample records. Shows what features would be extracted from the loaded data.2 paramsTest a domain extraction rulebook against sample records. Shows what features would be extracted from the loaded data.
domain_name*stringsample_sizeintegerpprl_auto_configAnalyze the loaded dataset and recommend optimal PPRL (privacy-preserving record linkage) configuration. Returns recommended fields, bloom filter parameters, threshold, and explanation.2 paramsAnalyze the loaded dataset and recommend optimal PPRL (privacy-preserving record linkage) configuration. Returns recommended fields, bloom filter parameters, threshold, and explanation.
security_levelstringstandard · high · paranoiddefault: highuse_llmbooleanpprl_linkRun privacy-preserving record linkage between two parties' data. Computes bloom filters, matches records without sharing raw data. Specify fields, threshold, and security level.5 paramsRun privacy-preserving record linkage between two parties' data. Computes bloom filters, matches records without sharing raw data. Specify fields, threshold, and security level.
fields*arrayfile_a*stringfile_b*stringsecurity_levelstringstandard · high · paranoiddefault: highthresholdnumberYour customer data lives in a CRM, a billing system, and three spreadsheets nobody owns. Some records are duplicates. Some are the same company spelled four different ways. Nobody can answer how many customers do we actually have, and every dashboard built on top inherits the doubt.
Splink-beating entity resolution, Arrow-native and Rust-fast with zero tuning, feeding a durable identity layer so messy records from every source become stable golden entities with whole-record, Customer-360 provenance.
Zero-config matching that beats expert-tuned Splink head-to-head on messy customer records, in an Arrow-native, Rust-authoritative engine verified from a laptop CSV to a 250M-row dedupe in 11.2 minutes. The identities it produces live in a transaction-native control plane carrying stable entity_ids, per-field provenance, merge/split, and a tamper-evident audit log, all one call away as a Customer 360. It even owns its primitives: byte-identical, faster-than-rapidfuzz / jellyfish / FAISS Rust kernels, not rented dependencies.
Python · TypeScript · SQL, at 4-decimal parity · native in Postgres + DuckDB · edge WASM · 70+ MCP tools · beats hand-tuned Splink · 250M rows in 11.2 min
Pair drilldown in the web workbench: cluster members, field-level diff, and a one-line NL explanation per pair. pip install goldenmatch[web] then goldenmatch serve-ui <project>. More screenshots →
v3.17.1: The polars-free first run actually works now. 3.17.0 claimed this and did not deliver it: auto-config puts negative evidence on the exact matchkey by default, and that path still bridged to polars, so
goldenmatch dedupe customers.csvexited 3 on a default install. Verified the way it should have been the first time --pip installinto a clean polars-free venv, then the documented command.v3.17.0: The documented first run works on a default install.
pip install goldenmatchfollowed bygoldenmatch dedupe customers.csv-- the quickstart on every doc surface -- exited 3 on a polars-free install, which is what a plain install has produced since polars became an optional extra. Three separate polars imports on the zero-config path (auto-config ingest, the Arrow lane's preflight decline, and the csv writer) are gone, with polars' exact csv bytes reproduced and parity-pinned.v3.13.0: Fellegi-Sunter training runs distributed on Spark. The E-step reads only the comparison vector, so identical vectors collapse to one counted row and the whole step becomes a Spark
GROUP BYover agreement patterns -- the cluster counts, the driver only fits. Training cost tracks DISTINCT vectors (bounded byprod(levels + 1)), not pairs: 1M -> 5M rows grew candidate pairs 5.00x and the distributed counting stage 5.25x, while distinct patterns grew 3.0% (433 -> 446) and driver-side EM stayed at 0.01s. Runs on jar-only executors viagoldenmatch-spark, off the same Rust kernel every other surface uses.
Most entity-resolution tools hand you clusters and stop. GoldenMatch keeps going: it resolves messy records into a durable golden entity, one per real-world customer, that survives re-runs, carries provenance on every field, and answers "who is this, and where did each value come from?" in a single call.
entity_id (UUIDv7) that persists across runs as new data arrives. Records are absorbed, entities merge or split, but the id an entity earns is the id downstream systems can rely on. Run-local cluster numbers reshuffle on every run; these don't.customer_360(entity_id) composes it into one read: golden record, per-field provenance, every linked source record, the event timeline, and the entity's relationship neighborhood:
// customer_360("018f...c2a1") (trimmed)
{
"entity_id": "018f2b7e-...-c2a1", "confidence": 0.97, "record_count": 3,
"sources": ["salesforce", "billing", "support"],
"golden_record": { "name": "Ada Lovelace", "email": "ada@analytical.io", "phone": "+1-555-0100" },
"field_provenance": [
{ "field": "email", "value": "ada@analytical.io",
"winning_source": "billing", "winning_record_id": "billing:8821",
"conflicting_values": [ { "value": "ada@ada.dev", "source": "salesforce" } ] },
{ "field": "phone", "value": "+1-555-0100", "winning_source": "salesforce" }
],
"timeline": [ { "kind": "created", "actor": "pipeline", "recorded_at": "2026-07-30T..." },
{ "kind": "absorbed_record", "reason": "matched billing:8821" } ],
"relationships": [ { "other_entity_id": "018f...9d0e", "kind": "shares_address" } ]
}
What ships today vs. what's emerging. The identity spine is production-grade and in
main: stableentity_ids, per-field provenance, survivorship, merge/split, the append-only log + audit chain, cross-channel stitching, the relationship overlay, and incremental resolution against a persisted index (a new record resolves without a full re-run). Thecustomer_360()serving view above and the source-registry layer that keeps it fresh from live systems are the newer, actively-landing pieces. The source connectors (Snowflake, BigQuery, Salesforce, HubSpot) ship today; the registry that wires them into the spine is emerging. See the Customer 360 design + ADR. We label the seam rather than blur it.
The golden entity lives in the control plane; the matching that builds it runs in the compute engine. That split is the next section.
The golden entity above is produced by two engines that optimize for genuinely different things, and keeping them distinct is the architecture, not an implementation detail (ADR 0047).
flowchart LR
src([source records])
e360([golden entities · Customer 360])
subgraph compute ["Identity Compute Engine: Arrow-native, Rust-authoritative"]
match[block · score · cluster]
end
subgraph control ["Identity Control Plane: transaction-native state machine"]
spine[stable ids · survivorship · merge/split · provenance · audit]
end
src --> compute -->|resolution batch + evidence| control --> e360
control -.->|persisted index| compute
| Identity Compute Engine | Identity Control Plane | |
|---|---|---|
| Shape | Arrow at bulk boundaries, Rust-authoritative kernels | Transaction-native state machine (SQLite default · Postgres) |
| Job | Block, score, cluster: throughput, vectorized, deterministic per run | Stable ids, survivorship, merge/split, provenance, append-only audit |
| State | Stateless per call; measurement-driven kernelization | Durable, transactional, replayable, auditable |
| Backends | DataFusion · Ray · Sail · Spark are replaceable execution backends, none synonymous with GoldenMatch | Storage backends conform to one externally-observable semantics |
Many surfaces, one answer. The same capabilities reach Python, edge-safe TypeScript (with an opt-in WASM backend running the same Rust kernels), SQL inside PostgreSQL and DuckDB, and MCP / REST / A2A, all governed by specification + conformance, not copy-paste. There is one authoritative owner per capability; pure-Python / standalone-TS paths are classified, conformance-tested fallbacks. Where a boundary can't cross byte-for-byte, we measure and label it rather than claim parity.
Why a platform engineer should care:
The identity layer is only as good as the matching underneath it, and the matching starts at zero config. dedupe_df(df) runs with no rules and no training data: it profiles the data, picks a defensible configuration, and returns golden records immediately. The config it chose comes back on result.config: inspectable, diffable, versionable. Never a black box.
historical_50k pairwise F1 0.827 vs 0.757, cluster B³ 0.862 vs 0.788, one shared evaluator, reproducible bake-off. Fuzzy, exact, probabilistic (Fellegi-Sunter), and LLM scorers, with EM-trained weights and calibrated scores.result.suggestions. Each is kept only if it doesn't worsen a health proxy, so a suggestion never makes results worse. dedupe_df(df, heal=True) applies and re-runs in one call. You close the gap to expert-tuned without being the expert.Runs on unstructured input, too: extract records from PDFs and images, then resolve them like any other source (
pip install goldenmatch[documents]).
The engine and the identity layer reach your stack through the surface you already use, with the same capabilities governed by conformance (one product, two engines) rather than re-implemented thinly per surface.
evaluate · Fellegi-Sunter scoring · GoldenFlow transforms. Resolve without moving data out of the warehouse.node:*-free (browsers, Cloudflare Workers, Vercel Edge, Deno); an opt-in WebAssembly backend (await enableWasm()) swaps in the same pyo3-free Rust kernels the Python wheels and SQL UDFs use, with pure-TS as the byte-identical default.spark.addArtifact("goldenmatch-spark.jar")) and are called over JNI, so executors need no goldenmatch virtualenv, no packed env, nothing installed. Fellegi-Sunter training runs distributed on that path -- the E-step is a Spark GROUP BY over agreement patterns, so the cluster does the counting and the driver only fits the model. Deployment story, not a throughput one: the JVM scoring path measured ~2.4x slower than the Python-worker path, and the reason to use it is that there is nothing to install.Surface parity is not the same as handing any pipeline phase from one language to the other byte-for-byte. Each verdict below is measured by a conformance harness, not assumed:
| Boundary | Verdict |
|---|---|
| Identity graph DB | ✅ byte-safe + cryptographically cross-verifiable (a seal written by one toolkit validates under the other) |
score → cluster and the end-to-end split-run | ✅ byte-safe, reproduces the single-language run |
Cluster JSON · config YAML · Learning Memory · record_fingerprint | ✅ portable |
| String scoring | 🟡 4-decimal tolerance; a pair on a threshold can flip (byte-identical only with the shared WASM scorer) |
| Standardize / dates · embeddings · auto-config controller | 🟠 divergent, not byte-portable |
| Distributed / Ray · document (VLM) ingest | ⛔ Python-only by architecture |
Rule of thumb: hand off at the cluster or identity boundary and it's seamless; don't split across standardize/dates, embeddings, or the controller and expect bit-exact reproduction. Full detail + the runnable harness that keeps these verdicts honest: Cross-language parity & phase-handoff limits.
GoldenMatch is the headline, but resolution is only as good as what feeds it. Five sibling tools clean, standardize, and map records before they reach the identity layer. Each stands alone, but they compose into one pipeline, orchestrated declaratively by GoldenPipe:
flowchart LR
raw([raw rows])
golden([golden entities])
subgraph orchestration ["GoldenPipe orchestrates"]
direction LR
infermap[InferMap] --> goldencheck[GoldenCheck] --> goldenflow[GoldenFlow] --> goldenmatch[GoldenMatch]
end
raw --> infermap
goldenmatch --> golden
| Package | Lang | Role in the pipeline | Install |
|---|---|---|---|
| InferMap | Python · TS | Schema mapping: auto-aligns columns across heterogeneous sources | pip install infermap · npm i infermap |
| GoldenCheck | Python · TS | Data-quality scanning: encoding, format validation, anomaly detection | pip install goldencheck · npm i goldencheck |
| GoldenFlow | Python · TS | Transforms & standardizers: phone, date, address, categorical | pip install goldenflow · npm i goldenflow |
| GoldenMatch | Python · TS | Zero-config entity resolution → the identity spine. Headline package. | pip install goldenmatch · npm i goldenmatch |
| GoldenAnalysis | Python · TS | Analysis & reporting: any stage's artifacts → a unified AnalysisReport + cross-run regression detection | pip install goldenanalysis · npm i goldenanalysis |
| GoldenPipe | Python · TS | Orchestrator: declarative YAML wiring the steps | pip install goldenpipe · npm i goldenpipe |
| golden-suite | Python | One-line meta-install: the whole suite + native acceleration | pip install golden-suite |
The deepest docs live in packages/python/goldenmatch/README.md (~1,300 lines: full feature list, CLI, architecture, benchmarks).
Not pipeline stages — the pieces the stages agree through, and the single front door an agent points at.
| Package | What it is | Install |
|---|---|---|
| goldencheck-types | Shared canonical field-type registry: one source of truth for what a field type means, across every package and both languages | pip install goldencheck-types · npm i goldencheck-types |
| goldensuite-mcp | One MCP server exposing every suite tool under a single endpoint (stdio or Streamable HTTP) | pip install goldensuite-mcp |
The suite owns its string-matching primitives instead of renting them: byte-identical drop-in replacements, published on their own so they're usable outside the suite too.
| Library | Replaces | What it is | Install |
|---|---|---|---|
| goldenfuzz | rapidfuzz | Fuzzy-string scorers + the full fuzz.* composite family + one-vs-many extract/cdist. Byte-identical (oracle-fuzzed), faster on short strings. | pip install goldenfuzz · cargo add goldenfuzz-core |
| goldenphonetic | jellyfish | Phonetic encoders: soundex / metaphone / nysiis / match-rating. Byte-identical (6,000-input + 2,500-pair fuzz corpus), pure-Rust zero-dep. | pip install goldenphonetic · cargo add goldenphonetic-core |
| goldenmatch-hnsw | FAISS IndexHNSWFlat | Pure-Rust HNSW approximate-nearest-neighbor index (zero C deps). Powers embedding-based blocking across Python, Rust, and TS/WASM. | pip install goldenmatch-hnsw |
Entity resolution is the stage most GraphRAG pipelines do worst: duplicate surface forms of one entity scatter across documents. Two packages put GoldenMatch's resolution there:
| Package | What it does | Status |
|---|---|---|
| goldenmatch-kg | Drop-in GoldenMatch resolution as the ER stage of existing KG frameworks (neo4j-graphrag, LlamaIndex, Graphiti). | in-repo · not published (by design) |
| goldengraph | Build-your-own-KG from text: text → LLM extraction → GoldenMatch resolution → durable bi-temporal store. Rust engine; ER is the differentiator. | in-repo · first PyPI release pending |
Measured, not asserted (ER-KG-Bench): resolution scores F1 0.602 on the labelled set, ahead of Neo4j-KGBuilder (0.456), neo4j-graphrag (0.403), and MS-GraphRAG / LightRAG / Cognee / mem0 (0.066). A resolved graph also does two things passage-window RAG structurally can't: exact aggregation (size-invariant where RAG recall collapses 0.99 → 0.64 across cluster-size buckets) and temporal as-of (1.000 vs 0.002 on past-date queries).
Every headline number maps back to a single committed runner (scripts/run_benchmarks.py); see docs/reproducing-benchmarks.md for per-number commands, dataset URLs, and expected output with tolerance.
docs/scale-envelope.md): per-backend ranges (in-memory/bucket to a few M · DuckDB out-of-core to ~50M · Ray distributed ≥ 50M), block-size failure modes, and a decision tree for picking a backend.Verified on the Ray tier: a full 100M-row dedupe on a 5-node Ray cluster in 9.2 min (554 s), 20,000,000 golden records recovered exactly, driver peak 0.36 GB RSS. (The larger 250M / 11.2 min headline is the Spark tier on a different cluster shape -- the two are separate lanes, not one number superseding the other, and this one is what carries the recall-complete guarantee below.) The default distributed path is recall-complete: duplicates merge correctly no matter how the input is partitioned (blocking-key shuffle scoring + distributed randomized-contraction WCC), and it stays driver-collect-free end to end. Recipe: configs/distributed-100m.yaml.
Fellegi-Sunter training, distributed on Spark: the E-step collapses to one Spark GROUP BY over agreement patterns, so training cost tracks the number of DISTINCT comparison vectors (bounded by prod(levels + 1)), not the pair count. Measured on a real 2-worker Spark cluster (jar-only executors, no Python installed), 1M -> 5M rows: candidate pairs grew 5.00x and the distributed counting stage 5.25x, while distinct patterns grew 3.0% (433 -> 446) and driver-side EM stayed at 0.01s. That is the property the tier rests on -- the cluster absorbs the data, the driver's work stays flat.
Head-to-head vs Splink on a real Spark cluster, at 50M rows. Both engines on the same 5-node cluster, the same fixture, the same shared metric implementation, and Splink configured the way its own performance guide prescribes -- break_lineage_method="parquet" onto a real distributed filesystem, shuffle partitions at 5x cluster cores, and identical 48 GB executors. Over 463,923,179 candidate pairs, scored identically by both:
| GoldenMatch | Splink | ratio | |
|---|---|---|---|
| wall | 552s | 1,054s | 1.91x |
| u / estimate | 3.5s | 30.8s | 8.78x |
| shuffle write | 86.8 GB | 212.1 GB | 2.44x |
| stages | 119 | 394 | 3.31x |
| executor CPU | 31,300s | 51,572s | 1.65x |
Reported with it, because a benchmark that only publishes its wins is not evidence: the margin still narrows with scale (2.54x at 1M -> 1.91x at 50M), single runs on this lane move ~16% so no one ratio should carry much weight, zero spill was scale-bounded on the build that curve measured (true at 50M, 56.4 GB at 100M, 201.3 GB at 250M -- since taken to zero at 250M by #2698, so the caveat describes the curve rather than the shipped path), the fixture is synthetic, and the accuracy figures in it are not an accuracy verdict -- for that, see the bake-off. Measured further since: 250M rows / 2.32 billion pairs in 670s on the same 5-node cluster, no executor deaths and zero failed tasks, at 0.289 seconds per million pairs -- 4.13x faster than the 2,766s that curve first measured, with 3.1x less executor CPU, zero spill and a byte-identical trained model. Cost stays linear in PAIRS rather than rows; the constant got four times smaller. Four earlier attempts at this comparison were invalid because we had misconfigured Splink; each defect and its effect is documented alongside the results. Full method, caveats and reproduce command.
Three reproducible real-world pipelines run this on public data at scale:
Dedupe a CSV in 30 seconds, zero config, writes <timestamp>_golden.csv:
pip install goldenmatch && goldenmatch dedupe customers.csv
import goldenmatch as gm
result = gm.dedupe("customers.csv") # zero-config
print(result) # DedupeResult(records=5000, clusters=847, match_rate=12.0%)
result.golden.write_csv("deduped.csv")
result = gm.dedupe("customers.csv", # or be explicit
exact=["email"], fuzzy={"name": 0.85, "zip": 0.95}, blocking=["zip"], threshold=0.85)
import { dedupe } from "goldenmatch"; // edge-safe: browsers, Vercel Edge, Workers, Deno
const result = dedupe(rows, { fuzzy: { name: 0.85 }, blocking: ["zip"], threshold: 0.85 });
The whole suite, configured for speed. golden-suite pulls in every package plus the native (Rust) kernels, pinned and defaulted to the perf-optimized config. Native wheels are hard dependencies on purpose: a platform without a wheel fails loudly rather than silently running the slow pure-Python path.
pip install golden-suite
golden-suite doctor # verify every package + native kernel is importable and healthy
golden-suite optimize # repair / re-enable the perf-optimized config
pip install golden-suite[mcp] # + aggregator MCP server (every tool, one endpoint)
pip install golden-suite[all] # everything
Just GoldenMatch. Fat optional extras, so you pay only for what you use (native acceleration is default on common platforms):
pip install goldenmatch # core (CSV in, CSV out) + native
pip install goldenmatch[documents] # + PDF/image ingest (resolve unstructured input)
pip install goldenmatch[embeddings] # + sentence-transformers, FAISS
pip install goldenmatch[llm] # + Claude / OpenAI for LLM boost
pip install goldenmatch[ray] # + Ray distributed backend (50M+ rows)
pip install goldenmatch[postgres] # + Postgres sync (also: [snowflake] [bigquery] [databricks] [salesforce])
pip install goldenmatch[mcp] # + MCP server (also: [agent] A2A, [web] browser workbench)
Web workbench. pip install 'goldenmatch[web]' then goldenmatch serve-ui my-project (opens http://localhost:5050): edit rules with live validation, preview against a sampled slice, label pairs (mirrored into Learning Memory), compare runs.
More: examples/, covering Python (quickstart, full pipeline, customer 360, PPRL, MCP client) · TypeScript (quickstart, Vercel Edge, MCP client) · Airflow.
Remote MCP. The hosted endpoint requires a bearer token -- it exposes tools that read and write files on the server, so it is not open. Ask the maintainer for one, or self-host: goldenmatch mcp-serve --transport http is the same server.
{ "mcpServers": { "goldenmatch": {
"url": "https://goldenmatch-mcp-production.up.railway.app/mcp/",
"headers": { "Authorization": "Bearer YOUR_TOKEN" }
} } }
Self-hosting on loopback needs no token; binding to a public interface without
GOLDENMATCH_MCP_TOKEN is refused rather than served open.
Containers. Every package ships as a multi-arch image (linux/amd64 + arm64) on GHCR, pull anonymously:
docker run -p 8300:8300 ghcr.io/benseverndev-oss/goldensuite-mcp:latest # one container, every tool
docker run -p 8200:8200 ghcr.io/benseverndev-oss/goldenmatch-mcp:latest # per-package (also goldencheck/goldenflow/goldenpipe/infermap)
docker run -e POSTGRES_PASSWORD=secret ghcr.io/benseverndev-oss/goldenmatch-extensions:latest # Postgres + extension
Airflow. 13 drop-in DAGs at examples/airflow/ (TaskFlow API, Airflow 2.7+ / 3.x; idempotent, marker-protected), grouped by lifecycle stage:
| Group | DAGs |
|---|---|
| Core pipeline | daily_dedupe, incremental_match, warehouse_native (Snowflake), customer_360, identity_graph |
| Privacy | pprl_linkage (two-party PPRL) |
| Onboarding & monitoring | schema_align_and_load, schema_drift_alarm, quality_gate |
| Feedback loop | review_worker, active_learning |
| Operationalize | reverse_etl (Salesforce/HubSpot), backfill |
goldenmatch/
├── packages/
│ ├── python/ goldenmatch · goldencheck · goldenflow · goldenpipe · infermap · goldenanalysis
│ │ goldensuite-mcp (aggregator) · golden-suite (meta) · goldengraph · goldenmatch-kg
│ ├── typescript/ full TS ports (edge-safe cores + WASM) · goldencheck-types
│ ├── rust/extensions/ Postgres pgrx + DuckDB UDFs + native kernels + owned libraries (own Cargo workspace)
│ ├── dbt/goldensuite/ dbt materializations, tests, macros
│ └── actions/goldencheck/ GitHub Action
├── examples/ python · typescript · airflow (drop-in DAGs)
├── context-network/ architecture decisions + design docs (ADRs, the two-engine frame, Customer 360)
├── docs/superpowers/ design specs and implementation plans
└── justfile · pyproject.toml (uv workspace) · pnpm-workspace.yaml (Turborepo) · .github/workflows/ci.yml
packages/rust/extensions/ is itself a Cargo workspace (the postgres crate is excluded for pgrx); Cargo commands run from inside it.packages/typescript/* form a single pnpm + Turborepo workspace.just install # uv sync + per-package npm install + cargo fetch
just test # all languages · just lint · just build
feature/<name> branches; merge via squash PR. Titles: feat: / fix: / docs:.packages/typescript/goldenmatch/tests/parity/ enforces 4-decimal Python ↔ TypeScript scorer parity.context-network/decisions/ and docs/superpowers/specs/.corepack enable # one-time, picks up pnpm@9.15.0
pnpm install
pnpm turbo run build test typecheck # full pipeline (cached after first run)
Windows: enable Developer Mode so pnpm install can create symlinks; if corepack enable needs admin, npm i -g pnpm@9.15.0 is equivalent.
This repo was formed on 2026-05-01 by folding 8 sibling repos into goldenmatch via git filter-repo (full history preserved). Built by Ben Severn. MIT, see LICENSE.