
banyandb-bydbql
Generate, validate, and execute read-only BanyanDB BydbQL for SkyWalking data.
- GitHub stars
- 355
Install
/plugin install banyandb-bydbql@banyandbRun it inside Claude Code after adding the marketplace with /plugin marketplace add apache/skywalking-banyandb.
Contents
Discovered from sourceSkills 5
BanyanDB BydbQL
Use this skill when the user asks to query BanyanDB, generate BydbQL, translate natural language to BydbQL, inspect BanyanDB data, validate BydbQL, or run read-only BanyanDB queries.
Workflow
- Read
references/safety.mdbefore generating, validating, or executing BydbQL. - Read
references/syntax.mdwhen query syntax is needed. - Read
references/examples.mdwhen mapping natural language to BydbQL. - Identify the BanyanDB resource type:
STREAM,MEASURE,TRACE, orPROPERTY. - If the group, resource name, or resource type is missing or ambiguous, use
list_groups_schemas. - If the request involves sorting or ranking (
ORDER BY,SHOW TOP, or wording such as "latest", "top", "highest", "lowest", "first", "last"), or you have already fetched schema, callget_generate_bydbql_promptand follow its index-rule guidance. This is required:get_generate_bydbql_promptis the only component that injects the indexed-field list and enforcesORDER BYfield substitution/omission.validate_bydbqlis parse-only, soORDER BY <non-indexed-field>validatestruebut fails at execution. - Generate exactly one read-only BydbQL statement. When it contains
ORDER BY, only sort by a field from the indexed-field list returned byget_generate_bydbql_prompt; substitute the closest indexed field, or omitORDER BYwhen no indexed field matches. - Call
validate_bydbqlbefore execution. - Execute with
list_resources_bydbqlonly when the user asks to run, query, fetch, show, or inspect raw BanyanDB data. - If
validate_bydbqlreturnsvalid=false, fix the query and validate again before any execution. - If execution fails due to a missing group, resource, or schema, use
list_groups_schemasto discover the correct names and retry only when the correction is clear.
Tool Use
Use the existing BanyanDB MCP tools:
list_groups_schemas: discover groups and resource names for streams, measures, traces, and properties.get_generate_bydbql_prompt: return the generation prompt text, including the live indexed-field list and theORDER BYindex-rule substitution/omission rules. It is the only source of index-ruleORDER BYsafety.validate_bydbql: run read-only safety checks and parse-only BydbQL syntax validation. This does not verify group, resource, tag, field, or index-rule existence.list_resources_bydbql: execute a validated BydbQL statement when data retrieval is requested.
The host model performs natural language to BydbQL translation using these references and live schema discovery. Promote get_generate_bydbql_prompt to a required step whenever ranking or sorting is involved (or once schema has been fetched), because it carries the only index-rule ORDER BY check; validate_bydbql is parse-only and cannot catch a sort on a non-indexed field.
Output
For generation-only BydbQL requests, return the single BydbQL statement and state that it was not executed.
For BydbQL execution requests, show the BydbQL statement, then summarize the returned data or error concisely.
Compiling SkyWalking BanyanDB
Follow these steps to compile the project.
Step 1: Generate protobuf and mock code
Run make generate first. This regenerates .pb.go files from .proto definitions and mock files via mockgen.
This step is required whenever:
- Proto files (
.proto) have been added or modified - Go interfaces used by mockgen have changed
If generate fails, check:
protocand Go protobuf plugins are installed- Proto file syntax is valid
- Interface signatures match mock expectations
Step 2: Build all binaries
Run make build to compile all project components: ui, banyand, bydbctl, mcp, fodc/agent, fodc/proxy.
Binaries are output to each component's build/bin/dev/ directory.
Troubleshooting
- Missing fields in .pb.go: Proto files were updated but
make generatewas not run. Run it first. - Mock generation failures: An interface changed but the mock wasn't regenerated.
make generatefixes this. - Import errors: Check import aliases match CLAUDE.md conventions (e.g.,
commonv1,databasev1).
Other useful targets
make clean— clean all build artifactsmake clean-build— clean only build binariesmake lint— run lintersmake test— run tests
Writing a BanyanDB Issue
BanyanDB's issues live in the apache/skywalking tracker, not in this
repository — apache/skywalking-banyandb has issues disabled. Label them
database, which is the component label for BanyanDB.
A good issue is one somebody can implement from without asking you a question. That is a higher bar than "clear", and it is the bar that decides whether an issue sits untouched for a year.
Size the work before polishing the issue
Issue length is not issue size. Inspect the design and the relevant production code before drafting. Count independently testable behaviour, integration seams, callers, format families, lifecycle domains, fixture work, and test suites. A short issue that says "replace the index" can be much larger than a long issue that fixes one query operation.
Always produce a short size audit during review, even if it will not appear in the filed body:
## Size audit
Classification: executable leaf | tracking parent
Boundary: <one seam, or the several seams that make this a parent>
Production activation: <the caller switched by this merge>
RED test: <the command and the observable failure on current main>
End to end: <the real operation proved by the merge>
Format/lifecycle scope: <sections or algorithms; read/write/publish/merge/GC/etc.>
Fixtures and oracle: <who supplies expected values independently of the code under test>
Focused suites: <commands>
Dependencies present on main: yes | no — <evidence>
Decision: <fits one run, or the exact split required>
Do not infer the answers only from the proposal. Use repository evidence such as interfaces, call sites, packages, fixtures, and existing tests. Treat line count as supporting evidence, never as the sole sizing rule.
Executable leaf versus tracking parent
An executable leaf must satisfy all of these:
- It exposes one agreed boundary and delivers one behaviour end to end.
- The same merge switches at least one named production or CLI caller to it.
- Every requirement has a RED test that fails on current
mainfor a stated reason and passes after the change. - One focused end-to-end scenario proves the real caller, not just an isolated codec or helper.
- Expected values and persisted bytes come from an independent oracle.
- Its prerequisites and the seam it builds on are already merged to
main. - A contract author can create the boundary and RED tests in one workflow turn, and an implementer can reasonably complete the production change in the next.
A tracking parent describes a milestone, replacement, or multi-PR outcome. It owns completion criteria, the dependency graph, and links to children, but it is not itself an implementation contract. Mark it clearly as a tracking parent and do not put it in an automated implementation queue.
Mandatory split triggers
Classify the issue as a parent or split it before filing when any of these is true:
- It mixes lifecycle domains such as read, write, publish/recovery, merge/GC, or replication.
- It introduces several independent public seams or several unrelated caller cutovers.
- It replaces a whole third-party subsystem rather than one observable operation.
- No test can be named that fails on current
mainwithout asserting private implementation details. - It needs the production implementation to generate its own expected fixture bytes; that is a circular oracle, not a test contract.
- A required boundary exists only in an open or planned PR. Keep the issue as a parent and decompose it after the blocker merges.
- It is foundation-only: the merge adds format readers, helpers, or abstractions that no production/CLI path uses.
Use this score as a second check when none of the hard triggers is decisive:
| add | signal |
|---|---|
| +2 | each additional public boundary after the first |
| +2 | each additional lifecycle domain after the first |
| +1 | each additional production caller family |
| +1 | each additional persisted format or algorithm family |
| +1 | each additional independent fixture family |
| +1 | each additional focused package test suite |
| +2 | a prerequisite or seam is not yet on main |
Score 0–2: normally a leaf. Score 3–4: split unless repository evidence
shows it is still one small behaviour. Score 5+: a tracking parent. A hard
trigger overrides the score.
For a TDD workflow, also cap a leaf at roughly four observable requirements, one fixture family, one activation point, and one focused end-to-end scenario. These are review budgets, not excuses to hide work by combining requirements.
RED-test feasibility catches false slices
A compatibility-preserving cutover often returns the same result before and after the change, so a new output assertion may pass against the legacy path and is not RED. State why the test fails before implementation. Absence of a new boundary can be a compile-time RED test, but acceptance must also prove that a real caller uses the boundary; do not use brittle source-text assertions as the only proof.
If a fixture is required, include its cost in the size audit. Persisted expected bytes should be generated by an independent legacy/reference writer and checked in with provenance before the native code consumes them. Do not create a fixture-only or codec-only ticket when the project requires every merge to activate production behaviour; put the smallest fixture and decoder beside the first real caller that uses them.
The body is the specification
Assume the implementer has your issue and the codebase, and nothing else — no context from a design meeting, no follow-up in the comments. Comments are where a specification goes to be missed; if a discussion changed the plan, edit the body.
Four things have to be derivable from it:
A boundary. The seam the change exposes: a Go interface, the exported
functions, the protobuf message, the HTTP surface. "Add an Authorizer that
decides whether a principal may perform an operation on a group" is a boundary.
"Improve permissions" is not.
Expected values from an independent source. State the concrete answers, not
just the behaviour. A test whose expected value is derived the way the
implementation derives it passes by construction and can never disagree with the
code — so the issue is where the real answers come from. Write a reader scoped to group "ecommerce" sees only that group, not scoping should work.
Acceptance criteria provable by a command. If a criterion is "a research document" or "benchmark results", nobody can tell whether it is met. Those are worth doing and belong in a proposal, not in an implementable issue.
Where it lives and how that area is tested. Name the packages or workspace. It saves the implementer a discovery pass and it scopes the suite they run:
| area | tests |
|---|---|
Go — banyand/, pkg/ | make test-ci PKG=./banyand/trace ./pkg/index |
Canopy — canopy/ | npm run -w web test, npm run -w server test |
| Canopy end-to-end | npx playwright test --config=e2e/playwright.config.ts |
| everything, before pushing | make generate && make build && make pre-push |
What a database issue owes that a service issue does not
Weight the criteria by what fails silently and late. If the change touches any of these, say so in the issue — the implementer should not have to infer it:
- Anything persisted or on the wire. A segment, an index, a protobuf
message: existing data must still read, and a rolling upgrade must survive a
mixed-version cluster. State what an old reader does with new bytes and what a
new reader does with old ones. The project labels this class
bydb file compatible change. - Durability and ordering. For a flush, a WAL append, a compaction or a snapshot: what is the state after a crash between any two steps?
- Concurrency. Goroutine launches go through the panic-recovery wrappers
(
run.Go/run.GoOrDie/run.GoWithSignal); a rawgo func(needs an explicit//panicdiag:allow-rawgodirective and the lint baseline only ever shrinks. - Resource bounds. An unbounded buffer, a query that materialises a whole series, a retention path that cannot reclaim.
A shape that works
## Summary
One paragraph: what this delivers and why.
## Boundary
The seam this exposes — the interface, exported functions, or protobuf/HTTP
surface. What callers may rely on, and what stays free to change.
## Requirements
R1. <observable behaviour, with its concrete expected value>
R2. ...
one per behaviour, each independently checkable.
## Acceptance criteria
- every requirement has a test that fails today and passes after
- end-to-end: <a real use of the feature, start to finish>
- <compatibility / durability / concurrency criteria, where they apply>
## Scope
Packages: banyand/trace, pkg/index — or canopy/web.
Out of scope: <the neighbouring thing this is NOT>
Breaking a design-ready issue into tickets
When an issue carries a design and a proof of concept, it is usually several pieces of work. Split it into vertical slices — each cutting a complete path through the layers it touches, each verifiable on its own — and file them as sub-issues of the base.
A slice is the right size when it delivers one behaviour end to end and its acceptance criteria can be checked by running something. A slice that only makes sense alongside the next one is not a slice; it is half of one.
Do not slice a storage replacement horizontally by format section (container,
postings, stored fields) if those tickets would be unused foundations. Slice
by fixture complexity against one real caller instead. A first read slice may
activate document count on a deliberately small fixture; a later slice can widen
that same caller to multiple segments and deletion masks; another caller can
then activate stored-field decoding. Each merge stays TDD-sized and live.
Decompose just in time. Keep downstream design workstreams as tracking parents until their blocker lands. Then inspect the boundary that actually merged and create their executable leaves. Pre-filing speculative leaves commits to stale seams and recreates horizontal slicing at the issue layer.
Order them by dependency and say so. Put Blocked by: #NNNN in the body of
anything that needs an earlier ticket, and create them in that order so the
numbers ascend with the graph. Do not start a blocked ticket before its blocker
has merged — not merely opened a pull request. Until the merge, main does not
contain the seam the next ticket builds on.
A pure refactor is a poor ticket if the project's convention is to land behaviour with a failing test first: a change that alters no behaviour has no test that fails today. Either land the refactor on its own as an explicit cleanup, or fold it into the first ticket that adds behaviour.
Wide, mechanical changes use expand–contract, one ticket per phase:
- Expand — add the new form alongside the old.
- Migrate — move call sites in batches sized by blast radius.
- Contract — delete the old form once nothing uses it.
Filing them
Create each child, then attach it to the base by the child's id, which is
not its number:
url=$(gh issue create --repo apache/skywalking \
--title "..." --body-file ./ticket-1.md --label database)
n=${url##*/}
id=$(gh api repos/apache/skywalking/issues/$n -q .id)
gh api --method POST repos/apache/skywalking/issues/<base>/sub_issues -F sub_issue_id=$id
-F, not -f: the endpoint takes an integer, and -f sends the string form.
Check the result with:
gh api repos/apache/skywalking/issues/<base> -q .sub_issues_summary
The base issue stays open as the umbrella, and sub_issues_summary is the
progress bar. Close it when the last ticket merges.
If automation selects the oldest issue with a queue label such as Backlog,
apply that label only to the oldest unblocked executable leaf. Never queue a
tracking parent or all sibling leaves at once. Create children in dependency
order so issue numbers reinforce rather than fight the execution order.
Before you file
- Did you show the size audit and explicitly classify the issue as a leaf or a tracking parent?
- Could someone write a failing test from this body alone, without asking you anything?
- Does the test actually fail on current
main, or would the legacy path already satisfy it? - Does every expected value come from the issue rather than from what the code would compute?
- Is every acceptance criterion provable by running something?
- Is it one boundary and one production activation, or a plan with phases?
- Does this merge activate a real caller rather than land dead foundation code?
- Are all prerequisite seams already merged to
main? - If it touches stored or wire data, does it say what happens to existing data and to a mixed-version cluster?
Creating a Pull Request for SkyWalking BanyanDB
Branch Rules
Always create a PR from a new branch. Never push directly to main. This is the same convention as the main Apache SkyWalking repo (https://github.com/apache/skywalking).
git checkout -b <descriptive-branch-name>
Local Checks Before Creating PR
Run these checks locally before pushing. They mirror the CI check job and PR-blocking tests in .github/workflows/ci.yml.
Required: Code Generation and Build
make generate
make build
Required: Linting and Formatting
make lint
make check
make check verifies formatting (gofumpt), go mod tidy, and ensures no uncommitted generated file diffs.
Why this order: build must succeed before lint so generated code exists. check validates consistency after linting.
After make lint passes: If lint introduced any fixes (e.g. auto-formatting, field alignment corrections), commit those changes before running make check. Do NOT update CHANGES.md for these fixup commits — just stage all modified tracked files and commit with a message like chore: fix lint issues. Then run make check on the clean tree.
Required: License Headers
make license-check
All source files must have Apache 2.0 license headers.
Required: Update CHANGES.md
Add a one-line entry under the current development version section in CHANGES.md (at the repo root). Place it under the appropriate subsection (### Features, ### Bug Fixes, etc.).
Unit Tests
Run these test packages. Each can run in parallel if the user's machine has enough cores, but it's fine to run them sequentially:
make test-ci PKG=./banyand/...
make test-ci PKG=./bydbctl/...
make test-ci PKG=./pkg/...
make test-ci PKG=./fodc/...
The CI uses these options: --vv --fail-fast --label-filter \!slow with coverage flags. For local runs, use a simplified form unless the user asks for full CI parity:
TEST_CI_OPTS="--vv --fail-fast --label-filter \!slow" make test-ci PKG=./banyand/...
Integration Tests
Run these after unit tests pass:
make test-ci PKG=./test/integration/standalone/...
make test-ci PKG=./test/integration/distributed/...
Checks NOT Practical Locally
These run in CI only — no need to run locally:
- e2e tests — require Docker + OAP stack (90 min timeout)
- fodc-e2e tests — require Kind Kubernetes cluster
- dependency-review — GitHub-specific action
- slow/flaky/property-repair tests — scheduled, not PR-blocking
Common issues
-
make lintfails with field alignment errors: The linter reports structs with suboptimal field ordering (e.g.fieldalignment: struct with X pointer bytes could be Y). Fix them automatically with:~/go/bin/fieldalignment -fix ./path/to/package/...Parse the lint output to find which packages have alignment issues, run
fieldalignment -fixon those packages, then re-runmake lintto confirm they're resolved. -
make lintfails with formatting errors: Runmake formatto auto-fix, then re-runmake lintto confirm. -
Test timeout: Integration tests can be slow; add
TEST_CI_OPTS="--timeout 60m"if they time out -
Missing tools:
make check-reqwill tell you what's missing -
make lintormake checkfails withbuf: not found: Installbufautomatically by runningmake -C api generate, then retry. -
make buildfails: Check if generated files are up to date withmake generate
Creating the PR
git push -u origin <branch-name>
gh pr create --title "<title>" --body "<body>"
Follow the standard PR format with a summary and test plan.
After the PR Is Merged
Merged branches must be removed, both locally and on origin, so they do not pile up. Do this once the PR has actually been merged — never for an open PR, and never for a PR that was closed without merging unless the user asks.
Confirm the merge state first:
gh pr view <number> --json state,mergedAt --jq '"\(.state) \(.mergedAt)"'
Only proceed when the state is MERGED. Then clean up:
git checkout main
git pull --ff-only origin main
git branch -D <branch-name>
git push origin --delete <branch-name>
git fetch --prune origin
Notes:
-
This repo only allows squash merges, so the squash commit on
mainnever matches the branch's own commits.git branch -dtherefore refuses to delete even a merged branch;-Dis required, which is why theMERGEDcheck above comes first. -
If
git push origin --deletereportsremote ref does not exist, GitHub already removed the remote branch. That is fine; continue withgit fetch --prune. -
Also prune any other local branches whose upstream is already gone (earlier merged PRs). List them, confirm with the user that each was merged, then delete:
git fetch --prune origin git branch -vv | grep ': gone\]' | awk '{print $1}'
Upgrade vendor dependencies
Run the vendor-update Make target across all projects:
make vendor-update
This runs go get -u ./... and go mod tidy -compat=1.26 in each project directory (ui, banyand, bydbctl, mcp, fodc/agent, fodc/proxy, pkg, test).
Update license dependencies
After the vendor upgrade completes, run make license-dep to update any license-related files that may have changed due to new or updated dependencies:
make license-dep
This ensures license headers and dependency license files stay in sync with the upgraded vendors.
Check for tool version drift
After upgrading, compare go.mod dependency versions against scripts/build/version.mk to find any tools whose library version was bumped and may need their tool version updated.
Version mapping
Read scripts/build/version.mk and go.mod in parallel. For each tool version variable, check if the corresponding library in go.mod was upgraded:
| version.mk variable | Binary | go.mod package |
|---|---|---|
BUF_VERSION | bin/buf | github.com/bufbuild/buf (if present) |
PROTOC_GEN_GO_VERSION | bin/protoc-gen-go | google.golang.org/protobuf |
PROTOC_GEN_GO_GRPC_VERSION | bin/protoc-gen-go-grpc | google.golang.org/grpc |
PROTOC_GEN_DOC_VERSION | bin/protoc-gen-doc | github.com/pseudomuto/protoc-gen-doc (if present) |
GRPC_GATEWAY_VERSION | bin/protoc-gen-grpc-gateway | github.com/grpc-ecosystem/grpc-gateway/v2 |
PROTOC_GEN_VALIDATE_VERSION | bin/protoc-gen-validate | github.com/envoyproxy/protoc-gen-validate |
GOLANGCI_LINT_VERSION | bin/golangci-lint | (no go.mod entry — only in version.mk) |
REVIVE_VERSION | bin/revive | (no go.mod entry — only in version.mk) |
LICENSE_EYE_VERSION | bin/license-eye | (no go.mod entry — only in version.mk) |
MOCKGEN_VERSION | bin/mockgen | go.uber.org/mock |
GINKGO_VERSION | bin/ginkgo | github.com/onsi/ginkgo/v2 |
GOVULNCHECK_VERSION | bin/govulncheck | golang.org/x/vuln |
BPF2GO_VERSION | (used in bpf2go.mk) | github.com/cilium/ebpf |
Only update version.mk if the library version in go.mod has changed and the tool version should follow. Many tools (golangci-lint, revive, license-eye) are not in go.mod — their versions only change when explicitly updated in version.mk, not from make vendor-update.
Update version.mk if needed
If a tool version should be bumped (library changed and tool version needs to match), update scripts/build/version.mk using the Edit tool. Keep the format exactly — e.g., PROTOC_GEN_GO_VERSION := v1.36.11.
Remove stale binaries
When version.mk is updated, remove the corresponding old binary from bin/:
rm bin/<binary-name>
This ensures the next build/install of that tool fetches the correct new version.
MCP servers 1
Sponsors
Details
- Marketplace
- apache/skywalking-banyandb
- Category
- Community