197 validated life sciences skills — genomics, proteomics, drug discovery, biostatistics, scientific computing

version
1.0.0
GitHub stars
393

Install

/plugin install sciagent-skills@sciagent-skills

Run it inside Claude Code after adding the marketplace with /plugin marketplace add jaechang-hits/SciAgent-Skills.

Contents

Discovered from source

Skills 205

PyMC Bayesian Modeling

Overview

PyMC is a Python library for Bayesian statistical modeling and probabilistic programming. It provides an expressive syntax for defining probabilistic models and efficient inference via MCMC (NUTS) and variational methods (ADVI). This skill covers the full Bayesian modeling cycle from model specification through diagnostics, comparison, and prediction.

When to Use

  • Estimating parameters with full uncertainty quantification (credible intervals, not just point estimates)
  • Fitting hierarchical/multilevel models to grouped or nested data
  • Performing prior and posterior predictive checks to validate model assumptions
  • Comparing candidate models using information criteria (LOO-CV, WAIC)
  • Building regression models (linear, logistic, Poisson) in a Bayesian framework
  • Handling missing data or measurement error as latent parameters
  • Modeling time series with autoregressive or random walk priors
  • Generating posterior predictions for new observations with uncertainty bounds
  • Use Stan/PyStan instead for compiled, more scalable Bayesian inference on large models; use statsmodels for frequentist statistical tests

Prerequisites

  • Python packages: pymc >= 5.0, arviz, numpy, matplotlib
  • Data: NumPy arrays or pandas DataFrames with numeric columns
  • Environment: CPU sufficient for most models; GPU via JAX backend for large models
pip install pymc arviz numpy matplotlib
# Optional: JAX backend for GPU acceleration
pip install pymc[jax]

Quick Start

import pymc as pm
import arviz as az
import numpy as np

# Simulate data
np.random.seed(42)
X = np.random.randn(100)
y = 2.5 + 1.3 * X + np.random.randn(100) * 0.5

# Build and fit model
with pm.Model() as model:
    alpha = pm.Normal("alpha", mu=0, sigma=5)
    beta = pm.Normal("beta", mu=0, sigma=5)
    sigma = pm.HalfNormal("sigma", sigma=1)
    mu = alpha + beta * X
    y_obs = pm.Normal("y_obs", mu=mu, sigma=sigma, observed=y)
    idata = pm.sample(1000, tune=1000, chains=4, random_seed=42)

print(az.summary(idata, var_names=["alpha", "beta", "sigma"]))
# Expected: alpha ~ 2.5, beta ~ 1.3, sigma ~ 0.5

Workflow

Step 1: Prepare Data

Standardize continuous predictors for better sampling efficiency. Use named coordinates for readable models and ArviZ integration.

import pymc as pm
import arviz as az
import numpy as np

# Load data
X = np.random.randn(200, 3)  # 200 obs, 3 predictors
y = X @ np.array([1.0, -0.5, 0.3]) + np.random.randn(200) * 0.8

# Standardize predictors
X_mean, X_std = X.mean(axis=0), X.std(axis=0)
X_scaled = (X - X_mean) / X_std

# Define coordinates for named dimensions
coords = {
    "predictors": ["var1", "var2", "var3"],
    "obs_id": np.arange(len(y)),
}
print(f"Data shape: X={X_scaled.shape}, y={y.shape}")

Step 2: Define Model and Set Priors

Specify the model structure inside a pm.Model() context. Use weakly informative priors, dims for named dimensions, and HalfNormal or Exponential for scale parameters.

with pm.Model(coords=coords) as model:
    # Priors — weakly informative, not flat
    alpha = pm.Normal("alpha", mu=0, sigma=1)
    beta = pm.Normal("beta", mu=0, sigma=1, dims="predictors")
    sigma = pm.HalfNormal("sigma", sigma=1)

    # Linear predictor
    mu = alpha + pm.math.dot(X_scaled, beta)

    # Likelihood
    y_obs = pm.Normal("y_obs", mu=mu, sigma=sigma, observed=y, dims="obs_id")

# Inspect model variables
print(model.basic_RVs)  # Lists: [alpha, beta, sigma, y_obs]

Step 3: Prior Predictive Check

Validate that priors produce plausible data ranges before fitting. Adjust priors if simulated data is unreasonable.

with model:
    prior_pred = pm.sample_prior_predictive(samples=1000, random_seed=42)

# Check prior-implied data range
prior_y = prior_pred.prior_predictive["y_obs"].values.flatten()
print(f"Prior predictive range: [{prior_y.min():.1f}, {prior_y.max():.1f}]")
print(f"Observed data range:    [{y.min():.1f}, {y.max():.1f}]")

az.plot_ppc(prior_pred, group="prior", num_pp_samples=100)

Step 4: Sample Posterior (MCMC)

Run NUTS sampling with multiple chains. Include log_likelihood=True if you plan model comparison later.

with model:
    idata = pm.sample(
        draws=2000,
        tune=1000,
        chains=4,
        target_accept=0.9,
        random_seed=42,
        idata_kwargs={"log_likelihood": True},
    )

print(f"Posterior shape: {idata.posterior['beta'].shape}")
# Expected: (4 chains, 2000 draws, 3 predictors)

Step 5: Diagnose Sampling

Check convergence before interpreting results. All three diagnostics (R-hat, ESS, divergences) must pass.

# Summary with convergence diagnostics
summary = az.summary(idata, var_names=["alpha", "beta", "sigma"])
print(summary[["mean", "sd", "hdi_3%", "hdi_97%", "r_hat", "ess_bulk"]])

# R-hat convergence check
bad_rhat = summary[summary["r_hat"] > 1.01]
if len(bad_rhat) > 0:
    print(f"WARNING: {len(bad_rhat)} parameters with R-hat > 1.01")
    print(bad_rhat[["r_hat"]])

# Effective sample size check
low_ess = summary[summary["ess_bulk"] < 400]
if len(low_ess) > 0:
    print(f"WARNING: {len(low_ess)} parameters with ESS < 400")

# Divergence check
n_div = idata.sample_stats.diverging.sum().item()
total = len(idata.posterior.draw) * len(idata.posterior.chain)
print(f"Divergences: {n_div}/{total} ({n_div / total * 100:.2f}%)")

# Visual diagnostics — trace plots and rank plots
az.plot_trace(idata, var_names=["alpha", "beta", "sigma"])
az.plot_rank(idata, var_names=["alpha", "beta", "sigma"])

Step 6: Posterior Predictive Check

Validate model fit by comparing simulated data from the posterior to observed data.

with model:
    pm.sample_posterior_predictive(idata, extend_inferencedata=True, random_seed=42)

az.plot_ppc(idata, num_pp_samples=100)
# Blue = observed data, grey = posterior simulations
# Systematic deviations indicate model misspecification

Step 7: Compare Models

Use LOO-CV or WAIC to compare candidate models. Lower information criterion is better.

# Fit multiple models with log_likelihood=True, then compare
# Example: compare linear vs a second model
idatas = {"linear": idata}  # add more fitted models here

comparison = az.compare(idatas, ic="loo")
print(comparison[["rank", "elpd_loo", "p_loo", "d_loo", "weight"]])

# Check LOO reliability via Pareto-k diagnostics
loo_result = az.loo(idata, pointwise=True)
high_k = (loo_result.pareto_k > 0.7).sum().item()
print(f"Observations with Pareto-k > 0.7: {high_k}")
# Interpretation: Dloo < 2 = similar models; Dloo > 10 = strong evidence

az.plot_compare(comparison)

Step 8: Generate Predictions

Produce posterior predictions for new data with full uncertainty propagation.

X_new = np.array([[0.5, -1.0, 0.2]])
X_new_scaled = (X_new - X_mean) / X_std

with model:
    pm.set_data({"X_scaled": X_new_scaled})
    post_pred = pm.sample_posterior_predictive(
        idata.posterior, var_names=["y_obs"], random_seed=42
    )

y_pred = post_pred.posterior_predictive["y_obs"]
print(f"Predicted mean: {y_pred.mean().item():.3f}")
print(f"94% HDI: {az.hdi(y_pred, hdi_prob=0.94).values}")

Key Parameters

ParameterDefaultRange / OptionsEffect
draws1000500-10000Number of posterior samples per chain
tune1000500-5000Warmup iterations (discarded); increase for complex posteriors
chains42-8Number of independent chains; minimum 4 for reliable R-hat
coresall CPUs1-NParallel chains; set equal to chains for full parallelism
target_accept0.80.8-0.99NUTS acceptance rate; increase to reduce divergences
init"auto""adapt_diag", "jitter+adapt_diag", "advi"Initialization strategy for sampler
random_seedNoneany intSeed for reproducibility
idata_kwargs{}{"log_likelihood": True}Store log-likelihood for LOO/WAIC model comparison
method (pm.fit)"advi""advi", "fullrank_advi", "svgd"Variational inference algorithm
n (pm.fit)100005000-100000VI optimization iterations
samples (prior pred)500100-5000Prior predictive samples for validation

Key Concepts

Prior/Distribution Selection Guide

DistributionUse WhenKey Parameters
Normal(mu, sigma)Unbounded real-valued parameter (standardized data)mu: center, sigma: spread
HalfNormal(sigma)Scale/standard deviation parameter (positive)sigma: spread of positive half
Exponential(lam)Scale parameter, alternative to HalfNormallam: rate (1/mean)
StudentT(nu, mu, sigma)Robust alternative to Normal (outlier-resistant)nu: degrees of freedom (<10 = heavier tails)
Beta(alpha, beta)Probability or proportion in [0,1]alpha=beta=2: weakly informative
Gamma(alpha, beta)Positive parameter (rate, concentration)alpha: shape, beta: rate
LogNormal(mu, sigma)Positive parameter with multiplicative effectsmu, sigma: of underlying Normal
LKJCorr(n, eta)Correlation matrix prioreta=1: uniform; eta>1: prefer identity
Dirichlet(a)Probability vector (sums to 1)a: concentration; uniform if all equal
Bernoulli(p / logit_p)Binary outcome likelihoodUse logit_p for numerical stability
Poisson(mu)Count data (equidispersed)mu: rate; use NegBinomial if overdispersed
NegativeBinomial(mu, alpha)Overdispersed count dataalpha: dispersion (smaller = more overdispersion)

Diagnostic Thresholds

MetricThresholdInterpretationAction if Failed
R-hat< 1.01Chains convergedRun longer chains; check multimodality
ESS bulk> 400Sufficient independent samplesIncrease draws; reparameterize
ESS tail> 400Reliable tail estimatesIncrease draws
Divergences0NUTS explored successfullyIncrease target_accept; non-centered param.
Pareto-k (LOO)< 0.7LOO estimate reliableUse WAIC or k-fold CV
Max tree depth< 10No trajectory truncationReparameterize or increase max_treedepth

Model Variants Overview

Problem TypeRecipeLikelihoodKey Feature
Grouped/nested dataHierarchical ModelNormal (varies)Non-centered parameterization, partial pooling
Binary outcomeLogistic RegressionBernoullilogit_p link function
Nonlinear/spatialGaussian ProcessNormalKernel-based covariance, flexible shape
Count data(use Poisson in Workflow)Poisson / NegBinomialLog link; NegBinomial for overdispersion
Time series(see references)AR / GaussianRandomWalkAutoregressive coefficients
Mixture/clustering(see references)Mixture / NormalMixtureComponent weights via Dirichlet

Common Recipes

Recipe: Hierarchical Model

When to use: data has natural grouping (patients within hospitals, students within schools). Non-centered parameterization avoids divergences from funnel geometry.

import pymc as pm
import arviz as az
import numpy as np

n_groups, n_per_group = 5, 30
group_idx = np.repeat(np.arange(n_groups), n_per_group)
group_names = [f"group_{i}" for i in range(n_groups)]

# Simulated grouped data
true_alphas = np.random.normal(3.0, 1.5, n_groups)
y_obs = np.random.normal(true_alphas[group_idx], 0.5)

with pm.Model(coords={"groups": group_names}) as hierarchical_model:
    # Hyperpriors (population level)
    mu_alpha = pm.Normal("mu_alpha", mu=0, sigma=5)
    sigma_alpha = pm.HalfNormal("sigma_alpha", sigma=2)

    # Non-centered parameterization
    alpha_offset = pm.Normal("alpha_offset", mu=0, sigma=1, dims="groups")
    alpha = pm.Deterministic(
        "alpha", mu_alpha + sigma_alpha * alpha_offset, dims="groups"
    )

    # Observation level
    sigma = pm.HalfNormal("sigma", sigma=1)
    y = pm.Normal("y", mu=alpha[group_idx], sigma=sigma, observed=y_obs)

    idata_hier = pm.sample(2000, tune=1000, target_accept=0.95, random_seed=42)

print(az.summary(idata_hier, var_names=["mu_alpha", "sigma_alpha", "alpha", "sigma"]))

Recipe: Logistic Regression

When to use: binary outcome variable (success/failure, disease/healthy). Use logit_p for numerical stability instead of computing probabilities directly.

import pymc as pm
import arviz as az
import numpy as np

# Simulated binary outcome
np.random.seed(42)
n, p = 200, 3
X_lr = np.random.randn(n, p)
true_beta = np.array([1.0, -0.5, 0.3])
prob = 1 / (1 + np.exp(-(0.2 + X_lr @ true_beta)))
y_binary = np.random.binomial(1, prob)

with pm.Model() as logistic_model:
    alpha = pm.Normal("alpha", mu=0, sigma=2)
    beta = pm.Normal("beta", mu=0, sigma=2, shape=p)
    logit_p = alpha + pm.math.dot(X_lr, beta)
    y = pm.Bernoulli("y", logit_p=logit_p, observed=y_binary)

    idata_logit = pm.sample(2000, tune=1000, target_accept=0.9, random_seed=42)

summary = az.summary(idata_logit, var_names=["alpha", "beta"])
print(summary[["mean", "sd", "hdi_3%", "hdi_97%"]])
# Odds ratios
beta_samples = idata_logit.posterior["beta"].values.reshape(-1, p)
print(f"Odds ratios (median): {np.median(np.exp(beta_samples), axis=0)}")

Recipe: Gaussian Process

When to use: modeling unknown nonlinear functions, spatial data, or smooth latent processes. Computationally expensive for n > 1000; consider sparse approximations for larger datasets.

import pymc as pm
import arviz as az
import numpy as np

# Simulated nonlinear data
np.random.seed(42)
X_gp = np.sort(np.random.uniform(0, 10, 60))[:, None]
y_gp = np.sin(X_gp[:, 0]) + np.random.randn(60) * 0.3

with pm.Model() as gp_model:
    # GP hyperparameters
    ls = pm.Gamma("lengthscale", alpha=2, beta=1)
    eta = pm.HalfNormal("amplitude", sigma=2)
    sigma_noise = pm.HalfNormal("noise", sigma=0.5)

    # Covariance function
    cov = eta**2 * pm.gp.cov.Matern52(1, ls=ls)
    gp = pm.gp.Marginal(cov_func=cov)

    # Marginal likelihood
    y_ = gp.marginal_likelihood("y", X=X_gp, y=y_gp, sigma=sigma_noise)
    idata_gp = pm.sample(1000, tune=1000, target_accept=0.9, random_seed=42)

# Predict on new points
X_new_gp = np.linspace(0, 10, 100)[:, None]
with gp_model:
    f_pred = gp.conditional("f_pred", X_new_gp)
    pred_samples = pm.sample_posterior_predictive(idata_gp, var_names=["f_pred"])

print(f"GP predictions shape: {pred_samples.posterior_predictive['f_pred'].shape}")

Expected Outputs

  • Trace plots (az.plot_trace) -- chain mixing visualization; healthy chains look like "fuzzy caterpillars"
  • Posterior summary (az.summary) -- mean, SD, HDI, R-hat, ESS per parameter
  • Prior/posterior predictive plots (az.plot_ppc) -- simulated vs observed data distributions
  • Forest plots (az.plot_forest) -- coefficient estimates with credible intervals
  • Model comparison table (az.compare) -- ranked models with LOO/WAIC, weights, warnings
  • Rank plots (az.plot_rank) -- uniform histograms indicate good mixing
  • Energy plot (az.plot_energy) -- HMC energy transition diagnostics
  • InferenceData file (idata.to_netcdf("results.nc")) -- serialized results for later analysis

Troubleshooting

ProblemCauseSolution
Divergent transitionsPosterior geometry difficult for NUTS (funnels, ridges)Increase target_accept=0.95; use non-centered parameterization; add stronger priors
Low ESS (< 400)High autocorrelation between samplesIncrease draws=5000; reparameterize to reduce correlation; try QR decomposition for correlated predictors
R-hat > 1.01Chains have not convergedIncrease tune=2000, draws=5000; check for multimodality; initialize with ADVI
Slow samplingComplex posterior or large datasetUse ADVI for initialization; reduce model complexity; increase cores; try JAX backend
Pareto-k > 0.7 in LOOInfluential observations affecting LOO estimateUse WAIC instead; investigate influential points; consider k-fold CV
Hit max tree depthModel geometry requires long trajectoriesReparameterize model; increase max_treedepth parameter
SamplingError at startPoor initializationTry init="adapt_diag" or init="jitter+adapt_diag"; provide initvals via MAP
Biased posterior (prior dominates)Priors too strong relative to dataWeaken priors (increase sigma); check prior predictive; verify data is correctly passed
ValueError: logp = -infParameter hit impossible regionCheck data for NaN/Inf; ensure positive params use HalfNormal/Gamma; verify likelihood matches data type

Best Practices

  1. Use weakly informative priors, not flat priors. Flat priors (Uniform over large ranges) cause sampling difficulties and are rarely appropriate.
  2. Always run prior predictive checks before fitting. If prior-implied data is implausible, adjust priors before sampling.
  3. Standardize continuous predictors to mean=0, sd=1 for better NUTS geometry and faster sampling.
  4. Use non-centered parameterization for hierarchical models: offset ~ Normal(0,1); param = mu + sigma * offset instead of param ~ Normal(mu, sigma).
  5. Run at least 4 chains for reliable R-hat and ESS diagnostics. Never trust a single chain.
  6. Check all three diagnostics (R-hat < 1.01, ESS > 400, 0 divergences) before interpreting posteriors.
  7. Include log_likelihood=True when fitting if you plan to compare models via LOO or WAIC.
  8. Use dims instead of shape for named dimensions -- integrates with ArviZ for labeled summaries and subsetting.
  9. Report credible intervals (HDI), not just point estimates. The posterior distribution is the result, not a single number.
  10. Save results with idata.to_netcdf("results.nc") for reproducibility and later re-analysis.

Bundled Resources

Two reference files provide deeper detail for on-demand consultation:

  • references/distributions_inference.md -- Distribution catalog organized by category (continuous, discrete, multivariate, mixture, time series, special/modifiers) with full parameter signatures, support, and common uses. Sampling and inference methods (NUTS, Metropolis, Slice, CompoundStep, SMC, ADVI, fullrank ADVI, SVGD, MAP) with code examples and a method selection guide table. Reparameterization tricks (non-centered, QR decomposition) with code. Covers: consolidated from original references/distributions.md (320 lines) and references/sampling_inference.md (424 lines). Relocated inline: core distribution selection guide table (Key Concepts), diagnostic threshold table (Key Concepts), basic ADVI usage (Recipe: Variational Inference in original, covered in references here). Omitted: shape broadcasting examples (trivial NumPy-style shape=5), basic dims usage (covered in Workflow Step 2), prior predictive sampling (covered in Workflow Step 3).

  • references/advanced_workflows.md -- Model comparison workflow (LOO, WAIC, Pareto-k reliability checks, model averaging with weighted predictions), comprehensive diagnostic report generation (trace plots, rank plots, autocorrelation, energy, ESS evolution), prior-posterior comparison patterns, data preparation best practices (standardization, centering, missing data imputation), prior selection guidelines (weakly informative vs informative with domain knowledge), named dimensions (dims) usage with xarray subsetting, save/load patterns (NetCDF, pickle). Mixture model pattern. Covers: consolidated from original references/workflows.md (526 lines). Relocated inline: core diagnostic checking logic (Workflow Step 5), model comparison basics (Workflow Step 7), prior predictive check (Workflow Step 3), linear/logistic/hierarchical model code (Recipes). Omitted: complete monolithic workflow template (redundant with 8-step Workflow section).

Per-reference-file disposition (all originals)

Original references/ (3 files):

  1. distributions.md (320 lines) -> consolidated with sampling_inference.md into new references/distributions_inference.md
  2. sampling_inference.md (424 lines) -> consolidated with distributions.md into new references/distributions_inference.md
  3. workflows.md (526 lines) -> migrated as new references/advanced_workflows.md

Original scripts/ (2 files):

  1. model_diagnostics.py (350 lines) -> check_diagnostics() logic (R-hat, ESS, divergences, tree depth checks) inlined in Workflow Step 5; create_diagnostic_report() plot generation patterns described in Expected Outputs and referenced in references/advanced_workflows.md; compare_prior_posterior() utility covered in references/advanced_workflows.md
  2. model_comparison.py (387 lines) -> compare_models() and check_loo_reliability() patterns inlined in Workflow Step 7 (using az.compare and az.loo directly); model_averaging() pattern covered in references/advanced_workflows.md; cross_validation_comparison() guidance covered in references/advanced_workflows.md; plot_model_comparison() is a thin wrapper around az.plot_compare (inlined in Step 7)

Related Skills

  • arviz (planned) -- posterior visualization and diagnostics; PyMC returns ArviZ InferenceData objects
  • bambi (planned) -- formula-based interface to PyMC (R-style y ~ x1 + x2 syntax); simpler API for standard GLMs
  • numpyro (planned) -- JAX-based probabilistic programming; faster NUTS via hardware acceleration
  • statsmodels-statistical-modeling -- frequentist regression and GLMs; use when Bayesian framework is not needed
  • scikit-learn-machine-learning -- ML classification/regression; use when prediction accuracy matters more than uncertainty quantification

References

scikit-survival -- Survival Analysis

Overview

scikit-survival is a Python library for time-to-event analysis built on scikit-learn. It handles right-censored data (observations where the event has not yet occurred) using Cox models, ensemble methods, survival SVMs, and non-parametric estimators. All models follow the scikit-learn fit/predict API and integrate with Pipelines, cross-validation, and GridSearchCV.

When to Use

  • Modeling time-to-event outcomes with right-censored data (clinical trials, reliability)
  • Fitting Cox proportional hazards models (standard or elastic net penalized)
  • Building ensemble survival models (Random Survival Forest, Gradient Boosting)
  • Training survival SVMs for margin-based learning on medium-sized datasets
  • Evaluating survival predictions with censoring-aware metrics (C-index, Brier score, AUC)
  • Estimating non-parametric survival curves (Kaplan-Meier, Nelson-Aalen)
  • Analyzing competing risks with cumulative incidence functions
  • High-dimensional survival data with automatic feature selection (CoxNet L1/L2)
  • For simpler parametric models (Weibull, log-normal AFT) or statistical tests (log-rank), use lifelines
  • For deep learning survival models, use pycox or torchlife

Prerequisites

pip install scikit-survival scikit-learn pandas numpy matplotlib

Python: >= 3.9. Dependencies: scikit-learn, numpy, scipy, pandas, joblib, osqp (for some SVM solvers).

Data format: Survival outcomes are NumPy structured arrays with (event, time) fields. Events are boolean (True = event occurred, False = censored). Times are positive floats.

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: timeAndEventColumns
    kind: required
    source: data
    ask: "Which column holds follow-up time, which holds the event indicator, and does the indicator mark the event or censoring?"
    default: null

  - id: D2
    param: eventDefinition
    kind: required
    source: user
    depends_on: [D1]
    ask: "Which event is being modelled, and are competing events treated as censoring?"
    default: null

  - id: D3
    param: covariates
    kind: required
    source: data
    ask: "Which variables enter the model, and are any measured after baseline?"
    default: null

  - id: D4
    param: modelFamily
    kind: required
    source: user
    ask: "A proportional-hazards model whose coefficients are interpretable, or an ensemble that predicts better but does not yield hazard ratios?"
    default: "Cox proportional hazards"

  - id: D5
    param: regularization
    kind: required
    source: user
    depends_on: [D3, D4]
    ask: "With more covariates than events, should coefficients be penalized - and toward selection or toward shrinkage?"
    default: "none"

  - id: D6
    param: validationScheme
    kind: required
    source: user
    ask: "How should performance be estimated - cross-validation, a held-out split, or an external cohort?"
    default: "cross-validation"

  - id: D7
    param: ensembleHyperparameters
    kind: optional_conditional
    source: user
    depends_on: [D4]
    ask: "How many trees, how deep, and at what learning rate?"
    default: "100 estimators, depth 3, learning rate 0.1"
    skip_if: "a proportional-hazards model was chosen"

D1 and D2 are asked before anything else because a flipped event indicator fits a model of the time to not having the event, and every hazard ratio comes back inverted without any warning. D3 carries the other silent error: a covariate measured after baseline leaks the outcome into the predictors and produces an impressive concordance index that will not replicate.

Quick Start

from sksurv.datasets import load_breast_cancer
from sksurv.ensemble import RandomSurvivalForest
from sksurv.metrics import concordance_index_ipcw
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

rsf = RandomSurvivalForest(n_estimators=100, random_state=42)
rsf.fit(X_train, y_train)

risk_scores = rsf.predict(X_test)
c_index = concordance_index_ipcw(y_train, y_test, risk_scores)[0]
print(f"C-index: {c_index:.3f}")  # e.g., 0.68

# Individual survival curves
surv_fns = rsf.predict_survival_function(X_test[:2])
for fn in surv_fns:
    print(f"5-year survival: {fn(365 * 5):.3f}")

Core API

Module 1: Data Preparation

Create structured survival arrays and preprocess features.

import numpy as np
import pandas as pd
from sksurv.util import Surv
from sksurv.preprocessing import OneHotEncoder, encode_categorical
from sksurv.datasets import load_gbsg2, load_breast_cancer
from sklearn.preprocessing import StandardScaler

# Create survival outcome from arrays
event = np.array([True, False, True, True, False])
time = np.array([120.0, 365.0, 200.0, 90.0, 400.0])
y = Surv.from_arrays(event=event, time=time)
print(y.dtype)  # [('event', '?'), ('time', '<f8')]

# From DataFrame columns
# y = Surv.from_dataframe("event_col", "time_col", df)

# Load built-in datasets
# Available: load_gbsg2, load_breast_cancer, load_veterans_lung_cancer,
#            load_whas500, load_aids, load_flchain
X, y = load_gbsg2()
print(f"Shape: {X.shape}, Events: {y['event'].sum()}, "
      f"Censoring rate: {1 - y['event'].mean():.1%}")

# Encode categoricals (survival-aware one-hot)
X_encoded = encode_categorical(X)  # auto-detect and encode all categorical cols

# Standardize (critical for Cox and SVM models)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X_encoded)
from sksurv.io import loadarff

# Load ARFF format (Weka format)
data = loadarff("survival_data.arff")
X_arff, y_arff = data[0], data[1]  # DataFrame, structured array

Module 2: Cox Proportional Hazards

Semi-parametric model: h(t|x) = h_0(t) * exp(beta^T x). Interpretable coefficients as log hazard ratios.

from sksurv.linear_model import CoxPHSurvivalAnalysis, CoxnetSurvivalAnalysis, IPCRidge

# Standard Cox PH model
cox = CoxPHSurvivalAnalysis(alpha=0.0, ties="breslow")
cox.fit(X_train, y_train)
print(f"Coefficients: {cox.coef_}")  # log hazard ratios
# Hazard ratio interpretation: exp(coef) = HR for 1-unit increase
risk_scores = cox.predict(X_test)  # Higher = higher risk

# Survival function for individual patients
surv_funcs = cox.predict_survival_function(X_test[:3])
for fn in surv_funcs:
    print(f"5-year survival: {fn(365 * 5):.3f}")
# Penalized Cox (elastic net) -- for high-dimensional data (p > n)
coxnet = CoxnetSurvivalAnalysis(
    l1_ratio=0.9,           # 0=Ridge, 1=Lasso, between=Elastic Net
    alpha_min_ratio=0.01,   # smallest alpha / largest alpha ratio
    n_alphas=100,           # steps in regularization path
)
coxnet.fit(X_train, y_train)

# Feature selection: non-zero coefficients
selected = np.where(coxnet.coef_ != 0)[0]
print(f"Selected {len(selected)} / {X_train.shape[1]} features")

# IPCRidge: accelerated failure time model (predicts log survival time)
ipcridge = IPCRidge(alpha=1.0)
ipcridge.fit(X_train, y_train)
log_survival_time = ipcridge.predict(X_test)

Module 3: Ensemble Methods

Non-parametric tree-based models for complex non-linear relationships.

from sksurv.ensemble import (
    RandomSurvivalForest,
    GradientBoostingSurvivalAnalysis,
    ComponentwiseGradientBoostingSurvivalAnalysis,
    ExtraSurvivalTrees,
)

# Random Survival Forest -- robust, minimal tuning
rsf = RandomSurvivalForest(
    n_estimators=200, min_samples_split=10, min_samples_leaf=15,
    max_features="sqrt", random_state=42, n_jobs=-1,
)
rsf.fit(X_train, y_train)
risk = rsf.predict(X_test)

# Gradient Boosting -- best performance, needs tuning
gbs = GradientBoostingSurvivalAnalysis(
    loss="coxph",           # "coxph" or "ipcwls" (AFT)
    n_estimators=300, learning_rate=0.05, max_depth=3,
    subsample=0.8, dropout_rate=0.1, random_state=42,
)
gbs.fit(X_train, y_train)

# ComponentwiseGB -- linear model with automatic feature selection
cgbs = ComponentwiseGradientBoostingSurvivalAnalysis(
    n_estimators=100, learning_rate=0.1,
)
cgbs.fit(X_train, y_train)
print(f"Non-zero coefficients: {np.sum(cgbs.coef_ != 0)}")

# ExtraSurvivalTrees -- more regularized than RSF, faster training
est = ExtraSurvivalTrees(n_estimators=100, random_state=42)
est.fit(X_train, y_train)

# Survival curves from any ensemble model
surv_funcs = rsf.predict_survival_function(X_test[:1])
chf_funcs = rsf.predict_cumulative_hazard_function(X_test[:1])

Module 4: Survival SVMs

Margin-based learning for survival ranking. Always standardize features.

from sksurv.svm import FastSurvivalSVM, FastKernelSurvivalSVM, HingeLossSurvivalSVM

# Linear SVM -- fast, for linear relationships
lsvm = FastSurvivalSVM(alpha=1.0, rank_ratio=1.0, max_iter=100, random_state=42)
lsvm.fit(X_train_scaled, y_train)
risk = lsvm.predict(X_test_scaled)

# Kernel SVM -- for non-linear relationships (rbf, poly, sigmoid)
ksvm = FastKernelSurvivalSVM(
    alpha=1.0, kernel="rbf", gamma="scale",
    max_iter=50, random_state=42,
)
ksvm.fit(X_train_scaled, y_train)

# Hinge loss variant
hsvm = HingeLossSurvivalSVM(alpha=1.0, random_state=42)
hsvm.fit(X_train_scaled, y_train)
# NaiveSurvivalSVM also available but slower (O(n^3))
from sksurv.kernels import ClinicalKernelTransform

# Clinical kernel: combines clinical + molecular features
# Weighs clinical variables separately from high-dimensional molecular data
transform = ClinicalKernelTransform(fit_once=True)
transform.prepare(X_train)  # auto-detect clinical features
X_kern = transform.fit_transform(X_train)

Module 5: Non-Parametric Estimation

Estimate survival and hazard curves without model assumptions.

from sksurv.nonparametric import kaplan_meier_estimator, nelson_aalen_estimator
import matplotlib.pyplot as plt

# Kaplan-Meier survival curve
time_km, surv_prob = kaplan_meier_estimator(y["event"], y["time"])
plt.step(time_km, surv_prob, where="post")
plt.xlabel("Time (days)")
plt.ylabel("Survival probability")
plt.title("Kaplan-Meier Estimate")
plt.savefig("km_curve.png", dpi=150)

# With confidence intervals
time_km, surv_prob, conf_int = kaplan_meier_estimator(
    y["event"], y["time"], conf_type="log-log",
)

# Nelson-Aalen cumulative hazard
time_na, cum_hazard = nelson_aalen_estimator(y["event"], y["time"])

# Stratified KM by group
for group_name, mask in [("Treated", treated_mask), ("Control", control_mask)]:
    t, s = kaplan_meier_estimator(y[mask]["event"], y[mask]["time"])
    plt.step(t, s, where="post", label=group_name)
plt.legend()

Module 6: Evaluation Metrics

Censoring-aware metrics for discrimination and calibration.

from sksurv.metrics import (
    concordance_index_censored,
    concordance_index_ipcw,
    cumulative_dynamic_auc,
    integrated_brier_score,
    brier_score,
    as_concordance_index_ipcw_scorer,
    as_integrated_brier_score_scorer,
)
import numpy as np

risk_scores = model.predict(X_test)

# Harrell's C-index (simple, biased with high censoring)
c_harrell = concordance_index_censored(
    y_test["event"], y_test["time"], risk_scores
)[0]

# Uno's C-index (recommended -- robust to censoring)
c_uno = concordance_index_ipcw(y_train, y_test, risk_scores)[0]
print(f"C-index (Harrell): {c_harrell:.3f}, (Uno): {c_uno:.3f}")

# Time-dependent AUC at clinically relevant timepoints
times = np.array([365, 730, 1095])  # 1, 2, 3 years
auc, mean_auc = cumulative_dynamic_auc(y_train, y_test, risk_scores, times)
print(f"AUC at 1/2/3yr: {auc}, Mean: {mean_auc:.3f}")

# Integrated Brier Score (measures discrimination + calibration)
surv_funcs = model.predict_survival_function(X_test)
preds = np.row_stack([fn(times) for fn in surv_funcs])
ibs = integrated_brier_score(y_train, y_test, preds, times)
print(f"IBS: {ibs:.3f}")  # Lower is better; compare vs KM baseline

Module 7: Competing Risks

Multiple mutually exclusive event types (e.g., death from cancer vs cardiovascular).

from sksurv.nonparametric import cumulative_incidence_competing_risks
from sksurv.linear_model import CoxPHSurvivalAnalysis
from sksurv.util import Surv
import numpy as np

# Cumulative Incidence Function (CIF) estimation
# y_competing: structured array with integer event codes
# (0=censored, 1=relapse, 2=death_in_remission)
time_pts, cif_relapse, cif_death = cumulative_incidence_competing_risks(y_competing)

import matplotlib.pyplot as plt
plt.step(time_pts, cif_relapse, where="post", label="Relapse")
plt.step(time_pts, cif_death, where="post", label="Death in remission")
plt.xlabel("Time")
plt.ylabel("Cumulative Incidence")
plt.legend()

# Cause-specific hazard models: fit separate Cox per event type
event_types = np.array([0, 1, 2, 1, 0, 2, 1])
times = np.array([10.2, 5.3, 8.1, 3.7, 12.5, 6.8, 4.2])

# Model for relapse (event type 1); treat type 2 as censored
y_relapse = Surv.from_arrays(event=(event_types == 1), time=times)
cox_relapse = CoxPHSurvivalAnalysis()
cox_relapse.fit(X, y_relapse)

# Model for death (event type 2); treat type 1 as censored
y_death = Surv.from_arrays(event=(event_types == 2), time=times)
cox_death = CoxPHSurvivalAnalysis()
cox_death.fit(X, y_death)
print(f"Relapse HR (age): {np.exp(cox_relapse.coef_[0]):.3f}")
print(f"Death HR (age): {np.exp(cox_death.coef_[0]):.3f}")

Key Concepts

Model Selection Guide

High-dimensional (p > n)?
├── Yes → CoxnetSurvivalAnalysis (elastic net)
└── No
    ├── Need interpretable coefficients?
    │   ├── Yes → CoxPHSurvivalAnalysis or ComponentwiseGB
    │   └── No
    │       ├── Large data (n > 1000) → GradientBoostingSurvivalAnalysis
    │       ├── Medium data → RandomSurvivalForest or FastKernelSurvivalSVM
    │       └── Small data → RandomSurvivalForest
    └── For max performance → compare multiple models

Censoring Types

scikit-survival primarily handles right-censoring (event not observed by end of study). Left-censoring and interval-censoring require other tools (e.g., lifelines). Censoring rate affects metric choice: >40% censoring favors Uno's C-index over Harrell's.

Concordance Index Interpretation

  • 0.5 = random prediction (no discrimination)
  • 0.6-0.7 = moderate discrimination
  • 0.7-0.8 = good discrimination (typical for clinical models)
  • 0.8+ = excellent (rare for clinical survival data)
  • C-index measures ranking ability, not calibration. Always pair with Brier score.

Evaluation Metric Decision Table

MetricMeasuresBest For
Harrell's C-indexRanking onlyQuick development, low censoring
Uno's C-indexRanking (censoring-adjusted)Publication, any censoring rate
Time-dependent AUCDiscrimination at fixed tClinical decision at specific horizons
Brier Score (single t)Calibration at fixed tProbability accuracy at timepoint
Integrated Brier ScoreOverall calibrationComprehensive model assessment

Common Workflows

Workflow 1: Standard Survival Pipeline

Goal: End-to-end analysis with preprocessing, model fitting, and evaluation.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import GridSearchCV, train_test_split
from sksurv.datasets import load_breast_cancer
from sksurv.ensemble import GradientBoostingSurvivalAnalysis
from sksurv.metrics import (
    concordance_index_ipcw, cumulative_dynamic_auc,
    integrated_brier_score, as_concordance_index_ipcw_scorer,
)
import numpy as np

X, y = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Build pipeline
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("model", GradientBoostingSurvivalAnalysis(random_state=42)),
])

# Hyperparameter search
param_grid = {
    "model__learning_rate": [0.01, 0.05, 0.1],
    "model__n_estimators": [100, 200],
    "model__max_depth": [3, 5],
}
cv = GridSearchCV(pipe, param_grid, cv=5,
                  scoring=as_concordance_index_ipcw_scorer(), n_jobs=-1)
cv.fit(X_train, y_train)
print(f"Best CV C-index: {cv.best_score_:.3f}, Params: {cv.best_params_}")

# Final evaluation on test set
best = cv.best_estimator_
risk = best.predict(X_test)
c_uno = concordance_index_ipcw(y_train, y_test, risk)[0]
times = np.percentile(y_test["time"][y_test["event"]], [25, 50, 75])
auc, mean_auc = cumulative_dynamic_auc(y_train, y_test, risk, times)
print(f"Test C-index: {c_uno:.3f}, Mean AUC: {mean_auc:.3f}")

Workflow 2: Model Comparison

Goal: Compare multiple model families and select the best.

from sksurv.linear_model import CoxPHSurvivalAnalysis
from sksurv.ensemble import RandomSurvivalForest, GradientBoostingSurvivalAnalysis
from sksurv.svm import FastSurvivalSVM
from sksurv.metrics import concordance_index_ipcw, integrated_brier_score
from sklearn.preprocessing import StandardScaler
import numpy as np

scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

models = {
    "CoxPH": CoxPHSurvivalAnalysis(),
    "RSF": RandomSurvivalForest(n_estimators=200, random_state=42),
    "GBS": GradientBoostingSurvivalAnalysis(n_estimators=200, random_state=42),
    "LinearSVM": FastSurvivalSVM(alpha=1.0, random_state=42),
}

times = np.percentile(y_test["time"][y_test["event"]], [25, 50, 75])
for name, model in models.items():
    X_tr = X_train_s if name in ("CoxPH", "LinearSVM") else X_train
    X_te = X_test_s if name in ("CoxPH", "LinearSVM") else X_test
    model.fit(X_tr, y_train)
    risk = model.predict(X_te)
    c = concordance_index_ipcw(y_train, y_test, risk)[0]
    print(f"{name:12s}: C-index = {c:.3f}")

Workflow 3: High-Dimensional Feature Selection

  1. Fit CoxnetSurvivalAnalysis with l1_ratio=0.9 (Lasso-dominant)
  2. Use GridSearchCV over alpha_min_ratio values with C-index scorer
  3. Extract non-zero coefficients from best model
  4. Refit standard CoxPH or RSF on selected features for final model
  5. Evaluate on held-out test set with Uno's C-index

Key Parameters

ParameterModuleDefaultRangeEffect
alphaCoxPH0.0>= 0Regularization (0 = none)
l1_ratioCoxNet0.50-1L1 vs L2 penalty (1=Lasso, 0=Ridge)
n_estimatorsRSF, GBS10050-1000Number of trees/iterations
max_depthRSF, GBSNone/31-20Tree depth (GBS: 3-5 recommended)
learning_rateGBS0.10.001-0.3Step size shrinkage per iteration
subsampleGBS1.00.5-1.0Fraction of samples per tree
dropout_rateGBS0.00-0.3Drop previous learners during training
max_featuresRSFNone"sqrt","log2",intFeatures per split
alphaFastSurvivalSVM1.00.01-100SVM regularization
kernelFastKernelSVM"rbf""linear","rbf","poly","sigmoid"Kernel function
gammaFastKernelSVM"scale""scale","auto",floatKernel coefficient

Best Practices

  1. Always standardize features for Cox and SVM models: Coefficients and margins are scale-dependent. Use StandardScaler inside a Pipeline to prevent leakage.

  2. Use Uno's C-index over Harrell's: More robust to censoring distribution differences; required for comparing models across datasets. Pass y_train as the first argument.

  3. Anti-pattern -- using built-in RSF feature importance: It is biased toward continuous and high-cardinality features. Use sklearn.inspection.permutation_importance() instead.

  4. Report multiple metrics: C-index alone measures only discrimination. Add Brier score for calibration and time-dependent AUC for time-specific performance.

  5. Anti-pattern -- ignoring proportional hazards assumption: Cox models assume constant hazard ratios over time. Validate with Schoenfeld residuals (available in lifelines) or use non-parametric alternatives (RSF, GBS).

  6. Ensure sufficient events per feature: Rule of thumb: 10+ events per feature for Cox. With fewer events, use CoxNet (l1_ratio=0.9) or ensemble methods.

  7. Use scikit-learn Pipelines: Wrapping preprocessing + model prevents data leakage and simplifies cross-validation and deployment.

  8. Anti-pattern -- treating competing events as censored for KM: Kaplan-Meier overestimates event probability when competing risks exist. Use cumulative_incidence_competing_risks() instead.

  9. GBS early stopping: For GradientBoosting, set n_estimators=1000 with n_iter_no_change=10 to automatically stop when validation performance plateaus.

Common Recipes

Recipe: Hyperparameter Tuning with Cross-Validation

from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sksurv.ensemble import RandomSurvivalForest
from sksurv.metrics import as_concordance_index_ipcw_scorer

pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("model", RandomSurvivalForest(random_state=42)),
])
param_grid = {
    "model__n_estimators": [100, 300, 500],
    "model__min_samples_split": [6, 10, 20],
    "model__max_depth": [None, 10, 20],
}
cv = GridSearchCV(pipe, param_grid, cv=5,
                  scoring=as_concordance_index_ipcw_scorer(), n_jobs=-1)
cv.fit(X, y)
print(f"Best: {cv.best_score_:.3f}, Params: {cv.best_params_}")

Recipe: Permutation Feature Importance

from sklearn.inspection import permutation_importance
from sksurv.metrics import as_concordance_index_ipcw_scorer

result = permutation_importance(
    rsf, X_test, y_test, n_repeats=15,
    scoring=as_concordance_index_ipcw_scorer(), random_state=42, n_jobs=-1,
)
sorted_idx = result.importances_mean.argsort()[::-1]
for i in sorted_idx[:10]:
    print(f"{X.columns[i]:20s}: {result.importances_mean[i]:.4f} "
          f"+/- {result.importances_std[i]:.4f}")

Recipe: Kaplan-Meier with Confidence Bands

from sksurv.nonparametric import kaplan_meier_estimator
import matplotlib.pyplot as plt

time, surv, conf_int = kaplan_meier_estimator(
    y["event"], y["time"], conf_type="log-log",
)
plt.step(time, surv, where="post", label="KM estimate")
plt.fill_between(time, conf_int[0], conf_int[1], alpha=0.25, step="post")
plt.xlabel("Time")
plt.ylabel("Survival probability")
plt.legend()
plt.savefig("km_with_ci.png", dpi=150)

Troubleshooting

ProblemCauseSolution
ValueError: y must be a structured arrayWrong outcome formatUse Surv.from_arrays(event, time) -- event must be bool, time must be float
concordance_index_ipcw raises errorMissing y_train argumentIPCW requires training set for censoring distribution; pass y_train as first arg
Cox model won't convergeCorrelated or unstandardized featuresStandardize with StandardScaler; switch to CoxnetSurvivalAnalysis
Very low C-index (< 0.5)Wrong sign or non-predictive featuresVerify event/time coding; check feature engineering
as_concordance_index_ipcw_scorer error in CVTime range mismatch between foldsEnsure all folds contain events at evaluated timepoints
SVM extremely slowLarge dataset with kernel SVMUse FastSurvivalSVM (linear, O(np)) for large n; subsample for kernel
RSF feature importance misleadingBuilt-in impurity importance biasedUse permutation_importance() from scikit-learn
Kaplan-Meier overestimatesCompeting events treated as censoringUse cumulative_incidence_competing_risks() instead

Bundled Resources

references/models_evaluation.md

Covers: Detailed Cox model family (CoxPH, CoxNet, IPCRidge with parameter guidance and interpretation), ensemble methods (RSF, GBS, ComponentwiseGB, ExtraSurvivalTrees with hyperparameter tuning, early stopping, loss functions), SVM models (FastSurvivalSVM, FastKernelSurvivalSVM, HingeLossSurvivalSVM, NaiveSurvivalSVM, MinlipSurvivalAnalysis with kernel selection), evaluation metrics (Harrell vs Uno C-index selection, time-dependent AUC plotting, Brier score with null model comparison, comprehensive evaluation pipeline), model comparison tables.

Relocated inline: Core usage patterns for all model families consolidated into Core API Modules 2-4 and Module 6. Model selection decision tree relocated to Key Concepts.

Omitted: CoxNet cross-validation alpha selection via scoring string (not a valid sklearn scorer usage -- use as_concordance_index_ipcw_scorer() as shown in Core API). NaiveSurvivalSVM detailed usage (slow, primarily for benchmarking). MinlipSurvivalAnalysis detailed usage (research-only). Redundant "when to use" prose duplicated across original reference files.

Consolidates 4 originals: cox-models.md (182 lines), ensemble-models.md (327 lines), svm-models.md (411 lines), evaluation-metrics.md (378 lines). Total: 1,298 lines.

references/data_competing_risks.md

Covers: Complete data preprocessing workflows (Surv arrays, loading custom CSV, ARFF I/O, categorical encoding methods, standardization, missing data imputation strategies, feature selection), data validation checks (negative times, censoring rate, events-per-feature), train-test splitting (random and stratified by event/time), complete preprocessing pipeline with sklearn, competing risks analysis (CIF estimation, data format with event types, stratified group comparison, cause-specific hazard modeling, complete practical example with simulated data).

Relocated inline: Core Surv array creation, encode_categorical, StandardScaler usage consolidated into Core API Module 1. CIF estimation and cause-specific Cox models relocated to Core API Module 7.

Omitted: Custom preprocessing function with docstring (duplicates sklearn Pipeline pattern shown in Workflow 1). Time-varying covariates note (scikit-survival does not support them -- mentioned only as a 1-line note in original). Fine-Gray sub-distribution model (not available in scikit-survival; noted as lifelines alternative).

Consolidates 2 originals: data-handling.md (494 lines), competing-risks.md (397 lines). Total: 891 lines.

Related Skills

  • scikit-learn-machine-learning -- general ML with same Pipeline/GridSearchCV API; non-survival tasks
  • statsmodels-statistical-modeling -- frequentist regression (OLS, GLM); non-censored outcomes
  • pymc-bayesian-modeling -- Bayesian survival models with prior specification
  • matplotlib-scientific-plotting -- customize Kaplan-Meier curves and survival plots
  • statistical-analysis -- test selection framework and reporting guidelines

References

Statistical Analysis

Overview

Statistical analysis is the systematic process of selecting appropriate tests, verifying assumptions, quantifying effect magnitudes, and reporting results. This knowhow guides test selection, assumption diagnostics, and APA-style reporting for frequentist and Bayesian analyses in academic research.

Key Concepts

Frequentist vs Bayesian Framework

AspectFrequentistBayesian
Core outputp-value, confidence intervalPosterior distribution, credible interval
Interpretation"How likely is this data if H0 is true?""How likely is H1 given the data?"
Null supportCannot support H0 (only fail to reject)Can quantify evidence for H0 via Bayes Factor
Prior infoNot usedIncorporated via prior distributions
Sample sizeRequires adequate powerWorks with any sample size
Best forStandard analyses, large samplesSmall samples, prior info, complex models

Statistical vs Practical Significance

A statistically significant result (p < .05) may be trivially small in practice. Always report:

  • Effect size: Magnitude of the effect (Cohen's d, eta-squared, r, R-squared)
  • Confidence interval: Precision of the estimate
  • Context: Clinical/practical relevance in the domain

Common Effect Sizes

TestEffect SizeSmallMediumLarge
t-testCohen's d0.200.500.80
t-test (small n)Hedges' g0.200.500.80
ANOVAeta-squared partial0.010.060.14
ANOVAomega-squared0.010.060.14
Correlationr0.100.300.50
RegressionR-squared0.020.130.26
Regressionf-squared0.020.150.35
Chi-squareCramer's V0.070.210.35
Chi-square 2x2phi coefficient0.100.300.50

Cohen's benchmarks are guidelines, not rigid thresholds -- domain context always matters.

Assumptions Overview

Most parametric tests require:

  1. Independence: Observations are independent of each other
  2. Normality: Data (or residuals) are approximately normally distributed
  3. Homogeneity of variance: Groups have similar variances (for group comparisons)
  4. Linearity: Relationship between variables is linear (for regression)

When assumptions are violated:

  • Normality violated, n > 30: Proceed -- parametric tests are robust with large samples
  • Normality violated, n < 30: Use non-parametric alternative
  • Variance heterogeneity: Use Welch's correction (t-test) or Welch's ANOVA
  • Linearity violated: Add polynomial terms, transform variables, or use GAMs

Test-Specific Assumption Workflows

T-test assumptions: (1) Check normality per group with Shapiro-Wilk + Q-Q plots. (2) Check homogeneity with Levene's test. (3) If normality violated: Mann-Whitney U (independent) or Wilcoxon signed-rank (paired). If variance heterogeneity: use Welch's t-test.

ANOVA assumptions: (1) Normality per group. (2) Homogeneity via Levene's test. (3) For repeated measures: check sphericity (Mauchly's test); if violated, apply Greenhouse-Geisser (epsilon < 0.75) or Huynh-Feldt (epsilon > 0.75) correction. (4) If normality violated: Kruskal-Wallis (independent) or Friedman (repeated).

Linear regression assumptions: (1) Linearity via residuals-vs-fitted plot. (2) Independence via Durbin-Watson test (1.5-2.5 acceptable). (3) Homoscedasticity via Breusch-Pagan test + scale-location plot. (4) Normality of residuals via Q-Q plot + Shapiro-Wilk. (5) Multicollinearity via VIF (>10 = severe, >5 = moderate).

Logistic regression assumptions: (1) Independence. (2) Linearity of log-odds with continuous predictors (Box-Tidwell test). (3) No perfect multicollinearity (VIF). (4) Adequate sample size (10-20 events per predictor minimum).

Specialized Test Categories

Beyond the main decision flowchart, several specialized test families address specific data types:

Survival / time-to-event analysis:

  • Log-rank test: Compares survival curves between groups (non-parametric)
  • Cox proportional hazards: Models time-to-event with covariates; assumes proportional hazards
  • Parametric survival models: Weibull, exponential, log-normal for known distributional forms
  • Use when outcome is time until an event (death, relapse, failure) with possible censoring

Count outcome models:

  • Poisson regression: For count data where mean approximately equals variance
  • Negative binomial regression: For overdispersed counts (variance > mean)
  • Zero-inflated models: For excess zeros beyond what Poisson/NB predicts
  • Use when outcome is a count (number of events, incidents, occurrences)

Agreement and reliability:

  • Cohen's kappa: Inter-rater agreement for categorical ratings (2 raters)
  • Fleiss' kappa / Krippendorff's alpha: Agreement for >2 raters
  • Intraclass correlation coefficient (ICC): Continuous ratings reliability
  • Cronbach's alpha: Internal consistency of multi-item scales
  • Bland-Altman analysis: Agreement between two measurement methods (continuous)
  • Use when assessing measurement reliability or inter-rater consistency

Categorical data extensions:

  • McNemar's test: Paired binary outcomes (2x2)
  • Cochran's Q test: Paired binary outcomes (3+ conditions)
  • Cochran-Armitage trend test: Ordered categories in contingency tables

Decision Framework

Test Selection Flowchart

What is your research question?
|
+-- Comparing GROUPS on a continuous outcome?
|   |
|   +-- How many groups?
|   |   +-- 2 groups
|   |   |   +-- Independent -> Independent t-test (or Mann-Whitney U)
|   |   |   +-- Paired/repeated -> Paired t-test (or Wilcoxon signed-rank)
|   |   +-- 3+ groups
|   |      +-- Independent -> One-way ANOVA (or Kruskal-Wallis)
|   |      +-- Repeated -> Repeated-measures ANOVA (or Friedman)
|   |
|   +-- Multiple factors? -> Factorial ANOVA / Mixed ANOVA
|   +-- With covariates? -> ANCOVA
|
+-- Testing a RELATIONSHIP between variables?
|   |
|   +-- Both continuous?
|   |   +-- Normal -> Pearson correlation
|   |   +-- Non-normal or ordinal -> Spearman correlation
|   |
|   +-- Predicting continuous outcome?
|   |   +-- 1 predictor -> Simple linear regression
|   |   +-- Multiple predictors -> Multiple linear regression
|   |
|   +-- Predicting categorical outcome?
|   |   +-- Binary -> Logistic regression
|   |   +-- Ordinal -> Ordinal logistic regression
|   |
|   +-- Predicting count outcome?
|   |   +-- Equidispersed -> Poisson regression
|   |   +-- Overdispersed -> Negative binomial regression
|   |   +-- Excess zeros -> Zero-inflated Poisson/NB
|   |
|   +-- Time-to-event outcome?
|       +-- Compare survival curves -> Log-rank test
|       +-- With covariates -> Cox proportional hazards
|
+-- Testing ASSOCIATION between categorical variables?
|   +-- Expected cell count >= 5 -> Chi-square test
|   +-- Expected cell count < 5 -> Fisher's exact test
|   +-- Ordered categories -> Cochran-Armitage trend test
|   +-- Paired categories -> McNemar's test
|
+-- Assessing AGREEMENT / RELIABILITY?
    +-- Categorical, 2 raters -> Cohen's kappa
    +-- Categorical, >2 raters -> Fleiss' kappa
    +-- Continuous ratings -> ICC
    +-- Two measurement methods -> Bland-Altman analysis
    +-- Internal consistency -> Cronbach's alpha

Quick Reference Table

Research QuestionData TypeNormal?TestNon-parametric Alternative
2 independent groupsContinuousYesIndependent t-testMann-Whitney U
2 paired groupsContinuousYesPaired t-testWilcoxon signed-rank
3+ independent groupsContinuousYesOne-way ANOVAKruskal-Wallis
3+ repeated groupsContinuousYesRepeated-measures ANOVAFriedman test
2 variablesContinuousYesPearson rSpearman rho
Predict continuousMixed--Linear regression--
Predict binaryMixed--Logistic regression--
Predict countsCount--Poisson / Negative binomial--
Time-to-eventSurvival--Cox PH / Log-rank--
2 categoricalCategorical--Chi-square / Fisher's exact--
Rater agreementCategorical--Cohen's kappa / Fleiss' kappa--
Method agreementContinuous--Bland-Altman / ICC--

Best Practices

  1. Pre-register analyses when possible to distinguish confirmatory from exploratory findings. Specify primary outcome, tests, and correction methods before data collection
  2. Always check assumptions before interpreting results. Run normality tests (Shapiro-Wilk), homogeneity tests (Levene's), and residual diagnostics. Document results even when assumptions are met
  3. Report effect sizes with confidence intervals for every test. p-values alone are insufficient -- effect sizes convey practical importance
  4. Report all planned analyses including non-significant findings. Selective reporting inflates false positive rates
  5. Use appropriate multiple comparison corrections. Bonferroni (conservative), Holm (step-down, less conservative), or FDR/Benjamini-Hochberg (for many tests). Choose based on the number of comparisons and acceptable error rate
  6. Visualize data before and after analysis. Box plots for group comparisons, scatter plots for correlations, residual plots for regression diagnostics
  7. Conduct sensitivity analyses to assess robustness: re-run with outliers removed, different transformations, or alternative tests
  8. Anti-pattern -- p-hacking: Testing multiple outcomes, subgroups, or model specifications until p < .05 inflates false positives. Pre-register to avoid
  9. Anti-pattern -- HARKing (Hypothesizing After Results are Known): Presenting exploratory findings as confirmatory undermines scientific integrity
  10. Anti-pattern -- misinterpreting non-significance: Failure to reject H0 does not mean H0 is true. Use Bayesian methods or equivalence testing to support null

Common Pitfalls

  1. Misinterpreting p-values as probability of the hypothesis being true. p-values measure P(data | H0), not P(H0 | data). How to avoid: Use precise language: "If the null hypothesis were true, the probability of observing data this extreme is p = ..."

  2. Confusing statistical significance with practical importance. A large sample can make trivially small effects significant. How to avoid: Always report and interpret effect sizes alongside p-values

  3. Running post-hoc power analysis after a non-significant result. Post-hoc power is a mathematical function of the p-value and adds no new information. How to avoid: Use sensitivity analysis instead -- determine what effect size the study could detect at 80% power

  4. Ignoring assumption violations and proceeding with parametric tests. How to avoid: Run assumption checks systematically. Use Welch's corrections, non-parametric alternatives, or transformations when violated

  5. Multiple comparisons without correction. Running 20 tests at alpha = .05 gives ~64% chance of at least one false positive. How to avoid: Apply Bonferroni, Holm, or FDR correction. Report both corrected and uncorrected p-values

  6. Treating ordinal data as continuous. Likert scales are ordinal -- means and standard deviations assume equal intervals. How to avoid: Use non-parametric tests (Mann-Whitney, Kruskal-Wallis) or ordinal regression

  7. Ignoring missing data patterns. Listwise deletion assumes MCAR, which is rarely true. How to avoid: Assess missingness mechanism (MCAR, MAR, MNAR). Use multiple imputation for MAR data

  8. Confusing correlation with causation. Observational studies cannot establish causal relationships regardless of effect size. How to avoid: Use causal language only for experimental designs with random assignment

  9. Not reporting non-significant results. Publication bias and file-drawer effect distort the literature. How to avoid: Report all pre-registered analyses. Consider registered reports

  10. Using one-tailed tests to "improve" significance. One-tailed tests should be pre-specified based on strong directional hypotheses. How to avoid: Default to two-tailed. Only use one-tailed when justified a priori

Workflow

Standard Analysis Pipeline

  1. Define research question and hypotheses

    • State H0 and H1 explicitly
    • Specify primary outcome and covariates
  2. Select statistical test (use Decision Framework above)

    • Match test to data type, design, and assumptions
    • Plan multiple comparison corrections if needed
  3. Conduct a priori power analysis

    • Specify target effect size (from literature or clinical relevance)
    • Set alpha = .05, power = .80 (minimum), determine required n
    • Libraries: statsmodels.stats.power, pingouin
  4. Inspect and clean data

    • Check for missing data patterns (MCAR/MAR/MNAR)
    • Identify outliers (IQR method: Q1 - 1.5IQR / Q3 + 1.5IQR, or z-scores > 3)
    • Verify variable types and coding
    • The original assumption_checks.py script provides automated normality, homogeneity, and outlier detection with visualization
  5. Check assumptions (see Test-Specific Assumption Workflows above)

    • Normality: Shapiro-Wilk test + Q-Q plots (visual primary for n > 50)
    • Homogeneity: Levene's test + box plots
    • Linearity: Residual plots (for regression)
    • Sphericity: Mauchly's test (for repeated measures)
    • Document results and remedial actions
  6. Run primary analysis

    • Execute planned test with appropriate library (scipy.stats, pingouin, statsmodels)
    • Calculate effect size and confidence interval
    • For Bayesian analyses: specify priors, run MCMC, check convergence (see references/bayesian_statistics.md)
  7. Conduct post-hoc and secondary analyses

    • Post-hoc pairwise comparisons (Tukey HSD, Bonferroni)
    • Sensitivity analyses (remove outliers, alternative methods)
    • Exploratory analyses (clearly labeled)
  8. Report results in APA format

    • Descriptive statistics (M, SD, n per group)
    • Test statistic, degrees of freedom, exact p-value
    • Effect size with confidence interval
    • See references/reporting_standards.md for templates

Bundled Resources

  • references/effect_sizes_and_power.md -- Detailed guide to calculating, interpreting, and reporting effect sizes (Cohen's d, Hedges' g, Glass's delta, eta-squared, omega-squared, partial eta-squared, phi coefficient, standardized beta, f-squared, Cramer's V, odds ratio); a priori, sensitivity, and correlation power analysis with code examples. Condensed from 582-line original.

  • references/bayesian_statistics.md -- Comprehensive Bayesian analysis guide: Bayes' theorem, prior specification, ROPE (Region of Practical Equivalence), prior sensitivity analysis, Bayesian t-test/ANOVA/correlation/regression, hierarchical models, model comparison (WAIC/LOO), convergence diagnostics. Condensed from 662-line original.

  • references/reporting_standards.md -- APA-style reporting templates for t-tests, ANOVA, regression, correlation, chi-square, non-parametric, and Bayesian analyses; pre-registration guidance; methods section templates (participants, design, measures); null results reporting; reporting checklist. Condensed from 470-line original.

Fully-Consolidated Files (no separate reference file)

  • test_selection_guide.md (130 lines original) -- Fully consolidated into Decision Framework (flowchart + Quick Reference Table) and Specialized Test Categories subsection in Key Concepts. Combined coverage: flowchart (~35 lines) + Quick Reference Table (~15 lines) + Specialized Test Categories (~35 lines) = ~85 lines covering all original capabilities. Original content on sample size considerations, multiple comparisons, and missing data was consolidated into Best Practices and Common Pitfalls. Omitted: study design considerations (RCTs, observational, clustered data) -- general guidance covered by statsmodels-statistical-modeling skill.

  • assumptions_and_diagnostics.md (370 lines original) -- Fully consolidated into Key Concepts (Assumptions Overview + Test-Specific Assumption Workflows) and Workflow Steps 4-5. Combined coverage: Assumptions Overview (~12 lines) + Test-Specific Assumption Workflows (~20 lines) + Workflow Steps 4-5 (~16 lines) = ~48 lines. The original contained detailed code blocks for each assumption check; since this is Knowhow (not Skill), code is referenced rather than reproduced. Key diagnostic thresholds preserved (VIF > 10, Durbin-Watson 1.5-2.5, variance ratio < 2-3). Omitted: extensive Python code blocks for individual checks (normality, homogeneity, linearity, logistic regression diagnostics) -- available in scipy.stats and pingouin documentation. Sample size rules of thumb covered in Workflow Step 3.

Script Disposition

  • assumption_checks.py (540 lines) -- Contains 6 functions: check_normality(), check_normality_per_group(), check_homogeneity_of_variance(), check_linearity(), detect_outliers(), comprehensive_assumption_check(). As Knowhow entry, script functions are referenced in Workflow Step 4 rather than reproduced inline. Key capabilities (Shapiro-Wilk, Levene's, IQR/z-score outlier detection, Q-Q plots) are described in Assumptions Overview and Test-Specific Assumption Workflows. Users needing automated checking should use scipy.stats and pingouin directly following the patterns described.

Intentional Omissions

  • Time series methods (ARIMA, ACF/PACF) -- specialized topic beyond core statistical testing scope
  • Mixed-effects models / GEE -- covered by statsmodels-statistical-modeling skill
  • Bootstrap and permutation tests -- mentioned in passing; detailed implementation deferred to computational statistics resources

Further Reading

  • Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.)
  • Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics (4th ed.)
  • Gelman, A., & Hill, J. (2006). Data Analysis Using Regression and Multilevel/Hierarchical Models
  • Kruschke, J. K. (2014). Doing Bayesian Data Analysis (2nd ed.)
  • APA Publication Manual: https://apastyle.apa.org/
  • Cross Validated (stats Q&A): https://stats.stackexchange.com/

Related Skills

  • statsmodels-statistical-modeling -- Implementing OLS, GLM, Logit, time-series models programmatically
  • pymc-bayesian-modeling -- Full Bayesian modeling with MCMC sampling
  • scikit-learn-machine-learning -- Predictive modeling, cross-validation, classification
  • matplotlib-scientific-plotting -- Creating publication-quality statistical figures
  • hypothesis-generation -- Structured hypothesis formulation before statistical testing

statsmodels

Overview

Statsmodels provides classical statistical modeling with rigorous inference for Python. It covers linear models, generalized linear models, discrete choice, time series, and comprehensive diagnostics. Unlike scikit-learn (prediction-focused), statsmodels emphasizes coefficient interpretation, p-values, confidence intervals, and model diagnostics.

When to Use

  • Fitting linear regression (OLS, WLS, GLS) with detailed coefficient tables and diagnostics
  • Running logistic regression with odds ratios and marginal effects for clinical/epidemiological studies
  • Analyzing count data with Poisson or negative binomial regression
  • Time series forecasting with ARIMA, SARIMAX, or exponential smoothing
  • Performing ANOVA, t-tests, or non-parametric tests with proper corrections
  • Testing model assumptions (heteroskedasticity, autocorrelation, normality of residuals)
  • Model comparison using AIC/BIC or likelihood ratio tests
  • Using R-style formula interface (y ~ x1 + x2 + C(group)) for intuitive model specification
  • For prediction-focused ML with cross-validation and hyperparameter tuning, use scikit-learn instead
  • For Bayesian modeling with posterior inference, use pymc instead

Prerequisites

  • Python packages: statsmodels, numpy, pandas, scipy
  • Optional: matplotlib (for diagnostic plots), patsy (for formula API, included with statsmodels)
  • Data: Tabular data as pandas DataFrames or NumPy arrays
pip install statsmodels numpy pandas matplotlib

Quick Start

import statsmodels.api as sm
import statsmodels.formula.api as smf
import pandas as pd
import numpy as np

# Generate sample data
np.random.seed(42)
n = 100
df = pd.DataFrame({
    "x1": np.random.randn(n),
    "x2": np.random.randn(n),
    "group": np.random.choice(["A", "B"], n)
})
df["y"] = 2 + 3 * df["x1"] - 1.5 * df["x2"] + np.random.randn(n)

# OLS with formula API (R-style)
results = smf.ols("y ~ x1 + x2 + C(group)", data=df).fit()
print(results.summary())
print(f"R²: {results.rsquared:.3f}, AIC: {results.aic:.1f}")

Core API

Module 1: Linear Regression (OLS, WLS, GLS)

Standard linear models with comprehensive diagnostics.

import statsmodels.api as sm
import numpy as np

# Generate data
np.random.seed(42)
X = np.random.randn(200, 3)
y = 1 + 2*X[:, 0] - 0.5*X[:, 1] + np.random.randn(200)

# ALWAYS add constant for intercept
X_const = sm.add_constant(X)
results = sm.OLS(y, X_const).fit()

print(results.summary())
print(f"\nCoefficients: {results.params}")
print(f"P-values: {results.pvalues}")
print(f"R²: {results.rsquared:.4f}")

# Predictions with confidence intervals
pred = results.get_prediction(X_const[:5])
print(pred.summary_frame())
# Robust standard errors (heteroskedasticity-consistent)
results_robust = sm.OLS(y, X_const).fit(cov_type="HC3")
print("Robust SEs:", results_robust.bse)

# Weighted Least Squares
weights = 1 / np.abs(results.resid + 0.1)  # Example weights
results_wls = sm.WLS(y, X_const, weights=weights).fit()
print(f"WLS R²: {results_wls.rsquared:.4f}")

Module 2: Generalized Linear Models (GLM)

Extend regression to non-normal outcomes (binary, count, continuous-positive).

import statsmodels.api as sm
import numpy as np

# Poisson regression for count data
np.random.seed(42)
X = np.random.randn(200, 2)
X_const = sm.add_constant(X)
y_counts = np.random.poisson(np.exp(0.5 + 0.3*X[:, 0]))

model = sm.GLM(y_counts, X_const, family=sm.families.Poisson())
results = model.fit()
print(results.summary())

# Rate ratios
rate_ratios = np.exp(results.params)
print(f"Rate ratios: {rate_ratios}")

# Check overdispersion
overdispersion = results.pearson_chi2 / results.df_resid
print(f"Overdispersion ratio: {overdispersion:.2f}")
if overdispersion > 1.5:
    print("→ Consider Negative Binomial model")

Module 3: Discrete Choice Models (Logit, Probit, Count)

Binary, multinomial, and count outcome models.

import statsmodels.api as sm
import numpy as np

# Logistic regression
np.random.seed(42)
X = np.random.randn(300, 2)
X_const = sm.add_constant(X)
prob = 1 / (1 + np.exp(-(0.5 + X[:, 0] - 0.5*X[:, 1])))
y_binary = np.random.binomial(1, prob)

logit_results = sm.Logit(y_binary, X_const).fit()
print(logit_results.summary())

# Odds ratios
odds_ratios = np.exp(logit_results.params)
print(f"Odds ratios: {odds_ratios}")

# Marginal effects (at means)
margeff = logit_results.get_margeff()
print(margeff.summary())

# Predicted probabilities
probs = logit_results.predict(X_const[:5])
print(f"Predicted P(Y=1): {probs}")

Module 4: Time Series (ARIMA, SARIMAX)

Univariate and multivariate time series modeling and forecasting.

import statsmodels.api as sm
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.stattools import adfuller
import numpy as np
import pandas as pd

# Generate time series
np.random.seed(42)
dates = pd.date_range("2020-01-01", periods=200, freq="D")
y = np.cumsum(np.random.randn(200)) + 50
ts = pd.Series(y, index=dates)

# Stationarity test
adf_result = adfuller(ts)
print(f"ADF statistic: {adf_result[0]:.4f}, p-value: {adf_result[1]:.4f}")
print("Stationary" if adf_result[1] < 0.05 else "Non-stationary → difference")

# Fit ARIMA
model = ARIMA(ts, order=(1, 1, 1))
results = model.fit()
print(results.summary())

# Forecast with confidence intervals
forecast = results.get_forecast(steps=30)
forecast_df = forecast.summary_frame()
print(f"30-day forecast:\n{forecast_df.head()}")
# Seasonal ARIMA (SARIMAX)
from statsmodels.tsa.statespace.sarimax import SARIMAX

# Monthly data with yearly seasonality
model_sarima = SARIMAX(ts, order=(1, 1, 1), seasonal_order=(1, 1, 1, 12))
results_sarima = model_sarima.fit(disp=False)
print(f"AIC: {results_sarima.aic:.1f}")

# Diagnostic plots
results_sarima.plot_diagnostics(figsize=(12, 8))

Module 5: Statistical Tests and Diagnostics

Assumption tests, hypothesis tests, and model validation.

import statsmodels.api as sm
from statsmodels.stats.diagnostic import het_breuschpagan, acorr_ljungbox
from statsmodels.stats.stattools import jarque_bera
import numpy as np

# Fit a model first
np.random.seed(42)
X = sm.add_constant(np.random.randn(200, 2))
y = 1 + 2*X[:, 1] + np.random.randn(200) * X[:, 1]  # Heteroskedastic
results = sm.OLS(y, X).fit()

# Heteroskedasticity test (Breusch-Pagan)
bp_stat, bp_p, _, _ = het_breuschpagan(results.resid, X)
print(f"Breusch-Pagan p-value: {bp_p:.4f} {'→ heteroskedastic' if bp_p < 0.05 else '→ OK'}")

# Normality test (Jarque-Bera)
jb_stat, jb_p, _, _ = jarque_bera(results.resid)
print(f"Jarque-Bera p-value: {jb_p:.4f} {'→ non-normal' if jb_p < 0.05 else '→ OK'}")

# Autocorrelation test (Ljung-Box)
lb_result = acorr_ljungbox(results.resid, lags=[10], return_df=True)
print(f"Ljung-Box p-value (lag 10): {lb_result['lb_pvalue'].values[0]:.4f}")
# Variance Inflation Factor (multicollinearity)
from statsmodels.stats.outliers_influence import variance_inflation_factor

vif_data = pd.DataFrame({
    "Variable": [f"x{i}" for i in range(X.shape[1])],
    "VIF": [variance_inflation_factor(X, i) for i in range(X.shape[1])]
})
print(vif_data)  # VIF > 10 suggests multicollinearity

Module 6: Formula API (R-style)

Intuitive model specification using formulas with automatic dummy coding.

import statsmodels.formula.api as smf
import pandas as pd
import numpy as np

np.random.seed(42)
df = pd.DataFrame({
    "y": np.random.randn(100),
    "x1": np.random.randn(100),
    "x2": np.random.randn(100),
    "group": np.random.choice(["A", "B", "C"], 100),
})

# Formula with categoricals (auto dummy-coded)
res = smf.ols("y ~ x1 + x2 + C(group)", data=df).fit()
print(res.summary())

# Interactions
res2 = smf.ols("y ~ x1 * x2", data=df).fit()  # x1 + x2 + x1:x2
print(f"Interaction term p-value: {res2.pvalues['x1:x2']:.4f}")

# Logit via formula
df["binary"] = (df["y"] > 0).astype(int)
logit_res = smf.logit("binary ~ x1 + x2 + C(group)", data=df).fit()
print(f"Logit AIC: {logit_res.aic:.1f}")

Common Workflows

Workflow 1: Complete Regression Analysis

Goal: Fit OLS, validate assumptions, use robust SEs if needed.

import statsmodels.api as sm
import statsmodels.formula.api as smf
from statsmodels.stats.diagnostic import het_breuschpagan
from statsmodels.stats.outliers_influence import variance_inflation_factor
import numpy as np
import pandas as pd

# 1. Fit initial model
np.random.seed(42)
df = pd.DataFrame({"y": np.random.randn(200), "x1": np.random.randn(200), "x2": np.random.randn(200)})
df["y"] = 2 + 3*df["x1"] - df["x2"] + np.random.randn(200)

results = smf.ols("y ~ x1 + x2", data=df).fit()

# 2. Check heteroskedasticity
bp_stat, bp_p, _, _ = het_breuschpagan(results.resid, results.model.exog)
print(f"Breusch-Pagan p: {bp_p:.4f}")

# 3. If heteroskedastic, use robust SEs
if bp_p < 0.05:
    results = smf.ols("y ~ x1 + x2", data=df).fit(cov_type="HC3")
    print("Using HC3 robust standard errors")

# 4. Check multicollinearity
X = results.model.exog
for i in range(1, X.shape[1]):  # skip constant
    print(f"VIF x{i}: {variance_inflation_factor(X, i):.2f}")

# 5. Final results
print(results.summary())
print(f"\nAIC: {results.aic:.1f}, BIC: {results.bic:.1f}")

Workflow 2: Model Comparison

Goal: Compare nested and non-nested models using appropriate criteria.

import statsmodels.formula.api as smf
from scipy import stats
import pandas as pd
import numpy as np

np.random.seed(42)
df = pd.DataFrame({"y": np.random.randn(200), "x1": np.random.randn(200),
                    "x2": np.random.randn(200), "x3": np.random.randn(200)})
df["y"] = 1 + 2*df["x1"] - df["x2"] + 0.1*df["x3"] + np.random.randn(200)

# Fit nested models
m1 = smf.ols("y ~ x1", data=df).fit()
m2 = smf.ols("y ~ x1 + x2", data=df).fit()
m3 = smf.ols("y ~ x1 + x2 + x3", data=df).fit()

# Compare via AIC/BIC (lower = better)
comparison = pd.DataFrame({
    "R²": [m.rsquared for m in [m1, m2, m3]],
    "AIC": [m.aic for m in [m1, m2, m3]],
    "BIC": [m.bic for m in [m1, m2, m3]],
}, index=["y~x1", "y~x1+x2", "y~x1+x2+x3"])
print(comparison)

# Likelihood ratio test (nested: m2 vs m3)
lr_stat = 2 * (m3.llf - m2.llf)
p_val = 1 - stats.chi2.cdf(lr_stat, df=m3.df_model - m2.df_model)
print(f"\nLR test (m3 vs m2): stat={lr_stat:.2f}, p={p_val:.4f}")

Workflow 3: Time Series Forecasting Pipeline

Goal: Test stationarity, identify model order, forecast.

from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.stattools import adfuller
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Generate data
np.random.seed(42)
ts = pd.Series(np.cumsum(np.random.randn(200)) + 100,
               index=pd.date_range("2020-01-01", periods=200, freq="D"))

# 1. Test stationarity
adf_p = adfuller(ts)[1]
print(f"ADF p-value: {adf_p:.4f} → {'stationary' if adf_p < 0.05 else 'non-stationary'}")

# 2. Identify order from ACF/PACF (on differenced series)
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 6))
plot_acf(ts.diff().dropna(), lags=20, ax=ax1)
plot_pacf(ts.diff().dropna(), lags=20, ax=ax2)
plt.savefig("acf_pacf.png", dpi=150, bbox_inches="tight")

# 3. Fit and forecast
model = ARIMA(ts[:180], order=(1, 1, 1))
results = model.fit()
forecast = results.get_forecast(steps=20)
fc_df = forecast.summary_frame()
print(f"ARIMA AIC: {results.aic:.1f}")
print(f"Forecast (first 5 days):\n{fc_df.head()}")

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
cov_typeOLS/WLS/GLM"nonrobust""HC0"-"HC3", "HAC", "cluster"Robust covariance estimator
familyGLMrequiredPoisson(), Binomial(), Gamma(), etc.Distribution family
orderARIMArequired(p, d, q) tupleAR order, differencing, MA order
seasonal_orderSARIMAX(0,0,0,0)(P, D, Q, s) tupleSeasonal ARIMA parameters
alphasummary(), conf_int()0.050.01-0.10Significance level for CIs
maxiterAll .fit()35-10050-1000Max optimization iterations
method.fit()model-dependent"newton", "bfgs", "lbfgs", "powell"Optimization algorithm
lagsACF/PACFNone10-50Number of lags to display

Best Practices

  1. Always add a constant for OLS/GLM: sm.add_constant(X) or use formula API (adds intercept automatically)

  2. Match model to outcome type: Binary → Logit/Probit, Counts → Poisson/NegBin, Continuous → OLS/WLS, Time series → ARIMA

  3. Check diagnostics before interpreting: Run Breusch-Pagan (heteroskedasticity), Jarque-Bera (normality), Ljung-Box (autocorrelation) on residuals

  4. Use robust SEs when assumptions fail: results = model.fit(cov_type="HC3") for heteroskedasticity-robust inference

  5. Report effect sizes, not just p-values: Include coefficients, confidence intervals, and R² alongside significance tests

  6. Prefer formula API for exploratory work: smf.ols("y ~ x1 * x2 + C(group)", data=df) is more readable and handles categoricals automatically

  7. Test stationarity before time series modeling: Use ADF test; difference if non-stationary

Common Recipes

Recipe: ANOVA with Post-hoc Tests

When to use: Comparing means across 3+ groups.

import statsmodels.formula.api as smf
from statsmodels.stats.multicomp import pairwise_tukeyhsd
import pandas as pd
import numpy as np

np.random.seed(42)
df = pd.DataFrame({"value": np.concatenate([np.random.normal(m, 1, 30) for m in [5, 6, 7]]),
                    "group": np.repeat(["A", "B", "C"], 30)})

# One-way ANOVA
anova = smf.ols("value ~ C(group)", data=df).fit()
print(sm.stats.anova_lm(anova))

# Post-hoc Tukey HSD
tukey = pairwise_tukeyhsd(df["value"], df["group"], alpha=0.05)
print(tukey)

Recipe: Power Analysis for Sample Size

When to use: Determining required sample size before a study.

from statsmodels.stats.power import TTestIndPower

analysis = TTestIndPower()
# What sample size for medium effect (d=0.5), 80% power, alpha=0.05?
n = analysis.solve_power(effect_size=0.5, alpha=0.05, power=0.8)
print(f"Required n per group: {n:.0f}")

# Power for given sample size
power = analysis.solve_power(effect_size=0.5, alpha=0.05, nobs1=50)
print(f"Power with n=50: {power:.3f}")

Recipe: Mixed Effects Model

When to use: Hierarchical/clustered data (patients within hospitals, students within schools).

import statsmodels.formula.api as smf
import pandas as pd
import numpy as np

np.random.seed(42)
df = pd.DataFrame({
    "y": np.random.randn(100), "x": np.random.randn(100),
    "group": np.repeat(range(10), 10)
})

# Random intercept model
model = smf.mixedlm("y ~ x", data=df, groups=df["group"])
results = model.fit()
print(results.summary())

Troubleshooting

ProblemCauseSolution
MissingDataErrorNaN values in dataDrop NAs: df.dropna() or impute before fitting
No intercept in resultsForgot sm.add_constant()Always add constant, or use smf.ols() formula API
ConvergenceWarningOptimization failedIncrease maxiter, try different method, or scale variables
Overdispersion in PoissonVariance > meanSwitch to NegativeBinomial or use GLM(family=NegativeBinomial())
Non-stationary time seriesTrend or unit rootDifference the series (ts.diff()) or increase d in ARIMA
Singular matrix errorPerfect multicollinearityRemove redundant variables; check VIF > 10
Different results from RDefault settings differCheck: constant term, link function, optimizer, SE type
PerfectSeparationError in LogitPredictor perfectly separates classesUse regularized logistic (penalized MLE) or Firth's method

References

Cellpose — Deep Learning Cell Segmentation

Overview

Cellpose uses a flow-based neural network to segment individual cells or nuclei in fluorescence microscopy images without manual parameter tuning. Pre-trained models (cyto3, nuclei, tissuenet) generalize across cell types, magnifications, and staining conditions — eliminating the need for manual threshold selection or watershed parameter optimization. Cellpose outputs integer label masks (each cell = unique integer) compatible with scikit-image regionprops for morphology measurement and with TrackPy for tracking. A built-in diameter estimator removes the need to specify cell size, though providing an approximate diameter improves accuracy.

When to Use

  • Segmenting cells or nuclei in fluorescence microscopy images where rule-based thresholding fails due to varying intensity or cell touching
  • Processing large microscopy datasets in batch without per-image parameter tuning
  • Segmenting diverse cell types (adherent cells, blood cells, bacteria, organoids) with a single model
  • Producing label masks for downstream region property measurement (area, intensity, shape) with scikit-image
  • 3D volumetric segmentation of z-stack microscopy data with do_3D=True
  • Use scikit-image watershed when cells are well-separated and rule-based thresholding is sufficient
  • Use StarDist as an alternative deep learning segmenter optimized for star-convex cells (neurons, nuclei)

Prerequisites

  • Python packages: cellpose, numpy, matplotlib
  • Optional: GPU with CUDA for 10-50× speedup (pip install cellpose[gui] for GUI)
  • Input: grayscale or multichannel TIFF/PNG images (2D or 3D arrays)
# Install Cellpose
pip install cellpose

# Install with GUI support
pip install cellpose[gui]

# Install with GPU (PyTorch CUDA)
pip install cellpose torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

# Verify
python -c "from cellpose import models; print('Cellpose ready')"

Quick Start

from cellpose import models
import numpy as np
from skimage import io

# Load image (grayscale or 2D array)
img = io.imread("cells.tif")  # shape: (H, W) or (H, W, C)

# Initialize model and segment
model = models.Cellpose(model_type="cyto3", gpu=False)
masks, flows, styles, diams = model.eval(img, diameter=0, channels=[0, 0])

print(f"Cells segmented: {masks.max()}")  # number of cells
print(f"Estimated diameter: {diams:.1f} px")
print(f"Mask shape: {masks.shape}")

Workflow

Step 1: Load and Inspect Images

Load microscopy images and inspect channel layout before segmentation.

import numpy as np
from skimage import io
import matplotlib.pyplot as plt

# Load single-channel fluorescence image
img_gray = io.imread("nucleus_dapi.tif")           # shape: (H, W)
img_rgb = io.imread("cells_multichannel.tif")      # shape: (H, W, C)

print(f"Grayscale shape: {img_gray.shape}, dtype: {img_gray.dtype}")
print(f"Multichannel shape: {img_rgb.shape}")

# Preview
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
axes[0].imshow(img_gray, cmap="gray")
axes[0].set_title("DAPI (nuclei)")
axes[1].imshow(img_rgb[..., 0], cmap="green")
axes[1].set_title("GFP channel")
plt.tight_layout()
plt.savefig("image_preview.png", dpi=100)
print("Saved: image_preview.png")

Step 2: Segment Cells with a Pre-trained Model

Run Cellpose with the appropriate pre-trained model.

from cellpose import models
import numpy as np
from skimage import io

# Available models: 'cyto3' (cells), 'nuclei', 'tissuenet', 'cyto2', 'CP'
model = models.Cellpose(model_type="cyto3", gpu=False)

img = io.imread("cells.tif")

# channels=[cytoplasm_channel, nucleus_channel]
# Use [0, 0] for grayscale; [1, 3] for green cytoplasm + blue nucleus (1-indexed)
masks, flows, styles, diams = model.eval(
    img,
    diameter=0,          # 0 = auto-estimate; or provide px estimate
    channels=[0, 0],     # grayscale
    flow_threshold=0.4,  # lower = fewer false positives; range 0.1-1.0
    cellprob_threshold=0.0,  # lower = more cells detected; range -6 to 6
)

print(f"Cells found: {masks.max()}")
print(f"Estimated cell diameter: {diams:.1f} pixels")
np.save("masks.npy", masks)

Step 3: Segment Nuclei from DAPI Channel

Use the nuclei model for DAPI-stained nuclei.

from cellpose import models
from skimage import io
import numpy as np

model = models.Cellpose(model_type="nuclei", gpu=False)
dapi = io.imread("dapi.tif")

# Nucleus-only segmentation: channels=[0, 0] (single channel)
masks, flows, styles, diams = model.eval(
    dapi,
    diameter=30,         # approximate nucleus diameter in pixels
    channels=[0, 0],
    flow_threshold=0.4,
    cellprob_threshold=0.0,
)

print(f"Nuclei segmented: {masks.max()}")
# Save label mask as TIFF for ImageJ/FIJI compatibility
from skimage import io as skio
skio.imsave("nuclei_masks.tif", masks.astype(np.uint16))
print("Saved: nuclei_masks.tif")

Step 4: Visualize Segmentation Results

Overlay masks on original images for quality control.

from cellpose import plot as cpplot
import matplotlib.pyplot as plt
import numpy as np
from skimage import io

img = io.imread("cells.tif")
masks = np.load("masks.npy")
flows_data = None  # load if you saved them: flows = np.load("flows.npy", allow_pickle=True)

# Cellpose built-in visualization
fig, axes = plt.subplots(1, 3, figsize=(15, 5))

# Original image
axes[0].imshow(img, cmap="gray")
axes[0].set_title(f"Original image")

# Label mask (each cell = unique color)
axes[1].imshow(masks, cmap="tab20")
axes[1].set_title(f"Segmentation masks ({masks.max()} cells)")

# Overlay: outline on original
from skimage.segmentation import find_boundaries
boundaries = find_boundaries(masks, mode="inner")
overlay = np.stack([img / img.max()] * 3, axis=-1)
overlay[boundaries] = [1, 0, 0]  # red outlines
axes[2].imshow(overlay)
axes[2].set_title("Outlines overlay")

plt.tight_layout()
plt.savefig("segmentation_result.png", dpi=150)
print("Saved: segmentation_result.png")

Step 5: Measure Cell Properties from Masks

Extract morphology and intensity measurements using scikit-image regionprops.

import numpy as np
import pandas as pd
from skimage.measure import regionprops_table
from skimage import io

masks = np.load("masks.npy")
img = io.imread("cells.tif")

# Measure morphology and intensity per cell
props = regionprops_table(
    masks, intensity_image=img,
    properties=["label", "area", "centroid", "eccentricity",
                 "mean_intensity", "max_intensity", "perimeter",
                 "equivalent_diameter_area"]
)
df = pd.DataFrame(props)
df.columns = ["cell_id", "area_px", "centroid_y", "centroid_x",
              "eccentricity", "mean_intensity", "max_intensity",
              "perimeter", "diameter_px"]

print(f"Cells measured: {len(df)}")
print(f"Median area: {df['area_px'].median():.0f} px²")
print(f"Median diameter: {df['diameter_px'].median():.1f} px")
print(df.head())

df.to_csv("cell_measurements.csv", index=False)

Step 6: Batch Segment Multiple Images

Process a directory of images and aggregate results.

from cellpose import models
from skimage import io
from skimage.measure import regionprops_table
import pandas as pd
import numpy as np
from pathlib import Path

model = models.Cellpose(model_type="cyto3", gpu=False)
image_dir = Path("images/")
output_dir = Path("results/")
output_dir.mkdir(exist_ok=True)

all_stats = []
for img_path in sorted(image_dir.glob("*.tif")):
    img = io.imread(img_path)
    masks, _, _, diams = model.eval(img, diameter=0, channels=[0, 0])
    
    # Save mask
    np.save(output_dir / f"{img_path.stem}_masks.npy", masks)
    
    # Measure
    if masks.max() > 0:
        props = regionprops_table(masks, intensity_image=img,
                                  properties=["label", "area", "mean_intensity"])
        df = pd.DataFrame(props)
        df["image"] = img_path.name
        df["est_diameter"] = diams
        all_stats.append(df)
    
    print(f"{img_path.name}: {masks.max()} cells, diameter={diams:.0f}px")

summary = pd.concat(all_stats, ignore_index=True)
summary.to_csv(output_dir / "all_cells.csv", index=False)
print(f"\nTotal cells: {len(summary)} across {summary['image'].nunique()} images")

Key Parameters

ParameterDefaultRange/OptionsEffect
model_type"cyto3""cyto3", "cyto2", "nuclei", "tissuenet", "CP", custom pathPre-trained model; cyto3 is most general; nuclei for DAPI-only
diameter300–500 pxApproximate cell diameter in pixels; 0 = auto-estimate from image
channels[0, 0][cyto, nucleus] (0=gray, 1=R, 2=G, 3=B)Channel indices for cytoplasm and nuclear stain
flow_threshold0.40.1–1.0Cell probability threshold from flow field; lower = stricter
cellprob_threshold0.0−6 to 6Cell probability cutoff; increase to find more cells
gpuFalseTrue, FalseEnable GPU inference (requires CUDA PyTorch)
do_3DFalseTrue, FalseEnable 3D volumetric segmentation of z-stacks
min_size15integer px²Minimum object size in pixels²; smaller objects discarded
batch_size8integerNumber of image tiles processed per GPU batch
normalizeTrueTrue, FalseNormalize image intensity before segmentation

Common Recipes

Recipe 1: Segment Multichannel Image (GFP + DAPI)

from cellpose import models
from skimage import io
import numpy as np

model = models.Cellpose(model_type="cyto3", gpu=False)

# Multichannel image: channel 1 = GFP (cytoplasm), channel 3 = DAPI (nucleus)
img_multi = io.imread("cells_gfp_dapi.tif")  # shape: (H, W, 3)

# channels=[cytoplasm_channel, nucleus_channel] (1-indexed for multichannel)
masks, flows, styles, diams = model.eval(
    img_multi,
    diameter=0,
    channels=[2, 3],  # GFP=channel2, DAPI=channel3 (1-indexed)
    flow_threshold=0.4,
)
print(f"Cells segmented: {masks.max()}, diameter: {diams:.0f}px")
np.save("masks_multichannel.npy", masks)

Recipe 2: Use Cellpose CLI for Directory Batch Processing

# CLI batch segmentation of all TIFFs in a directory
cellpose \
    --image_path images/ \
    --pretrained_model cyto3 \
    --diameter 0 \
    --chan 0 \
    --save_tif \
    --no_npy

# With GPU
cellpose \
    --image_path images/ \
    --pretrained_model nuclei \
    --diameter 30 \
    --chan 0 \
    --use_gpu \
    --save_tif

# Results saved as: images/*_cp_masks.tif
echo "Done. Masks saved in images/ directory."

Recipe 3: Fine-tune Cellpose on Custom Cell Type

from cellpose import models, train
import numpy as np
from skimage import io

# Prepare training data: list of images and corresponding masks
train_images = [io.imread(f"train/img_{i}.tif") for i in range(10)]
train_masks = [np.load(f"train/mask_{i}.npy") for i in range(10)]

# Fine-tune starting from cyto3
model = models.CellposeModel(model_type="cyto3")

# Train: saves model to models/ directory
model_path = train.train_seg(
    model.net,
    train_data=train_images,
    train_labels=train_masks,
    channels=[0, 0],
    save_path="models/",
    n_epochs=100,
    learning_rate=0.2,
    weight_decay=1e-5,
)
print(f"Fine-tuned model saved: {model_path}")

Expected Outputs

OutputFormatDescription
masks arraynumpy int32Label mask: 0=background, 1..N=unique cell IDs
flows listnumpy arraysFlow field components: [XY flows, cell prob, gradient]
styles arraynumpy floatStyle vector embedding (used for model similarity)
diams floatscalarEstimated average cell diameter in pixels
*_masks.npyNumPySaved mask array (from np.save)
*_cp_masks.tifTIFF uint16Mask TIFF (from CLI --save_tif); compatible with FIJI/ImageJ

Troubleshooting

ProblemCauseSolution
All cells merged into one maskDiameter too large or cells too closeReduce diameter; increase flow_threshold to 0.6–0.8
Very few cells detectedDiameter too small or cellprob_threshold too highIncrease cellprob_threshold to −2; use diameter=0 for auto
Many false positives (background labeled)Low flow_thresholdIncrease flow_threshold to 0.6–0.9; increase min_size
GPU out of memoryImage too large for GPU batchProcess in tiles; reduce batch_size; crop image
Poor generalization on new cell typeModel not trained on similar cellsTry all pre-trained models; fine-tune with 10-20 annotated images
3D segmentation very slowLarge z-stack on CPUEnable GPU; reduce z-stack depth; use anisotropy parameter
Mask values overflow uint8More than 255 cells in imageSave with dtype=np.uint16 or np.int32
Import error: No module named 'cellpose'Package not installedpip install cellpose or conda install -c conda-forge cellpose

References

FlowIO — Flow Cytometry File Handler

Overview

FlowIO is a lightweight Python library for reading and writing Flow Cytometry Standard (FCS) files. It parses FCS metadata, extracts event data as NumPy arrays, and creates new FCS files. Supports FCS versions 2.0, 3.0, and 3.1. Minimal dependencies — ideal for data pipelines and preprocessing before advanced analysis.

When to Use

  • Parsing FCS files to extract event data as NumPy arrays
  • Reading channel metadata (names, ranges, types) from FCS files
  • Converting flow cytometry data to pandas DataFrames or CSV
  • Creating new FCS files from NumPy arrays or processed data
  • Handling multi-dataset FCS files (separating combined datasets)
  • Batch processing directories of FCS files
  • Preprocessing flow cytometry data before downstream analysis
  • For compensation, gating, and FlowJo workspace support, use FlowKit instead
  • For advanced cytometry visualization (density plots, gating plots), use matplotlib or plotly

Prerequisites

pip install flowio numpy pandas

Requires Python 3.9+. No compiled dependencies — installs on any platform.

Quick Start

from flowio import FlowData

flow = FlowData("experiment.fcs")
print(f"Events: {flow.event_count}, Channels: {flow.channel_count}")
print(f"Channels: {flow.pnn_labels}")

events = flow.as_array()  # Shape: (n_events, n_channels)
print(f"Data shape: {events.shape}")

Core API

1. Reading FCS Files

The FlowData class is the primary interface for reading FCS files.

from flowio import FlowData

# Standard reading
flow = FlowData("sample.fcs")
print(f"Version: {flow.version}")          # '3.0', '3.1', etc.
print(f"Events: {flow.event_count}")
print(f"Channels: {flow.channel_count}")

# Event data
events = flow.as_array()                   # Preprocessed (gain, log scaling)
raw = flow.as_array(preprocess=False)      # Raw values
print(f"Shape: {events.shape}")            # (n_events, n_channels)

# Memory-efficient: metadata only (skip DATA segment)
flow_meta = FlowData("sample.fcs", only_text=True)
print(f"Instrument: {flow_meta.text.get('$CYT', 'Unknown')}")

# Handle problematic files
flow = FlowData("bad.fcs", ignore_offset_discrepancy=True)
flow = FlowData("bad.fcs", use_header_offsets=True)

# Exclude null channels
flow = FlowData("sample.fcs", null_channel_list=["Time", "Null"])

2. Channel Metadata

Extract channel names, types, and ranges from FCS files.

flow = FlowData("sample.fcs")

# Channel names
pnn = flow.pnn_labels   # Short names: ['FSC-A', 'SSC-A', 'FL1-A', ...]
pns = flow.pns_labels   # Descriptive: ['Forward Scatter', 'Side Scatter', 'FITC', ...]
pnr = flow.pnr_values   # Range/max values per channel

# Channel type indices
scatter_idx = flow.scatter_indices   # [0, 1] — FSC, SSC
fluoro_idx = flow.fluoro_indices     # [2, 3, 4] — fluorescence channels
time_idx = flow.time_index           # Time channel index (or None)

# Access by type
events = flow.as_array()
scatter_data = events[:, scatter_idx]
fluoro_data = events[:, fluoro_idx]

# Full metadata (TEXT segment dictionary)
text = flow.text
print(f"Date: {text.get('$DATE', 'N/A')}")
print(f"Instrument: {text.get('$CYT', 'N/A')}")

3. Creating FCS Files

Generate new FCS files from NumPy arrays.

import numpy as np
from flowio import create_fcs

# Basic creation
events = np.random.rand(10000, 5) * 1000
channels = ["FSC-A", "SSC-A", "FL1-A", "FL2-A", "Time"]
create_fcs("output.fcs", events, channels)

# With descriptive names and metadata
create_fcs(
    "output.fcs",
    events,
    channels,
    opt_channel_names=["Forward Scatter", "Side Scatter", "FITC", "PE", "Time"],
    metadata={"$SRC": "Python pipeline", "$DATE": "17-FEB-2026", "$CYT": "Synthetic"},
)
# Output: FCS 3.1, single-precision float

4. Multi-Dataset FCS Files

Handle FCS files containing multiple datasets.

from flowio import FlowData, read_multiple_data_sets, MultipleDataSetsError

# Detect multi-dataset files
try:
    flow = FlowData("sample.fcs")
except MultipleDataSetsError:
    datasets = read_multiple_data_sets("sample.fcs")
    print(f"Found {len(datasets)} datasets")
    for i, ds in enumerate(datasets):
        print(f"Dataset {i}: {ds.event_count} events, {ds.channel_count} channels")
        events = ds.as_array()

# Read specific dataset by offset
first = FlowData("multi.fcs", nextdata_offset=0)
next_offset = int(first.text.get("$NEXTDATA", "0"))
if next_offset > 0:
    second = FlowData("multi.fcs", nextdata_offset=next_offset)

5. Modifying and Re-Exporting

Read, modify, and save FCS data.

from flowio import FlowData, create_fcs

# Read original
flow = FlowData("original.fcs")
events = flow.as_array(preprocess=False)  # Use raw for modification

# Filter events (e.g., threshold on FSC)
mask = events[:, 0] > 500
filtered = events[mask]
print(f"Before: {len(events)}, After: {len(filtered)}")

# Save filtered data as new FCS
create_fcs(
    "filtered.fcs",
    filtered,
    flow.pnn_labels,
    opt_channel_names=flow.pns_labels,
    metadata={**flow.text, "$SRC": "Filtered"},
)

# Or write with updated metadata (no event modification)
flow.write_fcs("updated.fcs", metadata={"$SRC": "Updated"})

Key Concepts

FCS File Structure

FCS files consist of four segments:

SegmentContentFlowData attribute
HEADERVersion, byte offsetsflow.header
TEXTKey-value metadata ($DATE, $CYT, channel names)flow.text
DATAEvent data (binary/float)flow.events (bytes), flow.as_array()
ANALYSISOptional processed resultsflow.analysis

Preprocessing (as_array)

When preprocess=True (default), FlowIO applies:

  1. Gain scaling: Multiply by PnG gain values
  2. Log transform: Apply PnE exponential transform if present (value = a × 10^(b × raw))
  3. Time scaling: Convert time channel to proper units

Use preprocess=False when you need raw values for modification or custom transforms.

Common Workflows

Workflow: Batch FCS Summary

from pathlib import Path
from flowio import FlowData
import pandas as pd

fcs_files = list(Path("data/").glob("*.fcs"))
summaries = []
for f in fcs_files:
    try:
        flow = FlowData(str(f), only_text=True)
        summaries.append({
            "file": f.name, "version": flow.version,
            "events": flow.event_count, "channels": flow.channel_count,
            "date": flow.text.get("$DATE", "N/A"),
        })
    except Exception as e:
        print(f"Error: {f.name}: {e}")

df = pd.DataFrame(summaries)
print(df)

Workflow: FCS to DataFrame with Channel Statistics

from flowio import FlowData
import pandas as pd
import numpy as np

flow = FlowData("sample.fcs")
df = pd.DataFrame(flow.as_array(), columns=flow.pnn_labels)

# Per-channel statistics
for col in df.columns:
    print(f"{col}: mean={df[col].mean():.1f}, median={df[col].median():.1f}, std={df[col].std():.1f}")

# Export
df.to_csv("output.csv", index=False)
print(f"Exported {len(df)} events, {len(df.columns)} channels")

Key Parameters

ParameterFunctionDefaultOptionsEffect
preprocessas_array()TrueTrue/FalseApply gain/log scaling
only_textFlowData()FalseTrue/FalseSkip DATA segment (metadata only)
ignore_offset_discrepancyFlowData()FalseTrue/FalseTolerate HEADER/TEXT offset mismatch
use_header_offsetsFlowData()FalseTrue/FalsePrefer HEADER over TEXT offsets
ignore_offset_errorFlowData()FalseTrue/FalseSkip all offset validation
null_channel_listFlowData()NoneList of namesExclude channels during parsing
nextdata_offsetFlowData()Nonebyte offsetRead specific dataset in multi-dataset files
opt_channel_namescreate_fcs()NoneList of namesDescriptive channel names (PnS)
metadatacreate_fcs()NoneDictCustom TEXT segment key-value pairs

Best Practices

  1. Use only_text=True for metadata scanning: When processing many files, skip DATA segment parsing for 10-100x speedup.

  2. Use preprocess=False for data modification: Always work with raw values when filtering/modifying events, then re-export. Preprocessing is irreversible.

  3. Anti-pattern — modifying flow.events directly: FlowIO does not support in-place event modification. Extract with as_array(), modify, then create_fcs() to save.

  4. Preserve metadata on re-export: Pass flow.text as metadata to create_fcs() to retain original acquisition info.

  5. Check for multi-dataset files: Catch MultipleDataSetsError and use read_multiple_data_sets() — some instruments write multiple acquisitions into one file.

Common Recipes

Recipe: Extract Fluorescence Channels Only

from flowio import FlowData
import numpy as np

flow = FlowData("sample.fcs")
events = flow.as_array()
fluoro = events[:, flow.fluoro_indices]
names = [flow.pnn_labels[i] for i in flow.fluoro_indices]
print(f"Fluorescence channels: {names}, shape: {fluoro.shape}")

Recipe: File Inspection Report

from flowio import FlowData

flow = FlowData("unknown.fcs")
print(f"Version: {flow.version} | Events: {flow.event_count:,} | Channels: {flow.channel_count}")
for i, (pnn, pns) in enumerate(zip(flow.pnn_labels, flow.pns_labels)):
    ctype = "scatter" if i in flow.scatter_indices else "fluoro" if i in flow.fluoro_indices else "time" if i == flow.time_index else "other"
    print(f"  [{i}] {pnn:10s} | {pns:30s} | {ctype}")
for key in ["$DATE", "$CYT", "$INST", "$SRC"]:
    print(f"  {key}: {flow.text.get(key, 'N/A')}")

Recipe: Normalize Events to [0, 1] Range

When to use: Prepare fluorescence channels for machine learning or cross-sample comparison.

from flowio import FlowData
import numpy as np

flow = FlowData("sample.fcs")
events = flow.as_array()

# Normalize each fluorescence channel to [0, 1]
fluoro_idx = flow.fluoro_indices
fluoro = events[:, fluoro_idx]
pnr = np.array(flow.pnr_values)[fluoro_idx]  # Per-channel max range
normalized = fluoro / pnr
print(f"Normalized shape: {normalized.shape}, range: [{normalized.min():.3f}, {normalized.max():.3f}]")

Troubleshooting

ProblemCauseSolution
DataOffsetDiscrepancyErrorHEADER/TEXT offset mismatchUse ignore_offset_discrepancy=True
MultipleDataSetsErrorFile contains multiple datasetsUse read_multiple_data_sets() instead
FCSParsingErrorCorrupt or non-standard FCS fileTry ignore_offset_error=True; verify file is valid FCS
Out of memory on large filesMillions of events loaded at onceUse only_text=True for metadata; process in chunks by channel
Unexpected channel countNull/padding channels in fileUse null_channel_list=["Time", "Null"] to exclude
Modified data has wrong valuesApplied preprocessing before modificationUse preprocess=False for raw data when modifying events
Channel names missing (empty PnS)Instrument didn't set descriptive namesUse pnn_labels (short names) instead; PnS is optional in FCS spec

Related Skills

  • matplotlib-scientific-plotting — create scatter plots, density plots, and histograms from extracted cytometry data
  • scikit-learn-machine-learning — clustering and dimensionality reduction on cytometry event data

References

napari — Multi-dimensional Image Viewer

Overview

napari is a fast, interactive multi-dimensional viewer for scientific data built on PyQt5 and VisPy. It displays NumPy arrays and zarr arrays as layered visualizations — Image layers for raw data, Labels layers for segmentation masks, Points layers for cell centroids, and Shapes layers for ROI annotations. napari integrates with scikit-image, Cellpose, and StarDist via plugins, making it the standard visualization and annotation tool in Python bioimage analysis pipelines. For headless environments (HPC, CI), napari supports offscreen rendering and viewer.screenshot() for automated figure generation.

When to Use

  • Visually inspecting and quality-checking microscopy images and segmentation masks before quantitative analysis
  • Annotating training data for deep learning segmentation models (Cellpose, StarDist)
  • Overlaying multiple image channels (DAPI, GFP, mCherry) with independent contrast and colormap control
  • Reviewing 3D z-stacks and 4D time-lapse experiments with slider-based navigation
  • Exporting annotated screenshots or label masks from GUI for publication figures
  • Running plugin-based analysis (Cellpose napari plugin, StarDist plugin, n2v denoising) interactively
  • Use ImageJ/FIJI for macro/batch scripting with minimal Python dependency
  • Use ITK-SNAP as an alternative for medical imaging (DICOM, NIfTI) segmentation

Prerequisites

  • Python packages: napari, numpy, scikit-image
  • Qt backend: requires display server; for headless use QT_QPA_PLATFORM=offscreen
  • Optional plugins: napari-cellpose, napari-stardist, napari-animation
# Install with all backends
pip install "napari[all]"

# Or minimal install
pip install napari pyqt5

# Verify
python -c "import napari; print(napari.__version__)"
# 0.5.5

# Install useful plugins
pip install napari-cellpose napari-animation

Quick Start

import napari
import numpy as np
from skimage import data

# Open viewer with a sample image
viewer = napari.Viewer()
viewer.add_image(data.cells3d()[:, 1, :, :], name="DAPI", colormap="blue")
napari.run()   # blocks until viewer closed (use in scripts)

Core API

Module 1: Image Layer — Display Raw Images

Add and configure multi-channel image layers.

import napari
import numpy as np
from skimage import io

viewer = napari.Viewer()

# Add single grayscale image
img = io.imread("cells.tif")  # shape: (H, W)
viewer.add_image(img, name="phase contrast", colormap="gray",
                 contrast_limits=[0, img.max()])

# Add multichannel image (3 channels)
img_mc = io.imread("multichannel.tif")  # shape: (H, W, 3)
viewer.add_image(img_mc[..., 0], name="DAPI", colormap="blue", blending="additive")
viewer.add_image(img_mc[..., 1], name="GFP", colormap="green", blending="additive")
viewer.add_image(img_mc[..., 2], name="mCherry", colormap="red", blending="additive")

print(f"Layers: {[l.name for l in viewer.layers]}")

Module 2: Labels Layer — Visualize Segmentation Masks

Display and edit integer label masks from Cellpose, StarDist, or scikit-image.

import napari
import numpy as np
from skimage import io

viewer = napari.Viewer()

img = io.imread("cells.tif")
masks = np.load("masks.npy")  # integer label array: 0=background, 1..N=cells

# Add raw image
viewer.add_image(img, name="raw", colormap="gray")

# Add label mask (each cell gets a unique random color)
label_layer = viewer.add_labels(masks, name="cell_masks", opacity=0.5)

# Access labels for editing
print(f"Unique cells: {len(np.unique(masks)) - 1}")
print(f"Label layer data shape: {label_layer.data.shape}")

Module 3: Points Layer — Mark Cell Centroids

Add and style point markers for centroids, landmarks, or detected features.

import napari
import numpy as np
import pandas as pd
from skimage.measure import regionprops_table

viewer = napari.Viewer()

# Compute centroids from label mask
masks = np.load("masks.npy")
props = regionprops_table(masks, properties=["centroid", "label"])
centroids = np.column_stack([props["centroid-0"], props["centroid-1"]])

# Add centroids as Points layer
viewer.add_points(
    centroids,
    name=f"centroids ({len(centroids)} cells)",
    size=8,
    face_color="yellow",
    edge_color="black",
    edge_width=0.5,
)
print(f"Cells marked: {len(centroids)}")

Module 4: Shapes Layer — Draw ROIs and Annotations

Add bounding boxes, polygons, and line annotations.

import napari
import numpy as np

viewer = napari.Viewer()

# Add rectangles as ROIs (format: [[y1, x1], [y2, x2]])
rois = [
    np.array([[50, 100], [200, 300]]),   # ROI 1
    np.array([[300, 150], [450, 350]]),  # ROI 2
]

shapes_layer = viewer.add_shapes(
    rois,
    shape_type="rectangle",
    name="ROIs",
    edge_color="cyan",
    face_color="transparent",
    edge_width=2,
)

# Retrieve shapes data for analysis
for i, shape in enumerate(shapes_layer.data):
    y_min, x_min = shape.min(axis=0)
    y_max, x_max = shape.max(axis=0)
    print(f"ROI {i+1}: y={y_min:.0f}-{y_max:.0f}, x={x_min:.0f}-{x_max:.0f}")

Module 5: 3D and Time-lapse Visualization

Display z-stacks and time series with sliders.

import napari
import numpy as np
from skimage import data

viewer = napari.Viewer()

# 3D z-stack: shape (Z, H, W)
zstack = data.cells3d()[:, 1, :, :]   # nuclei channel
viewer.add_image(zstack, name="z-stack nuclei",
                 colormap="cyan", blending="additive")

# 4D time-lapse: shape (T, H, W) or (T, Z, H, W)
timelapse = np.random.randint(0, 65535, (10, 256, 256), dtype=np.uint16)
viewer.add_image(timelapse, name="timelapse", colormap="gray")

# napari shows axis sliders automatically for ndim > 2
print(f"z-stack shape: {zstack.shape} → slider for Z axis")
print(f"timelapse shape: {timelapse.shape} → sliders for T axis")

Module 6: Headless Screenshot Export

Export screenshots without a display (for HPC and CI environments).

import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"  # must be set BEFORE importing napari

import napari
import numpy as np
from skimage import io, data
import matplotlib
matplotlib.use("Agg")  # also set matplotlib backend

viewer = napari.Viewer(show=False)

img = data.cells3d()[30, 1, :, :]  # single z-slice
masks = (img > img.mean()).astype(int)  # simple threshold mask

viewer.add_image(img, name="DAPI", colormap="blue", blending="additive")
viewer.add_labels(masks.astype(np.int32), name="masks", opacity=0.5)

# Export screenshot
screenshot = viewer.screenshot(path="napari_export.png", canvas_only=True)
print(f"Screenshot saved: napari_export.png ({screenshot.shape})")
viewer.close()

Key Parameters

ParameterModuleDefaultEffect
colormapadd_image"gray"Colormap name (matplotlib cmaps + napari built-ins: "green", "blue", "cyan")
contrast_limitsadd_imageauto[min, max] intensity clipping for display
blendingadd_image"translucent""additive" for multichannel overlay; "opaque" for solid
opacityadd_labels0.70–1 transparency of label layer over image
face_coloradd_points"white"Point fill color (name, hex, or RGBA)
sizeadd_points10Point radius in data coordinates (pixels)
edge_widthadd_shapes1Shape outline width in pixels
showViewer()TrueFalse for headless/offscreen mode
ndisplayViewer()23 for 3D OpenGL rendering mode
canvas_onlyscreenshot()FalseTrue to exclude the napari toolbar from export

Common Workflows

Workflow 1: Review Cellpose Segmentation Quality

import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"

import napari
import numpy as np
from cellpose import models
from skimage import io
from skimage.measure import regionprops_table

# Segment with Cellpose
img = io.imread("cells.tif")
model = models.Cellpose(model_type="cyto3", gpu=False)
masks, _, _, diams = model.eval(img, diameter=0, channels=[0, 0])

# Visualize in napari (headless for export)
viewer = napari.Viewer(show=False)
viewer.add_image(img, name="raw", colormap="gray")
viewer.add_labels(masks, name=f"masks ({masks.max()} cells)", opacity=0.6)

# Add centroids
props = regionprops_table(masks, properties=["centroid"])
centroids = np.column_stack([props["centroid-0"], props["centroid-1"]])
viewer.add_points(centroids, name="centroids", size=6, face_color="yellow")

viewer.screenshot(path="segmentation_review.png", canvas_only=True)
viewer.close()
print(f"QC export: segmentation_review.png — {masks.max()} cells detected")

Workflow 2: Multi-channel FISH Image Analysis

import napari
import numpy as np
from skimage import io

# Load 4-channel FISH image: DAPI + 3 RNA probes
img = io.imread("fish_4channel.tif")  # shape: (H, W, 4)

viewer = napari.Viewer()
channels = [
    ("DAPI", "blue", img[..., 0]),
    ("probe_A_cy3", "yellow", img[..., 1]),
    ("probe_B_cy5", "red", img[..., 2]),
    ("probe_C_gfp", "green", img[..., 3]),
]

for name, colormap, channel in channels:
    viewer.add_image(channel, name=name, colormap=colormap,
                     blending="additive",
                     contrast_limits=[channel.min(), np.percentile(channel, 99.5)])

napari.run()

Common Recipes

Recipe 1: Export All Layers as Annotated Figure

import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"

import napari
import numpy as np
from skimage import io
import matplotlib.pyplot as plt

viewer = napari.Viewer(show=False)

img = io.imread("cells.tif")
masks = np.load("masks.npy")

viewer.add_image(img, name="raw", colormap="gray")
viewer.add_labels(masks, name="segmentation", opacity=0.5)

# Set camera zoom and position
viewer.camera.zoom = 1.5
viewer.camera.center = (img.shape[0] // 2, img.shape[1] // 2)

screenshot = viewer.screenshot(path="figure_panel.png", canvas_only=True)
viewer.close()

# Add scalebar with matplotlib
fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(screenshot)
ax.axis("off")
plt.tight_layout()
plt.savefig("figure_final.pdf", dpi=300, bbox_inches="tight")
print("Exported: figure_final.pdf")

Recipe 2: Batch Export Z-stack Projections

import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"

import napari
import numpy as np
from skimage import io
from pathlib import Path

output_dir = Path("projections")
output_dir.mkdir(exist_ok=True)

for img_path in sorted(Path("zstacks").glob("*.tif")):
    zstack = io.imread(img_path)  # shape: (Z, H, W)
    max_proj = zstack.max(axis=0)
    
    viewer = napari.Viewer(show=False)
    viewer.add_image(max_proj, name="max_projection", colormap="gray")
    viewer.screenshot(path=str(output_dir / f"{img_path.stem}_maxproj.png"), canvas_only=True)
    viewer.close()
    print(f"Exported: {img_path.stem}_maxproj.png")

print("All z-stack projections exported.")

Troubleshooting

ProblemCauseSolution
qt.qpa.plugin: Could not load the Qt platform plugin "xcb"Missing display or Qt platform pluginSet QT_QPA_PLATFORM=offscreen before importing napari; install libxcb-util-dev
napari window does not openRunning in SSH without X forwardingUse viewer = napari.Viewer(show=False) and export via screenshot()
Slow rendering of large imagesImage too large for GPU VRAMUse viewer.add_image(img, multiscale=True) for pyramidal rendering
Labels layer shows wrong colorsMask dtype overflowEnsure masks are int32 not uint8 (overflow at 255 cells)
napari.run() blocks Jupyter notebookQt event loop conflictUse %gui qt magic in Jupyter; or use viewer.show() without napari.run()
Screenshot is black/emptyViewer not fully rendered before screenshotAdd viewer.update() or slight delay before screenshot()
Plugin not appearing in menuPlugin not installed or wrong napari versionpip install napari-<plugin>; check napari version compatibility on napari-hub
3D rendering slowComplex geometry or large volumeSwitch viewer.dims.ndisplay = 2; reduce z-stack depth

References

OpenCV — Bio-image Computer Vision

Overview

OpenCV (cv2) provides optimized C++-backed image processing routines for preprocessing, segmentation, feature extraction, and video analysis of biological images. In life sciences, OpenCV is used for fluorescence image enhancement (background subtraction, CLAHE), morphological segmentation (watershed, contour detection), brightfield cell detection, and real-time microscopy stream processing. Unlike scikit-image (which emphasizes scientific measurement), OpenCV prioritizes computational speed and video support — making it ideal for preprocessing pipelines and real-time imaging applications.

When to Use

  • Preprocessing fluorescence or brightfield images: background subtraction, CLAHE, Gaussian/median blur
  • Detecting cell contours, blobs, or edges without deep learning (classical methods)
  • Processing video streams from live-cell imaging microscopes in real-time
  • Template matching for finding repeated structures (organelles, crystals, patterns)
  • Applying morphological operations (erosion, dilation, opening, closing) for mask refinement
  • Computing optical flow between video frames for cell tracking
  • Use scikit-image instead for scientific morphometry, regionprops, and scientific image I/O (TIFF metadata)
  • Use Cellpose or StarDist instead for deep-learning cell segmentation on fluorescence images

Prerequisites

  • Python packages: opencv-python, numpy, matplotlib
  • Optional: opencv-contrib-python for extra modules (SIFT, SURF, optical flow)
# Install OpenCV
pip install opencv-python

# Install with extra contributed modules (SIFT, SURF, etc.)
pip install opencv-contrib-python

# Verify
python -c "import cv2; print(cv2.__version__)"
# 4.10.0

Quick Start

import cv2
import numpy as np

# Read and display image info
img = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)
print(f"Shape: {img.shape}, dtype: {img.dtype}")
print(f"Min: {img.min()}, Max: {img.max()}")

# Apply Gaussian blur and threshold
blurred = cv2.GaussianBlur(img, (5, 5), 0)
_, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
print(f"Cells detected (rough): {np.sum(binary > 0)} foreground pixels")

Core API

Module 1: Image I/O and Color Space Conversion

Read, write, and convert images between color spaces.

import cv2
import numpy as np

# Read image (GRAYSCALE, COLOR, or UNCHANGED for 16-bit)
img_gray  = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)   # uint8
img_color = cv2.imread("rgb.tif", cv2.IMREAD_COLOR)         # BGR order!
img_16bit = cv2.imread("16bit.tif", cv2.IMREAD_UNCHANGED)   # uint16

print(f"Grayscale shape: {img_gray.shape}, dtype: {img_gray.dtype}")
print(f"Color shape:     {img_color.shape}")

# Color space conversions
img_rgb  = cv2.cvtColor(img_color, cv2.COLOR_BGR2RGB)   # BGR → RGB
img_hsv  = cv2.cvtColor(img_color, cv2.COLOR_BGR2HSV)   # BGR → HSV
img_gray2 = cv2.cvtColor(img_color, cv2.COLOR_BGR2GRAY) # BGR → gray

# Write image
cv2.imwrite("output.png", img_gray)
cv2.imwrite("output_16bit.tif", img_16bit)
print("Images written.")

Module 2: Filtering and Enhancement

Apply filters and contrast enhancement for image preprocessing.

import cv2
import numpy as np

img = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)

# Gaussian blur (noise reduction)
blurred = cv2.GaussianBlur(img, (7, 7), sigmaX=1.5)

# Median blur (salt-and-pepper noise)
median = cv2.medianBlur(img, 5)

# CLAHE: Contrast Limited Adaptive Histogram Equalization (for microscopy)
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
clahe_img = clahe.apply(img)

# Top-hat filter for bright spots on dark background
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (15, 15))
tophat = cv2.morphologyEx(img, cv2.MORPH_TOPHAT, kernel)

print(f"CLAHE range: [{clahe_img.min()}, {clahe_img.max()}]")
cv2.imwrite("clahe_enhanced.tif", clahe_img)

Module 3: Thresholding and Binary Segmentation

Convert grayscale images to binary masks using various thresholding methods.

import cv2
import numpy as np

img = cv2.imread("nuclei.tif", cv2.IMREAD_GRAYSCALE)

# Otsu's thresholding (automatic threshold selection)
thresh_val, otsu_mask = cv2.threshold(img, 0, 255,
                                      cv2.THRESH_BINARY + cv2.THRESH_OTSU)
print(f"Otsu threshold: {thresh_val:.0f}")

# Adaptive thresholding (handles uneven illumination)
adaptive = cv2.adaptiveThreshold(
    img, 255,
    cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
    cv2.THRESH_BINARY,
    blockSize=11,     # neighborhood size (odd)
    C=2,              # constant subtracted from mean
)

# For 16-bit images: normalize first
img_16 = cv2.imread("16bit_nuclei.tif", cv2.IMREAD_UNCHANGED)
img_8 = cv2.normalize(img_16, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
_, mask_16 = cv2.threshold(img_8, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

print(f"Otsu mask foreground: {mask_16.sum() / 255} pixels")

Module 4: Contour Detection and Measurement

Find and measure cell contours from binary masks.

import cv2
import numpy as np
import pandas as pd

img = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)
blurred = cv2.GaussianBlur(img, (5, 5), 0)
_, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

# Remove small objects with morphological opening
kernel = np.ones((3, 3), np.uint8)
cleaned = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel, iterations=2)

# Find contours
contours, hierarchy = cv2.findContours(cleaned, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
print(f"Objects detected: {len(contours)}")

# Measure each contour
records = []
for i, cnt in enumerate(contours):
    area = cv2.contourArea(cnt)
    if area < 50: continue  # skip tiny objects
    perimeter = cv2.arcLength(cnt, True)
    x, y, w, h = cv2.boundingRect(cnt)
    (cx, cy), radius = cv2.minEnclosingCircle(cnt)
    records.append({"cell_id": i, "area": area, "perimeter": perimeter,
                     "x": x, "y": y, "w": w, "h": h, "radius": radius})

df = pd.DataFrame(records)
print(f"Cells > 50 px²: {len(df)}")
print(df[["area", "perimeter", "radius"]].describe())

Module 5: Morphological Operations for Mask Refinement

Refine segmentation masks with morphological operations.

import cv2
import numpy as np

# Load binary mask (from thresholding or Cellpose)
mask = cv2.imread("rough_mask.png", cv2.IMREAD_GRAYSCALE)
_, mask = cv2.threshold(mask, 127, 255, cv2.THRESH_BINARY)

# Structural elements
ellipse = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (7, 7))
rect    = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))

# Opening: remove small bright noise
opened = cv2.morphologyEx(mask, cv2.MORPH_OPEN, ellipse, iterations=1)

# Closing: fill small holes inside cells
closed = cv2.morphologyEx(opened, cv2.MORPH_CLOSE, ellipse, iterations=2)

# Dilation: expand cell boundaries slightly
dilated = cv2.dilate(closed, ellipse, iterations=1)

# Distance transform for watershed seed generation
dist = cv2.distanceTransform(closed, cv2.DIST_L2, 5)
_, seeds = cv2.threshold(dist, 0.5 * dist.max(), 255, 0)
seeds = seeds.astype(np.uint8)
print(f"Potential cell centers: {cv2.connectedComponents(seeds)[0] - 1}")

Module 6: Video Processing for Live-Cell Imaging

Process video streams from time-lapse microscopy.

import cv2
import numpy as np

# Process a time-lapse video file
cap = cv2.VideoCapture("timelapse.avi")
fps = cap.get(cv2.CAP_PROP_FPS)
n_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
print(f"Video: {n_frames} frames at {fps} FPS")

# Background subtraction (remove static background)
bg_subtractor = cv2.createBackgroundSubtractorMOG2(
    history=50, varThreshold=25, detectShadows=False
)

frame_counts = []
frame_idx = 0
while cap.isOpened():
    ret, frame = cap.read()
    if not ret: break
    
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    fg_mask = bg_subtractor.apply(gray)
    
    # Count moving objects in this frame
    contours, _ = cv2.findContours(fg_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    moving = [c for c in contours if cv2.contourArea(c) > 100]
    frame_counts.append(len(moving))
    frame_idx += 1

cap.release()
print(f"Processed {frame_idx} frames. Mean moving objects: {np.mean(frame_counts):.1f}")

Key Parameters

ParameterModuleDefaultEffect
sigmaXGaussianBlurauto from ksizeGaussian standard deviation; larger = more smoothing
clipLimitcreateCLAHE40.0Maximum contrast amplification; 2.0–4.0 for microscopy
tileGridSizecreateCLAHE(8,8)Tile size for local histogram equalization
blockSizeadaptiveThresholdrequiredNeighborhood size for adaptive threshold (must be odd, ≥ 3)
CadaptiveThresholdrequiredConstant subtracted from mean; positive to subtract
iterationsmorphologyEx1Number of erosion/dilation cycles; higher = stronger effect
historyBackgroundSubtractorMOG2500Frames to model background; lower = faster adaptation
varThresholdBackgroundSubtractorMOG216Pixel variance threshold; higher = less sensitive
minAreacontour filter—Minimum cv2.contourArea(cnt) to keep; filter noise
cv2.IMREAD_UNCHANGEDimread—Preserve bit-depth (16-bit, 32-bit); required for scientific images

Common Workflows

Workflow 1: Fluorescence Nucleus Detection Pipeline

import cv2
import numpy as np
import pandas as pd

def detect_nuclei(image_path: str, min_area: int = 200) -> pd.DataFrame:
    """Detect DAPI-stained nuclei from a fluorescence image."""
    img = cv2.imread(image_path, cv2.IMREAD_UNCHANGED)
    
    # Normalize 16-bit to 8-bit
    if img.dtype == np.uint16:
        img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
    
    # Preprocess: CLAHE → Gaussian blur
    clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
    enhanced = clahe.apply(img)
    blurred = cv2.GaussianBlur(enhanced, (5, 5), 1.5)
    
    # Segment: Otsu threshold → morphological opening
    _, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
    cleaned = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel, iterations=1)
    
    # Find and measure contours
    contours, _ = cv2.findContours(cleaned, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    records = []
    for cnt in contours:
        area = cv2.contourArea(cnt)
        if area < min_area: continue
        M = cv2.moments(cnt)
        if M["m00"] == 0: continue
        cx = int(M["m10"] / M["m00"])
        cy = int(M["m01"] / M["m00"])
        records.append({"area": area, "cx": cx, "cy": cy,
                         "perimeter": cv2.arcLength(cnt, True)})
    
    return pd.DataFrame(records)

df = detect_nuclei("dapi.tif", min_area=300)
print(f"Nuclei detected: {len(df)}")
print(df.describe())

Workflow 2: Batch Process Image Directory

import cv2
import numpy as np
import pandas as pd
from pathlib import Path

def process_image(path: str) -> dict:
    img = cv2.imread(path, cv2.IMREAD_GRAYSCALE)
    if img is None:
        return {}
    blurred = cv2.GaussianBlur(img, (5, 5), 0)
    _, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
    contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    cells = [c for c in contours if cv2.contourArea(c) > 200]
    return {"file": Path(path).name, "cell_count": len(cells),
            "mean_area": np.mean([cv2.contourArea(c) for c in cells]) if cells else 0}

results = [process_image(str(p)) for p in sorted(Path("images").glob("*.tif"))]
df = pd.DataFrame([r for r in results if r])
print(df)
df.to_csv("batch_results.csv", index=False)
print("Saved: batch_results.csv")

Common Recipes

Recipe 1: Annotate Detected Cells on Image

import cv2
import numpy as np

img = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)
img_color = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)

blurred = cv2.GaussianBlur(img, (5, 5), 0)
_, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

for i, cnt in enumerate(contours):
    if cv2.contourArea(cnt) < 200: continue
    # Draw contour outline
    cv2.drawContours(img_color, [cnt], -1, (0, 255, 0), 2)
    # Label with cell number
    M = cv2.moments(cnt)
    if M["m00"] > 0:
        cx, cy = int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"])
        cv2.putText(img_color, str(i), (cx - 5, cy), cv2.FONT_HERSHEY_SIMPLEX,
                    0.4, (255, 255, 0), 1)

cv2.imwrite("annotated_cells.png", img_color)
print(f"Annotated {len(contours)} cells. Saved: annotated_cells.png")

Recipe 2: Background Subtraction with Rolling Ball

import cv2
import numpy as np

def rolling_ball_background(img: np.ndarray, radius: int = 50) -> np.ndarray:
    """Estimate and subtract background using a blur approximation."""
    kernel_size = 2 * radius + 1
    background = cv2.GaussianBlur(img, (kernel_size, kernel_size), radius / 3)
    corrected = cv2.subtract(img, background)
    return corrected

img = cv2.imread("uneven_fluorescence.tif", cv2.IMREAD_GRAYSCALE)
corrected = rolling_ball_background(img, radius=50)
cv2.imwrite("background_corrected.tif", corrected)
print(f"Background corrected. Range: [{corrected.min()}, {corrected.max()}]")

Troubleshooting

ProblemCauseSolution
imread returns NoneFile not found or unsupported formatUse absolute path; verify with Path(path).exists(); for TIFF use cv2.IMREAD_UNCHANGED
16-bit image shows as blackIMREAD_GRAYSCALE clips to uint8Use cv2.IMREAD_UNCHANGED and normalize: cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX)
BGR vs RGB color mismatchOpenCV uses BGR, matplotlib uses RGBConvert: rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) before plt.imshow()
Contours split one cell into manyBinary mask has holes or noiseApply cv2.MORPH_CLOSE before contour detection; increase Gaussian blur sigma
GaussianBlur requires odd kernelEven kernel size providedAlways use odd kernel sizes: 3, 5, 7, 9; ksize=(5,5) not (4,4)
CLAHE makes image worseclipLimit too highReduce clipLimit to 1.5–2.0; increase tileGridSize to (16,16)
Background subtraction removes cellsHistory too short for MOG2Increase history parameter; use static frame subtraction for microscopy
Performance slow on large imagesPython loop over pixelsUse vectorized NumPy operations or CUDA-accelerated cv2.cuda module

References

PyImageJ — Python Bridge to ImageJ/Fiji

Overview

PyImageJ provides a Python interface to ImageJ2 and Fiji through PyJNIus and scyjava, embedding a full Java Virtual Machine inside a Python process. It enables bidirectional data exchange between NumPy arrays and ImageJ's ImagePlus/ImgLib2 data structures, so you can preprocess images in Python, pass them into Fiji plugins (Bio-Formats, TrackMate, Analyze Particles, Weka segmentation), and return results back to pandas DataFrames. The library supports headless operation for scripting and batch processing, as well as GUI mode for interactive Fiji sessions.

When to Use

  • Running Fiji-specific plugins from Python: Bio-Formats multi-format I/O, TrackMate particle tracking, CLIJ2 GPU processing, or community Fiji update site plugins
  • Automating ImageJ macro pipelines headlessly without opening the Fiji GUI, e.g., batch processing an entire experiment overnight
  • Applying the ImageJ Ops framework (150+ image processing operations) with the full ImageJ type system
  • Converting between NumPy arrays (SciPy ecosystem) and ImageJ hyperstacks (TZCYX channel order) for round-trip processing
  • Parsing ImageJ Results tables and ROI Manager measurements into pandas DataFrames for downstream statistical analysis
  • Executing existing .ijm macro files as part of a Python workflow without rewriting them
  • Use scikit-image instead when you need pure Python processing without Fiji plugins — scikit-image is faster to install and avoids JVM overhead
  • Use napari instead for interactive multi-dimensional image visualization and annotation; PyImageJ does not replace a viewer

Prerequisites

  • Python packages: pyimagej, scyjava, numpy, pandas
  • Java: Java 8 or Java 11 (Java 17 is not supported); use conda for reliable Java management
  • Fiji/ImageJ2: Downloaded automatically on first init, or specify a local Fiji installation path
  • Environment: conda environment strongly recommended; pip-only installs often have JVM path issues
# Recommended: conda installation
conda create -n pyimagej -c conda-forge pyimagej openjdk=11
conda activate pyimagej

# Install additional dependencies
pip install pandas tifffile

# Verify
python -c "import imagej; ij = imagej.init('sc.fiji:fiji', mode='headless'); print(ij.getVersion())"

Quick Start

import imagej
import numpy as np

# Initialize Fiji in headless mode (downloads on first run, ~500 MB)
ij = imagej.init("sc.fiji:fiji", mode="headless")
print(f"ImageJ version: {ij.getVersion()}")

# Create a test image, process with Gaussian blur via Ops, convert back
arr = np.random.randint(0, 1000, (256, 256), dtype=np.uint16)
imp = ij.py.to_imageplus(arr)
blurred = ij.op().filter().gauss(imp.getProcessor(), 2.0)
result = ij.py.from_imageplus(imp)
print(f"Processed array shape: {result.shape}, dtype: {result.dtype}")

Core API

Module 1: Initialization

PyImageJ must be initialized once per Python session. The mode and endpoint determine which ImageJ distribution and GUI behavior to use.

import imagej

# Headless Fiji — most common for scripts and batch jobs
ij = imagej.init("sc.fiji:fiji", mode="headless")

# GUI mode — opens the Fiji window (requires a display)
ij = imagej.init("sc.fiji:fiji", mode="gui")

# Local Fiji installation — faster startup, no download
ij = imagej.init("/path/to/Fiji.app", mode="headless")

# Specific Fiji version
ij = imagej.init("sc.fiji:fiji:2.14.0", mode="headless")

# Bare ImageJ2 without Fiji plugins
ij = imagej.init("net.imagej:imagej", mode="headless")

print(f"ImageJ version: {ij.getVersion()}")
print(f"Headless: {ij.ui().isHeadless()}")

Module 2: Image I/O

Open and save images using ImageJ's I/O layer (which includes Bio-Formats for proprietary formats) and convert between ImageJ and NumPy representations.

import imagej
import numpy as np

ij = imagej.init("sc.fiji:fiji", mode="headless")

# Open any format Bio-Formats supports: CZI, LIF, ND2, ICS, TIFF, etc.
imp = ij.io().open("/data/experiment.czi")
print(f"Dimensions: {imp.getDimensions()}")   # [W, H, C, Z, T]
print(f"nSlices: {imp.getNSlices()}, nFrames: {imp.getNFrames()}")

# Save image
ij.io().save(imp, "/data/output.tif")
print("Saved output.tif")
# NumPy ↔ ImageJ conversion
arr = np.zeros((100, 100), dtype=np.uint16)
arr[30:70, 30:70] = 1000   # bright square

# NumPy → ImagePlus
imp = ij.py.to_imageplus(arr)
print(f"ImagePlus: {imp.getWidth()}×{imp.getHeight()}, type={imp.getType()}")

# ImagePlus → NumPy (returns a view where possible)
arr_back = ij.py.from_imageplus(imp)
print(f"NumPy array: shape={arr_back.shape}, dtype={arr_back.dtype}")

# Multi-channel array: shape (C, H, W)
rgb = np.random.randint(0, 255, (3, 256, 256), dtype=np.uint8)
imp_rgb = ij.py.to_imageplus(rgb)
print(f"Channels: {imp_rgb.getNChannels()}")

Module 3: Macro Execution

Run ImageJ macro language (IJM) snippets or macro files. Macros execute inside the ImageJ environment and can call any built-in ImageJ command.

import imagej

ij = imagej.init("sc.fiji:fiji", mode="headless")

# Run an inline macro string
ij.macro.run("print('Hello from ImageJ macro');")

# Run a macro with options string (key=value pairs)
# Options string mirrors the dialog parameters of ImageJ commands
macro_code = """
run("Gaussian Blur...", "sigma=2");
run("Auto Threshold", "method=Otsu white");
"""
ij.macro.run(macro_code)

# Run a macro file from disk
ij.macro.runMacroFile("/scripts/my_analysis.ijm")

# Run macro that returns a value via getResult or output string
result = ij.macro.run("""
x = 42 * 2;
return x;
""")
print(f"Macro returned: {result}")
# Macro with current image: open → process → measure
ij.io().open("/data/cells.tif")   # sets current active image

measure_macro = """
run("Set Measurements...", "area mean min integrated redirect=None decimal=3");
run("Analyze Particles...", "size=50-Infinity display clear summarize");
"""
ij.macro.run(measure_macro)
print("Analyze Particles complete; results in Results table")

Module 4: ImageJ Ops

ImageJ Ops is a framework of 150+ image processing operations with type-safe dispatch. Ops work on ImgLib2 Img objects and are the preferred way to call image processing algorithms programmatically.

import imagej
import numpy as np

ij = imagej.init("sc.fiji:fiji", mode="headless")

arr = np.random.randint(100, 900, (512, 512), dtype=np.uint16)
img = ij.py.to_java(arr)   # converts to ImgLib2 RandomAccessibleInterval

# Gaussian blur
blurred = ij.op().filter().gauss(img, 2.0)
blurred_np = ij.py.from_java(blurred)
print(f"Blurred: {blurred_np.shape}")

# Otsu threshold → binary image
binary = ij.op().threshold().otsu(img)
binary_np = ij.py.from_java(binary)
print(f"Binary unique values: {np.unique(binary_np)}")

# Morphological operations
from jnius import autoclass
BitType = autoclass("net.imglib2.type.logic.BitType")
opened = ij.op().morphology().open(binary, [3, 3])
opened_np = ij.py.from_java(opened)
print(f"After opening: {opened_np.shape}")
# Statistics ops
mean_val = ij.op().stats().mean(img)
std_val  = ij.op().stats().stdDev(img)
print(f"Mean intensity: {mean_val:.1f}, StdDev: {std_val:.1f}")

# Math ops: multiply image by scalar
scaled = ij.op().math().multiply(img, ij.py.to_java(2.0))
print(f"Scaled max: {ij.py.from_java(scaled).max()}")

Module 5: Plugin and Command Calls

SciJava commands are the primary way to invoke Fiji plugins programmatically. Commands accept a dict of named parameters mirroring the plugin dialog.

import imagej

ij = imagej.init("sc.fiji:fiji", mode="headless")

# Open a file using Bio-Formats opener command
future = ij.command().run(
    "loci.plugins.LociImporter",
    True,
    {"id": "/data/image.lif", "open_files": True, "autoscale": True}
)
module = future.get()
imp = module.getOutput("imp")
print(f"Opened via Bio-Formats: {imp.getDimensions()}")
# Run Analyze Particles as a SciJava command
ij.io().open("/data/binary_mask.tif")

future = ij.command().run(
    "ij.plugin.filter.ParticleAnalyzer",
    True,
    {
        "minSize": 50.0,
        "maxSize": float("inf"),
        "options": 0,      # SHOW_NONE
        "measurements": 1,  # AREA
    }
)
future.get()
print("Analyze Particles command complete")

# Alternatively, run via macro string for simpler plugin invocation
ij.macro.run("""
run("Analyze Particles...", "size=50-Infinity display clear summarize");
""")

Module 6: Results Table and ROI Analysis

Retrieve measurement results from ImageJ's Results table and ROI Manager after running Analyze Particles or other measurement commands.

import imagej
import pandas as pd

ij = imagej.init("sc.fiji:fiji", mode="headless")

# After running Analyze Particles, read the Results table
def results_to_dataframe(ij) -> pd.DataFrame:
    """Convert ImageJ Results table to pandas DataFrame."""
    rt = ij.ResultsTable.getResultsTable()
    if rt is None or rt.size() == 0:
        return pd.DataFrame()
    headings = list(rt.getHeadings())
    data = {col: [rt.getValue(col, i) for i in range(rt.size())]
            for col in headings}
    return pd.DataFrame(data)

# Run segmentation + measurement macro
ij.io().open("/data/cells.tif")
ij.macro.run("""
run("Gaussian Blur...", "sigma=1.5");
setAutoThreshold("Otsu dark");
run("Convert to Mask");
run("Analyze Particles...", "size=20-Infinity display clear");
""")

df = results_to_dataframe(ij)
print(f"Found {len(df)} objects")
print(df[["Area", "Mean", "IntDen"]].describe())
df.to_csv("particle_measurements.csv", index=False)
print("Saved particle_measurements.csv")
# Access the ROI Manager
def get_roi_manager(ij):
    """Return the ImageJ ROI Manager instance, creating if needed."""
    RoiManager = ij.py.jclass("ij.plugin.frame.RoiManager")
    rm = RoiManager.getInstance()
    if rm is None:
        rm = RoiManager(False)   # headless=False means no GUI window
    return rm

rm = get_roi_manager(ij)
roi_count = rm.getCount()
print(f"ROIs in manager: {roi_count}")

# Extract bounding boxes for all ROIs
rois = []
for i in range(roi_count):
    roi = rm.getRoi(i)
    bounds = roi.getBounds()
    rois.append({"index": i, "x": bounds.x, "y": bounds.y,
                 "width": bounds.width, "height": bounds.height})
roi_df = pd.DataFrame(rois)
print(roi_df.head())

Common Workflows

Workflow 1: Automated Fluorescence Quantification

Goal: Open a multi-channel TIFF stack, apply Gaussian blur, threshold nuclei channel, run Analyze Particles, and export per-cell measurements as CSV.

import imagej
import pandas as pd
import numpy as np
from pathlib import Path

ij = imagej.init("sc.fiji:fiji", mode="headless")

def quantify_nuclei(tiff_path: str, output_csv: str,
                    channel: int = 1, sigma: float = 1.5,
                    min_size: int = 50) -> pd.DataFrame:
    """
    Segment and measure nuclei in a fluorescence TIFF.

    Parameters
    ----------
    tiff_path  : path to single- or multi-channel TIFF
    output_csv : where to save results
    channel    : 1-based channel index for nuclear stain (e.g., DAPI)
    sigma      : Gaussian blur radius in pixels
    min_size   : minimum nucleus area in pixels
    """
    # Step 1: Open image
    imp = ij.io().open(tiff_path)
    print(f"Loaded: {Path(tiff_path).name}  dims={imp.getDimensions()}")

    # Step 2: Extract channel if multi-channel
    if imp.getNChannels() > 1:
        imp.setC(channel)

    # Step 3: Apply Gaussian blur and threshold via macro
    ij.macro.run(f"""
selectWindow("{imp.getTitle()}");
run("Gaussian Blur...", "sigma={sigma}");
setAutoThreshold("Otsu dark");
run("Convert to Mask");
run("Fill Holes");
run("Watershed");
""")

    # Step 4: Measure
    ij.macro.run(f"""
run("Set Measurements...", "area mean min centroid integrated shape redirect=None decimal=3");
run("Analyze Particles...", "size={min_size}-Infinity display clear include summarize");
""")

    # Step 5: Collect results
    rt = ij.ResultsTable.getResultsTable()
    if rt is None or rt.size() == 0:
        print("No objects detected")
        return pd.DataFrame()

    headings = list(rt.getHeadings())
    df = pd.DataFrame(
        {col: [rt.getValue(col, i) for i in range(rt.size())]
         for col in headings}
    )
    df["source_file"] = Path(tiff_path).stem
    df.to_csv(output_csv, index=False)
    print(f"Saved {len(df)} measurements → {output_csv}")
    return df


df = quantify_nuclei(
    tiff_path="/data/experiment_dapi.tif",
    output_csv="nuclei_measurements.csv",
    channel=1,
    sigma=1.5,
    min_size=50
)
print(df[["Area", "Mean", "Circ."]].describe())

Workflow 2: Batch Fiji Macro Processing

Goal: Process a folder of images with an existing Fiji .ijm macro file, collect the Results table from each image into a single DataFrame.

import imagej
import pandas as pd
from pathlib import Path

ij = imagej.init("sc.fiji:fiji", mode="headless")

def run_macro_on_image(ij, image_path: str, macro_file: str) -> pd.DataFrame:
    """Open one image, run a macro file, return its Results table."""
    ij.io().open(image_path)

    # Clear any previous results before running
    ij.macro.run("run(\"Clear Results\");")

    # Run macro file (macro must operate on the active image)
    ij.macro.runMacroFile(macro_file)

    rt = ij.ResultsTable.getResultsTable()
    if rt is None or rt.size() == 0:
        return pd.DataFrame()
    headings = list(rt.getHeadings())
    return pd.DataFrame(
        {col: [rt.getValue(col, i) for i in range(rt.size())]
         for col in headings}
    )


input_dir  = Path("/data/images")
macro_file = "/scripts/measure_cells.ijm"
output_csv = "batch_results.csv"

all_results = []
image_files = sorted(input_dir.glob("*.tif"))

for img_path in image_files:
    print(f"Processing: {img_path.name}")
    df = run_macro_on_image(ij, str(img_path), macro_file)
    if not df.empty:
        df["filename"] = img_path.name
        all_results.append(df)

    # Close all windows to free memory between images
    ij.macro.run("close('*');")

if all_results:
    combined = pd.concat(all_results, ignore_index=True)
    combined.to_csv(output_csv, index=False)
    print(f"Batch complete: {len(image_files)} images, "
          f"{len(combined)} total measurements → {output_csv}")
else:
    print("No results collected from any image")

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
modeInitialization"headless""headless", "gui", "interactive"Controls whether Fiji GUI window opens; use "headless" for scripts
endpointInitialization"sc.fiji:fiji"Maven coordinate or /path/to/Fiji.appSelects ImageJ2 distribution; sc.fiji:fiji includes all Fiji plugins
sigma (Gaussian blur)Macro / Ops2.00.5–10.0Spatial smoothing radius in pixels; higher reduces noise but blurs edges
minSize (Analyze Particles)Plugin / Macro0pixels² or 0–InfinitySmallest object area to include; eliminates noise particles
method (Auto Threshold)Macro / Ops"Otsu""Otsu", "Triangle", "MaxEntropy", "Huang", "Li"Threshold algorithm; Otsu works well for bimodal histograms
measurements bitmaskResults / MacrovariesOR combination of AREA=1, MEAN=2, CENTROID=4, etc.Selects which columns appear in the Results table
Java heap sizeInitialization (env)JVM default (~25% RAM)set via JAVA_TOOL_OPTIONSLimits memory for large stacks; set -Xmx8g for big images

Best Practices

  1. Initialize once per session and reuse ij: Starting a new JVM is expensive (5–15 seconds). Create ij at module level or pass it as a parameter rather than calling imagej.init() inside a loop.

    # At module top level — initialized once
    import imagej
    ij = imagej.init("sc.fiji:fiji", mode="headless")
    
    def process(path):
        imp = ij.io().open(path)   # reuse ij
        ...
    
  2. Use conda for Java management: pip-only installs frequently fail because JAVA_HOME is not set or the wrong JDK version is on PATH. A conda environment with openjdk=11 avoids 90% of JVM-not-found errors.

  3. Clear Results and ROI Manager between images in batch loops: ImageJ accumulates results across calls in the same session. Always call run("Clear Results") and rm.reset() before each image to prevent row contamination.

    ij.macro.run("run('Clear Results');")
    rm = get_roi_manager(ij)
    rm.reset()
    
  4. Prefer ij.macro.run() for simple commands over ij.command().run(): Macro strings are shorter, easier to read, and use the same syntax as the Fiji Macro Recorder. Use ij.command().run() only when you need programmatic access to command outputs (module return values).

  5. Convert to NumPy as late as possible: ij.py.from_imageplus() copies data from the JVM to Python. For multi-step processing inside ImageJ, keep data in ImagePlus or ImgLib2 form and convert only at the end to minimize memory-copy overhead.

  6. Use ij.py.to_java() / ij.py.from_java() for ImgLib2 Ops: The to_imageplus() / from_imageplus() pair works with the classic ImageProcessor; to_java() / from_java() target the modern ImgLib2 type system required by ij.op().

  7. Set a Fiji update site plugins list during init for reproducibility: Specify a pinned Fiji version (sc.fiji:fiji:2.14.0) rather than sc.fiji:fiji (latest) so your pipeline behavior does not change when Fiji releases new plugin updates.

Common Recipes

Recipe: TrackMate Headless Spot Detection

When to use: Run TrackMate particle tracking programmatically and retrieve detected spots as a DataFrame without opening the TrackMate GUI.

import imagej
import pandas as pd

ij = imagej.init("sc.fiji:fiji", mode="headless")

# TrackMate headless via macro (scripting interface)
trackmate_macro = """
run("TrackMate", "");
// For fully scripted TrackMate, use the Scripting Interface
// documented at https://imagej.net/plugins/trackmate/scripting
"""

# Scripted TrackMate via Jython-style Java interop
def run_trackmate_headless(ij, imp, radius=3.0, threshold=100.0):
    """Detect spots with LoG detector; return DataFrame of spot coordinates."""
    from jnius import autoclass

    # Import TrackMate classes
    Model          = autoclass("fiji.plugin.trackmate.Model")
    Settings       = autoclass("fiji.plugin.trackmate.Settings")
    TrackMate      = autoclass("fiji.plugin.trackmate.TrackMate")
    LogDetectorFactory = autoclass(
        "fiji.plugin.trackmate.detection.LogDetectorFactory")

    model    = Model()
    settings = Settings(imp)

    # Configure LoG spot detector
    settings.detectorFactory = LogDetectorFactory()
    settings.detectorSettings = {
        "DO_SUBPIXEL_LOCALIZATION": True,
        "RADIUS": radius,
        "TARGET_CHANNEL": 1,
        "THRESHOLD": threshold,
        "DO_MEDIAN_FILTERING": False,
    }

    tm = TrackMate(model, settings)
    tm.process()

    # Extract spot table
    spots = model.getSpots()
    spots.setVisible(True)
    records = []
    for spot in spots.iterable(True):
        records.append({
            "id":       spot.ID(),
            "x":        spot.getDoublePosition(0),
            "y":        spot.getDoublePosition(1),
            "z":        spot.getDoublePosition(2),
            "frame":    spot.getFeature("FRAME"),
            "quality":  spot.getFeature("QUALITY"),
        })
    return pd.DataFrame(records)


imp = ij.io().open("/data/timelapse.tif")
df  = run_trackmate_headless(ij, imp, radius=3.0, threshold=50.0)
print(f"Detected {len(df)} spots across {df['frame'].nunique()} frames")
df.to_csv("spots.csv", index=False)
print(df.head())

Recipe: Convert ImageJ Hyperstack to NumPy 5D Array (TZCYX)

When to use: Import a multi-dimensional Fiji hyperstack into Python as a 5D NumPy array with the standard TZCYX axis order used by most scientific image analysis libraries.

import imagej
import numpy as np

ij = imagej.init("sc.fiji:fiji", mode="headless")

def hyperstack_to_numpy(ij, imp) -> np.ndarray:
    """
    Convert an ImageJ hyperstack to a NumPy array with shape (T, Z, C, Y, X).

    ImageJ internal order is C-Z-T (slowest to fastest in stack index).
    This function reorders to the TZCYX convention used by tifffile, OME, etc.
    """
    nC = imp.getNChannels()
    nZ = imp.getNSlices()
    nT = imp.getNFrames()
    H  = imp.getHeight()
    W  = imp.getWidth()

    arr = np.zeros((nT, nZ, nC, H, W), dtype=np.uint16)

    for t in range(1, nT + 1):
        for z in range(1, nZ + 1):
            for c in range(1, nC + 1):
                idx = imp.getStackIndex(c, z, t)
                imp.setSlice(idx)
                arr[t-1, z-1, c-1] = ij.py.from_imageplus(imp)

    return arr


imp = ij.io().open("/data/4d_experiment.tif")
print(f"ImageJ dims (W,H,C,Z,T): {imp.getDimensions()}")

stack = hyperstack_to_numpy(ij, imp)
print(f"NumPy TZCYX shape: {stack.shape}")  # e.g., (10, 15, 2, 512, 512)
print(f"dtype: {stack.dtype}, max: {stack.max()}")

# Save as OME-TIFF with correct axis metadata
import tifffile
tifffile.imwrite(
    "output_TZCYX.ome.tif",
    stack,
    imagej=True,
    metadata={"axes": "TZCYX"},
    photometric="minisblack"
)
print("Saved output_TZCYX.ome.tif")

Troubleshooting

ProblemCauseSolution
JVMNotFoundException on initJAVA_HOME not set or wrong JDK versionInstall via conda: conda install -c conda-forge openjdk=11; avoid Java 17
RuntimeError: Fiji download failedNo internet or corporate proxyDownload Fiji manually from fiji.sc, then use imagej.init("/path/to/Fiji.app")
java.lang.OutOfMemoryError on large stacksJVM default heap is too smallSet export JAVA_TOOL_OPTIONS="-Xmx8g" before importing imagej
Macro run(...) silently does nothingNo active image when macro expects oneCall ij.io().open(path) before running processing macros; check ij.WindowManager.getImageCount()
Results table empty after Analyze ParticlesThreshold not applied, or mask not binaryVerify mask is 8-bit binary (0/255) with imp.getType() == 0; run Convert to Mask before Analyze Particles
AttributeError: 'NoneType' object on ij.ResultsTable.getResultsTable()No measurements run yet in this sessionConfirm Analyze Particles macro completed; run ij.macro.run("print(nResults);") to check count
Plugin class not found (ClassNotFoundException)Plugin not in this Fiji installationAdd the Fiji update site (e.g., TrackMate) or use sc.fiji:fiji endpoint which includes all default plugins
gui mode crashes with HeadlessExceptionNo display available (SSH/cluster)Use mode="headless" for remote environments; GUI mode requires DISPLAY or X11 forwarding

Related Skills

  • scikit-image-processing — pure Python image processing without JVM; use when Fiji plugins are not needed
  • napari-image-viewer — interactive multi-dimensional image viewer for Python; complement to PyImageJ for visualization
  • trackpy-particle-tracking — Python-native Crocker-Grier SPT; alternative to TrackMate for simple 2D tracking
  • cellpose-cell-segmentation — deep learning cell segmentation; can be run standalone or as a Fiji plugin
  • omero-integration — OMERO server image management; PyImageJ can process images retrieved via omero-py

References

scikit-image — Scientific Image Processing

Overview

scikit-image is a Python library for image processing in the SciPy ecosystem. It provides algorithms for reading/writing images, filtering (noise reduction, edge detection), geometric transforms, segmentation (thresholding, watershed, active contours), object measurement (area, intensity, shape descriptors), and feature detection. Images are represented as NumPy arrays, enabling seamless integration with NumPy, SciPy, matplotlib, and pandas. Widely used for fluorescence microscopy, histology, and general bioimage analysis.

When to Use

  • Preprocessing fluorescence microscopy images: background subtraction, denoising, illumination correction
  • Segmenting cells, nuclei, or organelles using thresholding or watershed
  • Measuring object properties: area, perimeter, intensity statistics, shape descriptors
  • Applying morphological operations: erosion, dilation, opening, closing, fill holes
  • Detecting keypoints or local features in biological images
  • Converting between image formats and color spaces
  • Use OpenCV instead for real-time video processing or GPU-accelerated operations
  • For deep-learning cell segmentation, use CellPose instead (better accuracy for touching cells)
  • Use napari instead for interactive multi-dimensional image visualization and annotation
  • For whole-slide image tiling, use PathML or histolab instead

Prerequisites

  • Python packages: scikit-image, numpy, scipy, matplotlib
  • Input requirements: Images as files (TIFF, PNG, JPEG) or NumPy arrays; fluorescence images as 2D/3D grayscale arrays
  • Environment: Python 3.9+
pip install scikit-image numpy scipy matplotlib

# For reading proprietary microscopy formats
pip install tifffile aicsimageio

# Verify
python -c "import skimage; print(skimage.__version__)"

Quick Start

from skimage import io, filters, measure
import numpy as np

# Load → denoise → threshold → measure
img = io.imread("cells.tif")
img_smooth = filters.gaussian(img, sigma=1.5)
threshold = filters.threshold_otsu(img_smooth)
binary = img_smooth > threshold

regions = measure.regionprops(measure.label(binary))
print(f"Found {len(regions)} objects")
print(f"Mean area: {np.mean([r.area for r in regions]):.1f} px²")

Core API

Module 1: Image I/O and Data Types

from skimage import io, img_as_float, img_as_uint
import numpy as np

# Read single image
img = io.imread("nuclei.tif")
print(f"Shape: {img.shape}, dtype: {img.dtype}")  # (512, 512), uint16

# Read image collection from directory
from skimage import io as ski_io
images = ski_io.ImageCollection("data/*.tif")
print(f"Loaded {len(images)} images")

# Type conversions (critical for correct arithmetic)
img_f = img_as_float(img)      # uint16 → float64, range [0, 1]
img_u8 = (img_f * 255).astype(np.uint8)  # → 8-bit

# Save image
io.imsave("output.tif", img_u8)
# Multi-channel fluorescence (TIFF with CZYX or ZCYX dims)
import tifffile

stack = tifffile.imread("multichannel.tif")  # shape: (C, Z, Y, X)
dapi = stack[0]   # DAPI channel
gfp = stack[1]    # GFP channel
print(f"DAPI: {dapi.shape}, GFP: {gfp.shape}")

# Maximum intensity projection along Z
mip = dapi.max(axis=0)
io.imsave("dapi_mip.tif", mip)

Module 2: Filters and Preprocessing

from skimage import filters, restoration
import numpy as np

# Gaussian blur (denoising, smoothing)
from skimage.filters import gaussian
smoothed = gaussian(img, sigma=2.0)

# Median filter (salt-and-pepper noise removal)
from skimage.filters import median
from skimage.morphology import disk
denoised = median(img, footprint=disk(3))

# Top-hat transform (background subtraction for uneven illumination)
from skimage.morphology import white_tophat, disk
background_removed = white_tophat(img, footprint=disk(50))
print(f"Background removed: range [{background_removed.min()}, {background_removed.max()}]")
# Edge detection
from skimage.filters import sobel, laplace, prewitt

edges_sobel = sobel(img_as_float(img))
edges_laplace = laplace(img_as_float(img))

# Difference of Gaussians (blob-like structure detection)
from skimage.filters import difference_of_gaussians
blob_enhanced = difference_of_gaussians(img_as_float(img), low_sigma=1, high_sigma=3)

# Contrast enhancement (CLAHE: local histogram equalization)
from skimage.exposure import equalize_adapthist
enhanced = equalize_adapthist(img_as_float(img), clip_limit=0.03)

Module 3: Thresholding and Segmentation

from skimage import filters, morphology, segmentation
from skimage.color import label2rgb
import numpy as np

# Automatic thresholding methods
from skimage.filters import (threshold_otsu, threshold_li,
                              threshold_triangle, threshold_yen)

img_f = img_as_float(img)
print(f"Otsu: {threshold_otsu(img_f):.3f}")
print(f"Li: {threshold_li(img_f):.3f}")

# Apply threshold and clean binary mask
binary = img_f > threshold_otsu(img_f)
binary_clean = morphology.remove_small_objects(binary, min_size=50)
binary_filled = morphology.remove_small_holes(binary_clean, area_threshold=100)
# Watershed segmentation (separate touching objects)
from skimage.segmentation import watershed
from skimage.feature import peak_local_max
from scipy import ndimage as ndi

# Distance transform → local maxima → watershed
distance = ndi.distance_transform_edt(binary_filled)
coords = peak_local_max(distance, min_distance=20, labels=binary_filled)
mask = np.zeros(distance.shape, dtype=bool)
mask[tuple(coords.T)] = True
markers = ndi.label(mask)[0]
labels = watershed(-distance, markers, mask=binary_filled)

print(f"Segmented objects: {labels.max()}")
overlay = label2rgb(labels, image=img_f, bg_label=0)

Module 4: Morphological Operations

from skimage.morphology import (erosion, dilation, opening, closing,
                                 disk, ball, binary_erosion, binary_dilation)

# Erosion and dilation
eroded = erosion(binary, footprint=disk(3))
dilated = dilation(binary, footprint=disk(5))

# Opening: erosion then dilation (removes small objects, smooths edges)
opened = opening(binary, footprint=disk(3))

# Closing: dilation then erosion (fills small holes)
closed = closing(binary, footprint=disk(5))

# Skeletonization
from skimage.morphology import skeletonize
skeleton = skeletonize(binary)
print(f"Skeleton pixels: {skeleton.sum()}")

Module 5: Measurement and Region Properties

from skimage import measure
import pandas as pd

# Label connected components
labeled = measure.label(binary_filled)

# Extract region properties
props = measure.regionprops(labeled, intensity_image=img_as_float(img))

# Convert to DataFrame
data = []
for r in props:
    data.append({
        "label": r.label,
        "area": r.area,
        "perimeter": r.perimeter,
        "eccentricity": r.eccentricity,
        "mean_intensity": r.mean_intensity,
        "max_intensity": r.max_intensity,
        "centroid_y": r.centroid[0],
        "centroid_x": r.centroid[1],
        "bbox": r.bbox,
    })

df = pd.DataFrame(data)
print(f"Objects: {len(df)}")
print(df[["area", "mean_intensity", "eccentricity"]].describe().round(2))
# Filter by property thresholds
cells = df[(df["area"] > 100) & (df["area"] < 5000) & (df["eccentricity"] < 0.9)]
print(f"Valid cells: {len(cells)}")

# Measure co-localization: fraction of channel-1 signal in channel-2 positive mask
from skimage.measure import regionprops_table
import numpy as np

# For multi-channel images
table = regionprops_table(
    labeled, intensity_image=np.stack([dapi, gfp], axis=-1),
    properties=["label", "area", "mean_intensity"]
)

Module 6: Feature Detection and Transforms

from skimage.feature import blob_log, blob_dog, corner_harris, corner_peaks
from skimage import transform

# Laplacian of Gaussian blob detection (nuclei, puncta)
blobs = blob_log(img_as_float(img), min_sigma=5, max_sigma=20,
                  num_sigma=5, threshold=0.05)
print(f"Blobs detected: {len(blobs)}")
# blobs columns: [y, x, sigma] where radius = sqrt(2) * sigma

# Difference of Gaussians (faster alternative)
blobs_dog = blob_dog(img_as_float(img), min_sigma=5, max_sigma=20, threshold=0.02)
# Geometric transforms
from skimage import transform

# Rescale
img_small = transform.rescale(img_as_float(img), 0.5)

# Rotate
img_rotated = transform.rotate(img_as_float(img), angle=15, resize=True)

# Affine registration (align two images)
from skimage.registration import phase_cross_correlation
shift, error, _ = phase_cross_correlation(ref_img, moving_img)
print(f"Alignment shift: {shift} px, error: {error:.4f}")

Key Concepts

Image Arrays and Conventions

scikit-image represents images as NumPy arrays. Shape conventions:

Image TypeShapedtype
Grayscale 2D(H, W)uint8, uint16, float64
RGB color(H, W, 3)uint8
Multichannel(H, W, C)any
Z-stack(Z, H, W)any

dtype matters: Most algorithms expect float64 in [0, 1]. Use img_as_float(img) before processing; convert back with img_as_uint(img) for saving.

Common Workflows

Workflow 1: Fluorescence Cell Segmentation and Measurement

Goal: Segment DAPI-stained nuclei and measure GFP fluorescence per nucleus.

from skimage import io, filters, morphology, measure, img_as_float
from skimage.segmentation import watershed
from skimage.feature import peak_local_max
from scipy import ndimage as ndi
import pandas as pd
import numpy as np
import tifffile

# Load 2-channel image (DAPI=ch0, GFP=ch1)
img = tifffile.imread("cells.tif")
dapi = img_as_float(img[0])
gfp = img_as_float(img[1])

# Segment nuclei from DAPI channel
dapi_smooth = filters.gaussian(dapi, sigma=2)
threshold = filters.threshold_otsu(dapi_smooth)
binary = dapi_smooth > threshold
binary = morphology.remove_small_objects(binary, min_size=200)
binary = morphology.remove_small_holes(binary, area_threshold=500)

# Watershed to separate touching nuclei
distance = ndi.distance_transform_edt(binary)
coords = peak_local_max(distance, min_distance=30, labels=binary)
mask = np.zeros_like(distance, dtype=bool)
mask[tuple(coords.T)] = True
markers = ndi.label(mask)[0]
labels = watershed(-distance, markers, mask=binary)

# Measure GFP per nucleus
props = measure.regionprops(labels, intensity_image=gfp)
df = pd.DataFrame([{
    "nucleus_id": p.label,
    "area_px2": p.area,
    "gfp_mean": p.mean_intensity,
    "gfp_max": p.max_intensity,
} for p in props])

df.to_csv("nucleus_measurements.csv", index=False)
print(f"Nuclei: {len(df)}, mean GFP: {df['gfp_mean'].mean():.3f}")

Workflow 2: Batch Image Processing

Goal: Apply the same preprocessing and measurement pipeline to a folder of images.

from pathlib import Path
from skimage import io, filters, measure, img_as_float, morphology
import pandas as pd

results = []
for img_path in sorted(Path("data/").glob("*.tif")):
    img = img_as_float(io.imread(img_path))
    if img.ndim == 3:
        img = img.mean(axis=-1)  # convert RGB to grayscale

    # Preprocess
    smooth = filters.gaussian(img, sigma=1.5)
    thresh = filters.threshold_otsu(smooth)
    binary = morphology.remove_small_objects(smooth > thresh, min_size=50)

    # Measure
    labeled = measure.label(binary)
    props = measure.regionprops(labeled, intensity_image=img)
    for p in props:
        results.append({
            "image": img_path.stem,
            "object_id": p.label,
            "area": p.area,
            "mean_intensity": p.mean_intensity,
        })

df = pd.DataFrame(results)
df.to_csv("batch_results.csv", index=False)
print(f"Processed {df['image'].nunique()} images, {len(df)} objects total")

Key Parameters

FunctionParameterDefaultRange/OptionsEffect
gaussiansigma1.00.5–10+Smoothing kernel size; larger = more blur
medianfootprintdisk(1)disk(1–10)Median filter neighborhood
threshold_otsu———Returns automatic threshold (no tuning)
remove_small_objectsmin_size64any integerRemove objects smaller than N pixels
watershed———Uses marker positions from peak_local_max
peak_local_maxmin_distance15–50Min separation between detected peaks (px)
blob_logmin_sigma / max_sigma1/50dependentExpected blob radius range
regionpropsintensity_imageNonearrayImage for intensity measurements

Best Practices

  1. Always check dtype before processing: Operations like subtraction on uint8 silently clip to 0. Convert to float: img = img_as_float(img) as the first step.

  2. Visualize intermediate results: For every segmentation pipeline, plot the binary mask overlaid on the original before measuring. Silent segmentation errors are the most common failure mode.

  3. Tune thresholds on representative samples: Otsu works well for bimodal histograms. For difficult images, compare Otsu/Li/Triangle with try_all_threshold(img) from skimage.

  4. Use watershed for touching objects: Simple thresholding cannot separate touching nuclei. Always apply watershed with distance transform markers for densely packed cells.

  5. Measure in physical units: Convert pixel measurements to microns using the pixel size from microscope metadata: area_um2 = area_px * pixel_size_um**2.

  6. Validate with known samples: Before batch processing, verify the pipeline on 3–5 images with manually counted objects. Spot-check object counts against expectations.

Common Recipes

Recipe: Measure Spot Intensity in Fluorescence Images

from skimage import io, filters, measure, img_as_float
from skimage.feature import blob_log
import numpy as np

img = img_as_float(io.imread("spots.tif"))
blobs = blob_log(img, min_sigma=2, max_sigma=8, num_sigma=5, threshold=0.05)

intensities = []
for y, x, sigma in blobs:
    r = int(np.ceil(np.sqrt(2) * sigma))
    region = img[max(0,int(y-r)):int(y+r), max(0,int(x-r)):int(x+r)]
    intensities.append({"y": y, "x": x, "radius": r, "mean_intensity": region.mean()})

import pandas as pd
df = pd.DataFrame(intensities)
print(f"Spots: {len(df)}, mean intensity: {df['mean_intensity'].mean():.4f}")

Recipe: Binary Mask from Multiple Channels

from skimage import filters, morphology, img_as_float
import numpy as np

# Segment objects positive in BOTH channels
ch1 = img_as_float(dapi)
ch2 = img_as_float(gfp)

mask1 = ch1 > filters.threshold_otsu(ch1)
mask2 = ch2 > filters.threshold_otsu(ch2)
combined_mask = mask1 & mask2  # objects in both channels

combined_mask = morphology.binary_closing(combined_mask)
print(f"Co-positive pixels: {combined_mask.sum()}")

Recipe: Save Labeled Overlay as Publication Figure

from skimage.color import label2rgb
from skimage import io, img_as_ubyte
import matplotlib.pyplot as plt

overlay = label2rgb(labels, image=img_as_float(dapi), bg_label=0, alpha=0.5)

fig, axes = plt.subplots(1, 2, figsize=(12, 6))
axes[0].imshow(dapi, cmap="gray")
axes[0].set_title(f"DAPI (raw)")
axes[1].imshow(overlay)
axes[1].set_title(f"Segmented ({labels.max()} nuclei)")
for ax in axes:
    ax.axis("off")
plt.tight_layout()
plt.savefig("segmentation_overlay.png", dpi=300, bbox_inches="tight")
print("Saved segmentation_overlay.png")

Troubleshooting

ProblemCauseSolution
OverflowError in arithmeticInteger overflow from uint8/uint16 opsConvert to float first: img = img_as_float(img)
Otsu threshold finds one classBimodal distribution absentTry threshold_li() or try_all_threshold(img) for comparison
Watershed over-segmentsMarkers too close togetherIncrease min_distance in peak_local_max; smooth distance map
All objects merged in binaryThreshold too lowCheck plt.hist(img.ravel()) for histogram; manually adjust
regionprops missing intensity statsintensity_image not providedPass: regionprops(labels, intensity_image=img)
3D images processed as 2DZ-stack not detectedCheck shape: img.shape; process per slice or use 3D functions
Tiny noise objects in binaryThreshold too aggressiveApply morphology.remove_small_objects(binary, min_size=50)

Related Skills

  • pathml — whole-slide image processing using scikit-image under the hood
  • matplotlib-scientific-plotting — visualizing segmentation results and measurement distributions
  • histolab-wsi-processing — scikit-image-compatible preprocessing for H&E WSI tiles

References

trackpy

Overview

trackpy is a Python library for single-particle tracking (SPT) in video microscopy. It implements the Crocker-Grier algorithm to locate bright spots in each frame with subpixel precision, then links those positions across frames into continuous trajectories. From trajectories, trackpy computes mean squared displacement (MSD), diffusion coefficients, and motion classifications (confined, normal, directed). It handles 2D fluorescence videos, 3D confocal z-stacks, and large image sequences via memory-efficient streaming through the pims image reader library.

When to Use

  • You have a fluorescence microscopy video of labeled particles (quantum dots, fluorescent beads, vesicles, receptors) and need to extract individual trajectories and diffusion coefficients.
  • You want to measure particle mobility: compute MSD curves and distinguish Brownian diffusion, directed motion, or confined motion from single-particle tracks.
  • You are analyzing colloid dynamics, lipid membrane diffusion, intracellular cargo transport, or virus-cell interactions where you need per-particle trajectory data.
  • You need 3D tracking from confocal z-stack time series to capture out-of-plane motion of particles or organelles.
  • You want to apply drift correction to remove stage drift before computing intrinsic particle motion statistics.
  • You need ensemble MSD averaged across hundreds of tracks to extract population-level diffusion behavior with statistical power.
  • Use TrackMate (Fiji/ImageJ plugin) instead when you need a graphical interface, manual curation of tracks, or integration with biological object segmenters (Cellpose, StarDist).
  • Use napari with napari-trackpy instead when you want interactive visualization and manual editing of trajectories alongside image data.

Prerequisites

  • Python packages: trackpy, pims, pandas, numpy, matplotlib, scipy
  • Data requirements: Grayscale or single-channel image sequence (TIF stack, AVI, or directory of PNG/TIF frames); particles should appear as bright Gaussian spots on a darker background (or use invert=True for dark spots on bright background)
  • Environment: Works in Jupyter notebooks and scripts; pims handles most microscopy formats; for ND2 or CZI files install pims-nd2 or aicsimageio
pip install trackpy pims pandas numpy matplotlib scipy
# For reading multi-channel or proprietary formats:
pip install pims[bioformats]   # Bioformats via JPype
pip install aicsimageio        # ND2, CZI, LIF via AICSImageIO

Quick Start

import trackpy as tp
import pims

# Load a TIF image stack (T frames × Y × X)
frames = pims.open("particles.tif")   # shape: (T, Y, X)

# Locate particles in all frames
f = tp.batch(frames, diameter=11, minmass=500)
print(f"Found {len(f)} particle detections across {f['frame'].nunique()} frames")

# Link into trajectories
t = tp.link(f, search_range=5, memory=3)

# Remove short-lived tracks (fewer than 10 frames)
t = tp.filter_stubs(t, threshold=10)
print(f"Retained {t['particle'].nunique()} trajectories")

# Compute ensemble MSD
imsd = tp.imsd(t, mpp=0.16, fps=10)   # mpp: microns per pixel, fps: frames per second
print(imsd.head())

Core API

Module 1: tp.locate() — Single-Frame Particle Detection

tp.locate() finds bright circular features in one image frame using a bandpass filter followed by local maximum detection. It returns a DataFrame with subpixel x/y positions, integrated mass, signal, and eccentricity for each detected particle.

import trackpy as tp
import pims
import matplotlib.pyplot as plt

frames = pims.open("particles.tif")
frame0 = frames[0]   # single 2D array

# Locate particles: diameter must be odd integer, roughly matching spot size in pixels
f0 = tp.locate(frame0, diameter=11, minmass=300, maxsize=None, separation=None)
print(f"Detected {len(f0)} particles in frame 0")
print(f0[['x', 'y', 'mass', 'size', 'ecc']].head())
# x, y: subpixel centroid; mass: integrated brightness; size: Gaussian width; ecc: eccentricity (0=circular)
# Diagnostic plot: annotate detected particles on the raw frame
fig, ax = plt.subplots(figsize=(8, 8))
tp.annotate(f0, frame0, ax=ax, imshow_style={"cmap": "gray"})
ax.set_title(f"Frame 0: {len(f0)} particles detected")
plt.tight_layout()
plt.savefig("locate_diagnostic.png", dpi=150)
print("Saved locate_diagnostic.png")

Module 2: tp.batch() — Multi-Frame Detection

tp.batch() applies tp.locate() to every frame in an image sequence and concatenates results into a single DataFrame with a frame column. It accepts any pims-compatible image reader or a list of 2D arrays.

import trackpy as tp
import pims

frames = pims.open("particles.tif")

# Locate particles across all frames (same parameters as tp.locate)
f = tp.batch(frames, diameter=11, minmass=300, processes=1)
# processes=1 uses serial processing; set processes="auto" for multicore (requires joblib)
print(f"Total detections: {len(f)}")
print(f"Frames with data: {f['frame'].nunique()} / {len(frames)}")
print(f"Mean particles per frame: {len(f)/f['frame'].nunique():.1f}")
print(f.groupby('frame').size().describe())
# Mass histogram: use to choose minmass cutoff
import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(6, 4))
f['mass'].hist(bins=40, ax=ax)
ax.axvline(300, color='red', linestyle='--', label='minmass=300')
ax.set_xlabel("Integrated mass")
ax.set_ylabel("Count")
ax.set_title("Mass distribution of detections")
ax.legend()
plt.tight_layout()
plt.savefig("mass_histogram.png", dpi=150)
print("Saved mass_histogram.png — use to refine minmass cutoff")

Module 3: tp.link() — Trajectory Linking

tp.link() connects particle detections across frames into trajectories by solving a bipartite assignment problem (Hungarian algorithm). It adds a particle column (integer trajectory ID) to the positions DataFrame. search_range (pixels) is the maximum displacement between frames; memory allows a particle to disappear for up to N frames before being dropped.

import trackpy as tp
import pims

frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=300)

# Link: search_range in pixels; memory handles brief disappearances (blinking, out-of-focus)
t = tp.link(f, search_range=5, memory=3)
print(f"Number of unique trajectories: {t['particle'].nunique()}")
print(f"Trajectory length distribution:")
print(t.groupby('particle').size().describe())
# Visualize all trajectories overlaid on the first frame
import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(8, 8))
tp.plot_traj(t, superimpose=frames[0], ax=ax)
ax.set_title(f"{t['particle'].nunique()} trajectories")
plt.tight_layout()
plt.savefig("trajectories.png", dpi=150)
print("Saved trajectories.png")

Module 4: tp.filter_stubs() — Short-Track Removal

tp.filter_stubs() removes trajectories shorter than a given number of frames. Short tracks arise from noise detections, particles entering/leaving the field of view, or linking errors. Removing them improves MSD reliability because short tracks contribute high-variance MSD estimates at long lag times.

import trackpy as tp
import pims

frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=300)
t = tp.link(f, search_range=5, memory=3)

before = t['particle'].nunique()
t_filt = tp.filter_stubs(t, threshold=10)   # keep only tracks with ≥10 frames
after = t_filt['particle'].nunique()
print(f"Tracks before filtering: {before}")
print(f"Tracks after filtering (≥10 frames): {after}")
print(f"Removed {before - after} short tracks ({100*(before-after)/before:.1f}%)")

Module 5: MSD Analysis — tp.imsd() and tp.emsd()

tp.imsd() computes per-particle mean squared displacement as a function of lag time, returning a DataFrame (lag time as index, particle ID as columns). tp.emsd() computes the ensemble-averaged MSD across all particles. Both require the physical scale (mpp, microns per pixel) and frame rate (fps).

import trackpy as tp
import pims
import matplotlib.pyplot as plt

frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=300)
t = tp.link(f, search_range=5, memory=3)
t = tp.filter_stubs(t, threshold=10)

mpp = 0.16    # microns per pixel (from microscope calibration)
fps = 10.0    # frames per second

# Individual MSD curves (one column per particle)
imsd = tp.imsd(t, mpp=mpp, fps=fps, max_lagtime=100)
print(f"IMSD shape: {imsd.shape}")  # (lag times) × (particles)

# Ensemble MSD
emsd = tp.emsd(t, mpp=mpp, fps=fps, max_lagtime=100)
print(f"EMSD at lag 1 s: {emsd.iloc[0]:.4f} µm²")
# Plot ensemble MSD and fit diffusion coefficient
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import linregress

mpp = 0.16
fps = 10.0

# Fit MSD = 4*D*t (2D Brownian) over first 10 lag times
lag_s = emsd.index.values[:10]   # lag times in seconds
msd_vals = emsd.values[:10]
slope, intercept, r, p, se = linregress(lag_s, msd_vals)
D = slope / 4   # diffusion coefficient in µm²/s
print(f"Diffusion coefficient D = {D:.4f} µm²/s  (R²={r**2:.3f})")

fig, ax = plt.subplots(figsize=(6, 5))
ax.plot(emsd.index, emsd.values, 'o-', label='Ensemble MSD')
ax.plot(lag_s, slope * lag_s + intercept, 'r--', label=f'Fit: D={D:.4f} µm²/s')
ax.set_xlabel("Lag time (s)")
ax.set_ylabel("MSD (µm²)")
ax.set_title("Ensemble Mean Squared Displacement")
ax.legend()
plt.tight_layout()
plt.savefig("emsd.png", dpi=150)
print("Saved emsd.png")

Module 6: Motion Analysis — Characterize and Drift Correction

tp.motion.characterize() computes per-trajectory statistics (mean velocity, net displacement, straightness). tp.subtract_drift() removes bulk stage drift from trajectories before MSD analysis.

import trackpy as tp
import pims

frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=300)
t = tp.link(f, search_range=5, memory=3)
t = tp.filter_stubs(t, threshold=10)

# Estimate and subtract drift (bulk movement of the sample/stage)
drift = tp.compute_drift(t)
print("Drift (first 5 frames):")
print(drift.head())

t_corrected = tp.subtract_drift(t.copy(), drift)
print(f"Drift subtracted from {t_corrected['particle'].nunique()} trajectories")
import trackpy as tp

# Characterize individual trajectories (requires tp.motion module)
from trackpy import motion

# Per-particle summary statistics
char = motion.characterize(t, mpp=0.16, fps=10.0)
print(char.columns.tolist())
# Columns: 'alpha' (anomalous exponent), 'D_app' (apparent diffusion), 'r^2' (fit quality)
print(char[['alpha', 'D_app']].describe())
# alpha ~ 1.0: Brownian; alpha < 1: confined/subdiffusion; alpha > 1: directed/superdiffusion

Common Workflows

Workflow 1: Full 2D Tracking Pipeline with MSD and Diffusion Coefficient

Goal: Load a fluorescence video, locate and link particles across all frames, filter short tracks, compute MSD, and extract diffusion coefficients.

import trackpy as tp
import pims
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import linregress

# ── 1. Load image sequence ──────────────────────────────────────────────────
frames = pims.open("fluorescence_video.tif")   # (T, Y, X) grayscale TIF stack
print(f"Loaded {len(frames)} frames, frame shape: {frames.frame_shape}")

# ── 2. Tune detection on a single frame ─────────────────────────────────────
f0 = tp.locate(frames[0], diameter=11, minmass=400)
print(f"Frame 0: {len(f0)} particles detected")
# Adjust diameter (odd integer ≥ spot size) and minmass until count looks right

# ── 3. Batch detect across all frames ───────────────────────────────────────
f = tp.batch(frames, diameter=11, minmass=400, processes=1)
print(f"Total detections: {len(f)} across {f['frame'].nunique()} frames")

# ── 4. Link into trajectories ────────────────────────────────────────────────
t = tp.link(f, search_range=6, memory=3)
print(f"Unique trajectories before filtering: {t['particle'].nunique()}")

# ── 5. Remove short trajectories ─────────────────────────────────────────────
t = tp.filter_stubs(t, threshold=15)
print(f"Trajectories after filtering (≥15 frames): {t['particle'].nunique()}")

# ── 6. Subtract stage drift ───────────────────────────────────────────────────
drift = tp.compute_drift(t)
t = tp.subtract_drift(t.copy(), drift)

# ── 7. Compute MSD ────────────────────────────────────────────────────────────
mpp = 0.16   # µm/pixel — from microscope calibration
fps = 10.0   # frames per second

emsd = tp.emsd(t, mpp=mpp, fps=fps, max_lagtime=50)
imsd = tp.imsd(t, mpp=mpp, fps=fps, max_lagtime=50)

# ── 8. Fit diffusion coefficient from linear regime (first 10 points) ─────────
n_fit = 10
lag_s = emsd.index.values[:n_fit]
msd_v = emsd.values[:n_fit]
slope, intercept, r, _, _ = linregress(lag_s, msd_v)
D = slope / 4   # MSD = 4Dt for 2D Brownian
print(f"Diffusion coefficient D = {D:.4f} µm²/s  (R²={r**2:.3f})")

# ── 9. Plot ────────────────────────────────────────────────────────────────────
fig, axes = plt.subplots(1, 2, figsize=(12, 5))

# Individual + ensemble MSD
axes[0].plot(imsd.index, imsd.values, alpha=0.2, color='steelblue', linewidth=0.8)
axes[0].plot(emsd.index, emsd.values, 'k-', linewidth=2, label='Ensemble MSD')
axes[0].plot(lag_s, slope * lag_s + intercept, 'r--', label=f'D={D:.4f} µm²/s')
axes[0].set_xlabel("Lag time (s)")
axes[0].set_ylabel("MSD (µm²)")
axes[0].set_title("MSD: individual (blue) + ensemble (black)")
axes[0].legend()

# Trajectory overlay
tp.plot_traj(t, superimpose=frames[0], ax=axes[1])
axes[1].set_title(f"{t['particle'].nunique()} trajectories")

plt.tight_layout()
plt.savefig("tracking_results.png", dpi=150, bbox_inches="tight")
print("Saved tracking_results.png")

# ── 10. Export trajectories ───────────────────────────────────────────────────
t.to_csv("trajectories.csv", index=False)
emsd.to_csv("ensemble_msd.csv")
print("Exported trajectories.csv and ensemble_msd.csv")

Workflow 2: 3D Particle Tracking from Confocal Z-Stacks

Goal: Track particles in 3D from a time-series of confocal z-stacks (T × Z × Y × X), link in 3D, and compute 3D MSD.

import trackpy as tp
import pims
import numpy as np
import matplotlib.pyplot as plt

# ── 1. Load 4D image stack (T × Z × Y × X) ─────────────────────────────────
# pims opens multi-page TIF; reshape into (T, Z, Y, X) as needed
raw = pims.open("confocal_3d_timeseries.tif")
# Assume each "frame" in pims is one Z-slice; reshape to (T, Z, Y, X)
T, n_z = 50, 20   # adjust to match acquisition
frames_4d = np.array(raw).reshape(T, n_z, raw.frame_shape[0], raw.frame_shape[1])
print(f"4D stack shape: {frames_4d.shape}")  # (T, Z, Y, X)

# ── 2. Detect in 3D (tp.locate works on 3D arrays) ─────────────────────────
# For 3D, pass a single 3D volume; diameter can be (z_diam, y_diam, x_diam)
f0_3d = tp.locate(frames_4d[0], diameter=(7, 11, 11), minmass=2000)
print(f"3D detections in t=0: {len(f0_3d)}")
print(f0_3d[['x', 'y', 'z', 'mass']].head())

# ── 3. Batch detect across all time points ──────────────────────────────────
detections = []
for t_idx in range(T):
    frame_3d = frames_4d[t_idx]    # Z × Y × X
    detected = tp.locate(frame_3d, diameter=(7, 11, 11), minmass=2000)
    detected['frame'] = t_idx
    detections.append(detected)

import pandas as pd
f3d = pd.concat(detections, ignore_index=True)
print(f"Total 3D detections: {len(f3d)}")

# ── 4. Link in 3D ────────────────────────────────────────────────────────────
# search_range in pixels; use a 3-tuple (z, y, x) for anisotropic voxels
t3d = tp.link(f3d, search_range=(3, 6, 6), memory=2)
t3d = tp.filter_stubs(t3d, threshold=10)
print(f"3D trajectories: {t3d['particle'].nunique()}")

# ── 5. Compute 3D MSD ────────────────────────────────────────────────────────
mpp_xy = 0.16   # µm/pixel in x, y
mpp_z = 0.30    # µm/pixel in z (z-step size)
fps = 1.0       # z-stack volume rate

# Scale z coordinates to µm
t3d_um = t3d.copy()
t3d_um['x'] *= mpp_xy
t3d_um['y'] *= mpp_xy
t3d_um['z'] *= mpp_z

emsd_3d = tp.emsd(t3d_um, mpp=1.0, fps=fps, max_lagtime=20)  # mpp=1 since already in µm
print(f"3D ensemble MSD (lag=1 s): {emsd_3d.iloc[0]:.4f} µm²")

# ── 6. Fit 3D diffusion coefficient (MSD = 6Dt for 3D) ─────────────────────
from scipy.stats import linregress
lag_s = emsd_3d.index.values[:8]
slope, _, r, _, _ = linregress(lag_s, emsd_3d.values[:8])
D_3d = slope / 6
print(f"3D Diffusion coefficient D = {D_3d:.4f} µm²/s  (R²={r**2:.3f})")

t3d.to_csv("trajectories_3d.csv", index=False)
print("Saved trajectories_3d.csv")

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
diameterlocate, batchrequiredodd integer ≥ 3 (or tuple for 3D)Approximate particle diameter in pixels; must be odd. Too small: split detections. Too large: merged detections
minmasslocate, batch1000 to ∞Minimum integrated brightness; primary filter against noise. Start at 0, plot mass histogram, set to separate noise peak
search_rangelinkrequired1–50 pixelsMax displacement between frames. Set to ~1.5× max expected per-frame movement
memorylink00–10 framesFrames a particle may be absent before track is broken; useful for blinking fluorophores
thresholdfilter_stubs1integer ≥ 1Minimum track length in frames; short tracks have unreliable MSD
max_lagtimeimsd, emsd100integerMaximum lag time in frames for MSD calculation; use ~10–20% of total frames for reliability
mppimsd, emsd1float > 0Microns per pixel; converts pixel units to physical units (µm)
fpsimsd, emsd1float > 0Frames per second; converts frame lag to seconds
separationlocate, batchdiameter+1integerMinimum center-to-center distance between features; prevents double-counting dense particles
invertlocate, batchFalseTrue, FalseSet True for dark particles on bright background (transmitted light imaging)

Best Practices

  1. Always tune diameter and minmass on a single frame first: Run tp.locate() on one representative frame and use tp.annotate() to visually check detections before committing to tp.batch(). Over-detection wastes time; under-detection misses particles.

    f0 = tp.locate(frames[0], diameter=11, minmass=200)
    tp.annotate(f0, frames[0])   # visual check in Jupyter
    
  2. Set search_range conservatively: Too large a search range causes spurious links between unrelated particles in dense samples. Estimate typical per-frame displacement from tp.locate() output scatter before linking.

  3. Subtract drift before computing MSD: Stage drift inflates MSD, causing overestimation of D. Always call tp.compute_drift() + tp.subtract_drift() before tp.emsd().

  4. Use only the linear regime for diffusion coefficient fitting: MSD curves become noisy at long lag times (few track pairs contribute). Fit only the first 10–20% of available lag times. Use log-log slope to detect non-Brownian behavior before fitting.

  5. Do not mix mpp units between locate and MSD steps: tp.locate() returns positions in pixels. mpp is applied only in tp.imsd()/tp.emsd(). Avoid rescaling positions manually before linking, as this breaks the pixel-unit search_range.

  6. For 3D tracking with anisotropic voxels: Pass diameter and search_range as tuples matching (z, y, x) axis order. The z-step is usually 2-5× coarser than xy pixel size; set the z component of diameter and search_range accordingly.

Common Recipes

Recipe: Drift Correction and Corrected MSD Comparison

When to use: Compare raw vs drift-corrected MSD to assess stage drift contribution.

import trackpy as tp
import pims
import matplotlib.pyplot as plt

frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=400, processes=1)
t = tp.link(f, search_range=6, memory=3)
t = tp.filter_stubs(t, threshold=15)

mpp, fps = 0.16, 10.0

# MSD without drift correction
emsd_raw = tp.emsd(t, mpp=mpp, fps=fps, max_lagtime=50)

# Subtract drift
drift = tp.compute_drift(t)
t_corr = tp.subtract_drift(t.copy(), drift)
emsd_corr = tp.emsd(t_corr, mpp=mpp, fps=fps, max_lagtime=50)

fig, ax = plt.subplots(figsize=(6, 5))
ax.loglog(emsd_raw.index, emsd_raw.values, 'r--', label='Raw MSD')
ax.loglog(emsd_corr.index, emsd_corr.values, 'b-', label='Drift-corrected MSD')
ax.set_xlabel("Lag time (s)")
ax.set_ylabel("MSD (µm²)")
ax.set_title("Effect of drift correction on MSD")
ax.legend()
plt.tight_layout()
plt.savefig("drift_correction_comparison.png", dpi=150)
print("Saved drift_correction_comparison.png")

Recipe: Classify Particles by Diffusion Regime

When to use: Separate particle population into confined, normal (Brownian), and directed motion based on log-log MSD slope (anomalous exponent alpha).

import trackpy as tp
import pims
import numpy as np
import pandas as pd
from scipy.stats import linregress

frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=400, processes=1)
t = tp.link(f, search_range=6, memory=3)
t = tp.filter_stubs(t, threshold=20)

drift = tp.compute_drift(t)
t = tp.subtract_drift(t.copy(), drift)

mpp, fps = 0.16, 10.0
imsd = tp.imsd(t, mpp=mpp, fps=fps, max_lagtime=30)

# Fit log-log slope (anomalous exponent alpha) for each particle
results = []
for pid in imsd.columns:
    curve = imsd[pid].dropna()
    if len(curve) < 5:
        continue
    log_lag = np.log(curve.index.values)
    log_msd = np.log(curve.values)
    slope, intercept, r, _, _ = linregress(log_lag[:10], log_msd[:10])
    D_app = np.exp(intercept) / 4   # apparent D from intercept
    results.append({'particle': pid, 'alpha': slope, 'D_app': D_app, 'r2': r**2})

df_char = pd.DataFrame(results)

# Classify by alpha
df_char['regime'] = pd.cut(
    df_char['alpha'],
    bins=[-np.inf, 0.7, 1.3, np.inf],
    labels=['confined', 'brownian', 'directed']
)
print(df_char['regime'].value_counts())
print(f"\nMean D by regime:\n{df_char.groupby('regime')['D_app'].mean()}")

df_char.to_csv("particle_classification.csv", index=False)
print("Saved particle_classification.csv")

Recipe: Filter by Eccentricity to Remove Aggregates

When to use: Exclude non-circular detections (doublets, aggregates, debris) that pass the mass threshold but are elongated.

import trackpy as tp
import pims

frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=400, processes=1)

# Eccentricity: 0 = perfect circle, 1 = line
# Remove elongated features (likely aggregates or debris)
f_round = f[f['ecc'] < 0.3]
print(f"Before ecc filter: {len(f)} detections")
print(f"After ecc filter (ecc<0.3): {len(f_round)} detections")
print(f"Removed: {len(f)-len(f_round)} elongated features")

t = tp.link(f_round, search_range=6, memory=3)
t = tp.filter_stubs(t, threshold=10)
print(f"Trajectories after eccentricity filtering: {t['particle'].nunique()}")

Troubleshooting

ProblemCauseSolution
Too many spurious detectionsminmass too low or diameter mismatched to spot sizePlot mass histogram; raise minmass to the valley between noise and signal peaks. Verify diameter matches actual spot width in pixels
Few or zero detectionsminmass too high, or particles are dim / out of focusLower minmass; check image contrast; apply background subtraction before locate
Very short trajectories (all stubs filtered out)search_range too small for particle velocity, or memory=0 with blinkingIncrease search_range to 1.5–2× max per-frame displacement; set memory=2 or 3 for blinking dyes
MSD curves are noisy or non-monotonic at long lag timesToo few tracks or fitting too many lag pointsUse only first 10–20% of lag times for fitting; ensure at least 50+ trajectories for ensemble MSD
Drift correction makes MSD worseToo few immobile reference particles; drift estimated from mobile particlesInclude fiducial beads or immobile particles; use tp.compute_drift() only on particles known to be immobile
MemoryError during tp.batch()All frames loaded into RAM at onceUse pims lazy reader (default); set processes=1; process frames in chunks using a loop over tp.locate()
3D locate returns 2D positions onlyPassed a 2D frame instead of a 3D volumeConfirm input array has 3 dimensions (Z, Y, X); check frames_4d[t_idx].ndim == 3
Linked trajectories fragment into many short segmentsParticles moving faster than search_range between framesIncrease search_range; increase memory; consider sub-sampling frames if frame rate is very high

References

matplotlib

Overview

Matplotlib is Python's foundational visualization library for creating static, animated, and interactive plots. It provides both a MATLAB-style pyplot interface and an object-oriented API for full control over figures, axes, and artists. Essential for generating publication-quality scientific figures.

When to Use

  • Creating publication-quality plots with precise control over every element (fonts, ticks, colors, spacing)
  • Building multi-panel figures with complex subplot layouts for papers
  • Generating standard scientific plot types: line, scatter, bar, histogram, heatmap, box, violin, contour
  • Exporting figures to vector formats (PDF, SVG) for journal submission
  • Creating 3D surface, scatter, or wireframe plots
  • Customizing colormaps and color schemes for accessibility (colorblind-friendly)
  • Integrating plots with NumPy arrays and pandas DataFrames
  • For quick statistical visualizations (distributions, regressions), use seaborn instead
  • For interactive/web-based plots with hover and zoom, use plotly instead

Prerequisites

  • Python packages: matplotlib, numpy
  • Optional: pandas (for DataFrame plotting), seaborn (for style presets)
  • Environment: Works in scripts, Jupyter notebooks (%matplotlib inline), and GUI apps
pip install matplotlib numpy

Quick Start

import matplotlib.pyplot as plt
import numpy as np

# Publication-ready figure template: set size, plot, label, save as PDF
fig, ax = plt.subplots(figsize=(6, 4))  # single-column journal width ≈ 6 cm → set here in inches

x = np.linspace(0, 2 * np.pi, 200)
ax.plot(x, np.sin(x), color="steelblue", lw=1.5, label="sin(x)")
ax.plot(x, np.cos(x), color="coral",    lw=1.5, label="cos(x)", linestyle="--")

ax.set_xlabel("x (radians)")
ax.set_ylabel("Amplitude")
ax.set_title("Sine and Cosine Waves")
ax.legend(frameon=False)
ax.spines[["top", "right"]].set_visible(False)  # clean axis style

plt.tight_layout()
plt.savefig("quickstart.pdf", bbox_inches="tight", dpi=300)
print("Saved quickstart.pdf")

Core API

Module 1: Figure and Axes Creation

The fundamental objects: Figure (canvas) and Axes (plotting area).

import matplotlib.pyplot as plt
import numpy as np

# Single plot (recommended: OO interface)
fig, ax = plt.subplots(figsize=(8, 5))
x = np.linspace(0, 2 * np.pi, 100)
ax.plot(x, np.sin(x), label="sin(x)")
ax.plot(x, np.cos(x), label="cos(x)")
ax.set_xlabel("x"); ax.set_ylabel("y")
ax.set_title("Trigonometric Functions")
ax.legend(); ax.grid(True, alpha=0.3)
plt.savefig("basic_plot.png", dpi=300, bbox_inches="tight")
print("Saved basic_plot.png")
# Multi-panel subplots
fig, axes = plt.subplots(2, 2, figsize=(10, 8), constrained_layout=True)
axes[0, 0].plot(x, np.sin(x)); axes[0, 0].set_title("sin(x)")
axes[0, 1].scatter(x[::5], np.cos(x[::5])); axes[0, 1].set_title("cos(x)")
axes[1, 0].bar(["A", "B", "C"], [3, 7, 5]); axes[1, 0].set_title("Bar")
axes[1, 1].hist(np.random.randn(500), bins=30); axes[1, 1].set_title("Histogram")
plt.savefig("subplots.png", dpi=300, bbox_inches="tight")
print("Saved subplots.png with 4 panels")

Module 2: Plot Types

Standard scientific chart types.

import matplotlib.pyplot as plt
import numpy as np

fig, axes = plt.subplots(2, 3, figsize=(15, 9), constrained_layout=True)

# Line plot — trends over time
x = np.linspace(0, 10, 50)
axes[0, 0].plot(x, np.exp(-x/3) * np.sin(x), "b-", linewidth=2)
axes[0, 0].set_title("Line Plot")

# Scatter plot — correlations
np.random.seed(42)
axes[0, 1].scatter(np.random.randn(100), np.random.randn(100), alpha=0.6, c=np.random.rand(100), cmap="viridis")
axes[0, 1].set_title("Scatter Plot")

# Bar chart — categorical comparisons
categories = ["Gene A", "Gene B", "Gene C", "Gene D"]
axes[0, 2].bar(categories, [4.2, 7.1, 3.5, 6.8], color="steelblue", edgecolor="black")
axes[0, 2].set_title("Bar Chart")

# Histogram — distributions
axes[1, 0].hist(np.random.randn(1000), bins=40, edgecolor="black", alpha=0.7)
axes[1, 0].set_title("Histogram")

# Box plot — statistical distributions
data = [np.random.randn(50) + i for i in range(4)]
axes[1, 1].boxplot(data, labels=["Ctrl", "Drug A", "Drug B", "Drug C"])
axes[1, 1].set_title("Box Plot")

# Heatmap — matrix data
matrix = np.random.rand(8, 8)
im = axes[1, 2].imshow(matrix, cmap="coolwarm", aspect="auto")
plt.colorbar(im, ax=axes[1, 2])
axes[1, 2].set_title("Heatmap")

plt.savefig("plot_types.png", dpi=300, bbox_inches="tight")
print("Saved 6 plot types to plot_types.png")

Module 3: Styling and Customization

Colors, fonts, styles, annotations.

import matplotlib.pyplot as plt
import numpy as np

# Use style sheets
plt.style.use("seaborn-v0_8-whitegrid")

# Custom rcParams for publication
plt.rcParams.update({
    "font.size": 12, "axes.labelsize": 14,
    "axes.titlesize": 16, "xtick.labelsize": 10,
    "ytick.labelsize": 10, "legend.fontsize": 11,
})

fig, ax = plt.subplots(figsize=(8, 5))
x = np.linspace(0, 5, 100)
ax.plot(x, np.exp(-x), "r--", linewidth=2, label="Exponential decay")
ax.fill_between(x, np.exp(-x) - 0.1, np.exp(-x) + 0.1, alpha=0.2, color="red")

# Annotations
ax.annotate("Half-life", xy=(0.693, 0.5), xytext=(2, 0.7),
            arrowprops=dict(arrowstyle="->", color="black"),
            fontsize=12, fontweight="bold")
ax.set_xlabel("Time (s)"); ax.set_ylabel("Signal")
ax.legend()
plt.savefig("styled_plot.png", dpi=300, bbox_inches="tight")
print("Saved styled_plot.png")

Module 4: Advanced Layouts

Mosaic layouts, GridSpec, insets.

import matplotlib.pyplot as plt
from matplotlib.gridspec import GridSpec
import numpy as np

# Mosaic layout — named axes
fig, axes = plt.subplot_mosaic(
    [["main", "right"], ["main", "bottom_right"]],
    figsize=(10, 7), constrained_layout=True,
    gridspec_kw={"width_ratios": [2, 1]}
)
x = np.linspace(0, 10, 200)
axes["main"].plot(x, np.sin(x) * np.exp(-x/5), "b-", linewidth=2)
axes["main"].set_title("Main Panel")
axes["right"].hist(np.random.randn(300), bins=20, orientation="horizontal")
axes["right"].set_title("Distribution")
axes["bottom_right"].bar(["A", "B"], [3, 5])
axes["bottom_right"].set_title("Summary")
plt.savefig("mosaic_layout.png", dpi=300, bbox_inches="tight")
print("Saved mosaic_layout.png")

Module 5: 3D Visualization

Surface, scatter, and wireframe plots.

import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np

fig = plt.figure(figsize=(10, 7))
ax = fig.add_subplot(111, projection="3d")

# Surface plot
u = np.linspace(0, 2 * np.pi, 50)
v = np.linspace(0, np.pi, 50)
X = np.outer(np.cos(u), np.sin(v))
Y = np.outer(np.sin(u), np.sin(v))
Z = np.outer(np.ones_like(u), np.cos(v))

ax.plot_surface(X, Y, Z, cmap="viridis", alpha=0.8)
ax.set_xlabel("X"); ax.set_ylabel("Y"); ax.set_zlabel("Z")
ax.set_title("3D Surface Plot")
plt.savefig("surface_3d.png", dpi=300, bbox_inches="tight")
print("Saved surface_3d.png")

Module 6: Export and Saving

Output to various formats with publication settings.

import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots(figsize=(6, 4))
ax.plot([1, 2, 3], [1, 4, 9], "ko-")
ax.set_title("Export Example")

# High-res PNG for presentations
fig.savefig("figure.png", dpi=300, bbox_inches="tight", facecolor="white")

# Vector PDF for journal submission
fig.savefig("figure.pdf", bbox_inches="tight")

# SVG for web
fig.savefig("figure.svg", bbox_inches="tight")

# Transparent background
fig.savefig("figure_transparent.png", dpi=300, bbox_inches="tight", transparent=True)

plt.close(fig)  # Free memory
print("Exported to PNG, PDF, SVG, and transparent PNG")

Common Workflows

Workflow 1: Multi-Panel Figure for Publication

Goal: Create a 4-panel figure combining different plot types for a paper.

import matplotlib.pyplot as plt
import numpy as np

np.random.seed(42)
fig, axes = plt.subplots(2, 2, figsize=(10, 8), constrained_layout=True)

# Panel A: Time series
t = np.linspace(0, 24, 100)
axes[0, 0].plot(t, 50 + 10 * np.sin(t * np.pi / 12), "b-", linewidth=2)
axes[0, 0].set_xlabel("Time (h)"); axes[0, 0].set_ylabel("Expression")
axes[0, 0].set_title("A", loc="left", fontweight="bold")

# Panel B: Volcano plot
fc = np.random.randn(500)
pval = -np.log10(np.random.uniform(0.0001, 1, 500))
colors = ["red" if abs(f) > 1 and p > 2 else "grey" for f, p in zip(fc, pval)]
axes[0, 1].scatter(fc, pval, c=colors, s=10, alpha=0.7)
axes[0, 1].axhline(2, ls="--", color="black", alpha=0.5)
axes[0, 1].set_xlabel("log₂ FC"); axes[0, 1].set_ylabel("-log₁₀ p-value")
axes[0, 1].set_title("B", loc="left", fontweight="bold")

# Panel C: Bar chart with error bars
means = [3.2, 5.1, 4.7, 6.3]
sems = [0.4, 0.6, 0.3, 0.5]
axes[1, 0].bar(["Ctrl", "Drug A", "Drug B", "Combo"], means, yerr=sems,
               capsize=5, color="steelblue", edgecolor="black")
axes[1, 0].set_ylabel("Response"); axes[1, 0].set_title("C", loc="left", fontweight="bold")

# Panel D: Heatmap
data = np.random.randn(6, 4)
im = axes[1, 1].imshow(data, cmap="RdBu_r", aspect="auto")
plt.colorbar(im, ax=axes[1, 1])
axes[1, 1].set_title("D", loc="left", fontweight="bold")

fig.savefig("publication_figure.pdf", bbox_inches="tight")
print("Saved publication_figure.pdf (4 panels)")

Workflow 2: Statistical Comparison Plot

Goal: Bar chart with individual data points and significance annotations.

import matplotlib.pyplot as plt
import numpy as np

np.random.seed(42)
groups = {"Control": np.random.normal(5, 1.2, 20),
          "Treatment A": np.random.normal(7, 1.5, 20),
          "Treatment B": np.random.normal(6, 1.0, 20)}

fig, ax = plt.subplots(figsize=(6, 5))
positions = range(len(groups))
for i, (name, data) in enumerate(groups.items()):
    ax.bar(i, np.mean(data), yerr=np.std(data)/np.sqrt(len(data)),
           capsize=5, color=["#4C72B0", "#DD8452", "#55A868"][i],
           edgecolor="black", alpha=0.8, width=0.6)
    # Overlay individual data points
    ax.scatter(np.full_like(data, i) + np.random.uniform(-0.15, 0.15, len(data)),
               data, color="black", s=15, alpha=0.5, zorder=5)

ax.set_xticks(positions); ax.set_xticklabels(groups.keys())
ax.set_ylabel("Measurement")

# Add significance bracket
y_max = max(max(d) for d in groups.values()) + 1
ax.plot([0, 0, 1, 1], [y_max, y_max + 0.2, y_max + 0.2, y_max], "k-", linewidth=1)
ax.text(0.5, y_max + 0.3, "**", ha="center", fontsize=14)

fig.savefig("comparison_plot.png", dpi=300, bbox_inches="tight")
print("Saved comparison_plot.png")

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
figsizeFigure creation(6.4, 4.8)(w, h) in inchesFigure dimensions
dpisavefig10072-600Resolution: 300 for print, 150 for web
bbox_inchessavefigNone"tight", NoneCrop whitespace around figure
constrained_layoutsubplotsFalseTrue/FalseAuto-adjust spacing to prevent overlap
cmapHeatmap/scatter"viridis""viridis", "coolwarm", "RdBu_r", etc.Colormap for data mapping
alphaAll plot types1.00.0-1.0Transparency (0=invisible, 1=opaque)
linewidthLine plots1.50.5-5.0Line thickness in points
sScatter201-500Marker size in points²
binsHistogram105-100 or arrayNumber of histogram bins
projectionadd_subplotNone"3d", "polar"Axes projection type

Best Practices

  1. Always use the OO interface (fig, ax = plt.subplots()) for production code. Reserve plt.plot() for quick interactive exploration only

  2. Use constrained_layout=True to prevent overlapping labels and titles:

    fig, ax = plt.subplots(figsize=(8, 5), constrained_layout=True)
    
  3. Choose accessible colormaps: Use viridis, cividis, or plasma (perceptually uniform, colorblind-safe). Avoid jet and rainbow

  4. Close figures after saving to prevent memory leaks in loops:

    plt.close(fig)  # After savefig
    
  5. Set DPI appropriately: 300 for print/journal, 150 for web/slides, 72 for screen-only

  6. Use rasterized=True for large datasets to reduce PDF/SVG file size:

    ax.scatter(x, y, rasterized=True)  # Vector labels + rasterized data
    
  7. Label panels consistently: Use bold letters (A, B, C, D) at top-left of each subplot for multi-panel figures

Common Recipes

Recipe: Custom Color Palette

When to use: Consistent colors across multiple figures in a paper.

import matplotlib.pyplot as plt

# Define a custom palette
palette = {"control": "#4C72B0", "treatment": "#DD8452", "combo": "#55A868"}

fig, ax = plt.subplots()
for group, color in palette.items():
    ax.bar(group, [5, 7, 6][list(palette.keys()).index(group)], color=color)
plt.savefig("custom_palette.png", dpi=300, bbox_inches="tight")

Recipe: Twin Y-Axes

When to use: Plotting two variables with different scales on the same figure.

import matplotlib.pyplot as plt
import numpy as np

fig, ax1 = plt.subplots(figsize=(8, 5))
x = np.arange(10)
ax1.bar(x, np.random.randint(10, 100, 10), alpha=0.7, color="steelblue", label="Count")
ax1.set_ylabel("Count", color="steelblue")

ax2 = ax1.twinx()
ax2.plot(x, np.cumsum(np.random.rand(10)), "r-o", linewidth=2, label="Cumulative")
ax2.set_ylabel("Cumulative", color="red")

fig.legend(loc="upper left", bbox_to_anchor=(0.15, 0.95))
plt.savefig("twin_axes.png", dpi=300, bbox_inches="tight")

Recipe: Inset Zoom Plot

When to use: Showing a zoomed-in region of a larger plot.

import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots(figsize=(8, 5))
x = np.linspace(0, 10, 500)
y = np.sin(x) * np.exp(-x / 5)
ax.plot(x, y, "b-", linewidth=2)

# Inset
axins = ax.inset_axes([0.5, 0.5, 0.4, 0.4])
axins.plot(x, y, "b-", linewidth=2)
axins.set_xlim(1, 3); axins.set_ylim(0.2, 0.8)
ax.indicate_inset_zoom(axins, edgecolor="black")
plt.savefig("inset_zoom.png", dpi=300, bbox_inches="tight")

Troubleshooting

ProblemCauseSolution
Overlapping labels/titlesNo layout managementAdd constrained_layout=True to plt.subplots()
UserWarning: tight_layoutIncompatible with constrained_layoutUse only one: constrained_layout OR tight_layout(), not both
Blurry figures in JupyterLow default DPISet %config InlineBackend.figure_format = 'retina'
Memory grows in loopsFigures not closedAdd plt.close(fig) after each savefig()
Font not found warningMissing system fontUse plt.rcParams["font.sans-serif"] = ["DejaVu Sans"]
Large PDF/SVG file sizeMany data points in vectorUse rasterized=True on heavy artists
3D plot rotation stuckInteractive backend issueUse %matplotlib widget in Jupyter or plt.show() in scripts
Colorbar wrong sizeDefault sizing doesn't match axesUse fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04)

References

3Dmol.js molecular visualization

Overview

3Dmol.js is a WebGL molecular viewer that runs entirely in the browser. This skill emits self-contained HTML files that load 3Dmol from a CDN and render a structure, a trajectory, or a vibrational mode — no server, no build step, no Python runtime to view. The bundled scripts/mol_viewer.py generates that HTML from any .xyz/.trj/.pdb/.sdf/.mol2/.cube file; the Core API below shows the underlying 3Dmol.js calls so you can hand-write or customize a viewer.

When to Use

  • Animate a transition-state imaginary vibrational mode (from a mode trajectory or dx/dy/dz vectors)
  • Play back a reaction path (IRC/NEB) or an MD trajectory with a speed control
  • Show a protein–ligand docking pose with cartoon + ligand sticks + a binding-site surface
  • Display an orbital or electron-density isosurface from a Gaussian .cube file
  • Hand a colleague one HTML file that opens in any browser, no install
  • Use py3Dmol instead for inline viewers inside a Jupyter notebook (same engine, Python API)
  • Use PyMOL/ChimeraX instead for publication ray-traced stills or heavy structural editing
  • Use rdkit-chemdraw-cdxml for 2D chemical structures, plotly/matplotlib for 2D plots

Prerequisites

  • Viewing: any modern browser with network access (the HTML pulls 3Dmol.js from a CDN)
  • Generator script: scripts/mol_viewer.py — Python 3 standard library only, no install
  • Optional: pip install py3Dmol for notebook use (wraps the same library)

No package is needed to produce or open the HTML. The generator lives in this skill's scripts/ folder (next to this SKILL.md). It can't be run in place from the skill directory, so use your file tools to read scripts/mol_viewer.py and save it into your working directory before running.

Quick Start

# animate a mode/trajectory file with play/pause + speed slider, in one call
python3 mol_viewer.py ts_imaginary_mode_000.trj --mode trajectory \
    --title "TS mode" --subtitle "-621.8 cm-1" --out ts_mode.html
# static structure:  python3 mol_viewer.py mol.xyz --out mol.html

Core API

All snippets assume <script src="https://3Dmol.org/build/3Dmol-min.js"></script> is loaded and a <div id="v"></div> exists.

Create a viewer and load a structure

createViewer binds to a div; addModel(data, format) loads coordinates. Always zoomTo() then render(). Supported format: xyz, pdb, sdf, mol2, cube, cif.

const viewer = $3Dmol.createViewer("v", {backgroundColor: "white"});
viewer.addModel(xyzString, "xyz");         // coordinates as a string, not a URL
viewer.setStyle({}, {stick: {radius: 0.15}, sphere: {scale: 0.28}});
viewer.zoomTo();
viewer.render();

Styles and coloring

setStyle(selection, styleSpec) — empty selection {} targets all atoms. Styles: stick, sphere, line, cross, cartoon. Color by element (default), a scheme, or a fixed color.

viewer.setStyle({}, {stick: {}, sphere: {scale: 0.25}});          // ball-and-stick
viewer.setStyle({elem: "C"}, {stick: {color: "gray"}});           // per-element override
viewer.setStyle({chain: "A"}, {cartoon: {color: "spectrum"}});    // protein ribbon
viewer.render();

Animate a trajectory

Load every frame with addModelsAsFrames, then animate. interval is the delay between frames in milliseconds (larger = slower) — do not use step, which skips frames and looks jumpy. loop: "backAndForth" makes a one-way path oscillate; reps: 0 loops forever.

viewer.addModelsAsFrames(trjString, "xyz");   // multi-frame .trj or multi-model .xyz/.pdb
viewer.setStyle({}, {stick: {radius: 0.14}, sphere: {scale: 0.28}});
viewer.zoomTo();
viewer.render();
viewer.animate({loop: "backAndForth", interval: 120, reps: 0});

Animate a vibrational normal mode

If a model's atoms carry displacement vectors (dx, dy, dz — extra columns on each XYZ line: elem x y z dx dy dz), model.vibrate(numFrames, amplitude, bothWays, arrowSpec) builds the oscillation frames. bothWays: true swings symmetrically about equilibrium; arrowSpec draws motion arrows.

const m = viewer.addModel(modeXyz, "xyz");        // each atom line: elem x y z dx dy dz
m.vibrate(10, 1.0, true, {radius: 0.08, color: "black"});   // 10 frames, full amplitude, arrows
viewer.setStyle({}, {stick: {radius: 0.14}, sphere: {scale: 0.28}});
viewer.zoomTo();
viewer.render();
viewer.animate({loop: "backAndForth", interval: 120, reps: 0});

If you only have a precomputed frame trajectory (e.g. pysisyphus ts_imaginary_mode_000.trj), use the trajectory path above instead — no dx/dy/dz needed.

Surfaces and volumetric isosurfaces

addSurface(type, style, atomsel) builds a molecular surface (VDW, SAS, SES, MS). For an orbital/density isosurface, load the .cube and call addVolumetricData.

viewer.addSurface($3Dmol.SurfaceType.VDW, {opacity: 0.75, color: "lightblue"}, {chain: "A"});
// isosurface from a Gaussian cube (positive and negative lobes):
viewer.addVolumetricData(cubeString, "cube", {isoval:  0.02, color: "blue", opacity: 0.85});
viewer.addVolumetricData(cubeString, "cube", {isoval: -0.02, color: "red",  opacity: 0.85});
viewer.render();

Labels and interactive speed control

addLabel(text, spec) annotates. For animations, a slider bound to interval (restarting via stopAnimate() + animate()) lets the viewer set the speed — the fix for "sometimes too fast".

viewer.addLabel("TS", {position: {x: 0, y: 0, z: 0}, backgroundColor: "black", fontSize: 14});
let interval = 140;
const play = () => viewer.animate({loop: "backAndForth", interval});
document.getElementById("spd").oninput = e => { interval = +e.target.value; viewer.stopAnimate(); play(); };
play();

Key Concepts

interval vs step. interval (ms) sets playback speed; every frame is shown. step plays every Nth frame — it skips motion and is the usual cause of a "too fast"/jumpy animation. Control speed with interval, never step.

Coordinates are strings, not URLs. addModel/addModelsAsFrames take the file contents. Embed them in the HTML as a JSON-encoded string so quotes and newlines survive (scripts/mol_viewer.py uses json.dumps; a raw backtick template breaks on backticks in data).

CDN and CSP. The page fetches 3Dmol.js from a CDN, so it needs network access when opened, and a strict Content-Security-Policy (e.g. inside some artifact sandboxes) will blank it. Open it as a normal local/hosted file.

Common Workflows

TS imaginary-mode animation (quantum-chemistry)

End-to-end HTML from a precomputed mode trajectory, with play/pause and a speed slider — the deliverable the neb-irc-activation-energy skill hands off.

python3 mol_viewer.py ts_imaginary_mode_000.trj --mode trajectory \
    --title "Transition-state mode" --subtitle "-621.8 cm-1" --out ts_mode.html
# open ts_mode.html; drag the slider if the oscillation is too fast

Reaction-path / MD playback

python3 mol_viewer.py trajectory.pdb --mode trajectory --style ballstick --out md.html
# any multi-model .xyz/.pdb works; backAndForth loop + interval control are built in

Docking pose: protein ribbon + ligand sticks + pocket surface

const viewer = $3Dmol.createViewer("v", {backgroundColor: "white"});
viewer.addModel(complexPdb, "pdb");
viewer.setStyle({}, {cartoon: {color: "spectrum"}});                 // protein
viewer.setStyle({resn: "LIG"}, {stick: {radius: 0.2}});             // ligand
viewer.addSurface($3Dmol.SurfaceType.VDW, {opacity: 0.6}, {resn: "LIG", byres: true, expand: 5});
viewer.zoomTo({resn: "LIG"});
viewer.render();

Key Parameters

ParameterMethodDefaultRange / OptionsEffect
intervalanimate5040–400 msFrame delay; larger = slower playback
loopanimateforwardforward/backward/backAndForthbackAndForth oscillates a one-way path
repsanimate00=∞, nNumber of loops
radiusstick0.30.1–0.3Bond cylinder thickness
scalesphere1.0 (vdW)0.2–0.4 for ball-and-stickAtom sphere size
amplitudevibrate1.00.5–2.0Normal-mode distortion size
numFramesvibrate108–20Frames per half-cycle
isovaladdVolumetricData—e.g. ±0.02Isosurface contour value (sign = lobe)
opacityaddSurface1.00–1Surface transparency

Best Practices

  • Control animation speed with interval (ms), never step.
  • Embed coordinates as a JSON-encoded string (json.dumps), not a raw backtick template.
  • Call zoomTo() before render(), and again after adding a large model.
  • Keep default element colors unless the analysis needs a specific scheme — don't bake a palette.
  • For large trajectories (>500 frames or >5k atoms), subsample frames; WebGL redraw is the limit.
  • Ship one CDN <script> tag; only vendor the ~1 MB 3Dmol-min.js inline if offline use is required.

Common Recipes

Recipe: generate a viewer in one call

python3 mol_viewer.py mode.xyz --mode vibrate --amplitude 1.2 --title "mode" --out mode.html
python3 mol_viewer.py mol.sdf  --style stick --out mol.html          # static

Recipe: inline viewer in a Jupyter notebook (py3Dmol)

import py3Dmol
view = py3Dmol.view(width=500, height=400)
view.addModel(open("mol.xyz").read(), "xyz")
view.setStyle({}, {"stick": {}, "sphere": {"scale": 0.25}})
view.zoomTo(); view.show()

Recipe: side-by-side viewers

const viewer = $3Dmol.createViewerGrid("v", {rows: 1, cols: 2});
viewer[0][0].addModel(reactantXyz, "xyz"); viewer[0][0].setStyle({}, {stick: {}});
viewer[0][1].addModel(productXyz, "xyz");  viewer[0][1].setStyle({}, {stick: {}});
viewer[0][0].zoomTo(); viewer[0][1].zoomTo(); viewer[0][0].render(); viewer[0][1].render();

Troubleshooting

ProblemCauseSolution
Blank white page3Dmol.js not loaded (offline / strict CSP)Open with network access; check the CDN <script> resolves
Animation too fast / jumpyUsing step, or a tiny intervalUse interval (ms); raise it; never set step
Vibration shows no motionModel lacks dx/dy/dz vectorsAdd mode vectors as extra XYZ columns, or use a precomputed frame .trj
Nothing renderedWrong format string or bad dataMatch format to the file; coordinates must be the file contents, not a path
JS syntax error in pageBacktick/quote in embedded dataEmbed via json.dumps (the generator does this)
Structure loads but no bondsXYZ without connectivity + line styleUse stick/sphere; 3Dmol infers bonds by distance
Surface slow or hangsLarge SES/MS on a big systemUse VDW, restrict the atomsel, or lower resolution

Bundled Resources

  • scripts/mol_viewer.py — emit a standalone 3Dmol HTML (static / trajectory / vibrate) from a structure file, with built-in play/pause + speed slider for animations

Related Skills

  • neb-irc-activation-energy — produces TS imaginary-mode trajectories and IRC paths that this skill animates
  • rdkit-chemdraw-cdxml — 2D chemical structure and reaction-scheme drawing
  • plotly-interactive-plots — interactive 2D scientific plots and dashboards

References

multipanel

Overview

A multi-panel figure is one figure, built one of two ways depending on what you have:

  • Option 1 — redraw every panel (you have the data or plotting code): draw each data panel with a python script into its own subfigure so it packs to its own labels — no empty bands, and axes need NOT align across the grid. Follow the discipline below so legends stay inside their panels, panel letters sit at each panel's own top-left, and text never overlaps.
  • Option 2 — composite finished images (you only have rendered PNG/PDF panels): paste them onto a plt.subplot_mosaic canvas — fine here, since images carry no tick labels to misalign — add panel letters, and export.

A mix is allowed: if one or two panels are image-only (no data/code), imshow them onto their own subfigure axes and redraw the rest into the same figure. Both modes export a vector PDF and a high-DPI PNG.

Always export the individual panels AND the composite. Every run outputs both: one standalone figure per panel (figure1A.png, figure1B.png, …) and the combined figure (combined_figure1.pdf + .png) — not just the composite. Because a matplotlib subfigure cannot be saved on its own, factor every data panel's plotting body into a draw_<letter>(ax) function (option 1); the same function then draws onto the composite's subfigure axis AND onto a fresh standalone figure, so the panels stay identical across both outputs with no duplicated drawing code. See "Exporting individual panels" below.

This skill covers composition. For how to draw each individual plot type (volcano, GSEA bar, heatmap, box/violin, PCA, Kaplan–Meier, …), use the sibling omics-plotting skill — copy each recipe's body onto a subfigure's axis rather than calling it as a standalone figure. Everything you need here (shared style, composite recipe, panel-label helper) is in this document.

When to use

  • The user asks for a multi-panel / composite / journal figure (panels A, B, C…) combining two or more plots into one page of image.
  • The user hands you or points out already-rendered panels (PNG/PDF) and wants them combined into one figure (image assembly — see "Assembling user-provided panels").
  • You are assembling a figure for a report, a paper submission, or a presentation and want all panels to read as one consistent system.

Do NOT use for

  • A single plot from a data table — use the sibling omics-plotting skill.
  • Interactive dashboards or web charts (this is static matplotlib output).
  • 3D molecular structure rendering (that is the structure viewer, not a plot).

Key Concepts

Redraw vs composite — two composition modes

There are two fundamentally different ways to build a composite, and the user chooses. Redraw (option 1) rebuilds every panel from data or code in one script, giving uniform style, fonts, colors, and panel letters — best when you hold the underlying data/DataFrame or the plotting code. Composite (option 2) pastes already-rendered PNG/PDF panels onto a canvas and only adds panel letters — image assembly, not plotting — best when you have only the finished images. A mix is allowed: image-only panels are imshow-pasted while data panels are redrawn, all into one figure.

Independent subfigures vs shared mosaic

The central layout decision. Giving each panel its own subfigure lets it run its own constrained_layout and pack tightly to its OWN labels — panels sit flush with no empty bands, and axes deliberately do NOT align across the grid. A single shared subplot_mosaic gridspec instead equalizes every column's margin to its widest y-label, leaving wide empty bands beside short-label panels. Independent subfigures are the default here because composites usually mix heterogeneous plot types; a shared mosaic is correct only when panels genuinely share a scale and are meant to be read against each other.

Panel letters in the subfigure frame

Panel letters (bold A, B, C…) must sit at each panel's OWN outer top-left, left of that panel's y-axis labels — never merged into the title and never snapped to a shared column x-position. Placing each letter at (0, 1) in its subfigure's coordinate frame (transform=sf.transSubfigure) guarantees it hugs its panel regardless of neighbors' label widths.

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: panelSelection
    kind: required
    source: user
    ask: "Which results earn a panel, and what claim should the assembled figure make?"
    default: null

  - id: D2
    param: panelOrder
    kind: required
    source: user
    depends_on: [D1]
    ask: "In what order should the panels be read, and which belong on the same row as a group?"
    default: "analysis order"

  - id: D3
    param: sharedScales
    kind: required
    source: user
    depends_on: [D1]
    ask: "Should panels that show the same quantity share one axis range, or each get its own?"
    default: "shared within a row of comparable panels"

  - id: D4
    param: figureDimensions
    kind: required
    source: user
    ask: "What column width and page format is this figure for?"
    default: "single column, portrait"

  - id: D5
    param: exportFormat
    kind: optional
    source: user
    ask: "Vector for submission, raster for a slide, or both?"
    default: "vector PDF plus a raster preview"

D3 is the only one here that changes what a reader concludes rather than how it looks. Independent y-axes make a small effect and a large one occupy the same visual height, and side-by-side panels invite exactly that comparison.

Decision Framework

Start from what you have, then how panels relate:

What sources do you have?
├─ Data / code for every panel .................. Option 1: redraw all
├─ Only finished PNG/PDF images ................. Option 2: composite images
└─ Mix (some data, some image-only) ............. Option 1 + imshow the image-only panels
        │
        ▼
How do the panels relate?
├─ Heterogeneous plot types (default) ........... Independent subfigures (tight pack, axes need NOT align)
└─ Same scale, read against each other .......... Shared subplot_mosaic (aligned axes)
        │
        ▼
Layout: sketch the grid [[...]], nest subfigures for spanning panels, fill every cell
SituationApproachLayout primitivePanel letters
Have data/code for all panelsRedraw (option 1)fig.subfigures(...) per panelsubfigure frame (0,1)
Only rendered imagesComposite (option 2)plt.subplot_mosaic + imshowmosaic axes top-left
Some data, some image-onlyRedraw + pastesubfigures + imshow leafsubfigure frame (0,1)
Panels share a common scaleShared mosaicsubplot_mosaic alignedaxes top-left
Spanning panel (e.g. bottom row)Nested subfigurestop[0].subfigures(1, 2)leaf subfigure frame

Workflow

  1. Ask which approach first — ask the user, then wait. Both approaches below are usually viable and the choice is the user's, so before drawing or writing any script, ask the user to choose between these two concrete options:

    • Option 1 — Redraw every panel into one unified figure (from data/code): consistent style, fonts, colors, and panel letters across all panels. Best when you have the underlying data (CSV/TSV/DataFrame) or the plotting code.
    • Option 2 — Composite already-rendered images: paste the finished PNG/PDF panels onto a canvas and add panel letters — image assembly, not plotting. Best when you only have the finished images (no data/code) or the user wants to keep the originals as-is.

    Skip the question only when one option is impossible (e.g. only images and no data/code → option 2 is forced; or a data table with no rendered images → option 1) and say why. If a mix (some panels have data, one or two are images-only), tell the user that the image-only panels will be pasted regardless (discipline in the intro).

  2. Decide the layout (the grid [[...]] sketch is just to plan the tiling; you build it with nested subfigures, not subplot_mosaic — see discipline #1). Fill every cell. — e.g. two on top, one spanning the bottom → [["A", "B"], ["C", "C"]] → top = fig.subfigures(2, 1); tc = top[0].subfigures(1, 2) (A,B in tc; C in top[1]). — e.g. three on top, two on the bottom → [["A", "B", "C"], ["D", "E", "E"]].

  • e.g. one big panel on the left, two stacked on the right → [["A", "B"], ["A", "C"]] → lr = fig.subfigures(1, 2); A = lr[0]; rr = lr[1].subfigures(2, 1).
  1. Gather each panel's source — a workspace-relative CSV/TSV (or DataFrame) for data panels, or a user-supplied PNG/PDF for image panels.
  2. Write one python script: paste the style block, factor each data panel's plotting body into a draw_<letter>(ax) function (so it can render onto both a subfigure axis and a standalone figure), build the subfigures (nest for spanning panels), call each draw_<letter> onto its axis (data) or imshow the image, collect the subfigures into a panels dict, and add panel letters with the helper. Then always save both outputs to workspace-relative paths under figures/:
    • the composite as figures/combined_figure1.pdf + figures/combined_figure1.png, and
    • each individual panel as figures/figure1A.png, figures/figure1B.png, … (plus matching .pdf) by rendering every draw_<letter> onto a fresh standalone figure. See "Exporting individual panels" for the exact loop.
  3. Report the saved paths back to the user — the combined figure and every individual panel file.

Shared style — paste at the top of the script

import matplotlib.pyplot as plt

# Publication style (colorblind-friendly, editable vector text, no top/right spines)
PUB_STYLE = {
    "figure.dpi": 110, "savefig.dpi": 300, "savefig.bbox": "tight",
    "font.family": "sans-serif",
    "font.sans-serif": ["Arial", "Liberation Sans", "Nimbus Sans", "Helvetica", "DejaVu Sans"],
    "font.size": 11, "axes.titlesize": 13, "axes.titleweight": "bold",
    "figure.titlesize": 13, "figure.titleweight": "bold",
    "axes.labelsize": 12, "axes.linewidth": 1.0,
    "axes.spines.top": False, "axes.spines.right": False,
    "xtick.labelsize": 10, "ytick.labelsize": 10,
    "xtick.direction": "out", "ytick.direction": "out",
    "legend.frameon": False, "legend.fontsize": 9,
    "svg.fonttype": "none", "pdf.fonttype": 42, "ps.fonttype": 42,
}
plt.rcParams.update(PUB_STYLE)

# Palette — reuse the SAME colors across every panel
UP, DOWN, NS = "#d73721", "#204897", "#d9d9d9"   # up / down / not-significant
PALETTE = ["#2a78d6", "#eb6834", "#1baf7a", "#eda100", "#e87ba4",
           "#008300", "#4a3aa7", "#e34948", "#12a4c0", "#a66a2e"]  # categorical (CVD-safe)
DIVERGING_CMAP = "RdBu_r"    # z-score / log2FC — set center=0, vmin=-vmax
SEQUENTIAL_CMAP = "viridis"  # magnitude / -log10 p / density

For a dense composite, lower the font: plt.rcParams.update({"font.size": 7, "axes.titlesize": 8, "axes.labelsize": 7, "legend.fontsize": 6}).

Multi-panel discipline

This is what keeps a composite clean — every rule prevents a specific failure.

  1. One figure, independent subfigures, constrained layout. Give each panel its own subfigure so it packs to its OWN labels: fig = plt.figure(layout="constrained", figsize=(width_mm/25.4, height_mm/25.4)), then sfs = fig.subfigures(nrows, ncols, width_ratios=..., height_ratios=...) and ax = sfs[r, c].subplots() per panel. Each subfigure runs its own constrained_layout, so a panel with long y-tick labels no longer shoves its column-neighbors' plots sideways — axes deliberately do NOT align across the grid; panels sit flush with no empty bands (a single shared subplot_mosaic gridspec, by contrast, equalizes each column's margin to its widest y-label and leaves a wide gap beside the short-label panels). Reserve a hair of margin so panel letters never clip: fig.get_layout_engine().set(rect=(0.012, 0, 0.988, 0.985)). Never add tight_layout() or manual subplots_adjust. Size in mm (single column = 88 mm, double = 180 mm).
    • Spanning panels: nest subfigures — e.g. two panels on top, one spanning the bottom → top = fig.subfigures(2, 1); tc = top[0].subfigures(1, 2) (A, B in tc[0], tc[1]; C in top[1]). One .subplots() per leaf subfigure.
    • Match each panel to its plot's shape via the subfigures' width_ratios/ height_ratios (pin with ax.set_box_aspect(...) if it still deforms): scatter panels (volcano/PCA) near-square; for bar / box / histogram, protect the value axis in both orientations — horizontal (barh, horizontal box) kept wide, vertical (bar, box, hist) kept tall. Never let a neighbor squeeze that axis flat.
    • When panels genuinely share a scale (same y-range, meant to be read against each other), a shared subplot_mosaic with aligned axes is the right choice instead — but this skill usually combines heterogeneous plot types, so independent subfigures are the default.
  2. Fill every cell. No empty grid slots. If a panel would be blank, span a neighbor across it: [["A", "B"], ["C", "C"]].
  3. Legends & colorbars belong to their own panel — a legend in that panel's free corner (ax.legend(loc="lower right", frameon=False)) or a colorbar on that one axis (fig.colorbar(im, ax=ax, fraction=0.025, pad=0.02)). Never float a figure-level legend in empty space or stack two in a margin; for a dot plot, keep only the colorbar and drop the size legend (count range → panel title).
    • Too wide? Thin a colorbar with aspect=40 + shrink=0.6 (and small fraction); tighten a legend with handlelength=1.0, handletextpad=0.2, borderpad=0.2, or fold long legends into ncol.
  4. Panel letters at each panel's OWN outer top-left. Bold capitals A, B, C… (lowercase fine; never numeric), placed to the LEFT of that panel's y-axis tick and axis labels — not merged into the title. Place each letter in its own subfigure's coordinate frame — sf.text(0.0, 1.0, letter, transform=sf.transSubfigure, ...) (the helper below). The subfigure's top-left corner is always left of that panel's y-labels and hugs that panel, so the letter never overlaps a wide label and never floats over an empty band. Do not snap letters to a shared column-x — with independent packing that would drag a short-label panel's letter far from its plot (the empty-gap failure the user sees). The reserved margin from discipline #1 keeps edge letters in-canvas.
  5. Text must stay readable — the #1 way composites go wrong.
    • Point labels (volcano/scatter): cap to ≤5 strongest hits in a small panel, italic ~6 pt, and repel with adjustText; if it is not installed, skip labels rather than dumping overlapping text.
    • Long category names (pathways/gene sets): put them on the y-axis (horizontal, one per row), never crammed/rotated on a narrow x-axis. If they must go on x, rotate (45° to save vertical space, or 90° when very long, ha="right" so the tick end aligns under its bar), wrap to ≤26 chars, and give enough width per column — crammed x-labels otherwise collide into unreadable text. Shorten over-long names (common with MSigDB/GO/Reactome): strip the DB prefix (HALLMARK_, GO_, REACTOME_, KEGG_), swap _→space and title-case, and replace verbose terms with standard abbreviations (e.g. HALLMARK_INTERFERON_GAMMA_RESPONSE → IFN-γ response); truncate with an ellipsis only if still too long. Keep the full name in the underlying data/tooltip, not on the axis tick.
    • Size-encoded markers (dot plot): floor the size range (s in ~[25,150]) so small dots stay visible.
    • In-cell heatmap numbers: annotate only when the values are needed (small font ~4–5 pt, no decimals); otherwise omit them and let the colorbar carry the values.
  6. Reuse one palette and axis convention across panels so the composite reads as a single system.
  7. Export vector PDF + PNG and report the relative path.

Panel-label helper

Place each letter at the top-left corner of its own subfigure. Because every panel lives in its own tightly-packed subfigure, that corner is always left of the panel's y-labels and hugs the panel — so letters never float over an empty band (the shared-column failure) and never overlap a wide y-label, no matter how the panels' label widths differ:

def add_panel_labels(panels, size=11):
    """Bold letter at each panel's OWN outer top-left, in its subfigure frame.

    panels : dict {letter: subfigure} — the subfigure that holds each panel's axes,
    collected as you build them (for a spanning panel, its leaf subfigure). Placing
    the letter at (0, 1) in the subfigure's coordinates puts it at that cell's top-left
    corner: always LEFT of the panel's y-labels and hugging the panel, with no
    dependence on any neighbor's label width. Reserve a hair of figure margin first
    (`fig.get_layout_engine().set(rect=(0.012, 0, 0.988, 0.985))`, discipline #1) so the
    letters of edge panels are not clipped at the canvas edge.
    """
    for letter, sf in panels.items():
        sf.text(0.0, 1.0, letter, transform=sf.transSubfigure,
                fontsize=size, fontweight="bold", va="top", ha="left")

Usage: collect the subfigures as you create them, e.g. panels = {"A": sfs[0, 0], "B": sfs[0, 1], "C": top[1]}, then call add_panel_labels(panels).

Exporting individual panels

Every run produces both the individual panels (figure1A.png, figure1B.png, …) and the composite (combined_figure1.pdf + .png) — this is the default output, not an extra. A matplotlib subfigure cannot be saved on its own, so put each panel's plotting body in a draw_<letter>(ax) function and call it twice: once onto the composite's subfigure axis, and once onto a fresh standalone figure. One source of truth per panel — the panels stay identical across both outputs.

import os

os.makedirs("plots", exist_ok=True)

# 1) Factor each DATA panel's body into a function of a single Axes.
#    (Copy the omics-plotting recipe body here, drawing onto `ax` instead of a new figure.)
def draw_A(ax):
    ax.scatter(df["log2FC"], -np.log10(df["padj"]), s=8, c=NS)  # volcano, etc.
    ax.set_xlabel("log2 fold change"); ax.set_ylabel("-log10 FDR")

def draw_B(ax):
    ...   # PCA / box / heatmap body onto ax

def draw_C(ax):
    ...

DATA_PANELS = {"A": draw_A, "B": draw_B, "C": draw_C}
# Per-panel standalone figure size (mm) — match each plot's shape (discipline #1).
PANEL_SIZE_MM = {"A": (88, 75), "B": (88, 75), "C": (180, 70)}
# Image-only panels stay separate: keep the PNG/PDF the user supplied as their
# standalone file, and only imshow them onto the composite axis (see intro).

# 2) Composite — draw each function onto its subfigure axis, add letters, save.
panels = {"A": sfs[0, 0], "B": sfs[0, 1], "C": top[1]}
for letter, sf in panels.items():
    DATA_PANELS[letter](sf.subplots())
add_panel_labels(panels)
fig.savefig("figures/combined_figure1.pdf")
fig.savefig("figures/combined_figure1.png", dpi=300)

# 3) Individual panels — same functions onto fresh standalone figures (no letter).
for letter, draw in DATA_PANELS.items():
    w_mm, h_mm = PANEL_SIZE_MM[letter]
    fp = plt.figure(layout="constrained", figsize=(w_mm / 25.4, h_mm / 25.4))
    draw(fp.subplots())
    fp.savefig(f"figures/figure1{letter}.pdf")
    fp.savefig(f"figures/figure1{letter}.png", dpi=300)
    plt.close(fp)

Output files (Figure 1 with panels A, B, C): figures/combined_figure1.pdf, figures/combined_figure1.png, figures/figure1A.{pdf,png}, figures/figure1B.{pdf,png}, figures/figure1C.{pdf,png}.

Notes:

  • No panel letter on standalones — the A/B/C label belongs to the composite frame only; a lone figure1A.png needs no letter baked in.
  • Size each standalone to its plot's shape (discipline #1) via PANEL_SIZE_MM: scatter/PCA near-square, barh/horizontal-box wide, vertical bar/box/hist tall — don't reuse one size for all.
  • Legends/colorbars still belong to their own axis (discipline #3) — since the body lives in draw_<letter>, attach them inside that function so they appear in both the composite and the standalone.
  • Image-only panels are already standalone files (the user's PNG/PDF); don't re-export them — just reference the originals.

Best Practices

  • One figure, one style. Never stitch separate PNGs or call standalone plot functions for a composite; copy their bodies onto each subfigure's axis.
  • Workspace-relative paths only. Save under figures/ (create it if needed); never absolute paths like /tmp or /home/....
  • Only plot data that exists. Never invent columns, groups, or values.
  • Label every axis, keep every legend inside its panel, fill every cell.
  • Always export both a vector .pdf and a .png (dpi≥300) under figures/.

Common Pitfalls

  • Building the whole figure as one shared subplot_mosaic gridspec. It equalizes each column's margin to its widest y-label, so a long-label panel shoves neighbors sideways, leaves empty bands, and strands letters snapped to the shared column edge. How to avoid: give each panel its own subfigure so it packs to its own labels (discipline #1); reserve a shared mosaic only for panels that genuinely share a scale.
  • Panel letters merged into titles, snapped to a shared column-x, or clipped at the edge. They then sit right of the y-labels, float far from their plot, or vanish off-canvas. How to avoid: place each letter at (0, 1) in its own subfigure frame (transform=sf.transSubfigure) with the add_panel_labels helper, and reserve a hair of margin (rect=(0.012, 0, 0.988, 0.985), discipline #1) so edge letters stay in-canvas.
  • Floating or bulky legends and colorbars. Per-plot figure-level legends collide in the margins, or a colorbar eats half the panel. How to avoid: attach each legend/colorbar to its own panel's axis, drop a composite dot plot's size legend, and thin a wide colorbar (aspect=40, shrink=0.6, small fraction) (discipline #3).
  • Value axis flattened — scatter dots merge or bars/boxes squash. A neighbor steals the space the plot's value direction needs. How to avoid: widen or heighten that cell via width_ratios / height_ratios (or pin with ax.set_box_aspect) instead of shrinking the plot; only then bump marker size.
  • Cramming long category names onto a narrow x-axis. Pathway/gene-set names collide into unreadable text. How to avoid: put long names on the horizontal y-axis, strip DB prefixes (HALLMARK_, GO_) and abbreviate, or rotate 90° and wrap to ≤26 chars with enough panel width.
  • Fixing cramped panels with manual spacing. Adding tight_layout() or subplots_adjust fights constrained_layout and makes it worse. How to avoid: instead increase figsize (in mm), adjust the ratios, or lower the font, and let constrained layout re-space.
  • Inconsistent or unreadable text. Over-labeled points overlap, heatmap cell numbers are too dense, and font sizes drift between panels. How to avoid: cap point labels to ≤5 (repel with adjustText, else skip); annotate heatmap cells only at ~4–5 pt with no decimals or drop them for the colorbar; keep the same font sizes across all panels, including any the user supplies (discipline #5).

Further Reading

omics-plotting

Overview

When the user wants a figure, generate it with matplotlib / seaborn, applying the shared style block below. The user can hand-tune colors, fonts, or spines per plot, but unless they ask for something specific, paste the style block and reuse the palette so a whole analysis reads as one figure system at a glance.

This skill is self-contained: everything you need (style, palette, recipes) is in this document.

When to use

  • The user asks for a plot / figure / chart / visualization from a results table or an in-memory DataFrame (DEG table, enrichment result, expression matrix, long-form measurements, survival table…).
  • You are preparing figures for a report, a paper submission or presentation and want a consistent publication style.

Combining several plots into one multi-panel composite, or assembling user-supplied PNG/PDF panels, is handled by the sibling multipanel skill — use this skill to draw each individual panel.

Do NOT use for

  • Interactive dashboards or web charts (this is static matplotlib output).
  • 3D molecular structure rendering (that is the structure viewer, not a plot).

Key Concepts

One consistent figure system

The core idea is that every figure from a single analysis should look like it came from the same publication. That is enforced by two shared objects: the PUB_STYLE rcParams block (fonts, spines, DPI, editable vector text) and a fixed PALETTE / directional color set (UP, DOWN, NS). Paste both at the top of every plot script and map the same group or direction to the same color across panels, so a reader can carry meaning from one figure to the next.

Diverging vs sequential colormaps

Color encoding is not free choice. Use the diverging colormap (DIVERGING_CMAP = "RdBu_r", always center=0, vmin=-vmax) for signed quantities where zero is meaningful — z-scores, log2 fold changes, correlations. Use the sequential colormap (SEQUENTIAL_CMAP = "viridis") for unsigned magnitudes — densities, -log10 p, counts. Mixing these (a sequential map on signed data) hides the sign and misleads the reader.

Data shape drives figure type

Each recipe expects a specific table shape: a per-gene DEG table (volcano, MA), a genes × samples matrix (heatmap), a samples × features matrix (PCA/UMAP), or long-form tidy rows (box/violin/bar, ridgeline, Kaplan–Meier). Identifying the shape first — then reading the header to confirm the real column names — is what selects the recipe. The column names in each recipe are defaults to override, not fixed requirements.

Decision Framework

Pick the figure type from what the data represents and what question it answers:

What does the table hold?
├─ Per-gene stats (log2FC, padj)
│   ├─ emphasize significance ......... Volcano
│   └─ emphasize expression level ..... MA plot
├─ genes × samples matrix
│   ├─ show patterns/clusters ......... Clustered expression heatmap (z-score)
│   └─ show sample-sample QC .......... Correlation heatmap
├─ Enrichment / gene-set result
│   ├─ signed effect (NES) ............ GSEA bar
│   └─ ratio + size + significance .... GSEA dot plot
├─ Long-form measurements (x, y)
│   ├─ compare distributions .......... Box / Violin
│   ├─ compare means .................. Bar (with error bars)
│   └─ many groups, shape matters ..... Ridgeline
├─ samples × features (high-dim) ...... PCA / UMAP / t-SNE
└─ time-to-event + group ............. Kaplan–Meier
Data you haveQuestionFigureColormap / palette
DEG tableWhich genes change, how significantly?VolcanoUP/DOWN/NS
DEG tableEffect vs abundanceMA plotUP/DOWN/NS
Expression matrixCluster structureClustered heatmapdiverging, center 0
Expression matrixSample QCCorrelation heatmapdiverging, [-1, 1]
Enrichment resultTop pathways, directionGSEA barUP/DOWN
Enrichment resultRatio + significance + sizeGSEA dot plotsequential
Long-formGroup distributionsBox / Violincategorical PALETTE
High-dim matrixGlobal sample layoutPCA / UMAP / t-SNEcategorical PALETTE
Survival tableGroup survival over timeKaplan–Meiercategorical PALETTE

Workflow

  1. Identify the data source — a workspace-relative CSV/TSV path or a DataFrame already in memory — and the figure type (pick from the table below). If the required columns are unclear, inspect the table's header first.
  2. Write one python script: paste the style block, load the data, draw the plot with the matching recipe, and save to a workspace-relative path under figures/.
  3. Report the saved path back to the user (and reference it in any report / deck by that relative path, e.g. ![Volcano](figures/volcano.png)).

Shared style — paste at the top of every plot script

import matplotlib.pyplot as plt

# Publication style (colorblind-friendly, editable vector text, no top/right spines)
PUB_STYLE = {
    "figure.dpi": 110, "savefig.dpi": 300, "savefig.bbox": "tight",
    "font.family": "sans-serif",
    "font.sans-serif": ["Arial", "Liberation Sans", "Nimbus Sans", "Helvetica", "DejaVu Sans"],
    "font.size": 11, "axes.titlesize": 13, "axes.titleweight": "bold",
    "figure.titlesize": 13, "figure.titleweight": "bold",
    "axes.labelsize": 12, "axes.linewidth": 1.0,
    "axes.spines.top": False, "axes.spines.right": False,
    "xtick.labelsize": 10, "ytick.labelsize": 10,
    "xtick.direction": "out", "ytick.direction": "out",
    "legend.frameon": False, "legend.fontsize": 9,
    "svg.fonttype": "none", "pdf.fonttype": 42, "ps.fonttype": 42,
}
plt.rcParams.update(PUB_STYLE)   # or: with plt.rc_context(PUB_STYLE): ...

# Palette — reuse the SAME colors across every panel of an analysis
UP, DOWN, NS = "#d73721", "#204897", "#d9d9d9"   # up / down / not-significant
PALETTE = ["#2a78d6", "#eb6834", "#1baf7a", "#eda100", "#e87ba4",
           "#008300", "#4a3aa7", "#e34948", "#12a4c0", "#a66a2e"]  # categorical (CVD-safe order)
GROUP_COLORS = {"Group1": "#204897", "Group2": "#e34948", "Group3": "#E7B800"}
DIVERGING_CMAP = "RdBu_r"    # z-score / log2FC heatmaps — set center=0, vmin=-vmax
SEQUENTIAL_CMAP = "viridis"  # magnitude / -log10 p / density

Multi-panel / composite figures

For combining several plots into one multi-panel journal figure (panels A, B, C…), or assembling already-rendered PNG/PDF panels the user supplies, use the sibling multipanel skill — it owns the composition discipline (one subplot_mosaic canvas, per-panel legends, correctly placed panel letters, text-legibility rules, image assembly). Draw each panel with the single-panel recipes below, then compose per that skill. The recipes here each build their own figure, so do not call them directly for a composite — copy the recipe body onto a mosaic axis as multipanel describes.

Plot catalogue

Pick the recipe by figure type. Columns listed are the defaults — override the column-name variables to match the actual table.

FigureInput shapeKey columns (defaults)
VolcanoDEG tablelog2FoldChange, padj; optional label column
MA plotDEG tablebaseMean, log2FoldChange, padj
Expression heatmapgenes × samples matrixnumeric matrix, optional index_col
Correlation heatmapsamples × features (numeric)all numeric columns
GSEA bar plotenrichment resultTerm, NES, FDR q-val
GSEA dot plotenrichment resultTerm, GeneRatio, Count, Adjusted P-value
Box / Violin / Barlong-formx (category), y (numeric), optional hue
Ridgelinelong-formnumeric x, categorical group
PCA / UMAP / t-SNEsamples × featuresnumeric features + optional group
Kaplan–Meiersurvival tabletime, event, group
ManhattanGWAS summary statsCHR, BP, P
QQ plotp-value vectorP
Foresteffect + CI tablelabel, estimate, ci_low, ci_high

Recipes

Each is a full python script body. Adjust column names, thresholds, and the save path. All save under figures/.

Volcano (-log10 p vs log2 fold change):

import numpy as np, pandas as pd
df = pd.read_csv("deg_results.csv").dropna(subset=["log2FoldChange", "padj"])
fc, p = df["log2FoldChange"].to_numpy(float), df["padj"].to_numpy(float)
nlp = -np.log10(np.clip(p, 1e-300, None))
fc_t, p_t = 0.58, 0.05
up, down = (fc >= fc_t) & (p < p_t), (fc <= -fc_t) & (p < p_t)
ns = ~(up | down)
fig, ax = plt.subplots(figsize=(7, 6))
ax.scatter(fc[ns], nlp[ns], c=NS, s=12, alpha=0.5, edgecolors="none", rasterized=True, label=f"NS ({ns.sum()})")
ax.scatter(fc[down], nlp[down], c=DOWN, s=18, alpha=0.85, edgecolors="none", label=f"Down ({down.sum()})")
ax.scatter(fc[up], nlp[up], c=UP, s=18, alpha=0.85, edgecolors="none", label=f"Up ({up.sum()})")
for v in (fc_t, -fc_t): ax.axvline(v, ls="--", lw=0.8, color="0.5")
ax.axhline(-np.log10(p_t), ls="--", lw=0.8, color="0.5")
lab = (df["gene"] if "gene" in df else pd.Series(df.index)).astype(str).to_numpy()
sig = np.where(up | down)[0]
top = sig[np.argsort(nlp[sig])[::-1][:10]]      # standalone: top ~10; composite panel: cut to <=5
try:                                            # repel labels so they never overlap
    from adjustText import adjust_text
    texts = [ax.text(fc[i], nlp[i], lab[i], fontsize=7) for i in top]
    adjust_text(texts, ax=ax, expand=(1.3, 1.6),
                arrowprops=dict(arrowstyle="-", color="0.6", lw=0.5))
except ImportError:                             # no adjustText -> label fewer, with an offset
    for i in top[:5]:
        ax.annotate(lab[i], (fc[i], nlp[i]), xytext=(6, 6), textcoords="offset points",
                    fontsize=7, ha="left", va="bottom")
ax.set_xlabel(r"$\log_{2}$ fold change"); ax.set_ylabel(r"$-\log_{10}$ padj")
ax.set_title("Volcano plot"); ax.legend(loc="upper right", markerscale=1.4)
fig.tight_layout(); fig.savefig("figures/volcano.png")

MA plot (log2 fold change vs mean expression; columns baseMean, log2FoldChange, optional padj):

import numpy as np, pandas as pd
df = pd.read_csv("deg_results.csv").dropna(subset=["baseMean", "log2FoldChange"])
x = np.log10(df["baseMean"].to_numpy(float) + 1)
fc = df["log2FoldChange"].to_numpy(float)
sig = (df["padj"].to_numpy(float) < 0.05) if "padj" in df else np.zeros(len(df), bool)
fig, ax = plt.subplots(figsize=(7, 5))
ax.scatter(x[~sig], fc[~sig], c=NS, s=10, alpha=0.5, edgecolors="none", rasterized=True, label="NS")
ax.scatter(x[sig], fc[sig], c=UP, s=14, alpha=0.85, edgecolors="none", label=f"padj<0.05 ({int(sig.sum())})")
ax.axhline(0, color="0.4", lw=0.8)
ax.set_xlabel(r"$\log_{10}$(mean norm. count + 1)"); ax.set_ylabel(r"$\log_{2}$ fold change")
ax.set_title("MA plot"); ax.legend(loc="upper right")
fig.tight_layout(); fig.savefig("figures/ma.png")

Clustered expression heatmap (z-scored, seaborn clustermap):

import seaborn as sns, pandas as pd
mat = pd.read_csv("expression.csv", index_col=0).select_dtypes("number")
g = sns.clustermap(mat, z_score=0, cmap=DIVERGING_CMAP, center=0,
                   figsize=(8, 8), xticklabels=True, yticklabels=mat.shape[0] <= 60,
                   cbar_kws={"label": "z-score"})
g.ax_col_dendrogram.set_title("Expression heatmap", pad=12)
g.figure.savefig("figures/heatmap.png")

Correlation heatmap (sample QC):

import numpy as np, seaborn as sns, pandas as pd
corr = pd.read_csv("expr.csv", index_col=0).select_dtypes("number").corr(method="pearson")
mask = np.triu(np.ones_like(corr, dtype=bool), k=1)
fig, ax = plt.subplots(figsize=(7, 6))
sns.heatmap(corr, mask=mask, cmap=DIVERGING_CMAP, vmin=-1, vmax=1, center=0,
            annot=True, # if the n>20 samples then set annot=False
            fmt=".2f", annot_kws={"size": 7}, square=True,
            linewidths=0.5, 
            cbar_kws={"label": "Pearson r", "shrink": 0.7}, 
            ax=ax)
ax.set_title("Sample correlation"); fig.tight_layout(); fig.savefig("figures/corr.png")

GSEA bar (top gene sets, colored by direction):

import pandas as pd
df = pd.read_csv("gsea.csv").dropna(subset=["Term", "NES"]).copy()
df = df.assign(_a=df["NES"].abs()).nlargest(15, "_a").sort_values("NES")
colors = [UP if v >= 0 else DOWN for v in df["NES"]]
fig, ax = plt.subplots(figsize=(7, 6))
ax.barh(df["Term"].astype(str), df["NES"], color=colors)
ax.axvline(0, color="0.4", lw=0.8); ax.set_xlabel("NES"); ax.set_title("GSEA")
fig.tight_layout(); fig.savefig("figures/gsea_bar.png")

GSEA dot plot (clusterProfiler-style; GeneRatio may be "k/n"):

import numpy as np, pandas as pd
from matplotlib.cm import ScalarMappable
from matplotlib.colors import Normalize
df = pd.read_csv("enrichment.csv").dropna(subset=["Term", "GeneRatio", "Adjusted P-value"]).copy()
df["_ratio"] = df["GeneRatio"].map(lambda v: float(v.split("/")[0]) / float(v.split("/")[1]) if isinstance(v, str) and "/" in v else float(v))
df["_nlp"] = -np.log10(np.clip(df["Adjusted P-value"].astype(float), 1e-300, None))
df = df.nlargest(15, "_nlp").sort_values("_ratio")
cnt = df["Count"].to_numpy(float); sizes = 40 + 220 * (cnt - cnt.min()) / (np.ptp(cnt) + 1e-9)
norm = Normalize(df["_nlp"].min(), df["_nlp"].max())
fig, ax = plt.subplots(figsize=(7, 6))
ax.scatter(df["_ratio"], df["Term"].astype(str), s=sizes, c=df["_nlp"], cmap=SEQUENTIAL_CMAP, norm=norm, edgecolors="0.3", linewidths=0.5, zorder=3)
ax.grid(axis="y", ls=":", color="0.8", zorder=0); ax.set_xlabel("Gene ratio"); ax.set_title("Enrichment")
cb = fig.colorbar(ScalarMappable(norm=norm, cmap=SEQUENTIAL_CMAP), ax=ax, shrink=0.6, pad=0.02)
cb.set_label(r"$-\log_{10}$ adj. $p$"); fig.tight_layout(); fig.savefig("figures/dotplot.png")

Box plot (long-form, jittered points, optional 2-group test):

import seaborn as sns, pandas as pd
from scipy import stats
df = pd.read_csv("measurements.csv"); x, y = "group", "value"
fig, ax = plt.subplots(figsize=(6, 5))
sns.boxplot(data=df, x=x, y=y, hue=x, palette=PALETTE[:df[x].nunique()], legend=False, fliersize=0, width=0.6, ax=ax)
sns.stripplot(data=df, x=x, y=y, color="0.25", size=3, alpha=0.6, ax=ax)
lv = list(df[x].dropna().unique())
if len(lv) == 2:  # optional significance star
    a, b = (df.loc[df[x] == l, y].dropna() for l in lv)
    p = stats.mannwhitneyu(a, b, alternative="two-sided").pvalue
    star = "****" if p < 1e-4 else "***" if p < 1e-3 else "**" if p < 1e-2 else "*" if p < 0.05 else "ns"
    ymax = df[y].max(); h = (ymax - df[y].min()) * 0.08
    ax.plot([0, 0, 1, 1], [ymax + h, ymax + 2*h, ymax + 2*h, ymax + h], lw=1.0, c="0.2")
    ax.text(0.5, ymax + 2*h, star, ha="center", va="bottom")
ax.set_title(f"{y} by {x}"); fig.tight_layout(); fig.savefig("figures/box.png")

For violin swap sns.boxplot → sns.violinplot(..., inner="box", cut=0); for bar of the mean use sns.barplot(..., errorbar="se", capsize=0.15).

Grouped embedding scatter (PCA / UMAP / t-SNE / factor scatter): (samples × features + group column):

import numpy as np, pandas as pd

def scatter_emb(ax, emb, grp, xi=0, yi=1, ev=None, name="PC", centered=True):
    """emb: (n_samples, n_comp). ev: variance explained % or None. UMAP/t-SNE -> centered=False."""
    emb, grp = np.asarray(emb), pd.Series(grp).astype(str)
    for g in [g for g in GROUP_COLORS if g in set(grp)]:     
        m = (grp == g).to_numpy()
        ax.scatter(emb[m, xi], emb[m, yi], c=GROUP_COLORS[g], s=45, alpha=.9,
                   edgecolors="white", linewidths=.4, label=g)
    if centered:                   # PC/factor are 0-centered and equally spaced
        ax.axhline(0, lw=.5, ls="--", c="0.75"); ax.axvline(0, lw=.5, ls="--", c="0.75")
        ax.set_aspect("equal", adjustable="datalim")
    lab = lambda i: f"{name}{i+1}" + (f" ({ev[i]:.1f}%)" if ev is not None else "")
    ax.set_xlabel(lab(xi)); ax.set_ylabel(lab(yi))
    ax.spines[["top", "right"]].set_visible(False); ax.legend(frameon=False, fontsize=7)

FOR PCA: samples × features + 'group' column

from sklearn.decomposition import PCA

df = pd.read_csv("samples_features.csv", index_col=0)    
grp = df.pop("group") if "group" in df else pd.Series("all", index=df.index)
X = df.select_dtypes("number")
p = PCA(n_components=min(5, *X.shape)).fit(X) 
emb, ev = p.transform(X), p.explained_variance_ratio_ * 100
fig, ax = plt.subplots(figsize=(70/25.4, 70/25.4), constrained_layout=True)
scatter_emb(ax, emb, grp, 0, 1, ev, "PC")
fig.savefig("figures/pca.png", dpi=400)

FOR UMAP/t-SNE: no ev

import umap           # t-SNE: from sklearn.manifold import TSNE

emb = umap.UMAP(n_neighbors=15, min_dist=0.1).fit_transform(X)   # TSNE(perplexity=30).fit_transform(X)
fig, ax = plt.subplots(figsize=(70/25.4, 70/25.4), constrained_layout=True)
scatter_emb(ax, emb, grp, name="UMAP", centered=False)    # 원점·축 스케일에 의미 없음
fig.savefig("figures/umap.png", dpi=400)

FOR factor scatter:

Z  = pd.DataFrame(model.get_factors()["group1"], index=meta.index)
r2 = model.get_variance_explained()["r2_per_factor"]["group1"]    # view × factor
ev = r2.sum(axis=0).to_numpy()                           
keep = [i for i, v in enumerate(ev) if v >= 5.0]        
fig, ax = plt.subplots(figsize=(70/25.4, 70/25.4), constrained_layout=True)
scatter_emb(ax, Z, meta["condition"], keep[0], keep[1], ev, "Factor")
fig.savefig("figures/factor_scatter.png", dpi=400)

Kaplan–Meier (survival by group):

import pandas as pd
from lifelines import KaplanMeierFitter
df = pd.read_csv("survival.csv"); kmf = KaplanMeierFitter()
fig, ax = plt.subplots(figsize=(6, 5))
for i, g in enumerate(sorted(df["group"].unique())):
    m = df["group"] == g
    kmf.fit(df.loc[m, "time"], df.loc[m, "event"], label=str(g))
    kmf.plot_survival_function(ax=ax, color=PALETTE[i % len(PALETTE)], ci_show=True)
ax.set_xlabel("Time"); ax.set_ylabel("Survival probability"); ax.set_ylim(0, 1.02)
ax.set_title("Kaplan–Meier"); fig.tight_layout(); fig.savefig("figures/km.png")

Manhattan (GWAS summary stats; columns CHR, BP, P):

import numpy as np, pandas as pd
df = pd.read_csv("gwas.csv").dropna(subset=["CHR", "BP", "P"]).copy()
df["CHR"] = df["CHR"].astype(str)
chrom_order = sorted(df["CHR"].unique(), key=lambda c: (len(c), c))
df["_nlp"] = -np.log10(np.clip(df["P"].astype(float), 1e-300, None))
offset, xticks, ticklabels, xpos = 0.0, [], [], np.zeros(len(df))  # lay chromosomes end-to-end
for c in chrom_order:
    m = (df["CHR"] == c).to_numpy()
    bp = df.loc[m, "BP"].astype(float)
    xpos[m] = bp + offset
    xticks.append(offset + bp.median()); ticklabels.append(c)
    offset = xpos[m].max() + 1
fig, ax = plt.subplots(figsize=(180/25.4, 70/25.4), constrained_layout=True)
for i, c in enumerate(chrom_order):
    m = (df["CHR"] == c).to_numpy()
    ax.scatter(xpos[m], df["_nlp"].to_numpy()[m], s=6, c=["#2a78d6", "#9fb3c8"][i % 2], edgecolors="none")
ax.axhline(-np.log10(5e-8), color=UP, lw=0.8, ls="--")   # genome-wide significance
ax.set_xticks(xticks); ax.set_xticklabels(ticklabels, fontsize=6)
ax.set_xlabel("Chromosome"); ax.set_ylabel(r"$-\log_{10} p$"); ax.set_title("Manhattan")
fig.savefig("figures/manhattan.png", dpi=400)

QQ plot (p-value calibration; reports genomic inflation λ):

import numpy as np, pandas as pd
from scipy import stats
p = pd.read_csv("gwas.csv")["P"].dropna().astype(float).to_numpy()
p = np.clip(np.sort(p), 1e-300, 1.0)
obs = -np.log10(p)
exp = -np.log10((np.arange(1, len(p) + 1) - 0.5) / len(p))
lam = np.median(stats.chi2.isf(p, 1)) / stats.chi2.isf(0.5, 1)   # genomic inflation
fig, ax = plt.subplots(figsize=(70/25.4, 70/25.4), constrained_layout=True)
ax.scatter(exp, obs, s=6, c="#2a78d6", edgecolors="none")
lim = float(max(exp.max(), obs.max()))
ax.plot([0, lim], [0, lim], color="0.4", lw=0.8, ls="--")
ax.set_xlabel(r"Expected $-\log_{10} p$"); ax.set_ylabel(r"Observed $-\log_{10} p$")
ax.set_title(f"QQ (λ = {lam:.3f})"); fig.savefig("figures/qq.png", dpi=400)

Forest (effect size ± 95% CI per study/variant; columns label, estimate, ci_low, ci_high):

import pandas as pd
df = pd.read_csv("effects.csv").iloc[::-1].reset_index(drop=True)  # first row at top
y = list(range(len(df)))
fig, ax = plt.subplots(figsize=(90/25.4, max(60, 14 * len(df)) / 25.4), constrained_layout=True)
ax.errorbar(df["estimate"], y,
            xerr=[df["estimate"] - df["ci_low"], df["ci_high"] - df["estimate"]],
            fmt="s", color="#204897", ecolor="0.4", capsize=2, markersize=4, lw=0.8)
ax.axvline(1.0, color=UP, lw=0.8, ls="--")   # null: OR/HR=1 (use 0.0 for log-scale beta)
ax.set_yticks(y); ax.set_yticklabels(df["label"])
ax.set_xlabel("Effect size (95% CI)"); ax.set_title("Forest")
fig.savefig("figures/forest.png", dpi=400)

Best Practices

  • Only plot data that exists. Never invent columns, groups, or values; if a needed column is missing, inspect the header and ask or adapt — do not fabricate.
  • Workspace-relative paths only. Save under figures/ (create it if needed); never write to absolute paths like /tmp or /home/....
  • Reuse the palette across a figure set so related panels share colors for the same group / direction. Use the diverging colormap (centered at 0) for z-scores and log2FC; the sequential colormap for magnitudes and -log10 p. However, if the user explicitly requests a different color, use it.
  • Label axes and give a real title. Include units, group n, and thresholds where relevant (e.g. volcano cutoff lines).
  • For vector / print output, also save a .pdf (fig.savefig("figures/x.pdf")); text stays editable because pdf.fonttype=42 / svg.fonttype="none".

Common Pitfalls

  • Wrong colormap family for the data. A sequential map on signed values (log2FC, z-score) hides the sign. How to avoid: use the diverging colormap with center=0, vmin=-vmax for signed data, and reserve the sequential colormap for unsigned magnitudes only.
  • Columns don't match the recipe (KeyError). The recipe's default column names rarely match the real table. How to avoid: read the header first and override the *_col / x / y variables to the real names; never invent a missing column — pick a figure the data supports instead.
  • Overcrowded / overlapping labels. Annotating every volcano point, or cramming long pathway names on an x-axis, produces unreadable overlap. How to avoid: label only the top ~10 by |log2FC| × -log10 p (≤5 inside a composite panel) and repel with adjustText; move long category names to a horizontal y-axis, or rotate 90° and wrap to ≤26 chars; skip labels rather than dumping colliding text.
  • Dot / bubble markers all one size or too small. A size encoding that maps to invisible dots carries no information. How to avoid: floor the size mapping (s in ~[25, 220]) so the smallest stays visible, and confirm the size column actually varies.
  • Legend or clustermap title collides. A legend sits on top of the data, or a clustermap title is cropped. How to avoid: move the legend to a free corner or outside the axes (or shrink its font); put a clustermap title on g.ax_col_dendrogram, not a figure suptitle.
  • Empty / all-NS plot. Nothing passes the threshold because the columns or scale are wrong. How to avoid: confirm the p-value and fold-change columns are numeric and that thresholds match the data scale (adjusted vs raw p).

For multi-panel composition issues (panel letters misplaced, legends floating in margins, panels colliding), see the multipanel skill.

Further Reading

Plotly Interactive Plots

Overview

Plotly is a Python library for producing interactive, web-ready figures backed by HTML and JavaScript. It exposes two complementary APIs: plotly.express (px) provides a high-level, DataFrame-oriented interface for generating common chart types in one line, while plotly.graph_objects (go) offers fine-grained control over every trace, axis, and layout property. Figures are fully interactive by default — supporting hover tooltips, zoom, pan, and click events — and can be embedded in web pages, Jupyter notebooks, or built into web applications using the Dash framework.

When to Use

  • You need hover tooltips that display gene names, p-values, or sample metadata without cluttering the static figure.
  • You are building a multi-panel interactive dashboard for dose-response curves, patient cohorts, or multi-condition comparisons.
  • You want to share figures as self-contained HTML files that non-programmers can explore in a browser.
  • You need 3D scatter or surface plots for structural biology, conformational landscapes, or PCA of high-dimensional data.
  • You are creating heatmaps of gene expression or correlation matrices where users need to zoom into specific gene clusters.
  • You require animation frames to show time-series or treatment-response trajectories.
  • Use seaborn instead when you need automatic statistical aggregation (confidence intervals, regression fits) with minimal code.
  • Use matplotlib when you need fine-grained control over every axis element for print-ready publication figures at exact journal specifications.

Prerequisites

  • Python packages: plotly, kaleido (static image export), pandas, numpy
  • Data requirements: pandas DataFrames or NumPy arrays; long-form (tidy) data works best with px
  • Environment: Jupyter Lab/Notebook (inline rendering), or save as HTML for browser display
pip install plotly kaleido pandas numpy

For Jupyter Lab inline rendering (if not automatic):

pip install "jupyterlab>=3" ipywidgets

Quick Start

import plotly.express as px
import pandas as pd

# Gene expression scatter with hover info
df = pd.DataFrame({
    "log2FC": [-3.1, 0.2, 1.8, 2.5, -0.5, 4.1],
    "neg_log10_padj": [8.2, 0.4, 2.1, 6.8, 0.1, 9.3],
    "gene": ["BRCA1", "MYC", "TP53", "EGFR", "CDKN1A", "KRAS"],
    "significance": ["sig", "ns", "ns", "sig", "ns", "sig"],
})

fig = px.scatter(
    df, x="log2FC", y="neg_log10_padj",
    color="significance", hover_name="gene",
    title="Volcano Plot — Treatment vs Control",
)
fig.show()

Core API

Module 1: px Scatter and Line — Relational Plots

px.scatter() and px.line() map DataFrame columns to visual encodings (color, symbol, size) and automatically populate hover tooltips from hover_data.

import plotly.express as px
import pandas as pd
import numpy as np

# Dose-response scatter: color by drug, symbol by cell line
np.random.seed(42)
df = pd.DataFrame({
    "dose_uM": np.tile([0.01, 0.1, 1, 10, 100], 4),
    "viability": np.clip(np.random.normal(
        [100, 90, 70, 40, 10] * 4, 5), 0, 110),
    "drug": ["DrugA"] * 5 + ["DrugA"] * 5 + ["DrugB"] * 5 + ["DrugB"] * 5,
    "cell_line": ["HCT116"] * 10 + ["MCF7"] * 10,
    "replicate": np.tile([1, 2, 3, 4, 5], 4),
})

fig = px.scatter(
    df, x="dose_uM", y="viability",
    color="drug", symbol="cell_line",
    log_x=True,
    hover_data={"replicate": True, "dose_uM": ":.2f"},
    labels={"viability": "Cell Viability (%)", "dose_uM": "Dose (µM)"},
    title="Dose-Response by Drug and Cell Line",
)
fig.show()
print(f"Figure has {len(fig.data)} traces")
# Time-course gene expression line plot
time_df = pd.DataFrame({
    "hour": list(range(0, 25, 4)) * 3,
    "expression": [1.0, 1.8, 3.2, 4.5, 3.8, 2.1, 1.2,
                   1.0, 2.5, 5.1, 6.8, 5.5, 3.2, 1.8,
                   1.0, 1.1, 1.0, 1.2, 1.1, 1.0, 0.9],
    "gene": ["MYC"] * 7 + ["EGFR"] * 7 + ["GAPDH"] * 7,
})

fig = px.line(
    time_df, x="hour", y="expression",
    color="gene", markers=True,
    labels={"expression": "Relative Expression (log2)", "hour": "Time (h)"},
    title="Time-Course Gene Expression",
)
fig.update_traces(line=dict(width=2.5), marker=dict(size=8))
fig.show()

Module 2: px Statistical Plots — Distributions and Categories

px.box(), px.violin(), px.histogram(), and px.strip() produce publication-ready distribution summaries with built-in grouping.

import plotly.express as px
import pandas as pd
import numpy as np

# Violin + strip overlay: expression by cell type
np.random.seed(7)
n = 60
cell_data = pd.DataFrame({
    "expression": np.concatenate([
        np.random.normal(4.2, 0.8, n),
        np.random.normal(6.5, 1.2, n),
        np.random.normal(2.8, 0.6, n),
    ]),
    "cell_type": ["T cell"] * n + ["B cell"] * n + ["NK cell"] * n,
    "patient_id": np.tile([f"P{i:02d}" for i in range(1, 11)], 18),
})

fig = px.violin(
    cell_data, x="cell_type", y="expression",
    color="cell_type", box=True, points="all",
    hover_data=["patient_id"],
    labels={"expression": "CD3E Expression (log2 CPM)"},
    title="CD3E Expression Across Cell Types",
)
fig.update_traces(jitter=0.3, pointpos=-1.5)
fig.show()
print(f"Cells per type: {cell_data.groupby('cell_type').size().to_dict()}")
# Histogram with rug: distribution of fold changes
fc_df = pd.DataFrame({
    "log2FC": np.concatenate([
        np.random.normal(0.1, 0.8, 500),   # not DE genes
        np.random.normal(2.5, 0.4, 50),    # upregulated
        np.random.normal(-2.3, 0.4, 40),   # downregulated
    ]),
    "category": ["background"] * 500 + ["up"] * 50 + ["down"] * 40,
})

fig = px.histogram(
    fc_df, x="log2FC", color="category",
    nbins=60, barmode="overlay", opacity=0.7,
    marginal="rug",
    labels={"log2FC": "log2 Fold Change", "count": "Gene Count"},
    title="Distribution of Fold Changes (DESeq2 Results)",
    color_discrete_map={"background": "gray", "up": "crimson", "down": "steelblue"},
)
fig.show()

Module 3: px Heatmap and Matrix — Gene Expression and Correlations

px.imshow() renders 2D arrays or DataFrames as color-encoded matrices, ideal for expression heatmaps and correlation matrices.

import plotly.express as px
import pandas as pd
import numpy as np

# Gene expression heatmap (genes × samples)
np.random.seed(12)
genes = [f"Gene_{g}" for g in ["BRCA1", "TP53", "EGFR", "MYC", "KRAS",
                                 "CDKN1A", "RB1", "PTEN", "VHL", "APC"]]
samples = [f"S{i:02d}" for i in range(1, 9)]

expr_matrix = pd.DataFrame(
    np.random.normal(0, 1.5, (10, 8)) +
    np.array([2, -1, 3, -2, 1, -3, 0, 2, -1, 3]).reshape(-1, 1),
    index=genes, columns=samples,
)

fig = px.imshow(
    expr_matrix,
    color_continuous_scale="RdBu_r",
    color_continuous_midpoint=0,
    aspect="auto",
    labels={"color": "log2 Expression (z-score)"},
    title="Gene Expression Heatmap",
)
fig.update_xaxes(side="top")
fig.update_layout(width=600, height=500)
fig.show()
print(f"Heatmap shape: {expr_matrix.shape} (genes × samples)")
# Correlation matrix heatmap
from itertools import combinations

markers = ["IL6", "TNF", "CXCL10", "IFNg", "IL10", "IL1B", "CCL2", "IL17A"]
np.random.seed(3)
raw = np.random.multivariate_normal(
    mean=np.zeros(8),
    cov=np.eye(8) * 0.3 + 0.7,
    size=80,
)
corr_df = pd.DataFrame(raw, columns=markers).corr()

fig = px.imshow(
    corr_df,
    color_continuous_scale="RdBu_r",
    color_continuous_midpoint=0,
    zmin=-1, zmax=1,
    text_auto=".2f",
    title="Cytokine Correlation Matrix (n=80 patients)",
)
fig.update_traces(textfont_size=10)
fig.show()

Module 4: go Graph Objects — Full Trace Control

plotly.graph_objects provides fine-grained access to every trace property: marker symbols, error bars, fill areas, and multi-trace layouts. Essential when px lacks the flexibility you need.

import plotly.graph_objects as go
import numpy as np

# Volcano plot built from scratch with go.Scatter
np.random.seed(99)
n_genes = 5000
log2fc = np.random.normal(0, 1.2, n_genes)
pval = np.random.uniform(0, 1, n_genes) ** 2  # skew toward low p-values
neg_log10_p = -np.log10(pval + 1e-300)
gene_names = [f"Gene_{i:04d}" for i in range(n_genes)]

# Classify genes
sig_mask = (np.abs(log2fc) > 1.5) & (neg_log10_p > 3)
up_mask = sig_mask & (log2fc > 0)
down_mask = sig_mask & (log2fc < 0)
ns_mask = ~sig_mask

fig = go.Figure()

# Non-significant background
fig.add_trace(go.Scatter(
    x=log2fc[ns_mask], y=neg_log10_p[ns_mask],
    mode="markers",
    name="Not significant",
    marker=dict(color="lightgray", size=4, opacity=0.5),
    text=[gene_names[i] for i in np.where(ns_mask)[0]],
    hovertemplate="<b>%{text}</b><br>log2FC: %{x:.2f}<br>-log10(p): %{y:.2f}<extra></extra>",
))

# Upregulated
fig.add_trace(go.Scatter(
    x=log2fc[up_mask], y=neg_log10_p[up_mask],
    mode="markers",
    name=f"Up ({up_mask.sum()} genes)",
    marker=dict(color="crimson", size=7, opacity=0.8),
    text=[gene_names[i] for i in np.where(up_mask)[0]],
    hovertemplate="<b>%{text}</b><br>log2FC: %{x:.2f}<br>-log10(p): %{y:.2f}<extra></extra>",
))

# Downregulated
fig.add_trace(go.Scatter(
    x=log2fc[down_mask], y=neg_log10_p[down_mask],
    mode="markers",
    name=f"Down ({down_mask.sum()} genes)",
    marker=dict(color="steelblue", size=7, opacity=0.8),
    text=[gene_names[i] for i in np.where(down_mask)[0]],
    hovertemplate="<b>%{text}</b><br>log2FC: %{x:.2f}<br>-log10(p): %{y:.2f}<extra></extra>",
))

# Threshold lines
fig.add_hline(y=3, line_dash="dash", line_color="black", line_width=1)
fig.add_vline(x=1.5, line_dash="dash", line_color="black", line_width=1)
fig.add_vline(x=-1.5, line_dash="dash", line_color="black", line_width=1)

fig.update_layout(
    title="Volcano Plot (Treatment vs Control, n=5000 genes)",
    xaxis_title="log2 Fold Change",
    yaxis_title="-log10(adjusted p-value)",
    legend=dict(x=0.01, y=0.99),
    width=750, height=550,
)
fig.show()
print(f"Up: {up_mask.sum()}, Down: {down_mask.sum()}, NS: {ns_mask.sum()}")
# Bar chart with error bars: mean ± SEM per treatment group
groups = ["Vehicle", "DrugA 1µM", "DrugA 10µM", "DrugB 1µM", "DrugB 10µM"]
means = [100.0, 82.3, 54.7, 91.2, 68.5]
sems = [3.2, 4.1, 3.8, 3.5, 4.7]

fig = go.Figure(go.Bar(
    x=groups, y=means,
    error_y=dict(type="data", array=sems, visible=True),
    marker_color=["gray", "lightsalmon", "crimson", "lightblue", "steelblue"],
    hovertemplate="%{x}<br>Mean: %{y:.1f}%<br>SEM: ±%{error_y.array:.1f}%<extra></extra>",
))

fig.update_layout(
    title="Cell Viability by Treatment (Mean ± SEM, n=6)",
    yaxis_title="Viability (%)", yaxis_range=[0, 120],
    xaxis_title="Treatment Group",
    showlegend=False,
)
fig.show()

Module 5: 3D and Specialized Charts

Plotly supports 3D scatter, surface plots, parallel coordinates, and treemaps — chart types unavailable in seaborn or standard matplotlib.

import plotly.express as px
import numpy as np
import pandas as pd

# 3D PCA scatter: cell clusters in embedding space
np.random.seed(42)
n_per_cluster = 80
cluster_centers = {"T cell": [3, 2, 1], "B cell": [-3, 1, 2], "Monocyte": [0, -3, -1]}

records = []
for ctype, center in cluster_centers.items():
    coords = np.random.normal(center, 0.8, (n_per_cluster, 3))
    for row in coords:
        records.append({
            "PC1": row[0], "PC2": row[1], "PC3": row[2],
            "cell_type": ctype,
            "score": np.random.uniform(0.5, 1.0),
        })

pca_df = pd.DataFrame(records)

fig = px.scatter_3d(
    pca_df, x="PC1", y="PC2", z="PC3",
    color="cell_type", size="score", opacity=0.7,
    hover_data={"score": ":.3f"},
    title="3D PCA — Single-Cell Transcriptomics",
)
fig.update_traces(marker=dict(sizeref=0.04))
fig.show()
print(f"Total cells: {len(pca_df)}, clusters: {pca_df['cell_type'].nunique()}")
import plotly.graph_objects as go
import numpy as np
import pandas as pd

# Parallel coordinates: multi-parameter drug screen
np.random.seed(5)
n_compounds = 200
drug_df = pd.DataFrame({
    "MW": np.random.normal(380, 60, n_compounds),
    "logP": np.random.uniform(-1, 6, n_compounds),
    "HBA": np.random.randint(2, 10, n_compounds),
    "HBD": np.random.randint(0, 6, n_compounds),
    "IC50_nM": np.random.lognormal(4, 1.5, n_compounds),
    "selectivity": np.random.uniform(1, 100, n_compounds),
})

fig = px.parallel_coordinates(
    drug_df,
    color="IC50_nM",
    color_continuous_scale="RdYlGn_r",
    dimensions=["MW", "logP", "HBA", "HBD", "IC50_nM", "selectivity"],
    labels={
        "MW": "MW (Da)", "logP": "logP",
        "HBA": "H-Bond Acceptors", "HBD": "H-Bond Donors",
        "IC50_nM": "IC50 (nM)", "selectivity": "Selectivity Index",
    },
    title="Drug Candidate Properties — Parallel Coordinates",
)
fig.show()

Module 6: Subplots and Export

make_subplots() creates multi-panel layouts with shared axes, mixed chart types, and independent traces per panel. fig.write_html() exports interactive figures; fig.write_image() exports static files via kaleido.

from plotly.subplots import make_subplots
import plotly.graph_objects as go
import numpy as np

# Two-panel: raw data + summary statistics
np.random.seed(77)
doses = [0.01, 0.1, 1, 10, 100]
drugs = {"DrugA": {"EC50": 1.0, "hill": 1.5}, "DrugB": {"EC50": 8.0, "hill": 0.9}}

def hill_curve(dose, ec50, hill, top=100, bottom=0):
    return bottom + (top - bottom) / (1 + (ec50 / dose) ** hill)

fig = make_subplots(
    rows=1, cols=2,
    subplot_titles=["Dose-Response Curves", "IC50 Comparison"],
    shared_yaxes=False,
)

colors = {"DrugA": "crimson", "DrugB": "steelblue"}
ic50_values = []

for drug, params in drugs.items():
    # Smooth fit curve
    x_fit = np.logspace(-2, 2, 200)
    y_fit = hill_curve(x_fit, params["EC50"], params["hill"])
    fig.add_trace(go.Scatter(
        x=x_fit, y=y_fit, mode="lines",
        name=f"{drug} fit", line=dict(color=colors[drug], width=2.5),
    ), row=1, col=1)

    # Noisy data points
    y_data = [hill_curve(d, params["EC50"], params["hill"]) +
              np.random.normal(0, 4) for d in doses]
    fig.add_trace(go.Scatter(
        x=doses, y=y_data, mode="markers",
        name=f"{drug} data", marker=dict(color=colors[drug], size=9),
        showlegend=False,
    ), row=1, col=1)
    ic50_values.append(params["EC50"])

# Bar chart of IC50 values
fig.add_trace(go.Bar(
    x=list(drugs.keys()), y=ic50_values,
    marker_color=list(colors.values()),
    showlegend=False,
    hovertemplate="%{x}<br>IC50: %{y:.2f} µM<extra></extra>",
), row=1, col=2)

fig.update_xaxes(type="log", title_text="Dose (µM)", row=1, col=1)
fig.update_yaxes(title_text="Viability (%)", row=1, col=1)
fig.update_xaxes(title_text="Drug", row=1, col=2)
fig.update_yaxes(title_text="IC50 (µM)", row=1, col=2)
fig.update_layout(title="Dose-Response Dashboard", height=450, width=850)
fig.show()
print(f"Subplots: {len(fig.data)} traces across 2 panels")
# Export to HTML (interactive) and PNG (static)
# Requires: pip install kaleido

fig.write_html("dose_response_dashboard.html")
print("Saved: dose_response_dashboard.html (interactive, shareable)")

fig.write_image("dose_response_dashboard.png", width=1200, height=600, scale=2)
print("Saved: dose_response_dashboard.png (300 DPI equivalent with scale=2)")

fig.write_image("dose_response_dashboard.svg")
print("Saved: dose_response_dashboard.svg (vector, editable in Inkscape/Illustrator)")

Common Workflows

Workflow 1: Interactive Volcano Plot with Gene Annotations

Goal: Build a fully annotated volcano plot from DESeq2 results, with gene-name hover tooltips, threshold lines, and highlighted hit labels for sharing as HTML.

import plotly.graph_objects as go
import pandas as pd
import numpy as np

# Simulate DESeq2 output (replace with pd.read_csv("deseq2_results.csv"))
np.random.seed(42)
n = 3000
df = pd.DataFrame({
    "gene": [f"GENE_{i:04d}" for i in range(n)],
    "log2FC": np.random.normal(0, 1.0, n),
    "padj": np.clip(np.random.exponential(0.1, n), 1e-20, 1.0),
    "baseMean": np.random.lognormal(5, 1.5, n),
})
# Inject some hits
df.loc[:20, "log2FC"] = np.random.uniform(2.5, 5, 21)
df.loc[:20, "padj"] = np.random.uniform(1e-15, 1e-5, 21)
df.loc[21:35, "log2FC"] = np.random.uniform(-4, -2, 15)
df.loc[21:35, "padj"] = np.random.uniform(1e-12, 1e-4, 15)

df["neg_log10_padj"] = -np.log10(df["padj"].clip(1e-300))

# Classify
FC_THRESH, P_THRESH = 1.5, 2.0   # |log2FC| > 1.5, -log10(padj) > 2
df["category"] = "NS"
df.loc[(df["log2FC"] > FC_THRESH) & (df["neg_log10_padj"] > P_THRESH), "category"] = "Up"
df.loc[(df["log2FC"] < -FC_THRESH) & (df["neg_log10_padj"] > P_THRESH), "category"] = "Down"

color_map = {"NS": "lightgray", "Up": "crimson", "Down": "steelblue"}
size_map = {"NS": 4, "Up": 7, "Down": 7}
opacity_map = {"NS": 0.4, "Up": 0.85, "Down": 0.85}

fig = go.Figure()

for cat in ["NS", "Up", "Down"]:
    sub = df[df["category"] == cat]
    fig.add_trace(go.Scatter(
        x=sub["log2FC"], y=sub["neg_log10_padj"],
        mode="markers",
        name=f"{cat} (n={len(sub)})",
        marker=dict(
            color=color_map[cat],
            size=size_map[cat],
            opacity=opacity_map[cat],
        ),
        customdata=sub[["gene", "padj", "baseMean"]].values,
        hovertemplate=(
            "<b>%{customdata[0]}</b><br>"
            "log2FC: %{x:.3f}<br>"
            "padj: %{customdata[1]:.2e}<br>"
            "baseMean: %{customdata[2]:.1f}<extra></extra>"
        ),
    ))

# Threshold lines
fig.add_hline(y=P_THRESH, line_dash="dot", line_color="black", line_width=1.2,
              annotation_text=f"padj=0.01", annotation_position="right")
fig.add_vline(x=FC_THRESH, line_dash="dot", line_color="black", line_width=1.2)
fig.add_vline(x=-FC_THRESH, line_dash="dot", line_color="black", line_width=1.2)

# Label top 5 upregulated hits by significance
top_up = df[df["category"] == "Up"].nlargest(5, "neg_log10_padj")
for _, row in top_up.iterrows():
    fig.add_annotation(
        x=row["log2FC"], y=row["neg_log10_padj"],
        text=row["gene"], showarrow=True,
        arrowhead=2, arrowsize=1, arrowcolor="crimson",
        font=dict(size=9, color="crimson"),
        xshift=8, yshift=5,
    )

fig.update_layout(
    title="Volcano Plot — Treatment vs Control (DESeq2)",
    xaxis_title="log2 Fold Change",
    yaxis_title="-log10(adjusted p-value)",
    legend=dict(x=0.01, y=0.99, bordercolor="lightgray", borderwidth=1),
    width=800, height=560,
    plot_bgcolor="white",
)
fig.update_xaxes(showgrid=True, gridcolor="lightgray", zeroline=True, zerolinecolor="darkgray")
fig.update_yaxes(showgrid=True, gridcolor="lightgray")

fig.write_html("volcano_interactive.html")
print(f"Up: {(df.category=='Up').sum()}, Down: {(df.category=='Down').sum()}")
print("Saved: volcano_interactive.html")

Workflow 2: Multi-Panel Dose-Response Dashboard with make_subplots

Goal: Display dose-response curves for multiple drugs across cell lines in a grid layout with a shared color scale and consistent formatting.

from plotly.subplots import make_subplots
import plotly.graph_objects as go
import numpy as np
import pandas as pd

# Simulated IC50 data for 3 drugs × 3 cell lines
np.random.seed(10)
drugs = ["DrugA", "DrugB", "DrugC"]
cell_lines = ["HCT116", "MCF7", "A549"]
doses = np.logspace(-2, 2, 7)  # 0.01 to 100 µM

def hill(x, ec50, hill_n, top=100, bottom=0):
    return bottom + (top - bottom) / (1 + (ec50 / x) ** hill_n)

ec50_table = {
    ("DrugA", "HCT116"): 0.5, ("DrugA", "MCF7"): 2.0, ("DrugA", "A549"): 8.0,
    ("DrugB", "HCT116"): 5.0, ("DrugB", "MCF7"): 0.8, ("DrugB", "A549"): 15.0,
    ("DrugC", "HCT116"): 12.0, ("DrugC", "MCF7"): 6.0, ("DrugC", "A549"): 1.2,
}

palette = px_colors = ["#EF553B", "#636EFA", "#00CC96", "#AB63FA", "#FFA15A",
                        "#19D3F3", "#FF6692", "#B6E880", "#FF97FF"]

fig = make_subplots(
    rows=len(drugs), cols=len(cell_lines),
    subplot_titles=[f"{d} / {c}" for d in drugs for c in cell_lines],
    shared_xaxes=True, shared_yaxes=True,
    vertical_spacing=0.08, horizontal_spacing=0.04,
)

for r, drug in enumerate(drugs, start=1):
    for c, cell_line in enumerate(cell_lines, start=1):
        ec50 = ec50_table[(drug, cell_line)]
        x_fit = np.logspace(-2, 2, 200)
        y_fit = hill(x_fit, ec50, hill_n=1.5)

        # Noisy replicate data
        y_data = np.array([hill(d, ec50, 1.5) + np.random.normal(0, 5) for d in doses])

        color = palette[(r - 1) * len(cell_lines) + (c - 1)]
        show_legend = (c == 1 and r == 1)

        fig.add_trace(go.Scatter(
            x=x_fit, y=y_fit, mode="lines",
            line=dict(color=color, width=2),
            name=f"{drug}/{cell_line}",
            showlegend=False,
            hovertemplate=f"{drug} in {cell_line}<br>Dose: %{{x:.2f}} µM<br>Viability: %{{y:.1f}}%<extra></extra>",
        ), row=r, col=c)

        fig.add_trace(go.Scatter(
            x=doses, y=np.clip(y_data, 0, 110), mode="markers",
            marker=dict(color=color, size=7, opacity=0.8),
            showlegend=False,
            hovertemplate=f"Measured<br>Dose: %{{x:.2f}} µM<br>Viability: %{{y:.1f}}%<extra></extra>",
        ), row=r, col=c)

        # IC50 annotation
        fig.add_annotation(
            x=np.log10(ec50), y=50,
            text=f"IC50={ec50:.1f}µM",
            font=dict(size=8), showarrow=False,
            xref=f"x{(r-1)*len(cell_lines)+c if (r-1)*len(cell_lines)+c > 1 else ''}",
            yref=f"y{(r-1)*len(cell_lines)+c if (r-1)*len(cell_lines)+c > 1 else ''}",
        )

# Apply log scale to all x-axes
for i in range(1, len(drugs) * len(cell_lines) + 1):
    axis_key = f"xaxis{i if i > 1 else ''}"
    fig.layout[axis_key].update(type="log", title_text="Dose (µM)" if i > 6 else "")

for i in range(1, len(drugs) * len(cell_lines) + 1):
    axis_key = f"yaxis{i if i > 1 else ''}"
    fig.layout[axis_key].update(range=[-5, 115],
                                 title_text="Viability (%)" if i in [1, 4, 7] else "")

fig.update_layout(
    title="Dose-Response Dashboard — 3 Drugs × 3 Cell Lines",
    height=700, width=900,
)
fig.write_html("dose_response_dashboard.html")
print("Saved: dose_response_dashboard.html")
print(f"Grid: {len(drugs)} drugs × {len(cell_lines)} cell lines = {len(drugs)*len(cell_lines)} panels")

Key Parameters

ParameterModule / FunctionDefaultRange / OptionsEffect
colorpx.*NoneColumn nameMaps a DataFrame column to trace color; auto-assigns palette
hover_datapx.*{}Dict or list of column namesExtra columns shown in hover tooltip
log_x / log_ypx.*FalseTrue, FalseApply log10 scale to x or y axis
facet_col / facet_rowpx.*NoneColumn nameSplit into subplot grid by a categorical variable
color_continuous_scalepx.imshow, px.scatter"plasma""RdBu_r", "Viridis", "Hot", etc.Colormap for continuous color mapping
color_continuous_midpointpx.imshowNoneAny numericCenters the diverging colormap at this value (use 0 for z-scores)
barmodepx.histogram, px.bar"relative""relative", "overlay", "group"How multiple bar traces are displayed
opacitygo.Scatter, px.*1.00.0–1.0Point/bar transparency
size / sizerefgo.Scatter6 / autoPositive numericMarker size; sizeref normalizes sizes across traces
line_dashfig.add_hline, go.Scatter"solid""solid", "dash", "dot", "dashdot"Line style for reference lines and traces
shared_xaxes / shared_yaxesmake_subplotsFalseTrue, False, "rows", "cols"Link axes across subplot panels
scalefig.write_image11–4Resolution multiplier for PNG export (use 2 for ~150 DPI)

Best Practices

  1. Prefer px for DataFrame data, fall back to go for multi-trace composition. Use px.scatter() and its siblings for 80% of plots. Switch to go when you need traces with different types in the same figure (e.g., scatter + filled area) or need fine-grained per-trace control.

    # Correct: px for simple grouped plots
    fig = px.box(df, x="treatment", y="expression", color="genotype")
    
    # Correct: go when px cannot express the structure
    fig = go.Figure()
    fig.add_trace(go.Scatter(x=x, y=y_upper, fill="tonexty", ...))
    fig.add_trace(go.Scatter(x=x, y=y_lower, ...))
    
  2. Always include hovertemplate for scientific figures. The default tooltip shows raw coordinates without units or gene names. A custom template with customdata provides full biological context.

    fig.add_trace(go.Scatter(
        customdata=df[["gene", "padj"]].values,
        hovertemplate="<b>%{customdata[0]}</b><br>padj: %{customdata[1]:.2e}<extra></extra>",
    ))
    
  3. Export HTML for sharing, PNG/SVG for journals. fig.write_html() produces a self-contained file with no external dependencies. Use scale=2 or higher with write_image() to achieve sufficient resolution for print.

  4. Don't use fig.show() in batch scripts. In non-interactive contexts (CI, HPC, cron jobs), fig.show() may open a browser window or fail. Use write_html() or write_image() exclusively.

  5. Use color_continuous_midpoint=0 for diverging palettes on z-score data. Without it, the midpoint color defaults to the data midpoint, not zero, misrepresenting symmetric fold changes or correlations.

    fig = px.imshow(corr_matrix, color_continuous_scale="RdBu_r",
                    color_continuous_midpoint=0, zmin=-1, zmax=1)
    
  6. Set plot_bgcolor="white" for publication figures. Plotly defaults to a light-gray grid background. White background with subtle gridlines is cleaner for most scientific contexts.

    fig.update_layout(plot_bgcolor="white")
    fig.update_xaxes(showgrid=True, gridcolor="lightgray")
    fig.update_yaxes(showgrid=True, gridcolor="lightgray")
    

Common Recipes

Recipe: Dropdown Menu to Toggle Between Conditions

When to use: Overlay multiple conditions in one figure with a dropdown button to show/hide individual traces cleanly.

import plotly.graph_objects as go
import numpy as np

conditions = ["Untreated", "DrugA", "DrugB"]
colors = ["gray", "crimson", "steelblue"]
np.random.seed(1)
x = np.linspace(0, 24, 49)

fig = go.Figure()
for i, (cond, color) in enumerate(zip(conditions, colors)):
    y = np.sin(x / 4 + i * 0.5) * (1 - i * 0.2) + np.random.normal(0, 0.05, len(x))
    fig.add_trace(go.Scatter(
        x=x, y=y, mode="lines+markers",
        name=cond, line=dict(color=color, width=2),
        visible=(i == 0),  # only first trace visible initially
    ))

# One button per condition (shows only that trace)
buttons = []
for i, cond in enumerate(conditions):
    visibility = [j == i for j in range(len(conditions))]
    buttons.append(dict(label=cond, method="update",
                        args=[{"visible": visibility}, {"title": f"Gene Expression — {cond}"}]))

# "Show All" button
buttons.append(dict(label="Show All", method="update",
                    args=[{"visible": [True] * len(conditions)}, {"title": "Gene Expression — All Conditions"}]))

fig.update_layout(
    updatemenus=[dict(type="dropdown", x=0.01, y=1.15, showactive=True, buttons=buttons)],
    title="Gene Expression — Untreated",
    xaxis_title="Time (h)", yaxis_title="Relative Expression",
)
fig.show()

Recipe: Annotating Specific Hits with Arrows

When to use: Label outliers, drug hits, or significant genes directly on the figure without cluttering non-annotated points.

import plotly.graph_objects as go
import numpy as np
import pandas as pd

np.random.seed(33)
df = pd.DataFrame({
    "x": np.random.normal(0, 1.5, 300),
    "y": np.random.normal(0, 1.5, 300),
    "gene": [f"G{i:03d}" for i in range(300)],
})
# Inject top hits
hits = pd.DataFrame({
    "x": [3.2, -2.8, 2.5, -3.5],
    "y": [4.1, 3.8, -3.2, -2.9],
    "gene": ["BRCA1", "TP53", "EGFR", "KRAS"],
})

fig = go.Figure()
fig.add_trace(go.Scatter(
    x=df["x"], y=df["y"], mode="markers",
    marker=dict(color="lightgray", size=5, opacity=0.6),
    text=df["gene"],
    hovertemplate="<b>%{text}</b><br>x: %{x:.2f}, y: %{y:.2f}<extra></extra>",
    name="Background",
))
fig.add_trace(go.Scatter(
    x=hits["x"], y=hits["y"], mode="markers",
    marker=dict(color="crimson", size=10, symbol="diamond"),
    text=hits["gene"],
    hovertemplate="<b>%{text}</b> [HIT]<br>x: %{x:.2f}, y: %{y:.2f}<extra></extra>",
    name="Hits",
))

for _, row in hits.iterrows():
    fig.add_annotation(
        x=row["x"], y=row["y"],
        text=f"<b>{row['gene']}</b>",
        showarrow=True, arrowhead=2, arrowwidth=1.5,
        arrowcolor="crimson", font=dict(size=11, color="crimson"),
        ax=25, ay=-30,  # arrow offset in pixels
        bgcolor="rgba(255,255,255,0.7)", bordercolor="crimson", borderwidth=1,
    )

fig.update_layout(
    title="Hit Identification with Arrow Annotations",
    xaxis_title="Score A", yaxis_title="Score B",
    plot_bgcolor="white",
)
fig.show()

Troubleshooting

ProblemCauseSolution
ValueError: kaleido is required for static image exportkaleido not installedpip install kaleido; verify with import kaleido
Blank figure in Jupyter NotebookRenderer not configuredRun import plotly.io as pio; pio.renderers.default = "notebook" or upgrade JupyterLab to ≥3
fig.show() opens blank browser tabNo data in figure or offline renderer issueCheck len(fig.data) > 0; use pio.renderers.default = "browser"
Hover tooltips show wrong valuescustomdata index mismatch in hovertemplateVerify customdata column order matches %{customdata[N]} indices in template
Colors not assigned consistently across tracespx re-orders palette when category counts differUse color_discrete_map={"Cat1": "#color1", ...} to pin colors explicitly
write_image produces blurry PNGDefault scale=1 is too low for printUse fig.write_image("fig.png", scale=2) for 150 DPI or scale=4 for 300 DPI
Subplots x-axes not all log-scaled after shared_xaxes=TrueShared axis only synchronizes range, not typeIterate over all xaxis keys in fig.layout and set type="log" explicitly
Large datasets slow to render in browserToo many individual points in a single scatter traceDownsample background noise points; keep labeled hits as a separate, smaller trace
fig.update_layout does not apply to subplot axesMulti-panel figures use indexed axes (xaxis2, xaxis3)Use fig.update_xaxes() (applies to all) or target fig.layout["xaxis2"] explicitly

Related Skills

  • seaborn-statistical-plots — use for statistical aggregation (confidence intervals, regression), publication-quality static figures with minimal code, and when matplotlib-level output is required
  • matplotlib-scientific-plotting — use for full control over every figure element, custom layouts, embedded text rendering, and journal-specification figure preparation
  • pydeseq2-differential-expression — volcano plot outputs from DESeq2 results are a primary input for the interactive volcano workflow above
  • scanpy-scrna-seq — Scanpy's UMAP embeddings can be visualized interactively in 3D with px.scatter_3d

References

scientific-visualization

Overview

Effective scientific visualization communicates data clearly, honestly, and accessibly. Poor chart choices, misleading axes, or inaccessible color palettes can obscure findings or introduce bias. This guide covers the full workflow of scientific figure preparation: from selecting the right chart type for your data structure through color theory, accessibility, and journal submission formatting requirements.

Key Concepts

Chart Type and Data Type Alignment

Every chart type is optimized for a specific data structure. Mismatches (e.g., pie charts for continuous distributions, bar charts for time series) hide structure and distort perception.

Data TypeRecommended ChartAvoid
Continuous distribution (1 group)Histogram, violin plot, ridge plotBar chart with mean only
Continuous distribution (2–5 groups)Violin + boxplot overlay, beeswarmGrouped bar chart
Two continuous variables, correlationScatter plot, hexbin (large N)Line chart without temporal order
Categorical counts / proportionsBar chart (horizontal for long labels)Pie chart (>4 categories)
Change over time (continuous)Line chartBar chart
Change over time (sparse events)Step chart, event rasterConnected scatter
Part-to-whole (≤5 parts)Stacked bar, waffle chart3D pie chart
High-dimensional (>5 variables)Heatmap (clustered), parallel coordinates3D scatter
Spatial dataMap, spatial heatmapBubble chart
Survival / time-to-eventKaplan-Meier curveBar chart of median survival

Color Theory for Science

Color encodes information. Misused color introduces artifacts and fails readers with color vision deficiency (CVD; ~8% of males).

Sequential palettes encode ordered numeric data from low to high (e.g., expression level, concentration). Use perceptually uniform palettes: viridis, magma, cividis. These also print in grayscale.

Diverging palettes encode data with a meaningful midpoint (e.g., fold-change centered at 0, correlation from -1 to +1). Use RdBu, coolwarm, or vlag. Always ensure the midpoint maps to white/neutral.

Qualitative palettes encode unordered categories. Use Okabe-Ito (CVD-safe), tab10 (matplotlib default), or ColorBrewer qualitative palettes. Limit to ≤8 distinguishable colors; use shape or pattern as redundant encoding beyond that.

Color don'ts:

  • Rainbow/jet colormap: not perceptually uniform; creates false contours
  • Red vs. green encoding: fails deuteranopia (~6% males)
  • Saturated color for background or large areas

Figure Composition and Layout

Scientific figures are typically multi-panel. Panel layout and labeling affect how readers parse information.

  • Panel labels: Bold uppercase letters (A, B, C) in the top-left corner; use 8–12 pt in the figure, larger in the caption reference.
  • Alignment: Align panel edges on a grid. Unaligned panels signal lack of attention to detail.
  • White space: Leave adequate margins; crowded panels reduce readability.
  • Figure size: Design for the target column width — single column (~85 mm / 3.35 in), 1.5 column (~114 mm / 4.5 in), or double column (~170 mm / 6.7 in) for Nature-family journals.
  • Font: Sans-serif (Arial, Helvetica) at 6–8 pt minimum in the final figure at publication resolution.

Journal Formatting Requirements

Major journals specify exact figure requirements for submission. Violating these causes desk-rejection delays.

Journal/StyleMax WidthResolutionColor ModeFontFile Format
Nature family89 mm (1-col), 183 mm (2-col)300 dpi (photos), 600 dpi (line art)RGB or CMYKArial 5–7 ptPDF, TIFF, EPS
Cell/iScience85 mm (1-col), 170 mm (2-col)300 dpi raster, 600 dpi halftoneRGBHelvetica 6–8 ptPDF, EPS, TIFF
ACS journals3.25 in (1-col), 7 in (2-col)600 dpi (color), 1200 dpi (b&w line art)RGB (screen), CMYK (print)Arial/Helvetica 4.5–7 ptTIFF, EPS, PDF
PLOS ONENo strict width300 dpi (raster), 600–1200 dpi (line art)RGBAnyTIFF, EPS, PDF

Decision Framework

Use this tree to select the right visualization for your analysis goal:

What is the primary message of this figure?
|
+-- Show a distribution or spread of values
|   +-- One group         --> Histogram or violin plot
|   +-- 2-5 groups        --> Violin + jitter (show all points if N < 100)
|   +-- Many groups       --> Ridge plot (joy plot)
|
+-- Compare quantities between categories
|   +-- Few categories (2-5)    --> Bar chart with error bars + individual points
|   +-- Many categories (>8)    --> Lollipop chart or dot plot (horizontal)
|   +-- Paired measurements     --> Slopegraph or paired dot plot
|
+-- Show a relationship between two continuous variables
|   +-- N < 1000                --> Scatter plot
|   +-- N > 1000                --> Hexbin or 2D density plot
|   +-- Time ordered            --> Line chart
|
+-- Show composition or part-to-whole
|   +-- 2-4 parts               --> Stacked bar or waffle chart
|   +-- Over time               --> Stacked area chart
|   +-- Avoid pie chart unless <= 3 parts and proportions are obvious
|
+-- Show high-dimensional data
|   +-- Genes x samples         --> Clustered heatmap (seaborn.clustermap)
|   +-- Embeddings (UMAP, PCA)  --> Scatter colored by metadata
|   +-- Feature importance      --> Horizontal bar chart (sorted)
|
+-- Show spatial or geographic data
|   +-- Microscopy              --> Image overlay with colorbar
|   +-- Geographic              --> Choropleth map
Analysis GoalChart TypeLibraryKey Consideration
Gene expression across groupsViolin + jitterseaborn, plotnineShow all points if N < 50; never bar+SEM only
Differential expressionVolcano plotmatplotlibLog2FC on x-axis, -log10(p) on y-axis
Clustering resultsUMAP scatterscanpy, matplotlibOne plot per annotation variable
Correlation matrixClustered heatmapseaborn.clustermapUse diverging palette centered at 0
Protein structureRibbon diagramPyMOL, ChimeraXNot covered here — use dedicated molecular graphics tools
Survival analysisKaplan-MeierlifelinesInclude confidence bands and at-risk table
Time courseLine chart with CImatplotlibShow uncertainty; connect group means, not individual points

Best Practices

  1. Show the data, not just summaries: For N < 100, overlay individual data points on violin or box plots using jitter or beeswarm. Bar charts with only mean ± SEM conceal distribution shape, outliers, and bimodality.

  2. Choose CVD-safe color palettes by default: Use Okabe-Ito or viridis/cividis for sequential data. Test your figure with a CVD simulator (e.g., Coblis) before submission.

  3. Design at final publication size from the start: Set your figure canvas to the exact column width of the target journal (e.g., 89 mm for Nature single-column). Rescaling after the fact makes fonts too small or too large, and changes aspect ratios.

  4. Label axes with units and use descriptive titles: Every axis must have a label with units in parentheses (e.g., "Expression level (log2 CPM)"). Avoid cryptic abbreviations without legend entries.

  5. Use vector formats for line art and text: Save figures as PDF or SVG when they contain text and lines. Rasterize only when submitting to a journal that requires TIFF. Vector figures scale without pixelation and remain editable.

  6. Match statistical annotations to the test performed: If you annotate significance stars (*), state in the caption which test was used, the exact p-value, and the sample size. "n.s." should still report the p-value.

  7. Avoid dual y-axes: Two different y-axes on one plot are almost always misleading — the apparent relationship depends on scale choices. Use two separate panels instead.

Common Pitfalls

  1. Bar chart for continuous distributions (dynamite plot)

    • What goes wrong: Mean ± SEM bars hide whether data is normally distributed, multimodal, or has outliers. Two datasets with identical bars can have completely different distributions.
    • How to avoid: Use violin plots, box-and-whisker with jitter, or beeswarm plots. Show all data points when N < 50.
  2. Truncated or broken y-axis that exaggerates differences

    • What goes wrong: Starting the y-axis at a non-zero value makes small differences appear large and misleads readers about effect size.
    • How to avoid: Start y-axis at zero for ratio/count data. If zooming in is genuinely needed, use an inset or a separate panel with a clear label indicating the truncation.
  3. Using rainbow/jet colormap for heatmaps

    • What goes wrong: Jet is not perceptually uniform — it creates false contour lines and fails under CVD and grayscale printing.
    • How to avoid: Use viridis, magma, or inferno for sequential; RdBu or coolwarm for diverging data. These are the defaults in seaborn >= 0.12.
  4. Overlapping data points without jitter or transparency

    • What goes wrong: Points stack on top of each other (overplotting), making dense regions appear sparse and hiding the true data density.
    • How to avoid: Add jitter (seaborn.stripplot(jitter=True)), use transparency (alpha=0.3), or switch to a hexbin / 2D density plot for large N.
  5. P-value annotations without effect size or sample size

    • What goes wrong: A statistically significant result (p < 0.05) with N = 10,000 may be scientifically trivial. Stars alone are uninformative.
    • How to avoid: Report exact p-values, effect sizes (Cohen's d, odds ratio), and sample sizes in the figure caption or directly on the plot.
  6. Figure text too small at publication size

    • What goes wrong: Designing at screen size and then shrinking to journal column width results in 3–4 pt text that fails minimum readability standards.
    • How to avoid: Design at final print size from the start. Check that axis labels are ≥ 6 pt at the final dimensions.
  7. Inconsistent style across panels in a multi-panel figure

    • What goes wrong: Different fonts, line widths, or color schemes across panels make the figure look unprofessional and harder to compare.
    • How to avoid: Define a shared matplotlib.rcParams style dictionary at the top of your figure script and apply it to all panels.

Workflow

  1. Define the message first

    • Write one sentence describing what you want the reader to take away from this figure.
    • Choose the chart type that most directly communicates that message using the Decision Framework above.
  2. Prepare the data

    • Aggregate, tidy, and validate your data before plotting.
    • Check for outliers that should be investigated, not silently dropped.
  3. Prototype at screen resolution

    • Build a draft figure in your preferred library (matplotlib, seaborn, ggplot2, plotnine).
    • Focus on getting data and chart type right; ignore styling at this stage.
  4. Apply journal-specific styling

    • Set canvas size to target journal column width.
    • Set font family and minimum size.
    • Choose CVD-safe palette.
    • Standardize line widths (0.5–1.0 pt for axes and ticks, 1.0–2.0 pt for data lines).
  5. Add annotations and labels

    • Panel labels (A, B, C) in bold.
    • Axis labels with units.
    • Statistical annotations with test name and exact p-value.
    • Legend with descriptive entries.
  6. Export at correct resolution

    • Vector: PDF or SVG for line art and text.
    • Raster: TIFF at 300 dpi (halftone/photos) or 600 dpi (line art).
    • Use plt.savefig("fig1.pdf", bbox_inches="tight") in matplotlib.
  7. Accessibility check

    • Simulate CVD using an online tool or GIMP color blindness filter.
    • Print in grayscale and verify panels remain distinguishable.

Protocol Guidelines

  1. For matplotlib/seaborn: Set rcParams at the top of every figure script; use fig.set_size_inches() to enforce journal dimensions; export with dpi=300 minimum.
  2. For R/ggplot2: Use theme_classic() or a custom theme; set ggsave(width=..., units="mm", dpi=300).
  3. For multi-panel assembly: Use matplotlib.gridspec, patchworklib (Python), or cowplot/patchwork (R) for aligned panel grids.
  4. For interactive figures: Plotly or Bokeh for HTML outputs; Vega-Lite for web embedding. Always provide a static fallback for publication.
  5. For poster presentations: Scale all fonts up by 1.5–2×; use larger markers (pt size 8–12) and thicker lines (2–3 pt); prefer high-contrast, dark background palettes.

Further Reading

Related Skills

  • matplotlib-figures — Python implementation of publication-quality figures with matplotlib and seaborn
  • data-visualization — general Python plotting recipes
  • biostatistics — statistical test selection to accompany figure annotations

Seaborn — Statistical Plots

Overview

Seaborn is a Python library for statistical data visualization built on top of matplotlib. It works directly with pandas DataFrames, automatically handles grouping by categorical variables, computes confidence intervals and kernel density estimates, and produces attractive publication-ready figures with minimal configuration. Seaborn separates axes-level functions (embeddable in custom layouts) from figure-level functions (with built-in faceting), enabling both quick exploratory analysis and structured multi-panel figures.

When to Use

  • Comparing gene expression, protein abundance, or measurement distributions across experimental conditions (treatment vs. control, cell lines, time points)
  • Generating grouped box plots, violin plots, or strip plots to show both summary statistics and individual data points simultaneously
  • Visualizing pairwise correlations in multi-gene or multi-feature datasets as annotated heatmaps
  • Plotting regression fits with confidence bands between continuous variables (e.g., cell viability vs. drug concentration)
  • Faceting a single plot type across multiple sample subsets, tissue types, or experimental batches in one call
  • Rapid exploratory analysis of a new dataset using pairplot to survey all pairwise relationships at once
  • Use matplotlib directly when you need pixel-level control over figure elements, complex mixed-type layouts, or non-statistical custom plots
  • Use plotly when the output must be interactive (hover tooltips, zoom, pan) or embedded in a web application

Prerequisites

  • Python packages: seaborn>=0.13, matplotlib, pandas, numpy
  • Data requirements: Pandas DataFrame in long-form (tidy) format; each observation is a row, each variable is a column
  • Environment: Standard Python environment; no GPU or special hardware required
pip install "seaborn>=0.13" matplotlib pandas numpy scipy

Quick Start

import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# Simulate gene expression across conditions
rng = np.random.default_rng(42)
df = pd.DataFrame({
    "gene":      ["BRCA1"] * 60 + ["TP53"] * 60,
    "condition": ["control", "treated"] * 60,
    "log2_expr": np.concatenate([
        rng.normal(5.2, 0.8, 60),
        rng.normal(6.1, 0.9, 60),
    ])
})

sns.set_theme(style="ticks", context="notebook")
sns.boxplot(data=df, x="gene", y="log2_expr", hue="condition", palette="Set2")
plt.ylabel("log2 Expression")
plt.title("Gene Expression by Condition")
plt.tight_layout()
plt.savefig("quickstart_boxplot.png", dpi=150)
print("Saved quickstart_boxplot.png")

Core API

1. Distribution Plots

Visualize univariate distributions and compare them across groups. histplot bins data; kdeplot fits a smooth density estimate; displot is the figure-level wrapper that adds faceting.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(0)
n = 200
df = pd.DataFrame({
    "log2_tpm":  np.concatenate([rng.normal(4.5, 1.1, n), rng.normal(6.0, 1.3, n)]),
    "sample":    ["tumor"] * n + ["normal"] * n,
})

fig, axes = plt.subplots(1, 3, figsize=(15, 4))

# Histogram with density normalization and stacked hue groups
sns.histplot(data=df, x="log2_tpm", hue="sample", stat="density",
             multiple="stack", bins=30, ax=axes[0])
axes[0].set_title("Histogram (stacked)")

# KDE with fill — bandwidth controlled by bw_adjust
sns.kdeplot(data=df, x="log2_tpm", hue="sample", fill=True,
            bw_adjust=0.8, alpha=0.4, ax=axes[1])
axes[1].set_title("KDE (filled)")

# ECDF — useful for comparing cumulative distributions
sns.ecdfplot(data=df, x="log2_tpm", hue="sample", ax=axes[2])
axes[2].set_title("ECDF")

plt.tight_layout()
plt.savefig("distributions.png", dpi=150)
print("Saved distributions.png")
# Bivariate KDE: joint distribution of two continuous variables
rng = np.random.default_rng(1)
df2 = pd.DataFrame({
    "log2_rna": rng.normal(5.5, 1.2, 300),
    "log2_prot": rng.normal(4.8, 1.0, 300) + 0.6 * rng.normal(5.5, 1.2, 300),
})
sns.kdeplot(data=df2, x="log2_rna", y="log2_prot",
            fill=True, levels=8, thresh=0.05, cmap="Blues")
plt.xlabel("log2 RNA (TPM)")
plt.ylabel("log2 Protein (iBAQ)")
plt.title("RNA–Protein Correlation Density")
plt.tight_layout()
plt.savefig("bivariate_kde.png", dpi=150)
print("Saved bivariate_kde.png")

2. Categorical Plots

Compare distributions or aggregated statistics across categorical groups. Axes-level functions (boxplot, violinplot, stripplot, swarmplot, barplot) accept an ax= parameter for embedding in custom layouts.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(2)
conditions = ["DMSO", "Drug A 1uM", "Drug A 10uM", "Drug B 1uM", "Drug B 10uM"]
df = pd.DataFrame({
    "condition": np.repeat(conditions, 30),
    "viability": np.concatenate([
        rng.normal(100, 5, 30),
        rng.normal(92, 7, 30),
        rng.normal(65, 10, 30),
        rng.normal(88, 8, 30),
        rng.normal(45, 12, 30),
    ])
})

fig, axes = plt.subplots(1, 3, figsize=(18, 5))

# Box plot — shows quartiles and outliers
sns.boxplot(data=df, x="condition", y="viability",
            palette="husl", width=0.5, ax=axes[0])
axes[0].set_xticklabels(axes[0].get_xticklabels(), rotation=30, ha="right")
axes[0].set_title("Box Plot")

# Violin — KDE shape + inner quartile lines
sns.violinplot(data=df, x="condition", y="viability",
               inner="quart", palette="muted", ax=axes[1])
axes[1].set_xticklabels(axes[1].get_xticklabels(), rotation=30, ha="right")
axes[1].set_title("Violin Plot")

# Strip plot overlaid on box — shows all individual points
sns.boxplot(data=df, x="condition", y="viability",
            palette="pastel", width=0.5, ax=axes[2])
sns.stripplot(data=df, x="condition", y="viability",
              color="black", alpha=0.4, size=3, jitter=True, ax=axes[2])
axes[2].set_xticklabels(axes[2].get_xticklabels(), rotation=30, ha="right")
axes[2].set_title("Box + Strip")

plt.tight_layout()
plt.savefig("categorical.png", dpi=150)
print("Saved categorical.png")
# Bar plot with mean ± 95% CI and individual points (swarm)
fig, ax = plt.subplots(figsize=(8, 5))
sns.barplot(data=df, x="condition", y="viability",
            estimator="mean", errorbar="ci", palette="Set3", ax=ax)
sns.swarmplot(data=df, x="condition", y="viability",
              color="black", size=3, alpha=0.5, ax=ax)
ax.set_ylabel("Cell Viability (%)")
ax.set_xticklabels(ax.get_xticklabels(), rotation=30, ha="right")
plt.tight_layout()
plt.savefig("barswarm.png", dpi=150)
print("Saved barswarm.png")

3. Relational Plots

Visualize relationships between continuous variables. scatterplot and lineplot are axes-level; relplot is the figure-level wrapper that supports col and row faceting.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(3)
n = 150
df = pd.DataFrame({
    "molecular_weight": rng.uniform(200, 800, n),
    "logP":             rng.uniform(-2, 6, n),
    "pIC50":            rng.normal(6.5, 1.2, n),
    "target_class":     rng.choice(["kinase", "GPCR", "protease"], n),
    "pass_lipinski":    rng.choice(["yes", "no"], n, p=[0.7, 0.3]),
})

# Scatter with hue (categorical color) + size (continuous) + style (marker)
sns.scatterplot(data=df, x="molecular_weight", y="pIC50",
                hue="target_class", size="logP", style="pass_lipinski",
                sizes=(30, 120), alpha=0.7)
plt.xlabel("Molecular Weight (Da)")
plt.ylabel("pIC50")
plt.title("Compound Bioactivity by Target Class")
plt.tight_layout()
plt.savefig("relational_scatter.png", dpi=150)
print("Saved relational_scatter.png")
# Line plot with automatic mean aggregation and SD error band across replicates
timepoints = [0, 1, 2, 4, 8, 24]
groups = ["untreated", "low_dose", "high_dose"]
rows = []
for grp, base in zip(groups, [100.0, 95.0, 80.0]):
    for tp in timepoints:
        for _ in range(5):  # 5 replicates
            rows.append({"timepoint_h": tp, "group": grp,
                         "confluency": base * np.exp(-0.02 * tp * (1 + rng.normal(0, 0.1)))})
time_df = pd.DataFrame(rows)

sns.lineplot(data=time_df, x="timepoint_h", y="confluency",
             hue="group", style="group", errorbar="sd", markers=True, dashes=False)
plt.xlabel("Time (h)")
plt.ylabel("Confluency (%)")
plt.title("Cell Growth Inhibition (mean ± SD, n=5)")
plt.tight_layout()
plt.savefig("lineplot.png", dpi=150)
print("Saved lineplot.png")

4. Regression Plots

Fit linear (or polynomial/lowess) models and visualize them with confidence bands. regplot is axes-level; lmplot is figure-level with faceting support.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(4)
n = 120
tumor_size = rng.uniform(0.5, 6.0, n)
survival_months = 40 - 5 * tumor_size + rng.normal(0, 4, n)
grade = rng.choice(["low", "high"], n, p=[0.5, 0.5])
df = pd.DataFrame({"tumor_size_cm": tumor_size,
                   "survival_months": survival_months,
                   "grade": grade})

fig, axes = plt.subplots(1, 2, figsize=(13, 5))

# Linear regression with 95% CI band
sns.regplot(data=df, x="tumor_size_cm", y="survival_months",
            ci=95, scatter_kws={"alpha": 0.4, "s": 25}, ax=axes[0])
axes[0].set_title("Linear Regression (95% CI)")

# Residuals plot — check for homoscedasticity
sns.residplot(data=df, x="tumor_size_cm", y="survival_months",
              scatter_kws={"alpha": 0.4, "s": 25}, ax=axes[1])
axes[1].axhline(0, color="red", linestyle="--", linewidth=1)
axes[1].set_title("Residuals vs Fitted")

plt.tight_layout()
plt.savefig("regression.png", dpi=150)
print("Saved regression.png")
# lmplot — figure-level: separate regression lines per grade (hue) + facets
g = sns.lmplot(data=df, x="tumor_size_cm", y="survival_months",
               hue="grade", col="grade", ci=95,
               scatter_kws={"alpha": 0.4}, height=4, aspect=1.1)
g.set_axis_labels("Tumor Size (cm)", "Survival (months)")
g.set_titles("{col_name} grade")
g.savefig("lmplot_faceted.png", dpi=150)
print("Saved lmplot_faceted.png")

5. Matrix Plots

Visualize rectangular data as color-encoded matrices. heatmap is axes-level; clustermap is figure-level and applies hierarchical clustering to rows and columns.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(5)
genes = [f"GENE{i}" for i in range(1, 9)]
samples = [f"S{i}" for i in range(1, 7)]

# Simulate log2 fold-change matrix (rows=genes, cols=samples)
lfc = pd.DataFrame(
    rng.normal(0, 1.5, size=(8, 6)),
    index=genes, columns=samples
)
# Inject a pattern: first 3 genes up in samples 1-3, down in 4-6
lfc.iloc[:3, :3] += 2.5
lfc.iloc[:3, 3:] -= 2.5

# Correlation heatmap of numeric features
df_num = pd.DataFrame(
    rng.standard_normal((80, 5)),
    columns=["GeneA", "GeneB", "GeneC", "GeneD", "GeneE"]
)
df_num["GeneB"] = df_num["GeneA"] * 0.85 + rng.normal(0, 0.3, 80)
corr = df_num.corr()

fig, axes = plt.subplots(1, 2, figsize=(14, 5))

sns.heatmap(corr, annot=True, fmt=".2f", cmap="coolwarm",
            center=0, square=True, linewidths=0.5, ax=axes[0])
axes[0].set_title("Pearson Correlation Heatmap")

sns.heatmap(lfc, cmap="RdBu_r", center=0, annot=True, fmt=".1f",
            linewidths=0.3, cbar_kws={"label": "log2FC"}, ax=axes[1])
axes[1].set_title("log2 Fold Change Matrix")

plt.tight_layout()
plt.savefig("heatmaps.png", dpi=150)
print("Saved heatmaps.png")
# Clustermap with hierarchical clustering and row/column color annotations
rng = np.random.default_rng(6)
n_genes, n_samples = 30, 16
expr = pd.DataFrame(
    rng.lognormal(mean=2.0, sigma=1.2, size=(n_genes, n_samples)),
    index=[f"GENE{i:03d}" for i in range(n_genes)],
    columns=[f"{'T' if i < 8 else 'N'}{i:02d}" for i in range(n_samples)]
)

# Column annotation colors (tumor vs normal)
col_colors = ["#D32F2F" if c.startswith("T") else "#1976D2" for c in expr.columns]

g = sns.clustermap(
    np.log2(expr + 1),
    cmap="viridis",
    standard_scale=0,          # z-score across rows (genes)
    method="ward",
    metric="euclidean",
    col_colors=col_colors,
    figsize=(12, 10),
    linewidths=0,
    cbar_pos=(0.02, 0.8, 0.03, 0.15),
    cbar_kws={"label": "Row z-score"},
)
g.ax_heatmap.set_xlabel("Sample")
g.ax_heatmap.set_ylabel("Gene")
plt.savefig("clustermap.png", dpi=150, bbox_inches="tight")
print("Saved clustermap.png")

6. Multi-Variable Grids

Survey all pairwise relationships with pairplot or display a bivariate distribution with marginals using jointplot. For fully custom grid layouts, use FacetGrid directly.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(7)
n = 60
df = pd.DataFrame({
    "cell_area":      rng.normal(350, 60, n * 3),
    "nucleus_area":   rng.normal(90, 15, n * 3),
    "mean_intensity": rng.exponential(500, n * 3),
    "aspect_ratio":   np.abs(rng.normal(1.3, 0.3, n * 3)),
    "cell_type":      (["HeLa"] * n + ["MCF7"] * n + ["A549"] * n),
})

# Pairplot — matrix of pairwise scatter + KDE on diagonal
g = sns.pairplot(df, hue="cell_type", corner=True,
                 diag_kind="kde", plot_kws={"alpha": 0.5, "s": 20})
g.savefig("pairplot.png", dpi=150)
print("Saved pairplot.png")
# Jointplot — bivariate KDE with marginal histograms
g = sns.jointplot(data=df, x="cell_area", y="nucleus_area",
                  hue="cell_type", kind="scatter",
                  marginal_kws={"fill": True, "alpha": 0.3})
g.set_axis_labels("Cell Area (µm²)", "Nucleus Area (µm²)")
g.savefig("jointplot.png", dpi=150)
print("Saved jointplot.png")
# FacetGrid — custom layout: KDE of mean_intensity per cell type
g = sns.FacetGrid(df, col="cell_type", height=3.5, aspect=1.1,
                  sharey=False)
g.map(sns.histplot, "mean_intensity", bins=20, kde=True, color="steelblue")
g.set_axis_labels("Mean Intensity (AU)", "Count")
g.set_titles("{col_name}")
g.tight_layout()
g.savefig("facetgrid_intensity.png", dpi=150)
print("Saved facetgrid_intensity.png")

Key Concepts

Figure-Level vs Axes-Level Functions

Seaborn has two tiers of functions with different return types and composability:

FeatureAxes-LevelFigure-Level
Examplesscatterplot, histplot, boxplot, heatmap, regplotrelplot, displot, catplot, lmplot
Returnsmatplotlib.axes.AxesFacetGrid / JointGrid / PairGrid
FacetingManual (create subplots yourself)Built-in (col=, row= params)
Sizingfigsize= on parent figureheight= + aspect= per facet panel
Placementax= parameterCannot be placed in an existing axes
Savingplt.savefig(...)g.savefig(...)
Use whenCombining different plot types in one figureQuick multi-panel exploratory views
# Axes-level: place in a pre-allocated subplot grid
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
sns.violinplot(data=df, x="cell_type", y="cell_area", ax=axes[0])
sns.scatterplot(data=df, x="cell_area", y="nucleus_area", hue="cell_type", ax=axes[1])

Long-Form vs Wide-Form Data

Seaborn semantic mappings (hue, size, style) require long-form (tidy) data where each variable is a column and each observation is a row. Some functions (heatmap, clustermap, lineplot) also accept wide-form.

# Wide-form: unsuitable for hue/style mappings
#   sample_A  sample_B  sample_C
# 0      5.1       6.2       4.8

# Long-form (preferred): melt wide → long
wide = pd.DataFrame({"sampleA": [5.1, 4.3], "sampleB": [6.2, 5.9]})
long = wide.melt(var_name="sample", value_name="log2_expr")
# → columns: sample, log2_expr

Common Workflows

Workflow 1: Differential Expression Scatter with Significance Thresholds

Goal: Visualize log2 fold-change vs -log10 p-value (volcano-style) with significance annotations, colored by regulation status, and labeled top hits.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
n = 500
lfc   = rng.normal(0, 1.5, n)
pvals = 10 ** (-rng.exponential(1.5, n))        # skewed toward low significance
pvals = np.clip(pvals, 1e-20, 1.0)
genes = [f"GENE{i:04d}" for i in range(n)]

df_de = pd.DataFrame({"gene": genes, "log2fc": lfc, "pvalue": pvals})
df_de["neg_log10_p"] = -np.log10(df_de["pvalue"])

# Classify regulation status
lfc_thresh = 1.0
padj_thresh = 0.05
df_de["sig"] = "NS"
df_de.loc[(df_de["log2fc"] >  lfc_thresh) & (df_de["pvalue"] < padj_thresh), "sig"] = "Up"
df_de.loc[(df_de["log2fc"] < -lfc_thresh) & (df_de["pvalue"] < padj_thresh), "sig"] = "Down"

palette = {"NS": "#AAAAAA", "Up": "#D32F2F", "Down": "#1976D2"}

sns.set_theme(style="ticks", context="paper", font_scale=1.1)
fig, ax = plt.subplots(figsize=(8, 6))

sns.scatterplot(data=df_de, x="log2fc", y="neg_log10_p",
                hue="sig", palette=palette,
                alpha=0.6, s=18, linewidth=0, ax=ax)

# Threshold lines
ax.axhline(-np.log10(padj_thresh), color="black", linestyle="--", linewidth=0.8)
ax.axvline( lfc_thresh,            color="black", linestyle="--", linewidth=0.8)
ax.axvline(-lfc_thresh,            color="black", linestyle="--", linewidth=0.8)

# Label top 5 most significant genes per direction
for direction in ["Up", "Down"]:
    top = df_de[df_de["sig"] == direction].nlargest(5, "neg_log10_p")
    for _, row in top.iterrows():
        ax.text(row["log2fc"], row["neg_log10_p"] + 0.3, row["gene"],
                fontsize=6, ha="center", va="bottom",
                color=palette[direction])

# Annotation counts
n_up   = (df_de["sig"] == "Up").sum()
n_down = (df_de["sig"] == "Down").sum()
ax.set_title(f"Volcano Plot  |  Up: {n_up}  Down: {n_down}")
ax.set_xlabel("log2 Fold Change")
ax.set_ylabel("-log10 p-value")
sns.despine(trim=True)
plt.tight_layout()
plt.savefig("volcano_plot.png", dpi=300, bbox_inches="tight")
print(f"Volcano: {n_up} up, {n_down} down — saved volcano_plot.png")

Workflow 2: Multi-Condition Comparison with Grouped Violin + Strip Plots

Goal: Compare gene expression (or any continuous measurement) across multiple treatments and time points, showing full distributions plus individual replicates.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(99)
genes    = ["BRCA1", "TP53", "EGFR"]
treats   = ["DMSO", "Drug A", "Drug B"]
timepoints = ["6h", "24h", "48h"]
rows = []
for gene in genes:
    base_expr = {"BRCA1": 7.5, "TP53": 6.2, "EGFR": 8.1}[gene]
    for treat in treats:
        treat_shift = {"DMSO": 0.0, "Drug A": -0.8, "Drug B": 0.6}[treat]
        for tp in timepoints:
            tp_shift = {"6h": 0.0, "24h": 0.3, "48h": 0.6}[tp]
            for _ in range(12):
                rows.append({
                    "gene":      gene,
                    "treatment": treat,
                    "timepoint": tp,
                    "log2_expr": base_expr + treat_shift + tp_shift + rng.normal(0, 0.5),
                })
df_mc = pd.DataFrame(rows)

sns.set_theme(style="whitegrid", context="paper", font_scale=1.0)
g = sns.catplot(
    data=df_mc,
    x="timepoint", y="log2_expr",
    hue="treatment",
    col="gene",
    kind="violin",
    inner="quart",
    dodge=True,
    palette="Set2",
    height=4, aspect=0.9,
    col_order=genes,
    order=timepoints,
)

# Overlay individual points
for ax in g.axes.flat:
    gene_label = ax.get_title()
    gene_name  = gene_label.split(" = ")[-1] if " = " in gene_label else gene_label
    subset = df_mc[df_mc["gene"] == gene_name]
    sns.stripplot(
        data=subset,
        x="timepoint", y="log2_expr",
        hue="treatment",
        dodge=True,
        jitter=True,
        size=2.5,
        alpha=0.4,
        palette="dark:black",
        order=timepoints,
        legend=False,
        ax=ax,
    )

g.set_axis_labels("Timepoint", "log2 Expression")
g.set_titles("{col_name}")
g.add_legend(title="Treatment")
sns.despine(trim=True)
g.tight_layout()
g.savefig("multigroup_violin.png", dpi=300, bbox_inches="tight")
print("Saved multigroup_violin.png")

Workflow 3: Pairwise Feature Exploration for Cell Morphology

Goal: Quickly survey pairwise relationships in a multi-feature cell morphology dataset using pairplot, then examine one key pair with a jointplot.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

rng = np.random.default_rng(12)
n_per_type = 80
df_morph = pd.DataFrame({
    "cell_area_um2":    np.concatenate([rng.normal(320, 50, n_per_type),
                                        rng.normal(420, 70, n_per_type),
                                        rng.normal(280, 40, n_per_type)]),
    "nucleus_area_um2": np.concatenate([rng.normal(85, 12, n_per_type),
                                        rng.normal(110, 18, n_per_type),
                                        rng.normal(75, 10, n_per_type)]),
    "eccentricity":     np.abs(np.concatenate([rng.normal(0.6, 0.12, n_per_type),
                                               rng.normal(0.8, 0.10, n_per_type),
                                               rng.normal(0.5, 0.09, n_per_type)])),
    "mean_dapi":        np.concatenate([rng.exponential(400, n_per_type),
                                        rng.exponential(600, n_per_type),
                                        rng.exponential(350, n_per_type)]),
    "cell_line":        ["HeLa"] * n_per_type + ["MCF7"] * n_per_type + ["U2OS"] * n_per_type,
})

# 1. Pairplot survey
g = sns.pairplot(df_morph, hue="cell_line", corner=True,
                 diag_kind="kde", plot_kws={"alpha": 0.5, "s": 15},
                 palette="Dark2")
g.savefig("morphology_pairplot.png", dpi=150)
print("Saved morphology_pairplot.png")

# 2. Focused jointplot for the most informative pair
g2 = sns.jointplot(data=df_morph, x="cell_area_um2", y="nucleus_area_um2",
                   hue="cell_line", kind="scatter",
                   marginal_kws={"fill": True, "alpha": 0.25},
                   palette="Dark2", alpha=0.6)
g2.set_axis_labels("Cell Area (µm²)", "Nucleus Area (µm²)")
g2.savefig("morphology_jointplot.png", dpi=150)
print("Saved morphology_jointplot.png")

Key Parameters

ParameterFunction(s)DefaultRange / OptionsEffect
hueAll plot functionsNoneColumn name (categorical or continuous)Color-encodes a variable; triggers automatic legend
stylescatterplot, lineplotNoneCategorical column nameEncodes variable with marker shape or line dash pattern
sizescatterplot, lineplotNoneCategorical or continuous columnEncodes variable via point or line size
col / rowFigure-level only (relplot, displot, catplot, lmplot)NoneCategorical column nameCreates one subplot panel per unique value
col_wrapFigure-level onlyNoneintWraps columns onto a new row after N panels
estimatorbarplot, pointplot"mean""mean", "median", any callableAggregation function applied within each category
errorbarbarplot, lineplot, pointplot("ci", 95)"ci", "sd", "se", "pi", NoneError bar type displayed around the estimate
stathistplot"count""count", "frequency", "density", "probability"Normalization applied to histogram bar heights
bw_adjustkdeplot, violinplot1.00.1–3.0KDE bandwidth multiplier; lower=spikier, higher=smoother
multiplehistplot, kdeplot"layer""layer", "stack", "dodge", "fill"How overlapping hue groups are drawn
innerviolinplot"box""box", "quart", "point", "stick", NoneInterior annotation inside the violin body
standard_scaleclustermapNone0 (rows), 1 (columns)Z-score normalization axis before clustering
dodgeboxplot, violinplot, stripplotVariesTrue, FalseSeparate hue-grouped elements along the axis
contextset_theme()"notebook""paper", "notebook", "talk", "poster"Scales font and line widths for output medium

Best Practices

  1. Prefer long-form DataFrames with named columns: Seaborn's semantic mapping (hue, style, size) reads variable names directly from column names. Passing raw arrays loses axis labels and legends. Use pd.melt() to convert wide-form data.

  2. Call set_theme() once at the top of a script: This sets the global style, context, and palette for all subsequent plots, ensuring consistency. Reset to defaults with sns.set_theme().

    sns.set_theme(style="ticks", context="paper", font_scale=1.1,
                  rc={"axes.spines.right": False, "axes.spines.top": False})
    
  3. Use axes-level functions for mixed-type custom layouts: Figure-level functions (relplot, catplot) create their own figure and cannot be placed in an existing Axes. When combining different plot types (e.g., scatter + violin + heatmap), allocate a plt.subplots() grid and use axes-level functions with ax=.

  4. Use colorblind-safe palettes: sns.set_palette("colorblind") or palette="colorblind" produces a palette distinguishable by readers with common color vision deficiencies. For diverging data, use "RdBu_r" or "coolwarm" with center=0.

  5. Overlay individual data points on summary plots: Violin and bar plots hide distribution shape and sample size. Overlaying a stripplot or swarmplot with alpha=0.4 and small size conveys data density without obscuring the summary statistic.

  6. Size figure-level plots with height and aspect, not figsize: Figure-level functions ignore figsize. Use height= (inches per panel) and aspect= (width-to-height ratio per panel). For axes-level, set figsize on the plt.subplots() call.

  7. Anti-pattern — calling plt.savefig() on a figure-level grid: Figure-level functions return a FacetGrid/JointGrid object. Save it with g.savefig("out.png", dpi=300, bbox_inches="tight"), not plt.savefig(), which may capture a blank figure.

Common Recipes

Recipe: Publication-Ready Figure with Custom Palette and 300 DPI Export

When to use: Preparing a multi-panel figure for journal submission or a slide deck.

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

sns.set_theme(style="ticks", context="paper", font_scale=1.2,
              rc={"pdf.fonttype": 42, "ps.fonttype": 42})

rng = np.random.default_rng(7)
df = pd.DataFrame({
    "condition": np.repeat(["Control", "Treated"], 40),
    "ki67_pct":  np.concatenate([rng.normal(18, 4, 40), rng.normal(32, 6, 40)]),
    "apoptosis": np.concatenate([rng.normal(5, 1.5, 40), rng.normal(12, 2.5, 40)]),
})

custom_palette = {"Control": "#4575B4", "Treated": "#D73027"}

fig, axes = plt.subplots(1, 2, figsize=(8, 4))

# Panel A
sns.boxplot(data=df, x="condition", y="ki67_pct",
            palette=custom_palette, width=0.45, linewidth=1.2, ax=axes[0])
sns.stripplot(data=df, x="condition", y="ki67_pct",
              color="black", alpha=0.35, size=3, jitter=True, ax=axes[0])
axes[0].set_ylabel("Ki67 Positive Cells (%)")
axes[0].set_xlabel("")
axes[0].set_title("A", loc="left", fontweight="bold")

# Panel B
sns.boxplot(data=df, x="condition", y="apoptosis",
            palette=custom_palette, width=0.45, linewidth=1.2, ax=axes[1])
sns.stripplot(data=df, x="condition", y="apoptosis",
              color="black", alpha=0.35, size=3, jitter=True, ax=axes[1])
axes[1].set_ylabel("Apoptotic Cells (%)")
axes[1].set_xlabel("")
axes[1].set_title("B", loc="left", fontweight="bold")

sns.despine(trim=True)
plt.tight_layout()
plt.savefig("figure1.pdf", dpi=300, bbox_inches="tight")
plt.savefig("figure1.png", dpi=300, bbox_inches="tight")
print("Saved figure1.pdf and figure1.png at 300 DPI")

Recipe: Clustered Heatmap with Row and Column Color Annotations

When to use: Displaying a gene expression matrix with sample group annotations and hierarchical clustering to reveal co-expression modules.

import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import numpy as np
import pandas as pd

rng = np.random.default_rng(21)
n_genes, n_samples = 40, 20
conditions = ["tumor"] * 10 + ["normal"] * 10

# Simulate expression: 3 co-expression modules
expr = pd.DataFrame(
    rng.lognormal(2.5, 0.8, (n_genes, n_samples)),
    index=[f"GENE{i:03d}" for i in range(n_genes)],
    columns=[f"{c[0].upper()}{i:02d}" for i, c in enumerate(conditions)],
)
# Module 1: genes 0-13 up in tumor
expr.iloc[:14, :10]  *= 3.0
# Module 2: genes 14-27 down in tumor
expr.iloc[14:28, :10] *= 0.3
# Module 3: genes 28-39 unchanged

log_expr = np.log2(expr + 1)

# Column colors: tumor=red, normal=blue
cond_pal = {"tumor": "#C62828", "normal": "#1565C0"}
col_colors = [cond_pal[c] for c in conditions]

# Row colors: module membership
module_pal = {"up": "#EF9A9A", "down": "#90CAF9", "stable": "#C8E6C9"}
row_modules = (["up"] * 14) + (["down"] * 14) + (["stable"] * 12)
row_colors  = [module_pal[m] for m in row_modules]

g = sns.clustermap(
    log_expr,
    cmap="RdYlBu_r",
    center=log_expr.values.mean(),
    standard_scale=0,           # z-score per gene (row)
    method="ward",
    metric="euclidean",
    col_colors=col_colors,
    row_colors=row_colors,
    figsize=(14, 12),
    linewidths=0,
    cbar_pos=(0.02, 0.85, 0.03, 0.12),
    cbar_kws={"label": "Row z-score"},
    dendrogram_ratio=(0.12, 0.08),
)
g.ax_heatmap.set_xlabel("Sample", fontsize=10)
g.ax_heatmap.set_ylabel("Gene",   fontsize=10)
g.ax_heatmap.set_title("Gene Expression Clustermap", fontsize=12, pad=80)

# Manual legend for column/row annotations
legend_handles = [
    mpatches.Patch(color="#C62828", label="Tumor"),
    mpatches.Patch(color="#1565C0", label="Normal"),
    mpatches.Patch(color="#EF9A9A", label="Up in tumor"),
    mpatches.Patch(color="#90CAF9", label="Down in tumor"),
    mpatches.Patch(color="#C8E6C9", label="Stable"),
]
g.ax_heatmap.legend(handles=legend_handles, bbox_to_anchor=(1.25, 1.05),
                    loc="upper left", frameon=False, fontsize=9)

plt.savefig("clustermap_annotated.png", dpi=300, bbox_inches="tight")
print("Saved clustermap_annotated.png")

Troubleshooting

ProblemCauseSolution
Legend placed outside plot, clipped in saved fileFigure-level functions place the legend outside by defaultAdd bbox_inches="tight" to savefig(): g.savefig("out.png", dpi=300, bbox_inches="tight")
TypeError: FacetGrid.savefig() or blank figure savedCalled plt.savefig() on a figure-level grid that owns its own figureUse g.savefig(...) instead of plt.savefig(...)
Overlapping x-axis category labelsLong label strings overlap at default rotationAdd plt.xticks(rotation=45, ha="right") and plt.tight_layout() after the plot call
ValueError: Could not interpret value ... for parameter 'hue'Data is in wide-form; hue mapping requires long-formConvert with df.melt(id_vars=[...], var_name="sample", value_name="expr")
KDE bandwidth too smooth (loses bimodality)Default bw_adjust=1.0 over-smooths small datasetsLower to bw_adjust=0.5; confirm peaks with histplot
clustermap ignores figsizeFigure-level functions do not accept figsize as a kwarg in older seabornPass figsize as a direct argument: sns.clustermap(..., figsize=(12, 10))
Violin plot is a thin line (no shape)Too few observations for KDE estimationSwitch to kind="box" or kind="strip"; or use cut=0 to restrict KDE to data range
Colors not distinguishable for many groupsDefault palette repeats with >6 categoriesUse sns.color_palette("husl", n_colors=N) or "tab20" for up to 20 distinct colors
Figure-level function ignores ax= parameterAxes-level distinction: figure-level functions create their own figureUse the corresponding axes-level function (scatterplot, histplot, etc.) with ax=

Related Skills

  • matplotlib-scientific-plotting — low-level figure building, custom annotations, non-statistical plot types, and multi-panel layouts that mix seaborn with raw matplotlib
  • plotly-interactive-plots — interactive charts with hover, zoom, and HTML/Dash export
  • pydeseq2-differential-expression — produces the log2FC and p-values that feed into volcano-style scatter plots
  • scikit-image-processing — generates cell morphology measurements visualized with seaborn categorical/distribution plots
  • scientific-visualization — decision guide for selecting the right chart type and color scheme before coding

References

Statistical Significance Annotation on Plots

Overview

Statistical significance annotations (asterisk notation) are visual markers placed on comparison plots to indicate the results of hypothesis tests between groups. They consist of brackets connecting two groups and asterisk symbols denoting the p-value range. Proper annotation ensures that the visual claims in a figure match the quantitative evidence, making plots publication-ready and scientifically rigorous. This guide covers the standard conventions, when and how to annotate, and a reusable matplotlib implementation.

Key Concepts

Standard Asterisk Notation

The widely adopted convention maps p-value ranges to asterisk symbols:

SymbolP-value RangeMeaning
nsp > 0.05Not significant
*p <= 0.05Significant
**p <= 0.01Highly significant
***p <= 0.001Very highly significant
****p <= 0.0001Extremely significant

The conversion function:

def pvalue_to_asterisk(p: float) -> str:
    """Convert a p-value to standard asterisk notation."""
    if p <= 0.0001:
        return "****"
    elif p <= 0.001:
        return "***"
    elif p <= 0.01:
        return "**"
    elif p <= 0.05:
        return "*"
    else:
        return "ns"

Adjusted vs Raw P-values

  • Single comparison (one t-test): Use raw p-value.
  • Multiple comparisons (pairwise tests across 3+ groups, multiple genes): Use adjusted p-values (FDR/Benjamini-Hochberg or Bonferroni). Annotating with raw p-values inflates significance.
  • Pre-computed results (DESeq2 padj, ANOVA post-hoc): Use the adjusted values already provided.

Comparison Selection

Not every pair of groups needs annotation. Select comparisons that:

  • Directly support the claim made in the analysis text
  • Are biologically meaningful (e.g., treatment vs control, not control-A vs control-B)
  • Are limited in number to keep the figure readable (typically 1-5 per panel)

Decision Framework

Does the plot compare groups?
├── No (scatter, heatmap, PCA, line trend) → Do NOT annotate
└── Yes (box, violin, bar, strip)
    ├── Does the analysis claim significance? → Annotate the claimed comparisons
    ├── Exploratory (no specific claim) → Annotate vs control only, or skip
    └── Too many groups (>6 pairwise) → Annotate key comparisons only
ScenarioAnnotate?Which pairs
DEG box plot: treatment vs controlYesTreatment vs Control
Multi-group ANOVA with post-hocYesSignificant post-hoc pairs only
Gene expression across 10 cell typesSelectivelyvs reference cell type only
PCA or UMAPNoN/A
Heatmap or volcano plotNoN/A
Correlation scatterNoReport r and p in text/legend
Exploratory bar plot, no hypothesisOptionalvs control if applicable

Best Practices

  1. Match annotations to text claims: Every asterisk on the plot must correspond to a statistical test described in the analysis. Never annotate without having computed the test.
  2. Use adjusted p-values for multiple comparisons: When testing more than one pair, always use FDR-corrected or Bonferroni-corrected p-values. State the correction method in the figure legend.
  3. Limit annotated pairs: Annotate only comparisons relevant to the analysis conclusion. Over-annotating clutters the figure and dilutes focus.
  4. Position brackets clearly: Place brackets above the data range with enough vertical offset to avoid overlapping with data points, error bars, or other brackets. Stack multiple brackets with consistent spacing.
  5. State the statistical test: Always note the test used (t-test, Mann-Whitney U, Wilcoxon, ANOVA + Tukey HSD, etc.) in the figure title, caption, or legend.
  6. Include sample sizes: Show n per group in the axis labels (e.g., "Control (n=30)") or figure legend.
  7. Use bold titles: Set fontweight='bold' on figure titles for publication readiness.

Common Pitfalls

  1. Annotating all pairwise comparisons in a multi-group plot

    • How to avoid: Select only hypothesis-driven pairs. For k groups, k*(k-1)/2 pairs quickly becomes unreadable. Show vs control or specific contrasts only.
  2. Using raw p-values when multiple comparisons were performed

    • How to avoid: Apply statsmodels.stats.multitest.multipletests(pvals, method='fdr_bh') or use adjusted p-values from upstream tools (DESeq2 padj).
  3. Bracket overlap with data or other brackets

    • How to avoid: Use incremental vertical offset for stacked brackets. Start the first bracket above the maximum data value + error bar, then add a fixed offset for each additional bracket.
  4. Asterisks without stating which test was used

    • How to avoid: Always include the test name in the plot title or annotation (e.g., "Mann-Whitney U test" or "Tukey HSD post-hoc").
  5. Inconsistent notation across figures

    • How to avoid: Use the same pvalue_to_asterisk() function throughout the analysis. Define it once and reuse.
  6. Annotating "ns" on every non-significant pair

    • How to avoid: Only show "ns" when the non-significance itself is a notable finding (e.g., showing no difference between two treatments). Omit ns annotations for pairs not being compared.
  7. Placing annotations below the data

    • How to avoid: Always place brackets and asterisks above the compared groups, never below.

Workflow

Step 1: Compute Statistical Tests

Run the appropriate test and collect p-values before plotting:

from scipy import stats

# Two-group comparison
stat, pval = stats.mannwhitneyu(group_a, group_b, alternative='two-sided')
# or for normal data:
stat, pval = stats.ttest_ind(group_a, group_b)

# Multi-group: ANOVA + post-hoc
from scipy.stats import f_oneway
stat, pval_anova = f_oneway(group_a, group_b, group_c)

# Post-hoc pairwise (if ANOVA significant)
from itertools import combinations
from statsmodels.stats.multitest import multipletests

pairs = list(combinations(["A", "B", "C"], 2))
groups = {"A": group_a, "B": group_b, "C": group_c}
raw_pvals = []
for g1, g2 in pairs:
    _, p = stats.mannwhitneyu(groups[g1], groups[g2], alternative='two-sided')
    raw_pvals.append(p)

# Adjust for multiple comparisons
rejected, adj_pvals, _, _ = multipletests(raw_pvals, method='fdr_bh')

Step 2: Add Bracket Annotations to the Plot

Use this helper function to draw brackets with asterisks on any matplotlib axes:

def add_significance_bracket(ax, x1, x2, y, p_value, dh=0.02, barh=0.015, fontsize=11):
    """Draw a significance bracket with asterisk notation between two x positions.

    Args:
        ax: matplotlib Axes object.
        x1, x2: x-axis positions of the two groups (0-indexed).
        y: y-coordinate for the bracket (top of bracket line).
        p_value: p-value for the comparison.
        dh: vertical offset above bracket for the text (in axes fraction).
        barh: height of the bracket tips (in axes fraction).
        fontsize: font size for the asterisk text.
    """
    asterisk = pvalue_to_asterisk(p_value)

    # Draw bracket: two tips and a connecting line
    ax.plot([x1, x1, x2, x2], [y - barh, y, y, y - barh],
            lw=1.2, color='black')
    # Place asterisk text centered above the bracket
    ax.text((x1 + x2) / 2, y + dh, asterisk,
            ha='center', va='bottom', fontsize=fontsize, fontweight='bold')

Step 3: Integrate with Seaborn Plots

import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# Example: box plot with significance annotation
fig, ax = plt.subplots(figsize=(6, 5))
sns.boxplot(data=df, x="group", y="value", ax=ax, palette="Set2")
sns.stripplot(data=df, x="group", y="value", ax=ax,
              color="black", alpha=0.4, size=3, jitter=True)

# Determine bracket y-position from data
y_max = df["value"].max()
y_range = df["value"].max() - df["value"].min()
offset = y_range * 0.08  # spacing between brackets

# Add brackets for each significant comparison
# pairs_with_pvals: list of (group1_idx, group2_idx, p_value)
pairs_with_pvals = [(0, 1, 0.003), (0, 2, 0.042)]

for i, (x1, x2, pval) in enumerate(pairs_with_pvals):
    bracket_y = y_max + offset * (i + 1)
    add_significance_bracket(ax, x1, x2, bracket_y, pval)

ax.set_title("Gene Expression by Treatment", fontweight='bold', fontsize=14)
ax.set_ylabel("Expression (log2 CPM)")
# Extend y-axis to fit brackets
ax.set_ylim(top=y_max + offset * (len(pairs_with_pvals) + 1.5))

plt.tight_layout()
plt.savefig("expression_comparison.png", dpi=150, bbox_inches='tight')

Step 4: Annotating Grouped Bar Plots

For bar plots with error bars, position brackets above the error bars:

fig, ax = plt.subplots(figsize=(7, 5))
bar_plot = sns.barplot(data=df, x="gene", y="fold_change", hue="condition",
                       ax=ax, palette="Set2", ci="sd", capsize=0.05)

# For grouped bars, calculate x positions manually
# Each gene has multiple bars offset by group
n_groups = df["condition"].nunique()
n_genes = df["gene"].nunique()
bar_width = 0.8 / n_groups

for gene_idx in range(n_genes):
    # x positions of the two bars within this gene group
    x1 = gene_idx - bar_width / 2
    x2 = gene_idx + bar_width / 2
    # Get the max value + error for this gene
    gene_data = df[df["gene"] == df["gene"].unique()[gene_idx]]
    y_top = gene_data["fold_change"].mean() + gene_data["fold_change"].std()
    p_val = pvals_per_gene[gene_idx]  # pre-computed

    if p_val <= 0.05:  # only annotate significant results
        add_significance_bracket(ax, x1, x2, y_top + 0.1, p_val)

ax.set_title("Fold Change by Condition", fontweight='bold', fontsize=14)
plt.tight_layout()
plt.savefig("fold_change_comparison.png", dpi=150, bbox_inches='tight')

Protocol Guidelines

  1. Always compute tests before plotting: The statistical test should be run and results stored before any plotting code. Do not compute p-values inside the plotting block.
  2. Use consistent style: Use the same add_significance_bracket function and pvalue_to_asterisk conversion across all figures in an analysis.
  3. Report test details in solution text: When presenting the figure, state: the test used, number of samples per group, and whether p-values are adjusted.
  4. Adjust y-axis limits: After adding brackets, extend the y-axis upper limit to prevent clipping. Use ax.set_ylim(top=...) or ax.margins(y=0.15).
  5. For DESeq2/edgeR results: Use padj (adjusted p-value) directly. Do not re-test the raw counts.

Further Reading

Related Skills

  • seaborn-statistical-plots — Seaborn plotting fundamentals; use this guide's annotation workflow on top of seaborn figures
  • scientific-visualization — General scientific figure design principles

BWA-MEM2 — DNA Short-Read Aligner

Overview

BWA-MEM2 aligns short DNA reads (Illumina, 50–250 bp) to a reference genome using the BWT-FM index. It is the standard aligner for whole-genome sequencing (WGS), whole-exome sequencing (WES), ChIP-seq, and ATAC-seq DNA alignment. BWA-MEM2 is 2× faster than the original BWA-MEM while producing identical results. It outputs SAM format with proper read group (@RG) headers required by GATK HaplotypeCaller and Picard tools. For paired-end reads, it marks proper pairs and resolves chimeric/split reads into supplementary alignments.

When to Use

  • Aligning WGS or WES Illumina reads to a reference genome for variant calling (SNP, indel, SV)
  • ChIP-seq or ATAC-seq DNA alignment to produce BAM files for peak calling with MACS3
  • Producing GATK-compatible BAM files with @RG read group tags
  • Aligning reads ≥ 50 bp; for shorter reads (< 50 bp), BWA-backtrack may be more appropriate
  • Re-aligning legacy FASTQ files to an updated reference genome assembly
  • Use STAR instead for RNA-seq reads that span splice junctions
  • Use Bowtie2 as an alternative for local alignment or when index size must be minimized

Prerequisites

  • Software: bwa-mem2 (conda or pre-compiled binary), samtools
  • Reference: genome FASTA (e.g., GRCh38, hg19, mm10)
  • RAM: ~28 GB for human genome index; 6–8 GB for mouse

Check before installing: The tool may already be available in the current environment (e.g., inside a pixi / conda env). Run command -v bwa-mem2 first and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool via pixi run bwa-mem2 rather than bare bwa-mem2.

# Install with conda (recommended)
conda install -c bioconda bwa-mem2 samtools

# Or download pre-compiled binary
wget https://github.com/bwa-mem2/bwa-mem2/releases/download/v2.2.1/bwa-mem2-2.2.1_x64-linux.tar.bz2
tar -jxf bwa-mem2-2.2.1_x64-linux.tar.bz2
export PATH="$PWD/bwa-mem2-2.2.1_x64-linux:$PATH"

# Verify
bwa-mem2 version
# 2.2.1

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: reference_bundle
    kind: required
    source: user
    ask: "Which species and genome assembly should reads be aligned against?"
    default: null

  - id: D2
    param: read_group
    kind: required
    source: user
    ask: "What sample name and library identity should be stamped into each alignment?"
    default: null

  - id: D3
    param: downstream_compatibility
    kind: derived
    source: upstream
    ask: "Will GATK or Picard consume this BAM, requiring supplementary alignments flagged and soft-clipped?"
    default: "-M -Y when a GATK/Picard step follows"

  - id: D4
    param: seedLength
    kind: optional
    source: user
    ask: "Do these reads need a shorter seed than the default to map sensitively?"
    default: 19

  - id: D5
    param: minAlignmentScore
    kind: optional
    source: user
    ask: "How weak an alignment is still worth reporting?"
    default: 30

  - id: D6
    param: interleavedInput
    kind: derived
    source: data
    ask: "Are both mates held in a single interleaved FASTQ?"
    default: false
    skip_if: "reads supplied as separate R1/R2 files"

  - id: D7
    param: threads
    kind: never_ask
    source: data
    reason: "Affects runtime only, not the alignments"
    default: "min(8, available_cores)"

D2 is not cosmetic. The sample name in the read group is what joint genotyping groups by; an omitted or duplicated one silently merges or splits samples downstream, long after the alignment itself looks fine.

Quick Start

# 1. Build genome index (~30 min, run once)
bwa-mem2 index GRCh38.fa

# 2. Align paired-end reads and sort
bwa-mem2 mem -t 16 -R "@RG\tID:sample1\tSM:sample1\tPL:ILLUMINA" \
    GRCh38.fa sample1_R1.fastq.gz sample1_R2.fastq.gz \
    | samtools sort -@ 8 -o sample1.sorted.bam

# 3. Index the BAM
samtools index sample1.sorted.bam
echo "Aligned reads: $(samtools view -c -F 4 sample1.sorted.bam)"

Workflow

Step 1: Download Reference Genome

Obtain the reference genome FASTA file matching the target assembly.

# Download GRCh38 primary assembly (human)
wget https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/405/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna.gz
gunzip GCA_000001405.15_GRCh38_no_alt_analysis_set.fna.gz
mv GCA_000001405.15_GRCh38_no_alt_analysis_set.fna GRCh38.fa

# Or use ENSEMBL/GENCODE
wget https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_47/GRCh38.primary_assembly.genome.fa.gz
gunzip GRCh38.primary_assembly.genome.fa.gz

echo "Reference size: $(du -sh GRCh38.fa)"

Step 2: Build BWA-MEM2 Index

Index the reference genome — required once per genome, takes ~25-35 min for human.

# Build index (~28 GB RAM required for human genome)
bwa-mem2 index GRCh38.fa

# This creates: GRCh38.fa.0123, GRCh38.fa.amb, GRCh38.fa.ann,
#               GRCh38.fa.bwt.2bit.64, GRCh38.fa.pac
echo "Index files: $(ls GRCh38.fa.* | wc -l) created"
ls -lh GRCh38.fa.*

Step 3: Align Paired-End Reads

Align FASTQ reads with a read group header required for GATK compatibility.

# Align with read group (required for GATK)
# @RG fields: ID (run ID), SM (sample name), PL (platform), LB (library), PU (flowcell)
bwa-mem2 mem \
    -t 16 \
    -R "@RG\tID:sample1_run1\tSM:sample1\tPL:ILLUMINA\tLB:lib1\tPU:flowcell1" \
    GRCh38.fa \
    sample1_R1.fastq.gz \
    sample1_R2.fastq.gz \
    | samtools sort -@ 8 -m 2G -o sample1.sorted.bam

samtools index sample1.sorted.bam
echo "Alignment complete."
echo "Total reads:   $(samtools view -c sample1.sorted.bam)"
echo "Mapped reads:  $(samtools view -c -F 4 sample1.sorted.bam)"

Step 4: Mark PCR Duplicates

Remove or mark optical and PCR duplicates before variant calling.

# Option A: samtools markdup (fast)
samtools fixmate -m sample1.sorted.bam sample1.fixmate.bam
samtools sort -@ 8 -o sample1.fixmate.sorted.bam sample1.fixmate.bam
samtools markdup -@ 8 sample1.fixmate.sorted.bam sample1.markdup.bam
samtools index sample1.markdup.bam

echo "Duplication rate:"
samtools flagstat sample1.markdup.bam | grep "duplicate"

# Option B: Picard MarkDuplicates (GATK best practices)
picard MarkDuplicates \
    INPUT=sample1.sorted.bam \
    OUTPUT=sample1.markdup.bam \
    METRICS_FILE=sample1.dupmetrics.txt \
    REMOVE_DUPLICATES=false \
    CREATE_INDEX=true
cat sample1.dupmetrics.txt | grep -A2 "ESTIMATED"

Step 5: Assess Alignment Quality

Generate alignment statistics and check key quality metrics.

# Full alignment statistics
samtools flagstat sample1.markdup.bam > sample1.flagstat.txt
cat sample1.flagstat.txt

# Coverage statistics
samtools coverage sample1.markdup.bam | head -30

# Parse key metrics with Python
python3 - << 'EOF'
from pathlib import Path

flagstat = Path("sample1.flagstat.txt").read_text()
for line in flagstat.splitlines():
    if any(kw in line for kw in ["total", "mapped", "properly paired", "duplicate"]):
        print(line)
EOF

Step 6: Complete WGS/WES Pipeline → Variant Calling

Pipe BWA-MEM2 output directly into GATK HaplotypeCaller.

#!/bin/bash
# Complete WGS alignment → variant calling pipeline
GENOME="GRCh38.fa"
SAMPLE="sample1"
R1="data/${SAMPLE}_R1.fastq.gz"
R2="data/${SAMPLE}_R2.fastq.gz"
THREADS=16
OUTDIR="results/${SAMPLE}"
mkdir -p "$OUTDIR"

# Step 1: Align + sort
bwa-mem2 mem -t $THREADS \
    -R "@RG\tID:${SAMPLE}\tSM:${SAMPLE}\tPL:ILLUMINA\tLB:lib1\tPU:run1" \
    $GENOME $R1 $R2 \
    | samtools sort -@ 8 -o $OUTDIR/${SAMPLE}.sorted.bam
samtools index $OUTDIR/${SAMPLE}.sorted.bam

# Step 2: Mark duplicates
samtools fixmate -m $OUTDIR/${SAMPLE}.sorted.bam - \
    | samtools sort -@ 8 \
    | samtools markdup -@ 8 - $OUTDIR/${SAMPLE}.markdup.bam
samtools index $OUTDIR/${SAMPLE}.markdup.bam

# Step 3: GATK variant calling
gatk HaplotypeCaller \
    -R $GENOME \
    -I $OUTDIR/${SAMPLE}.markdup.bam \
    -O $OUTDIR/${SAMPLE}.g.vcf.gz \
    -ERC GVCF \
    --native-pair-hmm-threads 4

echo "Pipeline complete: $OUTDIR/${SAMPLE}.g.vcf.gz"

Key Parameters

ParameterDefaultRange/OptionsEffect
-t11–64CPU threads; use 8–16 for production runs
-R—@RG\tID:...\tSM:...Read group string; required for GATK compatibility
-k1910–28Minimum seed length; lower = more sensitive for shorter reads
-w10050–500Band width for Smith-Waterman alignment
-MoffflagMark split/supplementary reads as secondary (BWA-MEM style); needed for Picard compatibility
-aoffflagOutput all alignments for single-end reads (for seeding; increases file size)
-poffflagTreat input as interleaved paired-end FASTQ
-c500100–10000Skip alignment for MEM count > threshold (reduces multi-mapper noise)
-T3020–60Minimum alignment score threshold; lower = report more low-quality alignments
-YoffflagUse soft clipping for supplementary alignments (recommended for GATK)

Common Recipes

Recipe 1: Batch Align Multiple Samples

#!/bin/bash
# Align all samples in parallel using GNU parallel or sequential loop
GENOME="GRCh38.fa"
SAMPLES=(ctrl_1 ctrl_2 treat_1 treat_2)
THREADS=12

for sample in "${SAMPLES[@]}"; do
    echo "=== Aligning $sample ==="
    bwa-mem2 mem -t $THREADS \
        -R "@RG\tID:${sample}\tSM:${sample}\tPL:ILLUMINA\tLB:lib1\tPU:run1" \
        $GENOME \
        data/${sample}_R1.fastq.gz \
        data/${sample}_R2.fastq.gz \
        | samtools sort -@ 4 -m 2G -o results/${sample}.sorted.bam
    samtools index results/${sample}.sorted.bam
    
    MAPPED=$(samtools view -c -F 4 results/${sample}.sorted.bam)
    TOTAL=$(samtools view -c results/${sample}.sorted.bam)
    echo "$sample: $MAPPED / $TOTAL reads mapped"
done

Recipe 2: Parse Alignment Metrics with Python

import subprocess
import pandas as pd
from pathlib import Path

samples = ["ctrl_1", "ctrl_2", "treat_1", "treat_2"]
metrics = []

for sample in samples:
    bam = f"results/{sample}.sorted.bam"
    result = subprocess.run(
        ["samtools", "flagstat", bam], capture_output=True, text=True
    )
    stats = {}
    for line in result.stdout.splitlines():
        if "total" in line:
            stats["total"] = int(line.split()[0])
        elif "mapped" in line and "%" in line:
            stats["mapped"] = int(line.split()[0])
            stats["pct_mapped"] = float(line.split("(")[1].split("%")[0])
        elif "properly paired" in line:
            stats["properly_paired"] = int(line.split()[0])
    stats["sample"] = sample
    metrics.append(stats)

df = pd.DataFrame(metrics).set_index("sample")
print(df[["total", "mapped", "pct_mapped", "properly_paired"]])
df.to_csv("alignment_metrics.tsv", sep="\t")

Expected Outputs

OutputFormatDescription
*.sorted.bamBAMCoordinate-sorted aligned reads; index with samtools index
*.sorted.bam.baiBAIBAM index; required for random access by GATK and IGV
*.flagstat.txtTextAlignment summary: total/mapped/paired/duplicate counts and percentages
*.markdup.bamBAMDuplicate-marked BAM; use as input to GATK HaplotypeCaller
*.dupmetrics.txtTextPicard duplication metrics with estimated library size

Troubleshooting

ProblemCauseSolution
Low mapping rate (< 85%)Genome mismatch, contamination, or low quality readsVerify genome assembly matches sample; run FastQC; trim adapters with Trim Galore
@RG header missing error in GATK-R flag not specified during alignmentRe-align with -R "@RG\tID:...\tSM:...\tPL:ILLUMINA"
Out of memory during indexingInsufficient RAM for genome indexBWA-MEM2 requires ~28 GB for human; use classic bwa with less RAM if needed
Unbalanced paired-end countsInterleaved FASTQ or file mismatchAdd -p for interleaved; verify R1/R2 read counts with zcat r1.fq.gz | wc -l
[E::bwa_idx_load_from_disk] errorIndex files missing or wrong prefixRe-run bwa-mem2 index genome.fa; ensure all .0123, .bwt.2bit.64 files exist
Slow alignment speedLow thread count or slow I/OUse -t 16 or more; store data on SSD; pipe directly to samtools sort
Supplementary alignments causing issuesSplit reads in downstream toolsAdd -M flag to mark split reads as secondary (Picard compatibility)
GATK base quality score recalibration failsMissing known variant VCFDownload dbSNP VCF for your genome assembly from NCBI or GATK resource bundle

References

Pysam — Genomic File Toolkit

Overview

Pysam provides a Pythonic interface to htslib for reading, manipulating, and writing genomic data files. It handles SAM/BAM/CRAM alignments, VCF/BCF variants, and FASTA/FASTQ sequences with efficient region-based random access. Also exposes samtools and bcftools as callable Python functions.

When to Use

  • Reading and querying BAM/CRAM alignment files (region extraction, read filtering)
  • Analyzing VCF/BCF variant files (genotype access, variant filtering, annotation)
  • Extracting reference sequences from indexed FASTA files
  • Calculating per-base coverage and pileup statistics
  • Building custom bioinformatics pipelines that combine alignment + variant + sequence data
  • Quality control of NGS data (mapping quality, flag filtering, coverage)
  • For alignment from FASTQ (read mapping), use STAR, BWA, or minimap2 instead
  • For variant calling from BAM, use GATK or DeepVariant instead

Prerequisites

pip install pysam

Note: Requires htslib C library (bundled with pip install on most platforms). On some Linux systems, may need libhts-dev or equivalent. Index files (.bai, .tbi, .fai) required for random access — create with pysam.index(), pysam.tabix_index(), or pysam.faidx().

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: mappingQualityFloor
    kind: required
    source: user
    ask: "Below what mapping confidence should reads be ignored when piling up or counting?"
    default: "0 - no filter"

  - id: D2
    param: baseQualityFloor
    kind: required
    source: user
    ask: "How confident must an individual base call be to be counted at a position?"
    default: 15

  - id: D3
    param: regionTruncation
    kind: required
    source: user
    ask: "Should a pileup report only positions inside the requested region, or every position touched by an overlapping read?"
    default: "not truncated - overlapping reads extend the reported range"

  - id: D4
    param: indexRequirement
    kind: derived
    source: data
    ask: "Is the file indexed, or must records be streamed sequentially?"
    default: "indexed random access"

  - id: D5
    param: iteratorSafety
    kind: never_ask
    source: user
    reason: "Whether simultaneous iterators are allowed is an API-usage concern with a small overhead cost, not a result-changing choice"
    default: false

D3 is the one that surprises people: an untruncated pileup returns positions outside the region that was asked for, so per-region statistics computed straight off the result cover a wider span than intended.

Quick Start

import pysam

# Read BAM file, fetch reads in a region
with pysam.AlignmentFile("sample.bam", "rb") as bam:
    for read in bam.fetch("chr1", 1000, 2000):
        print(f"{read.query_name}: pos={read.reference_start}, mapq={read.mapping_quality}")
    print(f"Total reads in region: {bam.count('chr1', 1000, 2000)}")

Core API

1. Alignment Files (SAM/BAM/CRAM)

Read, query, and write aligned sequencing reads.

import pysam

# Open BAM (binary) or SAM (text) file
bam = pysam.AlignmentFile("sample.bam", "rb")  # rb=read BAM, r=read SAM, rc=read CRAM

# Fetch reads overlapping a region (requires .bai index)
for read in bam.fetch("chr1", 10000, 20000):
    print(f"Name: {read.query_name}")
    print(f"  Position: {read.reference_start}-{read.reference_end}")
    print(f"  MAPQ: {read.mapping_quality}")
    print(f"  CIGAR: {read.cigarstring}")
    print(f"  Sequence: {read.query_sequence[:30]}...")
    break

# Count reads in region (fast, no iteration needed)
n_reads = bam.count("chr1", 10000, 20000)
print(f"Reads in region: {n_reads}")

# Filter reads by quality and flags
for read in bam.fetch("chr1", 10000, 20000):
    if read.mapping_quality >= 30 and not read.is_unmapped and not read.is_duplicate:
        pass  # Process high-quality, mapped, non-duplicate reads

bam.close()
# Write filtered reads to a new BAM file
with pysam.AlignmentFile("input.bam", "rb") as inbam:
    with pysam.AlignmentFile("filtered.bam", "wb", header=inbam.header) as outbam:
        for read in inbam.fetch("chr1", 10000, 20000):
            if read.mapping_quality >= 30:
                outbam.write(read)

# Index the output
pysam.index("filtered.bam")
print("Created filtered.bam + filtered.bam.bai")

2. Coverage and Pileup Analysis

Calculate per-base coverage statistics.

import pysam
import numpy as np

bam = pysam.AlignmentFile("sample.bam", "rb")

# Pileup: per-base coverage with read-level detail
for pileup_col in bam.pileup("chr1", 10000, 10100, min_mapping_quality=30):
    bases = [p.alignment.query_sequence[p.query_position]
             for p in pileup_col.pileups if not p.is_del and p.query_position is not None]
    print(f"Pos {pileup_col.reference_pos}: depth={pileup_col.nsegments}, bases={''.join(bases[:5])}")

# Quick coverage count per region (faster than pileup)
coverage = bam.count_coverage("chr1", 10000, 10100, quality_threshold=20)
# Returns tuple of 4 arrays (A, C, G, T counts per position)
total_cov = np.array(coverage).sum(axis=0)
print(f"Mean coverage: {total_cov.mean():.1f}x")

bam.close()

3. Variant Files (VCF/BCF)

Read, query, and filter genetic variants.

import pysam

# Open VCF/BCF file
vcf = pysam.VariantFile("variants.vcf.gz")

# Iterate all variants
for record in vcf.fetch("chr1", 10000, 50000):
    print(f"{record.chrom}:{record.pos} {record.ref}>{','.join(record.alts or [])}")
    print(f"  QUAL={record.qual}, FILTER={list(record.filter)}")
    print(f"  INFO: {dict(record.info)}")

    # Access genotypes per sample
    for sample in record.samples:
        gt = record.samples[sample]["GT"]
        print(f"  {sample}: GT={gt}")
    break

vcf.close()
# Filter variants and write to new VCF
with pysam.VariantFile("variants.vcf.gz") as vcf_in:
    with pysam.VariantFile("filtered.vcf.gz", "wz", header=vcf_in.header) as vcf_out:
        for record in vcf_in:
            if record.qual and record.qual >= 30 and "PASS" in record.filter:
                vcf_out.write(record)

pysam.tabix_index("filtered.vcf.gz", preset="vcf")
print("Created filtered.vcf.gz + filtered.vcf.gz.tbi")

4. Sequence Files (FASTA/FASTQ)

Random access to reference sequences and sequential reading of raw reads.

import pysam

# FASTA: random access (requires .fai index)
fasta = pysam.FastaFile("reference.fasta")
seq = fasta.fetch("chr1", 10000, 10050)
print(f"Sequence ({len(seq)} bp): {seq}")
print(f"Available contigs: {fasta.references[:5]}")
print(f"Contig lengths: {dict(zip(fasta.references[:3], fasta.lengths[:3]))}")
fasta.close()

# Create FASTA index if needed
# pysam.faidx("reference.fasta")
# FASTQ: sequential reading
with pysam.FastxFile("reads.fastq.gz") as fq:
    for i, entry in enumerate(fq):
        print(f"Read {entry.name}: {len(entry.sequence)} bp, mean_qual={sum(entry.get_quality_array())/len(entry.sequence):.1f}")
        if i >= 2:
            break

5. Read Groups and Sample Information

Extract and filter reads by read group (essential for multi-sample BAM files).

import pysam

bam = pysam.AlignmentFile("multisample.bam", "rb")

# Access read group information from BAM header
print("Read groups in file:")
for rg_dict in bam.header.get("RG", []):
    print(f"  ID: {rg_dict['ID']}, Sample: {rg_dict.get('SM', 'N/A')}, Library: {rg_dict.get('LB', 'N/A')}, Platform: {rg_dict.get('PL', 'N/A')}")

# Get all samples in the BAM (from RG headers)
samples = set()
for rg_dict in bam.header.get("RG", []):
    if "SM" in rg_dict:
        samples.add(rg_dict["SM"])
print(f"Samples in BAM: {sorted(samples)}")

bam.close()
# Filter reads by read group ID
def extract_reads_by_rg(bam_path, rg_id, output_path):
    """Extract all reads from a specific read group.

    WARNING: Uses fetch(until_eof=True), which scans the entire BAM sequentially.
    Multi-sample BAMs can be tens to hundreds of GB — this may be slow.
    For large files, prefer region-based filtering:
        for read in bam.fetch("chr1", start, end): ...
    Or use the samtools CLI equivalent (faster for one-off extractions):
        samtools view -b -r <rg_id> input.bam -o output.bam
    """
    with pysam.AlignmentFile(bam_path, "rb") as bam_in:
        with pysam.AlignmentFile(output_path, "wb", header=bam_in.header) as bam_out:
            for read in bam_in.fetch(until_eof=True):
                if read.has_tag("RG") and read.get_tag("RG") == rg_id:
                    bam_out.write(read)
    pysam.index(output_path)
    print(f"Extracted reads from RG:{rg_id} → {output_path}")

extract_reads_by_rg("multisample.bam", "SAMPLE_001_LaneA", "sample001_laneA.bam")
from collections import defaultdict
import pysam

# Count reads per sample
def reads_per_sample(bam_path):
    """Count reads per sample from read group information.

    Two distinct "unknown" cases are tracked separately:
    - "no_sm_field":  RG header entry exists but is missing the SM (sample name) field.
    - "undefined_rg": A read carries an RG tag not declared in the BAM header.
    """
    counts = defaultdict(int)
    rg_to_sample = {}

    with pysam.AlignmentFile(bam_path, "rb") as bam:
        # Build RG → sample mapping from header
        for rg_dict in bam.header.get("RG", []):
            rg_id = rg_dict["ID"]
            # (a) RG header entry lacks SM field
            rg_to_sample[rg_id] = rg_dict.get("SM", "no_sm_field")

        # Count reads per resolved sample name
        for read in bam.fetch(until_eof=True):
            if read.has_tag("RG"):
                rg_id = read.get_tag("RG")
                # (b) Read's RG tag is not declared in the header
                sample = rg_to_sample.get(rg_id, "undefined_rg")
                counts[sample] += 1

    return dict(counts)

sample_counts = reads_per_sample("multisample.bam")
for sample, count in sorted(sample_counts.items()):
    print(f"  {sample}: {count:,} reads")

6. Samtools/Bcftools CLI Access

Call samtools and bcftools commands from Python.

import pysam

# Sort BAM file
pysam.sort("-o", "sorted.bam", "input.bam")

# Index BAM
pysam.index("sorted.bam")

# View region as BAM
pysam.view("-b", "-o", "region.bam", "sorted.bam", "chr1:1000-2000")

# BCFtools: compress and index VCF
pysam.bcftools.view("-O", "z", "-o", "output.vcf.gz", "input.vcf")
pysam.tabix_index("output.vcf.gz", preset="vcf")

# Error handling
try:
    pysam.sort("-o", "output.bam", "nonexistent.bam")
except pysam.SamtoolsError as e:
    print(f"samtools error: {e}")

CLI equivalents (for reference — use Python API in automated pipelines):

# These are equivalent to the Python calls above:
samtools sort -o sorted.bam input.bam
samtools index sorted.bam
samtools view -b -o region.bam sorted.bam chr1:1000-2000
bcftools view -O z -o output.vcf.gz input.vcf

Key Concepts

Coordinate Systems

Critical: pysam uses 0-based, half-open coordinates (Python convention):

SystemStartEndExample: "bases 1000-2000"
pysam Python API0-basedexclusivefetch("chr1", 999, 2000)
samtools region string1-basedinclusivefetch("chr1:1000-2000")
VCF file format1-based—record.pos = 1-based, record.start = 0-based
BED format0-basedexclusivechr1\t999\t2000

Index File Requirements

File TypeIndex ExtensionCreate With
BAM.baipysam.index("file.bam")
CRAM.craipysam.index("file.cram")
FASTA.faipysam.faidx("file.fasta")
VCF.gz.tbipysam.tabix_index("file.vcf.gz", preset="vcf")
BCF.csipysam.tabix_index("file.bcf", preset="bcf")

Without an index, use fetch(until_eof=True) for sequential reading.

File Mode Strings

ModeFormatDirection
"rb"BAM (binary)Read
"r"SAM (text)Read
"rc"CRAMRead
"wb"BAMWrite
"w"SAMWrite
"wz"VCF.gz (compressed)Write

Common Workflows

Workflow 1: Coverage Analysis for Target Regions

Goal: Calculate coverage statistics for a set of target regions (e.g., exome capture targets).

import pysam
import numpy as np

def coverage_for_regions(bam_path, regions, min_mapq=30):
    """Calculate coverage stats for a list of (chrom, start, end) regions."""
    results = []
    with pysam.AlignmentFile(bam_path, "rb") as bam:
        for chrom, start, end in regions:
            cov = np.array(bam.count_coverage(chrom, start, end,
                                               quality_threshold=min_mapq))
            total = cov.sum(axis=0)
            results.append({
                "region": f"{chrom}:{start}-{end}",
                "mean_cov": total.mean(),
                "min_cov": total.min(),
                "pct_above_20x": (total >= 20).mean() * 100,
            })
    return results

regions = [("chr1", 10000, 10500), ("chr1", 20000, 20500), ("chr2", 5000, 5500)]
stats = coverage_for_regions("sample.bam", regions)
for s in stats:
    print(f"{s['region']}: mean={s['mean_cov']:.1f}x, min={s['min_cov']}x, ≥20x={s['pct_above_20x']:.1f}%")

Workflow 2: Variant Annotation with Read Support

Goal: For each variant in a VCF, count supporting reads from the BAM.

import pysam

def annotate_variants_with_reads(vcf_path, bam_path, output_path):
    """Add read support counts to each variant."""
    with pysam.VariantFile(vcf_path) as vcf_in:
        # Add INFO field to header
        vcf_in.header.add_line(
            '##INFO=<ID=READ_SUPPORT,Number=1,Type=Integer,Description="Reads supporting alt allele">'
        )
        with pysam.VariantFile(output_path, "w", header=vcf_in.header) as vcf_out:
            with pysam.AlignmentFile(bam_path, "rb") as bam:
                for record in vcf_in:
                    alt_count = 0
                    for col in bam.pileup(record.chrom, record.start, record.stop,
                                           min_mapping_quality=30, truncate=True):
                        if col.reference_pos == record.start:
                            for p in col.pileups:
                                if (not p.is_del and p.query_position is not None and
                                    p.alignment.query_sequence[p.query_position] in (record.alts or [])):
                                    alt_count += 1
                    record.info["READ_SUPPORT"] = alt_count
                    vcf_out.write(record)

annotate_variants_with_reads("variants.vcf", "sample.bam", "annotated.vcf")
print("Created annotated.vcf with READ_SUPPORT field")

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
mode stringAlignmentFile, VariantFile—"rb", "r", "rc", "wb", "w", "wz"File format and read/write direction
min_mapping_qualitypileup()00–60Filter reads below this MAPQ
quality_thresholdcount_coverage()150–40Minimum base quality to count
truncatepileup()FalseTrue/FalseTruncate pileup to exact region (True) vs include overlapping reads (False)
until_eoffetch()FalseTrue/FalseRead all records sequentially without index
multiple_iteratorsfetch()FalseTrue/FalseAllow multiple simultaneous iterators (slight overhead)
presettabix_index()—"vcf", "bed", "gff", "sam"File format for tabix indexing

Best Practices

  1. Always use context managers (with statement) for automatic file cleanup. Unclosed files can leak file descriptors.

  2. Create and verify index files first: Most random-access operations fail silently or raise cryptic errors without indexes. Check for .bai/.tbi/.fai files before queries.

  3. Use count() instead of iterating to count reads: bam.count("chr1", 1000, 2000) is much faster than sum(1 for _ in bam.fetch(...)).

  4. Use count_coverage() for coverage, pileup() for base-level detail: count_coverage() is faster when you only need depth numbers. Use pileup() only when you need per-read, per-base information.

  5. Anti-pattern — mixing 0-based and 1-based coordinates: Always double-check coordinate systems when combining pysam (0-based) with VCF files (1-based POS), BED files (0-based), or region strings (1-based). See Key Concepts table.

  6. Anti-pattern — forgetting truncate=True in pileup: Without truncate=True, pileup() extends to the full extent of overlapping reads, which can be much larger than the requested region.

Common Recipes

Recipe: Extract Gene Sequences from Reference

import pysam

def get_gene_sequence(fasta_path, chrom, start, end, strand="+"):
    """Extract gene sequence, reverse-complement if on minus strand."""
    with pysam.FastaFile(fasta_path) as fasta:
        seq = fasta.fetch(chrom, start, end)
        if strand == "-":
            complement = str.maketrans("ACGTacgt", "TGCAtgca")
            seq = seq.translate(complement)[::-1]
        return seq

seq = get_gene_sequence("reference.fasta", "chr1", 10000, 11000, strand="-")
print(f"Gene sequence ({len(seq)} bp): {seq[:50]}...")

Recipe: BAM Statistics Summary

import pysam

def bam_summary(bam_path):
    """Quick summary statistics for a BAM file."""
    with pysam.AlignmentFile(bam_path, "rb") as bam:
        stats = {"total": 0, "mapped": 0, "unmapped": 0, "duplicates": 0, "mapq_ge30": 0}
        for read in bam.fetch(until_eof=True):
            stats["total"] += 1
            if read.is_unmapped:
                stats["unmapped"] += 1
            else:
                stats["mapped"] += 1
                if read.is_duplicate:
                    stats["duplicates"] += 1
                if read.mapping_quality >= 30:
                    stats["mapq_ge30"] += 1
        return stats

summary = bam_summary("sample.bam")
for k, v in summary.items():
    print(f"  {k}: {v:,}")

Troubleshooting

ProblemCauseSolution
ValueError: could not open alignment fileMissing file or wrong mode stringCheck file path; use "rb" for BAM, "r" for SAM
ValueError: fetch called on bamfile without indexNo .bai index fileRun pysam.index("file.bam") first
Region returns unexpected readsReads overlapping boundaries are includedUse truncate=True in pileup() or filter by read.reference_start >= start
Coordinate off-by-one errorsMixing 0-based (pysam) with 1-based (VCF, samtools)See Key Concepts coordinate table; record.pos is 1-based, record.start is 0-based
PileupProxy accessed after iterator finishedPileup iterator went out of scopeStore needed data from pileup columns immediately, don't save PileupProxy references
SamtoolsError from CLI callsInvalid arguments or missing inputWrap in try/except pysam.SamtoolsError; check samtools docs for argument syntax
Very slow iterationIterating all reads without region queryUse fetch("chr1", start, end) for targeted queries; use indexed files
Read group filter returns 0 readsRG tag missing or wrong ID specifiedVerify RG tag exists: read.has_tag("RG"); list available RGs from bam.header.get("RG", [])

Related Skills

  • biopython-molecular-biology — sequence I/O and alignment; complementary for non-BAM sequence formats
  • pydeseq2-differential-expression — downstream analysis of read counts from BAM coverage data
  • scanpy-scrna-seq — single-cell analysis; pysam handles the upstream BAM processing

References

samtools — SAM/BAM/CRAM Alignment Toolkit

Overview

samtools is the standard command-line toolkit for processing sequence alignment files in SAM, BAM, and CRAM formats. It handles the complete alignment file lifecycle: format conversion, coordinate sorting, index creation, quality control statistics, read filtering, duplicate marking, and multi-file merging. samtools is a near-universal component of NGS pipelines between alignment (STAR, BWA) and downstream analysis (variant calling, peak calling, coverage).

When to Use

  • Sorting BAM files by coordinate after alignment (required before indexing)
  • Indexing sorted BAM files for random access and region queries
  • Converting between SAM, BAM, and CRAM formats to save storage
  • Generating alignment QC metrics: mapping rates, insert sizes, per-chromosome stats
  • Filtering reads by mapping quality, FLAG bits, or genomic regions
  • Marking or removing PCR duplicates before variant calling
  • Merging multiple BAM files from different lanes or samples
  • Calculating per-base depth or coverage breadth for target regions
  • Use pysam instead for Python-native BAM manipulation in custom scripts
  • Use deeptools bamCoverage instead when you need normalized bigWig coverage tracks
  • Use mosdepth instead for whole-genome per-base depth (faster, parallelized)

Prerequisites

  • Installation: samtools 1.17+ recommended
  • Input requirements: SAM/BAM/CRAM files; CRAM requires FASTA reference
  • Companion tools: samtools faidx for FASTA indexing; samtools sort before samtools index

Check before installing: The tool may already be available in the current environment (e.g., inside a pixi / conda env). Run command -v samtools first and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool via pixi run samtools rather than bare samtools.

# Bioconda (recommended)
conda install -c bioconda samtools

# Homebrew (macOS)
brew install samtools

# Verify
samtools --version | head -1

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: mappingQualityFloor
    kind: required
    source: user
    ask: "Below what mapping confidence should reads be dropped? Everything counted downstream inherits this cut."
    default: "0 - keep every alignment, including ambiguously placed ones"

  - id: D2
    param: flagFilter
    kind: required
    source: user
    ask: "Which read classes should be excluded - unmapped, secondary, supplementary, failed-QC, duplicates?"
    default: "keep all classes"

  - id: D3
    param: duplicateHandling
    kind: required
    source: user
    ask: "Should duplicates be marked and left in place, or physically removed from the file?"
    default: "marked, not removed - downstream tools can then choose"

  - id: D4
    param: sortOrder
    kind: derived
    source: upstream
    ask: "Does the next step need coordinate order, or name order for mate pairing?"
    default: "coordinate"

  - id: D5
    param: outputFormat
    kind: optional
    source: user
    ask: "Write BAM, or CRAM against a reference to save space?"
    default: "BAM"

  - id: D6
    param: opticalDuplicateDistance
    kind: optional_conditional
    source: data
    ask: "Should optical duplicates be distinguished from PCR duplicates, using this platform's pixel distance?"
    default: "not distinguished"

  - id: D7
    param: threadsAndMemory
    kind: never_ask
    source: data
    reason: "Compression threads and per-thread sort memory affect runtime and peak RAM, not the records"
    default: "available cores, 768M per thread"

D1 and D2 look like plumbing and are not. A MAPQ floor silently removes multi-mapping regions - paralogs, recent duplications, repeat-adjacent genes - from everything computed afterwards, and a run that keeps secondary alignments counts the same fragment several times. Neither is visible in the output file.

Quick Start

# Typical post-alignment workflow: sort → index → QC
samtools sort -@ 8 -o sorted.bam input.bam
samtools index sorted.bam
samtools flagstat sorted.bam

Core API

Module 1: BAM/SAM I/O and Format Conversion

Convert between SAM/BAM/CRAM formats and extract subsets.

# SAM → BAM (saves ~75% disk space)
samtools view -b -h input.sam -o output.bam

# BAM → CRAM (saves additional 40-50%)
samtools view -C -T reference.fa input.bam -o output.cram

# Filter: mapping quality ≥20, exclude unmapped (-F 4)
samtools view -q 20 -F 4 input.bam -o filtered.bam

# Extract specific region (requires index)
samtools view -h sorted.bam "chr1:1000000-2000000" -o region.bam

# Count reads matching filter
samtools view -c -F 4 input.bam
# Output: 45231923 (number of mapped reads)
# Extract reads as FASTQ (for realignment or de novo assembly)
samtools fastq -@ 4 -1 R1.fastq.gz -2 R2.fastq.gz -0 unpaired.fastq.gz input.bam

# Extract reads as FASTA
samtools fasta input.bam > reads.fasta

# Filter by read group
samtools view -r SAMPLE_001 multi_rg.bam -o sample001.bam

Module 2: Sorting and Indexing

Organize BAM files for efficient random access.

# Sort by coordinate (required before indexing)
samtools sort -@ 8 -m 2G input.bam -o sorted.bam

# Sort by read name (required for fixmate/markdup)
samtools sort -n -@ 8 input.bam -o namesorted.bam

# Index sorted BAM (creates sorted.bam.bai)
samtools index sorted.bam

# For chromosomes > 512 Mbp: use CSI index instead
samtools index -c sorted.bam

# Group reads by name (fast, for fixmate — no full sort needed)
samtools collate -o collated.bam input.bam

Module 3: Quality Control and Statistics

Generate alignment QC metrics and coverage reports.

# Quick summary: total, mapped, paired, properly paired
samtools flagstat sorted.bam
# Example output:
# 50000000 + 0 in total (QC-passed reads + QC-failed reads)
# 48523111 + 0 mapped (97.05% : N/A)
# 50000000 + 0 paired in sequencing
# 48490234 + 0 properly paired (96.98% : N/A)

# Per-chromosome mapped/unmapped read counts
samtools idxstats sorted.bam
# chr1  248956422  12345678  0
# chr2  242193529  11234567  0

# Comprehensive stats (insert sizes, GC content, base quality)
samtools stats -r reference.fa sorted.bam > full_stats.txt
grep "^SN" full_stats.txt | cut -f2,3  # Summary Numbers only

# Coverage report (min/max/mean per region/chromosome)
samtools coverage sorted.bam
# Per-base read depth for specific regions
samtools depth -b target_regions.bed sorted.bam > depth.txt
# Output: chr  pos  depth (e.g., chr1  1000  45)

# Statistics split by read group
samtools stats -S RG sorted.bam > per_rg_stats.txt

Module 4: Read Filtering and FLAG Operations

Filter reads using SAM FLAG bits for specific subsets.

# FLAG reference — common masks:
# 1    = paired         4  = unmapped
# 2    = proper pair    8  = mate unmapped
# 16   = reverse strand 64 = R1 (first in pair)
# 128  = R2             256= secondary alignment
# 1024 = PCR duplicate  2048= supplementary

# Extract properly paired, mapped reads (FLAG 2 set, 4 unset)
samtools view -f 2 -F 4 sorted.bam -o proper_pairs.bam

# Extract R1 reads only
samtools view -f 64 sorted.bam -o R1.bam

# Remove secondary and supplementary alignments
samtools view -F 2304 sorted.bam -o primary.bam

# Extract reads from BED file regions
samtools view -L regions.bed -b sorted.bam -o regions.bam

Module 5: Duplicate Handling

Mark or remove PCR duplicates before variant calling.

# Full duplicate marking workflow (collate → fixmate → sort → markdup)
samtools collate -@ 8 -o collated.bam input.bam
samtools fixmate -m -@ 8 collated.bam fixmated.bam
samtools sort -@ 8 -o sorted.bam fixmated.bam
samtools markdup -@ 8 sorted.bam marked.bam
samtools index marked.bam

# Check duplication rate
samtools flagstat marked.bam | grep "duplicates"
# Output: 2345678 + 0 duplicates (4.83%)
# NovaSeq optical duplicate detection (2500 pixel distance)
samtools markdup -d 2500 sorted.bam marked_novaseq.bam

# Remove duplicates instead of marking
samtools markdup -r sorted.bam deduped.bam

# Get duplication stats without writing output
samtools markdup -s sorted.bam /dev/null

Module 6: Multi-file Operations and Region Analysis

Merge BAM files and perform region-level analysis.

# Merge multiple BAM files (all must be sorted)
samtools merge -@ 8 merged.bam lane1.bam lane2.bam lane3.bam

# Merge files listed in a text file (one per line)
samtools merge -b bam_list.txt -@ 8 merged.bam

# Merge with read group tags from filenames
samtools merge -r merged.bam sample1.bam sample2.bam

# Extract specific chromosome region from merged output
samtools view -h merged.bam chr1 -b -o chr1.bam

Key Concepts

SAM FLAG Bits

FLAGS encode read properties as a sum of bit values. Common filtering patterns:

Common Filter-f (require)-F (exclude)Selects
Mapped reads—4All aligned reads
Proper pairs2—Properly paired, both mapped
Unique primary—2308No secondary/supplementary/duplicate
R1 only64—First-in-pair reads
Unmapped4—Failed to align

CRAM vs BAM vs SAM

FormatSizeSpeedRequires
SAM~10× BAMSlow I/ONothing
BAM1×Fast.bai index for random access
CRAM~0.6× BAMSlightly slowerReference FASTA + index

Use CRAM for long-term storage; BAM for active analysis.

Common Workflows

Workflow 1: Post-Alignment QC and Preparation

Goal: Convert aligner output to analysis-ready BAM with QC metrics.

#!/bin/bash
SAMPLE="sample_001"
REF="reference.fa"
THREADS=8

# 1. Sort and index (aligner often outputs unsorted SAM/BAM)
samtools sort -@ $THREADS -o ${SAMPLE}.sorted.bam ${SAMPLE}.bam
samtools index ${SAMPLE}.sorted.bam

# 2. QC metrics
samtools flagstat ${SAMPLE}.sorted.bam > ${SAMPLE}.flagstat.txt
samtools stats -r $REF ${SAMPLE}.sorted.bam > ${SAMPLE}.stats.txt
samtools coverage ${SAMPLE}.sorted.bam > ${SAMPLE}.coverage.txt

# 3. Per-chromosome stats
samtools idxstats ${SAMPLE}.sorted.bam > ${SAMPLE}.idxstats.txt

echo "QC complete: $(grep 'mapped (' ${SAMPLE}.flagstat.txt | head -1)"

Workflow 2: Full Duplicate-Marking Pipeline

Goal: Prepare BAM for GATK or other variant callers requiring deduplicated input.

#!/bin/bash
INPUT="aligned.bam"
FINAL="deduped.bam"
THREADS=8

# Collate → fixmate → sort → markdup
samtools collate -@ $THREADS -o collated.bam $INPUT
samtools fixmate -m -@ $THREADS collated.bam fixmated.bam
samtools sort -@ $THREADS -o sorted.bam fixmated.bam
samtools markdup -@ $THREADS -s sorted.bam $FINAL

# Clean up intermediates
rm collated.bam fixmated.bam sorted.bam

# Index and verify
samtools index $FINAL
samtools flagstat $FINAL | grep "duplic"
# Expected: 3-15% duplicates (WGS); 10-30% for amplicon

Key Parameters

ParameterCommandDefaultRange/OptionsEffect
-@Most01–N coresAdditional compression/I/O threads
-msort768Me.g., 2G, 4GMemory per thread for sorting
-qview00–60Minimum mapping quality filter
-fview0FLAG bitsInclude reads with ALL bits set
-Fview0FLAG bitsExclude reads with ANY bit set
-bview—flagOutput BAM format
-Cview—flagOutput CRAM (requires -T)
-Tview—FASTA pathReference for CRAM output
-dmarkdup00–2500Optical duplicate pixel distance
-rmarkdup—flagRemove duplicates (vs just mark)
-nsort—flagSort by read name instead of position
-cindex—flagCreate CSI index (needed for chr > 512 Mb)

Best Practices

  1. Always sort before indexing: samtools index requires coordinate-sorted input. Attempting to index an unsorted BAM will fail or produce incorrect results.

  2. Use -@ for all production runs: Most samtools commands are I/O-bound. Adding -@ 8 provides near-linear speedup for compression/decompression with minimal overhead.

  3. Run flagstat before any analysis: samtools flagstat runs in seconds and catches alignment failures (low mapping rate, unexpected paired-end rates) before wasting time on downstream steps.

  4. Use the collate → fixmate → sort → markdup pipeline: Running samtools markdup directly on coordinate-sorted BAM without fixmate produces incorrect duplicate detection. The mate information added by fixmate -m is essential.

  5. Prefer CRAM for archiving: CRAM reduces storage 40-50% vs BAM with no loss. Always store the reference FASTA alongside CRAM files.

  6. Use -L bed_file for targeted analyses: Restricting samtools view to BED-defined target regions (WES capture, amplicons) dramatically reduces I/O for downstream steps.

Common Recipes

Recipe: Batch Flagstat for Multiple Samples

# Process all BAM files in directory
for bam in *.sorted.bam; do
    echo "=== $bam ==="
    samtools flagstat $bam | grep -E "mapped|properly paired|duplicates"
done

Recipe: Extract Unmapped Reads for De Novo Assembly

# Pull both unmapped reads (useful for pathogen detection)
samtools view -f 4 -b input.bam -o unmapped.bam
samtools fastq -@ 4 -1 unmapped_R1.fastq -2 unmapped_R2.fastq unmapped.bam
echo "Unmapped pairs ready for de novo assembly"

Recipe: Downsample BAM to Target Coverage

# Estimate current depth, then subsample to ~30×
TOTAL=$(samtools flagstat input.bam | grep "mapped (" | head -1 | awk '{print $1}')
GENOME_SIZE=3100000000  # hg38
READ_LEN=150
CURRENT_COV=$(echo "scale=1; $TOTAL * $READ_LEN / $GENOME_SIZE" | bc)
TARGET_FRAC=$(echo "scale=3; 30 / $CURRENT_COV" | bc)
echo "Current: ${CURRENT_COV}×; subsample fraction: $TARGET_FRAC"
samtools view -b -s $TARGET_FRAC input.bam -o downsampled.bam
samtools index downsampled.bam

Troubleshooting

ProblemCauseSolution
[bam_index_build2] fail to indexBAM not sorted by coordinateSort first: samtools sort -o sorted.bam input.bam
BAI index too large for chromosomeChromosome > 512 MbpUse CSI index: samtools index -c input.bam
CRAM: reference not foundMissing or wrong reference FASTASet REF_PATH env var or use -T ref.fa
Duplicate marking incorrectfixmate step skippedRun full pipeline: collate → fixmate → sort → markdup
flagstat shows 0% properly pairedPaired-end BAM missing mate infoRun samtools fixmate to populate mate coordinates
Very slow sortingLow memory per threadIncrease -m 4G; reduce -@ if memory-limited
Region query returns nothingBAM not indexed or wrong coordsRun samtools index; use 1-based coords: chr1:1000-2000
[E::hts_open_format] fail to openFile path wrong or BAM corruptVerify path; test with samtools quickcheck file.bam

Related Skills

  • deeptools-ngs-analysis — normalized bigWig coverage tracks and ChIP-seq visualization downstream of samtools
  • pysam-genomic-files — Python API for BAM manipulation in custom scripts
  • bedtools-genomic-intervals — genomic interval operations on BAM/BED files produced by samtools

References

STAR — Spliced RNA-seq Aligner

Overview

STAR (Spliced Transcripts Alignment to a Reference) aligns RNA-seq reads to a genome in a splice-aware manner, identifying novel and annotated splice junctions in a single pass. It generates coordinate-sorted BAM files compatible with samtools, IGV, deeptools, and GATK. STAR's 2-pass mode re-aligns reads using junctions discovered in the first pass, improving sensitivity for novel splice sites. With --quantMode GeneCounts, STAR simultaneously produces gene-level read count tables without requiring a separate featureCounts or HTSeq step.

When to Use

  • Aligning bulk RNA-seq reads to a reference genome when downstream tools require a BAM file (variant calling, visualization, deeptools)
  • Running ENCODE-compliant RNA-seq pipelines that mandate genome alignment
  • Discovering novel splice junctions and alternative splicing events in the dataset
  • Generating gene count tables alongside BAM alignment in a single step with --quantMode GeneCounts
  • Processing long reads or reads with high mismatch rates by tuning --outFilterMismatchNmax
  • Use Salmon instead when you only need transcript/gene quantification and do not need a BAM file — Salmon is 20-50× faster

Prerequisites

  • Software: STAR ≥ 2.7.0 (conda or compiled binary)
  • Reference files: genome FASTA + GTF annotation (same assembly)
  • RAM: 30–32 GB for human/mouse genome index; 8–16 GB for smaller genomes
  • Disk: ~25 GB for human genome index, ~5–10 GB per sample BAM

Check before installing: The tool may already be available in the current environment (e.g., inside a pixi / conda env). Run command -v STAR first and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool via pixi run STAR rather than bare STAR.

# Install with conda (recommended)
conda install -c bioconda star

# Verify
STAR --version
# STAR_2.7.11a

# Or compile from source
git clone https://github.com/alexdobin/STAR
cd STAR/source && make STAR

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: reference_bundle
    kind: required
    source: user
    ask: "Which species, genome assembly, and matching GTF annotation release should be used?"
    default: null

  - id: D2
    param: --sjdbOverhang
    kind: derived
    source: data
    ask: "What is the longest trimmed read length observed in the FASTQ files?"
    default: "maximum trimmed read length minus 1"

  - id: D3
    param: --quantMode
    kind: derived
    source: upstream
    ask: "Does the downstream workflow require STAR gene counts, transcriptome BAM for RSEM, or only the genomic BAM?"
    default: "no quantMode; emit a coordinate-sorted genomic BAM"

  - id: D4
    param: --twopassMode
    kind: optional
    source: user
    ask: "Should alignment prioritize sensitive discovery of novel splice junctions?"
    default: "None"

  - id: D5
    param: --outFilterMultimapNmax
    kind: optional
    source: user
    ask: "How many genomic loci may a read match before it is treated as too ambiguous to align?"
    default: 10

  - id: D6
    param: --outFilterMismatchNmax
    kind: optional
    source: user
    ask: "Does this assay require a non-default maximum number of alignment mismatches?"
    default: 10

  - id: D7
    param: --alignIntronMax
    kind: optional_conditional
    source: literature
    ask: "Does the organism or assay require an intron-length limit different from the reference bundle's standard setting?"
    default: 1000000

  - id: D8
    param: --genomeSAindexNbases
    kind: never_ask
    source: data
    reason: "Calculated from genome length to size the index; it affects memory use, not the biological result."
    default: "min(14, floor(log2(genome_length) / 2 - 1))"

  - id: D9
    param: --runThreadN
    kind: never_ask
    source: data
    reason: "Affects runtime only, not the alignments"
    default: "min(8, available_cores)"

D2 is measured from the input FASTQ rather than chosen by the user. The GTF, genome FASTA, and STAR index must all come from the same reference release. GeneCounts produces STAR's own counts; omit it when featureCounts will produce the count matrix. TranscriptomeSAM is for transcriptome-BAM consumers such as RSEM, not ordinary Salmon quantification.

Quick Start

# 1. Generate genome index (~30 min, run once)
STAR --runMode genomeGenerate \
     --runThreadN 8 \
     --genomeDir genome/star_index \
     --genomeFastaFiles genome/GRCh38.fa \
     --sjdbGTFfile genome/gencode.v47.gtf \
     --sjdbOverhang 100    # ReadLength - 1

# 2. Align paired-end reads (~10-20 min)
STAR --runThreadN 8 \
     --genomeDir genome/star_index \
     --readFilesIn sample_R1.fastq.gz sample_R2.fastq.gz \
     --readFilesCommand zcat \
     --outSAMtype BAM SortedByCoordinate \
     --outFileNamePrefix results/sample/

# 3. Index the BAM
samtools index results/sample/Aligned.sortedByCoord.out.bam

Workflow

Step 1: Prepare Reference Files

Download a genome FASTA and matching GTF annotation (same assembly version).

# Download GRCh38 genome and GENCODE annotation
wget https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_47/GRCh38.primary_assembly.genome.fa.gz
wget https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_47/gencode.v47.primary_assembly.annotation.gtf.gz

gunzip GRCh38.primary_assembly.genome.fa.gz gencode.v47.primary_assembly.annotation.gtf.gz
mkdir -p genome/star_index

echo "Genome and GTF ready."
ls -lh GRCh38.primary_assembly.genome.fa gencode.v47.primary_assembly.annotation.gtf

Step 2: Generate Genome Index

Build the STAR genome index — required once per genome/read-length combination.

# Standard human genome index (requires ~32 GB RAM)
STAR --runMode genomeGenerate \
     --runThreadN 16 \
     --genomeDir genome/star_index/ \
     --genomeFastaFiles GRCh38.primary_assembly.genome.fa \
     --sjdbGTFfile gencode.v47.primary_assembly.annotation.gtf \
     --sjdbOverhang 100

# For small genomes (e.g., E. coli ~4.6 Mb), reduce genomeSAindexNbases
# STAR --runMode genomeGenerate \
#      --genomeSAindexNbases 11 \
#      --genomeDir genome/ecoli_index/ ...

echo "Index complete: $(ls genome/star_index/ | wc -l) files"

Step 3: Align RNA-seq Reads

Align single-end or paired-end FASTQ files to the indexed genome.

# Single-end alignment
STAR --runThreadN 8 \
     --genomeDir genome/star_index/ \
     --readFilesIn sample1.fastq.gz \
     --readFilesCommand zcat \
     --outSAMtype BAM SortedByCoordinate \
     --outSAMattributes NH HI AS NM MD \
     --outFileNamePrefix results/sample1/

# Paired-end alignment
STAR --runThreadN 8 \
     --genomeDir genome/star_index/ \
     --readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
     --readFilesCommand zcat \
     --outSAMtype BAM SortedByCoordinate \
     --outSAMattributes NH HI AS NM MD \
     --outFileNamePrefix results/sample1/

echo "BAM: results/sample1/Aligned.sortedByCoord.out.bam"

Step 4: Run 2-Pass Alignment for Improved Sensitivity

Two-pass mode collects splice junctions from the first pass and uses them as annotation for the second pass.

# First pass — collect splice junctions
STAR --runThreadN 8 \
     --genomeDir genome/star_index/ \
     --readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
     --readFilesCommand zcat \
     --outSAMtype None \
     --outFileNamePrefix pass1/sample1/

# Second pass — realign with all junctions from pass 1
SJ_FILES=$(ls pass1/*/SJ.out.tab | tr '\n' ' ')

STAR --runThreadN 8 \
     --genomeDir genome/star_index/ \
     --readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
     --readFilesCommand zcat \
     --sjdbFileChrStartEnd $SJ_FILES \
     --outSAMtype BAM SortedByCoordinate \
     --outFileNamePrefix results/sample1/

# Alternative: single-command 2-pass
STAR --runThreadN 8 \
     --genomeDir genome/star_index/ \
     --readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
     --readFilesCommand zcat \
     --twopassMode Basic \
     --outSAMtype BAM SortedByCoordinate \
     --outFileNamePrefix results/sample1/

Step 5: Check Alignment Statistics

Parse the alignment log to assess mapping rate and read quality.

# View the alignment summary
cat results/sample1/Log.final.out

# Parse key metrics with python
python3 - << 'EOF'
import re, sys
from pathlib import Path

log = Path("results/sample1/Log.final.out").read_text()
metrics = {}
for line in log.splitlines():
    if "|" in line:
        key, _, val = line.partition("|")
        metrics[key.strip()] = val.strip()

print(f"Unique mapping:     {metrics.get('Uniquely mapped reads %', 'N/A')}")
print(f"Multi-mapping:      {metrics.get('% of reads mapped to multiple loci', 'N/A')}")
print(f"Too many mismatches:{metrics.get('% of reads unmapped: too many mismatches', 'N/A')}")
print(f"Total input reads:  {metrics.get('Number of input reads', 'N/A')}")
EOF

Step 6: Generate Gene Count Tables

Enable simultaneous gene counting during alignment using --quantMode GeneCounts.

# Align and count simultaneously
STAR --runThreadN 8 \
     --genomeDir genome/star_index/ \
     --readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
     --readFilesCommand zcat \
     --outSAMtype BAM SortedByCoordinate \
     --quantMode GeneCounts \
     --outFileNamePrefix results/sample1/

# ReadsPerGene.out.tab has 4 columns:
# gene_id  unstranded  stranded_fwd  stranded_rev
head results/sample1/ReadsPerGene.out.tab

# Load into pandas (select column based on library strandedness)
python3 - << 'EOF'
import pandas as pd

df = pd.read_csv("results/sample1/ReadsPerGene.out.tab",
                 sep="\t", header=None, skiprows=4,
                 names=["gene_id", "unstranded", "fwd", "rev"])
# For unstranded library: use column 2 (unstranded)
counts = df.set_index("gene_id")["unstranded"]
print(f"Genes with counts > 0: {(counts > 0).sum()}")
print(counts[counts > 0].sort_values(ascending=False).head())
EOF

Key Parameters

ParameterDefaultRange/OptionsEffect
--runThreadN11–64CPU threads for alignment
--sjdbOverhang99ReadLength-1Splice junction overhang; set to ReadLength-1
--outSAMtypeSAMBAM SortedByCoordinate, BAM UnsortedOutput format and sort order
--outFilterMismatchNmax100–33Max mismatches per read; lower for stricter mapping
--outFilterMultimapNmax101–9999Max genomic loci per read; reads exceeding limit marked unmapped
--quantMode–GeneCounts, TranscriptomeSAMEnable gene counting or transcriptome BAM
--twopassModeNoneNone, BasicEnable 2-pass alignment for novel junction discovery
--alignIntronMax10000001–1e9Maximum intron length; reduce for bacterial genomes
--outReadsUnmappedNoneFastxWrite unmapped reads to FASTQ
--genomeSAindexNbases1410–14SA index size; set log2(GenomeSize)/2 − 1 for small genomes

Common Recipes

Recipe 1: Batch Align All Samples

#!/bin/bash
# Align all paired-end samples in a directory
SAMPLES=(ctrl_1 ctrl_2 treat_1 treat_2)
INDEX="genome/star_index"
DATA="data"
OUT="results"
THREADS=12

mkdir -p "$OUT"
for sample in "${SAMPLES[@]}"; do
    echo "Aligning: $sample"
    mkdir -p "$OUT/$sample"
    STAR --runThreadN "$THREADS" \
         --genomeDir "$INDEX" \
         --readFilesIn "$DATA/${sample}_R1.fastq.gz" "$DATA/${sample}_R2.fastq.gz" \
         --readFilesCommand zcat \
         --outSAMtype BAM SortedByCoordinate \
         --quantMode GeneCounts \
         --twopassMode Basic \
         --outFileNamePrefix "$OUT/$sample/"
    samtools index "$OUT/$sample/Aligned.sortedByCoord.out.bam"
    echo "Done: $sample — $(grep 'Uniquely mapped reads %' $OUT/$sample/Log.final.out | awk '{print $NF}')"
done

Recipe 2: Build Gene Count Matrix Across Samples

import pandas as pd
from pathlib import Path

results_dir = Path("results")
samples = ["ctrl_1", "ctrl_2", "treat_1", "treat_2"]
strandedness = "unstranded"  # or "fwd" / "rev"

col_map = {"unstranded": 1, "fwd": 2, "rev": 3}
col = col_map[strandedness]

counts = {}
for sample in samples:
    count_file = results_dir / sample / "ReadsPerGene.out.tab"
    df = pd.read_csv(count_file, sep="\t", header=None, skiprows=4)
    counts[sample] = df.set_index(0)[col]

matrix = pd.DataFrame(counts)
matrix = matrix[matrix.sum(axis=1) > 0]  # drop zero-count genes
matrix.to_csv("gene_count_matrix.tsv", sep="\t")
print(f"Count matrix: {matrix.shape} (genes × samples)")
print(matrix.head())

Recipe 3: Integrate with DESeq2 via pydeseq2

import pandas as pd
from pydeseq2.dds import DeseqDataSet
from pydeseq2.default_inference import DefaultInference

# Load count matrix from STAR output
counts = pd.read_csv("gene_count_matrix.tsv", sep="\t", index_col=0).T
metadata = pd.DataFrame({
    "condition": ["control", "control", "treated", "treated"]
}, index=counts.index)

# Run DESeq2
dds = DeseqDataSet(counts=counts, metadata=metadata,
                   design_factors="condition",
                   inference=DefaultInference(n_cpus=4))
dds.deseq2()
print("DESeq2 complete — see dds.varm['LFC'] for results")

Expected Outputs

OutputFormatDescription
Aligned.sortedByCoord.out.bamBAMCoordinate-sorted aligned reads; index with samtools index
SJ.out.tabTSVSplice junction table with coverage, motif, and novelty flags
Log.final.outTextAlignment statistics: unique mapping %, multimappers %, etc.
ReadsPerGene.out.tabTSVGene counts (4 columns: unstranded/fwd/rev) when --quantMode GeneCounts
Unmapped.out.mate1/2FASTQUnmapped reads (when --outReadsUnmapped Fastx)
Log.outTextVerbose run log; check for warnings and parameter echoes

Troubleshooting

ProblemCauseSolution
Unique mapping < 60%Wrong genome assembly or species contaminationVerify genome FASTA matches sample species; run FastQC to check overrepresented sequences
Fatal error: genome files not foundWrong --genomeDir path or incomplete indexRe-run genomeGenerate; check genomeDir contains Genome, SA, SAindex files
Out of memory during genome generationNot enough RAM for genome SA indexAdd --genomeSAindexNbases 13 (or lower) for small genomes; request ≥32 GB RAM for human
.gz files not decompressedMissing --readFilesCommand zcatAdd --readFilesCommand zcat for gzip-compressed inputs
Error: number of input files differR1/R2 read count mismatchVerify FASTQ files with `zcat file.fastq.gz
ReadsPerGene.out.tab missing--quantMode GeneCounts not setRe-run with --quantMode GeneCounts or use featureCounts on BAM
Very high multimapping (>20%)Highly repetitive genome or wrong --outFilterMultimapNmaxReduce --outFilterMultimapNmax; use --outSAMmultNmax 1 to output only one alignment per read
Genome index takes too longLarge genome + slow diskUse SSD storage; pre-built indices available from ENCODE and Ensembl

References

Bakta Genome Annotation

Overview

Bakta is a command-line pipeline for rapid, standardized annotation of bacterial and archaeal genomes and plasmids. It combines Prodigal for CDS prediction, tRNAscan-SE/Aragorn/Barrnap/Infernal for non-coding RNA, PILER-CR/PILERCR for CRISPR detection, and a tiered DIAMOND/HMM search against a curated UniRef100 + IPS/UPS database to assign gene names, EC numbers, GO terms, and COG categories. Bakta produces NCBI-compatible outputs (GFF3, GenBank, EMBL, INSDC-formatted FASTA, plus a JSON summary and a circular Circos plot) for a typical 5 Mb genome in 5–15 minutes on 8 CPUs.

When to Use

  • Annotating bacterial or archaeal genome assemblies (Illumina, PacBio, Nanopore) with NCBI-compatible locus tags and product names
  • Annotating plasmids and other circular replicons separately with --plasmid and --complete flags
  • Producing JSON-structured annotation outputs that can be parsed without GenBank or GFF3 detours
  • Generating a publication-ready circular genome plot via the bundled bakta_plot command
  • Annotating MAGs (metagenome-assembled genomes) with --meta to disable Prodigal training
  • Use Prokka instead when you need viral/mitochondrial kingdoms or when you must reproduce a legacy Prokka pipeline exactly
  • Use PGAP instead when submitting to NCBI GenBank with full standards compliance
  • Use Bakta when you want faster runs, regularly updated UniRef-derived databases, AMRFinderPlus integration, and a JSON summary out of the box

Prerequisites

  • Software: Bakta ≥ 1.9, Python 3.8+, Prodigal, tRNAscan-SE, Aragorn, Barrnap, Infernal, DIAMOND, HMMER3, PILER-CR, BLAST+, AMRFinderPlus
  • Database: Bakta DB (full ~70 GB, or light ~3 GB) downloaded once with bakta_db download
  • Python packages (for output parsing): biopython, pandas, matplotlib
  • Input: assembled genome in FASTA format (one or more contigs)
  • Hardware: ≥ 16 GB RAM for full DB, ≥ 4 GB RAM for light DB; ≥ 8 CPUs recommended

Check before installing: The tool may already be available in the current environment (e.g., inside a pixi / conda env). Run command -v bakta first and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool via pixi run bakta rather than bare bakta.

# Install Bakta via conda/mamba (recommended)
mamba install -c conda-forge -c bioconda bakta

# Verify installation
bakta --version
# bakta 1.9.4

# Download the light database (~3 GB, faster, fewer functional hits)
bakta_db download --output db/ --type light

# Or full database (~70 GB, comprehensive UniRef100 coverage)
# bakta_db download --output db/ --type full

# Install Python parsing dependencies
pip install biopython pandas matplotlib

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: database
    kind: required
    source: user
    ask: "Annotate against the light database or the full one? The light database leaves more genes described only generically."
    default: null

  - id: D2
    param: organismMetadata
    kind: required
    source: user
    ask: "Which genus, species, and strain should be recorded in the output?"
    default: null

  - id: D3
    param: repliconTopology
    kind: required
    source: data
    ask: "Are these contigs complete circular replicons, plasmids, or draft fragments?"
    default: "draft fragments - no circularity assumed"

  - id: D4
    param: assemblyOrigin
    kind: required
    source: data
    ask: "Is this an isolate genome, or a bin recovered from a metagenome?"
    default: "isolate - gene prediction trains on the assembly itself"

  - id: D5
    param: translationTable
    kind: required
    source: user
    ask: "Which genetic code does this organism use?"
    default: "11, the bacterial and archaeal code"

  - id: D6
    param: locusTagPrefix
    kind: optional
    source: user
    ask: "Which prefix should locus tags carry, if they will be submitted or cross-referenced?"
    default: "generated automatically"

  - id: D7
    param: minContigLength
    kind: optional_conditional
    source: data
    ask: "Should short contigs be skipped?"
    default: "no minimum"

  - id: D8
    param: threadsAndPlotting
    kind: never_ask
    source: data
    reason: "Thread count and the circular plot affect runtime and output files, not the annotation"
    default: "available cores, plot enabled"

D5 is asked rather than defaulted because the exceptions are real and silent: Mycoplasma and relatives reassign a stop codon, so annotating them with the standard bacterial code truncates a large fraction of their genes into fragments that still look like ordinary short CDS calls.

Quick Start

# Annotate a bacterial genome — results in results/ directory
bakta genome.fasta \
    --db db/bakta_db_light \
    --output results/ \
    --prefix sample1 \
    --threads 8

# Inspect the JSON summary for feature counts
python -c "
import json
with open('results/sample1.json') as f:
    d = json.load(f)
print('Genus:', d['genome'].get('genus'))
print('Length:', d['genome']['size'], 'bp')
print('CDS:', sum(1 for f in d['features'] if f['type'] == 'cds'))
print('tRNA:', sum(1 for f in d['features'] if f['type'] == 'tRNA'))
"

Workflow

Step 1: Install Bakta and Download the Database

Install Bakta and prepare the reference database. The database download is one-time and reused across runs.

# Create a dedicated conda environment (avoids dependency conflicts)
mamba create -n bakta_env -c conda-forge -c bioconda bakta python=3.11 -y
mamba activate bakta_env

# Verify Bakta and its dependencies
bakta --version
# bakta 1.9.4

bakta --help | head -20

# Download the light database (sufficient for routine annotation)
mkdir -p db/
bakta_db download --output db/ --type light
# Downloads ~3 GB; expands to ~5 GB on disk

# Verify the database was extracted correctly
ls db/bakta_db_light/
# antifam.h3f  bakta.db  expert  oric.fna  pfam.h3f  rfam-go.tsv  ...

# (Optional) Update AMRFinderPlus DB used by Bakta for AMR gene calling
amrfinder -u

# Install Python parsing tools
pip install biopython pandas matplotlib

Step 2: Prepare the Input Assembly

Bakta requires clean FASTA headers without spaces or special characters. Pre-clean and optionally filter short contigs.

from Bio import SeqIO
import re

input_fasta = "genome.fasta"
records = list(SeqIO.parse(input_fasta, "fasta"))
print(f"Input assembly: {len(records)} contigs")
total_bases = sum(len(r) for r in records)
print(f"Total bases: {total_bases:,}")
print(f"Largest contig: {max(len(r) for r in records):,} bp")

# Bakta preferred: short, alphanumeric, unique IDs
cleaned = []
for i, rec in enumerate(records, 1):
    new_id = f"contig_{i:04d}"
    new_rec = rec.__class__(rec.seq, id=new_id, description="")
    cleaned.append(new_rec)

SeqIO.write(cleaned, "genome_clean.fasta", "fasta")
print(f"Wrote genome_clean.fasta with {len(cleaned)} contigs")
# Filter out short contigs (<200 bp) which contribute little to annotation
awk 'BEGIN{RS=">"; ORS=""} NR>1 {n=split($0, a, "\n"); seq=""; for(i=2;i<=n;i++) seq=seq a[i]; if (length(seq) >= 200) print ">" $0}' \
    genome_clean.fasta > genome_filtered.fasta

echo "Filtered assembly: $(grep -c '>' genome_filtered.fasta) contigs"

Step 3: Run Standard Bakta Annotation

Run Bakta with genus/species hints. Locus tags are auto-generated from the strain field.

# Standard annotation for a draft bacterial genome
bakta genome_clean.fasta \
    --db db/bakta_db_light \
    --output annotation/ \
    --prefix E_coli_K12 \
    --genus Escherichia \
    --species coli \
    --strain K12 \
    --locus-tag ECOLI \
    --threads 8 \
    --keep-contig-headers

# Expected runtime: 5–15 min for ~5 Mb genome on 8 CPUs (light DB)

echo "Bakta annotation outputs:"
ls annotation/
# E_coli_K12.embl   E_coli_K12.faa     E_coli_K12.ffn
# E_coli_K12.fna    E_coli_K12.gbff    E_coli_K12.gff3
# E_coli_K12.hypotheticals.faa  E_coli_K12.hypotheticals.tsv
# E_coli_K12.json   E_coli_K12.log     E_coli_K12.png
# E_coli_K12.svg    E_coli_K12.tsv     E_coli_K12.txt

Step 4: Parse the JSON Summary

Bakta's JSON output is the canonical, machine-readable annotation. Parse it directly for downstream pipelines.

import json
import pandas as pd
from collections import Counter

with open("annotation/E_coli_K12.json") as f:
    bakta = json.load(f)

# Genome-level metadata
genome = bakta["genome"]
print(f"Organism: {genome.get('genus')} {genome.get('species')} {genome.get('strain')}")
print(f"Size: {genome['size']:,} bp across {len(bakta['sequences'])} sequences")
print(f"GC content: {genome['gc']:.2%}")

# Feature type counts
features = bakta["features"]
type_counts = Counter(f["type"] for f in features)
print("\nFeature counts:")
for ftype, n in sorted(type_counts.items(), key=lambda x: -x[1]):
    print(f"  {ftype:>10}: {n}")

# Build a tidy CDS DataFrame
cds_rows = []
for f in features:
    if f["type"] != "cds":
        continue
    cds_rows.append({
        "locus_tag": f.get("locus", ""),
        "contig":    f.get("contig", ""),
        "start":     f.get("start"),
        "stop":      f.get("stop"),
        "strand":    f.get("strand"),
        "gene":      f.get("gene", ""),
        "product":   f.get("product", ""),
        "length_aa": len(f.get("aa", "")),
    })

cds_df = pd.DataFrame(cds_rows)
print(f"\nTotal CDS: {len(cds_df)}")
print(cds_df.head(5).to_string(index=False))

Step 5: Parse the TSV Feature Table

The TSV output is convenient for spreadsheet workflows and quick filtering.

import pandas as pd

# Bakta TSV begins with comment lines starting with '#'
df = pd.read_csv("annotation/E_coli_K12.tsv", sep="\t", comment="#",
                 names=["sequence_id", "type", "start", "stop", "strand",
                        "locus_tag", "gene", "product", "dbxrefs"])
print(f"Total features: {len(df)}")
print(f"Feature types: {df['type'].value_counts().to_dict()}")

# Hypothetical vs annotated CDS
cds = df[df["type"] == "cds"].copy()
hypothetical = cds["product"].str.contains("hypothetical", case=False, na=True)
print(f"\nCDS with assigned function: {(~hypothetical).sum()} / {len(cds)}")
print(f"Hypothetical proteins: {hypothetical.sum()}")

# Cross-references (UniRef, KEGG, EC, GO, etc.) parsed from the dbxrefs column
def split_xrefs(xref_str):
    if not isinstance(xref_str, str) or xref_str in ("", "-"):
        return []
    return [x.strip() for x in xref_str.split(",")]

cds["dbxref_list"] = cds["dbxrefs"].apply(split_xrefs)
ec_hits = cds[cds["dbxref_list"].apply(lambda xs: any(x.startswith("EC:") for x in xs))]
print(f"CDS with EC numbers: {len(ec_hits)}")
print(ec_hits[["locus_tag", "gene", "product"]].head(5).to_string(index=False))

Step 6: Render the Circular Genome Plot

Bakta emits a Circos-style PNG/SVG by default. Regenerate with custom styling using bakta_plot.

# Re-render the plot from the existing JSON with a different style
bakta_plot --output annotation/ \
           --prefix E_coli_K12_replot \
           --type cog \
           --dpi 300 \
           annotation/E_coli_K12.json

ls annotation/E_coli_K12_replot.*
# E_coli_K12_replot.png  E_coli_K12_replot.svg

# Display the rendered PNG inline in a Jupyter notebook
python <<'PY'
from pathlib import Path
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

img = mpimg.imread("annotation/E_coli_K12.png")
fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(img)
ax.axis("off")
ax.set_title("Bakta circular genome annotation")
plt.savefig("bakta_circular_thumbnail.png", dpi=120, bbox_inches="tight")
print("Saved bakta_circular_thumbnail.png")
PY

Step 7: Compute Annotation Quality Statistics

Summarize hypothetical-protein rate, gene density, and feature coverage to assess annotation quality.

import json
import pandas as pd
import matplotlib.pyplot as plt

with open("annotation/E_coli_K12.json") as f:
    bakta = json.load(f)

genome_size = bakta["genome"]["size"]
features = bakta["features"]

# Per-feature-type counts and density (per Mb)
counts = {}
for f in features:
    counts[f["type"]] = counts.get(f["type"], 0) + 1
density = {k: v / (genome_size / 1e6) for k, v in counts.items()}

cds = [f for f in features if f["type"] == "cds"]
n_cds = len(cds)
n_hypo = sum(1 for f in cds if "hypothetical" in (f.get("product") or "").lower())
n_known = n_cds - n_hypo
coding_density = sum(abs(f["stop"] - f["start"] + 1) for f in cds) / genome_size

print(f"Genome: {genome_size:,} bp")
print(f"CDS: {n_cds}  ({density.get('cds', 0):.1f} per Mb)")
print(f"  Known function: {n_known}  Hypothetical: {n_hypo}")
print(f"Coding density: {coding_density:.1%}")
print(f"tRNA: {counts.get('tRNA', 0)}   rRNA: {counts.get('rRNA', 0)}")
print(f"ncRNA: {counts.get('ncRNA', 0)}  CRISPR: {counts.get('crispr', 0)}")

# Bar plot of feature type counts
fig, ax = plt.subplots(figsize=(8, 4))
items = sorted(counts.items(), key=lambda x: -x[1])
labels = [k for k, _ in items]
values = [v for _, v in items]
bars = ax.bar(labels, values, color="#2980B9", edgecolor="white")
for bar, v in zip(bars, values):
    ax.text(bar.get_x() + bar.get_width() / 2, v + max(values) * 0.01,
            str(v), ha="center", fontsize=9)
ax.set_ylabel("Count")
ax.set_title("Bakta feature counts by type")
plt.xticks(rotation=20)
plt.tight_layout()
plt.savefig("bakta_feature_counts.png", dpi=150, bbox_inches="tight")
print("Saved bakta_feature_counts.png")

Step 8: Batch Annotation Across Multiple Genomes

Run Bakta over a directory of assemblies and aggregate per-sample summary statistics.

#!/bin/bash
# batch_bakta.sh — annotate all FASTA files in a directory
INPUT_DIR="genomes/"
OUTPUT_DIR="annotations/"
DB="db/bakta_db_light"
mkdir -p "$OUTPUT_DIR"

for FASTA in "$INPUT_DIR"/*.fasta; do
    SAMPLE=$(basename "$FASTA" .fasta)
    echo "Annotating: $SAMPLE"
    bakta "$FASTA" \
        --db "$DB" \
        --output "${OUTPUT_DIR}/${SAMPLE}" \
        --prefix "$SAMPLE" \
        --threads 4 \
        --skip-plot \
        --force
done

echo "Batch annotation complete."
# Aggregate per-genome summaries from each sample's JSON file
from pathlib import Path
import json
import pandas as pd

annotation_dir = Path("annotations/")
rows = []
for json_file in sorted(annotation_dir.glob("*/*.json")):
    with open(json_file) as f:
        bakta = json.load(f)
    sample = json_file.stem
    counts = {}
    for feat in bakta["features"]:
        counts[feat["type"]] = counts.get(feat["type"], 0) + 1
    rows.append({
        "sample":  sample,
        "size":    bakta["genome"]["size"],
        "gc":      bakta["genome"]["gc"],
        "CDS":     counts.get("cds", 0),
        "tRNA":    counts.get("tRNA", 0),
        "rRNA":    counts.get("rRNA", 0),
        "ncRNA":   counts.get("ncRNA", 0),
        "crispr":  counts.get("crispr", 0),
    })

summary_df = pd.DataFrame(rows)
print(f"Annotated genomes: {len(summary_df)}")
print(summary_df.to_string(index=False))
summary_df.to_csv("batch_bakta_summary.csv", index=False)
print("Saved: batch_bakta_summary.csv")

Key Parameters

ParameterDefaultRange / OptionsEffect
--db—path to Bakta DB directoryRequired; selects light or full reference DB
--genus—any genus name stringSets organism metadata in GenBank/EMBL output
--species—any species name stringCombined with --genus for organism qualifier
--strain—any stringAdds strain qualifier to organism metadata
--locus-tagauto3–24 alphanumeric charsPrefix for locus tags (e.g., ECOLI_00001)
--completeoffflagTreat all input contigs as complete circular replicons
--plasmidoffflagTreat all input contigs as plasmids (circular)
--metaoffflagDisable Prodigal training (use for MAGs / metagenomic bins)
--translation-table11NCBI table IDs (1, 4, 11, 25, …)Genetic code used by Prodigal for CDS translation
--min-contig-length1any integer (bp)Skip contigs shorter than this length
--threads11–CPU countParallel threads for DIAMOND/HMMER searches
--skip-plotoffflagSkip the slow circular plot rendering step
--keep-contig-headersoffflagPreserve original contig IDs instead of renaming
--proteins—path to GenBank or FASTA fileCustom expert protein DB used before UniRef search

Common Recipes

Recipe: Plasmid-only Annotation

When to use: A finished circular plasmid sequence that should be annotated without chromosome assumptions.

bakta plasmid.fasta \
    --db db/bakta_db_light \
    --output plasmid_annotation/ \
    --prefix pBR322 \
    --plasmid \
    --complete \
    --threads 4

# Inspect plasmid-typing results (incompatibility group, replication initiator)
grep -E "rep|inc" plasmid_annotation/pBR322.tsv | head -10

Recipe: MAG (Metagenome-Assembled Genome) Annotation

When to use: Annotating a bin from metagenomic assembly where Prodigal cannot train on the full sequence.

bakta MAG_bin_42.fasta \
    --db db/bakta_db_light \
    --output mag_annotation/ \
    --prefix MAG_bin_42 \
    --meta \
    --min-contig-length 500 \
    --skip-plot \
    --threads 8

cat mag_annotation/MAG_bin_42.txt | head -20

Recipe: Annotation with Custom Expert Protein Database

When to use: You have curated reference proteins (e.g., a virulence factor catalog) that should take priority over UniRef.

# Custom proteins are searched before the bundled UniRef DB
bakta genome.fasta \
    --db db/bakta_db_light \
    --proteins virulence_factors.faa \
    --output annotation_custom/ \
    --prefix novel_strain \
    --threads 8

echo "Annotations referencing custom DB:"
grep "User-provided" annotation_custom/novel_strain.tsv | head

Recipe: Convert Bakta GFF3 to a Pandas DataFrame

When to use: You need feature coordinates and qualifiers for downstream coordinate-overlap analyses.

import pandas as pd

def parse_gff3(gff_file):
    rows = []
    with open(gff_file) as f:
        for line in f:
            if line.startswith("##FASTA"):
                break
            if line.startswith("#") or not line.strip():
                continue
            parts = line.rstrip("\n").split("\t")
            if len(parts) < 9:
                continue
            attrs = {}
            for item in parts[8].split(";"):
                if "=" in item:
                    k, v = item.split("=", 1)
                    attrs[k] = v
            rows.append({
                "seqname":   parts[0],
                "feature":   parts[2],
                "start":     int(parts[3]),
                "end":       int(parts[4]),
                "strand":    parts[6],
                "locus_tag": attrs.get("locus_tag", attrs.get("ID", "")),
                "gene":      attrs.get("gene", ""),
                "product":   attrs.get("product", ""),
            })
    return pd.DataFrame(rows)

gff_df = parse_gff3("annotation/E_coli_K12.gff3")
print(f"Features parsed: {len(gff_df)}")
print(gff_df[gff_df["feature"] == "CDS"].head(5).to_string(index=False))

Expected Outputs

Output FileFormatDescription
{prefix}.gff3GFF3Genome annotation with feature coordinates and attributes; INSDC-compliant
{prefix}.gbffGenBank Flat FileAnnotated GenBank record for use in Geneious, BioPython, NCBI submission prep
{prefix}.emblEMBLEMBL-format annotation for ENA submission
{prefix}.fnaFASTANucleotide sequences of the input contigs
{prefix}.ffnFASTANucleotide sequences of all annotated features
{prefix}.faaFASTAProtein sequences for all CDS features
{prefix}.hypotheticals.faaFASTAProtein sequences flagged as hypothetical (for further investigation)
{prefix}.hypotheticals.tsvTSVDetailed table for hypothetical CDS with low-confidence hits
{prefix}.tsvTSVFull feature table: seqid, type, start, stop, strand, locus_tag, gene, product, dbxrefs
{prefix}.jsonJSONMachine-readable summary with genome metadata + every feature with full qualifiers
{prefix}.png / .svgImageCircos circular genome plot colored by COG category
{prefix}.txtTextPlain-text summary of feature counts and per-replicon stats
{prefix}.logTextFull Bakta run log including timing per step

Troubleshooting

ProblemCauseSolution
Error: database not found at <path>DB path incorrect, or DB not extractedRe-run bakta_db download --output db/ --type light; pass full path to --db
KILLED during DIAMOND stepOut of memory on full DBSwitch to --type light, reduce --threads, or run on a host with ≥ 32 GB RAM
Very high hypothetical-protein rate (>60 %)Divergent strain or light DB without close hitsRe-run with the full DB or supply a curated --proteins reference set
tRNAscan-SE: Error opening fileMissing tRNAscan-SE installation in conda envmamba install -c bioconda trnascan-se and rerun
Bakta complains about contig headersSpaces or special characters in FASTA IDsPre-clean headers with awk '/^>/{print ">contig_"++i; next}{print}'
Plot generation hangs or failsCircos / matplotlib backend issueRe-run with --skip-plot; render later via bakta_plot from the JSON file
AMRFinderPlus: database not up-to-date warningAMRFinderPlus DB staleamrfinder -u to refresh, or pass --skip-amr to bypass
Different locus tag prefix than expected--locus-tag not set, default uses random prefixExplicitly pass --locus-tag MYORG to control prefix
Run is much slower than reportedDefault --threads 1Set --threads to physical core count; use --skip-plot for batch jobs

References

Prokka Genome Annotation

Overview

Prokka is a command-line pipeline for rapid annotation of prokaryotic genomes (bacteria, archaea, and viruses). It uses a tiered search strategy: protein-coding genes (CDS) are predicted with Prodigal and searched first against a genus-specific database, then RefSeq proteins, then Pfam/TIGRFAMs HMMs. Non-coding RNA genes (rRNA, tRNA, tmRNA) are identified with Barrnap, Aragorn, and Infernal. Prokka processes a single FASTA assembly in minutes and outputs a comprehensive annotation in GFF3, GenBank, FASTA, and tabular formats.

When to Use

  • Annotating a newly assembled bacterial or archaeal genome from Illumina, PacBio, or Nanopore assemblies
  • Getting functional protein annotations (CDS with product names, EC numbers, GO terms) from a draft or complete genome
  • Preparing annotation files for downstream comparative genomics (Roary pan-genome, OrthoFinder)
  • Annotating viral or phage genomes when kingdom-specific databases are important
  • Performing metagenome-assembled genome (MAG) annotation with the --metagenome flag
  • Parsing annotated outputs in Python with BioPython for downstream sequence or feature analysis
  • Use PGAP (NCBI Prokaryotic Genome Annotation Pipeline) instead when the goal is NCBI GenBank submission with standards compliance
  • Use Bakta instead for faster annotation with built-in NCBI-compatible outputs and a more regularly updated database

Prerequisites

  • Software: Prokka ≥ 1.14, Perl 5, Prodigal, Barrnap, HMMER3, BLAST+, Aragorn, Infernal, tbl2asn
  • Python packages (for output parsing): biopython, pandas, matplotlib
  • Input: assembled genome in FASTA format (complete or draft with multiple contigs)
  • Environment: conda strongly recommended to handle the Perl and C dependency stack

Check before installing: The tool may already be available in the current environment (e.g., inside a pixi / conda env). Run command -v prokka first and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool via pixi run prokka rather than bare prokka.

# Install Prokka via conda/mamba (recommended)
conda install -c conda-forge -c bioconda prokka

# Or with mamba (faster)
mamba install -c conda-forge -c bioconda prokka

# Verify installation and database setup
prokka --version
# prokka 1.14.6

# Check that required tools are on PATH
prokka --depends
# prokka needs: awk, sed, grep, makeblastdb, blastp, hmmscan, ...

# Install Python parsing dependencies
pip install biopython pandas matplotlib

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: kingdom
    kind: required
    source: user
    ask: "Is this a bacterial, archaeal, viral, or mitochondrial assembly?"
    default: "Bacteria"

  - id: D2
    param: organismMetadata
    kind: required
    source: user
    ask: "Which genus, species, and strain should be recorded, and used to prioritise a genus-specific protein database?"
    default: null

  - id: D3
    param: assemblyOrigin
    kind: required
    source: data
    ask: "Is this an isolate genome, or a bin recovered from a metagenome?"
    default: "isolate - gene prediction trains on the assembly itself"

  - id: D4
    param: rnaPrediction
    kind: optional
    source: user
    ask: "Should rRNA, tRNA, and other non-coding RNA be predicted alongside coding genes?"
    default: "tRNA and rRNA predicted; Rfam search off as it is slow"

  - id: D5
    param: customProteins
    kind: optional
    source: user
    ask: "Is there a curated protein set that should be searched before the general reference database?"
    default: "none"

  - id: D6
    param: hitEvalue
    kind: optional
    source: user
    ask: "How confident must a database hit be before it names a gene?"
    default: "1e-6"

  - id: D7
    param: minContigLength
    kind: optional_conditional
    source: data
    ask: "Should short contigs be skipped - and does the assembly have enough of them to matter?"
    default: "no minimum"

  - id: D8
    param: cpus
    kind: never_ask
    source: data
    reason: "Affects runtime only, not the annotation"
    default: "min(8, available_cores)"

D3 changes how genes are predicted rather than how they are named. Prodigal trains its model on the input; a metagenomic bin holding more than one organism trains a blended model, and the resulting gene boundaries are wrong in a way that no downstream step flags.

Quick Start

# Annotate a bacterial genome assembly — results in results/ directory
prokka genome.fasta \
    --outdir results/ \
    --prefix sample1 \
    --kingdom Bacteria \
    --cpus 4

# Check output summary
cat results/sample1.txt
# Organism: Genus species strain
# Contigs: 1
# Bases: 4639675
# CDS: 4140
# rRNA: 22
# tRNA: 86

echo "Annotation complete. Key output files:"
ls results/sample1.{gff,gbk,faa,ffn,tsv}

Workflow

Step 1: Install and Verify Prokka

Install Prokka and confirm all dependent tools are accessible in the current environment.

# Create a dedicated conda environment
conda create -n prokka_env -c conda-forge -c bioconda prokka python=3.10 -y
conda activate prokka_env

# Verify Prokka version and all tool dependencies
prokka --version
# prokka 1.14.6

prokka --depends
# Checking that required tools are installed...
# OK: makeblastdb is installed (2.13.0+)
# OK: blastp is installed (2.13.0+)
# OK: hmmscan is installed (3.3.2)
# OK: prodigal is installed (2.6.3)
# OK: barrnap is installed (0.9)

# Check available genus-specific databases bundled with Prokka
ls $(conda info --base)/envs/prokka_env/db/genus/
# Archaea  Bacteria  Mitochondria  Viruses

# Install Python parsing tools
pip install biopython pandas matplotlib

Step 2: Prepare the Input Genome

Clean and rename contigs to comply with Prokka's header requirements before annotation.

from Bio import SeqIO
import re

# Load and inspect assembly
input_fasta = "genome.fasta"
records = list(SeqIO.parse(input_fasta, "fasta"))
print(f"Input assembly: {len(records)} contigs")
total_bases = sum(len(r) for r in records)
print(f"Total bases: {total_bases:,}")
print(f"Largest contig: {max(len(r) for r in records):,} bp")
print(f"N50 approx: see assembly stats tool")

# Rename contigs to short IDs compatible with Prokka (max 37 chars)
# Prokka requires: no spaces, no special characters in header
cleaned = []
for i, rec in enumerate(records, 1):
    new_id = f"contig_{i:04d}"
    new_rec = rec.__class__(rec.seq, id=new_id, description=f"len={len(rec.seq)}")
    cleaned.append(new_rec)

SeqIO.write(cleaned, "genome_clean.fasta", "fasta")
print(f"\nWrote genome_clean.fasta with {len(cleaned)} renamed contigs")
# genome_clean.fasta: contig_0001 through contig_NNNN
# Alternatively, clean headers with a simple bash one-liner
awk '/^>/{print ">contig_" ++i; next}{print}' genome.fasta > genome_clean.fasta

# Filter out short contigs (< 200 bp) to reduce annotation noise
awk '/^>/{header=$0; next} length($0) >= 200 {print header; print}' \
    genome_clean.fasta > genome_filtered.fasta

echo "Filtered assembly ready: $(grep -c '>' genome_filtered.fasta) contigs"

Step 3: Run Basic Prokka Annotation

Run Prokka with standard options for a bacterial genome, specifying genus/species for database selection.

# Basic annotation with genus/species hint (uses genus-specific protein database first)
prokka genome_clean.fasta \
    --outdir annotation/ \
    --prefix E_coli_K12 \
    --kingdom Bacteria \
    --genus Escherichia \
    --species coli \
    --strain K12 \
    --cpus 8 \
    --mincontiglen 200

# Expected runtime: 2–10 minutes for a typical 4–6 Mb bacterial genome

echo "Prokka annotation output files:"
ls annotation/
# E_coli_K12.err   E_coli_K12.faa   E_coli_K12.ffn
# E_coli_K12.fna   E_coli_K12.gbk   E_coli_K12.gff
# E_coli_K12.log   E_coli_K12.sqn   E_coli_K12.tbl
# E_coli_K12.tsv   E_coli_K12.txt

Step 4: Parse Annotation Summary (TSV)

Load the TSV output for a quick overview of annotated features and their functional assignments.

import pandas as pd

# Load the annotation TSV (tab-delimited feature table)
tsv_file = "annotation/E_coli_K12.tsv"
df = pd.read_csv(tsv_file, sep="\t")
print(f"Total features: {len(df)}")
print(f"Columns: {list(df.columns)}")
# Columns: [locus_tag, ftype, length_bp, gene, EC_number, COG, product]

# Feature type summary
print("\nFeature type counts:")
print(df["ftype"].value_counts().to_string())
# CDS     4140
# tRNA      86
# rRNA      22
# tmRNA      1

# Functional gene annotations (non-hypothetical CDS)
cds_df = df[df["ftype"] == "CDS"].copy()
hypothetical = cds_df["product"].str.contains("hypothetical", case=False, na=True)
print(f"\nCDS with known function: {(~hypothetical).sum()}")
print(f"Hypothetical proteins: {hypothetical.sum()}")

# Genes with EC numbers (enzymes)
ec_annotated = cds_df[cds_df["EC_number"].notna() & (cds_df["EC_number"] != "")]
print(f"CDS with EC numbers: {len(ec_annotated)}")
print(ec_annotated[["locus_tag", "gene", "EC_number", "product"]].head(5).to_string(index=False))

Step 5: Parse GenBank Output with BioPython

Read the GenBank file to access per-gene sequences, qualifiers, and feature coordinates.

from Bio import SeqIO
import pandas as pd

# Parse GenBank file
gbk_file = "annotation/E_coli_K12.gbk"
records = list(SeqIO.parse(gbk_file, "genbank"))
print(f"Contigs in GenBank: {len(records)}")

# Iterate over CDS features and extract details
rows = []
for rec in records:
    for feat in rec.features:
        if feat.type != "CDS":
            continue
        qualifiers = feat.qualifiers
        rows.append({
            "contig":      rec.id,
            "locus_tag":   qualifiers.get("locus_tag", ["?"])[0],
            "gene":        qualifiers.get("gene", [""])[0],
            "product":     qualifiers.get("product", ["hypothetical protein"])[0],
            "EC_number":   qualifiers.get("EC_number", [""])[0],
            "protein_id":  qualifiers.get("protein_id", [""])[0],
            "start":       int(feat.location.start),
            "end":         int(feat.location.end),
            "strand":      feat.location.strand,
            "aa_length":   len(qualifiers.get("translation", [""])[0]),
        })

features_df = pd.DataFrame(rows)
print(f"CDS features extracted: {len(features_df)}")
print(features_df.head(3).to_string(index=False))

# Retrieve protein sequence for a specific gene
gene_name = "dnaA"
gene_feat = features_df[features_df["gene"] == gene_name]
if not gene_feat.empty:
    idx = gene_feat.index[0]
    print(f"\n{gene_name}: {features_df.loc[idx, 'aa_length']} aa")

Step 6: Visualize Annotation Statistics

Generate a summary barplot of feature types and functional annotation coverage.

import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches

tsv_file = "annotation/E_coli_K12.tsv"
df = pd.read_csv(tsv_file, sep="\t")

cds = df[df["ftype"] == "CDS"].copy()
n_cds = len(cds)
n_known = (~cds["product"].str.contains("hypothetical", case=False, na=True)).sum()
n_hypo = n_cds - n_known
n_ec = cds["EC_number"].notna().sum()
n_rrna = (df["ftype"] == "rRNA").sum()
n_trna = (df["ftype"] == "tRNA").sum()

fig, axes = plt.subplots(1, 2, figsize=(11, 4))

# Left panel: feature type counts
types = df["ftype"].value_counts()
colors_left = ["#2980B9", "#27AE60", "#E74C3C", "#F39C12"] + ["#95A5A6"] * len(types)
axes[0].bar(types.index, types.values, color=colors_left[:len(types)], edgecolor="white")
axes[0].set_title("Annotated Feature Counts")
axes[0].set_xlabel("Feature Type")
axes[0].set_ylabel("Count")
for i, (label, val) in enumerate(types.items()):
    axes[0].text(i, val + 5, str(val), ha="center", fontsize=9)

# Right panel: CDS functional annotation breakdown
labels = ["Known function", "Hypothetical protein", "Enzyme (EC number)"]
sizes  = [n_known, n_hypo, n_ec]
colors_right = ["#2980B9", "#BDC3C7", "#E74C3C"]
bars = axes[1].bar(labels, sizes, color=colors_right, edgecolor="white")
axes[1].set_title(f"CDS Functional Annotation\n(n={n_cds} total CDS)")
axes[1].set_ylabel("Count")
axes[1].tick_params(axis="x", rotation=15)
for bar, val in zip(bars, sizes):
    axes[1].text(bar.get_x() + bar.get_width() / 2,
                 val + 10, str(val), ha="center", fontsize=9)

plt.suptitle("Prokka Genome Annotation Summary", fontsize=12, fontweight="bold")
plt.tight_layout()
plt.savefig("prokka_annotation_summary.png", dpi=150, bbox_inches="tight")
print(f"Saved prokka_annotation_summary.png")
print(f"CDS: {n_cds}  |  Known function: {n_known}  |  Hypothetical: {n_hypo}")
print(f"rRNA: {n_rrna}  |  tRNA: {n_trna}  |  EC-annotated CDS: {n_ec}")

Step 7: Batch Annotation Across Multiple Genomes

Annotate multiple genome assemblies sequentially and collect summary statistics.

#!/bin/bash
# batch_prokka.sh — annotate all FASTA files in a directory

INPUT_DIR="genomes/"
OUTPUT_DIR="annotations/"
mkdir -p "$OUTPUT_DIR"

for FASTA in "$INPUT_DIR"/*.fasta; do
    SAMPLE=$(basename "$FASTA" .fasta)
    echo "Annotating: $SAMPLE"
    prokka "$FASTA" \
        --outdir "${OUTPUT_DIR}/${SAMPLE}" \
        --prefix "$SAMPLE" \
        --kingdom Bacteria \
        --cpus 4 \
        --mincontiglen 200 \
        --quiet
    echo "  Done: ${OUTPUT_DIR}/${SAMPLE}/${SAMPLE}.txt"
done

echo "Batch annotation complete."
# Collect summary statistics from all Prokka .txt files
from pathlib import Path
import pandas as pd

annotation_dir = Path("annotations/")
rows = []

for txt_file in sorted(annotation_dir.glob("*/*.txt")):
    sample = txt_file.stem
    stats = {}
    with open(txt_file) as f:
        for line in f:
            line = line.strip()
            if ": " in line:
                key, val = line.split(": ", 1)
                stats[key.strip()] = val.strip()
    rows.append({
        "sample":   sample,
        "contigs":  int(stats.get("Contigs", 0)),
        "bases":    int(stats.get("Bases", 0)),
        "CDS":      int(stats.get("CDS", 0)),
        "rRNA":     int(stats.get("rRNA", 0)),
        "tRNA":     int(stats.get("tRNA", 0)),
    })

summary_df = pd.DataFrame(rows)
print(f"Annotated genomes: {len(summary_df)}")
print(summary_df.to_string(index=False))
summary_df.to_csv("batch_annotation_summary.csv", index=False)
print("\nSaved: batch_annotation_summary.csv")

Step 8: Compare Annotations Between Strains

Use the protein FASTA outputs for cross-strain comparison with identity-based clustering.

from Bio import SeqIO, pairwise2
import pandas as pd

# Load protein sequences from two strains
def load_proteins(faa_file):
    """Return dict of locus_tag -> protein sequence."""
    return {rec.id: str(rec.seq)
            for rec in SeqIO.parse(faa_file, "fasta")}

strain_a = load_proteins("annotation_A/strain_A.faa")
strain_b = load_proteins("annotation_B/strain_B.faa")

print(f"Strain A proteins: {len(strain_a)}")
print(f"Strain B proteins: {len(strain_b)}")

# Compare gene counts and size distributions
import matplotlib.pyplot as plt
import numpy as np

len_a = [len(seq) for seq in strain_a.values()]
len_b = [len(seq) for seq in strain_b.values()]

fig, ax = plt.subplots(figsize=(8, 4))
bins = np.linspace(0, 1500, 50)
ax.hist(len_a, bins=bins, alpha=0.6, label=f"Strain A (n={len(len_a)})", color="#2980B9")
ax.hist(len_b, bins=bins, alpha=0.6, label=f"Strain B (n={len(len_b)})", color="#E74C3C")
ax.set_xlabel("Protein Length (aa)")
ax.set_ylabel("Count")
ax.set_title("Protein Length Distribution by Strain")
ax.legend()
plt.tight_layout()
plt.savefig("strain_protein_length_comparison.png", dpi=150, bbox_inches="tight")
print("Saved strain_protein_length_comparison.png")

# Find proteins unique to each strain by size/count difference
print(f"\nSize difference: {abs(len(strain_a) - len(strain_b))} proteins")
print("→ Use Roary or OrthoFinder for formal pan-genome analysis")

Key Parameters

ParameterDefaultRange / OptionsEffect
--kingdomBacteriaBacteria, Archaea, Viruses, MitochondriaSelects gene prediction model and default databases
--genus—any genus name stringPrioritizes genus-specific protein database for CDS annotation
--species—any species name stringCombined with --genus for locus_tag prefix and organism metadata
--strain—any stringAdded to organism metadata in GenBank output
--proteins—path to FASTA fileCustom protein database prepended before RefSeq search
--hmms—path to HMM fileAdditional custom HMM database for specialized annotation
--evalue1e-61e-4–1e-9E-value cutoff for BLAST and HMMER hits
--cpus81–CPU countParallel processes for BLAST and HMMER searches
--mincontiglen1any integerSkip contigs shorter than this length (bp)
--metagenomeoffflagDisables Prodigal training on this genome (for MAGs)
--rfamoffflagEnable Infernal rRNA/ncRNA search against Rfam (slower)
--norrnaoffflagSkip rRNA prediction (use when assembly has no rRNA genes)
--notrnaoffflagSkip tRNA prediction

Common Recipes

Recipe: Annotation with Custom Protein Database

When to use: You have closely related reference proteins (e.g., characterized isolate) to improve annotation accuracy.

# Use a custom protein database to enhance annotation of a novel strain
# Custom proteins are searched first, before internal Prokka databases
prokka genome.fasta \
    --proteins reference_proteins.faa \
    --outdir custom_annotation/ \
    --prefix novel_strain \
    --kingdom Bacteria \
    --cpus 4

echo "Custom DB annotation complete:"
grep "CDS" custom_annotation/novel_strain.txt

Recipe: Metagenome-Assembled Genome (MAG) Annotation

When to use: Annotating a MAG where Prodigal cannot train on the full genome sequence.

# --metagenome disables Prodigal model training (uses meta mode)
# --mincontiglen 500 discards short, potentially chimeric contigs
prokka MAG_bin_42.fasta \
    --metagenome \
    --kingdom Bacteria \
    --outdir mag_annotation/ \
    --prefix MAG_bin_42 \
    --mincontiglen 500 \
    --cpus 4

echo "MAG annotation complete:"
cat mag_annotation/MAG_bin_42.txt

Recipe: Extract Specific Gene Sequences

When to use: Retrieve nucleotide or protein sequences for a target gene or pathway from the annotation.

from Bio import SeqIO

# Extract all sequences for a specific gene name from protein FASTA
faa_file = "annotation/E_coli_K12.faa"
target_gene = "dnaA"

matches = []
for rec in SeqIO.parse(faa_file, "fasta"):
    # Prokka FASTA header: >locus_tag gene product
    if target_gene in rec.description:
        matches.append(rec)

print(f"Proteins matching '{target_gene}': {len(matches)}")
for rec in matches:
    print(f"  {rec.id}: {len(rec.seq)} aa — {rec.description}")

# Save matching sequences
if matches:
    SeqIO.write(matches, f"{target_gene}_proteins.faa", "fasta")
    print(f"Saved: {target_gene}_proteins.faa")

Recipe: Convert GFF to Pandas DataFrame

When to use: Work with genomic coordinates for feature overlap analysis or visualization.

import pandas as pd

def parse_gff(gff_file):
    """Parse Prokka GFF3 file into a DataFrame (skips sequence section)."""
    rows = []
    with open(gff_file) as f:
        for line in f:
            if line.startswith("##FASTA"):
                break
            if line.startswith("#") or not line.strip():
                continue
            parts = line.rstrip("\n").split("\t")
            if len(parts) < 9:
                continue
            attr_dict = {}
            for item in parts[8].split(";"):
                if "=" in item:
                    k, v = item.split("=", 1)
                    attr_dict[k] = v
            rows.append({
                "seqname":  parts[0],
                "source":   parts[1],
                "feature":  parts[2],
                "start":    int(parts[3]),
                "end":      int(parts[4]),
                "score":    parts[5],
                "strand":   parts[6],
                "frame":    parts[7],
                "locus_tag": attr_dict.get("ID", ""),
                "gene":     attr_dict.get("gene", ""),
                "product":  attr_dict.get("product", ""),
            })
    return pd.DataFrame(rows)

gff_df = parse_gff("annotation/E_coli_K12.gff")
print(f"GFF features: {len(gff_df)}")
print(gff_df[gff_df["feature"] == "CDS"].head(5)[
    ["seqname", "start", "end", "strand", "gene", "product"]
].to_string(index=False))

Expected Outputs

Output FileFormatDescription
{prefix}.gffGFF3Genome annotation with feature coordinates and attributes; includes FASTA sequence at the end
{prefix}.gbkGenBankFull GenBank-format annotation for use in Geneious, BioPython, and submission prep
{prefix}.faaFASTAPredicted protein sequences for all CDS features
{prefix}.ffnFASTANucleotide sequences for all annotated features (CDS, rRNA, tRNA)
{prefix}.fnaFASTANucleotide FASTA of the complete assembly (contigs)
{prefix}.tsvTSVTab-delimited summary table: locus_tag, ftype, length_bp, gene, EC_number, COG, product
{prefix}.txtTextOne-line summary counts: Contigs, Bases, CDS, rRNA, tRNA, tmRNA
{prefix}.tblTBLFeature table format for tbl2asn GenBank submission
{prefix}.sqnSQNASN.1 format for NCBI submission (generated by tbl2asn)
{prefix}.errTextWarnings and errors from tbl2asn validation step
{prefix}.logTextFull Prokka run log with timing per step

Troubleshooting

ProblemCauseSolution
FATAL: Can't find any tRNA genesNo tRNA detected on short or highly fragmented assemblyAdd --notrna flag; check if assembly is too fragmented (N50 < 1 kb)
Can't exec "makeblastdb"BLAST+ not on PATHconda install -c bioconda blast; ensure prokka env is activated
Contig ID too long (>37 chars)Assembler produced long contig headersPre-process FASTA to shorten headers: awk '/^>/{print ">contig_"++i; next}{print}'
Very high hypothetical protein rate (>60%)Divergent organism with few database matchesAdd --proteins with closely related strain FAA; consider --genus flag
ERROR: Argument --proteins: file does not existPath to custom protein file is incorrectUse absolute path; verify file exists with ls -la custom.faa
tbl2asn error: multiple /productDuplicate product qualifiers in annotationIgnore if exporting for local use; for NCBI submission, use PGAP instead
Annotation is very slow (>30 min for ~5 Mb)--cpus not set or set to 1; --rfam enabledSet --cpus to available thread count; disable --rfam for faster runs
rRNA count is 0 for complete genome--norrna flag was set, or barrnap threshold too strictRemove --norrna; check barrnap is installed with barrnap --version

References

Roary Pan-Genome Pipeline

Overview

Roary is a high-throughput pan-genome pipeline for prokaryotes that takes per-sample GFF3 annotations (typically from Prokka or Bakta) and produces a clustered gene presence/absence matrix across the entire input set. It first reduces redundancy with CD-HIT iterative clustering, then performs an all-vs-all BLASTP within each pre-cluster, and finally applies MCL graph clustering to define orthologous gene families. The output partitions the gene space into core (≥ 99 %), soft-core (95–99 %), shell (15–95 %), and cloud (< 15 %) genes and optionally builds a concatenated core-gene alignment suitable for phylogenetic inference.

When to Use

  • Computing a pan-genome from a set of bacterial isolate annotations (10–10,000 genomes)
  • Producing a gene_presence_absence.csv matrix for downstream GWAS, accessory-gene mining, or core-gene phylogenetics
  • Building a concatenated core-gene multi-FASTA alignment for ML/Bayesian phylogenetic trees
  • Generating a pan-genome reference FASTA to use as a non-redundant gene catalog
  • Comparative genomics across closely related strains where >95 % nucleotide identity is expected
  • Use Panaroo instead when assemblies are highly fragmented or annotations are noisy (Panaroo aggressively cleans annotation errors)
  • Use PIRATE instead when paralog-aware clustering with multiple identity thresholds is needed
  • Use PPanGGOLiN instead when graph-based, statistically grounded gene-family partitioning is preferred over fixed-frequency cutoffs

Prerequisites

  • Software: Roary ≥ 3.13, Perl 5, BLAST+, CD-HIT, MCL, BEDTools, PRANK or MAFFT (for -e core alignment), FastTree (optional)
  • Python packages (for output parsing): pandas, matplotlib, seaborn, biopython, dendropy
  • Input: per-sample GFF3 files with embedded FASTA at the end (Prokka/Bakta default output)
  • Hardware: ≥ 8 GB RAM for ~50 genomes; 32 GB+ recommended for ~500 genomes
  • Naming: each GFF3 filename becomes the sample column header in the output matrix; use sample_id.gff style

Check before installing: The tool may already be available in the current environment (e.g., inside a pixi / conda env). Run command -v roary first and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool via pixi run roary rather than bare roary.

# Install Roary via conda/mamba (recommended)
mamba install -c conda-forge -c bioconda roary

# Verify installation
roary --version
# 3.13.0

# Verify dependent tools
which cd-hit blastp mcl bedtools mafft
# /opt/conda/bin/cd-hit
# /opt/conda/bin/blastp
# /opt/conda/bin/mcl
# /opt/conda/bin/bedtools
# /opt/conda/bin/mafft

# Install Python parsing dependencies
pip install pandas matplotlib seaborn biopython dendropy

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: inputAnnotations
    kind: derived
    source: upstream
    ask: "Which GFF3 files, and were they all produced by the same annotation tool and version?"
    default: "the annotation stage output"

  - id: D2
    param: identityThreshold
    kind: required
    source: user
    ask: "How similar must two proteins be to count as the same gene? Lower values merge diverged orthologs; higher values split them."
    default: "95%"

  - id: D3
    param: coreDefinition
    kind: required
    source: user
    ask: "In what fraction of isolates must a gene appear before it counts as core?"
    default: "99%"

  - id: D4
    param: paralogSplitting
    kind: required
    source: user
    ask: "Should duplicated genes within a genome be split into separate families, or kept together?"
    default: "split"

  - id: D5
    param: coreAlignment
    kind: required
    source: upstream
    ask: "Is a concatenated core-gene alignment needed for a downstream phylogeny?"
    default: "not produced - it is the slowest stage by a wide margin"

  - id: D6
    param: alignmentSpeed
    kind: optional
    source: user
    depends_on: [D5]
    ask: "Align every gene accurately, or use the fast route sufficient for tree building?"
    default: "fast alignment"
    skip_if: "no core alignment requested"

  - id: D7
    param: clusterInflation
    kind: optional
    source: user
    ask: "How tightly should the clustering step group families?"
    default: 1.5

  - id: D8
    param: maxClusters
    kind: optional_conditional
    source: data
    ask: "Is the expected number of gene families above the default cap?"
    default: 50000

  - id: D9
    param: processes
    kind: never_ask
    source: data
    reason: "Affects runtime only, not the gene families"
    default: "min(8, available_cores)"

D1 is derived and carries a consistency requirement rather than a choice: Roary compares annotations, so GFF3 files from different annotation tools or database versions produce apparent presence/absence differences that are annotation artefacts, not biology. D2 and D3 together decide the size of the core genome, which is usually the headline number of the analysis.

Quick Start

# Run Roary on all GFF3 files in current directory; emit core gene alignment
roary -e --mafft -p 8 -o pangenome -f roary_out/ *.gff

# Inspect summary statistics
cat roary_out/summary_statistics.txt
# Core genes        (99% <= strains <= 100%) 2823
# Soft core genes   (95% <= strains <  99%)   78
# Shell genes       (15% <= strains <  95%)  1542
# Cloud genes       ( 0% <= strains <  15%)  3104
# Total genes                                 7547

Workflow

Step 1: Install Roary and Verify Dependencies

Install Roary in a dedicated environment to avoid Perl module conflicts.

# Create a dedicated conda environment
mamba create -n roary_env -c conda-forge -c bioconda roary mafft fasttree python=3.11 -y
mamba activate roary_env

# Verify Roary
roary --version
# 3.13.0

# Confirm pipeline dependencies are reachable
for tool in cd-hit blastp mcl bedtools mafft FastTree; do
    if command -v $tool >/dev/null; then
        echo "OK: $tool"
    else
        echo "MISSING: $tool"
    fi
done

# Install Python parsing tools
pip install pandas matplotlib seaborn biopython dendropy

Step 2: Prepare Per-Sample GFF3 Files from Prokka or Bakta

Roary requires GFF3 files with an embedded FASTA section (the ##FASTA block). Both Prokka .gff and Bakta .gff3 outputs satisfy this format.

# Verify GFF3 files include the embedded FASTA section
mkdir -p roary_input/
cp annotations/*/sample*.gff roary_input/

for GFF in roary_input/*.gff; do
    if grep -q "^##FASTA" "$GFF"; then
        echo "OK: $GFF"
    else
        echo "MISSING ##FASTA in: $GFF"
    fi
done

# Roary uses GFF filename (without .gff) as the sample column name.
# Rename or symlink to ensure unique, descriptive names.
cd roary_input/
ls *.gff | head
# strain_A.gff  strain_B.gff  strain_C.gff
# Sanity-check sample size and unique names
from pathlib import Path

gff_files = sorted(Path("roary_input/").glob("*.gff"))
print(f"GFF files: {len(gff_files)}")

names = [g.stem for g in gff_files]
duplicates = [n for n in names if names.count(n) > 1]
assert not duplicates, f"Duplicate sample names: {set(duplicates)}"

# Show first 5
for g in gff_files[:5]:
    size_mb = g.stat().st_size / 1e6
    print(f"  {g.name}: {size_mb:.1f} MB")

Step 3: Run Roary with Core Gene Alignment

The -e --mafft combination produces the concatenated core-gene alignment used downstream for phylogenetics.

# Run Roary on the prepared GFF directory
# -e: extract core gene alignment per gene, then concatenate
# --mafft: use MAFFT for alignment (faster than default PRANK)
# -p 8: 8 parallel processes
# -i 95: min BLASTP percentage identity (default 95)
# -cd 99: % isolates a gene must be in to be "core" (default 99)
roary -e --mafft \
    -p 8 \
    -i 95 \
    -cd 99 \
    -o pangenome \
    -f roary_out/ \
    roary_input/*.gff

# Expected runtime:
#   ~50 genomes:  10–20 min
#   ~500 genomes: 4–8 hours

ls roary_out/
# accessory_binary_genes.fa            gene_presence_absence.csv
# accessory_binary_genes.fa.newick     gene_presence_absence.Rtab
# accessory.header.embl                number_of_conserved_genes.Rtab
# accessory.tab                        number_of_genes_in_pan_genome.Rtab
# blast_identity_frequency.Rtab        number_of_new_genes.Rtab
# clustered_proteins                   number_of_unique_genes.Rtab
# core_accessory.header.embl           pan_genome_reference.fa
# core_accessory.tab                   summary_statistics.txt
# core_gene_alignment.aln

Step 4: Parse the Gene Presence/Absence Matrix

The CSV output is the canonical pan-genome matrix: rows are gene families, columns include metadata and one column per sample (filled with locus tags or empty).

import pandas as pd
import numpy as np

df = pd.read_csv("roary_out/gene_presence_absence.csv", low_memory=False)
print(f"Gene families: {len(df)}")
print(f"Columns: {len(df.columns)}")
# Standard metadata columns end with 'Inference', then sample columns follow
metadata_cols = list(df.columns[:14])
sample_cols = list(df.columns[14:])
print(f"Samples: {len(sample_cols)}")

# Build a binary presence/absence matrix (1 = locus tag present, 0 = empty)
binary = df[sample_cols].notna().astype(int)
binary.index = df["Gene"]
print(f"Binary matrix shape: {binary.shape}")

# Pan-genome partition by frequency
n_samples = len(sample_cols)
freq = binary.sum(axis=1) / n_samples
core      = freq[freq >= 0.99]
soft_core = freq[(freq >= 0.95) & (freq < 0.99)]
shell     = freq[(freq >= 0.15) & (freq < 0.95)]
cloud     = freq[freq < 0.15]

print(f"\nPan-genome partition (n={n_samples} genomes):")
print(f"  Core      (>=99 %): {len(core):>5}")
print(f"  Soft-core (95–99 %): {len(soft_core):>5}")
print(f"  Shell    (15–95 %): {len(shell):>5}")
print(f"  Cloud      (<15 %): {len(cloud):>5}")
print(f"  Total            : {len(freq):>5}")

Step 5: Visualize the Pan-Genome Frequency Distribution

A frequency histogram exposes whether the input set has a U-shaped (open) or core-skewed (closed) pan-genome.

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

df = pd.read_csv("roary_out/gene_presence_absence.csv", low_memory=False)
sample_cols = list(df.columns[14:])
n = len(sample_cols)
binary = df[sample_cols].notna().astype(int)
freq = binary.sum(axis=1)

fig, axes = plt.subplots(1, 2, figsize=(11, 4))

# Histogram of gene frequencies
axes[0].hist(freq, bins=range(1, n + 2), color="#2980B9", edgecolor="white")
axes[0].axvline(0.99 * n, color="red", linestyle="--", label="99 % core cutoff")
axes[0].axvline(0.15 * n, color="orange", linestyle="--", label="15 % shell cutoff")
axes[0].set_xlabel("Number of genomes containing the gene")
axes[0].set_ylabel("Number of gene families")
axes[0].set_title("Pan-genome frequency distribution")
axes[0].legend()

# Pie chart of pan-genome partition
freqp = freq / n
core_n      = (freqp >= 0.99).sum()
soft_n      = ((freqp >= 0.95) & (freqp < 0.99)).sum()
shell_n     = ((freqp >= 0.15) & (freqp < 0.95)).sum()
cloud_n     = (freqp < 0.15).sum()
axes[1].pie([core_n, soft_n, shell_n, cloud_n],
            labels=[f"Core ({core_n})", f"Soft-core ({soft_n})",
                    f"Shell ({shell_n})", f"Cloud ({cloud_n})"],
            colors=["#27AE60", "#3498DB", "#F39C12", "#95A5A6"],
            autopct="%1.1f%%", startangle=90)
axes[1].set_title(f"Pan-genome partition (n={n} genomes)")

plt.tight_layout()
plt.savefig("pangenome_distribution.png", dpi=150, bbox_inches="tight")
print(f"Saved pangenome_distribution.png  (total families: {len(freq)})")

Step 6: Build a Phylogenetic Tree from the Core Gene Alignment

Use FastTree on the concatenated core-gene alignment to recover strain relationships.

# FastTree expects an aligned multi-FASTA; Roary produces this with `-e`
FastTree -nt -gtr -nosupport \
    < roary_out/core_gene_alignment.aln \
    > roary_out/core_gene_tree.nwk

echo "Tree built: roary_out/core_gene_tree.nwk"
# (Optional) IQ-TREE for support values:
# iqtree -s core_gene_alignment.aln -m GTR+G -bb 1000 -nt 8
# Visualize the FastTree result alongside the gene presence/absence matrix
import dendropy
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

tree = dendropy.Tree.get(path="roary_out/core_gene_tree.nwk", schema="newick")
leaves = [leaf.taxon.label for leaf in tree.leaf_node_iter()]
print(f"Tree leaves: {len(leaves)}")
print(f"Tree length: {tree.length():.4f} substitutions/site")

# Print first 10 leaves with their root-to-tip distances
for leaf in list(tree.leaf_node_iter())[:10]:
    dist = leaf.distance_from_root()
    print(f"  {leaf.taxon.label:<25}  d={dist:.4f}")

Step 7: Produce Roary's Built-in Summary Plots

Roary ships with a roary_plots.py companion that builds a tree-anchored matrix figure.

# Use the bundled python helper from the Roary install
# (download from: https://github.com/sanger-pathogens/Roary/blob/master/contrib/roary_plots/roary_plots.py)
python roary_plots.py \
    roary_out/core_gene_tree.nwk \
    roary_out/gene_presence_absence.csv \
    --format png \
    --labels

ls *.png
# pangenome_frequency.png
# pangenome_matrix.png
# pangenome_pie.png

Step 8: Compute Per-Genome Accessory Gene Counts

Identify which genomes carry the most unique (cloud) genes — useful for strain-specific gene investigation.

import pandas as pd

df = pd.read_csv("roary_out/gene_presence_absence.csv", low_memory=False)
sample_cols = list(df.columns[14:])
n = len(sample_cols)
binary = df[sample_cols].notna().astype(int)
binary.index = df["Gene"]

freq = binary.sum(axis=1)
cloud_mask = freq < (0.15 * n)
shell_mask = (freq >= 0.15 * n) & (freq < 0.95 * n)

per_genome = pd.DataFrame({
    "genes_total":   binary.sum(axis=0),
    "cloud_genes":   binary.loc[cloud_mask].sum(axis=0),
    "shell_genes":   binary.loc[shell_mask].sum(axis=0),
})
per_genome = per_genome.sort_values("cloud_genes", ascending=False)
print(per_genome.head(10).to_string())

per_genome.to_csv("per_genome_accessory_counts.csv")
print("\nSaved per_genome_accessory_counts.csv")

Key Parameters

ParameterDefaultRange / OptionsEffect
-i9570–99 (% identity)Minimum BLASTP percentage identity for clustering
-cd9990–100 (%)Minimum % of isolates a gene must be in to be called "core"
-p11–CPU countNumber of parallel processes for BLAST and MCL stages
-eoffflagExtract concatenated core gene multi-FASTA alignment (per-gene MSA + concat)
--mafftoffflag (with -e)Use MAFFT for alignment (faster than default PRANK)
-soffflagDo not split paralogs into separate gene families
-noffflagUse fast core gene alignment with MAFFT (skips per-gene PRANK)
-g50000any integerMaximum number of clusters expected (cap on output size)
-iv1.51.2–5.0MCL inflation value; higher = tighter clusters
-roffflagGenerate R plots (presence/absence heatmap, pan-genome curves)
-oclustered_proteinsstringOutput prefix for clustered proteins file
-f.pathOutput directory
-zoffflagDon't delete intermediate files (useful for debugging)

Common Recipes

Recipe: Core Gene Alignment from Existing Roary Output

When to use: You already ran Roary without -e and now need a phylogenetic alignment.

# Re-run only the alignment step using the extracted core genes
# query_pan_genome from the Roary suite extracts gene-family multi-FASTAs
query_pan_genome -a intersection \
    -g roary_out/clustered_proteins \
    -o core_gene_list.txt \
    roary_input/*.gff

echo "Core genes: $(wc -l < core_gene_list.txt)"
# Then re-run Roary with -e to materialize the alignment:
# roary -e --mafft -p 8 -f roary_out2/ roary_input/*.gff

Recipe: Lower Identity Threshold for Diverse Genus-Level Sets

When to use: Comparing genomes across multiple closely related species (e.g., genus-level pan-genome).

# Drop identity to 70 % to capture orthologs across species boundaries
roary -e --mafft \
    -p 8 \
    -i 70 \
    -cd 99 \
    -f roary_genus/ \
    roary_input/*.gff

cat roary_genus/summary_statistics.txt
# Expect a smaller core and a much larger shell + cloud

Recipe: Roary on a Subset of Strains

When to use: You want a pan-genome of a specific subclade without re-annotating all genomes.

# Stage GFF files for the subset
mkdir -p subset/
for SAMPLE in strain_A strain_B strain_E strain_K; do
    cp roary_input/${SAMPLE}.gff subset/
done

roary -e --mafft -p 4 -f roary_subset/ subset/*.gff
echo "Subset pan-genome:"
cat roary_subset/summary_statistics.txt

Recipe: Cross-Tabulate Accessory Genes Against Strain Metadata

When to use: You want to find genes enriched in a specific phenotype or geographic group.

import pandas as pd
from scipy.stats import fisher_exact

# Load presence/absence and strain metadata
pa = pd.read_csv("roary_out/gene_presence_absence.csv", low_memory=False)
sample_cols = list(pa.columns[14:])
binary = pa[sample_cols].notna().astype(int)
binary.index = pa["Gene"]
binary.columns = sample_cols

# Metadata: sample_id, phenotype (e.g., "resistant" / "susceptible")
meta = pd.read_csv("strain_metadata.csv").set_index("sample_id")
group_resistant = meta[meta["phenotype"] == "resistant"].index
group_susceptible = meta[meta["phenotype"] == "susceptible"].index
group_resistant = [g for g in group_resistant if g in binary.columns]
group_susceptible = [g for g in group_susceptible if g in binary.columns]

# Fisher's exact test per gene family
results = []
for gene, row in binary.iterrows():
    a = int(row[group_resistant].sum())
    b = len(group_resistant) - a
    c = int(row[group_susceptible].sum())
    d = len(group_susceptible) - c
    if a + c == 0 or a + c == len(group_resistant) + len(group_susceptible):
        continue  # skip invariant genes
    odds, pval = fisher_exact([[a, b], [c, d]])
    results.append({"gene": gene, "n_resistant": a, "n_susceptible": c,
                    "odds_ratio": odds, "pvalue": pval})

results_df = pd.DataFrame(results).sort_values("pvalue")
print(f"Gene-phenotype associations (top 10):")
print(results_df.head(10).to_string(index=False))
results_df.to_csv("phenotype_associated_genes.csv", index=False)

Expected Outputs

Output FileFormatDescription
gene_presence_absence.csvCSVPan-genome matrix; rows = gene families, columns = sample locus tags
gene_presence_absence.RtabTSVBinary 0/1 matrix in R-friendly format
summary_statistics.txtTextCounts of core / soft-core / shell / cloud / total gene families
pan_genome_reference.faFASTANon-redundant nucleotide reference of one representative per gene family
clustered_proteinsTextCluster definitions: cluster name → constituent locus tags
core_gene_alignment.alnFASTA (aligned)Concatenated multi-FASTA of core gene alignments (only with -e)
accessory_binary_genes.faFASTABinary 0/1 sequences of accessory presence (input to FastTree)
accessory_binary_genes.fa.newickNewickAccessory gene phylogeny (built from binary FASTA)
number_of_conserved_genes.RtabTSVRarefaction curve of core genes vs. genome count
number_of_genes_in_pan_genome.RtabTSVRarefaction curve of pan-genome growth
number_of_new_genes.RtabTSVNew genes added per genome
number_of_unique_genes.RtabTSVPer-genome unique gene counts
blast_identity_frequency.RtabTSVDistribution of BLAST identities used during clustering

Troubleshooting

ProblemCauseSolution
GFF file does not contain ##FASTAAnnotation file is not a Prokka/Bakta-style GFF3 with embedded sequencesRe-export from Prokka/Bakta; do not strip the FASTA tail
Could not run blastpBLAST+ missing from the conda envmamba install -c bioconda blast and re-activate the env
Roary produces 0 core genesOne sample has very different annotations or is misidentifiedInspect gene_presence_absence.csv; remove outlier with roary --check_input style sanity checks
Job runs out of memory on > 200 genomesDefault identity (95 %) creates many BLAST jobsIncrease -i to 90 if accuracy permits; split into batches; use Panaroo on RAM-tight machines
Paralogs split into many tiny clustersMCL inflation too high or duplicate locus tags across samplesAdd -s to keep paralogs together, or rerun Prokka/Bakta with unique --locus-tag per sample
FastTree: bad alignment on core_gene_alignment.alnAlignment built with PRANK was truncatedRe-run with --mafft for MAFFT-based alignment
Output column names look mangled or missingGFF filenames had spaces or special charactersRename input GFFs to [a-zA-Z0-9_]+.gff before running
Same genome counted twiceDuplicate GFF filenames (e.g., copied symlinks)Run `ls roary_input/*.gff
Pan-genome size grows linearly without saturationTruly open pan-genome (expected for many bacteria)This is biological, not a bug; report rarefaction curves and use Heaps' law for confirmation

References

Arboreto GRN Inference

Overview

Arboreto infers gene regulatory networks (GRNs) from gene expression data using parallelized tree-based regression. For each target gene, it trains a regression model with all other genes (or a specified TF list) as features and emits TF-target-importance triplets. It provides two interchangeable algorithms -- GRNBoost2 (gradient boosting, fast) and GENIE3 (Random Forest, classic) -- sharing identical input/output formats. Computation is Dask-parallelized, scaling from laptop cores to HPC clusters.

When to Use

  • Inferring transcription factor-to-target gene regulatory relationships from bulk RNA-seq expression data
  • Building gene regulatory networks from single-cell RNA-seq count matrices (cells as rows, genes as columns)
  • Generating the adjacency matrix (Step 1) of the pySCENIC regulatory analysis pipeline
  • Comparing regulatory network structure across experimental conditions (e.g., control vs treatment)
  • Producing consensus regulatory networks by running inference across multiple random seeds
  • Validating GRN results by comparing GRNBoost2 and GENIE3 outputs on the same dataset
  • For downstream regulon identification and activity scoring, use arboreto output with pySCENIC
  • For single-cell preprocessing (QC, normalization, clustering) before GRN inference, use scanpy-scrna-seq

Prerequisites

  • Python packages: arboreto, pandas, numpy, dask, distributed, scikit-learn, scipy
  • Data requirements: Gene expression matrix (genes as columns, observations as rows) in TSV/CSV; optionally a TF list file (one gene name per line)
  • Environment: Python 3.8+; optional networkx, matplotlib for visualization
pip install arboreto distributed networkx matplotlib

Pre-flight Interview

Settle these with the user before writing any analysis code.

decisions:
  - id: D1
    param: algorithm
    kind: required
    source: user
    ask: "Infer the network with gradient boosting (faster) or random forests (the original GENIE3 formulation)?"
    default: "GRNBoost2"

  - id: D2
    param: expressionMatrix
    kind: required
    source: upstream
    ask: "Which processed expression matrix, and which cells or samples within it, should the network be inferred from?"
    default: null

  - id: D3
    param: regulatorList
    kind: required
    source: literature
    ask: "Restrict candidate regulators to a curated transcription-factor list, or let every gene be a candidate regulator?"
    default: "a species-matched TF list"

  - id: D4
    param: edgeThreshold
    kind: required
    source: user
    ask: "The output ranks every regulator-target pair - how many of the top edges should be kept as the network?"
    default: null

  - id: D5
    param: seed
    kind: never_ask
    source: data
    reason: "Fixes the draw for reproducibility; does not change what the data supports"
    default: 0

  - id: D6
    param: daskClient
    kind: never_ask
    source: data
    reason: "Execution backend; affects runtime only"
    default: "local"

D3 is the decision that separates a usable network from a hairball. Leaving regulators unrestricted lets any correlated gene appear as a regulator, and the result still returns a fully populated ranked table. D4 has no safe default: the algorithm scores all pairs, so where the list is cut is the user's call about the network they intend to interpret.

Quick Start

Complete GRN inference in a single block. The if __name__ == '__main__': guard is required because Dask spawns worker processes via multiprocessing.

import pandas as pd
from arboreto.algo import grnboost2
from arboreto.utils import load_tf_names

if __name__ == '__main__':
    # Load expression matrix (observations x genes)
    expression_matrix = pd.read_csv('expression_data.tsv', sep='\t')
    tf_names = load_tf_names('tf_list.txt')  # optional TF filter

    # Infer GRN (uses all local cores by default)
    network = grnboost2(expression_data=expression_matrix,
                        tf_names=tf_names, seed=777)

    # Filter top links and save
    top_network = network[network['importance'] > 1.0]
    top_network.to_csv('grn_output.tsv', sep='\t', index=False, header=False)
    print(f"Inferred {len(network)} links, kept {len(top_network)} above threshold")
    # Example: Inferred 185432 links, kept 12876 above threshold

Workflow

Step 1: Load Expression Data

Arboreto accepts a pandas DataFrame (recommended) or NumPy array. Rows are observations (cells or samples), columns are genes. Gene names must be column headers.

import pandas as pd

# From TSV (genes as columns, observations as rows)
expression_matrix = pd.read_csv('expression_data.tsv', sep='\t')
print(f"Shape: {expression_matrix.shape}  "
      f"({expression_matrix.shape[0]} observations x {expression_matrix.shape[1]} genes)")
# Example: Shape: (5000, 18654)  (5000 observations x 18654 genes)

# From AnnData (e.g., after scanpy preprocessing)
import anndata as ad
adata = ad.read_h5ad('preprocessed.h5ad')
expression_matrix = pd.DataFrame(
    adata.X.toarray() if hasattr(adata.X, 'toarray') else adata.X,
    columns=adata.var_names.tolist()
)
print(f"Converted AnnData: {expression_matrix.shape}")
# Example: Converted AnnData: (5000, 18654)

Step 2: Load Transcription Factor List

Providing a TF list restricts regulators to known transcription factors, reducing computation time and improving biological relevance. If omitted, all genes are treated as potential regulators.

from arboreto.utils import load_tf_names

# From file (one TF name per line)
tf_names = load_tf_names('human_tfs.txt')
print(f"Loaded {len(tf_names)} transcription factors")
# Example: Loaded 1639 transcription factors

# Or define directly
tf_names = ['MYC', 'TP53', 'SOX2', 'NANOG', 'POU5F1']

# Verify TFs exist in expression matrix columns
tf_in_data = [tf for tf in tf_names if tf in expression_matrix.columns]
print(f"TFs found in expression data: {len(tf_in_data)}/{len(tf_names)}")
# Example: TFs found in expression data: 1583/1639

Step 3: Configure Dask Client (Optional)

By default arboreto creates an internal Dask client using all local cores. Create an explicit client for resource control, monitoring, or cluster deployment.

from distributed import LocalCluster, Client

# Custom local client with resource limits
local_cluster = LocalCluster(
    n_workers=8,
    threads_per_worker=1,   # avoid GIL contention in scikit-learn
    memory_limit='4GB'       # per worker
)
client = Client(local_cluster)
print(f"Dashboard: {client.dashboard_link}")
# Example: Dashboard: http://127.0.0.1:8787/status

Step 4: Run GRN Inference

Call grnboost2() (recommended) or genie3(). Both share the same signature and output format.

from arboreto.algo import grnboost2

if __name__ == '__main__':
    network = grnboost2(
        expression_data=expression_matrix,
        tf_names=tf_names,         # 'all' if no TF list
        client_or_address=client,  # omit to use default local scheduler
        seed=777,                  # for reproducibility
        verbose=True               # print progress
    )
    print(f"Inferred {len(network)} regulatory links")
    print(network.head())
    # Example:
    #       TF  target  importance
    # 0   MYC   CDK4       3.214
    # 1   MYC   CCND1      2.871
    # 2  TP53  CDKN1A      2.654

Step 5: Filter Results

Raw output contains links for every TF-target pair with non-zero importance. Filter to retain high-confidence regulatory interactions.

# Strategy 1: Importance threshold
threshold = 1.0
filtered = network[network['importance'] > threshold]
print(f"Threshold {threshold}: {len(filtered)} links "
      f"({len(filtered)/len(network)*100:.1f}% retained)")
# Example: Threshold 1.0: 12876 links (6.9% retained)

# Strategy 2: Top N links per target gene
top_n = 10
top_per_target = (network.groupby('target')
                  .apply(lambda g: g.nlargest(top_n, 'importance'))
                  .reset_index(drop=True))
print(f"Top {top_n} per target: {len(top_per_target)} links")
# Example: Top 10 per target: 186540 links

Step 6: Save Network

Save as a TSV file with three columns: TF, target, importance.

# Save full network
network.to_csv('full_network.tsv', sep='\t', index=False)
print(f"Saved full network: {len(network)} links")

# Save filtered network (without header for pySCENIC compatibility)
filtered.to_csv('filtered_network.tsv', sep='\t', index=False, header=False)
print(f"Saved filtered network: {len(filtered)} links")

# Clean up Dask client if explicitly created
client.close()
local_cluster.close()

Step 7: Visualize Results (Optional)

Plot the top regulatory interactions as a directed network graph.

import networkx as nx
import matplotlib.pyplot as plt

# Build directed graph from top links
top_links = network.nlargest(50, 'importance')
G = nx.from_pandas_edgelist(
    top_links, source='TF', target='target',
    edge_attr='importance', create_using=nx.DiGraph()
)

# Draw network
fig, ax = plt.subplots(figsize=(12, 10))
pos = nx.spring_layout(G, k=2, seed=42)
nx.draw_networkx_nodes(G, pos, node_size=300, node_color='lightblue', ax=ax)
nx.draw_networkx_labels(G, pos, font_size=7, ax=ax)
edges = nx.draw_networkx_edges(
    G, pos, edge_color=[G[u][v]['importance'] for u, v in G.edges()],
    edge_cmap=plt.cm.Reds, width=1.5, arrows=True, ax=ax
)
plt.colorbar(edges, ax=ax, label='Importance')
ax.set_title('Top 50 Regulatory Interactions')
plt.tight_layout()
plt.savefig('grn_network.png', dpi=300, bbox_inches='tight')
print("Saved grn_network.png")

Key Parameters

ParameterDefaultRange / OptionsEffect
expression_data(required)DataFrame or ndarrayExpression matrix, observations x genes
tf_names'all'list of str or 'all'Restrict regulators to known TFs; 'all' uses every gene
gene_namesNonelist of strRequired when expression_data is a NumPy array
client_or_address'local'Client, str, or 'local'Dask client instance or scheduler address
seedNoneintRandom seed for reproducibility; always set for replicable results
verboseFalseboolPrint progress messages during inference

Key Concepts

Algorithm Selection Guide

Both algorithms follow the same multiple-regression strategy (train one model per target gene, extract feature importances) but differ in the underlying regressor.

FeatureGRNBoost2GENIE3
MethodStochastic gradient boosting with early stoppingRandom Forest (or ExtraTrees)
SpeedFast -- optimized for large datasetsSlower -- higher per-model cost
MemoryLower (early stopping limits tree depth)Higher (full forests per target)
Best forDefault choice; 10k+ observations, single-cell scaleComparison with published GENIE3 results; validation
Importfrom arboreto.algo import grnboost2from arboreto.algo import genie3
Output formatTF, target, importance (identical)TF, target, importance (identical)

Decision rule: Start with GRNBoost2. Use GENIE3 only when reproducing published GENIE3 analyses or as an independent validation of GRNBoost2 results.

Output Format

The network DataFrame has three columns, sorted by descending importance:

ColumnTypeDescription
TFstrTranscription factor (regulator) gene name
targetstrTarget gene name
importancefloatRegulatory importance score (higher = stronger predicted regulation)

Importance scores are not p-values. They represent feature importance from the tree-based model. Filter by absolute threshold or top-N per target for downstream analysis.

The if __name__ == '__main__': Requirement

Dask uses Python's multiprocessing module to spawn worker processes. Without the main guard, each spawned process re-executes the script, leading to infinite process spawning. Always wrap arboreto calls in if __name__ == '__main__': when running as a script. This is not needed inside Jupyter notebooks.

Common Recipes

Recipe: Distributed Computing on a Cluster

When to use: datasets too large for a single machine (e.g., 50k+ cells, 20k+ genes). Requires a Dask distributed scheduler running on the cluster.

# On head node: start scheduler
dask-scheduler
# Output: Scheduler at tcp://10.118.224.134:8786

# On each compute node: start workers
dask-worker tcp://10.118.224.134:8786 --nprocs 4 --nthreads 1 --memory-limit 16GB
from distributed import Client
from arboreto.algo import grnboost2
import pandas as pd

if __name__ == '__main__':
    client = Client('tcp://10.118.224.134:8786')
    print(f"Connected to cluster: {client.scheduler_info()['workers'].__len__()} workers")
    # Example: Connected to cluster: 16 workers

    expression_data = pd.read_csv('large_scrna.tsv', sep='\t')
    tf_names = pd.read_csv('tf_list.txt', header=None)[0].tolist()

    network = grnboost2(
        expression_data=expression_data,
        tf_names=tf_names,
        client_or_address=client,
        seed=42, verbose=True
    )
    network.to_csv('cluster_grn.tsv', sep='\t', index=False)
    print(f"Cluster inference complete: {len(network)} links")
    client.close()

Recipe: Consensus Network from Multiple Seeds

When to use: improve robustness by aggregating networks from multiple random seeds. Links appearing consistently across runs are more reliable.

import pandas as pd
from distributed import LocalCluster, Client
from arboreto.algo import grnboost2

if __name__ == '__main__':
    client = Client(LocalCluster(n_workers=8, threads_per_worker=1))
    expression_data = pd.read_csv('expression_data.tsv', sep='\t')
    tf_names = pd.read_csv('tf_list.txt', header=None)[0].tolist()

    seeds = [42, 123, 456, 789, 1001]
    all_networks = []

    for seed in seeds:
        net = grnboost2(expression_data=expression_data,
                        tf_names=tf_names,
                        client_or_address=client, seed=seed)
        net['seed'] = seed
        all_networks.append(net)
        print(f"Seed {seed}: {len(net)} links")

    combined = pd.concat(all_networks, ignore_index=True)

    # Consensus: mean importance across seeds, keep links found in 3+ runs
    consensus = (combined.groupby(['TF', 'target'])
                 .agg(mean_importance=('importance', 'mean'),
                      n_seeds=('seed', 'nunique'))
                 .reset_index())
    consensus = consensus[consensus['n_seeds'] >= 3]
    consensus = consensus.sort_values('mean_importance', ascending=False)
    consensus.to_csv('consensus_network.tsv', sep='\t', index=False)
    print(f"Consensus network: {len(consensus)} links (found in 3+ of {len(seeds)} runs)")
    # Example: Consensus network: 28453 links (found in 3+ of 5 runs)

    client.close()

Recipe: pySCENIC Integration

When to use: downstream regulatory analysis after arboreto GRN inference. Arboreto produces the adjacency matrix (Step 1 of SCENIC); pySCENIC performs regulon identification (Step 2) and activity scoring (Step 3).

from arboreto.algo import grnboost2
from arboreto.utils import load_tf_names
import pandas as pd

if __name__ == '__main__':
    # Step 1 (arboreto): GRN inference
    expression_matrix = pd.read_csv('scrna_expression.tsv', sep='\t')
    tf_names = load_tf_names('allTFs_hg38.txt')

    adjacencies = grnboost2(
        expression_data=expression_matrix,
        tf_names=tf_names, seed=777
    )
    adjacencies.to_csv('adjacencies.tsv', sep='\t', index=False, header=False)
    print(f"SCENIC Step 1 complete: {len(adjacencies)} adjacencies")
    # Example: SCENIC Step 1 complete: 234567 adjacencies

    # Steps 2-3 use pySCENIC CLI (requires pyscenic installation):
    # pyscenic ctx adjacencies.tsv hg38_ranking.feather \
    #     --annotations_fname motifs-v10nr.hgnc-m0.001-o0.0.tbl \
    #     --output regulons.csv
    #
    # pyscenic aucell scrna_expression.tsv regulons.csv \
    #     --output auc_matrix.csv
    print("Next: run pyscenic ctx (Step 2) and pyscenic aucell (Step 3)")

Expected Outputs

  • full_network.tsv -- complete TF-target-importance table; columns: TF, target, importance
  • filtered_network.tsv -- high-confidence subset after threshold or top-N filtering
  • consensus_network.tsv -- aggregated network from multi-seed runs; columns: TF, target, mean_importance, n_seeds
  • adjacencies.tsv -- arboreto output formatted for pySCENIC input (no header)
  • grn_network.png -- directed graph visualization of top regulatory interactions

Troubleshooting

ProblemCauseSolution
RuntimeError: freeze_support() or infinite process spawningMissing if __name__ == '__main__': guardWrap all arboreto calls in main guard; not needed in Jupyter
MemoryError during inferenceExpression matrix too large for available RAMFilter low-variance genes first, reduce to top 5-10k genes, or use distributed cluster
Empty or near-empty networkTF names do not match expression matrix column namesVerify TF names overlap with expression_matrix.columns; check case sensitivity
Very slow inferenceUsing GENIE3 on large dataset, or too few Dask workersSwitch to GRNBoost2; create explicit Dask client with more workers
ModuleNotFoundError: No module named 'arboreto'Package not installedpip install arboreto
All importance scores are 0 or identicalExpression matrix has no variance (e.g., all zeros after filtering)Check data quality; ensure matrix contains non-zero expression values with variance
TypeError with NumPy array inputMissing gene_names parameterProvide gene_names= list matching the column count of the array

Bundled Resources

This entry is fully self-contained with no references/ directory.

Original reference file dispositions:

  • references/algorithms.md (139 lines) -- Algorithm comparison, parameter details, custom regressor kwargs. Consolidated into Key Concepts (Algorithm Selection Guide table) and Key Parameters. Custom regressor kwargs (regressor_type, regressor_kwargs) omitted: advanced tuning rarely needed; users can pass scikit-learn kwargs directly per the arboreto API docs.
  • references/basic_inference.md (152 lines) -- Input formats, TF loading, basic workflow, output format. Fully consolidated into Workflow Steps 1-2, Key Concepts (Output Format table), and Quick Start.
  • references/distributed_computing.md (242 lines) -- Local client, cluster setup, monitoring, performance tips. Consolidated into Workflow Step 3 and Recipe: Distributed Computing. Omitted: Dask dashboard panel descriptions (monitoring UI detail beyond scope), detailed network/storage tuning (cluster-admin concern).
  • scripts/basic_grn_inference.py (98 lines) -- CLI wrapper with argparse. Core load-infer-save logic absorbed into Workflow Steps 1-6 and Quick Start. Argparse/CLI boilerplate stripped.

Narrative use-case dispositions (from original Common Use Cases):

  • Single-cell RNA-seq analysis -- absorbed into When to Use bullets and Workflow Steps 1/4
  • Bulk RNA-seq with TF filtering -- absorbed into Workflow Step 2 and Quick Start
  • Comparative analysis (multiple conditions) -- omitted as standalone recipe: trivial loop over conditions, pattern shown in Consensus Network recipe

Related Skills

  • scanpy-scrna-seq -- upstream single-cell preprocessing (QC, normalization, HVG selection) before GRN inference
  • scvi-tools-single-cell -- deep generative models for batch correction and imputation prior to network inference
  • pyscenic (planned) -- downstream regulon identification and cellular activity scoring using arboreto adjacencies
  • networkx-graph-analysis -- graph-theoretic analysis of inferred regulatory networks (centrality, communities)

References

  • Arboreto GitHub repository -- source code, README, examples
  • Arboreto PyPI -- installation and version history
  • Moerman et al. (2019) "GRNBoost2 and Arboreto: efficient and scalable inference of gene regulatory networks" Bioinformatics 35(12):2159-2161 -- algorithm paper
  • Aibar et al. (2017) "SCENIC: single-cell regulatory network inference and clustering" Nature Methods 14:1083-1086 -- SCENIC pipeline paper
  • Dask distributed documentation -- Dask client, cluster setup, dashboard

Biopython: Computational Molecular Biology Toolkit

Overview

Biopython is the standard open-source Python library for computational molecular biology, providing modular APIs for sequence handling, biological file parsing, NCBI database access, BLAST searches, protein structure analysis, and phylogenetics. It supports Python 3 and requires NumPy.

When to Use

  • Parse and convert biological file formats (FASTA, GenBank, FASTQ, PDB, mmCIF, PHYLIP)
  • Fetch sequences or publications from NCBI databases (GenBank, PubMed, Protein) programmatically
  • Run and parse BLAST searches (remote NCBI or local BLAST+)
  • Perform pairwise or multiple sequence alignments with custom scoring
  • Analyze 3D protein structures — distances, angles, DSSP, superimposition
  • Build and visualize phylogenetic trees from sequence alignments
  • Calculate sequence statistics (GC content, molecular weight, melting temperature)
  • Batch-process thousands of sequences with custom filtering logic
  • Use pysam instead for reading SAM/BAM/CRAM alignment files and working with mapped reads; use scikit-bio instead for advanced ecological diversity metrics

Prerequisites

  • Python packages: biopython, numpy, matplotlib (for tree visualization)
  • Data requirements: Sequence files (FASTA, GenBank, FASTQ) or accession IDs for NCBI access
  • Environment: Python 3.8+; NCBI Entrez requires email registration
pip install biopython numpy matplotlib

Quick Start

from Bio import SeqIO
from Bio.Seq import Seq
from Bio.SeqUtils import gc_fraction

# Parse a FASTA file and compute basic statistics
records = list(SeqIO.parse("sequences.fasta", "fasta"))
print(f"Sequences loaded: {len(records)}")

seq = records[0].seq
print(f"ID: {records[0].id}")
print(f"Length: {len(seq)} bp")
print(f"GC content: {gc_fraction(seq)*100:.1f}%")
print(f"Reverse complement: {seq.reverse_complement()[:30]}...")
print(f"Protein translation: {seq.translate()[:10]}...")

Core API

Module 1: Sequence Objects (Bio.Seq)

Create and manipulate DNA, RNA, and protein sequences.

from Bio.Seq import Seq

# Create sequence and perform standard operations
dna = Seq("ATGGCCATTGTAATGGGCCGCTGAAAGGGTGCCCGATAG")
print(f"Length: {len(dna)} bp")
print(f"Complement: {dna.complement()}")
print(f"Reverse complement: {dna.reverse_complement()}")
print(f"Transcription: {dna.transcribe()}")
print(f"Translation: {dna.translate()}")
print(f"Translation (to stop): {dna.translate(to_stop=True)}")
# Length: 39 bp
# Translation: MAIVMGR*KGAR*
# Translation (to stop): MAIVMGR
from Bio.Seq import Seq

# Alternative genetic codes (e.g., mitochondrial)
mito_dna = Seq("ATGGCCATTGTAATGGGCCGCTGA")
std_protein = mito_dna.translate(table=1)      # Standard
mito_protein = mito_dna.translate(table=2)     # Vertebrate mitochondrial
print(f"Standard:      {std_protein}")
print(f"Mitochondrial: {mito_protein}")

# Find all start codons
coding_dna = Seq("ATGAAACCCATGGGGTTTAAATAG")
positions = [i for i in range(len(coding_dna) - 2) if coding_dna[i:i+3] == "ATG"]
print(f"ATG positions: {positions}")
# ATG positions: [0, 9]

Module 2: Sequence I/O (Bio.SeqIO)

Read, write, and convert biological file formats.

from Bio import SeqIO

# Parse FASTA file — returns SeqRecord iterator
records = list(SeqIO.parse("sequences.fasta", "fasta"))
print(f"Loaded {len(records)} sequences")
for rec in records[:3]:
    print(f"  {rec.id}: {len(rec.seq)} bp — {rec.description}")

# Parse GenBank — rich annotation access
for rec in SeqIO.parse("genome.gb", "genbank"):
    print(f"{rec.id}: {len(rec.features)} features, {len(rec.seq)} bp")
    for feat in rec.features[:5]:
        print(f"  {feat.type}: {feat.location}")

# Convert between formats
count = SeqIO.convert("input.gb", "genbank", "output.fasta", "fasta")
print(f"Converted {count} records: GenBank → FASTA")
from Bio import SeqIO
from Bio.SeqRecord import SeqRecord
from Bio.Seq import Seq

# Write sequences to file
records = [
    SeqRecord(Seq("ATCGATCG"), id="seq1", description="test sequence 1"),
    SeqRecord(Seq("GCTAGCTA"), id="seq2", description="test sequence 2"),
]
count = SeqIO.write(records, "output.fasta", "fasta")
print(f"Wrote {count} records to output.fasta")

# Filter sequences by length (streaming — memory efficient)
long_seqs = (rec for rec in SeqIO.parse("large_file.fasta", "fasta") if len(rec.seq) >= 200)
count = SeqIO.write(long_seqs, "filtered.fasta", "fasta")
print(f"Kept {count} sequences >= 200 bp")

# Index large FASTA for random access
idx = SeqIO.index("large_file.fasta", "fasta")
print(f"Indexed {len(idx)} sequences")
rec = idx["target_sequence_id"]
print(f"Retrieved: {rec.id}, {len(rec.seq)} bp")

Module 3: NCBI Database Access (Bio.Entrez)

Programmatic search and download from NCBI databases.

from Bio import Entrez, SeqIO

Entrez.email = "your.email@example.com"
# Entrez.api_key = "your_key"  # Optional: 10 req/s instead of 3 req/s

# Search PubMed
handle = Entrez.esearch(db="pubmed", term="CRISPR Cas9 2024", retmax=5)
results = Entrez.read(handle)
handle.close()
print(f"Found {results['Count']} articles, retrieved {len(results['IdList'])} IDs")
print(f"IDs: {results['IdList']}")

# Fetch GenBank record by accession
handle = Entrez.efetch(db="nucleotide", id="EU490707", rettype="gb", retmode="text")
record = SeqIO.read(handle, "genbank")
handle.close()
print(f"{record.id}: {record.description}")
print(f"Length: {len(record.seq)} bp, Features: {len(record.features)}")
from Bio import Entrez
import time

Entrez.email = "your.email@example.com"

# Batch download with rate limiting
handle = Entrez.esearch(db="protein", term="insulin[Protein Name] AND human[Organism]", retmax=20)
results = Entrez.read(handle)
handle.close()

# Fetch summaries in batch
ids = results["IdList"][:10]
handle = Entrez.esummary(db="protein", id=",".join(ids))
summaries = Entrez.read(handle)
handle.close()

for doc in summaries:
    print(f"  {doc['AccessionVersion']}: {doc['Title'][:60]}...")
print(f"\nFetched {len(summaries)} protein summaries")

Module 4: BLAST Operations (Bio.Blast)

Run and parse BLAST searches against NCBI or local databases.

from Bio.Blast import NCBIWWW, NCBIXML

# Remote BLAST search (nucleotide)
query_seq = "ATCGATCGATCGATCGATCGATCGATCGATCG"
result_handle = NCBIWWW.qblast("blastn", "nt", query_seq, hitlist_size=5)
blast_record = NCBIXML.read(result_handle)
result_handle.close()

print(f"Query: {blast_record.query[:50]}")
print(f"Database: {blast_record.database}")
print(f"Hits: {len(blast_record.alignments)}")

for aln in blast_record.alignments[:3]:
    hsp = aln.hsps[0]
    print(f"\n  {aln.title[:60]}...")
    print(f"  E-value: {hsp.expect:.2e}, Identity: {hsp.identities}/{hsp.align_length}")
    print(f"  Score: {hsp.score}, Bits: {hsp.bits:.1f}")
from Bio.Blast.Applications import NcbiblastpCommandline
from Bio.Blast import NCBIXML

# Local BLAST (requires BLAST+ installed)
blastp_cline = NcbiblastpCommandline(
    query="query.fasta",
    db="swissprot",
    evalue=1e-5,
    outfmt=5,  # XML output
    out="blast_results.xml",
    num_threads=4,
)
print(f"Command: {blastp_cline}")
# stdout, stderr = blastp_cline()  # Execute

# Parse local BLAST XML results
with open("blast_results.xml") as f:
    for record in NCBIXML.parse(f):
        print(f"Query: {record.query}")
        for aln in record.alignments[:3]:
            print(f"  Hit: {aln.hit_def[:50]}, E={aln.hsps[0].expect:.2e}")

Module 5: Pairwise Alignment (Bio.Align)

Global and local pairwise sequence alignment with customizable scoring.

from Bio import Align

# Global alignment
aligner = Align.PairwiseAligner()
aligner.mode = "global"
aligner.match_score = 2
aligner.mismatch_score = -1
aligner.open_gap_score = -5
aligner.extend_gap_score = -0.5

alignments = aligner.align("ACCGGTAACG", "ACGGTAAC")
print(f"Score: {alignments.score}")
print(f"Number of alignments: {len(alignments)}")
print(f"Best alignment:\n{alignments[0]}")
from Bio import Align
from Bio.Align import substitution_matrices

# Protein alignment with BLOSUM62
aligner = Align.PairwiseAligner()
aligner.mode = "local"
aligner.substitution_matrix = substitution_matrices.load("BLOSUM62")
aligner.open_gap_score = -10
aligner.extend_gap_score = -0.5

seq1 = "MVLSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTKTYFPHFDLSH"
seq2 = "MVHLTPEEKSAVTALWGKVNVDEVGGEALGRLLVVYPWTQRFFESFGDLST"
alignments = aligner.align(seq1, seq2)
print(f"Score: {alignments.score}")
print(f"Best alignment:\n{alignments[0]}")

Module 6: Protein Structure Analysis (Bio.PDB)

Parse PDB/mmCIF files and analyze 3D protein structures.

from Bio.PDB import PDBParser, PPBuilder

# Parse PDB structure
parser = PDBParser(QUIET=True)
structure = parser.get_structure("1CRN", "1crn.pdb")

# Navigate SMCRA hierarchy: Structure > Model > Chain > Residue > Atom
model = structure[0]
for chain in model:
    residues = list(chain.get_residues())
    print(f"Chain {chain.id}: {len(residues)} residues")

# Extract sequence from structure
ppb = PPBuilder()
for pp in ppb.build_peptides(structure):
    print(f"Peptide: {pp.get_sequence()[:50]}... ({len(pp.get_sequence())} aa)")

# Calculate CA-CA distance
chain_a = model["A"]
ca1 = chain_a[10]["CA"]
ca2 = chain_a[20]["CA"]
distance = ca1 - ca2
print(f"CA distance (res 10-20): {distance:.2f} Angstrom")
from Bio.PDB import PDBParser, Superimposer
import numpy as np

# Structure superimposition (RMSD calculation)
parser = PDBParser(QUIET=True)
struct1 = parser.get_structure("s1", "structure1.pdb")
struct2 = parser.get_structure("s2", "structure2.pdb")

# Get CA atoms for alignment
atoms1 = [res["CA"] for res in struct1[0]["A"].get_residues() if "CA" in res]
atoms2 = [res["CA"] for res in struct2[0]["A"].get_residues() if "CA" in res]

# Superimpose (requires same number of atoms)
n = min(len(atoms1), len(atoms2))
sup = Superimposer()
sup.set_atoms(atoms1[:n], atoms2[:n])
sup.apply(struct2.get_atoms())
print(f"RMSD: {sup.rms:.3f} Angstrom over {n} CA atoms")

Module 7: Phylogenetics (Bio.Phylo)

Build, manipulate, and visualize phylogenetic trees.

from Bio import Phylo, AlignIO
from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor
import io

# Build tree from multiple sequence alignment
alignment = AlignIO.read("aligned_sequences.fasta", "fasta")
print(f"Alignment: {len(alignment)} sequences, {alignment.get_alignment_length()} positions")

# Calculate distance matrix and build NJ tree
calculator = DistanceCalculator("identity")
dm = calculator.get_distance(alignment)
print(f"Distance matrix:\n{dm}")

constructor = DistanceTreeConstructor()
nj_tree = constructor.nj(dm)
upgma_tree = constructor.upgma(dm)

# Visualize
Phylo.draw_ascii(nj_tree)

# Save tree
Phylo.write(nj_tree, "tree.nwk", "newick")
print("Saved tree.nwk")

Module 8: Sequence Utilities (Bio.SeqUtils)

Compute sequence statistics and physicochemical properties.

from Bio.Seq import Seq
from Bio.SeqUtils import gc_fraction, molecular_weight, MeltingTemp

dna = Seq("ATCGATCGATCGATCGATCG")
print(f"GC content: {gc_fraction(dna):.2%}")
print(f"Molecular weight: {molecular_weight(dna, seq_type='DNA'):.2f} Da")
print(f"Melting temp (basic): {MeltingTemp.Tm_Wallace(dna):.1f} C")
print(f"Melting temp (NN):    {MeltingTemp.Tm_NN(dna):.1f} C")

# Protein analysis
from Bio.SeqUtils.ProtParam import ProteinAnalysis
protein = ProteinAnalysis("MVLSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTK")
print(f"\nProtein MW: {protein.molecular_weight():.2f} Da")
print(f"Isoelectric point: {protein.isoelectric_point():.2f}")
print(f"Aromaticity: {protein.aromaticity():.4f}")
print(f"Instability index: {protein.instability_index():.2f}")
print(f"GRAVY: {protein.gravy():.4f}")
aa_pct = protein.get_amino_acids_percent()
print(f"Top 3 amino acids: {sorted(aa_pct.items(), key=lambda x: -x[1])[:3]}")

Common Workflows

Workflow 1: Gene Sequence Retrieval and Analysis Pipeline

Goal: Fetch a gene from NCBI, analyze its properties, translate to protein, and compute statistics.

from Bio import Entrez, SeqIO
from Bio.Seq import Seq
from Bio.SeqUtils import gc_fraction
from Bio.SeqUtils.ProtParam import ProteinAnalysis

Entrez.email = "your.email@example.com"

# Step 1: Fetch gene from GenBank
handle = Entrez.efetch(db="nucleotide", id="NM_007294.4", rettype="gb", retmode="text")
record = SeqIO.read(handle, "genbank")
handle.close()
print(f"Gene: {record.description}")
print(f"Length: {len(record.seq)} bp")

# Step 2: Extract CDS
cds_features = [f for f in record.features if f.type == "CDS"]
if cds_features:
    cds = cds_features[0]
    cds_seq = cds.location.extract(record).seq
    print(f"CDS: {len(cds_seq)} bp, GC: {gc_fraction(cds_seq):.2%}")

    # Step 3: Translate
    protein_seq = cds_seq.translate(to_stop=True)
    print(f"Protein: {len(protein_seq)} aa")
    print(f"First 30 aa: {protein_seq[:30]}...")

    # Step 4: Protein properties
    analysis = ProteinAnalysis(str(protein_seq))
    print(f"MW: {analysis.molecular_weight():.0f} Da")
    print(f"pI: {analysis.isoelectric_point():.2f}")
    print(f"Instability: {analysis.instability_index():.1f}")
    print(f"GRAVY: {analysis.gravy():.3f}")

Workflow 2: Comparative Sequence Analysis with BLAST and Phylogeny

Goal: BLAST a protein sequence, fetch homologs, align, and build a phylogenetic tree.

from Bio.Blast import NCBIWWW, NCBIXML
from Bio import Entrez, SeqIO, AlignIO, Phylo
from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor
from Bio.Align.Applications import MuscleCommandline
import time

Entrez.email = "your.email@example.com"

# Step 1: BLAST search
query = "MVLSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTKTYFPHFDLSH"
result_handle = NCBIWWW.qblast("blastp", "swissprot", query, hitlist_size=10)
blast_record = NCBIXML.read(result_handle)
print(f"BLAST hits: {len(blast_record.alignments)}")

# Step 2: Collect homolog accessions
accessions = []
for aln in blast_record.alignments[:8]:
    acc = aln.accession
    accessions.append(acc)
    print(f"  {acc}: E={aln.hsps[0].expect:.2e}, {aln.hit_def[:50]}...")

# Step 3: Fetch sequences and save for alignment
handle = Entrez.efetch(db="protein", id=",".join(accessions), rettype="fasta", retmode="text")
records = list(SeqIO.parse(handle, "fasta"))
handle.close()
SeqIO.write(records, "homologs.fasta", "fasta")
print(f"Saved {len(records)} homolog sequences")

# Step 4: Align (requires MUSCLE installed)
# muscle_cline = MuscleCommandline(input="homologs.fasta", out="aligned.fasta")
# muscle_cline()

# Step 5: Build phylogenetic tree from alignment
alignment = AlignIO.read("aligned.fasta", "fasta")
calculator = DistanceCalculator("blosum62")
dm = calculator.get_distance(alignment)
tree = DistanceTreeConstructor().nj(dm)
Phylo.draw_ascii(tree)
Phylo.write(tree, "homologs.nwk", "newick")
print("Saved phylogenetic tree to homologs.nwk")

Workflow 3: Batch Sequence Processing and Quality Filtering

Goal: Process a large FASTQ/FASTA dataset — filter by quality/length, compute statistics, and export.

from Bio import SeqIO
from Bio.SeqUtils import gc_fraction
import pandas as pd

# Step 1: Stream through large file and collect stats
stats = []
passed = []

for rec in SeqIO.parse("reads.fastq", "fastq"):
    seq_len = len(rec.seq)
    gc = gc_fraction(rec.seq)
    avg_qual = sum(rec.letter_annotations["phred_quality"]) / seq_len

    stats.append({"id": rec.id, "length": seq_len, "gc": gc, "avg_qual": avg_qual})

    # Filter: length >= 100 and avg quality >= 20
    if seq_len >= 100 and avg_qual >= 20:
        passed.append(rec)

# Step 2: Summary statistics
df = pd.DataFrame(stats)
print(f"Total reads: {len(df)}")
print(f"Passed QC:   {len(passed)} ({len(passed)/len(df)*100:.1f}%)")
print(f"\nLength:  mean={df['length'].mean():.0f}, median={df['length'].median():.0f}")
print(f"GC:      mean={df['gc'].mean():.2%}, std={df['gc'].std():.2%}")
print(f"Quality: mean={df['avg_qual'].mean():.1f}, min={df['avg_qual'].min():.1f}")

# Step 3: Export filtered reads
count = SeqIO.write(passed, "filtered_reads.fastq", "fastq")
print(f"\nExported {count} filtered reads to filtered_reads.fastq")

# Step 4: Save statistics
df.to_csv("read_statistics.csv", index=False)
print(f"Saved statistics to read_statistics.csv")

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
Seq.translate(table=)Bio.Seq1 (Standard)1-33NCBI genetic code table for translation
Seq.translate(to_stop=)Bio.SeqFalseTrue, FalseStop at first stop codon vs translate entire sequence
SeqIO.parse(format=)Bio.SeqIO—"fasta", "genbank", "fastq", "phylip"File format for parsing
Entrez.emailBio.Entrez(required)Valid emailNCBI requires email for tracking; set before any Entrez call
Entrez.api_keyBio.EntrezNoneNCBI API key stringIncreases rate limit from 3 to 10 requests/second
PairwiseAligner.modeBio.Align"global""global", "local"Needleman-Wunsch vs Smith-Waterman algorithm
PairwiseAligner.open_gap_scoreBio.Align-1-20 to 0Penalty for opening a gap; more negative = fewer gaps
PairwiseAligner.substitution_matrixBio.AlignNone"BLOSUM62", "BLOSUM45", "PAM250"Scoring matrix for protein alignment
PDBParser(QUIET=)Bio.PDBFalseTrue, FalseSuppress parser warnings for non-standard PDB files
DistanceCalculator(model=)Bio.Phylo"identity""identity", "blosum62"Distance model for tree construction

Common Recipes

Recipe: Restriction Enzyme Analysis

When to use: Find restriction sites in a DNA sequence for cloning design.

from Bio.Seq import Seq
from Bio.Restriction import EcoRI, BamHI, HindIII, RestrictionBatch, Analysis

seq = Seq("GAATTCAAAGGATCCTTTTAAGCTTGGGAATTC")

# Single enzyme
print(f"EcoRI cuts at: {EcoRI.search(seq)}")
print(f"BamHI cuts at: {BamHI.search(seq)}")

# Batch analysis
batch = RestrictionBatch([EcoRI, BamHI, HindIII])
analysis = Analysis(batch, seq)
result = analysis.full()
for enzyme, sites in result.items():
    if sites:
        print(f"{enzyme}: cuts at positions {sites}")

Recipe: Motif Discovery and Position Weight Matrix

When to use: Analyze transcription factor binding sites or consensus patterns.

from Bio import motifs
from Bio.Seq import Seq

# Create motif from observed binding sites
instances = [
    Seq("TACGAT"),
    Seq("TAGCAT"),
    Seq("TACGGT"),
    Seq("TAGCAT"),
    Seq("TACGAT"),
]
m = motifs.create(instances)
print(f"Consensus: {m.consensus}")
print(f"Degenerate: {m.degenerate_consensus}")
print(f"\nPosition Weight Matrix:")
print(m.counts)

# Score a new sequence against the motif
pwm = m.counts.normalize(pseudocounts=0.5)
pssm = pwm.log_odds()
test_seq = Seq("AATACGATCCC")
for pos, score in pssm.search(test_seq, threshold=0.0):
    print(f"  Position {pos}: score={score:.2f}")

Recipe: GenomeDiagram — Visualize Genomic Features

When to use: Create a circular or linear genome map from a GenBank file.

from Bio import SeqIO
from Bio.Graphics import GenomeDiagram
from reportlab.lib import colors
from reportlab.lib.units import cm

# Load annotated genome
record = SeqIO.read("plasmid.gb", "genbank")

# Create diagram
diagram = GenomeDiagram.Diagram(record.name)
track = diagram.new_track(1, name="Annotated Features", greytrack=True)
feature_set = track.new_set()

# Color features by type
color_map = {"CDS": colors.blue, "gene": colors.green, "promoter": colors.red}
for feature in record.features:
    if feature.type in color_map:
        feature_set.add_feature(feature, color=color_map[feature.type], label=True, label_size=8)

# Draw circular map
diagram.draw(format="circular", circular=True, pagesize=(20*cm, 20*cm), start=0, end=len(record))
diagram.write("genome_map.pdf", "PDF")
print(f"Saved genome_map.pdf ({len(record.features)} features)")

Recipe: Batch PubMed Literature Search

When to use: Programmatically search and download publication metadata.

from Bio import Entrez
import time

Entrez.email = "your.email@example.com"

# Search PubMed with complex query
query = "(CRISPR[Title]) AND (2024[Date - Publication]) AND (review[Publication Type])"
handle = Entrez.esearch(db="pubmed", term=query, retmax=100)
results = Entrez.read(handle)
handle.close()
print(f"Found {results['Count']} articles")

# Fetch abstracts in batches
ids = results["IdList"]
batch_size = 20
articles = []
for i in range(0, len(ids), batch_size):
    batch = ids[i:i+batch_size]
    handle = Entrez.efetch(db="pubmed", id=",".join(batch), rettype="xml")
    records = Entrez.read(handle)
    handle.close()
    for article in records["PubmedArticle"]:
        info = article["MedlineCitation"]["Article"]
        title = info.get("ArticleTitle", "N/A")
        abstract = info.get("Abstract", {}).get("AbstractText", ["N/A"])[0]
        articles.append({"title": title, "abstract": str(abstract)[:200]})
    time.sleep(0.4)  # Rate limiting

for a in articles[:5]:
    print(f"  {a['title'][:70]}...")
print(f"\nRetrieved {len(articles)} articles")

Troubleshooting

ProblemCauseSolution
HTTPError 400 from EntrezInvalid accession/ID or malformed queryValidate accessions; check query syntax with NCBI web interface first
HTTPError 429 (Too Many Requests)Exceeding NCBI rate limit (3 req/s)Set Entrez.api_key for 10 req/s; add time.sleep(0.4) between calls
ValueError: No records found in SeqIO.read()Empty file or wrong format stringUse SeqIO.parse() to check if file has records; verify format matches actual content
Bio.PDB.PDBExceptions.PDBConstructionWarningNon-standard atoms or occupancy issuesUse PDBParser(QUIET=True) or fix PDB with pdb-tools; check for alternate conformations
BLAST search times outLarge query or busy NCBI serversUse local BLAST+ for large-scale searches; set NCBIWWW.qblast(hitlist_size=N) to limit results
Alignment has sequences of different lengthsUnaligned sequences passed to AlignIOAlign sequences first with MUSCLE/Clustal before loading as alignment
SeqIO.index() raises ValueErrorDuplicate IDs in FASTA fileDeduplicate IDs with SeqIO.to_dict() or pre-process with awk
ImportError: No module named BioBiopython not installed in active environmentpip install biopython; verify with python -c "import Bio; print(Bio.__version__)"
translate() gives unexpected *Stop codons in middle of sequenceCheck reading frame; use Seq.translate(table=N) with correct genetic code
GenomeDiagram blank outputNo features matched filter criteriaCheck feature.type values in your GenBank file; print types to debug

References

Biopython: Sequence Analysis Toolkit

Overview

Biopython provides a comprehensive suite of modules for sequence-centric bioinformatics: reading and writing every major biological file format (FASTA, FASTQ, GenBank, GFF), querying NCBI databases programmatically, running BLAST searches and parsing results, aligning sequences pairwise or in multiple-sequence alignments, and building and visualizing phylogenetic trees. This skill focuses on analysis workflows — from NCBI data retrieval through alignment to phylogenetic inference.

For PCR primer design, restriction enzyme digestion, cloning simulation, protein structure analysis (Bio.PDB), and molecular weight/Tm calculations, see biopython-molecular-biology.

When to Use

  • Download a gene family from NCBI Nucleotide/Protein, align sequences, and construct a phylogenetic tree
  • Parse GenBank or GFF3 annotation files and extract CDS sequences for a set of features
  • Run a BLAST search against NCBI nt or nr, filter significant hits, and fetch their full sequences
  • Compute pairwise sequence identities or score alignments with BLOSUM62/PAM250 matrices
  • Index a large multi-FASTA or FASTQ file with SeqIO.index() for random-access retrieval without loading all sequences into RAM
  • Convert between sequence formats (FASTA ↔ GenBank ↔ FASTQ ↔ PHYLIP) in a single call
  • Traverse, root, prune, and annotate a Newick or Nexus phylogenetic tree programmatically
  • Use pysam instead when working with SAM/BAM/CRAM alignment files and mapped reads
  • Use scikit-bio instead when you need ecological diversity metrics (UniFrac, beta diversity) or ordination methods such as PCoA
  • Use gget instead for quick gene lookups and one-liner NCBI queries without writing an Entrez pipeline

Prerequisites

  • Python packages: biopython, numpy, matplotlib
  • Optional tools: MUSCLE or ClustalW installed locally (for Bio.Align.Applications wrappers)
  • NCBI access: Set Entrez.email before any E-utilities call; obtain a free API key at https://www.ncbi.nlm.nih.gov/account/ for 10 req/s (default is 3 req/s)
  • Local BLAST: BLAST+ installed separately (conda install -c bioconda blast) for offline searches

Check before installing: The tool may already be available in the current environment (e.g., inside a pixi / conda env). Run command -v python first and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool via pixi run python rather than bare python.

pip install biopython numpy matplotlib
conda install -c bioconda blast  # optional, for local BLAST

Quick Start

from Bio import SeqIO, Entrez
from Bio.SeqUtils import gc_fraction

# Fetch a GenBank record and display basic stats
Entrez.email = "your.email@example.com"
handle = Entrez.efetch(db="nucleotide", id="NM_007294", rettype="gb", retmode="text")
record = SeqIO.read(handle, "genbank")
handle.close()

print(f"ID: {record.id}")
print(f"Length: {len(record.seq)} bp")
print(f"GC content: {gc_fraction(record.seq)*100:.1f}%")
print(f"Features: {len(record.features)}")
print(f"First feature: {record.features[0].type} at {record.features[0].location}")
# ID: NM_007294.4
# Length: 7207 bp
# GC content: 47.3%
# Features: 9

Core API

Module 1: SeqIO — File Parsing and Format Conversion

Bio.SeqIO reads and writes every major sequence format (FASTA, FASTQ, GenBank, EMBL, PHYLIP, Nexus). SeqIO.parse() returns an iterator of SeqRecord objects; SeqIO.index() builds an on-disk or in-memory dictionary for large files.

from Bio import SeqIO

# Parse FASTA and FASTQ
fasta_records = list(SeqIO.parse("sequences.fasta", "fasta"))
print(f"FASTA: {len(fasta_records)} sequences")

# FASTQ: access quality scores
for rec in SeqIO.parse("reads.fastq", "fastq"):
    quals = rec.letter_annotations["phred_quality"]
    avg_q = sum(quals) / len(quals)
    print(f"  {rec.id}: {len(rec.seq)} bp, mean Q={avg_q:.1f}")
    break  # show one example

# Parse GenBank with feature access
for rec in SeqIO.parse("chromosome.gb", "genbank"):
    cdss = [f for f in rec.features if f.type == "CDS"]
    print(f"{rec.id}: {len(cdss)} CDS features")
    for cds in cdss[:3]:
        gene = cds.qualifiers.get("gene", ["unknown"])[0]
        print(f"  {gene}: {cds.location}")
from Bio import SeqIO

# SeqIO.index() for random access without loading all records
# Useful for large reference FASTA files (genomes, nr database subsets)
idx = SeqIO.index("large_genome.fasta", "fasta")
print(f"Index contains {len(idx)} sequences")

# Retrieve specific sequences by ID in O(1)
target = idx["chr1"]
print(f"chr1: {len(target.seq):,} bp")
region = target.seq[1_000_000:1_001_000]
print(f"Region [1M-1M+1kb]: {region[:60]}...")

# Format conversion: GenBank → FASTA in one call
n = SeqIO.convert("annotation.gb", "genbank", "sequences.fasta", "fasta")
print(f"Converted {n} records to FASTA")

Module 2: Seq and SeqRecord — Sequence Objects and Feature Annotations

Bio.Seq represents a biological sequence with in-place operations. Bio.SeqRecord wraps a Seq with an ID, description, and a dictionary of feature annotations. Features use FeatureLocation with strand (+1, -1).

from Bio.Seq import Seq
from Bio.SeqRecord import SeqRecord
from Bio.SeqFeature import SeqFeature, FeatureLocation
from Bio.SeqUtils import gc_fraction, MeltingTemp

dna = Seq("ATGAAACCCGGGTTTTAA")
print(f"GC content: {gc_fraction(dna)*100:.1f}%")
print(f"Reverse complement: {dna.reverse_complement()}")
print(f"Transcript: {dna.transcribe()}")
print(f"Translation: {dna.translate()}")
print(f"Translation (to stop): {dna.translate(to_stop=True)}")

# Tm calculation for a short primer
primer = Seq("ATGAAACCCGGG")
tm = MeltingTemp.Tm_Wallace(primer)
print(f"Tm (Wallace): {tm:.1f}°C")
from Bio.Seq import Seq
from Bio.SeqRecord import SeqRecord
from Bio.SeqFeature import SeqFeature, FeatureLocation

# Build a SeqRecord with annotated features
gene_seq = Seq("ATGAAACCCGGGTTTTAAATCGATCG" * 10)
record = SeqRecord(gene_seq, id="MY_GENE_001", description="synthetic example gene")

# Annotate a CDS feature
cds = SeqFeature(
    FeatureLocation(0, 18, strand=+1),
    type="CDS",
    qualifiers={"gene": ["myGene"], "product": ["hypothetical protein"]}
)
record.features.append(cds)

# Extract and translate the feature
cds_seq = cds.location.extract(record.seq)
protein = cds_seq.translate(to_stop=True)
print(f"CDS: {cds_seq}")
print(f"Protein: {protein}")

# Slice record preserving feature annotations
sub = record[0:60]
print(f"Subrecord: {len(sub.seq)} bp, {len(sub.features)} features")

Module 3: Entrez — Programmatic NCBI Database Access

Bio.Entrez wraps the NCBI E-utilities API: esearch finds records matching a query, efetch retrieves full records, elink finds related records across databases, and esummary returns document summaries. Always set Entrez.email and respect the 3 req/s rate limit (10 req/s with API key).

from Bio import Entrez, SeqIO
import time

Entrez.email = "your.email@example.com"
# Entrez.api_key = "YOUR_API_KEY"  # for 10 req/s

# esearch: find IDs matching a query
handle = Entrez.esearch(db="nucleotide", term="BRCA1[Gene] AND Homo sapiens[Organism] AND mRNA[Filter]", retmax=10)
search_results = Entrez.read(handle)
handle.close()

print(f"Total hits: {search_results['Count']}")
ids = search_results["IdList"]
print(f"Retrieved IDs: {ids}")

# efetch: download full GenBank records
for acc_id in ids[:3]:
    handle = Entrez.efetch(db="nucleotide", id=acc_id, rettype="gb", retmode="text")
    record = SeqIO.read(handle, "genbank")
    handle.close()
    print(f"  {record.id}: {len(record.seq)} bp — {record.description[:60]}")
    time.sleep(0.4)  # stay within rate limit
from Bio import Entrez
import time

Entrez.email = "your.email@example.com"

# elink: cross-database links (e.g., PubMed article → related nucleotide sequences)
handle = Entrez.elink(dbfrom="pubmed", db="nucleotide", id="29087512")
link_results = Entrez.read(handle)
handle.close()

linked_ids = []
for linkset in link_results:
    for db_links in linkset.get("LinkSetDb", []):
        if db_links["DbTo"] == "nucleotide":
            linked_ids = [lnk["Id"] for lnk in db_links["Link"]]
            break

print(f"Nucleotide sequences linked to PubMed 29087512: {len(linked_ids)}")
print(f"First IDs: {linked_ids[:5]}")

# esummary: lightweight metadata without downloading full records
if linked_ids:
    handle = Entrez.esummary(db="nucleotide", id=",".join(linked_ids[:5]))
    summaries = Entrez.read(handle)
    handle.close()
    for doc in summaries:
        print(f"  {doc['AccessionVersion']}: {doc['Title'][:70]}")

Module 4: BLAST — Remote and Local Sequence Similarity Search

Bio.Blast.NCBIWWW.qblast() submits queries to NCBI BLAST servers and returns XML handles. Bio.Blast.NCBIXML.parse() (or .read() for single queries) yields Blast objects with alignments and hsps. For large-scale searches, use local BLAST+ via subprocess.

from Bio.Blast import NCBIWWW, NCBIXML
from Bio.Seq import Seq

# Remote BLASTP against Swiss-Prot (small, reviewed database)
query = Seq("MTEYKLVVVGAGGVGKSALTIQLIQNHFVDEYDPTIEDSY")
result_handle = NCBIWWW.qblast("blastp", "swissprot", str(query), hitlist_size=10)

# Parse results
blast_record = NCBIXML.read(result_handle)
print(f"Query: {blast_record.query}")
print(f"Database: {blast_record.database}")
print(f"Hits: {len(blast_record.alignments)}")

E_VALUE_THRESH = 1e-5
for alignment in blast_record.alignments[:5]:
    for hsp in alignment.hsps:
        if hsp.expect < E_VALUE_THRESH:
            identity_pct = hsp.identities / hsp.align_length * 100
            print(f"\n  Hit: {alignment.title[:70]}")
            print(f"  E-value: {hsp.expect:.2e}, Identity: {identity_pct:.1f}%, Score: {hsp.score}")
            print(f"  Query:  {hsp.query[:60]}")
            print(f"  Match:  {hsp.match[:60]}")
            print(f"  Sbjct:  {hsp.sbjct[:60]}")
import subprocess
from Bio.Blast import NCBIXML
from Bio import SeqIO

# Local BLAST+ via subprocess (faster for large batches)
# Requires: makeblastdb and blastp installed (conda install -c bioconda blast)

# Step 1: Build a local database from a FASTA file
subprocess.run(
    ["makeblastdb", "-in", "ref_proteins.fasta", "-dbtype", "prot", "-out", "ref_db"],
    check=True
)

# Step 2: Run blastp against local database
result = subprocess.run(
    ["blastp", "-query", "query.fasta", "-db", "ref_db",
     "-outfmt", "5",           # XML output for NCBIXML parsing
     "-evalue", "1e-5",
     "-num_threads", "4",
     "-out", "blast_results.xml"],
    check=True
)

# Step 3: Parse XML results
with open("blast_results.xml") as fh:
    for blast_rec in NCBIXML.parse(fh):
        print(f"Query: {blast_rec.query_id}")
        for aln in blast_rec.alignments[:3]:
            hsp = aln.hsps[0]
            print(f"  Hit: {aln.hit_id}, E={hsp.expect:.2e}, Id={hsp.identities}/{hsp.align_length}")

Module 5: Phylo — Tree Parsing, Manipulation, and Visualization

Bio.Phylo reads Newick, Nexus, PhyloXML, and NeXML formats. Trees are represented as Clade objects with nested children. Key methods: find_clades() (generator over nodes), common_ancestor(), distance(), root_with_outgroup(), and prune(). Visualization uses matplotlib.

from Bio import Phylo
import io

# Parse a Newick tree from a string
newick = "((Homo_sapiens:0.01, Pan_troglodytes:0.012):0.08, (Mus_musculus:0.15, Rattus_norvegicus:0.14):0.12, Drosophila_melanogaster:0.85);"
tree = Phylo.read(io.StringIO(newick), "newick")

print(f"Number of terminals: {tree.count_terminals()}")
print(f"Total branch length: {tree.total_branch_length():.3f}")
print(f"Terminals: {[t.name for t in tree.get_terminals()]}")

# Root with outgroup and compute distances
tree.root_with_outgroup("Drosophila_melanogaster")
human = tree.find_any("Homo_sapiens")
chimp = tree.find_any("Pan_troglodytes")
mouse = tree.find_any("Mus_musculus")
print(f"\nDistance Human–Chimp: {tree.distance(human, chimp):.4f}")
print(f"Distance Human–Mouse: {tree.distance(human, mouse):.4f}")

# Traverse all internal nodes
for clade in tree.find_clades(order="level"):
    if not clade.is_terminal():
        children = [c.name or "internal" for c in clade.clades]
        print(f"  Internal node → {children}")
from Bio import Phylo
import matplotlib.pyplot as plt
import io

newick = "((Homo_sapiens:0.01, Pan_troglodytes:0.012):0.08, (Mus_musculus:0.15, Rattus_norvegicus:0.14):0.12, Drosophila_melanogaster:0.85);"
tree = Phylo.read(io.StringIO(newick), "newick")
tree.root_with_outgroup("Drosophila_melanogaster")
tree.ladderize()   # sort clades by size for a clean layout

# Annotate bootstrap support (if present in node labels)
for clade in tree.find_clades():
    if clade.confidence is not None:
        clade.name = f"{clade.confidence:.0f}"

fig, ax = plt.subplots(figsize=(8, 5))
Phylo.draw(tree, axes=ax, do_show=False, show_confidence=True)
ax.set_title("Primate + Outgroup Phylogeny")
plt.tight_layout()
plt.savefig("phylogeny.png", dpi=150, bbox_inches="tight")
print("Saved phylogeny.png")

Module 6: PairwiseAligner — Pairwise Sequence Alignment

Bio.Align.PairwiseAligner replaces the legacy pairwise2 module (deprecated since Biopython 1.80). It supports local and global alignment, gap open/extend penalties, and any substitution matrix from Bio.Align.substitution_matrices.

from Bio.Align import PairwiseAligner, substitution_matrices

# Global protein alignment with BLOSUM62
aligner = PairwiseAligner()
aligner.mode = "global"
aligner.substitution_matrix = substitution_matrices.load("BLOSUM62")
aligner.open_gap_score = -11
aligner.extend_gap_score = -1

seq1 = "MTEYKLVVVGAGGVGKSALTIQLIQNHFVDEYDPTIEDSY"
seq2 = "MTEYKLVVVGAVGVGKSALTIQLIQNHFVDEYDPTIEDSY"  # G12V variant

alignments = aligner.align(seq1, seq2)
best = alignments[0]
print(f"Score: {best.score:.1f}")
print(f"Number of alignments: {aligner.score(seq1, seq2):.0f} (score only, faster)")
print(best)   # formatted alignment string

identity = sum(a == b for a, b in zip(*best) if a != "-") / best.shape[1]
print(f"Identity: {identity*100:.1f}%")
from Bio.Align import PairwiseAligner, substitution_matrices

# Local DNA alignment
aligner = PairwiseAligner()
aligner.mode = "local"
aligner.match_score = 2
aligner.mismatch_score = -1
aligner.open_gap_score = -5
aligner.extend_gap_score = -0.5

query = "ACGTACGTACGT"
subject = "TTTTACGTACGTACGTTTTT"
alignments = list(aligner.align(query, subject))
print(f"Local alignments found: {len(alignments)}")
best = alignments[0]
print(f"Score: {best.score}")
print(f"Aligned region in subject: [{best.coordinates[1][0]}:{best.coordinates[1][-1]}]")
print(best)

# Batch pairwise identity matrix
seqs = ["ACGTACGT", "ACGTATGT", "TTGTACGT", "ACGTACGG"]
n = len(seqs)
aligner2 = PairwiseAligner(mode="global", match_score=1, mismatch_score=-1)
print("\nPairwise identity matrix:")
for i in range(n):
    row = []
    for j in range(n):
        score = aligner2.score(seqs[i], seqs[j])
        max_len = max(len(seqs[i]), len(seqs[j]))
        row.append(f"{score/max_len:.2f}")
    print("  " + "  ".join(row))

Key Concepts

SeqRecord and Feature Coordinates

SeqRecord stores sequence metadata and a list of SeqFeature objects. Features use zero-based, half-open FeatureLocation(start, end, strand) — the same convention as Python slicing. Use feature.location.extract(record.seq) to obtain the strand-correct subsequence. Compound locations (e.g., spliced exons) use CompoundLocation.

from Bio import SeqIO

# Extract all CDS protein translations from a GenBank record
for rec in SeqIO.parse("gene.gb", "genbank"):
    for feat in rec.features:
        if feat.type == "CDS":
            gene = feat.qualifiers.get("gene", ["?"])[0]
            product = feat.qualifiers.get("product", ["unknown"])[0]
            cds_nt = feat.location.extract(rec.seq)
            protein = cds_nt.translate(to_stop=True)
            print(f"{gene} ({product}): {len(protein)} aa — {protein[:10]}...")

Phylo Clade Objects

A Clade is both a node and the subtree rooted at that node. Terminal clades (leaves) have .name set; internal clades may have .confidence (bootstrap) and .branch_length. Use tree.find_clades() with terminal=True/False to filter. The root is tree.root (also a Clade). Phylo.read() returns a Tree wrapper; tree.root gives the root Clade.

Common Workflows

Workflow 1: Gene Family Phylogeny — NCBI Download → MUSCLE Alignment → Tree

Goal: Download all BRCA1 orthologs from Vertebrata, align with MUSCLE, and build a neighbor-joining tree.

import subprocess
import time
from Bio import Entrez, SeqIO, Phylo, AlignIO
from Bio.Align import MultipleSeqAlignment
import matplotlib.pyplot as plt

Entrez.email = "your.email@example.com"

# Step 1: Search NCBI Protein for BRCA1 orthologs
handle = Entrez.esearch(
    db="protein",
    term="BRCA1[Gene] AND Vertebrata[Organism] AND RefSeq[Filter]",
    retmax=20
)
results = Entrez.read(handle); handle.close()
ids = results["IdList"]
print(f"Found {results['Count']} sequences, using {len(ids)}")

# Step 2: Fetch sequences in FASTA format
handle = Entrez.efetch(db="protein", id=",".join(ids), rettype="fasta", retmode="text")
with open("brca1_orthologs.fasta", "w") as fh:
    fh.write(handle.read())
handle.close()
print(f"Saved {len(ids)} sequences to brca1_orthologs.fasta")
time.sleep(1)

# Step 3: Align with MUSCLE (must be installed: conda install -c bioconda muscle)
subprocess.run(
    ["muscle", "-align", "brca1_orthologs.fasta", "-output", "brca1_aligned.fasta"],
    check=True
)
alignment = AlignIO.read("brca1_aligned.fasta", "fasta")
print(f"Alignment: {len(alignment)} sequences × {alignment.get_alignment_length()} columns")

# Step 4: Build a neighbor-joining tree using Bio.Phylo + distance matrix
from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor

calculator = DistanceCalculator("identity")
dm = calculator.get_distance(alignment)
constructor = DistanceTreeConstructor(calculator, method="nj")
tree = constructor.build_tree(alignment)

# Step 5: Root and save
tree.root_at_midpoint()
tree.ladderize()
Phylo.write(tree, "brca1_nj.nwk", "newick")
print("Saved NJ tree to brca1_nj.nwk")

# Step 6: Visualize
fig, ax = plt.subplots(figsize=(10, 8))
Phylo.draw(tree, axes=ax, do_show=False)
ax.set_title("BRCA1 Ortholog NJ Tree")
plt.tight_layout()
plt.savefig("brca1_tree.png", dpi=150, bbox_inches="tight")
print("Saved brca1_tree.png")

Workflow 2: BLAST Pipeline — FASTA Query → Remote BLAST → Fetch Top Hits

Goal: Take a query FASTA, BLAST against NCBI nr, filter significant hits, retrieve full GenBank records for the top matches, and write a summary CSV.

import csv
import time
from Bio import SeqIO, Entrez
from Bio.Blast import NCBIWWW, NCBIXML

Entrez.email = "your.email@example.com"

# Step 1: Load query sequence
query_record = SeqIO.read("query.fasta", "fasta")
print(f"Query: {query_record.id} ({len(query_record.seq)} aa)")

# Step 2: Remote BLAST against nr protein database
print("Submitting BLAST job (may take 1-5 minutes)...")
result_handle = NCBIWWW.qblast(
    "blastp", "nr", str(query_record.seq),
    hitlist_size=20,
    expect=1e-5,
    word_size=6,
    matrix_name="BLOSUM62"
)

# Step 3: Parse results and filter
E_THRESH = 1e-5
MIN_IDENTITY = 50.0
top_hits = []

blast_record = NCBIXML.read(result_handle)
for alignment in blast_record.alignments:
    for hsp in alignment.hsps:
        if hsp.expect > E_THRESH:
            continue
        identity_pct = hsp.identities / hsp.align_length * 100
        if identity_pct < MIN_IDENTITY:
            continue
        accession = alignment.accession
        top_hits.append({
            "accession": accession,
            "title": alignment.title[:80],
            "length": alignment.length,
            "score": hsp.score,
            "evalue": hsp.expect,
            "identity_pct": round(identity_pct, 1),
            "coverage": round(hsp.align_length / blast_record.query_length * 100, 1),
        })

print(f"Filtered to {len(top_hits)} significant hits")

# Step 4: Fetch full GenBank records for top 5 hits
accessions = [h["accession"] for h in top_hits[:5]]
time.sleep(1)
handle = Entrez.efetch(db="protein", id=",".join(accessions), rettype="gb", retmode="text")
gb_records = list(SeqIO.parse(handle, "genbank"))
handle.close()
SeqIO.write(gb_records, "top_blast_hits.gb", "genbank")
print(f"Saved {len(gb_records)} full GenBank records to top_blast_hits.gb")

# Step 5: Write CSV summary
with open("blast_summary.csv", "w", newline="") as fh:
    writer = csv.DictWriter(fh, fieldnames=top_hits[0].keys())
    writer.writeheader()
    writer.writerows(top_hits)
print(f"Saved blast_summary.csv ({len(top_hits)} hits)")

Workflow 3: FASTQ Quality Filter and Format Pipeline

Goal: Read a FASTQ file, filter reads by mean quality and length, and output a FASTA for downstream alignment.

from Bio import SeqIO
from Bio.SeqUtils import gc_fraction

input_fastq = "raw_reads.fastq"
output_fasta = "filtered_reads.fasta"
MIN_QUAL = 20     # mean Phred quality
MIN_LEN = 100     # minimum read length
MAX_LEN = 300     # maximum read length

def passes_qc(record, min_q=MIN_QUAL, min_l=MIN_LEN, max_l=MAX_LEN):
    quals = record.letter_annotations["phred_quality"]
    mean_q = sum(quals) / len(quals)
    length = len(record.seq)
    return mean_q >= min_q and min_l <= length <= max_l

kept = 0
total = 0
with open(output_fasta, "w") as out_fh:
    for record in SeqIO.parse(input_fastq, "fastq"):
        total += 1
        if passes_qc(record):
            # Convert FASTQ → FASTA (drops quality scores)
            SeqIO.write(record, out_fh, "fasta")
            kept += 1

print(f"Total reads: {total:,}")
print(f"Passed QC: {kept:,} ({kept/total*100:.1f}%)")
print(f"Saved to: {output_fasta}")

Key Parameters

ParameterModule / FunctionDefaultRange / OptionsEffect
retmaxEntrez.esearch()201–10000Max IDs returned per search; use history server for >10000
hitlist_sizeNCBIWWW.qblast()501–5000Max BLAST alignments returned
expectNCBIWWW.qblast()10.01e-100–1000E-value cutoff for BLAST hit reporting
matrix_nameNCBIWWW.qblast()"BLOSUM62""BLOSUM45", "BLOSUM80", "PAM250"Substitution matrix; BLOSUM62 for general use
modePairwiseAligner"global""global", "local"Needleman-Wunsch (global) vs Smith-Waterman (local)
open_gap_scorePairwiseAligner-1Negative floatPenalty for opening a gap; increase magnitude to penalize more
extend_gap_scorePairwiseAligner0Negative floatPer-residue gap extension penalty
methodDistanceTreeConstructor"nj""nj", "upgma"NJ (unrooted) vs UPGMA (rooted, assumes clock)
formatPhylo.read/write()required"newick", "nexus", "phyloxml"Tree file format
rettypeEntrez.efetch()required"fasta", "gb", "xml"Record format returned; pair with matching SeqIO format string

Best Practices

  1. Always set Entrez.email and respect rate limits. NCBI blocks IPs that exceed 3 req/s without a key. Add time.sleep(0.4) between requests in loops, or use Entrez.api_key for 10 req/s.

    from Bio import Entrez
    import time
    Entrez.email = "your.email@institution.edu"
    Entrez.api_key = "YOUR_API_KEY"    # from https://www.ncbi.nlm.nih.gov/account/
    # In any batch loop: time.sleep(0.11)  # ~9 req/s with key
    
  2. Use SeqIO.index() for large FASTA files instead of list(SeqIO.parse(...)). Loading all records into a list consumes O(N) memory; index() reads only the byte offsets and fetches records on demand.

    # Bad for large files: loads everything into RAM
    # records = {r.id: r for r in SeqIO.parse("genome.fasta", "fasta")}
    
    # Good: on-disk index, O(1) access
    from Bio import SeqIO
    idx = SeqIO.index("genome.fasta", "fasta")
    rec = idx["chr22"]          # fetches only this record
    
  3. Use PairwiseAligner not the legacy pairwise2. Bio.pairwise2 is deprecated since Biopython 1.80 and will be removed. PairwiseAligner is faster, supports substitution matrices directly, and returns Alignment objects with coordinate arrays.

  4. Close Entrez handles immediately after reading. Handles are HTTP connections; leaving them open risks timeouts and resource exhaustion.

    handle = Entrez.efetch(db="nucleotide", id="NM_007294", rettype="gb", retmode="text")
    record = SeqIO.read(handle, "genbank")
    handle.close()    # do not skip this
    
  5. Prefer local BLAST for batch searches. Remote NCBIWWW.qblast() is suitable for ad hoc queries but can queue for minutes on NCBI servers. For screening >100 sequences, build a local BLAST+ database with makeblastdb and call blastp/blastn via subprocess with -outfmt 5 (XML) for NCBIXML parsing.

  6. Root trees before measuring distances. Phylo.distance() measures the sum of branch lengths along the path between two nodes. On an unrooted tree, the path is still unique, but midpoint-rooting or outgroup-rooting makes biological sense for visualizations and clade assertions.

  7. Strip alignment gaps before building SeqRecord collections. BLAST and alignment results may include gap characters (-). Seq operations like .translate() will raise errors on gapped sequences; strip with seq.replace("-", "") or use ungap().

Common Recipes

Recipe: Batch Entrez Fetch with History Server

When to use: Downloading more than 500 records from NCBI — avoids URL length limits and keeps search results server-side.

from Bio import Entrez, SeqIO
import time

Entrez.email = "your.email@example.com"

# Search and store results on NCBI history server
handle = Entrez.esearch(db="nucleotide", term="16S rRNA[Gene] AND Bacteria[Organism]",
                        retmax=500, usehistory="y")
results = Entrez.read(handle); handle.close()
webenv = results["WebEnv"]
query_key = results["QueryKey"]
count = int(results["Count"])
print(f"Total: {count} sequences")

# Fetch in batches of 200
batch_size = 200
all_records = []
for start in range(0, min(count, 1000), batch_size):
    handle = Entrez.efetch(
        db="nucleotide", rettype="fasta", retmode="text",
        retstart=start, retmax=batch_size,
        webenv=webenv, query_key=query_key
    )
    batch = list(SeqIO.parse(handle, "fasta"))
    handle.close()
    all_records.extend(batch)
    print(f"  Fetched {len(all_records)}/{min(count, 1000)}")
    time.sleep(0.5)

SeqIO.write(all_records, "16S_sequences.fasta", "fasta")
print(f"Saved {len(all_records)} sequences")

Recipe: Parse GFF3 with Sequence Features

When to use: Extract gene/CDS sequences from a GFF3 annotation paired with a reference FASTA.

from Bio import SeqIO

# Load genome FASTA into indexed dict
genome = SeqIO.to_dict(SeqIO.parse("genome.fasta", "fasta"))

# Parse GFF3 manually (Biopython does not have a native GFF3 parser;
# use the gffutils package for complex queries, or parse directly for simple cases)
genes = []
with open("annotation.gff3") as fh:
    for line in fh:
        if line.startswith("#") or not line.strip():
            continue
        fields = line.strip().split("\t")
        if len(fields) < 9 or fields[2] != "CDS":
            continue
        chrom, _, feat_type, start, end, _, strand, _, attrs = fields
        start, end = int(start) - 1, int(end)   # GFF3 is 1-based
        if chrom not in genome:
            continue
        seq = genome[chrom].seq[start:end]
        if strand == "-":
            seq = seq.reverse_complement()
        attr_dict = dict(kv.split("=") for kv in attrs.strip().split(";") if "=" in kv)
        gene_id = attr_dict.get("gene_id", attr_dict.get("ID", "unknown"))
        genes.append((gene_id, seq, strand))

print(f"Extracted {len(genes)} CDS features")
for gid, seq, strand in genes[:3]:
    print(f"  {gid} ({strand}): {len(seq)} bp — protein: {seq.translate(to_stop=True)[:8]}...")

Recipe: Compute a Pairwise Identity Matrix from a Multiple Alignment

When to use: Quickly assess sequence diversity within an alignment file (FASTA, PHYLIP, Clustal) and identify outlier sequences.

from Bio import AlignIO
import numpy as np

alignment = AlignIO.read("aligned_sequences.fasta", "fasta")
n = len(alignment)
names = [rec.id for rec in alignment]
length = alignment.get_alignment_length()

# Build identity matrix
identity_matrix = np.zeros((n, n))
for i in range(n):
    for j in range(n):
        if i == j:
            identity_matrix[i, j] = 1.0
            continue
        matches = sum(
            a == b and a != "-"
            for a, b in zip(str(alignment[i].seq), str(alignment[j].seq))
        )
        aligned_cols = sum(a != "-" and b != "-"
                           for a, b in zip(str(alignment[i].seq), str(alignment[j].seq)))
        identity_matrix[i, j] = matches / aligned_cols if aligned_cols else 0.0

print("Pairwise identity matrix:")
print(f"{'':20s} " + "  ".join(f"{n[:8]:>8s}" for n in names))
for i, name in enumerate(names):
    row = "  ".join(f"{identity_matrix[i,j]*100:8.1f}" for j in range(n))
    print(f"{name[:20]:20s} {row}")

# Find most divergent pair
min_id = np.min(identity_matrix[identity_matrix > 0])
idx = np.unravel_index(np.argmin(np.where(identity_matrix > 0, identity_matrix, 1)), identity_matrix.shape)
print(f"\nMost divergent pair: {names[idx[0]]} vs {names[idx[1]]} ({min_id*100:.1f}%)")

Troubleshooting

ProblemCauseSolution
urllib.error.HTTPError: 429 Too Many RequestsExceeding NCBI rate limit (3 req/s)Add time.sleep(0.4) between calls, or register a free API key at https://www.ncbi.nlm.nih.gov/account/ for 10 req/s
RuntimeError: Too many requests were made without pausingNCBI detects rapid-fire requestsSet Entrez.email (required), reduce loop frequency, batch IDs into comma-joined strings in a single efetch call
Bio.Application.ApplicationError: blastall not foundLegacy blastall replaced by BLAST+Replace NcbiblastpCommandline with subprocess.run(["blastp", ...]) and BLAST+ tools
AttributeError: 'PairwiseAligner' object has no attribute 'align'Biopython < 1.78Upgrade: pip install --upgrade biopython (PairwiseAligner requires ≥ 1.72; stable from 1.78)
Bio.pairwise2 deprecation warningUsing the legacy pairwise2 APIReplace with from Bio.Align import PairwiseAligner (removed in Biopython 1.84+)
ValueError: Sequence contains letters not in the alphabetNon-standard characters (e.g., N, ambiguity codes) in sequence before translate()Strip or replace ambiguous bases; use seq.translate(table=1) which handles ambiguous codons
Empty BLAST results / StopIteration from NCBIXML.read()Empty or malformed XML; network timeout; query too shortCheck query sequence length (>10 aa recommended); switch from .read() to .parse() and check blast_record.alignments length
Phylo.draw() hangs or produces blank plotMissing matplotlib backendCall import matplotlib; matplotlib.use("Agg") before importing Phylo for headless environments
TreeConstruction distance matrix dimension mismatchAlignment contains duplicate IDsDeduplicate SeqRecord IDs before constructing the alignment: {r.id: r for r in records}.values()

Related Skills

  • biopython-molecular-biology — restriction digestion, PCR primer design, protein structure analysis (Bio.PDB), molecular weight and Tm calculations; complement to this skill
  • pysam-genomic-files — SAM/BAM/CRAM alignment file access, pileup, and region queries; use when working with read alignments
  • scikit-bio — ecological diversity statistics (UniFrac, Bray-Curtis), ordination (PCoA), and microbiome analysis; more specialized than Biopython's phylogenetics for diversity analyses
  • gget-genomic-databases — one-liner gene lookups, BLAST, and database queries without setting up an Entrez pipeline
  • etetoolkit — advanced phylogenetic tree visualization, annotation with NCBI taxonomy, and publication-quality tree rendering

References

ARCHS4 Database

Overview

ARCHS4 (All RNA-seq and ChIP-seq Sample and Signature Search) is a resource of uniformly aligned and processed human and mouse RNA-seq data from NCBI GEO and SRA, covering 1 million+ samples. The REST API at https://maayanlab.cloud/archs4/api/ provides gene-level expression profiles, z-score normalized tissue expression, co-expression networks, and sample metadata search — all without authentication. Large-scale bulk queries can also use the downloadable HDF5 expression matrices.

When to Use

  • Retrieving tissue-specific or cell-type-specific expression z-scores for a gene of interest across hundreds of tissue types
  • Finding genes co-expressed with a query gene (co-expression network construction or guilt-by-association analysis)
  • Searching for RNA-seq samples by tissue, disease, or metadata keyword to identify candidate datasets for reanalysis
  • Comparing expression profiles of multiple genes across tissues to prioritize candidates for wet-lab follow-up
  • Accessing uniformly processed gene expression matrices (HDF5 format) for large-scale cross-study analysis
  • Validating differential expression results by checking whether a gene's expression direction matches population-level tissue profiles
  • Use omics-plotting SKILL to render bar charts and expression heatmaps from the query results
  • For variant-level population allele frequencies use gnomad-database; ARCHS4 provides expression evidence only
  • For Enrichr pathway enrichment from a gene list use gget-genomic-databases (gget enrichr); ARCHS4 is for expression lookups

Prerequisites

  • Python packages: requests, pandas, matplotlib, seaborn
  • Data requirements: gene symbols (HGNC format, e.g., TP53, BRCA1); sample GEO/SRA IDs for direct sample queries
  • Environment: internet connection; no API key or account required
  • Rate limits: ~10 requests/second; add time.sleep(0.1) between sequential gene queries to avoid throttling
pip install requests pandas matplotlib seaborn

Quick Start

import requests

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def archs4_get(endpoint: str, params: dict = None) -> dict:
    """Send a GET request to the ARCHS4 API and return parsed JSON."""
    r = requests.get(f"{ARCHS4_BASE}/{endpoint}", params=params, timeout=30)
    r.raise_for_status()
    return r.json()

# Quick check: top tissues expressing TP53
data = archs4_get("meta/genes/TP53/zscore")
tissues = data.get("values", [])
print(f"TP53 tissue expression entries: {len(tissues)}")
top5 = sorted(tissues, key=lambda x: x.get("zscore", 0), reverse=True)[:5]
for t in top5:
    print(f"  {t['tissue']:<40}  z={t['zscore']:.2f}")
# TP53 tissue expression entries: 200
#   thymus                                   z=2.81
#   testis                                   z=2.44

Core API

Query 1: Gene Expression Z-Scores Across Tissues

Retrieve z-score normalized expression for a gene across all available tissue types. Z-scores are computed per-sample relative to the population distribution; positive values indicate above-average expression.

import requests
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def get_gene_tissue_zscore(gene_symbol: str, species: str = "human") -> pd.DataFrame:
    """Return tissue z-score expression profile for a gene.

    Parameters
    ----------
    gene_symbol : str
        HGNC gene symbol (e.g., 'TP53').
    species : str
        'human' or 'mouse' (default: 'human').
    """
    endpoint = f"meta/genes/{gene_symbol}/zscore"
    r = requests.get(
        f"{ARCHS4_BASE}/{endpoint}",
        params={"species": species},
        timeout=30
    )
    r.raise_for_status()
    data = r.json()
    records = data.get("values", [])
    df = pd.DataFrame(records)
    return df.sort_values("zscore", ascending=False).reset_index(drop=True)

df = get_gene_tissue_zscore("MYC")
print(f"MYC tissue z-scores: {len(df)} tissue types")
print(df[["tissue", "zscore"]].head(10).to_string(index=False))
# MYC tissue z-scores: 200
#                     tissue  zscore
#                      colon    3.12
#             small intestine    2.98
#                    placenta    2.74
# Query mouse tissues for a gene
df_mouse = get_gene_tissue_zscore("Myc", species="mouse")
print(f"Mouse Myc: top 5 tissues")
print(df_mouse[["tissue", "zscore"]].head(5).to_string(index=False))

Query 2: Co-expressed Genes

Find genes whose expression is most correlated with a query gene across all ARCHS4 samples. Useful for identifying pathway partners, regulators, or candidate targets.

import requests
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def get_coexpressed_genes(gene_symbol: str, top_n: int = 50,
                           species: str = "human") -> pd.DataFrame:
    """Return genes co-expressed with the query gene.

    Parameters
    ----------
    gene_symbol : str
        HGNC gene symbol.
    top_n : int
        Number of correlated genes to return (default: 50).
    species : str
        'human' or 'mouse' (default: 'human').
    """
    r = requests.get(
        f"{ARCHS4_BASE}/meta/genes/{gene_symbol}/correlations",
        params={"species": species, "limit": top_n},
        timeout=30
    )
    r.raise_for_status()
    data = r.json()
    records = data.get("values", [])
    df = pd.DataFrame(records)
    return df.sort_values("correlation", ascending=False).reset_index(drop=True)

coexp = get_coexpressed_genes("PCNA", top_n=20)
print(f"Top co-expressed genes with PCNA (n={len(coexp)}):")
print(coexp[["gene", "correlation"]].head(10).to_string(index=False))
# Top co-expressed genes with PCNA (n=20):
#   gene  correlation
#   RFC4         0.91
#   RFC2         0.89
#   MCM6         0.87
# Extract gene list for downstream enrichment
gene_list = coexp["gene"].tolist()
print(f"Co-expression gene list: {gene_list[:10]}")
# Pass gene_list to Enrichr or pathway analysis tools

Query 3: Sample Search

Search for RNA-seq samples by metadata keyword (tissue, disease condition, cell type, treatment). Returns GEO/SRA sample identifiers with metadata fields.

import requests
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def search_samples(keyword: str, species: str = "human",
                   limit: int = 100) -> pd.DataFrame:
    """Search ARCHS4 samples by metadata keyword.

    Parameters
    ----------
    keyword : str
        Search term (e.g., 'breast cancer', 'liver', 'HeLa').
    species : str
        'human' or 'mouse'.
    limit : int
        Maximum number of samples to return.
    """
    r = requests.get(
        f"{ARCHS4_BASE}/samples/search",
        params={"query": keyword, "species": species, "limit": limit},
        timeout=30
    )
    r.raise_for_status()
    data = r.json()
    records = data.get("samples", [])
    return pd.DataFrame(records)

samples = search_samples("pancreatic cancer", limit=50)
print(f"Samples matching 'pancreatic cancer': {len(samples)}")
if len(samples) > 0:
    print(samples[["sample_id", "series_id", "title"]].head(5).to_string(index=False))
# Samples matching 'pancreatic cancer': 50
#   sample_id  series_id  title
#   GSM2345678  GSE123456  Pancreatic ductal adenocarcinoma - sample 1

Query 4: Gene-Level Metadata Summary

Retrieve summary statistics and metadata for a gene including the number of samples expressing it, expression percentile, and available annotation.

import requests

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def get_gene_metadata(gene_symbol: str, species: str = "human") -> dict:
    """Return metadata and expression summary for a gene."""
    r = requests.get(
        f"{ARCHS4_BASE}/meta/genes/{gene_symbol}",
        params={"species": species},
        timeout=30
    )
    r.raise_for_status()
    return r.json()

meta = get_gene_metadata("GAPDH")
print(f"Gene: {meta.get('gene_symbol', 'N/A')}")
print(f"Species: {meta.get('species', 'N/A')}")
print(f"Ensembl ID: {meta.get('ensembl_gene_id', 'N/A')}")
print(f"Description: {meta.get('description', 'N/A')[:80]}")
# Compare metadata for a panel of housekeeping genes
import time

housekeeping = ["GAPDH", "ACTB", "B2M", "HPRT1", "RPLP0"]
for gene in housekeeping:
    meta = get_gene_metadata(gene)
    print(f"  {gene:<8}  {meta.get('ensembl_gene_id', 'N/A')}")
    time.sleep(0.1)

Query 5: Visualization — Tissue Expression Barplot

Fetch the top-tissue z-scores, then read skills/data-visualization/omics-plotting/SKILL.md and follow its "Box / Violin / Bar" recipe on the exported CSV (→ figures/BRCA1_tissue_expression.png).

import requests
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def tissue_zscores(gene_symbol: str, top_n: int = 20, species: str = "human") -> pd.DataFrame:
    """Fetch top tissue z-scores for a gene and save for plotting."""
    r = requests.get(
        f"{ARCHS4_BASE}/meta/genes/{gene_symbol}/zscore",
        params={"species": species},
        timeout=30,
    )
    r.raise_for_status()
    df = pd.DataFrame(r.json().get("values", [])).sort_values("zscore", ascending=False).head(top_n)
    df.to_csv(f"{gene_symbol}_tissue_zscores.csv", index=False)
    return df

df = tissue_zscores("BRCA1", top_n=15)
print(f"Top tissues for BRCA1: {len(df)} -> BRCA1_tissue_zscores.csv")
# Render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Box / Violin / Bar" recipe (horizontal bar of zscore by tissue).

Query 6: HDF5 Bulk Data Access

Download or stream from ARCHS4's precomputed HDF5 expression matrices for large-scale cross-sample analysis. The HDF5 files contain gene × sample count matrices for human and mouse.

import requests

# HDF5 files are available for bulk download from the ARCHS4 data portal
# URL pattern: https://maayanlab.cloud/archs4/download#expression
# Human gene-level: human_gene_v2.6.h5
# Mouse gene-level: mouse_gene_v2.6.h5

def get_h5_download_urls() -> dict:
    """Return download URLs for ARCHS4 HDF5 expression matrices."""
    base = "https://maayanlab.cloud/archs4"
    return {
        "human_gene": f"{base}/files/human_gene_v2.6.h5",
        "mouse_gene": f"{base}/files/mouse_gene_v2.6.h5",
        "human_transcript": f"{base}/files/human_transcript_v2.6.h5",
        "mouse_transcript": f"{base}/files/mouse_transcript_v2.6.h5",
    }

urls = get_h5_download_urls()
for key, url in urls.items():
    print(f"  {key:<22}  {url}")

# To work with a downloaded HDF5 file:
try:
    import h5py
    import numpy as np

    h5_path = "human_gene_v2.6.h5"   # after download

    def extract_gene_from_h5(h5_path: str, gene_symbol: str,
                              n_samples: int = 1000) -> dict:
        """Extract expression values for a gene from the HDF5 matrix."""
        with h5py.File(h5_path, "r") as f:
            genes = [g.decode() for g in f["meta"]["genes"]["gene_symbol"][:]]
            if gene_symbol not in genes:
                raise ValueError(f"{gene_symbol} not found in HDF5")
            idx = genes.index(gene_symbol)
            expr = f["data"]["expression"][idx, :n_samples]
            sample_ids = [s.decode() for s in f["meta"]["samples"]["geo_accession"][:n_samples]]
        return {"gene": gene_symbol, "expression": expr, "sample_ids": sample_ids}

    result = extract_gene_from_h5(h5_path, "TP53", n_samples=500)
    print(f"TP53 expression: mean={result['expression'].mean():.2f},"
          f" max={result['expression'].max():.2f} (n={len(result['expression'])} samples)")
except ImportError:
    print("h5py not installed. Install with: pip install h5py")
except FileNotFoundError:
    print("HDF5 file not downloaded yet. Use the URLs above to download first.")

Key Concepts

Z-Score Normalization

ARCHS4 reports gene expression as z-scores computed relative to all samples for that gene. A z-score of 0 means expression at the population mean; a z-score of 2.0 means expression 2 standard deviations above the mean. Z-scores are more interpretable across datasets than raw counts because they account for library size differences and batch effects introduced by uniform alignment across studies.

# Example: Positive z-score = above-average expression for that gene
# z > 2.0 → top ~2.5% of samples for that gene
# z < -2.0 → bottom ~2.5% of samples for that gene
# Use absolute z-score thresholds consistently when comparing across genes

HDF5 vs REST API

Access methodBest forLimitations
REST API (/zscore, /correlations)Quick single-gene queries, explorationAggregated profiles only, no per-sample access
REST API (/samples/search)Discovering relevant datasetsReturns metadata, not expression values
HDF5 downloadBulk analysis, custom co-expression, MLRequires 30–60 GB disk; download once

Species and Gene Symbol Conventions

ARCHS4 indexes human samples using HGNC gene symbols (uppercase, e.g., TP53) and mouse samples using MGI symbols (first letter uppercase, e.g., Trp53). The species parameter accepts "human" or "mouse". Mixed-case or ensemble IDs will return empty results.

Common Workflows

Workflow 1: Multi-Gene Tissue Expression Heatmap

Goal: Compare tissue expression profiles of a gene panel and visualize as a heatmap to identify tissue-specific vs ubiquitous expression patterns.

import requests, time
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

gene_panel = ["MYC", "TP53", "BRCA1", "EGFR", "KRAS", "CDK4"]
top_n_tissues = 25

def get_tissue_zscores(gene: str) -> pd.Series:
    r = requests.get(
        f"{ARCHS4_BASE}/meta/genes/{gene}/zscore",
        params={"species": "human"},
        timeout=30
    )
    r.raise_for_status()
    records = r.json().get("values", [])
    df = pd.DataFrame(records).set_index("tissue")["zscore"]
    return df

# Build expression matrix (genes × tissues)
all_data = {}
for gene in gene_panel:
    try:
        all_data[gene] = get_tissue_zscores(gene)
        print(f"  Fetched {gene}")
    except Exception as e:
        print(f"  Warning: {gene} failed — {e}")
    time.sleep(0.1)

matrix = pd.DataFrame(all_data).T   # genes × tissues
# Select top tissues by max absolute z-score
tissue_importance = matrix.abs().max(axis=0).sort_values(ascending=False)
top_tissues = tissue_importance.head(top_n_tissues).index
matrix_subset = matrix[top_tissues]

# Save the gene × tissue matrix; render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Expression heatmap"
# recipe (RdBu_r, center=0) -> figures/archs4_panel_heatmap.png
matrix_subset.to_csv("archs4_panel_matrix.csv")
print(f"Panel matrix ready: {matrix_subset.shape} -> archs4_panel_matrix.csv")

Workflow 2: Co-expression Network Seed Expansion

Goal: Start from a seed gene, retrieve co-expressed partners, then query their co-expressed genes in turn to build a two-hop co-expression neighborhood.

import requests, time
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def get_coexp(gene: str, top_n: int = 20, species: str = "human") -> list:
    r = requests.get(
        f"{ARCHS4_BASE}/meta/genes/{gene}/correlations",
        params={"species": species, "limit": top_n},
        timeout=30
    )
    r.raise_for_status()
    return [rec["gene"] for rec in r.json().get("values", [])]

seed_gene = "PCNA"
min_correlation = 0.80

# Hop 1: direct co-expressed partners
hop1_genes = get_coexp(seed_gene, top_n=30)
print(f"Hop 1 partners of {seed_gene}: {len(hop1_genes)}")
time.sleep(0.1)

# Hop 2: co-expressed genes of each partner
edges = set()
for gene in hop1_genes[:10]:   # limit for demonstration
    partners = get_coexp(gene, top_n=20)
    for partner in partners:
        if partner != seed_gene:
            edges.add((gene, partner))
    time.sleep(0.1)

# Summarize the network
network_df = pd.DataFrame(list(edges), columns=["source", "target"])
hub_counts = network_df["source"].value_counts()
print(f"\nTwo-hop network: {len(edges)} edges")
print(f"Top hub genes:")
print(hub_counts.head(5))

network_df.to_csv(f"{seed_gene}_coexp_network.csv", index=False)
print(f"\nSaved {seed_gene}_coexp_network.csv")

Workflow 3: Sample Discovery and Dataset Summary

Goal: Search for samples by disease keyword, summarize how many GEO series are available, and export sample metadata for downstream reanalysis selection.

import requests, time
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def search_and_summarize(keyword: str, species: str = "human",
                          limit: int = 200) -> pd.DataFrame:
    """Search samples and return a tidy metadata DataFrame."""
    r = requests.get(
        f"{ARCHS4_BASE}/samples/search",
        params={"query": keyword, "species": species, "limit": limit},
        timeout=30
    )
    r.raise_for_status()
    records = r.json().get("samples", [])
    return pd.DataFrame(records)

keyword = "colorectal cancer"
df = search_and_summarize(keyword, limit=150)
print(f"Samples matching '{keyword}': {len(df)}")

if len(df) > 0:
    # Summarize by GEO series
    series_counts = df["series_id"].value_counts()
    print(f"\nTop GEO series (by sample count):")
    print(series_counts.head(8).to_string())

    # Export sample list
    df.to_csv(f"{keyword.replace(' ', '_')}_samples.csv", index=False)
    print(f"\nSaved {keyword.replace(' ', '_')}_samples.csv ({len(df)} samples)")
    print(f"Unique GEO series: {df['series_id'].nunique()}")

Key Parameters

ParameterEndpointDefaultRange / OptionsEffect
speciesAll gene endpoints"human""human", "mouse"Selects the species-specific sample index
limit/correlations, /samples/search1001–500Number of results returned
gene_symbol (path)/meta/genes/{gene}/zscore, /correlations—HGNC symbol (human) or MGI symbol (mouse)Query gene; case-sensitive
query/samples/search—free-text stringMetadata keyword search across title, tissue, source fields
offset/samples/search0integerPagination offset for large result sets
correlation (response field)/correlations—-1.0–1.0Pearson correlation coefficient; filter > 0.7 for high co-expression
zscore (response field)/zscore—continuous floatExpression z-score; > 2.0 = high expression
page_size (HDF5)HDF5 sliceallany integerNumber of samples to extract per read from HDF5

Best Practices

  1. Use z-score thresholds consistently: Because z-scores are gene-specific, a z-score of 2.0 for a ubiquitous gene (GAPDH) and a tissue-restricted gene (TTR, liver) have different interpretive meaning. Always annotate which gene you are comparing and the tissue background.

  2. Sleep between batch queries: ARCHS4 enforces a soft rate limit of ~10 requests/second. Add time.sleep(0.1) between sequential gene queries to avoid 429 Too Many Requests errors.

  3. Download HDF5 for large-scale analyses: For queries covering 50+ genes or requiring per-sample expression values, the REST API is impractical. Download the HDF5 file once and use h5py slicing for fast matrix access; this avoids hitting rate limits and is 100× faster for bulk extraction.

  4. Match gene symbol conventions by species: Human queries require HGNC uppercase symbols (e.g., TP53); mouse queries require MGI-style symbols (e.g., Trp53). Using the wrong case returns empty results without an error.

  5. Validate co-expression findings across datasets: ARCHS4 co-expression aggregates across all tissue types. A high correlation may be driven by a single tissue or study. Cross-check with tissue-specific queries or manually inspect the top contributing GEO series.

Common Recipes

Recipe: Quick Tissue Specificity Check

When to use: Rapidly determine whether a gene is broadly expressed (housekeeping) or tissue-restricted before designing experiments.

import requests

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def tissue_specificity_summary(gene_symbol: str) -> None:
    """Print a summary of high and low expression tissues for a gene."""
    r = requests.get(
        f"{ARCHS4_BASE}/meta/genes/{gene_symbol}/zscore",
        params={"species": "human"},
        timeout=30
    )
    r.raise_for_status()
    records = r.json().get("values", [])
    zscores = [rec["zscore"] for rec in records if rec.get("zscore") is not None]
    top_high = sorted(records, key=lambda x: x.get("zscore", 0), reverse=True)[:5]
    top_low = sorted(records, key=lambda x: x.get("zscore", float("inf")))[:3]
    print(f"\n{gene_symbol} — {len(zscores)} tissues")
    print(f"  Range: [{min(zscores):.2f}, {max(zscores):.2f}]  "
          f"Mean: {sum(zscores)/len(zscores):.2f}")
    print("  High expression:")
    for t in top_high:
        print(f"    {t['tissue']:<35}  z={t['zscore']:.2f}")
    print("  Low expression:")
    for t in top_low:
        print(f"    {t['tissue']:<35}  z={t['zscore']:.2f}")

tissue_specificity_summary("TTR")   # Transthyretin — liver-specific

Recipe: Batch Gene Co-Expression Table

When to use: Generate a pairwise correlation table for a gene panel from a list of differentially expressed genes.

import requests, time
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

def batch_coexpr_table(gene_list: list, top_n: int = 10) -> pd.DataFrame:
    """For each gene in gene_list, return its top co-expressed genes."""
    rows = []
    for gene in gene_list:
        try:
            r = requests.get(
                f"{ARCHS4_BASE}/meta/genes/{gene}/correlations",
                params={"species": "human", "limit": top_n},
                timeout=30
            )
            r.raise_for_status()
            for rec in r.json().get("values", []):
                rows.append({
                    "query_gene": gene,
                    "coexp_gene": rec.get("gene"),
                    "correlation": rec.get("correlation"),
                })
            time.sleep(0.1)
        except Exception as e:
            print(f"Warning: {gene} skipped — {e}")
    return pd.DataFrame(rows)

deg_list = ["MYC", "CCND1", "CDK4", "RB1", "E2F1"]
coexp_table = batch_coexpr_table(deg_list, top_n=10)
print(f"Co-expression entries: {len(coexp_table)}")
print(coexp_table.groupby("query_gene")["coexp_gene"].count())
coexp_table.to_csv("deg_coexpression_table.csv", index=False)
print("Saved deg_coexpression_table.csv")

Recipe: Export Sample IDs for GEO Download

When to use: Identify relevant GEO accessions to download raw count matrices for a meta-analysis.

import requests
import pandas as pd

ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"

keyword = "glioblastoma"
r = requests.get(
    f"{ARCHS4_BASE}/samples/search",
    params={"query": keyword, "species": "human", "limit": 200},
    timeout=30
)
r.raise_for_status()
samples = pd.DataFrame(r.json().get("samples", []))
if len(samples) > 0:
    # Get unique GEO series accessions
    series = samples["series_id"].dropna().unique()
    print(f"Unique GEO series for '{keyword}': {len(series)}")
    for s in series[:10]:
        n = (samples["series_id"] == s).sum()
        print(f"  {s}  ({n} samples)")
    # Export series list for GEO download script
    pd.Series(series, name="geo_series").to_csv(
        f"{keyword}_geo_series.txt", index=False
    )
    print(f"\nSaved {keyword}_geo_series.txt")

Troubleshooting

ProblemCauseSolution
HTTP 404 for gene queryGene symbol not found in ARCHS4 indexVerify HGNC symbol spelling; check species parameter matches gene convention (human: uppercase, mouse: first-letter-upper)
HTTP 429 Too Many RequestsExceeded ~10 req/s rate limitAdd time.sleep(0.1) between requests; for batch queries use a 0.5 s delay
Empty values list in z-score responseGene is not expressed in any indexed tissue, or wrong speciesSwitch species; verify gene is protein-coding and has GEO coverage
Empty samples list from searchKeyword not matched in metadata fieldsTry broader or alternative keywords (e.g., "liver" instead of "hepatic")
HDF5 gene not foundSymbol mismatch between HDF5 version and queryCheck available genes in f["meta"]["genes"]["gene_symbol"][:]; try Ensembl ID or alias
requests.exceptions.TimeoutSlow API response under loadIncrease timeout=60; retry with exponential backoff
Z-scores all near zeroGene has very low or absent expression across tissuesCheck the gene's expression in raw counts; the gene may be non-coding or very lowly expressed

Related Skills

  • gnomad-database — Population variant frequencies; use after ARCHS4 to identify variants in highly expressed genes
  • gget-genomic-databases — Enrichr pathway enrichment for ARCHS4 co-expression gene lists (gget enrichr)
  • pydeseq2-differential-expression — Differential expression analysis on bulk RNA-seq; ARCHS4 HDF5 matrices can serve as reference cohorts

References

BioServices Multi-Database Access

Overview

BioServices provides a unified Python interface to 40+ bioinformatics web services including UniProt, KEGG, ChEMBL, ChEBI, PubChem, UniChem, PSICQUIC, QuickGO, and BLAST. Each service is accessed through a consistent object-oriented API with built-in caching, rate limiting, and output format handling.

When to Use

  • Querying protein information from UniProt (search, retrieve, ID mapping)
  • Discovering KEGG pathways and extracting gene/interaction networks
  • Cross-referencing compounds across ChEMBL, ChEBI, PubChem, and KEGG
  • Running BLAST sequence similarity searches against UniProtKB
  • Mapping identifiers between biological databases (UniProt, Ensembl, KEGG, RefSeq, PDB)
  • Retrieving Gene Ontology annotations via QuickGO
  • Finding protein-protein interactions via PSICQUIC (IntAct, MINT, BioGRID)
  • Batch converting thousands of biological identifiers with error handling
  • For single-database deep queries → use gget (Ensembl), pubchempy (PubChem), or chembl-database-bioactivity skill
  • For pathway visualization → use pathway analysis tools (Cytoscape, NetworkX) after retrieving data with bioservices

Prerequisites

pip install bioservices
# Optional: pandas for tabular output, matplotlib for visualization
pip install pandas matplotlib

API Rate Limits: Most services have rate limits. bioservices handles basic throttling internally, but for batch operations add explicit delays:

  • UniProt mapping: ~1 request/second for batch jobs
  • KEGG: 10 requests/second (be conservative with pathway parsing)
  • ChEMBL/ChEBI: 5-10 requests/second
  • BLAST: 1 job at a time (async polling, ~30-300s per job)

Quick Start

from bioservices import UniProt, KEGG
import time

# Protein lookup
u = UniProt(verbose=False)
result = u.search("ABL1_HUMAN", frmt="tsv", columns="accession,gene_names,organism_name,length")
print(result[:200])

# Pathway discovery
k = KEGG(verbose=False)
pathways = k.get_pathway_by_gene("hsa:25", "hsa")  # ABL1
print(f"ABL1 participates in {len(pathways)} pathways")
for pid, name in list(pathways.items())[:3]:
    print(f"  {pid}: {name}")

Core API

1. Protein Analysis (UniProt)

from bioservices import UniProt
u = UniProt(verbose=False)

# Search by protein name or gene
result = u.search("BRCA1 AND organism_id:9606", frmt="tsv",
                  columns="accession,gene_names,protein_name,length,go_p")
print(result[:300])

# Retrieve full entry
entry = u.retrieve("P38398", frmt="txt")  # Swiss-Prot flat file
fasta = u.retrieve("P38398", frmt="fasta")
print(fasta[:200])
# ID mapping: gene names → UniProt accessions
result = u.mapping(fr="Gene_Name", to="UniProtKB", query="BRCA1 TP53 ABL1", taxId=9606)
print(f"Mapped {len(result['results'])} entries")
for r in result['results']:
    print(f"  {r['from']} → {r['to']['primaryAccession']}")

2. Pathway Discovery (KEGG)

from bioservices import KEGG
k = KEGG(verbose=False)

# List pathways for an organism
pathways = k.pathwayIds  # All reference pathways
human_pathways = k.list("pathway", "hsa")
print(f"Human pathways: {len(human_pathways.strip().splitlines())}")

# Get pathway details
pathway_data = k.get("hsa04110")  # Cell cycle
parsed = k.parse(pathway_data)
print(f"Pathway: {parsed.get('NAME', 'Unknown')}")
print(f"Genes: {len(parsed.get('GENE', {}))}")
# KGML parsing for interaction networks
from bioservices import KEGG
k = KEGG(verbose=False)

kgml = k.get("hsa04110", "kgml")  # XML pathway representation
# Parse KGML for entries and relations
import xml.etree.ElementTree as ET
root = ET.fromstring(kgml)
entries = root.findall("entry")
relations = root.findall("relation")
print(f"Entries: {len(entries)}, Relations: {len(relations)}")

# Extract interaction types
from collections import Counter
rel_types = Counter()
for rel in relations:
    for subtype in rel.findall("subtype"):
        rel_types[subtype.get("name")] += 1
print(f"Interaction types: {dict(rel_types)}")

3. Compound Databases (ChEMBL, ChEBI, UniChem, PubChem)

from bioservices import ChEMBL, ChEBI, UniChem
import time

# ChEMBL compound lookup
chembl = ChEMBL(verbose=False)
result = chembl.get_molecule("CHEMBL25")  # Aspirin
print(f"Name: {result['pref_name']}")
print(f"MW: {result['molecule_properties']['full_mwt']}")
print(f"SMILES: {result['molecule_structures']['canonical_smiles']}")

time.sleep(0.2)

# ChEBI entity lookup
chebi = ChEBI(verbose=False)
entity = chebi.getCompleteEntity("CHEBI:15365")  # Aspirin
print(f"ChEBI Name: {entity.chebiAsciiName}")
print(f"Formula: {entity.formulae[0].data if entity.formulae else 'N/A'}")
# Cross-database compound mapping via UniChem
from bioservices import UniChem
uc = UniChem()

# Map ChEMBL ID to other databases
# Source IDs: 1=ChEMBL, 2=DrugBank, 3=PDB, 4=IUPHAR, 7=ChEBI, 22=PubChem
mappings = uc.get_mapping("CHEMBL25", 1)  # From ChEMBL
for m in mappings[:5]:
    print(f"  Source {m['src_id']}: {m['src_compound_id']}")

4. Sequence Analysis (BLAST)

from bioservices import NCBIblast
import time

blast = NCBIblast(verbose=False)

sequence = ">query\nMKTAYIAKQRQISFVKSHFSRQLE..."  # Truncated for brevity
job_id = blast.run(
    program="blastp",
    database="uniprotkb_swissprot",
    sequence=sequence,
    stype="protein",
    email="user@example.com"  # Required by NCBI
)
print(f"Job submitted: {job_id}")

# Poll for results (async)
while blast.getStatus(job_id) == "RUNNING":
    time.sleep(10)
    print("Waiting...")

result_types = blast.getResultTypes(job_id)
alignment = blast.getResult(job_id, "out")  # Text alignment
print(alignment[:500])

5. Identifier Mapping

from bioservices import UniProt
u = UniProt(verbose=False)

# Batch mapping: UniProt → multiple databases
accessions = "P00520 P12931 P04637 P38398"

# UniProt → PDB
result = u.mapping(fr="UniProtKB_AC-ID", to="PDB", query=accessions)
for r in result.get('results', []):
    print(f"  {r['from']} → {r['to']}")

# UniProt → Ensembl
result = u.mapping(fr="UniProtKB_AC-ID", to="Ensembl", query=accessions)
print(f"Mapped {len(result.get('results', []))} entries to Ensembl")
# KEGG-based identifier extraction
from bioservices import KEGG
k = KEGG(verbose=False)

# Get cross-references from a KEGG entry
entry = k.get("hsa:25")  # ABL1
parsed = k.parse(entry)
# Extract database links
for key in ['DBLINKS', 'PATHWAY']:
    if key in parsed:
        print(f"{key}: {parsed[key]}")

6. Gene Ontology & Protein Interactions

from bioservices import QuickGO
go = QuickGO(verbose=False)

# Get GO annotations for a protein
annotations = go.Annotation(geneProductId="UniProtKB:P00520", taxonId="9606")
if hasattr(annotations, 'shape'):  # DataFrame
    print(f"Annotations: {len(annotations)}")
    for aspect in ['biological_process', 'molecular_function', 'cellular_component']:
        subset = annotations[annotations['goAspect'] == aspect]
        print(f"  {aspect}: {len(subset)} terms")
# Protein-protein interactions via PSICQUIC
from bioservices import PSICQUIC
psicquic = PSICQUIC(verbose=False)

# Query IntAct for interactions
interactions = psicquic.query("intact", "identifier:P00520", frmt="tab25")
lines = interactions.strip().split("\n") if interactions else []
print(f"Interactions found: {len(lines)}")
for line in lines[:3]:
    fields = line.split("\t")
    print(f"  {fields[0]} <-> {fields[1]}")  # Interactor A <-> Interactor B

Key Concepts

Service Initialization & Verbosity

All services share a common pattern: Service(verbose=False) suppresses HTTP request logging. Set verbose=True during debugging to see full request/response details.

Output Format Handling

ServiceDefault FormatAvailable FormatsNotes
UniProtXMLtsv, fasta, json, txt, xml, gffUse tsv with columns= for tabular
KEGGTextkgml, imageParse text with k.parse()
ChEMBLJSONJSON onlyDict access on response
ChEBISOAP XMLObject attributesAccess via .chebiAsciiName etc.
NCBIblastTextout, xml, json, idsAsync: submit → poll → retrieve
PSICQUICPSI-MI TABtab25, tab27, xml25, countTab-separated interaction records
QuickGODataFrametsv, jsonPandas DataFrame when available

Common Organism Codes (KEGG)

CodeOrganismTaxonomy ID
hsaHomo sapiens9606
mmuMus musculus10090
rnoRattus norvegicus10116
dmeDrosophila melanogaster7227
sceSaccharomyces cerevisiae559292
ecoEscherichia coli K-1283333
athArabidopsis thaliana3702

UniProt Mapping Database Codes

Common database pairs for u.mapping(fr=, to=):

CategoryDatabase CodeExample
ProteinUniProtKB_AC-IDP00520
GeneGene_Name, GeneIDABL1, 25
StructurePDB2HYY
SequenceRefSeq_Protein, EnsemblNP_005148, ENSG00000097007
PathwayKEGG, Reactomehsa:25, R-HSA-123
OntologyGOGO:0006468

Full mapping database list: see references/identifier_mapping_guide.md.

Common Workflows

Workflow 1: Complete Protein Analysis Pipeline

from bioservices import UniProt, KEGG, QuickGO, PSICQUIC
import time

def analyze_protein(uniprot_id):
    """End-to-end protein analysis: metadata → pathways → GO → interactions."""
    results = {}

    # Step 1: UniProt metadata
    u = UniProt(verbose=False)
    entry = u.retrieve(uniprot_id, frmt="tsv",
                       columns="accession,gene_names,protein_name,length,organism_name")
    results['uniprot'] = entry
    print(f"[1] UniProt entry retrieved")

    # Step 2: KEGG pathways
    k = KEGG(verbose=False)
    mapping = u.mapping(fr="UniProtKB_AC-ID", to="KEGG", query=uniprot_id)
    kegg_ids = [r['to'] for r in mapping.get('results', [])]
    if kegg_ids:
        pathways = k.get_pathway_by_gene(kegg_ids[0], kegg_ids[0][:3])
        results['pathways'] = pathways or {}
        print(f"[2] Found {len(results['pathways'])} pathways")
    time.sleep(0.3)

    # Step 3: GO annotations
    go = QuickGO(verbose=False)
    annotations = go.Annotation(geneProductId=f"UniProtKB:{uniprot_id}")
    results['go_annotations'] = annotations
    print(f"[3] GO annotations retrieved")
    time.sleep(0.3)

    # Step 4: Protein interactions
    psicquic = PSICQUIC(verbose=False)
    interactions = psicquic.query("intact", f"identifier:{uniprot_id}", frmt="tab25")
    lines = interactions.strip().split("\n") if interactions else []
    results['interactions'] = len(lines)
    print(f"[4] Found {len(lines)} interactions")

    return results

# Usage
results = analyze_protein("P00520")  # ABL1

Workflow 2: Cross-Database Compound Search

from bioservices import KEGG, ChEMBL, ChEBI, UniChem
import time

def search_compound(compound_name):
    """Search compound across KEGG → ChEBI → ChEMBL with cross-references."""
    k = KEGG(verbose=False)

    # Step 1: KEGG compound search
    kegg_results = k.find("compound", compound_name)
    if not kegg_results:
        print(f"No KEGG results for '{compound_name}'")
        return None
    kegg_id = kegg_results.strip().split("\t")[0]  # First hit
    kegg_entry = k.parse(k.get(kegg_id))
    print(f"[1] KEGG: {kegg_id} — {kegg_entry.get('NAME', ['Unknown'])[0]}")
    print(f"    Formula: {kegg_entry.get('FORMULA', 'N/A')}")
    time.sleep(0.2)

    # Step 2: ChEBI cross-reference
    dblinks = kegg_entry.get('DBLINKS', {})
    chebi_id = dblinks.get('ChEBI', [''])[0] if isinstance(dblinks, dict) else None
    if chebi_id:
        chebi = ChEBI(verbose=False)
        entity = chebi.getCompleteEntity(f"CHEBI:{chebi_id}")
        print(f"[2] ChEBI: {entity.chebiId} — {entity.chebiAsciiName}")
        time.sleep(0.2)

    # Step 3: ChEMBL via UniChem
    uc = UniChem()
    try:
        chembl_mappings = uc.get_mapping(kegg_id.replace("cpd:", ""), 6)  # 6=KEGG
        if chembl_mappings:
            chembl_id = [m for m in chembl_mappings if m['src_id'] == '1']
            if chembl_id:
                chembl = ChEMBL(verbose=False)
                mol = chembl.get_molecule(chembl_id[0]['src_compound_id'])
                print(f"[3] ChEMBL: {mol['molecule_chembl_id']} — MW: {mol['molecule_properties']['full_mwt']}")
    except Exception as e:
        print(f"[3] ChEMBL mapping failed: {e}")

    return kegg_entry

result = search_compound("aspirin")

Workflow 3: Batch Identifier Conversion

from bioservices import UniProt
import time

def batch_convert_ids(ids, from_db, to_db, chunk_size=100):
    """Convert biological IDs in chunks with error handling."""
    u = UniProt(verbose=False)
    all_results = []
    failed = []

    for i in range(0, len(ids), chunk_size):
        chunk = ids[i:i+chunk_size]
        query = " ".join(chunk)
        try:
            result = u.mapping(fr=from_db, to=to_db, query=query)
            mapped = result.get('results', [])
            all_results.extend(mapped)
            print(f"  Chunk {i//chunk_size + 1}: {len(mapped)} mapped")
        except Exception as e:
            print(f"  Chunk {i//chunk_size + 1} failed: {e}")
            failed.extend(chunk)
        time.sleep(1)  # Rate limit between chunks

    print(f"\nTotal mapped: {len(all_results)}, Failed: {len(failed)}")
    return all_results, failed

# Usage: Gene names → UniProt accessions
gene_list = ["BRCA1", "TP53", "ABL1", "EGFR", "BRAF", "KRAS", "MYC", "PTEN"]
results, failed = batch_convert_ids(gene_list, "Gene_Name", "UniProtKB_AC-ID")
for r in results[:5]:
    print(f"  {r['from']} → {r['to']['primaryAccession']}")

Key Parameters

ParameterFunction/EndpointDefaultRangeEffect
verboseAll servicesTrueboolHTTP request logging
frmtUniProt search/retrievexmltsv/fasta/json/txtOutput format
columnsUniProt search—comma-separatedTSV columns to return
taxIdUniProt mapping—NCBI taxonomy IDOrganism filter for mapping
programNCBIblast—blastp/blastn/blastx/tblastnBLAST program type
databaseNCBIblast—uniprotkb_swissprot/nr/etcTarget database
emailNCBIblast—email addressRequired by NCBI policy
chunk_sizeBatch operations10010-500IDs per batch request
frmtPSICQUIC querytab25tab25/tab27/xml25/countInteraction output format

Best Practices

  1. Always set verbose=False in production — default True floods stdout with HTTP details. Use True only for debugging failed requests

  2. Handle None/empty results gracefully — web services frequently return None, empty strings, or partial data. Always check before parsing:

    result = k.get("hsa04110")
    if result and not isinstance(result, int):  # KEGG returns 404 as int
        parsed = k.parse(result)
    
  3. Use explicit time.sleep() between batch requests — even though bioservices has internal throttling, batch operations benefit from explicit delays (0.2-1.0s) to avoid 429 errors

  4. Cache service objects, not results — create one UniProt() instance and reuse it across calls. Each constructor call initializes a new HTTP session

  5. Anti-pattern — parsing KEGG text manually: Always use k.parse() to parse KEGG entry text into structured dicts. Manual string splitting breaks on multi-line fields

  6. Specify organism in UniProt searches — unfiltered searches return results across all species. Add AND organism_id:9606 for human-specific results

Common Recipes

Recipe 1: Pathway Network Statistics

from bioservices import KEGG
import xml.etree.ElementTree as ET
from collections import Counter

def pathway_stats(organism="hsa", limit=10):
    """Analyze pathway network statistics for an organism."""
    k = KEGG(verbose=False)
    pathway_list = k.list("pathway", organism).strip().split("\n")
    stats = []
    for line in pathway_list[:limit]:
        pid = line.split("\t")[0]
        kgml = k.get(pid, "kgml")
        if not kgml or isinstance(kgml, int):
            continue
        root = ET.fromstring(kgml)
        genes = len([e for e in root.findall("entry") if e.get("type") == "gene"])
        rels = len(root.findall("relation"))
        stats.append({"pathway": pid, "genes": genes, "relations": rels})
    return sorted(stats, key=lambda x: x['relations'], reverse=True)

top = pathway_stats("hsa", limit=5)
for s in top:
    print(f"  {s['pathway']}: {s['genes']} genes, {s['relations']} interactions")

Recipe 2: GO Term Enrichment Preparation

from bioservices import QuickGO, UniProt
import time

def get_go_terms(uniprot_ids, aspect="biological_process"):
    """Collect GO annotations for a list of proteins."""
    go = QuickGO(verbose=False)
    all_terms = {}
    for uid in uniprot_ids:
        try:
            annot = go.Annotation(geneProductId=f"UniProtKB:{uid}", taxonId="9606")
            if hasattr(annot, 'shape') and len(annot) > 0:
                subset = annot[annot['goAspect'] == aspect]
                all_terms[uid] = list(subset['goId'].unique())
        except Exception:
            all_terms[uid] = []
        time.sleep(0.3)
    return all_terms

terms = get_go_terms(["P00520", "P12931", "P04637"])
for uid, go_ids in terms.items():
    print(f"  {uid}: {len(go_ids)} {go_ids[:3]}")

Recipe 3: Multi-Hop Identifier Mapping

from bioservices import UniProt, KEGG
import time

def gene_to_pathways(gene_name, organism="hsa"):
    """Map gene name → UniProt → KEGG → pathways (multi-hop)."""
    u = UniProt(verbose=False)

    # Hop 1: Gene name → UniProt
    result = u.mapping(fr="Gene_Name", to="UniProtKB_AC-ID", query=gene_name, taxId=9606)
    if not result.get('results'):
        return None
    uniprot_id = result['results'][0]['to']['primaryAccession']
    print(f"  {gene_name} → UniProt: {uniprot_id}")
    time.sleep(0.3)

    # Hop 2: UniProt → KEGG
    kegg_result = u.mapping(fr="UniProtKB_AC-ID", to="KEGG", query=uniprot_id)
    if not kegg_result.get('results'):
        return None
    kegg_id = kegg_result['results'][0]['to']
    print(f"  UniProt: {uniprot_id} → KEGG: {kegg_id}")
    time.sleep(0.3)

    # Hop 3: KEGG gene → pathways
    k = KEGG(verbose=False)
    pathways = k.get_pathway_by_gene(kegg_id, organism)
    print(f"  KEGG: {kegg_id} → {len(pathways or {})} pathways")
    return pathways

pathways = gene_to_pathways("BRCA1")
if pathways:
    for pid, name in list(pathways.items())[:5]:
        print(f"    {pid}: {name}")

Troubleshooting

ProblemCauseSolution
ConnectionError on service initNetwork/firewall blockingCheck internet; some services need port 443
UniProt search returns emptyQuery syntax errorUse AND/OR operators; check field names in UniProt docs
KEGG get() returns 404 (int)Invalid pathway/gene IDVerify ID format: hsa:25 (gene), hsa04110 (pathway), cpd:C00001 (compound)
BLAST job timeoutLarge sequence or busy serverIncrease polling interval; try uniprotkb_swissprot (smaller than nr)
UniProt mapping returns no resultsWrong database codesUse exact codes: UniProtKB_AC-ID, Gene_Name, PDB, Ensembl (case-sensitive)
AttributeError on ChEBI entitySOAP response parsingCheck entity is not None; some ChEBI IDs lack certain fields
PSICQUIC returns empty stringNo interactions in databaseTry multiple databases: intact, mint, biogrid
429 Too Many RequestsRate limit exceededAdd time.sleep(0.5) between requests; reduce batch size
Inconsistent mapping resultsMultiple isoforms/entriesFilter by isReviewed=true for canonical UniProt entries

Bundled Resources

references/services_catalog.md

Catalog of 20+ bioservices-accessible services organized by category: protein/gene resources (UniProt, KEGG, HGNC, MyGeneInfo), chemical compound resources (ChEBI, ChEMBL, UniChem, PubChem), sequence analysis (NCBIblast), pathway/interaction resources (Reactome, PSICQUIC, IntactComplex, OmniPath), gene ontology (QuickGO), genomic resources (BioMart, ArrayExpress, ENA), structural biology (PDB, Pfam), and specialized resources (BioModels, COG, BiGG). Includes method signatures, key parameters, and output formats for each service. General patterns (error handling, verbosity, caching) relocated to SKILL.md Best Practices. Original services_reference.md content partially relocated to Core API (UniProt, KEGG, ChEMBL, ChEBI, BLAST, PSICQUIC, QuickGO sections); remaining services retained in catalog.

references/identifier_mapping_guide.md

Comprehensive guide to cross-database identifier conversion: UniProt mapping service (100+ database pairs with database code tables), UniChem compound mapping (source database IDs), KEGG identifier conversions, 4 common mapping patterns with code, and troubleshooting. Original identifier_mapping.md content partially relocated to Core API section 5 (basic mapping examples); advanced patterns, full database code tables, and multi-hop strategies retained in guide.

Scripts disposition (4 original scripts, 1,444 lines total):

  • protein_analysis_workflow.py (409 lines, 7 functions: search_protein, retrieve_sequence, run_blast, discover_pathways, find_interactions, get_go_annotations, main): End-to-end pipeline → Common Workflow 1 "Complete Protein Analysis Pipeline"
  • compound_cross_reference.py (379 lines, 7 functions: search_kegg_compound, get_kegg_info, get_chembl_id, get_chebi_info, get_chembl_info, save_results, main): Multi-DB compound search → Common Workflow 2 "Cross-Database Compound Search"
  • batch_id_converter.py (348 lines, 8 functions: normalize_database_code, read_ids_from_file, batch_convert, save_mapping_csv, save_failed_ids, print_mapping_summary, list_common_databases, main): Batch conversion → Common Workflow 3 "Batch Identifier Conversion" + Core API section 5
  • pathway_analysis.py (310 lines, 8 functions: get_all_pathways, analyze_pathway, analyze_all_pathways, save_pathway_summary, save_interactions_sif, save_detailed_pathway_info, print_statistics, main): KGML network analysis → Core API section 2 (KGML parsing) + Recipe 1 "Pathway Network Statistics"
  • CLI argument parsing, CSV/SIF file I/O, and summary printing utilities from all scripts: omitted — trivial boilerplate not essential for API understanding

Omitted original content: workflow_patterns.md (811 lines, 7 detailed workflows) — 4 workflows consolidated into Common Workflows 1-3 and Recipe 3; remaining 3 workflows (gene annotation pipeline, interaction network analysis, comparative species analysis) omitted as they combine patterns already shown in Core API modules without introducing new API calls.

Related Skills

  • gget-genomic-databases — deeper Ensembl/BLAST/enrichment queries via gget (simpler API for single-database access)
  • chembl-database-bioactivity — dedicated ChEMBL bioactivity queries beyond compound lookup
  • pubchem-compound-search — dedicated PubChem compound/assay queries with similarity search

References

cBioPortal Database

Overview

cBioPortal for Cancer Genomics is a public repository of cancer genomics data including TCGA, ICGC, and hundreds of curated studies spanning 100+ cancer types. It provides somatic mutation profiles, copy number alterations (CNA), gene expression, clinical data (survival, stage, treatment history), and methylation data for tens of thousands of patient samples. Data is accessible via a REST API at https://www.cbioportal.org/api/ with no authentication required.

When to Use

  • Retrieving somatic mutation profiles (variant type, amino acid change) for a gene across TCGA studies
  • Querying copy number alteration data (amplification, deep deletion) for candidate cancer driver genes
  • Accessing clinical data — overall survival, disease-free survival, tumor stage — for survival curve analysis
  • Identifying which cancer studies have molecular profiling data for a specific cancer type (e.g., breast, lung)
  • Downloading gene expression (RNA-seq FPKM/RSEM) data from specific TCGA cohorts for differential expression analysis
  • Correlating genomic alterations with clinical outcomes in a specific study
  • Use omics-plotting SKILL to render bar charts, mutation-frequency heatmaps, and Kaplan–Meier curves from the query results
  • Use gnomad-database instead when you need population-level variant allele frequencies in healthy individuals
  • For drug-gene interaction lookups use opentargets-database; cBioPortal provides the genomic alteration data, not drug interaction annotations

Prerequisites

  • Python packages: requests, pandas, matplotlib
  • Data requirements: Entrez gene symbols (e.g., TP53), cBioPortal study IDs (e.g., tcga_brca), molecular profile IDs
  • Environment: internet connection; no API key required
  • Rate limits: no strict rate limits; use time.sleep(0.2) between batch requests for polite access
pip install requests pandas matplotlib

Quick Start

import requests
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    """GET request to cBioPortal REST API, returns parsed JSON."""
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

# List available cancer types
cancer_types = cbio_get("cancer-types")
print(f"Total cancer types: {len(cancer_types)}")
# Total cancer types: 87

# Find TCGA breast cancer study
studies = cbio_get("studies", params={"keyword": "breast"})
brca = [s for s in studies if "tcga_brca" in s["studyId"]]
if brca:
    s = brca[0]
    print(f"Study: {s['name']}")
    print(f"  studyId: {s['studyId']}")
    print(f"  Samples: {s['allSampleCount']}")
# Study: Breast Invasive Carcinoma (TCGA, PanCancer Atlas)
#   studyId: brca_tcga_pan_can_atlas_2018
#   Samples: 1084

Core API

Query 1: Cancer Types and Studies

List available cancer types and find studies by cancer type or keyword.

import requests
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

# Get all cancer types
cancer_types = cbio_get("cancer-types")
ct_df = pd.DataFrame(cancer_types)[["cancerTypeId", "name", "dedicatedColor"]]
print(f"Cancer types: {len(ct_df)}")
print(ct_df.head(5).to_string(index=False))

# Find all studies for a cancer type
lung_studies = cbio_get("studies", params={"keyword": "lung adenocarcinoma"})
print(f"\nLung adenocarcinoma studies: {len(lung_studies)}")
for s in lung_studies[:3]:
    print(f"  {s['studyId']:40s}  n={s['allSampleCount']}")
# Get detailed study metadata including available data types
study_id = "brca_tcga_pan_can_atlas_2018"
study = cbio_get(f"studies/{study_id}")
print(f"Study: {study['name']}")
print(f"  Reference genome: {study.get('referenceGenome', 'n/a')}")
print(f"  All sample count: {study['allSampleCount']}")

# List molecular profiles for the study
profiles = cbio_get("molecular-profiles", params={"studyId": study_id})
print(f"\nMolecular profiles ({len(profiles)} total):")
for p in profiles:
    print(f"  {p['molecularProfileId']:55s}  [{p['molecularAlterationType']}]")

Query 2: Somatic Mutations

Retrieve mutation data for a gene or set of genes in a study's mutation profile.

import requests, json
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_post(endpoint, body):
    """POST request to cBioPortal REST API."""
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=60)
    r.raise_for_status()
    return r.json()

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

# Get all samples for a study
study_id = "brca_tcga_pan_can_atlas_2018"
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]
print(f"Total samples: {len(sample_ids)}")

# Mutation profile ID follows pattern: {studyId}_mutations
profile_id = f"{study_id}_mutations"

# Fetch mutations for TP53 (Entrez gene ID = 7157)
body = {
    "sampleIds": sample_ids[:200],   # first 200 samples
    "entrezGeneIds": [7157]           # TP53
}
mutations = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch", body)
print(f"TP53 mutations in first 200 samples: {len(mutations)}")

# Summarize by mutation type
mut_df = pd.DataFrame(mutations)
print("\nMutation type distribution:")
print(mut_df["mutationType"].value_counts().head(8).to_string())
# Missense_Mutation    102
# Nonsense_Mutation     28
# Splice_Site           14
# Frame_Shift_Del       12

Query 3: Copy Number Alterations

Fetch discrete CNA data (amplification = 2, gain = 1, diploid = 0, loss = -1, deep deletion = -2).

import requests
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_post(endpoint, body):
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=60)
    r.raise_for_status()
    return r.json()

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

study_id = "brca_tcga_pan_can_atlas_2018"
# CNA profile: discrete copy number data
cna_profile_id = f"{study_id}_gistic"   # GISTIC-derived discrete CNA

samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples][:300]

# Fetch CNA for ERBB2 (Entrez 2064) and MYC (Entrez 4609)
body = {
    "sampleIds": sample_ids,
    "entrezGeneIds": [2064, 4609]   # ERBB2, MYC
}
cna_data = cbio_post(
    f"molecular-profiles/{cna_profile_id}/molecular-data/fetch", body
)
print(f"CNA records retrieved: {len(cna_data)}")

cna_df = pd.DataFrame(cna_data)
# CNA values: 2=amplification, 1=gain, 0=diploid, -1=loss, -2=deep deletion
cna_label = {2: "AMP", 1: "GAIN", 0: "DIPLOID", -1: "LOSS", -2: "HOMDEL"}

print("\nERBB2 CNA distribution:")
erbb2 = cna_df[cna_df["entrezGeneId"] == 2064]
erbb2_counts = erbb2["value"].map(lambda x: cna_label.get(int(x), str(x))).value_counts()
print(erbb2_counts.to_string())
# DIPLOID    210
# AMP         62
# GAIN        18
# LOSS        10

Query 4: Clinical Data

Retrieve per-sample or per-patient clinical attributes including survival, tumor stage, and treatment.

import requests
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

study_id = "brca_tcga_pan_can_atlas_2018"

# List available clinical attributes for this study
attrs = cbio_get(f"studies/{study_id}/clinical-attributes")
attr_df = pd.DataFrame(attrs)[["clinicalAttributeId", "displayName", "datatype", "patientAttribute"]]
print(f"Clinical attributes: {len(attr_df)}")
# Show survival-related attributes
survival_attrs = attr_df[attr_df["clinicalAttributeId"].str.contains("SURVIVAL|MONTHS|STATUS", na=False)]
print(survival_attrs[["clinicalAttributeId", "displayName"]].to_string(index=False))

# Fetch OS_STATUS and OS_MONTHS for all patients
clinical = cbio_get(f"studies/{study_id}/clinical-data",
                    params={"clinicalDataType": "PATIENT",
                            "projection": "DETAILED"})
clin_df = pd.DataFrame(clinical)
# Pivot to patient × attribute matrix
clin_pivot = clin_df.pivot_table(
    index="patientId", columns="clinicalAttributeId",
    values="value", aggfunc="first"
)
print(f"\nPatients: {len(clin_pivot)}")
if "OS_STATUS" in clin_pivot.columns:
    print("OS status counts:")
    print(clin_pivot["OS_STATUS"].value_counts().to_string())
# OS status counts:
# 0:LIVING    765
# 1:DECEASED  319

Query 5: Gene Expression Data

Retrieve mRNA expression values (RSEM or FPKM) from RNA-seq profiles.

import requests
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_post(endpoint, body):
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=60)
    r.raise_for_status()
    return r.json()

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

study_id = "brca_tcga_pan_can_atlas_2018"
# RNA-seq profile (RSEM normalized values)
rna_profile_id = f"{study_id}_rna_seq_v2_mrna_median_normed_log2"

samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples][:100]

# Fetch expression for ESR1 (Entrez 2099), ERBB2 (2064), PGR (5241)
body = {
    "sampleIds": sample_ids,
    "entrezGeneIds": [2099, 2064, 5241]   # ESR1, ERBB2, PGR
}
expr_data = cbio_post(
    f"molecular-profiles/{rna_profile_id}/molecular-data/fetch", body
)
expr_df = pd.DataFrame(expr_data)
print(f"Expression records: {len(expr_df)}")

# Pivot to gene × sample matrix
expr_pivot = expr_df.pivot_table(
    index="sampleId", columns="entrezGeneId", values="value"
)
expr_pivot.columns = ["ERBB2", "ESR1", "PGR"]   # rename by gene symbol
print(f"\nExpression matrix: {expr_pivot.shape}")
print(expr_pivot.describe().round(2))

Query 6: Gene Details and Batch Lookup

Look up gene metadata (symbol, Entrez ID, type) required to construct mutation and CNA queries.

import requests
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

def cbio_post(endpoint, body):
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=30)
    r.raise_for_status()
    return r.json()

# Single gene lookup by Hugo symbol
gene = cbio_get("genes/TP53")
print(f"TP53: entrezGeneId={gene['entrezGeneId']}, type={gene['type']}")
# TP53: entrezGeneId=7157, type=protein-coding

# Batch gene lookup — convert Hugo symbols to Entrez IDs
gene_symbols = ["BRCA1", "BRCA2", "TP53", "PIK3CA", "PTEN", "KRAS", "EGFR"]
body = {"geneIds": gene_symbols, "geneIdType": "HUGO_GENE_SYMBOL"}
gene_list = cbio_post("genes/fetch", body)

gene_map = {g["hugoGeneSymbol"]: g["entrezGeneId"] for g in gene_list}
gene_df = pd.DataFrame(gene_list)[["hugoGeneSymbol", "entrezGeneId", "type"]]
print(f"\nResolved {len(gene_df)} genes:")
print(gene_df.to_string(index=False))
# hugoGeneSymbol  entrezGeneId            type
#          BRCA1         672   protein-coding
#          BRCA2         675   protein-coding
#           TP53        7157   protein-coding

Query 7: Visualization — Mutation Frequency Barplot

Compute mutation frequency across TCGA studies for a cancer driver gene, then read skills/data-visualization/omics-plotting/SKILL.md and follow its "Box / Violin / Bar" recipe on the exported CSV (→ figures/TP53_mutation_frequency.png).

import requests, time
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

def cbio_post(endpoint, body):
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=60)
    r.raise_for_status()
    return r.json()

# Focus on a curated set of TCGA PanCancer Atlas studies
STUDIES = {
    "brca_tcga_pan_can_atlas_2018": "BRCA",
    "luad_tcga_pan_can_atlas_2018": "LUAD",
    "coad_tcga_pan_can_atlas_2018": "COAD",
    "prad_tcga_pan_can_atlas_2018": "PRAD",
    "gbm_tcga_pan_can_atlas_2018": "GBM",
}

GENE_ENTREZ = 7157   # TP53
GENE_SYMBOL = "TP53"

rows = []
for study_id, label in STUDIES.items():
    try:
        samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
        sample_ids = [s["sampleId"] for s in samples]
        n_total = len(sample_ids)
        profile_id = f"{study_id}_mutations"
        body = {"sampleIds": sample_ids, "entrezGeneIds": [GENE_ENTREZ]}
        muts = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch", body)
        mutated_samples = len({m["sampleId"] for m in muts})
        rows.append({"study": label, "n_mutated": mutated_samples,
                     "n_total": n_total,
                     "freq": mutated_samples / n_total * 100})
        time.sleep(0.2)
    except Exception as e:
        print(f"  Skipping {study_id}: {e}")

df = pd.DataFrame(rows).sort_values("freq", ascending=True)
df.to_csv(f"{GENE_SYMBOL}_mutation_frequency.csv", index=False)
print(df[["study", "n_mutated", "n_total", "freq"]].to_string(index=False))
# Render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Box / Violin / Bar" recipe (horizontal bar of freq by study).

Key Concepts

cBioPortal Data Model

cBioPortal organizes data in a three-tier hierarchy: Cancer Studies → Molecular Profiles → Sample-level data. A single study (e.g., brca_tcga_pan_can_atlas_2018) contains multiple molecular profiles, each covering one data type. Before querying mutation or expression data, always retrieve the molecular profile list with GET /molecular-profiles?studyId={studyId} to confirm the correct profile ID.

Molecular Profile ID Conventions

Data TypeTypical Profile ID SuffixAlteration Type
Somatic mutations_mutationsMUTATION_EXTENDED
Discrete CNA (GISTIC)_gisticCOPY_NUMBER_ALTERATION
Continuous CNA (log2)_log2CNACOPY_NUMBER_ALTERATION
RNA-seq (log2 RSEM)_rna_seq_v2_mrna_median_normed_log2MRNA_EXPRESSION
Methylation_methylation_hm27 or _hm450METHYLATION

Not all studies have all profile types. Always verify with GET /molecular-profiles?studyId={studyId}.

Entrez Gene IDs

The REST API mutation and molecular data endpoints require Entrez Gene IDs (integers), not Hugo symbols. Use GET /genes/{hugoSymbol} or POST /genes/fetch to resolve symbols to IDs before batch queries.

Common Workflows

Workflow 1: Somatic Mutation Landscape for a Gene Panel

Goal: Retrieve mutations for multiple cancer driver genes across an entire TCGA study and export to CSV.

import requests, time
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

def cbio_post(endpoint, body):
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=120)
    r.raise_for_status()
    return r.json()

study_id = "luad_tcga_pan_can_atlas_2018"
profile_id = f"{study_id}_mutations"

# Resolve gene symbols to Entrez IDs
gene_symbols = ["KRAS", "EGFR", "TP53", "BRAF", "STK11", "KEAP1", "RB1"]
gene_list = cbio_post("genes/fetch",
                       {"geneIds": gene_symbols, "geneIdType": "HUGO_GENE_SYMBOL"})
gene_map = {g["entrezGeneId"]: g["hugoGeneSymbol"] for g in gene_list}
entrez_ids = list(gene_map.keys())

# Fetch all samples
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]
print(f"Study: {study_id} — {len(sample_ids)} samples")

# Batch mutations in chunks of 500 samples to avoid timeouts
chunk_size = 500
all_muts = []
for i in range(0, len(sample_ids), chunk_size):
    chunk = sample_ids[i:i + chunk_size]
    body = {"sampleIds": chunk, "entrezGeneIds": entrez_ids}
    muts = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch", body)
    all_muts.extend(muts)
    time.sleep(0.1)

mut_df = pd.DataFrame(all_muts)
mut_df["hugoSymbol"] = mut_df["entrezGeneId"].map(gene_map)
print(f"Total mutations: {len(mut_df)}")
print("\nMutation counts per gene:")
print(mut_df.groupby("hugoSymbol")["sampleId"].nunique()
      .sort_values(ascending=False).to_string())

mut_df.to_csv(f"{study_id}_driver_mutations.csv", index=False)
print(f"\nSaved: {study_id}_driver_mutations.csv")

Workflow 2: Survival Analysis — CNA Status vs. Overall Survival

Goal: Compare overall survival between patients with ERBB2 amplification vs. diploid/loss in TCGA BRCA.

import requests
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

def cbio_post(endpoint, body):
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=60)
    r.raise_for_status()
    return r.json()

study_id = "brca_tcga_pan_can_atlas_2018"
cna_profile_id = f"{study_id}_gistic"

# Get all samples
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]

# Fetch ERBB2 CNA (Entrez 2064)
cna_data = cbio_post(
    f"molecular-profiles/{cna_profile_id}/molecular-data/fetch",
    {"sampleIds": sample_ids, "entrezGeneIds": [2064]}
)
cna_df = pd.DataFrame(cna_data)[["sampleId", "value"]].rename(columns={"value": "erbb2_cna"})
cna_df["erbb2_cna"] = cna_df["erbb2_cna"].astype(int)
cna_df["erbb2_status"] = cna_df["erbb2_cna"].map(
    {2: "Amplified", 1: "Gain", 0: "Diploid", -1: "Loss", -2: "Deep Deletion"})

# Fetch clinical data (OS_STATUS, OS_MONTHS)
clinical = cbio_get(f"studies/{study_id}/clinical-data",
                    params={"clinicalDataType": "PATIENT", "projection": "DETAILED"})
clin_df = pd.DataFrame(clinical)
clin_pivot = clin_df.pivot_table(
    index="patientId", columns="clinicalAttributeId", values="value", aggfunc="first"
).reset_index()

# Map samples to patients
sample_patient = cbio_get(f"studies/{study_id}/samples", params={"projection": "DETAILED"})
sp_df = pd.DataFrame(sample_patient)[["sampleId", "patientId"]]

# Merge CNA + clinical via patient ID
merged = (cna_df
          .merge(sp_df, on="sampleId")
          .merge(clin_pivot[["patientId", "OS_STATUS", "OS_MONTHS"]],
                 on="patientId", how="inner"))
merged = merged.dropna(subset=["OS_STATUS", "OS_MONTHS"])
merged["OS_MONTHS"] = pd.to_numeric(merged["OS_MONTHS"], errors="coerce")
merged["event"] = (merged["OS_STATUS"] == "1:DECEASED").astype(int)

# Survival table for the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Kaplan–Meier" recipe (columns: time, event, group)
km_input = (merged.loc[merged["erbb2_status"].isin(["Amplified", "Diploid"]),
                       ["OS_MONTHS", "event", "erbb2_status"]]
            .rename(columns={"OS_MONTHS": "time", "erbb2_status": "group"}))
km_input.to_csv("erbb2_survival.csv", index=False)
print(f"Survival table: {len(km_input)} patients -> erbb2_survival.csv")
print(km_input["group"].value_counts().to_string())
# Render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Kaplan–Meier" recipe -> figures/erbb2_survival.png

Workflow 3: Multi-Study Alteration Frequency Heatmap

Goal: Build a gene × cancer-type alteration frequency matrix across TCGA studies.

import requests, time
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

def cbio_post(endpoint, body):
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=90)
    r.raise_for_status()
    return r.json()

STUDIES = {
    "brca_tcga_pan_can_atlas_2018": "BRCA",
    "luad_tcga_pan_can_atlas_2018": "LUAD",
    "coad_tcga_pan_can_atlas_2018": "COAD",
    "gbm_tcga_pan_can_atlas_2018":  "GBM",
}
GENE_SYMBOLS = ["TP53", "KRAS", "PIK3CA", "EGFR", "PTEN"]

# Resolve genes
gene_list = cbio_post("genes/fetch",
                       {"geneIds": GENE_SYMBOLS, "geneIdType": "HUGO_GENE_SYMBOL"})
gene_map = {g["entrezGeneId"]: g["hugoGeneSymbol"] for g in gene_list}
entrez_ids = list(gene_map.keys())

freq_matrix = pd.DataFrame(index=GENE_SYMBOLS, columns=list(STUDIES.values()), dtype=float)

for study_id, label in STUDIES.items():
    try:
        samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
        sample_ids = [s["sampleId"] for s in samples]
        n_total = len(sample_ids)
        profile_id = f"{study_id}_mutations"
        body = {"sampleIds": sample_ids, "entrezGeneIds": entrez_ids}
        muts = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch", body)
        mut_df = pd.DataFrame(muts) if muts else pd.DataFrame()
        for eid, symbol in gene_map.items():
            if mut_df.empty:
                freq_matrix.loc[symbol, label] = 0.0
            else:
                n_mut = mut_df[mut_df["entrezGeneId"] == eid]["sampleId"].nunique()
                freq_matrix.loc[symbol, label] = n_mut / n_total * 100
        time.sleep(0.2)
    except Exception as e:
        print(f"  {label}: {e}")

freq_matrix = freq_matrix.fillna(0).astype(float)
freq_matrix.to_csv("mutation_frequency_matrix.csv")
print(freq_matrix.to_string())
# Render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Expression heatmap" recipe (gene x cancer-type, YlOrRd,
# annotated) -> figures/mutation_frequency_heatmap.png

Key Parameters

ParameterFunction/EndpointDefaultRange / OptionsEffect
studyIdAll study endpoints—any valid cBioPortal study IDSelects the cancer study
molecularProfileIdmutations/fetch, molecular-data/fetch—{studyId}_mutations, {studyId}_gistic, etc.Selects the data type profile
entrezGeneIdsmutations/fetch, molecular-data/fetch—list of integer Entrez IDsGenes to query; use POST /genes/fetch to resolve symbols
sampleIdsmutations/fetch, molecular-data/fetch—list of sample ID stringsSamples to retrieve; use GET /studies/{id}/samples for all
clinicalDataTypeclinical-data"SAMPLE""SAMPLE", "PATIENT"Whether to return sample-level or patient-level clinical attributes
projectionsamples, clinical-data"SUMMARY""ID", "SUMMARY", "DETAILED", "META"Response verbosity; "ID" fastest for ID-only fetches
keywordstudies""free textFilter studies by name/cancer type keyword

Best Practices

  1. Fetch sample IDs before data queries: All mutation and molecular data endpoints require explicit sampleIds. Retrieve them with GET /studies/{studyId}/samples?projection=ID before each query.

  2. Verify profile IDs from the API: Profile IDs are not guaranteed to follow the _mutations / _gistic pattern in every study. Always confirm with GET /molecular-profiles?studyId={studyId} rather than guessing.

  3. Chunk large sample sets: The API can time out on requests with thousands of sample IDs. Batch requests in chunks of 500 samples with time.sleep(0.1) between chunks.

  4. Use Entrez IDs, not Hugo symbols, in data fetch endpoints: The mutation and molecular data endpoints accept entrezGeneIds (integers). Resolve symbols first with POST /genes/fetch.

  5. Don't hard-code Entrez IDs: Gene IDs can be looked up dynamically via the API. Hard-coded IDs become incorrect if the gene model changes. Use POST /genes/fetch to resolve gene symbols at runtime.

Common Recipes

Recipe: List All Molecular Profiles for a Study

When to use: Before running any data query — verify which profile IDs are available.

import requests

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

study_id = "brca_tcga_pan_can_atlas_2018"
profiles = cbio_get("molecular-profiles", params={"studyId": study_id})
for p in profiles:
    print(f"{p['molecularProfileId']:55s}  {p['molecularAlterationType']}")
# brca_tcga_pan_can_atlas_2018_mutations          MUTATION_EXTENDED
# brca_tcga_pan_can_atlas_2018_gistic             COPY_NUMBER_ALTERATION
# brca_tcga_pan_can_atlas_2018_log2CNA            COPY_NUMBER_ALTERATION
# brca_tcga_pan_can_atlas_2018_rna_seq_v2_mrna_median_normed_log2  MRNA_EXPRESSION

Recipe: Download Full Mutation MAF for a Study

When to use: Export all somatic mutations from a study into MAF-compatible format for downstream analysis.

import requests, time
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=60)
    r.raise_for_status()
    return r.json()

def cbio_post(endpoint, body):
    r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
                      headers={"Accept": "application/json",
                               "Content-Type": "application/json"},
                      timeout=120)
    r.raise_for_status()
    return r.json()

study_id = "coad_tcga_pan_can_atlas_2018"
profile_id = f"{study_id}_mutations"

samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]

all_mutations = []
for i in range(0, len(sample_ids), 300):
    chunk = sample_ids[i:i + 300]
    muts = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch",
                     {"sampleIds": chunk, "entrezGeneIds": []})  # empty = all genes
    all_mutations.extend(muts)
    time.sleep(0.1)

mut_df = pd.DataFrame(all_mutations)
cols = ["hugoGeneSymbol", "sampleId", "chr", "startPosition", "endPosition",
        "referenceAllele", "variantAllele", "mutationType",
        "proteinChange", "variantType"]
available = [c for c in cols if c in mut_df.columns]
mut_df[available].to_csv(f"{study_id}_mutations.csv", index=False)
print(f"Saved {len(mut_df)} mutations → {study_id}_mutations.csv")

Recipe: Query Patient-Level Clinical Attribute

When to use: Extract a specific clinical variable (e.g., tumor stage, age at diagnosis) for all patients.

import requests
import pandas as pd

BASE_URL = "https://www.cbioportal.org/api"

def cbio_get(endpoint, params=None):
    r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
                     headers={"Accept": "application/json"}, timeout=30)
    r.raise_for_status()
    return r.json()

study_id = "brca_tcga_pan_can_atlas_2018"

# Fetch a specific clinical attribute for all patients
attr_id = "TUMOR_STAGE"
clinical = cbio_get(f"studies/{study_id}/clinical-data",
                    params={"clinicalDataType": "PATIENT",
                            "projection": "DETAILED"})

clin_df = pd.DataFrame(clinical)
if "clinicalAttributeId" in clin_df.columns:
    stage_df = clin_df[clin_df["clinicalAttributeId"] == attr_id][["patientId", "value"]]
    print(f"Patients with {attr_id} annotation: {len(stage_df)}")
    print(stage_df["value"].value_counts().head(10).to_string())

Troubleshooting

ProblemCauseSolution
404 Not Found on profile endpointMolecular profile does not exist for studyList profiles with GET /molecular-profiles?studyId={id}; confirm the profile ID
Empty mutations listGene has no mutations in the selected samples/profileVerify study has a mutation profile; check sample IDs belong to the same study
requests.exceptions.TimeoutLarge sample set (>1000) in a single requestChunk requests to 300–500 samples; increase timeout to 120s
entrezGeneIds key error in responseHugo symbol passed instead of Entrez IDUse POST /genes/fetch to resolve symbols to integer Entrez IDs first
CNA values returned as stringsvalue field is string in JSONCast with pd.to_numeric() or int(value) before comparison
Expression profile not foundStudy uses non-standard profile namingCheck profile list; look for MRNA_EXPRESSION alteration type in GET /molecular-profiles
Survival analysis has many NA valuesClinical attribute absent for some patientsUse dropna() on OS columns; check attribute availability with GET /studies/{id}/clinical-attributes

Related Skills

  • gnomad-database — population variant allele frequencies for healthy cohorts (complement to cBioPortal somatic data)
  • cnvkit-copy-number — CNVkit pipeline for generating SEG/CNA files that can be loaded into cBioPortal
  • pydeseq2-differential-expression — differential expression analysis that can be applied to cBioPortal RNA-seq exports

References

ClinPGx (PharmGKB) Pharmacogenomics Database

Overview

PharmGKB rebranded as ClinPGx in 2024 and the API moved from api.pharmgkb.org to api.clinpgx.org. The old host now returns 404/405; every example here uses the new endpoints. Two complementary APIs are used together:

  • ClinPGx Data API (api.clinpgx.org/v1) — record-style access to genes, drugs, variants, clinical annotations, guideline annotations, drug labels, and pathways. Responses wrap data as {"data": [...], "status": "success"}. Filters use dotted property paths (e.g. relatedChemicals.name=clopidogrel, levelOfEvidence.term=1A).
  • CPIC PostgREST API (api.cpicpgx.org/v1) — relational lookup of genotype → drug recommendation rows. PostgREST filter syntax (column=eq.value, JSON cs.{...} for jsonb containment). Returns flat JSON arrays.

Use ClinPGx for what is known about a gene/drug/variant; use CPIC for how to prescribe given a phenotype. The pattern is ClinPGx for annotations, CPIC for recommendations.

When to Use

  • Retrieving CPIC genotype-specific dosing recommendations for a gene-drug pair (e.g., CYP2C19 + clopidogrel) — use CPIC
  • Looking up all pharmacogenomic clinical annotations for a drug or evidence level — use ClinPGx data/clinicalAnnotation
  • Finding all CPIC/DPWG guideline annotations for a pharmacogene — use ClinPGx data/guidelineAnnotation
  • Resolving a gene symbol, drug name, or rsID to ClinPGx PA identifiers — use data/{gene,drug,variant}
  • Free-text search across all ClinPGx record types (genes, drugs, variants, annotations) — use POST /site/search
  • Retrieving FDA/EMA pharmacogenomic drug label annotations — use ClinPGx data/label
  • Building precision-medicine prescribing workflows that combine annotation evidence with phenotype-specific recommendations
  • For germline disease pathogenicity (not PGx) use clinvar-database
  • For somatic cancer pharmacogenomics use cosmic-database or opentargets-database

Prerequisites

  • Python packages: requests, pandas — both already in standard environments
  • Data requirements: HGNC gene symbols, drug names (lowercase generic), dbSNP rsIDs, or PA identifiers
  • Environment: internet connection; no authentication required for either host
  • Rate limits: the ClinPGx host occasionally returns HTTP 429; insert time.sleep(0.3–0.5) between sequential calls. CPIC is more permissive.

If you are inside a pixi/conda environment that already provides requests and pandas, skip the install — invoke scripts with pixi run python ....

pip install requests pandas

Quick Start

import requests

CLINPGX = "https://api.clinpgx.org/v1"
CPIC    = "https://api.cpicpgx.org/v1"

# CPIC genotype → recommendation: clopidogrel + CYP2C19 Poor Metabolizer
drug = requests.get(f"{CPIC}/drug", params={"name": "eq.clopidogrel"}).json()[0]
recs = requests.get(f"{CPIC}/recommendation",
                    params={"drugid": f"eq.{drug['drugid']}",
                            "phenotypes": 'cs.{"CYP2C19":"Poor Metabolizer"}'}).json()
print(f"clopidogrel CYP2C19=PM: {len(recs)} recommendation(s)")
for rec in recs[:2]:
    print(f"  [{rec['classification']}] {rec['drugrecommendation'][:80]}…")

# ClinPGx side: how many CPIC guideline annotations cover CYP2C19?
glines = requests.get(f"{CLINPGX}/data/guidelineAnnotation",
                      params={"relatedGenes.symbol": "CYP2C19",
                              "source": "CPIC", "view": "base"}).json()["data"]
print(f"CYP2C19 CPIC guidelines: {len(glines)}")

Core API

Module 1: Free-text site search

POST /site/search with a JSON body {"query": "<term>"} is the canonical entry point when you don't know the PA ID. It searches across drugs, genes, variants, clinical annotations, guideline annotations, and labels in one shot.

import requests

CLINPGX = "https://api.clinpgx.org/v1"

r = requests.post(f"{CLINPGX}/site/search",
                  json={"query": "rs4149056"}, timeout=15)
r.raise_for_status()
hits = r.json()["data"]["hits"]
print(f"Total hits: {r.json()['data']['total']}")
for h in hits[:5]:
    print(f"  id={h.get('id')}  name={h.get('name')[:80]}")
# Broader concept search
r = requests.post(f"{CLINPGX}/site/search",
                  json={"query": "TPMT azathioprine"}, timeout=15)
hits = r.json()["data"]["hits"]
print(f"TPMT+azathioprine hits: {len(hits)}")
for h in hits[:5]:
    print(f"  {h.get('id'):>15}  {h.get('name','')[:80]}")

Module 2: Gene, drug, and variant record lookup

The /data/{type} endpoints accept simple property filters. All return {"data": [...], "status": "success"} — use view=base for summary, view=max for full nested objects.

import requests

CLINPGX = "https://api.clinpgx.org/v1"

# Gene by HGNC symbol
gene = requests.get(f"{CLINPGX}/data/gene",
                    params={"symbol": "CYP2D6", "view": "base"}).json()["data"][0]
print(f"{gene['symbol']}  id={gene['id']}  {gene['name']}")

# Drug by name (lowercase generic preferred)
drug = requests.get(f"{CLINPGX}/data/drug",
                    params={"name": "warfarin", "view": "base"}).json()["data"][0]
print(f"{drug['name']}  id={drug['id']}")

# Variant by rsID
var = requests.get(f"{CLINPGX}/data/variant",
                   params={"name": "rs4149056", "view": "base"}).json()["data"][0]
print(f"{var['name']}  id={var['id']}  significance={var.get('clinicalSignificance')}")
# Direct record fetch when you already have a PA ID
r = requests.get(f"{CLINPGX}/data/drug/PA449088", params={"view": "max"}).json()
d = r["data"]
print(f"PA449088 → {d['name']}  (objCls={d['objCls']})")

Module 3: Clinical annotations

data/clinicalAnnotation records associate a variant (location) with one or more drugs (relatedChemicals) and an evidence level (levelOfEvidence.term). The two supported filters are relatedChemicals.name= and levelOfEvidence.term=. There is no working gene= filter on this endpoint — see Module 4 for gene-driven access.

import requests, pandas as pd

CLINPGX = "https://api.clinpgx.org/v1"

# All clinical annotations for clopidogrel
data = requests.get(f"{CLINPGX}/data/clinicalAnnotation",
                    params={"relatedChemicals.name": "clopidogrel",
                            "view": "base"}).json()["data"]
print(f"clopidogrel annotations: {len(data)}")

rows = []
for ann in data[:10]:
    loc = ann.get("location") or {}
    drugs = ", ".join(c.get("name", "") for c in ann.get("relatedChemicals", []))
    rows.append({
        "id": ann["id"],
        "variant": loc.get("displayName"),
        "gene": (loc.get("genes") or [{}])[0].get("symbol"),
        "drug": drugs,
        "level": (ann.get("levelOfEvidence") or {}).get("term"),
        "score": ann.get("score"),
    })
print(pd.DataFrame(rows).to_string(index=False))
# All Level 1A clinical annotations (highest evidence)
data = requests.get(f"{CLINPGX}/data/clinicalAnnotation",
                    params={"levelOfEvidence.term": "1A",
                            "view": "base"}).json()["data"]
print(f"Level 1A annotations: {len(data)}")

drug_to_count = {}
for ann in data:
    for c in ann.get("relatedChemicals") or []:
        drug_to_count[c["name"]] = drug_to_count.get(c["name"], 0) + 1
top = sorted(drug_to_count.items(), key=lambda x: -x[1])[:10]
for d, n in top:
    print(f"  {n:3}  {d}")

Module 4: Guideline annotations (gene-driven access)

data/guidelineAnnotation supports both relatedGenes.symbol= and relatedChemicals.name=, plus source= (CPIC, DPWG, CPNDS, RNPGx). This is the canonical way to get gene→guideline coverage.

import requests

CLINPGX = "https://api.clinpgx.org/v1"

# All CPIC guidelines mentioning CYP2C19
data = requests.get(f"{CLINPGX}/data/guidelineAnnotation",
                    params={"relatedGenes.symbol": "CYP2C19",
                            "source": "CPIC",
                            "view": "base"}).json()["data"]
print(f"CYP2C19 CPIC guidelines: {len(data)}")
for g in data[:5]:
    print(f"  PA{g['id']}: {g['name'][:80]}")
# Guidelines for a specific drug across all bodies (CPIC, DPWG, …)
data = requests.get(f"{CLINPGX}/data/guidelineAnnotation",
                    params={"relatedChemicals.name": "clopidogrel",
                            "view": "base"}).json()["data"]
by_source = {}
for g in data:
    for s in (g.get("crossReferences") or []):
        by_source.setdefault(s.get("resource", "?"), 0)
        by_source[s["resource"]] = by_source.get(s["resource"], 0) + 1
print(f"clopidogrel guidelines: {len(data)} ({list({g.get('source') for g in data})})")

Module 5: Regulatory drug labels (FDA / EMA)

data/label records are PharmGKB-curated annotations of FDA/EMA pharmacogenomic labeling. Filter by relatedChemicals.name= and source= (FDA, EMA, HCSC, PMDA, Swissmedic).

import requests, pandas as pd

CLINPGX = "https://api.clinpgx.org/v1"

data = requests.get(f"{CLINPGX}/data/label",
                    params={"relatedChemicals.name": "warfarin",
                            "source": "FDA",
                            "view": "base"}).json()["data"]
print(f"warfarin FDA labels: {len(data)}")

rows = [{
    "name": d["name"][:60],
    "biomarker_status": d.get("biomarkerStatus"),
    "testing_required": d.get("testingRequired"),
    "alternate_drug": d.get("alternateDrugAvailable"),
} for d in data]
print(pd.DataFrame(rows).to_string(index=False))

Module 6: CPIC genotype → recommendation chain

CPIC's PostgREST API uses column=eq.value for equality and column=cs.{...} for JSONB containment. The standard lookup chain is drug → drugid → recommendation, optionally filtered by phenotype.

import requests

CPIC = "https://api.cpicpgx.org/v1"

# Resolve drug name to drugid (RxNorm-prefixed)
drug = requests.get(f"{CPIC}/drug",
                    params={"name": "eq.clopidogrel"}).json()[0]
print(f"clopidogrel drugid: {drug['drugid']}")

# All phenotype-specific recommendations for clopidogrel
recs = requests.get(f"{CPIC}/recommendation",
                    params={"drugid": f"eq.{drug['drugid']}"}).json()
print(f"Total recommendations: {len(recs)}")
for rec in recs[:3]:
    print(f"  {rec['phenotypes']}  [{rec['classification']}]")
    print(f"    {rec['drugrecommendation'][:90]}…")
# Phenotype filter via jsonb containment (cs.{...})
# The phenotypes column is a jsonb dict; cs. checks that the query is a subset.
recs = requests.get(f"{CPIC}/recommendation",
                    params={"drugid": f"eq.{drug['drugid']}",
                            "phenotypes": 'cs.{"CYP2C19":"Poor Metabolizer"}'}
                    ).json()
for rec in recs:
    print(f"  [{rec['classification']}] {rec['drugrecommendation'][:90]}…")

# Gene-driven: list every drug with a CPIC pair for CYP2C19
pairs = requests.get(f"{CPIC}/pair",
                     params={"genesymbol": "eq.CYP2C19"}).json()
print(f"\nCYP2C19 CPIC pairs: {len(pairs)}")
drug_ids = sorted({p["drugid"] for p in pairs})
print(f"Sample drug IDs: {drug_ids[:5]}")

Key Concepts

Two-host architecture

QuestionUseWhy
What clinical annotations exist for this drug?ClinPGx data/clinicalAnnotationAnnotation-level evidence with curated levelOfEvidence.term
What CPIC guidelines cover this gene?ClinPGx data/guidelineAnnotationFilter by relatedGenes.symbol; no working gene= filter on clinicalAnnotation
Given phenotype X, what should I prescribe?CPIC recommendation + phenotypesStructured genotype→action rows; CPIC is the prescribing-rule oracle
What FDA labels mention this drug + gene?ClinPGx data/label?source=FDACurated regulatory PGx labeling
Free-text "anything about X"ClinPGx POST /site/searchCross-record-type fan-out

PharmGKB / ClinPGx evidence levels

Levels 1A → 4 in decreasing evidence quality:

  • 1A — Annotation of a variant–drug pair in a clinical guideline or FDA label (strongest)
  • 1B — Significant association replicated in multiple studies
  • 2A — Variant in a known PGx gene, significant association
  • 2B — Moderate evidence, often single study
  • 3 — Limited evidence (single study or unreplicated)
  • 4 — Case reports / biological plausibility only

Filter via levelOfEvidence.term on data/clinicalAnnotation. The term is a string, not an enum ("1A" not 1A).

ClinPGx response envelope and view modes

Every ClinPGx /data/... response is {"data": [...] | {...}, "status": "success" | "fail"}. On failure the body is {"status": "fail", "data": {"errors": [{"message": "..."}]}} — always read both keys.

  • view=base (default) — flat summary record; recommended for bulk filters
  • view=max — full nested objects (relatedDiseases, allelePhenotypes, scoreDetails, …). Larger payload, slower; use only for single-record details.

Common Workflows

Workflow 1: Pharmacogene panel CPIC coverage

Goal: Given a patient's pharmacogene panel, count how many CPIC guideline annotations cover each gene.

import requests, pandas as pd, time

CLINPGX = "https://api.clinpgx.org/v1"
pharmacogenes = ["CYP2D6", "CYP2C19", "CYP2C9", "DPYD", "TPMT", "SLCO1B1"]

rows = []
for g in pharmacogenes:
    data = requests.get(f"{CLINPGX}/data/guidelineAnnotation",
                        params={"relatedGenes.symbol": g,
                                "source": "CPIC", "view": "base"},
                        timeout=20).json()["data"]
    drugs = sorted({c["name"] for guideline in data
                                for c in (guideline.get("relatedChemicals") or [])})
    rows.append({"gene": g, "cpic_guidelines": len(data),
                 "n_drugs": len(drugs), "sample": ", ".join(drugs[:3])})
    time.sleep(0.3)

df = pd.DataFrame(rows).sort_values("cpic_guidelines", ascending=False)
print(df.to_string(index=False))
df.to_csv("pharmacogene_cpic_coverage.csv", index=False)

Workflow 2: Drug panel — CPIC prescribing rule lookup

Goal: Given a prescribed drug list, identify which have CPIC genotype-specific recommendations and surface the rule rows.

import requests, pandas as pd, time

CPIC = "https://api.cpicpgx.org/v1"
drugs = ["warfarin", "clopidogrel", "codeine", "simvastatin",
         "metoprolol", "omeprazole", "azathioprine", "tacrolimus"]

rows = []
for name in drugs:
    drug = requests.get(f"{CPIC}/drug", params={"name": f"eq.{name}"}, timeout=15).json()
    if not drug:
        rows.append({"drug": name, "in_cpic": False, "n_recs": 0, "phenotypes": ""}); continue
    did = drug[0]["drugid"]
    recs = requests.get(f"{CPIC}/recommendation",
                        params={"drugid": f"eq.{did}"}, timeout=15).json()
    phens = sorted({f"{k}={v}" for rec in recs
                                  for k, v in (rec.get("phenotypes") or {}).items()})
    rows.append({"drug": name, "in_cpic": True, "n_recs": len(recs),
                 "phenotypes": "; ".join(phens[:3])})
    time.sleep(0.3)

df = pd.DataFrame(rows).sort_values(["in_cpic", "n_recs"], ascending=[False, False])
print(df.to_string(index=False))

Workflow 3: Variant → drug interactions (rsID-driven)

Goal: Starting from a single rsID (e.g., SLCO1B1 *5 = rs4149056), find every clinical annotation that involves it.

The Data API does not accept rsID as a filter property. Use POST /site/search to discover related annotation IDs, then fetch each by ID.

import requests

CLINPGX = "https://api.clinpgx.org/v1"
rsid = "rs4149056"

hits = requests.post(f"{CLINPGX}/site/search",
                     json={"query": rsid}, timeout=15).json()["data"]["hits"]
print(f"{rsid}: {len(hits)} hits")

# Filter hits that look like clinical annotations
ann_hits = [h for h in hits if h.get("name", "").lower().startswith("clinical annotation")]
print(f"Clinical-annotation hits: {len(ann_hits)}")
for h in ann_hits[:5]:
    print(f"  id={h['id']}  {h['name'][:90]}")

# Dereference one annotation by ID for full detail
if ann_hits:
    ann = requests.get(f"{CLINPGX}/data/clinicalAnnotation/{ann_hits[0]['id']}",
                       params={"view": "max"}, timeout=15).json()["data"]
    drugs = ", ".join(c["name"] for c in (ann.get("relatedChemicals") or []))
    print(f"\nFirst annotation:")
    print(f"  drugs: {drugs}")
    print(f"  level: {(ann.get('levelOfEvidence') or {}).get('term')}")

Key Parameters

ParameterModule / EndpointDefaultRange / OptionsEffect
viewall /data/...basebase, min, maxField detail level; max includes all nested arrays (slow but complete)
relatedChemicals.nameclinicalAnnotation, variantAnnotation, guidelineAnnotation, label, pathway—lowercase generic drug nameFilter records related to a drug
relatedGenes.symbolguidelineAnnotation, pathway—HGNC gene symbolFilter records related to a gene (not available on clinicalAnnotation)
levelOfEvidence.termclinicalAnnotation—"1A", "1B", "2A", "2B", "3", "4"Minimum evidence level
sourceguidelineAnnotation, label—CPIC, DPWG, FDA, EMA, HCSC, PMDA, SwissmedicIssuing body
symboldata/gene—HGNC gene symbolGene record lookup
namedata/drug, data/variant—drug name or rsIDRecord lookup by canonical name
CPIC column=eq.valueall api.cpicpgx.org/v1/...—PostgREST equalityFilter by exact match
CPIC phenotypes=cs.{json}recommendation—JSON-encoded jsonb subsetFilter by phenotype containment (must URL-encode if special chars)

Best Practices

  1. Resolve PA identifiers once. Never hand-construct ClinPGx PA IDs. Call data/{type}?{symbol|name}=... (or site/search) once and cache the returned id for reuse — gene/PA128 for CYP2D6, drug/PA449088 for clopidogrel, variant/PA166154579 for rs4149056.

  2. Pick the right host for the question. Use ClinPGx for what is annotated and CPIC for what to prescribe. Trying to derive genotype-specific recommendations from ClinPGx alone misses the structured recommendation.phenotypes rows.

  3. Filter by evidence level upfront when building clinical workflows. levelOfEvidence.term=1A returns 312 actionable annotations across all of ClinPGx; Level 3/4 records are exploratory and shouldn't drive prescribing.

  4. Don't filter clinicalAnnotation by gene — filter by guidelineAnnotation with relatedGenes.symbol. The clinicalAnnotation endpoint has no working gene property and returns HTTP 400 for any attempt.

  5. Use view=base for bulk filters, view=max for single-record drill-downs. A list query with view=max can time out or hit 429; the difference is roughly 5–10× payload size.

  6. Throttle the ClinPGx host. Insert time.sleep(0.3) between sequential queries in loops; the API returns occasional HTTP 429s on tight loops. CPIC tolerates faster iteration.

  7. URL-encode cs.{...} jsonb filters when phenotype values contain spaces or special characters. requests.get(..., params={"phenotypes": 'cs.{"CYP2C19":"Poor Metabolizer"}'}) works because requests does the encoding; a manual URL string needs urllib.parse.quote.

Common Recipes

Recipe 1 — Free-text discovery via site/search

When to use: you have an arbitrary string (rsID, drug name, gene, allele) and want to find related ClinPGx records without knowing which endpoint to hit.

import requests
r = requests.post("https://api.clinpgx.org/v1/site/search",
                  json={"query": "VKORC1 warfarin"}, timeout=15)
hits = r.json()["data"]["hits"]
for h in hits[:10]:
    print(f"  {h.get('id'):>15}  {h.get('name','')[:80]}")

Recipe 2 — Top drugs by Level 1A annotation count

When to use: build a leaderboard of the most actionable PGx drugs.

import requests, pandas as pd
data = requests.get("https://api.clinpgx.org/v1/data/clinicalAnnotation",
                    params={"levelOfEvidence.term": "1A", "view": "base"},
                    timeout=30).json()["data"]
counts = {}
for ann in data:
    for c in ann.get("relatedChemicals") or []:
        counts[c["name"]] = counts.get(c["name"], 0) + 1
df = pd.DataFrame(sorted(counts.items(), key=lambda x: -x[1]),
                  columns=["drug", "n_1A_annotations"]).head(15)
print(df.to_string(index=False))

Recipe 3 — Patient genotype → drug recommendations

When to use: given a phenotype call from a PGx test, surface every CPIC recommendation row.

import requests
CPIC = "https://api.cpicpgx.org/v1"

genotype = {"CYP2C19": "Poor Metabolizer"}
drug = "clopidogrel"

did = requests.get(f"{CPIC}/drug", params={"name": f"eq.{drug}"}).json()[0]["drugid"]
import json
recs = requests.get(f"{CPIC}/recommendation",
                    params={"drugid": f"eq.{did}",
                            "phenotypes": f"cs.{json.dumps(genotype)}"}).json()
for rec in recs:
    print(f"[{rec['classification']}] {rec['drugrecommendation']}")
    print(f"  implications: {rec['implications']}")

Recipe 4 — Robust session with retry

When to use: long-running loops over many genes / drugs / variants.

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

s = requests.Session()
s.headers.update({"Accept": "application/json"})
s.mount("https://", HTTPAdapter(max_retries=Retry(
    total=4, backoff_factor=1.0,
    status_forcelist=[429, 500, 502, 503, 504],
    allowed_methods=["GET", "POST"])))

r = s.get("https://api.clinpgx.org/v1/data/gene",
          params={"symbol": "CYP2D6", "view": "base"}, timeout=20)
r.raise_for_status()
print(r.json()["data"][0]["name"])

Troubleshooting

ProblemCauseSolution
HTTP 404/405 on https://api.pharmgkb.org/v1/...Old PharmGKB host is dead; the service rebranded to ClinPGx in 2024Migrate to https://api.clinpgx.org/v1/.... Old /clinicalAnnotation?gene=X is now data/clinicalAnnotation with different filters.
{"status":"fail","data":{"errors":[{"message":"No such property: 'gene'"}]}}data/clinicalAnnotation does not accept gene= or relatedGenes.symbol=Use data/guidelineAnnotation?relatedGenes.symbol=X for gene-driven access, or ?relatedChemicals.name=Y for drug-driven.
{"status":"fail","data":{"errors":[{"message":"Missing criteria."}]}}A data/{type} list query has no filter and no IDAdd at least one filter (name=, symbol=, relatedChemicals.name=, …) or fetch by ID via data/{type}/{paId}.
HTTP 405 on GET /site/search?query=...site/search only accepts POST with a JSON bodyUse requests.post(url, json={"query": "..."}).
HTTP 429 mid-loopHit ClinPGx rate limitInsert time.sleep(0.3–0.5) between calls; use the Retry session in Recipe 4.
HTTP 400 on https://api.cpicpgx.org/v1/recommendation?phenotypes=cs.{...}The cs. JSON wasn't URL-encodedPass via requests params={"phenotypes": 'cs.{"CYP2C19":"Poor Metabolizer"}'} (auto-encoded) or urllib.parse.quote manually.
Empty data list for an obviously-real drugDrug name mismatch (brand vs. generic; capitalization)Try lowercase generic name; fall back to POST /site/search to fan out and find the canonical PA ID.
data/variant?name=rs... returns 1 record but data/clinicalAnnotation?location.name=rs... returns 404rsID is stored under location.displayName/location.rsid, not exposed as a filterable propertyUse site/search to discover annotation IDs by rsID, then dereference each with data/clinicalAnnotation/{id}. (Workflow 3.)

Related Skills

  • clinvar-database — germline pathogenicity / clinical significance for variants found in PharmGKB (complementary; ClinVar is disease-focused, ClinPGx is drug-response-focused)
  • opentargets-database — drug-target associations and safety signals overlapping ClinPGx pharmacogene targets
  • chembl-database-bioactivity — bioactivity and binding data for the drugs annotated in ClinPGx
  • cosmic-database — somatic cancer mutations and tumor-specific PGx (orthogonal to germline PGx covered here)

References

ClinVar Clinical Variants Database

Overview

ClinVar is NCBI's public archive of interpretations of variants submitted by clinical laboratories, researchers, and expert panels. It contains 2M+ variants with clinical significance classifications (Pathogenic, Likely Pathogenic, VUS, Likely Benign, Benign) for over 6,000 conditions. Access is free and requires no authentication via NCBI E-utilities.

When to Use

  • Checking whether a specific variant (rsID, HGVS, or genomic position) has a clinical significance classification
  • Retrieving all pathogenic/likely-pathogenic variants in a gene of interest
  • Identifying conflicting interpretations between submitting laboratories
  • Pulling condition/phenotype associations for a variant (MIM, MeSH, HPO terms)
  • Building variant filtering pipelines that prioritize clinically actionable variants
  • For somatic cancer variants, also check cosmic-database; for GWAS associations use gwas-database

Prerequisites

  • Python packages: requests, xml.etree.ElementTree (stdlib)
  • Data requirements: gene symbols, rsIDs, HGVS strings, or ClinVar Variation IDs
  • Environment: internet connection; NCBI Entrez email required (set email parameter)
  • Rate limits: 3 requests/second unauthenticated; 10/second with API key (free at https://www.ncbi.nlm.nih.gov/account/)
pip install requests
# No additional packages required; xml.etree is part of Python stdlib

Quick Start

import requests

EMAIL = "your@email.com"  # required by NCBI policy

def clinvar_search(query, retmax=10):
    """Search ClinVar and return a list of ClinVar Variation IDs."""
    r = requests.get(
        "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi",
        params={"db": "clinvar", "term": query, "retmax": retmax,
                "retmode": "json", "email": EMAIL}
    )
    r.raise_for_status()
    return r.json()["esearchresult"]["idlist"]

# Find pathogenic BRCA1 variants
ids = clinvar_search("BRCA1[gene] AND pathogenic[clinsig]", retmax=5)
print(f"Found variation IDs: {ids}")

Core API

Query 1: Search Variants by Gene and Clinical Significance

Use ESearch to find ClinVar Variation IDs matching a structured query.

import requests

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def esearch(query, retmax=200):
    r = requests.get(f"{BASE}/esearch.fcgi",
                     params={"db": "clinvar", "term": query,
                             "retmax": retmax, "retmode": "json", "email": EMAIL})
    r.raise_for_status()
    result = r.json()["esearchresult"]
    return result["idlist"], int(result["count"])

# Gene-specific pathogenic variants
ids, total = esearch("BRCA2[gene] AND (pathogenic[clinsig] OR likely pathogenic[clinsig])")
print(f"Pathogenic/LP BRCA2 variants: {total} total, retrieved {len(ids)}")
print(f"First 5 IDs: {ids[:5]}")
# By rsID
ids, _ = esearch("rs80357906[rs]")
print(f"Variant IDs for rs80357906: {ids}")

# By condition name
ids, total = esearch("breast cancer[dis] AND pathogenic[clinsig]")
print(f"Pathogenic variants for breast cancer: {total}")

Query 2: Fetch Variant Summary Records

Retrieve structured summary data (JSON) for a list of Variation IDs.

import requests, json

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def esummary(ids):
    """Fetch ESummary records for a list of ClinVar variation IDs."""
    r = requests.post(f"{BASE}/esummary.fcgi",
                      data={"db": "clinvar", "id": ",".join(ids),
                            "retmode": "json", "email": EMAIL})
    r.raise_for_status()
    return r.json()["result"]

ids, _ = esearch_func = lambda q: requests.get(
    f"{BASE}/esearch.fcgi",
    params={"db": "clinvar", "term": q, "retmax": 5, "retmode": "json", "email": EMAIL}
).json()["esearchresult"]["idlist"]

# Manual example with known IDs
sample_ids = ["12375", "17684", "54270"]
result = esummary(sample_ids)

for vid in result.get("uids", []):
    rec = result[vid]
    # ClinVar 2024 schema: clinical_significance was replaced by germline_classification
    # (also: clinical_impact_classification, oncogenicity_classification — same shape, often empty)
    gc = rec.get("germline_classification", {})
    print(f"\nVariation {vid}: {rec.get('title')}")
    print(f"  ClinSig  : {gc.get('description')}")
    print(f"  Review   : {gc.get('review_status')}")
    print(f"  Gene     : {rec.get('genes', [{}])[0].get('symbol')}")

Query 3: Fetch Full XML Records

Retrieve the complete variant record in XML for detailed submitter and condition data.

import requests
import xml.etree.ElementTree as ET

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def efetch_xml(variation_ids):
    # ClinVar 2024 XML overhaul: "clinvarset" rettype returns an empty stub.
    # Use rettype="vcv" + is_variationid="true" to get the new <VariationArchive> records.
    r = requests.post(f"{BASE}/efetch.fcgi",
                      data={"db": "clinvar", "id": ",".join(variation_ids),
                            "rettype": "vcv", "is_variationid": "true",
                            "retmode": "xml", "email": EMAIL})
    r.raise_for_status()
    return ET.fromstring(r.text)

root = efetch_xml(["17677"])  # BRCA1 c.5266dupC (rs80357906)

# Aggregate (germline) classification — one per VariationArchive
for va in root.iter("VariationArchive"):
    name = va.get("VariationName")
    gc = va.find("./ClassifiedRecord/Classifications/GermlineClassification")
    desc = gc.find("Description") if gc is not None else None
    rstat = gc.find("ReviewStatus") if gc is not None else None
    print(f"{name}: {desc.text if desc is not None else 'n/a'} "
          f"({rstat.text if rstat is not None else 'n/a'})")

    # Per-submitter assertions
    for ca in va.iter("ClinicalAssertion"):
        acc = ca.find("ClinVarAccession")
        cls = ca.find("Classification/GermlineClassification")
        if acc is not None and cls is not None:
            print(f"  {acc.get('SubmitterName', '?')}: {cls.text}")

Query 4: ClinVar FTP Bulk Data

For large-scale queries, download and parse the full variant summary file.

import urllib.request
import gzip, csv, io

# Full summary (tab-separated, ~300 MB compressed)
URL = "https://ftp.ncbi.nlm.nih.gov/pub/clinvar/tab_delimited/variant_summary.txt.gz"

# Stream and parse without full download
with urllib.request.urlopen(URL) as resp:
    with gzip.open(resp, "rt", encoding="utf-8") as f:
        reader = csv.DictReader(f, delimiter="\t")
        pathogenic_brca1 = []
        for row in reader:
            if row["GeneSymbol"] == "BRCA1" and "Pathogenic" in row["ClinicalSignificance"]:
                pathogenic_brca1.append({
                    "name": row["Name"],
                    "clinsig": row["ClinicalSignificance"],
                    "condition": row["PhenotypeList"],
                    "rsid": row["RS# (dbSNP)"],
                })
        print(f"Pathogenic BRCA1 variants: {len(pathogenic_brca1)}")
        for v in pathogenic_brca1[:3]:
            print(f"  {v['name']} | {v['clinsig']} | rs{v['rsid']}")

Query 5: Review Status and Conflicting Interpretations

Filter variants by review status (evidence quality) and find conflicts.

import requests

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

# Stars correspond to review levels:
# 0 = no assertion criteria, 1 = criteria provided (single),
# 2 = criteria provided (multiple), 3 = expert panel, 4 = practice guideline

def search_by_review_stars(gene, min_stars=2):
    """Search for variants with at least min_stars review status."""
    star_terms = {1: "criteria provided, single submitter",
                  2: "criteria provided, multiple submitters, no conflicts",
                  3: "reviewed by expert panel",
                  4: "practice guideline"}
    terms = [f'"{star_terms[s]}"[review status]' for s in range(min_stars, 5) if s in star_terms]
    query = f"{gene}[gene] AND (" + " OR ".join(terms) + ")"
    r = requests.get(f"{BASE}/esearch.fcgi",
                     params={"db": "clinvar", "term": query, "retmax": 100,
                             "retmode": "json", "email": EMAIL})
    return r.json()["esearchresult"]

result = search_by_review_stars("BRCA1", min_stars=3)
print(f"Expert-reviewed BRCA1 variants: {result['count']}")

Query 6: Variant-to-Condition Mapping

Extract condition (phenotype) data from ClinVar records.

import requests, json

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def get_conditions(variation_ids):
    """Return condition data for a list of ClinVar variation IDs."""
    r = requests.post(f"{BASE}/esummary.fcgi",
                      data={"db": "clinvar", "id": ",".join(variation_ids),
                            "retmode": "json", "email": EMAIL})
    r.raise_for_status()
    result = r.json()["result"]
    conditions = {}
    for vid in result.get("uids", []):
        rec = result[vid]
        # trait_set moved under germline_classification in the 2024 ClinVar JSON
        trait_set = rec.get("germline_classification", {}).get("trait_set", [])
        conditions[vid] = [t.get("trait_name") for t in trait_set]
    return conditions

sample_ids = ["12375", "17684", "54270"]
cond_map = get_conditions(sample_ids)
for vid, conds in cond_map.items():
    print(f"Variation {vid}: {', '.join(conds)}")

Key Concepts

ClinVar Variation ID vs. rsID

ClinVar assigns its own stable Variation ID (integer) to each interpreted variant record. This differs from dbSNP rsIDs. A single rsID can correspond to multiple ClinVar Variation IDs if different alleles or interpretations are submitted separately.

Review Stars and Evidence Quality

ClinVar's "review status" encodes the level of evidence:

  • 0 stars: No assertion criteria provided
  • 1 star: Criteria provided, single submitter
  • 2 stars: Multiple submitters, no conflict
  • 3 stars: Reviewed by expert panel (e.g., ENIGMA, ClinGen)
  • 4 stars: Practice guideline

Common Workflows

Workflow 1: Gene Pathogenicity Report

Goal: Retrieve all high-confidence pathogenic variants in a gene and export to CSV.

import requests, json, time, pandas as pd

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def search_gene_pathogenic(gene, clinsig="pathogenic"):
    query = f"{gene}[gene] AND {clinsig}[clinsig]"
    r = requests.get(f"{BASE}/esearch.fcgi",
                     params={"db": "clinvar", "term": query, "retmax": 500,
                             "retmode": "json", "email": EMAIL})
    return r.json()["esearchresult"]["idlist"]

def fetch_summaries(ids):
    records = []
    for i in range(0, len(ids), 100):
        batch = ids[i:i+100]
        r = requests.post(f"{BASE}/esummary.fcgi",
                          data={"db": "clinvar", "id": ",".join(batch),
                                "retmode": "json", "email": EMAIL})
        result = r.json()["result"]
        for vid in result.get("uids", []):
            rec = result[vid]
            # ClinVar 2024 schema: clinical_significance → germline_classification; trait_set nested inside it
            gc = rec.get("germline_classification", {})
            records.append({
                "variation_id": vid,
                "name": rec.get("title"),
                "clinsig": gc.get("description"),
                "review_status": gc.get("review_status"),
                "gene": ",".join(g.get("symbol", "") for g in rec.get("genes", [])),
                "conditions": "; ".join(t.get("trait_name", "") for t in gc.get("trait_set", [])),
            })
        time.sleep(0.15)
    return records

gene = "BRCA1"
ids = search_gene_pathogenic(gene)
print(f"Found {len(ids)} pathogenic variants in {gene}")

records = fetch_summaries(ids)
df = pd.DataFrame(records)
df.to_csv(f"{gene}_pathogenic_variants.csv", index=False)
print(f"Saved {len(df)} records → {gene}_pathogenic_variants.csv")
print(df[["name", "clinsig", "review_status"]].head())

Workflow 2: Variant Classification Check

Goal: Check ClinVar status for a list of user-provided rsIDs or HGVS notations.

import requests, time, pandas as pd

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

variants = ["rs80357906", "rs80357220", "rs28897672"]
results = []

for rsid in variants:
    r = requests.get(f"{BASE}/esearch.fcgi",
                     params={"db": "clinvar", "term": f"{rsid}[rs]",
                             "retmax": 5, "retmode": "json", "email": EMAIL})
    ids = r.json()["esearchresult"]["idlist"]
    if not ids:
        results.append({"rsid": rsid, "variation_id": None, "clinsig": "Not in ClinVar"})
        continue

    r2 = requests.post(f"{BASE}/esummary.fcgi",
                       data={"db": "clinvar", "id": ",".join(ids[:1]),
                             "retmode": "json", "email": EMAIL})
    rec = r2.json()["result"][ids[0]]
    gc = rec.get("germline_classification", {})  # 2024 ClinVar JSON
    results.append({
        "rsid": rsid,
        "variation_id": ids[0],
        "clinsig": gc.get("description", "Unknown"),
        "review_status": gc.get("review_status"),
    })
    time.sleep(0.15)

df = pd.DataFrame(results)
print(df.to_string(index=False))

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
retmaxESearch201–10000Max records returned per query
retmodeESearch/ESummary"xml""json", "xml"Response format
rettypeEFetch"vcv""vcv"Record type for XML fetch (legacy clinvarset returns empty stub since 2024)
is_variationidEFetch"false""true"/"false"Set to "true" when fetching by ClinVar Variation ID with rettype=vcv
clinsig query fieldESearch—"pathogenic", "likely pathogenic", "VUS"Filter by clinical significance
review status query fieldESearch—0–4 star termsFilter by evidence quality
emailAllrequiredvalid emailNCBI policy; prevents blocking

Best Practices

  1. Always set email: NCBI requires an email in all E-utility calls for rate-limit attribution and policy compliance.

  2. Use FTP bulk download for large queries: For more than ~1000 variants, download variant_summary.txt.gz from the ClinVar FTP rather than looping over EFetch — it's faster and avoids rate limits.

  3. Filter by review status: Automated pipelines should filter to ≥2-star variants to reduce noise from single-submitter assertions without peer review.

  4. Use API key for production: Register at https://www.ncbi.nlm.nih.gov/account/ to get a free API key (api_key parameter) and triple your rate limit (3 → 10 req/s).

  5. Handle VUS separately: "Conflicting interpretations of pathogenicity" is its own ClinSig category — don't combine it with "VUS" in filters; they have different implications for clinical decision-making.

Common Recipes

Recipe: Check if rsID Is in ClinVar

When to use: Quick lookup for a single known variant.

import requests

EMAIL = "your@email.com"
rsid = "rs80357906"

r = requests.get(
    "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi",
    params={"db": "clinvar", "term": f"{rsid}[rs]",
            "retmax": 1, "retmode": "json", "email": EMAIL}
)
count = int(r.json()["esearchresult"]["count"])
print(f"{rsid}: {'found' if count else 'NOT'} in ClinVar ({count} records)")

Recipe: Download Variant Summary TSV

When to use: Bulk analysis — load entire ClinVar into a pandas DataFrame.

import pandas as pd

url = "https://ftp.ncbi.nlm.nih.gov/pub/clinvar/tab_delimited/variant_summary.txt.gz"
# Only human GRCh38 pathogenic variants
df = pd.read_csv(url, sep="\t", compression="gzip",
                 usecols=["#AlleleID", "Name", "GeneSymbol", "ClinicalSignificance",
                          "ReviewStatus", "PhenotypeList", "Assembly", "RS# (dbSNP)"])
df = df[(df["Assembly"] == "GRCh38") & (df["ClinicalSignificance"].str.contains("Pathogenic", na=False))]
print(f"Pathogenic variants (GRCh38): {len(df)}")
df.to_csv("clinvar_pathogenic_grch38.csv", index=False)

Recipe: Search by OMIM Disease ID

When to use: Find all ClinVar variants associated with a specific OMIM condition.

import requests

EMAIL = "your@email.com"
omim_id = "604370"  # BRCA1-associated breast-ovarian cancer

r = requests.get(
    "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi",
    params={"db": "clinvar", "term": f"{omim_id}[MIM]",
            "retmax": 20, "retmode": "json", "email": EMAIL}
)
result = r.json()["esearchresult"]
print(f"Variants for OMIM {omim_id}: {result['count']} total")
print(f"First IDs: {result['idlist'][:5]}")

Troubleshooting

ProblemCauseSolution
HTTP 429 or no responseRate limit exceededAdd time.sleep(0.35) between requests; use API key
Empty idlist for rsID queryrsID not indexed in ClinVarTry HGVS notation or gene+position query instead
Missing clinsig in summaryVariant has no interpretationCheck review_status; "no interpretation for the single variant" means no ClinSig yet
XML parse error in EFetchIncomplete response (timeout)Set requests.get(..., timeout=30) and retry once
<ClinVarResult-Set><set/></ClinVarResult-Set> empty stubUsing legacy rettype="clinvarset" (deprecated in 2024)Switch to rettype="vcv" + is_variationid="true"; parse <VariationArchive> root
KeyError: clinical_significance in ESummary parsingField renamed in 2024 ClinVar JSONUse rec["germline_classification"] (also clinical_impact_classification, oncogenicity_classification); trait_set now nested inside germline_classification
Conflicting results for same rsIDMultiple submissions with different interpretationsGroup by review_status and prefer higher-star entries
FTP download failsLarge file / slow connectionUse pandas.read_csv with chunksize=100000 or pre-filter with grep

Related Skills

  • gwas-database — GWAS Catalog for population-level SNP-trait associations (complement to ClinVar's clinical assertions)
  • ensembl-database — Ensembl VEP for predicting variant consequences without requiring prior clinical curation
  • cosmic-database — Somatic cancer variant database (complementary to ClinVar's germline focus)
  • pubmed-database — Retrieve supporting publications cited in ClinVar submissions

References

COSMIC Somatic Cancer Mutations Database

Overview

COSMIC (Catalogue Of Somatic Mutations In Cancer) is the world's largest expert-curated database of somatic mutations in cancer, covering 6.7M+ coding mutations, 40,000+ cancer samples, 19,000+ genes across all cancer types. It includes the Cancer Gene Census (critical cancer genes), mutational signatures (SBS, DBS, ID), drug resistance variants, copy number data, gene expression, and methylation. The REST API v3.1 enables programmatic queries; most features are freely accessible after registration.

When to Use

  • Checking whether a specific somatic variant in a cancer gene is annotated in COSMIC (frequency, cancer type distribution)
  • Retrieving all somatic mutations in a gene of interest across COSMIC cancer samples
  • Accessing COSMIC Cancer Gene Census classifications (Tier 1/2, role: oncogene/TSG/fusion)
  • Looking up mutational signature attributions for samples or cancer types
  • Identifying drug resistance variants (pharmacogenomic data) from COSMIC drug resistance database
  • Building cancer driver gene lists for bioinformatic pipelines
  • For germline/inherited variants use clinvar-database; for drug-target associations use opentargets-database

Prerequisites

  • Python packages: requests, pandas
  • Data requirements: gene symbols (HGNC), COSMIC mutation IDs (COSM), sample IDs, or genomic coordinates
  • Environment: internet connection; free account registration at https://cancer.sanger.ac.uk/cosmic/register
  • Rate limits: authenticated requests only; 10 requests/second max; API key required
pip install requests pandas
# Register at https://cancer.sanger.ac.uk/cosmic/register to obtain API credentials

Quick Start

import requests
import base64

# COSMIC API requires base64-encoded email:password authentication
EMAIL = "your_registered@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

# Get mutations for KRAS gene
r = requests.get(f"{BASE}/mutations",
                 headers=HEADERS,
                 params={"gene_name": "KRAS", "limit": 5})
r.raise_for_status()
data = r.json()
print(f"Total KRAS mutations: {data['meta']['total']}")
for m in data["data"][:3]:
    print(f"  {m['mutation_id']:15s} AA: {m.get('mutation_aa')} | Cancer: {m.get('primary_site')}")

Core API

Query 1: Gene Mutations Search

Retrieve all COSMIC somatic mutations for a gene, with cancer type and amino acid change.

import requests, base64, pandas as pd

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

def get_gene_mutations(gene, limit=100, cancer_site=None):
    params = {"gene_name": gene, "limit": limit}
    if cancer_site:
        params["primary_site"] = cancer_site
    r = requests.get(f"{BASE}/mutations", headers=HEADERS, params=params)
    r.raise_for_status()
    return r.json()

data = get_gene_mutations("TP53", limit=20)
print(f"Total TP53 mutations in COSMIC: {data['meta']['total']}")

rows = []
for m in data["data"][:10]:
    rows.append({
        "mutation_id": m.get("mutation_id"),
        "mutation_aa": m.get("mutation_aa"),
        "mutation_cds": m.get("mutation_cds"),
        "primary_site": m.get("primary_site"),
        "histology": m.get("primary_histology"),
        "count": m.get("count"),
    })
df = pd.DataFrame(rows)
print(df.head())
# Filter by cancer site
data_lung = get_gene_mutations("TP53", cancer_site="lung", limit=20)
print(f"\nTP53 mutations in lung cancer: {data_lung['meta']['total']}")

Query 2: Cancer Gene Census

Retrieve the COSMIC Cancer Gene Census — classified cancer driver genes.

import requests, base64, pandas as pd

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

r = requests.get(f"{BASE}/genes", headers=HEADERS, params={"limit": 100})
r.raise_for_status()
data = r.json()
print(f"Total genes in COSMIC: {data['meta']['total']}")

# Get Cancer Gene Census genes
r_cgc = requests.get(f"{BASE}/genes",
                     headers=HEADERS,
                     params={"cgc_tier": "1", "limit": 50})
cgc_data = r_cgc.json()
print(f"\nCGC Tier 1 genes: {cgc_data['meta']['total']}")

rows = []
for g in cgc_data["data"][:15]:
    rows.append({
        "gene": g.get("gene_name"),
        "tier": g.get("cgc_tier"),
        "role": g.get("role_in_cancer"),
        "mutation_types": g.get("mutation_types"),
        "tumour_types": str(g.get("tumour_types_somatic", []))[:80],
    })
df = pd.DataFrame(rows)
print(df.to_string(index=False))

Query 3: Specific Mutation Lookup

Retrieve details for a known COSMIC mutation ID (COSM…).

import requests, base64

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

# KRAS G12D mutation
mutation_id = "COSM521"
r = requests.get(f"{BASE}/mutations/{mutation_id}", headers=HEADERS)
r.raise_for_status()
m = r.json()

print(f"Mutation ID : {m.get('mutation_id')}")
print(f"Gene        : {m.get('gene_name')}")
print(f"AA change   : {m.get('mutation_aa')}")
print(f"CDS change  : {m.get('mutation_cds')}")
print(f"Substitution: {m.get('mutation_description')}")
print(f"Count       : {m.get('count')} samples")
print(f"Cancer types: {str(m.get('cancer_types', []))[:100]}")

Query 4: Sample-Level Mutation Data

Retrieve all somatic mutations for a specific cancer sample.

import requests, base64, pandas as pd

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

# Search for a specific sample
r = requests.get(f"{BASE}/samples",
                 headers=HEADERS,
                 params={"primary_site": "breast", "limit": 5})
r.raise_for_status()
samples = r.json()["data"]
print(f"Example breast cancer samples:")
for s in samples[:3]:
    print(f"  {s.get('sample_id')}: {s.get('sample_name')} | {s.get('primary_histology')}")

# Get mutations for a specific sample
if samples:
    sample_id = samples[0]["sample_id"]
    r2 = requests.get(f"{BASE}/samples/{sample_id}/mutations", headers=HEADERS)
    if r2.ok:
        muts = r2.json()["data"]
        print(f"\nMutations in sample {sample_id}: {len(muts)}")
        for m in muts[:5]:
            print(f"  {m.get('gene_name'):10s} {m.get('mutation_aa')}")

Query 5: Mutational Signatures

Retrieve COSMIC mutational signature data for cancer types.

import requests, base64, pandas as pd

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

# List available mutational signatures
r = requests.get(f"{BASE}/signatures", headers=HEADERS)
r.raise_for_status()
sigs = r.json()["data"]
print(f"COSMIC mutational signatures: {len(sigs)}")
for s in sigs[:5]:
    print(f"  {s.get('signature_name')}: {s.get('aetiology', '')[:80]}")
# Get signature attributions by cancer type
r2 = requests.get(f"{BASE}/signatures/attributions",
                  headers=HEADERS,
                  params={"cancer_type": "Breast", "limit": 10})
if r2.ok:
    attributions = r2.json()["data"]
    for a in attributions[:5]:
        print(f"  {a.get('signature_name')}: {a.get('attribution_proportion'):.2%} in breast cancer")

Query 6: Drug Resistance Variants

Query the COSMIC drug resistance database for variants conferring drug resistance.

import requests, base64, pandas as pd

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

# Get drug resistance variants
r = requests.get(f"{BASE}/resistance_mutations",
                 headers=HEADERS,
                 params={"gene": "EGFR", "limit": 20})
if r.ok:
    data = r.json()
    print(f"EGFR drug resistance variants: {data['meta'].get('total', 'n/a')}")
    for v in data.get("data", [])[:5]:
        print(f"  {v.get('mutation_aa'):20s} Drug: {v.get('drug')} | Resistance: {v.get('resistance_type')}")
else:
    print(f"Drug resistance API: {r.status_code} - endpoint may require specific access level")

Key Concepts

Cancer Gene Census Tiers

COSMIC's Cancer Gene Census classifies genes into:

  • Tier 1: Well-established cancer drivers with documented mutations and molecular mechanisms in cancer
  • Tier 2: Genes with strong evidence for roles in cancer but less functional characterization

Mutation ID Stability

COSMIC mutation IDs (COSM…) are stable identifiers for specific amino acid changes in a gene. The same COSM ID appears across all samples with that mutation, allowing cross-study comparison.

Common Workflows

Workflow 1: Gene Hotspot Mutation Analysis

Goal: Identify the most frequently occurring somatic mutations in a cancer gene.

import requests, base64, pandas as pd
from collections import Counter

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

def get_all_gene_mutations(gene, max_records=1000):
    """Paginate through all COSMIC mutations for a gene."""
    all_muts = []
    skip = 0
    limit = 200
    while len(all_muts) < max_records:
        r = requests.get(f"{BASE}/mutations",
                         headers=HEADERS,
                         params={"gene_name": gene, "limit": limit, "skip": skip})
        r.raise_for_status()
        batch = r.json()["data"]
        if not batch:
            break
        all_muts.extend(batch)
        total = r.json()["meta"]["total"]
        skip += limit
        if skip >= total:
            break
    return all_muts

# Get hotspots for KRAS
mutations = get_all_gene_mutations("KRAS", max_records=500)
print(f"Retrieved {len(mutations)} KRAS somatic mutations")

# Rank by amino acid change frequency
aa_counter = Counter(m["mutation_aa"] for m in mutations if m.get("mutation_aa"))
hotspots = pd.DataFrame(aa_counter.most_common(15), columns=["mutation_aa", "sample_count"])
print("\nKRAS hotspot mutations:")
print(hotspots.head(10).to_string(index=False))
hotspots.to_csv("KRAS_hotspots.csv", index=False)

Workflow 2: Cancer Gene Census Export

Goal: Export the full Cancer Gene Census as a structured table for downstream pipeline use.

import requests, base64, pandas as pd, time

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}

all_genes = []
for tier in [1, 2]:
    skip = 0
    while True:
        r = requests.get(f"{BASE}/genes",
                         headers=HEADERS,
                         params={"cgc_tier": str(tier), "limit": 100, "skip": skip})
        r.raise_for_status()
        batch = r.json()["data"]
        if not batch:
            break
        all_genes.extend(batch)
        if len(batch) < 100:
            break
        skip += 100
        time.sleep(0.1)

rows = [{
    "gene": g.get("gene_name"),
    "tier": g.get("cgc_tier"),
    "role_in_cancer": g.get("role_in_cancer"),
    "mutation_types": g.get("mutation_types"),
    "somatic_tumours": str(g.get("tumour_types_somatic", [])),
    "germline_tumours": str(g.get("tumour_types_germline", [])),
    "chr": g.get("chromosomal_location"),
} for g in all_genes]

df = pd.DataFrame(rows)
df.to_csv("COSMIC_cancer_gene_census.csv", index=False)
print(f"Exported {len(df)} Cancer Gene Census genes → COSMIC_cancer_gene_census.csv")
print(df.groupby("tier")["gene"].count())

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
gene_nameMutations—HGNC symbolFilter mutations by gene
primary_siteMutations/Samples—tissue type stringFilter by primary tumor site
limitAll101–200Records per page
skipAll0integerPagination offset
cgc_tierGenes—"1", "2"Cancer Gene Census tier
mutation_idMutations—COSM ID stringLookup specific mutation

Best Practices

  1. Authenticate via Base64: COSMIC uses HTTP Basic Auth with base64-encoded email:password. Store credentials in environment variables, not in code.

  2. Paginate large gene queries: Popular cancer genes (TP53, KRAS) have 100,000+ mutation records; use skip/limit pagination and cache results locally.

  3. Use COSM IDs for cross-study comparison: Amino acid change strings may have formatting variations (p.G12D vs G12D); use COSMIC mutation IDs (COSM…) for unambiguous references.

  4. Check data license for commercial use: COSMIC data is free for academic use but requires a commercial license for industry applications. Verify at https://cancer.sanger.ac.uk/cosmic/license.

  5. Complement with clinical data: COSMIC captures somatic mutations from cancer sequencing; complement with clinvar-database for germline pathogenicity and opentargets-database for therapeutic significance.

Common Recipes

Recipe: Top Mutated Genes in a Cancer Type

When to use: Identify frequently mutated genes in a specific cancer type.

import requests, base64, pandas as pd
from collections import Counter

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

HEADERS = {"Authorization": f"Basic {token}"}
r = requests.get("https://cancer.sanger.ac.uk/cosmic/api/mutations",
                 headers=HEADERS,
                 params={"primary_site": "lung", "limit": 200})
data = r.json()["data"]
gene_counts = Counter(m.get("gene_name") for m in data if m.get("gene_name"))
df = pd.DataFrame(gene_counts.most_common(10), columns=["gene", "mutations"])
print(df.to_string(index=False))

Recipe: Check if a Variant Is in COSMIC

When to use: Look up whether a specific amino acid change is recorded in COSMIC.

import requests, base64

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

HEADERS = {"Authorization": f"Basic {token}"}
gene = "KRAS"
aa_change = "p.G12D"

r = requests.get("https://cancer.sanger.ac.uk/cosmic/api/mutations",
                 headers=HEADERS,
                 params={"gene_name": gene, "limit": 200})
all_muts = r.json()["data"]
matches = [m for m in all_muts if aa_change in (m.get("mutation_aa") or "")]
print(f"{gene} {aa_change}: {'FOUND' if matches else 'NOT FOUND'} in COSMIC ({len(matches)} records)")
if matches:
    print(f"  Sample count: {sum(m.get('count', 0) for m in matches)}")

Recipe: Download CGC Tier 1 Gene List

When to use: Get a simple list of Tier 1 cancer driver genes for filtering.

import requests, base64

EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()

HEADERS = {"Authorization": f"Basic {token}"}
r = requests.get("https://cancer.sanger.ac.uk/cosmic/api/genes",
                 headers=HEADERS,
                 params={"cgc_tier": "1", "limit": 200})
genes = [g["gene_name"] for g in r.json()["data"]]
print(f"CGC Tier 1 genes ({len(genes)}): {', '.join(genes[:10])}...")
with open("cosmic_tier1_genes.txt", "w") as f:
    f.write("\n".join(genes))

Troubleshooting

ProblemCauseSolution
HTTP 401 UnauthorizedMissing or incorrect API credentialsCheck base64 encoding: base64.b64encode(f"{email}:{password}".encode())
HTTP 403 ForbiddenAccess requires different tierSome endpoints need commercial license; check COSMIC license page
Empty data arrayNo records match filterBroaden query; check spelling of gene symbol or site name
Very slow for large genesTP53/KRAS have 100K+ recordsPaginate with small limit=200; cache results to local CSV
Rate limit errors>10 req/sAdd time.sleep(0.15) between requests
Different AA notation formatVarious mutation string formatsNormalize with RDKit or use COSM IDs for exact matching

Related Skills

  • clinvar-database — Germline pathogenicity classifications complementing COSMIC's somatic focus
  • opentargets-database — Drug-target associations for COSMIC cancer driver genes
  • ensembl-database — Variant consequence predictions (VEP) for COSMIC variants
  • gwas-database — Population-level SNP associations for cancer risk (vs. COSMIC's somatic mutations)

References

dbSNP Database

Overview

NCBI dbSNP is the primary public repository for short human genetic variants, cataloguing over 1 billion SNPs, indels, and MNVs with allele frequencies, functional annotations, and cross-references to ClinVar, gnomAD, and 1000 Genomes. Variants are identified by stable rsIDs (reference SNP cluster IDs). Access is free via two APIs: the legacy NCBI E-utilities and the newer NCBI Variation Services REST API, which returns structured JSON.

When to Use

  • Looking up allele frequencies and variant class for a known rsID
  • Searching all dbSNP variants in a gene or chromosomal region by name or coordinates
  • Resolving rsIDs to genomic coordinates (GRCh38/GRCh37) and HGVS notation
  • Checking whether a variant of interest has clinical significance links to ClinVar entries
  • Batch-fetching hundreds of rsIDs efficiently using epost+efetch history server
  • Cross-referencing a list of variant positions to dbSNP rsIDs for downstream annotation
  • For clinical pathogenicity classifications use clinvar-database; dbSNP provides IDs and frequency but not curated clinical significance
  • For population frequency stratified by ancestry use gnomad-database; dbSNP MAF is a single aggregate frequency

Prerequisites

  • Python packages: requests, pandas, matplotlib, xml.etree.ElementTree (stdlib)
  • Data requirements: rsIDs (rs80357906), gene symbols, or chromosomal coordinates
  • Environment: internet connection; NCBI Entrez email required for E-utilities (set email parameter)
  • Rate limits: 3 requests/second without API key; 10 requests/second with free NCBI API key. Register at https://www.ncbi.nlm.nih.gov/account/ — add &api_key=YOUR_KEY to all requests
pip install requests pandas matplotlib
# xml.etree.ElementTree is part of Python stdlib — no additional install needed

Quick Start

import requests
import json

EMAIL = "your@email.com"      # required by NCBI policy
BASE_EUTILS = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
BASE_VARIATION = "https://api.ncbi.nlm.nih.gov/variation/v0"

def fetch_snp_by_rsid(rsid: str) -> dict:
    """Fetch a dbSNP record by rsID using the NCBI Variation Services API (structured JSON)."""
    rs_num = str(rsid).lstrip("rs")
    r = requests.get(f"{BASE_VARIATION}/refsnp/{rs_num}", timeout=15)
    r.raise_for_status()
    return r.json()

record = fetch_snp_by_rsid("rs1800497")  # DRD2 Taq1A
print(f"rsID: rs{record['refsnp_id']}")
print(f"Variant type: {record['primary_snapshot_data'].get('variant_type')}")
# Top-level keys: citations, create_date, dbsnp1_merges, last_update_build_id,
# last_update_date, lost_obs_movements, mane_select_ids, present_obs_movements,
# primary_snapshot_data, refsnp_id. (No top-level `organism` field.)
# rsID: rs1800497
# Variant type: snv

Core API

Query 1: rsID Lookup via E-utilities

Fetch the full SNP record for a single rsID using efetch with db=snp. Returns an XML document with alleles, placements, and frequency data.

import requests
import xml.etree.ElementTree as ET

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def efetch_snp_xml(rsid: str) -> ET.Element:
    """Fetch dbSNP XML record for a single rsID via the docsum rettype.
    Note: rettype="xml" returns a namespaced ExchangeSet; rettype="docsum"
    returns the simpler eSummaryResult/DocumentSummary tree without namespaces."""
    rs_num = str(rsid).lstrip("rs")
    r = requests.get(f"{BASE}/efetch.fcgi",
                     params={"db": "snp", "id": rs_num,
                             "rettype": "docsum", "retmode": "xml",
                             "email": EMAIL},
                     timeout=20)
    r.raise_for_status()
    return ET.fromstring(r.text)

root = efetch_snp_xml("rs80357906")

# Parse the DocumentSummary record (MAF/MAFALLELE were removed in 2024;
# GLOBAL_MAFS is a sub-tree — use ESummary JSON below for easier access)
for docsum in root.iter("DocumentSummary"):
    rs_id = docsum.get("uid")
    snp_class = docsum.findtext("SNP_CLASS", "Unknown")
    chr_pos = docsum.findtext("CHRPOS", "N/A")
    clin_sig = docsum.findtext("CLINICAL_SIGNIFICANCE", "N/A")
    print(f"rs{rs_id} | Class: {snp_class} | Position: {chr_pos}")
    print(f"  ClinSig: {clin_sig}")
# rs80357906 | Class: delins | Position: 17:43057062
#   ClinSig: pathogenic,risk-factor,uncertain-significance
# Fetch using ESummary for structured JSON (preferred for batch)
def esummary_snp(rsid: str) -> dict:
    rs_num = str(rsid).lstrip("rs")
    r = requests.get(f"{BASE}/esummary.fcgi",
                     params={"db": "snp", "id": rs_num,
                             "retmode": "json", "email": EMAIL},
                     timeout=15)
    r.raise_for_status()
    result = r.json()["result"]
    return result.get(rs_num, {})

rec = esummary_snp("rs80357906")
print(f"rs{rec.get('snp_id')}:")
print(f"  Class       : {rec.get('snp_class')}")            # e.g., 'delins'
# `maf`/`mafallele` were removed from ESummary in 2024 — use `global_mafs`
# (list of {study, freq}) and pick a study (e.g., 'GnomAD_genomes') or the
# global aggregate ('TOPMED'/'1000Genomes').
for m in rec.get('global_mafs', [])[:4]:
    print(f"  MAF[{m['study']:18s}]: {m['freq']}")
print(f"  ChrPos      : {rec.get('chrpos')}")               # 17:43057062
print(f"  ClinSig     : {rec.get('clinical_significance')}")
print(f"  FxnClass    : {rec.get('fxn_class')}")

Query 2: Gene Variant Search

Search dbSNP for all variants in a gene using esearch. Returns a list of rsIDs matching the gene.

import requests

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def esearch_snp(query: str, retmax: int = 100) -> tuple[list, int]:
    """Search dbSNP using a query string. Returns (id_list, total_count)."""
    r = requests.get(f"{BASE}/esearch.fcgi",
                     params={"db": "snp", "term": query,
                             "retmax": retmax, "retmode": "json",
                             "email": EMAIL},
                     timeout=15)
    r.raise_for_status()
    result = r.json()["esearchresult"]
    return result["idlist"], int(result["count"])

# All variants in BRCA1
ids, total = esearch_snp("BRCA1[gene] AND human[orgn]", retmax=20)
print(f"BRCA1 variants in dbSNP: {total:,} total")
print(f"First 5 rsIDs: {['rs' + i for i in ids[:5]]}")

# Only clinical variants (linked to ClinVar)
ids_clin, total_clin = esearch_snp(
    "BRCA1[gene] AND human[orgn] AND clinsig[filter]", retmax=50)
print(f"BRCA1 variants with clinical significance: {total_clin:,}")

Query 3: Chromosomal Region Search

Search for all variants in a genomic region using chromosome coordinates.

import requests

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def search_region(chrom: str, start: int, stop: int,
                  assembly: str = "GRCh38", retmax: int = 200) -> tuple[list, int]:
    """Find all dbSNP variants in a chromosomal region."""
    query = f"{chrom}[CHR] AND {start}:{stop}[CHRPOS37]" if assembly == "GRCh37" else \
            f"{chrom}[CHR] AND {start}:{stop}[CHRPOS]"
    r = requests.get(f"{BASE}/esearch.fcgi",
                     params={"db": "snp", "term": query,
                             "retmax": retmax, "retmode": "json",
                             "email": EMAIL},
                     timeout=20)
    r.raise_for_status()
    result = r.json()["esearchresult"]
    return result["idlist"], int(result["count"])

# PCSK9 exon 4 region (GRCh38)
ids, total = search_region("1", 55039700, 55040200)
print(f"Variants in chr1:55039700-55040200: {total:,} total")
print(f"Retrieved {len(ids)} rsIDs: {['rs' + i for i in ids[:5]]}")

Query 4: Variant Summary — MAF, Alleles, Clinical Significance

Retrieve structured summary data for variant records using ESummary, extracting MAF, alleles, and database cross-links.

import requests, json

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def fetch_snp_summaries(rsids: list) -> dict:
    """Fetch ESummary records for a list of rsIDs. Returns dict keyed by rs number."""
    ids_str = ",".join(str(r).lstrip("rs") for r in rsids)
    r = requests.post(f"{BASE}/esummary.fcgi",
                      data={"db": "snp", "id": ids_str,
                            "retmode": "json", "email": EMAIL},
                      timeout=20)
    r.raise_for_status()
    return r.json()["result"]

rsids = ["rs80357906", "rs80357220", "rs28897672", "rs1801133"]
result = fetch_snp_summaries(rsids)

for rs_num in rsids:
    uid = str(rs_num).lstrip("rs")
    rec = result.get(uid, {})
    # `global_mafs` is a list of {study, freq}; pick the first or filter by study
    gmafs = rec.get("global_mafs", [])
    maf_str = gmafs[0]["freq"] if gmafs else "N/A"
    maf_study = gmafs[0]["study"] if gmafs else ""
    print(f"\n{rs_num}:")
    print(f"  Class        : {rec.get('snp_class', 'N/A')}")
    print(f"  MAF          : {maf_str} (from {maf_study})")
    print(f"  Location     : {rec.get('chrpos', 'N/A')}")
    print(f"  ClinSig      : {rec.get('clinical_significance', 'N/A')}")
    print(f"  Function     : {rec.get('fxn_class', 'N/A')}")

Query 5: Batch rsID Query with EPost+EFetch

Efficiently upload hundreds of rsIDs to the NCBI history server using EPost, then retrieve them in batches with EFetch.

import requests, time, pandas as pd

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def epost_ids(id_list: list) -> tuple[str, str]:
    """Upload rsIDs to NCBI history server. Returns (WebEnv, query_key)."""
    ids_str = ",".join(str(i).lstrip("rs") for i in id_list)
    r = requests.post(f"{BASE}/epost.fcgi",
                      data={"db": "snp", "id": ids_str, "email": EMAIL},
                      timeout=30)
    r.raise_for_status()
    import xml.etree.ElementTree as ET
    root = ET.fromstring(r.text)
    webenv = root.findtext("WebEnv")
    query_key = root.findtext("QueryKey")
    return webenv, query_key

def efetch_history(webenv: str, query_key: str,
                   retstart: int = 0, retmax: int = 100) -> dict:
    """Retrieve records from NCBI history server using ESummary."""
    r = requests.get(f"{BASE}/esummary.fcgi",
                     params={"db": "snp", "WebEnv": webenv,
                             "query_key": query_key, "retstart": retstart,
                             "retmax": retmax, "retmode": "json",
                             "email": EMAIL},
                     timeout=30)
    r.raise_for_status()
    return r.json()["result"]

rsid_batch = ["rs80357906", "rs80357220", "rs28897672", "rs1801133",
              "rs429358", "rs7412", "rs1800497"]

webenv, query_key = epost_ids(rsid_batch)
print(f"Posted {len(rsid_batch)} IDs | WebEnv: {webenv[:40]}...")

records = []
for start in range(0, len(rsid_batch), 100):
    result = efetch_history(webenv, query_key, retstart=start, retmax=100)
    for uid in result.get("uids", []):
        rec = result[uid]
        # global_mafs replaced maf/mafallele in 2024 — flatten the first entry
        gmafs = rec.get("global_mafs", [])
        records.append({
            "rsid": f"rs{uid}",
            "snp_class": rec.get("snp_class"),
            "maf": gmafs[0]["freq"] if gmafs else None,
            "maf_study": gmafs[0]["study"] if gmafs else None,
            "chrpos": rec.get("chrpos"),
            "clinical_sig": rec.get("clinical_significance"),
        })
    time.sleep(0.5)

df = pd.DataFrame(records)
print(f"\nRetrieved {len(df)} records:")
print(df.to_string(index=False))

Query 6: NCBI Variation Services API

The newer REST API returns structured JSON with detailed allele placements, frequencies, and variant type annotations. Preferred for programmatic rsID resolution.

import requests
import pandas as pd

BASE_VARIATION = "https://api.ncbi.nlm.nih.gov/variation/v0"

def fetch_refsnp(rsid: str) -> dict:
    """Fetch structured JSON from NCBI Variation Services API."""
    rs_num = str(rsid).lstrip("rs")
    r = requests.get(f"{BASE_VARIATION}/refsnp/{rs_num}", timeout=15)
    r.raise_for_status()
    return r.json()

def parse_allele_frequencies(record: dict) -> list:
    """Extract allele frequencies from a Variation Services record.
    freq[i] keys: study_name, study_version, local_row_id, observation, allele_count, total_count.
    The asserted allele identifier lives in freq.observation.inserted_sequence (NOT at ann.allele)."""
    freqs = []
    snapshot = record.get("primary_snapshot_data", {})
    for ann in snapshot.get("allele_annotations", []):
        for f in ann.get("frequency", []):
            obs = f.get("observation", {})
            freqs.append({
                "allele": obs.get("inserted_sequence"),
                "study": f.get("study_name"),
                "allele_count": f.get("allele_count"),
                "total_count": f.get("total_count"),
                "freq": (f["allele_count"] / f["total_count"]
                         if f.get("total_count") else None),
            })
    return freqs

record = fetch_refsnp("rs1800497")   # DRD2 Taq1A variant
print(f"rsID: rs{record['refsnp_id']}")
print(f"Variant type: {record['primary_snapshot_data'].get('variant_type')}")  # 'snv'

placements = record["primary_snapshot_data"].get("placements_with_allele", [])
for placement in placements[:2]:
    seq_id = placement.get("seq_id")
    for allele in placement.get("alleles", []):
        spdi = allele.get("allele", {}).get("spdi", {})
        print(f"  Placement: {seq_id} | SPDI: {spdi.get('inserted_sequence')}")

freqs = parse_allele_frequencies(record)
if freqs:
    df_freq = pd.DataFrame(freqs[:5])
    print(f"\nAllele frequencies ({len(freqs)} entries):")
    print(df_freq.to_string(index=False))

Key Concepts

rsID vs. ss ID (Submitted SNP)

dbSNP uses two IDs: rs IDs (Reference SNP cluster IDs) are stable public identifiers assigned after clustering submitted variants. ss IDs (Submitted SNP IDs) are assigned to individual laboratory submissions before clustering. Use rs IDs for all queries — ss IDs are internal and submission-specific. A single rs ID may cluster multiple ss IDs from different submissions.

MAF vs. Clinical Significance

  • MAF (Minor Allele Frequency): The aggregate frequency of the minor allele across all dbSNP submissions. This is a population-level statistic aggregated from 1000 Genomes, gnomAD, TOPMED, and other studies. It does not indicate whether the variant is pathogenic.
  • Clinical significance: A link field pointing to ClinVar classifications (Pathogenic, VUS, Benign, etc.). A variant can have a very low MAF (rare) yet be classified as Benign, or a moderate MAF yet be Pathogenic in a specific context (e.g., founder variants). Use clinvar-database for the full pathogenicity record.

Variant Classes in dbSNP

ClassDescriptionExample
snvSingle nucleotide variant (A>T)rs80357906
indelInsertion or deletionrs786201005
mnvMulti-nucleotide variantrs1057519737
insPure insertionrs113993960
delPure deletionrs66767301
microsatelliteSTR (short tandem repeat)rs5030655

Common Workflows

Workflow 1: Batch rsID Annotation from a Variant List

Goal: Given a list of rsIDs from a variant call pipeline, retrieve MAF, position, and clinical significance for all variants in one run.

import requests, time, pandas as pd

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def epost_snp(rsids):
    ids = ",".join(str(r).lstrip("rs") for r in rsids)
    r = requests.post(f"{BASE}/epost.fcgi",
                      data={"db": "snp", "id": ids, "email": EMAIL}, timeout=30)
    r.raise_for_status()
    import xml.etree.ElementTree as ET
    root = ET.fromstring(r.text)
    return root.findtext("WebEnv"), root.findtext("QueryKey")

def esummary_history(webenv, query_key, start, retmax=100):
    r = requests.get(f"{BASE}/esummary.fcgi",
                     params={"db": "snp", "WebEnv": webenv,
                             "query_key": query_key, "retstart": start,
                             "retmax": retmax, "retmode": "json", "email": EMAIL},
                     timeout=30)
    r.raise_for_status()
    return r.json()["result"]

# Example: VCF post-processing — annotate a list of called variants
variant_rsids = [
    "rs80357906", "rs80357220", "rs28897672", "rs1801133",
    "rs429358", "rs7412", "rs1800497", "rs2230199",
]

print(f"Posting {len(variant_rsids)} rsIDs to NCBI history server...")
webenv, query_key = epost_snp(variant_rsids)

records = []
for start in range(0, len(variant_rsids), 100):
    result = esummary_history(webenv, query_key, start=start, retmax=100)
    for uid in result.get("uids", []):
        rec = result[uid]
        gmafs = rec.get("global_mafs", [])  # 2024 schema (replaced maf/mafallele)
        records.append({
            "rsid": f"rs{uid}",
            "snp_class": rec.get("snp_class"),
            "maf": gmafs[0]["freq"] if gmafs else None,
            "maf_study": gmafs[0]["study"] if gmafs else None,
            "chrpos_grch38": rec.get("chrpos"),
            "gene": rec.get("genes", [{}])[0].get("name") if rec.get("genes") else None,
            "clinical_significance": rec.get("clinical_significance"),
            "fxn_class": rec.get("fxn_class"),
        })
    time.sleep(0.5)

df = pd.DataFrame(records)
df.to_csv("variant_annotations.csv", index=False)
print(f"Annotated {len(df)} variants → variant_annotations.csv")
print(df[["rsid", "snp_class", "maf", "chrpos_grch38", "clinical_significance"]].to_string(index=False))

Workflow 2: Gene Variant Class Distribution Visualization

Goal: Search all dbSNP variants in a gene and plot their variant class distribution.

import requests, time
import pandas as pd
import matplotlib.pyplot as plt

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def search_gene_variants(gene: str, retmax: int = 500) -> list:
    r = requests.get(f"{BASE}/esearch.fcgi",
                     params={"db": "snp", "term": f"{gene}[gene] AND human[orgn]",
                             "retmax": retmax, "retmode": "json", "email": EMAIL},
                     timeout=20)
    r.raise_for_status()
    return r.json()["esearchresult"]["idlist"]

def fetch_summaries_batch(ids: list, batch_size: int = 100) -> list:
    records = []
    for i in range(0, len(ids), batch_size):
        batch = ids[i:i+batch_size]
        ids_str = ",".join(batch)
        r = requests.post(f"{BASE}/esummary.fcgi",
                          data={"db": "snp", "id": ids_str,
                                "retmode": "json", "email": EMAIL},
                          timeout=30)
        r.raise_for_status()
        result = r.json()["result"]
        for uid in result.get("uids", []):
            rec = result[uid]
            records.append({"rsid": f"rs{uid}", "snp_class": rec.get("snp_class", "unknown")})
        time.sleep(0.5)
    return records

gene = "CFTR"
print(f"Searching dbSNP for {gene} variants...")
ids = search_gene_variants(gene, retmax=300)
print(f"Found {len(ids)} IDs; fetching summaries...")

records = fetch_summaries_batch(ids)
df = pd.DataFrame(records)
class_counts = df["snp_class"].value_counts()

# Plot variant class distribution
fig, ax = plt.subplots(figsize=(8, 5))
colors = ["#4472C4", "#ED7D31", "#A9D18E", "#FF0000", "#FFC000", "#7030A0"]
bars = ax.bar(class_counts.index, class_counts.values,
              color=colors[:len(class_counts)], edgecolor="white")
ax.bar_label(bars, padding=3, fontsize=9)
ax.set_xlabel("Variant Class")
ax.set_ylabel("Count")
ax.set_title(f"dbSNP Variant Classes in {gene} (n={len(df)})")
plt.tight_layout()
plt.savefig(f"{gene}_variant_classes.png", dpi=150, bbox_inches="tight")
print(f"Saved {gene}_variant_classes.png")
print(class_counts.to_string())
# snv                 241
# indel                38
# mnv                  12
# del                   7
# ins                   2

Key Parameters

ParameterFunction/EndpointDefaultRange / OptionsEffect
dbAll E-utilitiesrequired"snp"Database selector; must be "snp" for dbSNP queries
idefetch, esummary, epostrequiredrsID number(s) without rs prefixVariant identifier(s) to fetch
termesearchrequireddbSNP query stringSearch expression with field tags: [gene], [CHR], [CHRPOS], [rs], clinsig[filter]
retmaxesearch201–10000Maximum records returned per search
retmodeesearch, esummary"xml""json", "xml"Response format; use "json" for easy parsing
rettypeefetch"docsum""docsum", "xml"Record type for efetch responses
WebEnv + query_keyesummary, efetch—from epost responseHistory server tokens for batch retrieval
emailAll E-utilitiesrequiredvalid email stringNCBI policy; used for rate attribution
api_keyAll E-utilitiesoptionalNCBI API key stringRaises rate limit from 3 to 10 req/sec

Best Practices

  1. Register for a free NCBI API key: Adds api_key=YOUR_KEY to requests and triples your rate limit (3 → 10 req/sec) with no other changes. Register at https://www.ncbi.nlm.nih.gov/account/.

  2. Use epost+esummary for batches of more than 10 rsIDs: Avoid looping individual efetch calls. EPost uploads all IDs in one request to the history server; subsequent ESummary calls retrieve them in configurable batches of up to 500.

  3. Prefer NCBI Variation Services API for structured JSON: The /variation/v0/refsnp/{rs_num} endpoint returns a fully structured JSON with SPDI allele representations, placements, and frequency tables. Easier to parse than E-utilities XML for modern applications.

  4. Check snp_class before interpreting MAF: Indels and MNVs use different allele counting conventions than SNVs. Treat multi-allelic sites carefully — the reported MAF may refer to one allele among several.

  5. Combine dbSNP with ClinVar lookups: dbSNP records the clinical_significance field as a string (e.g., "pathogenic") but does not contain submitter details, review status, or HGVS details. Use clinvar-database for the full pathogenicity record when clinical interpretation is required.

Common Recipes

Recipe: Quick rsID Existence and Class Check

When to use: Check whether a variant is registered in dbSNP before downstream annotation.

import requests

EMAIL = "your@email.com"

def check_rsid(rsid: str) -> dict:
    """Check if an rsID exists in dbSNP and return basic info."""
    rs_num = str(rsid).lstrip("rs")
    r = requests.get(
        "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi",
        params={"db": "snp", "id": rs_num, "retmode": "json", "email": EMAIL},
        timeout=10
    )
    result = r.json().get("result", {})
    rec = result.get(rs_num, {})
    if not rec or "error" in rec:
        return {"rsid": rsid, "found": False}
    gmafs = rec.get("global_mafs", [])
    return {
        "rsid": rsid,
        "found": True,
        "snp_class": rec.get("snp_class"),
        "chrpos": rec.get("chrpos"),
        "maf": gmafs[0]["freq"] if gmafs else None,
        "maf_study": gmafs[0]["study"] if gmafs else None,
        "clinical_significance": rec.get("clinical_significance"),
    }

for rsid in ["rs80357906", "rs9999999999", "rs1800497"]:
    info = check_rsid(rsid)
    if info["found"]:
        print(f"{rsid}: {info['snp_class']} | pos={info['chrpos']} | MAF={info['maf']} ({info['maf_study']})")
    else:
        print(f"{rsid}: NOT FOUND in dbSNP")
# rs80357906: delins | pos=17:43057062 | MAF=G=0.0008929/4 (Estonian)
# rs9999999999: NOT FOUND in dbSNP
# rs1800497: snv | pos=11:113400106 | MAF=G=0.42... (study varies)

Recipe: Resolve Gene Variants to rsIDs and Coordinates

When to use: Convert a gene name to a list of rsIDs for use in downstream tools (PLINK, ANNOVAR, etc.).

import requests, time, pandas as pd

EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"

def gene_to_rsids(gene: str, max_variants: int = 200) -> pd.DataFrame:
    """Search dbSNP for a gene and return rsIDs with GRCh38 coordinates."""
    # Step 1: search
    r = requests.get(f"{BASE}/esearch.fcgi",
                     params={"db": "snp", "term": f"{gene}[gene] AND human[orgn]",
                             "retmax": max_variants, "retmode": "json", "email": EMAIL},
                     timeout=15)
    r.raise_for_status()
    ids = r.json()["esearchresult"]["idlist"]
    if not ids:
        return pd.DataFrame()
    time.sleep(0.4)

    # Step 2: fetch summaries
    r2 = requests.post(f"{BASE}/esummary.fcgi",
                       data={"db": "snp", "id": ",".join(ids),
                             "retmode": "json", "email": EMAIL},
                       timeout=30)
    r2.raise_for_status()
    result = r2.json()["result"]
    rows = []
    for uid in result.get("uids", []):
        rec = result[uid]
        gmafs = rec.get("global_mafs", [])
        rows.append({
            "rsid": f"rs{uid}",
            "chrpos_grch38": rec.get("chrpos"),
            "snp_class": rec.get("snp_class"),
            "maf": gmafs[0]["freq"] if gmafs else None,
        })
    return pd.DataFrame(rows)

df = gene_to_rsids("APOE", max_variants=50)
print(f"APOE variants retrieved: {len(df)}")
print(df.head(8).to_string(index=False))
df.to_csv("APOE_rsids.csv", index=False)

Troubleshooting

ProblemCauseSolution
HTTP 429 or connection refusedRate limit exceeded (3 req/sec)Add time.sleep(0.35) between requests; register for API key to get 10 req/sec
ESummary returns {"error": "Invalid uid"}rsID does not exist in dbSNPCheck rsID spelling; verify with NCBI browser; variant may be a novel call not yet in dbSNP
esearch returns 0 results for a geneGene symbol mismatch or missing human[orgn] filterTry adding AND human[orgn]; check NCBI gene symbol at https://www.ncbi.nlm.nih.gov/gene
KeyError: 'maf' / 'mafallele' in ESummary parsingFields removed in the 2024 dbSNP ESummary schemaUse rec["global_mafs"] — list of {"study": ..., "freq": "<allele>=<value>/<count>"}; pick a study (GnomAD_genomes, TOPMED, ALFA) and parse the freq string
global_mafs emptyVariant has no aggregated population frequency in dbSNPUse gnomAD via gnomad-database directly for population frequencies
root.iter("DocumentSummary") returns 0 with rettype="xml"EFetch XML root is namespaced ({https://www.ncbi.nlm.nih.gov/SNP/docsum}ExchangeSet)Use rettype="docsum" (no namespace) or pass the full namespaced tag to iter()
Variation Services API returns 404rsID not found or wrong URL formatConfirm integer rs number (no rs prefix) in /refsnp/{rs_num} endpoint
EPost XML parsing failsNon-XML response (rate limit HTML error page)Check response status code first; add retry logic with time.sleep(1)
Batch efetch returns fewer records than postedSome rsIDs were merged or retiredCross-check against NCBI merge history; retired rsIDs redirect to current active rs

Related Skills

  • clinvar-database — ClinVar pathogenicity classifications for variants identified by rsID (complement to dbSNP)
  • gnomad-database — Population allele frequencies by ancestry group (more detailed than dbSNP MAF)
  • gwas-database — GWAS Catalog for SNP-trait associations from published GWAS studies
  • ensembl-database — Ensembl REST API for variant consequences and gene annotations
  • snpeff-variant-annotation — Annotate VCF files with SnpEff and SnpSift, which adds dbSNP rsIDs and functional predictions

References

DepMap CRISPR Gene Effect Analysis Guide

Overview

This guide covers the correct interpretation and analysis of DepMap CRISPR gene effect (Chronos) data. The most critical and common error in DepMap analyses is failing to negate the CRISPR scores when computing correlations with "essentiality." A secondary but equally damaging mistake is using bulk correlation shortcuts that mishandle per-gene NaN patterns. This guide provides the mandatory sign convention, the correct per-gene NaN-safe Spearman correlation implementation, and data loading/alignment procedures.

Key Concepts

DepMap CRISPR Score Convention

The CRISPR gene effect score (produced by the Chronos algorithm) quantifies how gene knockout affects cell viability:

  • Negative score: gene knockout reduces cell viability -- the gene is essential for that cell line
  • Zero score: no measurable effect on viability
  • Positive score: gene knockout increases viability (rare, may indicate tumor-suppressive behavior)

The DepMap portal distributes these scores in the file CRISPRGeneEffect.csv. Each row is a cell line (DepMap ID, e.g., ACH-000001) and each column is a gene in the format GENE_NAME (ENTREZ_ID), e.g., A1BG (1).

Essentiality Sign Interpretation

Because negative raw scores indicate essentiality, any analysis that asks about "essentiality" or "dependency" requires negating the raw CRISPR scores:

  • "Correlation with essentiality" = correlation with -CRISPRGeneEffect (negated)
  • "Higher essentiality" = more negative raw score = more positive negated score
  • "Most essential gene" = gene with the most negative raw score

If you correlate expression with raw CRISPR scores and find 3 genes with correlation <= -0.6 and 0 genes with correlation >= 0.6, then the correct answer for "genes with strong positive correlation with essentiality" is 3, not 0. The negative correlations with raw scores ARE the positive correlations with essentiality.

Data Structure: CRISPRGeneEffect Format

The standard DepMap data files use a consistent structure:

  • Index: DepMap cell line identifiers (ACH-XXXXXX)
  • Columns: Gene identifiers in GENE_NAME (ENTREZ_ID) format
  • Values: Floating-point scores (may contain NaN for genes not screened in a given cell line)
  • Companion files: Expression data (OmicsExpressionProteinCodingGenesTPMLogp1BatchCorrected.csv) uses the same index/column format, enabling direct alignment

Different genes have different patterns of missing data across cell lines. This is because not all genes are screened in all cell lines, and quality control may remove specific gene-cell line combinations.

Decision Framework

Question: How should I compute correlations with DepMap CRISPR data?
├── Does the question mention "essentiality" or "dependency"?
│   ├── Yes → Negate CRISPR scores before correlating (see Best Practices #1)
│   └── No (raw gene effect) → Use raw scores directly
├── How should I compute correlations?
│   ├── Per-gene correlation → scipy.stats.spearmanr in a loop (see Best Practices #2)
│   └── Matrix-wide correlation → AVOID; use per-gene loop instead
└── How should I handle missing data?
    ├── Pairwise NaN removal → CORRECT (see Best Practices #3)
    └── Global row/column dropping → INCORRECT; loses too much data
ScenarioRecommended ApproachRationale
Correlating expression with "essentiality"Negate CRISPR scores, then per-gene SpearmanSign convention requires negation; per-gene handles NaN correctly
Correlating expression with raw gene effectPer-gene Spearman on raw scoresNo negation needed, but NaN-safe per-gene loop still required
Ranking genes by essentiality across cell linesRank by most negative mean raw scoreMore negative = more essential across the panel
Identifying selectively essential genesCompare score distributions across subgroupsUse per-subgroup mean/median of raw scores, then compare
Filtering genes before correlationRequire minimum 10 valid cell line pairsGenes with too few observations yield unreliable correlations

Best Practices

  1. Always negate CRISPR scores when the analysis asks about "essentiality": The raw DepMap convention is that negative = essential. When a question or hypothesis refers to "essentiality," "dependency," or "gene importance," negate the scores so that higher values mean more essential. Explicitly state the sign convention in your results.

  2. Use scipy.stats.spearmanr per gene in a loop: Bulk matrix shortcuts (DataFrame.corrwith, DataFrame.rank().corrwith()) handle NaN inconsistently across columns. The only reliable method is to compute Spearman correlation gene by gene using scipy.stats.spearmanr with pairwise-complete observations.

  3. Apply pairwise NaN removal, not global dropping: Different genes have different missing-data patterns. Dropping rows globally (any NaN in any column) discards far too much data. Instead, for each gene, mask out only the cell lines where either the expression or CRISPR value is NaN.

  4. Set a minimum valid-pair threshold: Genes with very few non-NaN cell line pairs produce unreliable correlation estimates. Require at least 10 (preferably 20+) valid pairs before computing a correlation. Skip genes below this threshold.

  5. Report NaN summary before analysis: Before computing correlations, print the total NaN count per dataset, the number of common cell lines, and the number of common genes. This provides an audit trail and helps catch data loading errors early.

  6. Verify dataset alignment before computation: Always intersect cell line IDs and gene columns between datasets before analysis. Misaligned indices produce silent errors -- correlations computed on mismatched rows are meaningless.

  7. State the sign convention explicitly in results: When reporting correlation results, always include a statement like "CRISPR scores were negated so that positive values represent higher essentiality." This prevents downstream misinterpretation.

Common Pitfalls

  1. Forgetting to negate CRISPR scores for essentiality analysis: The raw DepMap score convention (negative = essential) is counterintuitive. Omitting the negation reverses every correlation sign, leading to completely inverted conclusions.

    • How to avoid: Always check whether the analysis question uses the word "essentiality" or "dependency." If so, negate the CRISPR scores. Add a comment in the code: # Negate: in DepMap, negative = essential.
  2. Using bulk DataFrame correlation methods: Methods like DataFrame.corrwith(method='spearman') or DataFrame.rank().corrwith() silently mishandle NaN values, potentially shifting correlations enough to push genes above or below significance thresholds.

    • How to avoid: Always use a per-gene loop with scipy.stats.spearmanr. See the reference implementation in the Workflow section below.
  3. Dropping rows globally instead of pairwise: Calling dropna() on the entire DataFrame before correlation removes all cell lines that have any NaN in any gene, drastically reducing sample size.

    • How to avoid: Apply NaN masking inside the per-gene loop: mask = ~(np.isnan(x) | np.isnan(y)). This preserves the maximum number of observations per gene.
  4. Not checking for sufficient valid pairs: Computing Spearman correlation on fewer than 10 observations produces unstable, unreliable estimates that may appear significant by chance.

    • How to avoid: Add a guard clause: if mask.sum() < 10: continue. Adjust the threshold upward (e.g., 20) for more conservative analysis.
  5. Misinterpreting correlation signs without stated convention: Reporting "positive correlation" without stating whether CRISPR scores were negated leaves results ambiguous. Reviewers cannot tell if "positive" means "higher expression associates with more essential" or "less essential."

    • How to avoid: Always include a sentence in the results section stating the sign convention used. For example: "CRISPR scores were negated so that positive correlation indicates higher expression is associated with greater essentiality."
  6. Failing to align datasets before computation: Expression and CRISPR datasets may have different cell lines or different gene sets. Computing correlations without explicit alignment can silently match wrong rows or produce index errors.

    • How to avoid: Always compute common_lines = expr.index.intersection(crispr.index) and common_genes = expr.columns.intersection(crispr.columns), then subset both DataFrames before any computation.
  7. Ignoring the gene column format: DepMap gene columns use the format GENE_NAME (ENTREZ_ID). Attempting to match against plain gene symbols (e.g., TP53 instead of TP53 (7157)) will produce empty intersections.

    • How to avoid: Inspect column formats with df.columns[:5] before attempting any join or intersection. Parse gene names if needed: df.columns.str.extract(r'^(.+?)\s*\(')[0].

Workflow

  1. Step 1: Load DepMap data

    import pandas as pd
    
    # Load CRISPR gene effect data
    crispr = pd.read_csv('CRISPRGeneEffect.csv', index_col=0)
    
    # Load expression data
    expr = pd.read_csv(
        'OmicsExpressionProteinCodingGenesTPMLogp1BatchCorrected.csv',
        index_col=0
    )
    
    # Column format: "GENE_NAME (ENTREZ_ID)" e.g., "A1BG (1)"
    # Index: DepMap cell line IDs e.g., "ACH-000001"
    
  2. Step 2: Align datasets

    # Find common cell lines and genes
    common_lines = crispr.index.intersection(expr.index)
    common_genes = crispr.columns.intersection(expr.columns)
    
    print(f"Common cell lines: {len(common_lines)}")
    print(f"Common genes: {len(common_genes)}")
    
    # Subset to common
    crispr_aligned = crispr.loc[common_lines, common_genes]
    expr_aligned = expr.loc[common_lines, common_genes]
    
  3. Step 3: Report NaN summary

    expr_nan = expr_aligned.isna().sum().sum()
    crispr_nan = crispr_aligned.isna().sum().sum()
    print(f"Expression NaN count: {expr_nan}")
    print(f"CRISPR NaN count: {crispr_nan}")
    
  4. Step 4: Negate CRISPR scores if computing essentiality correlations

    # Negate: in DepMap, negative raw score = essential
    # After negation, positive = essential
    essentiality = -crispr_aligned
    
  5. Step 5: Compute per-gene NaN-safe Spearman correlation

    from scipy.stats import spearmanr
    import numpy as np
    
    def compute_per_gene_spearman(expression_df, crispr_df, negate_crispr=True):
        """Compute Spearman correlation per gene with proper NaN handling.
    
        Args:
            expression_df: DataFrame (cell_lines x genes)
            crispr_df: DataFrame (cell_lines x genes)
            negate_crispr: If True, negate CRISPR scores to represent essentiality
    
        Returns:
            Series of Spearman correlations indexed by gene name
        """
        # Align cell lines and genes
        common_lines = expression_df.index.intersection(crispr_df.index)
        common_genes = expression_df.columns.intersection(crispr_df.columns)
    
        expr = expression_df.loc[common_lines, common_genes]
        crispr = crispr_df.loc[common_lines, common_genes]
    
        if negate_crispr:
            crispr = -crispr
    
        # Print NaN summary BEFORE analysis
        expr_nan = expr.isna().sum().sum()
        crispr_nan = crispr.isna().sum().sum()
        print(f"Expression NaN count: {expr_nan}")
        print(f"CRISPR NaN count: {crispr_nan}")
        print(f"Common cell lines: {len(common_lines)}")
        print(f"Common genes: {len(common_genes)}")
    
        # Per-gene Spearman correlation with pairwise NaN removal
        correlations = {}
        for gene in common_genes:
            x = expr[gene].values
            y = crispr[gene].values
    
            # Remove pairs where either value is NaN
            mask = ~(np.isnan(x) | np.isnan(y))
            if mask.sum() < 10:  # Skip genes with too few valid pairs
                continue
    
            rho, pval = spearmanr(x[mask], y[mask])
            correlations[gene] = rho
    
        return pd.Series(correlations).sort_values(ascending=False)
    
  6. Step 6: Apply threshold and report results

    correlations = compute_per_gene_spearman(expr_aligned, crispr_aligned,
                                             negate_crispr=True)
    
    threshold = 0.6
    strong_positive = correlations[correlations >= threshold]
    strong_negative = correlations[correlations <= -threshold]
    
    print(f"Genes with correlation >= {threshold}: {len(strong_positive)}")
    print(f"Genes with correlation <= -{threshold}: {len(strong_negative)}")
    print(f"\nNote: CRISPR scores were negated so that positive correlation")
    print(f"indicates higher expression associated with greater essentiality.")
    
  7. Step 7: Validate -- check for anti-patterns

    Verify that none of these bulk shortcuts were used anywhere in the analysis:

    # WRONG: Bulk rank-then-correlate shortcut
    ranked_expr = expression_df.rank()
    ranked_crispr = crispr_df.rank()
    correlations = ranked_expr.corrwith(ranked_crispr)  # NaN handling is unreliable
    
    # WRONG: Bulk corrwith with method='spearman'
    correlations = expression_df.corrwith(crispr_df, method='spearman')  # Same issue
    

    If any of these patterns appear in the code, replace them with the per-gene loop from Step 5.

Further Reading

Related Skills

  • nan-safe-correlation -- General techniques for NaN-safe correlation computation across omics datasets; this guide applies those principles specifically to DepMap CRISPR data
  • degenerate-input-filtering -- Upstream data quality filtering to remove low-variance or degenerate features before correlation analysis; recommended as a preprocessing step before DepMap essentiality correlation

ENA Database — European Nucleotide Archive Programmatic Access

Overview

The European Nucleotide Archive (ENA) is EMBL-EBI's comprehensive nucleotide sequence database, encompassing raw sequencing reads, genome assemblies, annotated sequences, and associated metadata. It mirrors and extends INSDC data (GenBank, DDBJ). All access is via REST APIs with no authentication required.

When to Use

  • Searching for sequencing studies, samples, or experiments by organism, project, or keyword
  • Downloading raw FASTQ/BAM files for reanalysis of public sequencing datasets
  • Retrieving genome assemblies with quality statistics (N50, contig count, genome size)
  • Fetching nucleotide sequences in FASTA or EMBL flat-file format by accession
  • Exploring taxonomic lineage and finding organisms by partial name
  • Cross-referencing ENA records with external databases (ArrayExpress, UniProt, PDB)
  • Building bulk download lists for large-scale sequencing projects
  • For multi-database Python queries (ENA + UniProt + KEGG), prefer bioservices instead
  • For NCBI-specific queries (PubMed literature, GenBank records), use pubmed-database or Biopython Entrez

Prerequisites

pip install requests

API constraints:

  • Rate limit: 50 requests per second across all ENA APIs
  • No authentication required
  • Large result sets: use pagination (limit + offset) or streaming (limit=0 for TSV download)
  • Portal API base: https://www.ebi.ac.uk/ena/portal/api
  • Browser API base: https://www.ebi.ac.uk/ena/browser/api
  • Taxonomy API base: https://www.ebi.ac.uk/ena/taxonomy/rest
  • Cross-ref API base: https://www.ebi.ac.uk/ena/xref/rest

Quick Start

import requests
import time

BASE_PORTAL = "https://www.ebi.ac.uk/ena/portal/api"
BASE_BROWSER = "https://www.ebi.ac.uk/ena/browser/api"
BASE_TAXONOMY = "https://www.ebi.ac.uk/ena/taxonomy/rest"
BASE_XREF = "https://www.ebi.ac.uk/ena/xref/rest"

def ena_query(endpoint, params=None, base=BASE_PORTAL):
    """Reusable ENA API caller with rate-limit compliance."""
    resp = requests.get(f"{base}/{endpoint}", params=params)
    resp.raise_for_status()
    time.sleep(0.02)  # 50 req/sec limit
    return resp

# Search for human RNA-seq studies
resp = ena_query("search", params={
    "result": "study",
    "query": 'tax_tree(9606)',   # `library_strategy` is a `read_run`/`read_experiment` field, not a `study` field
    "fields": "study_accession,study_title",
    "format": "json",
    "limit": 3,
})
studies = resp.json()
for s in studies:
    print(f"{s['study_accession']}: {s['study_title'][:60]}")
# PRJEB12345: Transcriptome analysis of human liver tissue...

Core API

Module 1: Portal API Search

The Portal API provides advanced metadata search across all ENA data types with boolean query syntax, field selection, and pagination.

# Search read runs for a specific study
resp = ena_query("search", params={
    "result": "read_run",
    "query": 'study_accession="PRJEB1787"',
    "fields": "run_accession,sample_accession,instrument_model,read_count,base_count",
    "format": "json",
    "limit": 5,
})
runs = resp.json()
for r in runs:
    print(f"{r['run_accession']} — {r.get('instrument_model', 'N/A')}, "
          f"{int(r.get('read_count', 0)):,} reads")
# ERR123456 — Illumina HiSeq 2000, 45,231,890 reads

# Count total results without fetching data
count_resp = ena_query("count", params={
    "result": "read_run",
    "query": 'study_accession="PRJEB1787"',
})
print(f"Total runs: {count_resp.text.strip()}")
# Total runs: 142

Module 2: Browser API Retrieval

Fetch individual records by accession in multiple formats: XML, FASTA, EMBL flat-file, or plain text.

# Retrieve XML metadata for a study
resp = ena_query("xml/PRJEB1787", base=BASE_BROWSER)
print(resp.text[:300])
# <?xml version="1.0" encoding="UTF-8"?><PROJECT_SET>...

# Retrieve FASTA sequence for a coding sequence
resp = ena_query("fasta/M10051.1", base=BASE_BROWSER)
print(resp.text[:200])
# >ENA|M10051|M10051.1 Human insulin mRNA, complete cds.
# AGCCCTCCAGGACAGGCTGCAT...

# Retrieve EMBL flat-file format
resp = ena_query("embl/M10051.1", base=BASE_BROWSER)
print(resp.text[:300])
# ID   M10051; SV 1; linear; mRNA; STD; HUM; 786 BP.
# ...

Module 3: File Reports and Downloads

Get download URLs for FASTQ, submitted, and analysis files. File reports return FTP and Aspera paths.

# Get FASTQ file URLs for specific runs
resp = ena_query("filereport", params={
    "accession": "ERR000589",
    "result": "read_run",
    "fields": "run_accession,fastq_ftp,fastq_bytes,fastq_md5",
    "format": "json",
})
files = resp.json()
for f in files:
    ftp_urls = f.get("fastq_ftp", "").split(";")
    sizes = f.get("fastq_bytes", "").split(";")
    for url, size in zip(ftp_urls, sizes):
        if url:
            print(f"ftp://{url}  ({int(size)/1e6:.1f} MB)")
# ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR000/ERR000589/ERR000589_1.fastq.gz  (234.5 MB)
# ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR000/ERR000589/ERR000589_2.fastq.gz  (241.2 MB)

Module 4: Taxonomy Queries

Look up organisms by taxonomy ID, scientific name, or partial name match.

# Lookup by taxonomy ID
resp = ena_query("tax-id/9606", base=BASE_TAXONOMY)
tax = resp.json()
print(f"{tax['scientificName']} (taxId: {tax['taxId']}, rank: {tax['rank']})")
# Homo sapiens (taxId: 9606, rank: species)
print(f"Lineage: {tax['lineage'][:80]}...")

# Search by scientific name — endpoint returns a list (one entry per matching taxon)
resp = ena_query("scientific-name/Arabidopsis thaliana", base=BASE_TAXONOMY)
matches = resp.json()
result = matches[0] if isinstance(matches, list) else matches
print(f"Tax ID: {result['taxId']}, Common: {result.get('commonName', 'N/A')}")
# Tax ID: 3702, Common: thale cress

# Suggest organisms by partial name
resp = ena_query("suggest-for-search/salmo", base=BASE_TAXONOMY)
suggestions = resp.json()
for s in suggestions[:3]:
    print(f"  {s['scientificName']} (taxId: {s['taxId']})")
# Salmo salar (taxId: 8030)
# Salmo trutta (taxId: 8032)
# Salmonella enterica (taxId: 28901)

Module 5: Cross-Reference Service

Find links between ENA records and external databases (ArrayExpress, UniProt, PDB, etc.).

# Find cross-references for an ENA accession
resp = ena_query("json/search", base=BASE_XREF, params={
    "accession": "M10051",
})
xrefs = resp.json()
for x in xrefs[:5]:
    print(f"  {x['Source']} → {x['Source Primary Accession']} "
          f"({x.get('Source Description', '')[:50]})")
# UniProt → P01308 (Insulin precursor)
# PDB → 1A7F (Crystal structure of human insulin)

# Search cross-references by external database
resp = ena_query("json/search", base=BASE_XREF, params={
    "source": "UniProt",
    "accession": "P01308",
})
xrefs = resp.json()
for x in xrefs[:3]:
    print(f"  ENA: {x['Target Primary Accession']} — {x.get('Target Description', '')[:60]}")

Module 6: CRAM Reference Registry

Retrieve reference sequences used in CRAM files by MD5 or SHA1 checksum. Essential for CRAM decompression.

# Look up reference by MD5 checksum
md5 = "aef131c3b4b05d8e2b3f907faba5af9b"  # example
try:
    resp = ena_query(
        f"cram/md5/{md5}",
        base="https://www.ebi.ac.uk/ena/cram"
    )
    print(f"Reference found: {len(resp.content)} bytes")
except requests.HTTPError as e:
    if e.response.status_code == 404:
        print("Reference not found — check MD5 checksum")
    else:
        raise

Key Concepts

ENA Data Hierarchy

LevelAccession PrefixDescriptionContains
StudyPRJEB/ERPResearch projectSamples, Experiments
SampleERS/SAMEABiological sampleMetadata, taxonomy
ExperimentERXLibrary/sequencing setupRuns
RunERRSequencing runRaw read files (FASTQ)
AnalysisERZDerived analysisAssemblies, alignments
AssemblyGCAGenome assemblyContigs, scaffolds
SequenceAccession.versionAnnotated sequenceFeatures, coding seqs

Query Syntax Operators

OperatorExampleDescription
Equalityinstrument_model="Illumina NovaSeq 6000"Exact match
Wildcardstudy_title="*melanoma*"Partial match
Rangebase_count>=1000000Numeric comparison
Taxonomy treetax_tree(9606)Taxon and all descendants
Exact taxontax_eq(9606)Exact taxon only
Date rangefirst_public>=2023-01-01Date filtering
BooleanAND, OR, NOTCombine conditions
Grouping(A OR B) AND CParenthetical grouping

Result Types

Result TypeDescriptionKey Fields
studyResearch projectsstudy_accession, study_title, center_name
sampleBiological samplessample_accession, tax_id, scientific_name
read_runSequencing runsrun_accession, read_count, base_count, fastq_ftp
read_experimentExperimentsexperiment_accession, library_strategy, instrument_model
analysisDerived analysesanalysis_accession, analysis_type
assemblyGenome assembliesassembly_accession, assembly_level, genome_representation
sequenceAnnotated sequencesaccession, sequence_length, mol_type
wgs_setWGS scaffold setsset_accession, set_size
tsa_setTranscriptome assembliesset_accession, set_size
codingCoding sequencesaccession, gene, product
noncodingNon-coding featuresaccession, description
taxonTaxonomy entriestax_id, scientific_name, lineage

Discoverable Fields

Use the returnFields endpoint to discover available fields for any result type:

resp = ena_query("returnFields", params={"result": "read_run"})
fields = resp.text.strip().split("\n")
print(f"Available fields for read_run: {len(fields)}")
print(fields[:10])
# ['accession', 'altitude', 'assembly_quality', 'assembly_software', ...]

Common Workflows

Workflow 1: Study Exploration Pipeline

Search for a study, list its samples, then retrieve run metadata.

import json

# Step 1: Find studies by organism — `study_title="*…*"` wildcards no longer match;
# use `tax_tree()` against the species tax ID (SARS-CoV-2 = 2697049).
resp = ena_query("search", params={
    "result": "study",
    "query": 'tax_tree(2697049) AND first_public>=2023-01-01',
    "fields": "study_accession,study_title,center_name",
    "format": "json",
    "limit": 3,
})
studies = resp.json()
study_acc = studies[0]["study_accession"]
print(f"Selected: {study_acc} — {studies[0]['study_title'][:60]}")

# Step 2: List samples in the study
resp = ena_query("search", params={
    "result": "sample",
    "query": f'study_accession="{study_acc}"',
    "fields": "sample_accession,scientific_name,collection_date",
    "format": "json",
    "limit": 5,
})
samples = resp.json()
print(f"Found {len(samples)} samples (showing first 5)")
for s in samples:
    print(f"  {s['sample_accession']} — {s.get('scientific_name', 'N/A')}")

# Step 3: Get run metadata for each sample
for s in samples[:2]:
    resp = ena_query("search", params={
        "result": "read_run",
        "query": f'sample_accession="{s["sample_accession"]}"',
        "fields": "run_accession,instrument_model,read_count,library_strategy",
        "format": "json",
    })
    runs = resp.json()
    for r in runs:
        print(f"  {r['run_accession']}: {r.get('library_strategy','N/A')}, "
              f"{int(r.get('read_count',0)):,} reads")
    time.sleep(0.02)

Workflow 2: Bulk FASTQ Download URL Collection

Search for runs matching criteria and collect download URLs.

# Step 1: Search for Illumina RNA-Seq runs from a specific organism
resp = ena_query("search", params={
    "result": "read_run",
    "query": ('tax_tree(10090) AND library_strategy="RNA-Seq" '
              'AND instrument_platform="ILLUMINA" AND read_count>=10000000'),
    "fields": "run_accession,study_accession,read_count",
    "format": "json",
    "limit": 10,
})
runs = resp.json()
print(f"Found {len(runs)} runs meeting criteria")

# Step 2: Get file reports with download URLs
download_list = []
for run in runs[:5]:
    acc = run["run_accession"]
    resp = ena_query("filereport", params={
        "accession": acc,
        "result": "read_run",
        "fields": "run_accession,fastq_ftp,fastq_bytes,fastq_md5",
        "format": "json",
    })
    for f in resp.json():
        urls = f.get("fastq_ftp", "").split(";")
        md5s = f.get("fastq_md5", "").split(";")
        for url, md5 in zip(urls, md5s):
            if url:
                download_list.append({"url": f"ftp://{url}", "md5": md5, "run": acc})
    time.sleep(0.02)

print(f"\nDownload list: {len(download_list)} files")
for d in download_list[:4]:
    print(f"  {d['run']}: {d['url'].split('/')[-1]}")

Workflow 3: Taxonomic Assembly Exploration

Find organisms, search their assemblies, and check quality statistics.

# Step 1: Resolve organism by exact scientific name (returns a list, take the first match).
# `suggest-for-search` only returns prefix matches and may not include the species you want.
resp = ena_query("scientific-name/Drosophila melanogaster", base=BASE_TAXONOMY)
matches = resp.json()
target_tax = matches[0]["taxId"]
print(f"Selected: {matches[0]['scientificName']} (taxId={target_tax})")

# Step 2: Search assemblies for this organism. `n50` is no longer a valid `assembly`
# field — use `base_count` and `program` for what's available.
resp = ena_query("search", params={
    "result": "assembly",
    "query": f'tax_eq({target_tax}) AND assembly_level="chromosome"',
    "fields": ("assembly_accession,assembly_name,assembly_level,"
               "genome_representation,base_count"),
    "format": "json",
    "limit": 5,
})
assemblies = resp.json()
for a in assemblies:
    size = int(a.get("base_count", 0))
    print(f"  {a['assembly_accession']}: {a.get('assembly_name','N/A')}, "
          f"Size={size/1e6:.1f} Mb")
# GCA_000001215.4: Release 6 plus ISO1 MT, Size=143.7 Mb

Key Parameters

EndpointParameterDefaultDescription
searchresult(required)Result type: study, sample, read_run, assembly, etc.
searchquery(required)Boolean query string with field operators
searchfieldsallComma-separated field names to return
searchformattsvOutput format: json, tsv, xml
searchlimit100000Max results (0 = unlimited streaming)
searchoffset0Skip first N results (pagination)
searchsortFields—Field(s) to sort by
filereportaccession(required)Study, sample, or run accession
filereportresult(required)read_run, analysis, etc.
xml/accession path(required)Any ENA accession (Browser API)
fasta/accession path(required)Sequence accession (Browser API)

Best Practices

  1. Use tax_tree() over tax_eq() for organism queries — it includes subspecies and strains automatically
  2. Request only needed fields — reduces response size and server load significantly
  3. Prefer JSON format for programmatic access; TSV for large bulk exports (lower overhead)
  4. Use limit=0 for streaming large result sets directly to file, avoiding memory issues
  5. Check fastq_ftp and submitted_ftp — some runs have submitted files but no processed FASTQ
  6. Verify downloads with MD5 — file reports include fastq_md5 for integrity checking
  7. Anti-pattern: Do not fetch all fields then filter client-side — use query syntax server-side

Common Recipes

Recipe: Assembly Quality Filtering

# Find chromosome-level, full-representation assemblies.
# `n50` is no longer a valid `assembly` field — filter by base_count instead.
resp = ena_query("search", params={
    "result": "assembly",
    "query": ('tax_tree(7742) AND assembly_level="chromosome" '
              'AND genome_representation="full"'),
    "fields": "assembly_accession,scientific_name,base_count,assembly_level",
    "format": "json",
    "limit": 10,
})
for a in resp.json():
    size = int(a.get("base_count", 0))
    if size > 100_000_000:   # >100 Mb (proxy for "well-assembled")
        print(f"{a['assembly_accession']}: {a.get('scientific_name','?')}, Size={size/1e6:.0f} Mb")

Recipe: Cross-Database Linking

# Find UniProt/PDB cross-references for an ENA sequence
resp = ena_query("json/search", base=BASE_XREF, params={
    "accession": "M10051",
})
xrefs = resp.json()
by_source = {}
for x in xrefs:
    src = x.get("Source", "unknown")
    by_source.setdefault(src, []).append(x["Source Primary Accession"])
for src, accs in by_source.items():
    print(f"  {src}: {', '.join(accs[:5])}")
# UniProt: P01308
# PDB: 1A7F, 1AI0, 1BEN

Recipe: Retry Session with Exponential Backoff

from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

def ena_session():
    """Create a requests session with retry logic for ENA APIs."""
    session = requests.Session()
    retry = Retry(
        total=3, backoff_factor=1.0,
        status_forcelist=[429, 500, 502, 503, 504],
    )
    session.mount("https://", HTTPAdapter(max_retries=retry))
    return session

session = ena_session()
resp = session.get(f"{BASE_PORTAL}/search", params={
    "result": "study",
    "query": 'tax_tree(9606)',
    "fields": "study_accession",
    "format": "json",
    "limit": 5,
})
print(f"Status: {resp.status_code}, results: {len(resp.json())}")

Troubleshooting

ProblemCauseSolution
400 Bad Request on searchInvalid query syntax or unknown field nameUse returnFields endpoint to verify field names; check query operator syntax
400 with result paramInvalid result typeCheck result types table above; common: read_run not run
Empty results for known dataWrong taxonomy operatorUse tax_tree() (includes descendants) not tax_eq() (exact only)
fastq_ftp field is emptySubmitted files not processed to FASTQCheck submitted_ftp field instead; some datasets only have BAM/CRAM
429 Too Many RequestsExceeded 50 req/sec rate limitAdd time.sleep(0.02) between requests; use retry session with backoff
Timeout on large queriesResult set too large for single requestUse limit + offset pagination, or limit=0 with streaming to file
XML parsing errorsMalformed XML for some recordsUse JSON format instead (format=json) when available
Wrong sequence versionAccession without version suffixAlways use versioned accessions (e.g., M10051.1 not M10051) for Browser API
CRAM reference not foundMD5 checksum mismatch or non-INSDC referenceVerify MD5; check if reference is from a custom genome (not in registry)

Bundled Resources

This skill is self-contained. The original entry had a separate references/api_reference.md (490 lines) covering all 6 API endpoints in detail. That content has been fully consolidated inline:

  • Portal API (search, count, returnFields, filereport) — Core API Modules 1 and 3, Key Parameters table
  • Browser API (XML, FASTA, EMBL retrieval) — Core API Module 2
  • Taxonomy REST API (tax-id, scientific-name, suggest-for-search) — Core API Module 4
  • Cross-Reference Service (json/search) — Core API Module 5
  • CRAM Reference Registry (md5/sha1 lookup) — Core API Module 6
  • Rate limiting and error handling — Prerequisites, Troubleshooting, Recipe 3 (retry session)
  • Query syntax and result types — Key Concepts section (3 tables)
  • Pagination and bulk download — Key Parameters, Best Practices, Workflow 2
  • Omitted: detailed EMBL format field-by-field breakdown (rarely needed programmatically); Aspera download command examples (tool-specific, not requests-based)

Related Skills

  • bioservices-multi-database — unified Python interface covering ENA via bioservices; prefer for multi-database workflows
  • pubmed-database — PubMed literature search via NCBI E-utilities
  • pysam-genomic-files — downstream processing of FASTQ/BAM/CRAM files retrieved from ENA
  • biopython-molecular-biology — NCBI Entrez access and sequence parsing (GenBank/FASTA)
  • ncbi-blast (planned) — BLAST sequence similarity search

References

ENCODE Database

Overview

The ENCODE (Encyclopedia of DNA Elements) Project has generated thousands of functional genomics experiments — TF ChIP-seq, ATAC-seq, DNase-seq, histone ChIP-seq, and RNA-seq — across 1000+ human and mouse cell types and tissues. The ENCODE Portal REST API provides structured JSON access to experiment metadata, file download URLs, and SCREEN cCRE (candidate cis-Regulatory Elements) annotations. All data is freely accessible without authentication for most endpoints.

When to Use

  • Downloading TF ChIP-seq peak files (BED) for a specific transcription factor and cell type to annotate regulatory regions
  • Finding ATAC-seq or DNase-seq peaks in a cell type to identify open chromatin regions near a gene of interest
  • Retrieving cCREs (candidate cis-Regulatory Elements) overlapping a genomic region from ENCODE SCREEN
  • Building reference regulatory tracks for variant annotation pipelines (e.g., annotating VCF variants against ENCODE peak sets)
  • Exploring which experiments are available for a biosample (cell line, tissue, developmental stage) before planning a wet-lab experiment
  • Querying all ChIP-seq experiments for a transcription factor across multiple cell types for comparative regulatory analysis
  • Use regulomedb-database instead when you want pre-computed regulatory scores for specific SNPs — RegulomeDB integrates ENCODE data with eQTL and motif evidence into a single score
  • Use deeptools-ngs-analysis instead when you have your own BAM files and need to generate bigWig coverage tracks; ENCODE database is for retrieving existing deposited data

Prerequisites

  • Python packages: requests, pandas, matplotlib
  • Data requirements: experiment accessions (e.g., ENCSR000AKC), biosample names (e.g., K562), TF target names (e.g., CTCF, TP53), or genomic regions (chr7:117548628-117748628)
  • Environment: internet connection; no authentication required for public data; add Authorization: Bearer {api_key} header for submitter access
  • Rate limits: no published hard limit; add time.sleep(0.5) for large batch queries to avoid connection resets
pip install requests pandas matplotlib

Quick Start

import requests

BASE = "https://www.encodeproject.org"

def search_experiments(assay="TF ChIP-seq", target="CTCF", biosample="K562", limit=5):
    """Find ENCODE experiments matching assay type, target, and biosample."""
    params = {
        "type": "Experiment",
        "assay_title": assay,
        "target.label": target,
        "biosample_ontology.term_name": biosample,   # `biosample_summary` is a verbose freetext string; filter by ontology term name
        "status": "released",
        "format": "json",
        "limit": limit,
    }
    r = requests.get(f"{BASE}/search/", params=params, timeout=30)
    r.raise_for_status()
    data = r.json()
    experiments = data.get("@graph", [])
    print(f"Found {data.get('total', 0)} experiments for {target} ChIP-seq in {biosample}")
    for exp in experiments:
        print(f"  {exp['accession']}  {exp.get('biosample_summary', '')}  {exp.get('lab', {}).get('title', '')}")
    return experiments

exps = search_experiments(assay="TF ChIP-seq", target="CTCF", biosample="K562")

Core API

Query 1: Experiment Search — Find Experiments by Assay, Biosample, Target

Search the ENCODE Portal for experiments matching structured criteria.

import requests, pandas as pd

BASE = "https://www.encodeproject.org"

def search_experiments(assay_title=None, target=None, biosample=None,
                       organism="Homo sapiens", status="released", limit=50):
    """
    Search ENCODE experiments with flexible filters.
    Returns: pd.DataFrame of matching experiments.
    """
    params = {
        "type": "Experiment",
        "status": status,
        "replicates.library.biosample.donor.organism.scientific_name": organism,
        "format": "json",
        "limit": limit,
    }
    if assay_title:
        params["assay_title"] = assay_title
    if target:
        params["target.label"] = target
    if biosample:
        params["biosample_ontology.term_name"] = biosample   # filter by ontology term, not the freetext `biosample_summary`

    r = requests.get(f"{BASE}/search/", params=params, timeout=30)
    r.raise_for_status()
    data = r.json()
    total = data.get("total", 0)
    print(f"Total matching experiments: {total} (showing {min(limit, total)})")

    records = []
    for exp in data.get("@graph", []):
        records.append({
            "accession": exp.get("accession"),
            "assay": exp.get("assay_title"),
            "biosample": exp.get("biosample_summary"),
            "target": exp.get("target", {}).get("label", ""),
            "lab": exp.get("lab", {}).get("title", ""),
            "date_released": exp.get("date_released", ""),
        })
    df = pd.DataFrame(records)
    print(df.to_string(index=False))
    return df

# CTCF ChIP-seq in HCT116 colon cancer cells
df = search_experiments(assay_title="TF ChIP-seq", target="CTCF", biosample="HCT116")
# ATAC-seq experiments in multiple cell types
df_atac = search_experiments(assay_title="ATAC-seq", limit=20)
print(f"\nUnique cell types: {df_atac['biosample'].nunique()}")

Query 2: File Download — Get Metadata and Download URLs for BED/bigWig Files

Retrieve file metadata for a specific experiment and obtain download URLs.

import requests, pandas as pd

BASE = "https://www.encodeproject.org"

def get_experiment_files(accession, file_format="bed", output_type="peaks",
                         assembly="GRCh38"):
    """
    Get file download URLs for a specific ENCODE experiment.
    accession: experiment accession, e.g. 'ENCSR000AKC'
    file_format: 'bed', 'bigWig', 'fastq', 'bam'
    output_type: 'peaks', 'signal', 'alignments', 'reads'
    Returns: pd.DataFrame of matching files with download URLs.
    """
    params = {
        "type": "File",
        "dataset": f"/experiments/{accession}/",
        "file_format": file_format,
        "output_type": output_type,
        "assembly": assembly,
        "status": "released",
        "format": "json",
        "limit": 50,
    }
    r = requests.get(f"{BASE}/search/", params=params, timeout=30)
    r.raise_for_status()
    data = r.json()
    files = data.get("@graph", [])
    print(f"Found {len(files)} {file_format} files ({output_type}) for {accession}")

    records = []
    for f in files:
        records.append({
            "file_accession": f.get("accession"),
            "file_format": f.get("file_format"),
            "output_type": f.get("output_type"),
            "assembly": f.get("assembly"),
            "file_size_mb": round(f.get("file_size", 0) / 1e6, 2),
            "download_url": BASE + f.get("href", ""),
            "biological_replicate": str(f.get("biological_replicates", [])),
        })
    df = pd.DataFrame(records)
    print(df[["file_accession", "output_type", "assembly", "file_size_mb"]].to_string(index=False))
    return df

# Get peak BED files for a CTCF ChIP-seq experiment
df_files = get_experiment_files("ENCSR000AKC", file_format="bed", output_type="peaks")
import urllib.request

def download_encode_file(download_url, output_path):
    """Download an ENCODE file from its href URL."""
    print(f"Downloading {download_url} → {output_path}")
    urllib.request.urlretrieve(download_url, output_path)
    import os
    size_mb = os.path.getsize(output_path) / 1e6
    print(f"Downloaded {size_mb:.1f} MB → {output_path}")

# Download the first peak file
if len(df_files) > 0:
    url = df_files.iloc[0]["download_url"]
    download_encode_file(url, "CTCF_peaks.bed.gz")

Query 3: cCRE Region Query — ENCODE SCREEN Candidate cis-Regulatory Elements

Query ENCODE SCREEN for candidate cis-Regulatory Elements in a genomic region using the SCREEN API.

import requests, pandas as pd

SCREEN_BASE = "https://api.encodeproject.org/screen"

def search_ccres_by_region(chrom, start, end, assembly="GRCh38", limit=500):
    """
    Find candidate cis-Regulatory Elements (cCREs) in a genomic region.
    cCRE groups: PLS (promoter-like), pELS (proximal enhancer-like),
                 dELS (distal enhancer-like), DNase-H3K4me3, CTCF-only.
    Returns: pd.DataFrame of cCREs with scores.
    """
    payload = {
        "assembly": assembly,
        "coord_chrom": chrom,
        "coord_start": start,
        "coord_end": end,
        "limit": limit,
    }
    r = requests.post(f"{SCREEN_BASE}/search/", json=payload, timeout=30)
    r.raise_for_status()
    data = r.json()
    ccres = data.get("results", data.get("cCREs", []))
    print(f"Found {len(ccres)} cCREs in {chrom}:{start}-{end}")

    records = []
    for c in ccres:
        records.append({
            "accession": c.get("accession"),
            "chrom": c.get("chrom"),
            "start": c.get("start"),
            "end": c.get("end"),
            "group": c.get("group"),
            "dnase_zscore": round(c.get("dnase_zscore", 0), 2),
            "h3k4me3_zscore": round(c.get("h3k4me3_zscore", 0), 2),
            "h3k27ac_zscore": round(c.get("h3k27ac_zscore", 0), 2),
            "ctcf_zscore": round(c.get("ctcf_zscore", 0), 2),
        })
    df = pd.DataFrame(records)
    if len(df):
        print(df["group"].value_counts().to_string())
    return df

# cCREs in the TP53 locus (GRCh38)
df_ccres = search_ccres_by_region("chr17", 7_661_779, 7_887_538)
print(f"\nTotal cCREs: {len(df_ccres)}")
print(df_ccres.sort_values("h3k27ac_zscore", ascending=False).head(5).to_string(index=False))

Query 4: Gene cCRE Lookup — Find cCREs Near a Gene

Retrieve cCREs in the vicinity of a specific gene using SCREEN's gene-centric endpoint.

import requests, pandas as pd

SCREEN_BASE = "https://api.encodeproject.org/screen"
ENCODE_BASE = "https://www.encodeproject.org"

def get_gene_ccres(gene_symbol, assembly="GRCh38", window_kb=50):
    """
    Get cCREs near a gene using ENCODE SCREEN gene search.
    Returns: pd.DataFrame of nearby cCREs.
    """
    # Step 1: resolve gene to genomic coordinates via ENCODE gene search
    r = requests.get(
        f"{ENCODE_BASE}/search/",
        params={"type": "Gene", "symbol": gene_symbol, "assembly": assembly,
                "format": "json", "limit": 1},
        timeout=30
    )
    r.raise_for_status()
    genes = r.json().get("@graph", [])
    if not genes:
        print(f"Gene {gene_symbol} not found in ENCODE")
        return pd.DataFrame()
    gene = genes[0]
    chrom = gene.get("locations", [{}])[0].get("chromosome", "")
    start = gene.get("locations", [{}])[0].get("start", 0)
    end = gene.get("locations", [{}])[0].get("end", 0)
    print(f"{gene_symbol}: {chrom}:{start}-{end}")

    # Step 2: search cCREs in expanded window
    window = window_kb * 1000
    df = search_ccres_by_region(chrom, max(0, start - window), end + window, assembly=assembly)
    df["distance_to_gene"] = df.apply(
        lambda row: max(0, max(start - row["end"], row["start"] - end)), axis=1
    )
    return df.sort_values("distance_to_gene")

df_brca1_ccres = get_gene_ccres("BRCA1", window_kb=100)
print(f"\ncCREs near BRCA1: {len(df_brca1_ccres)}")
print(df_brca1_ccres[["accession", "group", "start", "end", "h3k27ac_zscore"]].head(10).to_string(index=False))

Query 5: Biosample Browser — List Available Cell Types and Tissues

Enumerate biosample terms (cell lines, primary tissues, developmental stages) available in ENCODE.

import requests, pandas as pd

BASE = "https://www.encodeproject.org"

def list_biosamples(organism="Homo sapiens", biosample_type=None, limit=200):
    """
    List biosamples available in ENCODE.
    biosample_type: 'cell line', 'primary cell', 'tissue', 'in vitro differentiated cells'
    Returns: pd.DataFrame of biosample terms with experiment counts.
    """
    params = {
        "type": "Biosample",
        "donor.organism.scientific_name": organism,
        "format": "json",
        "limit": limit,
    }
    if biosample_type:
        params["biosample_ontology.classification"] = biosample_type

    r = requests.get(f"{BASE}/search/", params=params, timeout=30)
    r.raise_for_status()
    data = r.json()
    biosamples = data.get("@graph", [])
    print(f"Found {data.get('total', 0)} biosamples (showing {len(biosamples)})")

    records = []
    for bs in biosamples:
        records.append({
            "biosample_name": bs.get("biosample_ontology", {}).get("term_name", bs.get("accession")),
            "classification": bs.get("biosample_ontology", {}).get("classification"),
            "tissue": bs.get("biosample_ontology", {}).get("organ_slims", [""])[0] if bs.get("biosample_ontology", {}).get("organ_slims") else "",
            "accession": bs.get("accession"),
        })
    df = pd.DataFrame(records).drop_duplicates("biosample_name").reset_index(drop=True)
    print(df["classification"].value_counts().to_string())
    return df

df_bs = list_biosamples(organism="Homo sapiens", biosample_type="cell line")
print(f"\nUnique human cell lines: {len(df_bs)}")
print(df_bs.head(10).to_string(index=False))

Query 6: Target/TF Query — All ChIP-seq Experiments for a Transcription Factor

Retrieve all released ChIP-seq experiments for a given TF across all available cell types.

import requests, pandas as pd

BASE = "https://www.encodeproject.org"

def get_tf_experiments(tf_name, assay="TF ChIP-seq", assembly="GRCh38", limit=200):
    """
    Find all ENCODE experiments for a given transcription factor.
    Returns: pd.DataFrame of experiments sorted by biosample.
    """
    params = {
        "type": "Experiment",
        "assay_title": assay,
        "target.label": tf_name,
        "files.assembly": assembly,
        "status": "released",
        "format": "json",
        "limit": limit,
    }
    r = requests.get(f"{BASE}/search/", params=params, timeout=30)
    r.raise_for_status()
    data = r.json()
    total = data.get("total", 0)
    print(f"{tf_name} {assay}: {total} total experiments (assembly {assembly})")

    records = []
    for exp in data.get("@graph", []):
        records.append({
            "accession": exp.get("accession"),
            "biosample": exp.get("biosample_summary"),
            "lab": exp.get("lab", {}).get("title", ""),
            "date_released": exp.get("date_released", ""),
        })
    df = pd.DataFrame(records).sort_values("biosample")
    print(f"Showing {len(df)} experiments across {df['biosample'].nunique()} cell types/tissues")
    print(df.head(10).to_string(index=False))
    return df

df_tp53 = get_tf_experiments("TP53", assay="TF ChIP-seq")

Query 7: Peak Set Retrieval — Get Optimal Peak Set for an Experiment

Retrieve the optimal replicated peak set (IDR-filtered or pooled) from an experiment.

import requests, pandas as pd

BASE = "https://www.encodeproject.org"

def get_optimal_peaks(accession, assembly="GRCh38"):
    """
    Retrieve the optimal peak file (IDR-filtered or pooled) from a ChIP-seq experiment.
    Returns: dict with file accession, download URL, and file metadata.
    """
    r = requests.get(f"{BASE}/experiments/{accession}/?format=json", timeout=30)
    r.raise_for_status()
    exp = r.json()

    # Preferred output types in order of preference
    preferred = ["IDR thresholded peaks", "optimal IDR thresholded peaks",
                 "peaks", "replicated peaks"]

    matching = []
    for f in exp.get("files", []):
        if (f.get("file_format") == "bed"
                and f.get("assembly") == assembly
                and f.get("status") == "released"
                and f.get("output_type") in preferred):
            matching.append({
                "file_accession": f.get("accession"),
                "output_type": f.get("output_type"),
                "file_size_mb": round(f.get("file_size", 0) / 1e6, 2),
                "download_url": BASE + f.get("href", ""),
                "biological_replicates": f.get("biological_replicates"),
            })

    if not matching:
        print(f"No peak BED files found for {accession} ({assembly})")
        return None

    # Pick the highest-preference output type
    for pref in preferred:
        for f in matching:
            if f["output_type"] == pref:
                print(f"Selected: {f['file_accession']}  ({f['output_type']}, {f['file_size_mb']} MB)")
                print(f"URL: {f['download_url']}")
                return f

    best = matching[0]
    print(f"Fallback: {best['file_accession']}  ({best['output_type']})")
    return best

peak_file = get_optimal_peaks("ENCSR000AKC", assembly="GRCh38")

Key Concepts

ENCODE Data Tiers and File Hierarchy

Each ENCODE experiment has multiple associated files in a hierarchy:

Experiment (e.g., ENCSR000AKC: CTCF ChIP-seq in K562)
├── Raw data: FASTQ files (reads, per replicate)
├── Alignments: BAM files (per replicate, GRCh38)
├── Signal tracks: bigWig (fold-change over control, p-value signal)
└── Peak calls (BED):
    ├── Replicate 1 peaks (narrow/broad)
    ├── Replicate 2 peaks
    ├── Pooled peaks
    ├── IDR thresholded peaks  ← optimal for most analyses
    └── Optimal IDR thresholded peaks  ← preferred default

For most downstream analyses, use IDR thresholded peaks or optimal IDR thresholded peaks — these are the reproducibility-filtered, high-confidence peak sets.

cCRE Classification System

ENCODE SCREEN classifies cCREs into five functional groups based on epigenomic signal z-scores:

GroupFull NameSignalsInterpretation
PLSPromoter-Like SequenceHigh DNase + High H3K4me3 + High H3K27acActive promoter region
pELSProximal Enhancer-Like SequenceHigh DNase + High H3K27ac, ≤2 kb from TSSProximal enhancer
dELSDistal Enhancer-Like SequenceHigh DNase + High H3K27ac, >2 kb from TSSDistal enhancer
DNase-H3K4me3—High DNase + High H3K4me3, low H3K27acUnusual promoter signature
CTCF-only—High CTCF, low H3K27acInsulator/boundary element

Z-scores >1.64 (p < 0.05) are considered significant in each signal category.

Common Workflows

Workflow 1: Download TF Peak Files for a Cell Type

Goal: Retrieve CTCF ChIP-seq peak BED files for a specific cell line and load them into a DataFrame for variant annotation.

import requests, pandas as pd, time, gzip, io

BASE = "https://www.encodeproject.org"

def download_tf_peaks_for_cell_type(tf_name, biosample, assembly="GRCh38"):
    """Find and download IDR peak BED for a TF in a specific cell type."""
    # Step 1: find experiments
    params = {
        "type": "Experiment",
        "assay_title": "TF ChIP-seq",
        "target.label": tf_name,
        "biosample_ontology.term_name": biosample,   # filter by ontology term, not the freetext `biosample_summary`
        "files.assembly": assembly,
        "status": "released",
        "format": "json",
        "limit": 5,
    }
    r = requests.get(f"{BASE}/search/", params=params, timeout=30)
    r.raise_for_status()
    exps = r.json().get("@graph", [])
    if not exps:
        print(f"No {tf_name} experiments found for {biosample}")
        return None
    accession = exps[0]["accession"]
    print(f"Using experiment {accession}")

    # Step 2: find optimal peak file
    peak = get_optimal_peaks(accession, assembly=assembly)
    if not peak:
        return None

    # Step 3: download and parse BED
    r2 = requests.get(peak["download_url"], timeout=120, stream=True)
    r2.raise_for_status()
    content = r2.content
    if peak["download_url"].endswith(".gz"):
        content = gzip.decompress(content)
    lines = content.decode("utf-8").strip().split("\n")
    rows = [l.split("\t") for l in lines if l and not l.startswith("#")]

    # Narrow peak BED6+4 columns
    cols = ["chrom", "start", "end", "name", "score", "strand",
            "signalValue", "pValue", "qValue", "peak"]
    df = pd.DataFrame(rows, columns=cols[:len(rows[0])] if rows else cols)
    df["start"] = pd.to_numeric(df["start"])
    df["end"] = pd.to_numeric(df["end"])
    print(f"Loaded {len(df):,} peaks for {tf_name} in {biosample} ({assembly})")
    print(f"Genome coverage: {(df['end'] - df['start']).sum() / 1e6:.1f} Mb")
    return df

df_peaks = download_tf_peaks_for_cell_type("CTCF", "K562")
if df_peaks is not None:
    df_peaks.to_csv("CTCF_K562_peaks.bed", sep="\t", index=False, header=False)
    print("Saved CTCF_K562_peaks.bed")

Workflow 2: cCRE Category Bar Chart for a Genomic Region

Goal: Query SCREEN for cCREs in a disease locus and visualize the distribution of regulatory element types.

import requests, pandas as pd
import matplotlib.pyplot as plt

SCREEN_BASE = "https://api.encodeproject.org/screen"

def ccre_category_chart(chrom, start, end, assembly="GRCh38", label="Region"):
    """Query SCREEN cCREs and plot category distribution."""
    payload = {"assembly": assembly, "coord_chrom": chrom,
               "coord_start": start, "coord_end": end, "limit": 1000}
    r = requests.post(f"{SCREEN_BASE}/search/", json=payload, timeout=30)
    r.raise_for_status()
    data = r.json()
    ccres = data.get("results", data.get("cCREs", []))

    if not ccres:
        print(f"No cCREs found in {chrom}:{start}-{end}")
        return pd.DataFrame()

    df = pd.DataFrame(ccres)
    print(f"cCREs in {chrom}:{start}-{end}: {len(df)}")

    # Count by group
    group_counts = df["group"].value_counts()
    group_colors = {
        "PLS": "#d62728",        # red — promoters
        "pELS": "#ff7f0e",       # orange — proximal enhancers
        "dELS": "#1f77b4",       # blue — distal enhancers
        "DNase-H3K4me3": "#9467bd",  # purple
        "CTCF-only": "#2ca02c",  # green — insulators
    }
    colors = [group_colors.get(g, "#aec7e8") for g in group_counts.index]

    fig, axes = plt.subplots(1, 2, figsize=(12, 5))

    # Bar chart of category counts
    axes[0].bar(group_counts.index, group_counts.values, color=colors, edgecolor="black")
    axes[0].set_xlabel("cCRE Category")
    axes[0].set_ylabel("Count")
    axes[0].set_title(f"cCRE Categories — {label}\n({chrom}:{start}-{end})")
    for i, (cat, cnt) in enumerate(group_counts.items()):
        axes[0].text(i, cnt + 0.3, str(cnt), ha="center", va="bottom", fontsize=9)

    # H3K27ac z-score distribution by category
    if "h3k27ac_zscore" in df.columns:
        groups_present = [g for g in ["PLS", "pELS", "dELS"] if g in df["group"].values]
        data_to_plot = [df.loc[df["group"] == g, "h3k27ac_zscore"].dropna().tolist()
                        for g in groups_present]
        axes[1].boxplot(data_to_plot, labels=groups_present,
                        patch_artist=True,
                        boxprops=dict(facecolor="lightsteelblue"))
        axes[1].set_xlabel("cCRE Group")
        axes[1].set_ylabel("H3K27ac Z-score")
        axes[1].set_title("H3K27ac Signal Strength by cCRE Group")
        axes[1].axhline(1.64, color="red", linestyle="--", label="p<0.05 threshold")
        axes[1].legend()

    plt.tight_layout()
    plt.savefig("ccre_category_chart.png", dpi=150, bbox_inches="tight")
    print(f"Saved ccre_category_chart.png")
    print(group_counts.to_string())
    return df

# BRCA1/BRCA2 locus — chr17 region
df_ccres = ccre_category_chart("chr17", 43_044_295, 43_170_000, label="BRCA1 locus")

Workflow 3: Build a TF Peak Atlas Across Multiple Cell Types

Goal: Systematically collect peak BED accessions for one TF across all available cell types for a comparative regulatory analysis.

import requests, pandas as pd, time

BASE = "https://www.encodeproject.org"

def build_tf_peak_atlas(tf_name, assembly="GRCh38", max_experiments=50):
    """
    Collect peak file metadata for a TF across all available cell types.
    Returns: pd.DataFrame with one row per experiment (download URL included).
    """
    params = {
        "type": "Experiment",
        "assay_title": "TF ChIP-seq",
        "target.label": tf_name,
        "files.assembly": assembly,
        "status": "released",
        "format": "json",
        "limit": max_experiments,
    }
    r = requests.get(f"{BASE}/search/", params=params, timeout=30)
    r.raise_for_status()
    exps = r.json().get("@graph", [])
    print(f"{tf_name}: {len(exps)} experiments across {assembly}")

    atlas_records = []
    for exp in exps:
        acc = exp.get("accession")
        peak = get_optimal_peaks(acc, assembly=assembly)
        if peak:
            atlas_records.append({
                "experiment": acc,
                "biosample": exp.get("biosample_summary", ""),
                "lab": exp.get("lab", {}).get("title", ""),
                "file_accession": peak["file_accession"],
                "output_type": peak["output_type"],
                "file_size_mb": peak["file_size_mb"],
                "download_url": peak["download_url"],
            })
        time.sleep(0.3)

    df = pd.DataFrame(atlas_records)
    print(f"Peak files found: {len(df)} / {len(exps)} experiments")
    df.to_csv(f"{tf_name}_peak_atlas_{assembly}.csv", index=False)
    print(f"Saved {tf_name}_peak_atlas_{assembly}.csv")
    return df

# Collect CTCF peak atlas
df_atlas = build_tf_peak_atlas("CTCF", max_experiments=20)
print(f"\nCell types covered: {df_atlas['biosample'].nunique()}")

Key Parameters

ParameterEndpoint / FunctionDefaultRange / OptionsEffect
typeAll search endpointsrequired"Experiment", "File", "Gene", "Biosample"Result type to search
assay_titleExperiment search—"TF ChIP-seq", "ATAC-seq", "DNase-seq", "Histone ChIP-seq", "RNA-seq"Filter by assay type
target.labelExperiment search—TF name string, e.g. "CTCF", "TP53"Filter by target protein
biosample_summaryExperiment search—Cell type string, e.g. "K562", "HeLa-S3"Filter by biosample
files.assemblyExperiment search—"GRCh38", "GRCh37", "mm10"Filter by genome assembly
output_typeFile search—"peaks", "IDR thresholded peaks", "signal", "alignments"Filter by file output type
statusAll search endpoints—"released", "in progress", "archived"Filter by release status
limitAll search endpoints251–500Max results per page
coord_chromSCREEN cCRE searchrequiredchromosome string, e.g. "chr17"Chromosome for region query
coord_startSCREEN cCRE searchrequiredintegerRegion start coordinate
coord_endSCREEN cCRE searchrequiredintegerRegion end coordinate

Best Practices

  1. Use IDR thresholded peaks for reproducible analyses: Raw replicate peaks contain many false positives. Always prefer "output_type": "IDR thresholded peaks" or "optimal IDR thresholded peaks" for any variant annotation or motif analysis.

  2. Filter by status=released and specify assembly: ENCODE stores data for multiple genome builds. Always include files.assembly=GRCh38 (or your target assembly) to avoid mixing coordinate systems from archived experiments.

  3. Add time.sleep(0.5) in loops over experiments: The ENCODE Portal does not publish a hard rate limit, but aggressive sequential requests will trigger connection resets. Polite delays prevent this in atlas-building workflows.

  4. Prefer the SCREEN API for cCRE queries, not the Portal search: The ENCODE Portal search can find cCRE-related files, but the dedicated SCREEN API (api.encodeproject.org/screen) is purpose-built for region-based cCRE lookup and returns structured z-score data.

  5. Check biosample_summary vs biosample_ontology.term_name: The biosample_summary field (used in search) sometimes differs from the official ontology term name. If a biosample search returns zero results, try browsing ENCODE to find the exact summary string used.

Common Recipes

Recipe: Find All ENCODE Assay Types Available

When to use: Explore what assay types are available before planning a targeted search.

import requests

BASE = "https://www.encodeproject.org"

r = requests.get(
    f"{BASE}/search/",
    params={"type": "Experiment", "status": "released", "format": "json",
            "limit": 0, "field": "assay_title"},
    timeout=30
)
r.raise_for_status()
# Facets contain aggregated counts per assay type
for facet in r.json().get("facets", []):
    if facet.get("field") == "assay_title":
        for term in sorted(facet.get("terms", []), key=lambda x: -x["doc_count"])[:20]:
            print(f"  {term['doc_count']:6d}  {term['key']}")

Recipe: Get Experiment Metadata by Accession

When to use: Retrieve full details for a known ENCODE accession (e.g., from a published paper).

import requests, json

BASE = "https://www.encodeproject.org"

def get_experiment_metadata(accession):
    r = requests.get(f"{BASE}/experiments/{accession}/?format=json", timeout=30)
    r.raise_for_status()
    exp = r.json()
    print(f"Accession  : {exp['accession']}")
    print(f"Assay      : {exp.get('assay_title')}")
    print(f"Biosample  : {exp.get('biosample_summary')}")
    print(f"Target     : {exp.get('target', {}).get('label', 'N/A')}")
    print(f"Lab        : {exp.get('lab', {}).get('title')}")
    print(f"Files      : {len(exp.get('files', []))}")
    print(f"Released   : {exp.get('date_released')}")
    return exp

exp = get_experiment_metadata("ENCSR000AKC")

Recipe: List bigWig Signal Files for Visualization

When to use: Download signal bigWig tracks to load into IGV or UCSC Genome Browser.

import requests, pandas as pd

BASE = "https://www.encodeproject.org"

def get_signal_bigwigs(accession, assembly="GRCh38"):
    """Get fold-change and p-value signal bigWig files for an experiment."""
    params = {
        "type": "File",
        "dataset": f"/experiments/{accession}/",
        "file_format": "bigWig",
        "output_type": "fold change over control",
        "assembly": assembly,
        "status": "released",
        "format": "json",
        "limit": 10,
    }
    r = requests.get(f"{BASE}/search/", params=params, timeout=30)
    r.raise_for_status()
    files = r.json().get("@graph", [])
    print(f"Found {len(files)} bigWig signal files for {accession}")
    for f in files:
        print(f"  {f['accession']}  {f['output_type']}  {f.get('biological_replicates')}  "
              f"{round(f.get('file_size', 0)/1e6, 1)} MB")
        print(f"  URL: {BASE}{f['href']}")
    return files

bigwigs = get_signal_bigwigs("ENCSR000AKC")

Troubleshooting

ProblemCauseSolution
HTTP 404 on experiment endpointAccession does not exist or was revokedVerify accession on https://www.encodeproject.org; check status field in search results
Empty @graph in search resultsOverly restrictive filters or no matching dataRelax one filter at a time; check total field to see if data exists before pagination
HTTP 503 or connection timeoutServer overload from rapid sequential requestsAdd time.sleep(0.5) between requests; retry with exponential backoff
cCRE search returns []SCREEN API endpoint path changed or region on non-canonical chromosomeVerify API URL; use canonical chromosomes (chr1–22, chrX, chrY) only
bigWig/BED files not foundAssembly mismatch (GRCh37 vs GRCh38)Always include files.assembly in queries; verify the experiment has files in your target assembly
target.label filter returns no resultsExact label mismatch (case-sensitive)Browse ENCODE to find the exact string (e.g., "eGFP-TP53" vs "TP53")
File download fails mid-transferLarge file + network timeoutUse stream=True in requests with chunked writing; increase timeout to 300

Related Skills

  • regulomedb-database — Pre-computed regulatory scores for variants integrating ENCODE ChIP-seq, DNase-seq, eQTL, and motif data
  • gwas-database — NHGRI-EBI GWAS Catalog for published SNP-trait associations; combine with ENCODE peaks for functional annotation of GWAS hits
  • macs3-peak-calling — Call peaks from your own ChIP-seq or ATAC-seq BAM files; use ENCODE peaks as reference comparison sets
  • deeptools-ngs-analysis — Generate bigWig coverage tracks, correlation heatmaps, and profile plots from your own aligned BAM files

References

Ensembl Genome Database

Overview

Ensembl is a comprehensive genome annotation database covering 300+ vertebrate and non-vertebrate species. The Ensembl REST API provides programmatic access to gene models, transcript/protein sequences, variant annotations, cross-references, regulatory features, and comparative genomics without requiring any login or API key.

When to Use

  • Retrieving official gene and transcript annotations (stable IDs, biotype, genomic coordinates) for human or model organism genes
  • Converting between gene identifier namespaces (HGNC symbol ↔ Ensembl ID ↔ RefSeq ↔ UniProt)
  • Fetching genomic or cDNA/CDS/protein sequences for a gene or transcript
  • Looking up variant consequences and functional impact (VEP) for a list of SNPs
  • Querying regulatory features (promoters, enhancers, CTCF sites) in a genomic region
  • Performing comparative genomics queries (orthologs, paralogs, gene trees) across species
  • For local offline access to large genomic annotations, use pyensembl instead
  • For pathway and metabolic annotations, use kegg-database or reactome-database instead

Prerequisites

  • Python packages: requests
  • Data requirements: gene symbols, Ensembl stable IDs (ENSG…/ENST…/ENSP…), or genomic coordinates
  • Environment: internet connection required; no API key needed
  • Rate limits: max ~15 requests/second; use expand=1 and batch endpoints to minimize calls
pip install requests

Quick Start

import requests

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

def ensembl_get(endpoint, params=None):
    r = requests.get(f"{BASE}{endpoint}", headers=HEADERS, params=params)
    r.raise_for_status()
    return r.json()

# Look up human BRCA1
gene = ensembl_get("/lookup/symbol/homo_sapiens/BRCA1", params={"expand": 1})
print(f"ID: {gene['id']}, Chr: {gene['seq_region_name']}:{gene['start']}-{gene['end']}")
print(f"Transcripts: {len(gene.get('Transcript', []))}")

Core API

Query 1: Gene Lookup by Symbol or Stable ID

Retrieve gene metadata from a gene symbol or Ensembl stable ID.

import requests

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

# By gene symbol
r = requests.get(
    f"{BASE}/lookup/symbol/homo_sapiens/TP53",
    headers=HEADERS,
    params={"expand": 1}
)
gene = r.json()
print(f"Ensembl ID : {gene['id']}")
print(f"Location   : {gene['seq_region_name']}:{gene['start']}-{gene['end']} ({gene['strand']})")
print(f"Biotype    : {gene['biotype']}")
print(f"Transcripts: {len(gene.get('Transcript', []))}")
# By stable ID (works for genes, transcripts, proteins)
r = requests.get(
    f"{BASE}/lookup/id/ENSG00000141510",
    headers=HEADERS,
    params={"expand": 0}
)
obj = r.json()
print(f"Symbol: {obj.get('display_name')}, Species: {obj.get('species')}")

Query 2: Batch Lookup

Retrieve information for multiple IDs in one call (POST endpoint).

import requests, json

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

# Batch lookup by symbols
symbols = ["BRCA1", "BRCA2", "TP53", "EGFR", "MYC"]
r = requests.post(
    f"{BASE}/lookup/symbol/homo_sapiens",
    headers=HEADERS,
    data=json.dumps({"symbols": symbols})
)
results = r.json()
for sym, data in results.items():
    if data:
        print(f"{sym}: {data['id']} ({data['seq_region_name']}:{data['start']}-{data['end']})")

Query 3: Sequence Retrieval

Fetch genomic, cDNA, CDS, or protein sequences.

import requests

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "text/plain"}

# Protein sequence for canonical transcript
r = requests.get(
    f"{BASE}/sequence/id/ENST00000269305",
    headers=HEADERS,
    params={"type": "protein"}
)
seq = r.text
print(f"Protein sequence ({len(seq)} aa): {seq[:60]}...")
# Genomic region sequence
HEADERS_JSON = {"Content-Type": "application/json"}
r = requests.get(
    f"{BASE}/sequence/region/human/17:43044295..43125364",
    headers=HEADERS_JSON,
    params={"coord_system_version": "GRCh38"}
)
result = r.json()
print(f"Retrieved {len(result['seq'])} bp of genomic sequence")

Query 4: Cross-References (ID Mapping)

Map Ensembl IDs to external database identifiers.

import requests

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

# All xrefs for a gene
r = requests.get(
    f"{BASE}/xrefs/id/ENSG00000141510",
    headers=HEADERS
)
xrefs = r.json()

# Group by database
from collections import defaultdict
by_db = defaultdict(list)
for x in xrefs:
    by_db[x["dbname"]].append(x["primary_id"])

for db in ["HGNC", "RefSeq_gene_name", "Uniprot_gn", "MIM_gene"]:
    if db in by_db:
        print(f"{db}: {by_db[db]}")

Query 5: Variant Consequence Annotation (VEP)

Predict functional consequences of variants via REST VEP endpoint.

import requests, json

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

# Annotate a list of hgvs notations
variants = ["17:g.43094692C>T", "13:g.32929387C>T"]
r = requests.post(
    f"{BASE}/vep/human/hgvs",
    headers=HEADERS,
    data=json.dumps({"hgvs_notations": variants})
)
for v in r.json():
    print(f"\nVariant: {v.get('input')}")
    for tc in v.get("transcript_consequences", [])[:2]:
        print(f"  Gene: {tc.get('gene_symbol')}, Impact: {tc.get('impact')}, Consequence: {tc.get('consequence_terms')}")
# Annotate by rsID
r = requests.get(
    f"{BASE}/vep/human/id/rs699",
    headers=HEADERS
)
v = r.json()[0]
print(f"rsID rs699 in gene: {v['transcript_consequences'][0]['gene_symbol']}")
print(f"Consequence: {v['transcript_consequences'][0]['consequence_terms']}")

Query 6: Regulatory Features

Query regulatory build features in a genomic region.

import requests

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

# Regulatory features in BRCA1 region
r = requests.get(
    f"{BASE}/overlap/region/human/17:43044000-43126000",
    headers=HEADERS,
    params={"feature": "regulatory"}
)
features = r.json()
print(f"Found {len(features)} regulatory features")
for f in features[:5]:
    print(f"  {f.get('feature_type')}: {f.get('start')}-{f.get('end')} ({f.get('description', 'n/a')})")

Query 7: Comparative Genomics (Orthologs / Gene Trees)

Find orthologs and paralogs across species.

import requests

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

# Get mouse ortholog for human TP53
r = requests.get(
    f"{BASE}/homology/symbol/human/TP53",
    headers=HEADERS,
    params={"target_species": "mus_musculus", "type": "orthologues"}
)
data = r.json()
for homo in data["data"][0]["homologies"][:3]:
    tgt = homo["target"]
    print(f"Mouse ortholog: {tgt['id']} ({tgt.get('perc_id', 'n/a')}% identity)")

Key Concepts

Stable IDs and Versioning

Ensembl uses stable IDs with optional version suffixes (e.g., ENSG00000141510.17). Genes (ENSG), transcripts (ENST), proteins (ENSP), and exons (ENSE) each have their own prefix. IDs are preserved across releases when possible; retired IDs can still be resolved via the archive API.

Assembly Versions

Human genome: GRCh38 (current) and GRCh37 (legacy, via grch37.rest.ensembl.org). Always specify which assembly your coordinates belong to when making region-based queries.

Common Workflows

Workflow 1: Gene-to-Protein Information Pipeline

Goal: Retrieve all key annotations for a gene list — coordinates, transcripts, xrefs, and canonical protein sequence.

import requests, json, time

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

def batch_lookup(symbols, species="homo_sapiens"):
    r = requests.post(
        f"{BASE}/lookup/symbol/{species}",
        headers=HEADERS,
        data=json.dumps({"symbols": symbols, "expand": 1})
    )
    return r.json()

def canonical_transcript(gene_data):
    """Return the ID of the canonical (longest CDS) transcript."""
    transcripts = gene_data.get("Transcript", [])
    coding = [t for t in transcripts if t.get("biotype") == "protein_coding"]
    if not coding:
        return None
    return max(coding, key=lambda t: t.get("Translation", {}).get("length", 0))

genes = ["BRCA1", "BRCA2", "TP53"]
lookup = batch_lookup(genes)

for sym in genes:
    g = lookup.get(sym)
    if not g:
        print(f"{sym}: not found")
        continue
    canon = canonical_transcript(g)
    print(f"\n{sym} ({g['id']})")
    print(f"  Location: {g['seq_region_name']}:{g['start']}-{g['end']}")
    if canon:
        prot_len = canon.get("Translation", {}).get("length", "n/a")
        print(f"  Canonical transcript: {canon['id']} ({prot_len} aa)")
    time.sleep(0.1)  # be polite

Workflow 2: Variant Annotation Pipeline

Goal: Annotate a VCF-style variant list with gene, consequence, and impact.

import requests, json, pandas as pd

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

# Input: list of hgvs notations
hgvs_list = [
    "17:g.43094692C>T",
    "17:g.43063873A>G",
    "13:g.32929387C>T",
]

# Annotate in batches of 200
def vep_batch(hgvs_batch):
    r = requests.post(
        f"{BASE}/vep/human/hgvs",
        headers=HEADERS,
        data=json.dumps({"hgvs_notations": hgvs_batch})
    )
    r.raise_for_status()
    return r.json()

records = []
for ann in vep_batch(hgvs_list):
    for tc in ann.get("transcript_consequences", []):
        if tc.get("canonical") == 1:
            records.append({
                "variant": ann["input"],
                "gene": tc.get("gene_symbol"),
                "consequence": ",".join(tc.get("consequence_terms", [])),
                "impact": tc.get("impact"),
                "biotype": tc.get("biotype"),
            })

df = pd.DataFrame(records)
print(df.to_string(index=False))
df.to_csv("vep_results.csv", index=False)
print(f"\nSaved {len(df)} variant annotations → vep_results.csv")

Key Parameters

ParameterModuleDefaultRange / OptionsEffect
expandLookup00 or 1Include nested transcripts/translations
typeSequence"genomic""genomic", "cDNA", "CDS", "protein"Sequence type to return
target_speciesHomologyNoneSpecies name or taxon IDFilter homologs to target species
featureOverlaprequired"gene", "transcript", "regulatory", "variation"Feature type to retrieve
coord_system_versionRegion"GRCh38""GRCh38", "GRCh37"Genome assembly
content_typeAllvia header"application/json", "text/plain"Response format

Best Practices

  1. Use batch endpoints: POST /lookup/symbol/{species} and POST /vep/human/hgvs accept up to 1000 IDs; single-ID GET requests in a loop will hit rate limits quickly.

  2. Pin assembly version: For region-based queries always specify coord_system_version=GRCh38 (or use grch37.rest.ensembl.org for legacy coordinates) to avoid silent mismatch errors.

  3. Cache responses: Gene metadata rarely changes between Ensembl releases; cache results to disk (joblib.Memory) to avoid redundant API calls during development.

    from joblib import Memory
    mem = Memory("cache/", verbose=0)
    cached_lookup = mem.cache(batch_lookup)
    
  4. Use expand=0 for metadata: When you only need gene coordinates and biotype (not transcript details), keep expand=0 for smaller payloads and faster responses.

  5. Check canonical flag in VEP: VEP returns consequences for all overlapping transcripts; filter on tc.get("canonical") == 1 to get the biologically most relevant consequence per variant.

Common Recipes

Recipe: Symbol → Ensembl ID Mapping Table

When to use: Build a lookup table from gene symbols to Ensembl IDs for downstream analysis.

import requests, json, pandas as pd

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

symbols = ["EGFR", "KRAS", "BRAF", "PIK3CA", "PTEN", "AKT1", "MYC", "RB1"]
r = requests.post(
    f"{BASE}/lookup/symbol/homo_sapiens",
    headers=HEADERS,
    data=json.dumps({"symbols": symbols})
)
data = r.json()
rows = [{"symbol": s, "ensembl_id": d["id"] if d else None,
         "chrom": d["seq_region_name"] if d else None} for s, d in data.items()]
df = pd.DataFrame(rows)
df.to_csv("symbol_to_ensembl.csv", index=False)
print(df.to_string(index=False))

Recipe: Region Gene Overlap

When to use: Find all genes overlapping a genomic interval (e.g., a GWAS locus).

import requests, pandas as pd

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

chrom, start, end = "17", 43044295, 43125364
r = requests.get(
    f"{BASE}/overlap/region/human/{chrom}:{start}-{end}",
    headers=HEADERS,
    params={"feature": "gene", "biotype": "protein_coding"}
)
genes = r.json()
df = pd.DataFrame([{
    "id": g["id"], "name": g.get("external_name"),
    "start": g["start"], "end": g["end"], "strand": g["strand"]
} for g in genes])
print(df.to_string(index=False))
print(f"\n{len(df)} protein-coding genes in region")

Recipe: Species List

When to use: Check which species are available in Ensembl before querying.

import requests

BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}

r = requests.get(f"{BASE}/info/species", headers=HEADERS)
species_list = r.json()["species"]
print(f"Total species: {len(species_list)}")
vertebrates = [s for s in species_list if s.get("division") == "EnsemblVertebrates"]
print(f"Vertebrates: {len(vertebrates)}")
for s in vertebrates[:5]:
    print(f"  {s['common_name']} ({s['name']}): {s['assembly']}")

Troubleshooting

ProblemCauseSolution
HTTP 429 Too Many RequestsExceeding ~15 req/s rate limitAdd time.sleep(0.1) between requests; use batch POST endpoints
HTTP 400 Bad Request on VEPMalformed HGVS notationVerify format: chr:g.posREF>ALT (e.g., 17:g.43094692C>T)
Gene not foundGene symbol not in EnsemblTry alternative symbol; check species name (use homo_sapiens not human for symbols)
Region query returns wrong genesAssembly mismatchSet coord_system_version=GRCh38 or use grch37.rest.ensembl.org
Old ID not resolvingRetired Ensembl IDQuery GET /archive/id/{id} to get current mapping
HTTP 503 Service UnavailableServer maintenanceRetry after a few minutes; check Ensembl status at status.ensembl.org

Related Skills

  • gget-genomic-databases — CLI/Python wrapper covering Ensembl + 20 other databases; use for quick lookups without raw API code
  • biopython-molecular-biology — Biopython's Entrez module for NCBI databases (alternative for RefSeq/GenBank queries)
  • kegg-database — Pathway/metabolic annotations for the same gene set
  • reactome-database — Pathway enrichment and hierarchy queries

References

gene-databasegeo-databasegget-genomic-databasesgnomad-databasegwas-databasejaspar-databasekegg-databasemonarch-databasemouse-phenome-databasequickgo-databaseregulomedb-databaseremap-databaseucsc-genome-browseretetoolkithomer-motif-analysisbedtools-genomic-intervalsdeeptools-ngs-analysisgenimlgtarsmacs3-peak-callingbusco-status-interpretationfastp-fastq-preprocessingmultiqc-qc-reportsdeseq2-differential-expressionfeaturecounts-rna-countinggseapy-gene-enrichmentpydeseq2-differential-expressionsalmon-rna-quantificationscikit-bioanndata-data-structurecell-type-annotationcelltypist-cell-annotationcellxgene-censusharmony-batch-correctionpopv-cell-annotationscanpy-scrna-seqscvi-tools-single-cellmultimodal-omics-integrationspatial-domain-detectionspatial-mapping-tangramspatial-panel-designspatial-transcriptomics-analysis-squidpyspatial-transcriptomics-cell-cell-communicationspatial-transcriptomics-cell-type-annotationspatial-transcriptomics-deconvolution-cell2locationtrajectory-inferencebcftools-variant-manipulationcnvkit-copy-numbergatk-variant-callingplink2-gwas-analysissnpeff-variant-annotationvcf-variant-filteringbenchling-integrationopentrons-protocol-apiprotocolsio-integrationpylabrobotwestern-blot-quantificationplannotate-plasmid-annotationsgrna-design-guideviennarna-structure-predictionadaptyv-bioesm-protein-language-modelhmdb-databaseinterpro-databasematchms-spectral-matchingmaxquant-proteomicsmetabolomics-workbench-databasepride-databasepyopenms-mass-spectrometryuniprot-protein-databaseaeonastropy-astronomydask-parallel-computingdegenerate-input-filteringexploratory-data-analysisgeopandas-geospatialhypogenic-hypothesis-generationmatlab-scientific-computingnan-safe-correlationneb-irc-activation-energynetworkx-graph-analysisneurokit2neuropixels-analysisnextflow-workflow-enginepolars-dataframespyhealthpymatgenpymooscikit-learn-machine-learningshap-model-explainabilitysimpy-discrete-event-simulationsnakemake-workflow-enginespikeinterface-electrophysiologysympy-symbolic-mathtorch-geometric-graph-neural-networkstransformers-bio-nlpumap-learnuspto-databasevaex-dataframeszarr-pythonbiorxiv-databasecancer-research-figure-guidecell-figure-guidecitation-managementclinical-decision-support-documentselife-figure-guidegeneral-figure-guidehypothesis-generationlancet-figure-guidelatex-research-postersliterature-reviewnature-figure-guidenejm-figure-guideopenalex-databasepeer-review-methodologypnas-figure-guidepubmed-databasescience-figure-guidescientific-brainstormingscientific-critical-thinkingscientific-literature-searchscientific-manuscript-writingscientific-schematicsscientific-slidesaizynthfinder-retrosynthesisalphafold-database-accessautodock-vina-dockingchembl-database-bioactivityclinicaltrials-database-searchdailymed-databasedatamol-cheminformaticsddinter-databasedeepchemdiffdockdrugbank-database-accessemdb-databasefda-databasegtopdb-databasemdanalysis-trajectorymdtraj-trajectory-analysismedchemmolfeat-molecular-featurizationopentargets-databasepdb-databasepubchem-compound-searchpytdc-therapeutics-data-commonsrdkit-chemdraw-cdxmlrdkit-cheminformaticsrowansar-analysissmina-molecular-dockingtorchdrugunichem-databasezinc-databasebrenda-databasecellchat-cell-communicationcobrapy-metabolic-modelingkegg-pathway-analysislamindb-data-managementlibsbml-network-modelingmofaplus-multi-omicsmuon-multiomics-singlecellomics-analysis-guidereactome-databasestring-database-ppi

Sponsors

Details

Marketplace
jaechang-hits/SciAgent-Skills
Author
jaechang-hits
Category
Scientific Research
License
CC-BY-4.0
Keywords
life-sciences, bioinformatics, drug-discovery, genomics, scientific-computing