
sciagent-skills
197 validated life sciences skills — genomics, proteomics, drug discovery, biostatistics, scientific computing
- version
- 1.0.0
- GitHub stars
- 393
Install
/plugin install sciagent-skills@sciagent-skillsRun it inside Claude Code after adding the marketplace with /plugin marketplace add jaechang-hits/SciAgent-Skills.
Contents
Discovered from sourceSkills 205
PyMC Bayesian Modeling
Overview
PyMC is a Python library for Bayesian statistical modeling and probabilistic programming. It provides an expressive syntax for defining probabilistic models and efficient inference via MCMC (NUTS) and variational methods (ADVI). This skill covers the full Bayesian modeling cycle from model specification through diagnostics, comparison, and prediction.
When to Use
- Estimating parameters with full uncertainty quantification (credible intervals, not just point estimates)
- Fitting hierarchical/multilevel models to grouped or nested data
- Performing prior and posterior predictive checks to validate model assumptions
- Comparing candidate models using information criteria (LOO-CV, WAIC)
- Building regression models (linear, logistic, Poisson) in a Bayesian framework
- Handling missing data or measurement error as latent parameters
- Modeling time series with autoregressive or random walk priors
- Generating posterior predictions for new observations with uncertainty bounds
- Use Stan/PyStan instead for compiled, more scalable Bayesian inference on large models; use statsmodels for frequentist statistical tests
Prerequisites
- Python packages:
pymc >= 5.0,arviz,numpy,matplotlib - Data: NumPy arrays or pandas DataFrames with numeric columns
- Environment: CPU sufficient for most models; GPU via JAX backend for large models
pip install pymc arviz numpy matplotlib
# Optional: JAX backend for GPU acceleration
pip install pymc[jax]
Quick Start
import pymc as pm
import arviz as az
import numpy as np
# Simulate data
np.random.seed(42)
X = np.random.randn(100)
y = 2.5 + 1.3 * X + np.random.randn(100) * 0.5
# Build and fit model
with pm.Model() as model:
alpha = pm.Normal("alpha", mu=0, sigma=5)
beta = pm.Normal("beta", mu=0, sigma=5)
sigma = pm.HalfNormal("sigma", sigma=1)
mu = alpha + beta * X
y_obs = pm.Normal("y_obs", mu=mu, sigma=sigma, observed=y)
idata = pm.sample(1000, tune=1000, chains=4, random_seed=42)
print(az.summary(idata, var_names=["alpha", "beta", "sigma"]))
# Expected: alpha ~ 2.5, beta ~ 1.3, sigma ~ 0.5
Workflow
Step 1: Prepare Data
Standardize continuous predictors for better sampling efficiency. Use named coordinates for readable models and ArviZ integration.
import pymc as pm
import arviz as az
import numpy as np
# Load data
X = np.random.randn(200, 3) # 200 obs, 3 predictors
y = X @ np.array([1.0, -0.5, 0.3]) + np.random.randn(200) * 0.8
# Standardize predictors
X_mean, X_std = X.mean(axis=0), X.std(axis=0)
X_scaled = (X - X_mean) / X_std
# Define coordinates for named dimensions
coords = {
"predictors": ["var1", "var2", "var3"],
"obs_id": np.arange(len(y)),
}
print(f"Data shape: X={X_scaled.shape}, y={y.shape}")
Step 2: Define Model and Set Priors
Specify the model structure inside a pm.Model() context. Use weakly informative priors, dims for named dimensions, and HalfNormal or Exponential for scale parameters.
with pm.Model(coords=coords) as model:
# Priors — weakly informative, not flat
alpha = pm.Normal("alpha", mu=0, sigma=1)
beta = pm.Normal("beta", mu=0, sigma=1, dims="predictors")
sigma = pm.HalfNormal("sigma", sigma=1)
# Linear predictor
mu = alpha + pm.math.dot(X_scaled, beta)
# Likelihood
y_obs = pm.Normal("y_obs", mu=mu, sigma=sigma, observed=y, dims="obs_id")
# Inspect model variables
print(model.basic_RVs) # Lists: [alpha, beta, sigma, y_obs]
Step 3: Prior Predictive Check
Validate that priors produce plausible data ranges before fitting. Adjust priors if simulated data is unreasonable.
with model:
prior_pred = pm.sample_prior_predictive(samples=1000, random_seed=42)
# Check prior-implied data range
prior_y = prior_pred.prior_predictive["y_obs"].values.flatten()
print(f"Prior predictive range: [{prior_y.min():.1f}, {prior_y.max():.1f}]")
print(f"Observed data range: [{y.min():.1f}, {y.max():.1f}]")
az.plot_ppc(prior_pred, group="prior", num_pp_samples=100)
Step 4: Sample Posterior (MCMC)
Run NUTS sampling with multiple chains. Include log_likelihood=True if you plan model comparison later.
with model:
idata = pm.sample(
draws=2000,
tune=1000,
chains=4,
target_accept=0.9,
random_seed=42,
idata_kwargs={"log_likelihood": True},
)
print(f"Posterior shape: {idata.posterior['beta'].shape}")
# Expected: (4 chains, 2000 draws, 3 predictors)
Step 5: Diagnose Sampling
Check convergence before interpreting results. All three diagnostics (R-hat, ESS, divergences) must pass.
# Summary with convergence diagnostics
summary = az.summary(idata, var_names=["alpha", "beta", "sigma"])
print(summary[["mean", "sd", "hdi_3%", "hdi_97%", "r_hat", "ess_bulk"]])
# R-hat convergence check
bad_rhat = summary[summary["r_hat"] > 1.01]
if len(bad_rhat) > 0:
print(f"WARNING: {len(bad_rhat)} parameters with R-hat > 1.01")
print(bad_rhat[["r_hat"]])
# Effective sample size check
low_ess = summary[summary["ess_bulk"] < 400]
if len(low_ess) > 0:
print(f"WARNING: {len(low_ess)} parameters with ESS < 400")
# Divergence check
n_div = idata.sample_stats.diverging.sum().item()
total = len(idata.posterior.draw) * len(idata.posterior.chain)
print(f"Divergences: {n_div}/{total} ({n_div / total * 100:.2f}%)")
# Visual diagnostics — trace plots and rank plots
az.plot_trace(idata, var_names=["alpha", "beta", "sigma"])
az.plot_rank(idata, var_names=["alpha", "beta", "sigma"])
Step 6: Posterior Predictive Check
Validate model fit by comparing simulated data from the posterior to observed data.
with model:
pm.sample_posterior_predictive(idata, extend_inferencedata=True, random_seed=42)
az.plot_ppc(idata, num_pp_samples=100)
# Blue = observed data, grey = posterior simulations
# Systematic deviations indicate model misspecification
Step 7: Compare Models
Use LOO-CV or WAIC to compare candidate models. Lower information criterion is better.
# Fit multiple models with log_likelihood=True, then compare
# Example: compare linear vs a second model
idatas = {"linear": idata} # add more fitted models here
comparison = az.compare(idatas, ic="loo")
print(comparison[["rank", "elpd_loo", "p_loo", "d_loo", "weight"]])
# Check LOO reliability via Pareto-k diagnostics
loo_result = az.loo(idata, pointwise=True)
high_k = (loo_result.pareto_k > 0.7).sum().item()
print(f"Observations with Pareto-k > 0.7: {high_k}")
# Interpretation: Dloo < 2 = similar models; Dloo > 10 = strong evidence
az.plot_compare(comparison)
Step 8: Generate Predictions
Produce posterior predictions for new data with full uncertainty propagation.
X_new = np.array([[0.5, -1.0, 0.2]])
X_new_scaled = (X_new - X_mean) / X_std
with model:
pm.set_data({"X_scaled": X_new_scaled})
post_pred = pm.sample_posterior_predictive(
idata.posterior, var_names=["y_obs"], random_seed=42
)
y_pred = post_pred.posterior_predictive["y_obs"]
print(f"Predicted mean: {y_pred.mean().item():.3f}")
print(f"94% HDI: {az.hdi(y_pred, hdi_prob=0.94).values}")
Key Parameters
| Parameter | Default | Range / Options | Effect |
|---|---|---|---|
draws | 1000 | 500-10000 | Number of posterior samples per chain |
tune | 1000 | 500-5000 | Warmup iterations (discarded); increase for complex posteriors |
chains | 4 | 2-8 | Number of independent chains; minimum 4 for reliable R-hat |
cores | all CPUs | 1-N | Parallel chains; set equal to chains for full parallelism |
target_accept | 0.8 | 0.8-0.99 | NUTS acceptance rate; increase to reduce divergences |
init | "auto" | "adapt_diag", "jitter+adapt_diag", "advi" | Initialization strategy for sampler |
random_seed | None | any int | Seed for reproducibility |
idata_kwargs | {} | {"log_likelihood": True} | Store log-likelihood for LOO/WAIC model comparison |
method (pm.fit) | "advi" | "advi", "fullrank_advi", "svgd" | Variational inference algorithm |
n (pm.fit) | 10000 | 5000-100000 | VI optimization iterations |
samples (prior pred) | 500 | 100-5000 | Prior predictive samples for validation |
Key Concepts
Prior/Distribution Selection Guide
| Distribution | Use When | Key Parameters |
|---|---|---|
Normal(mu, sigma) | Unbounded real-valued parameter (standardized data) | mu: center, sigma: spread |
HalfNormal(sigma) | Scale/standard deviation parameter (positive) | sigma: spread of positive half |
Exponential(lam) | Scale parameter, alternative to HalfNormal | lam: rate (1/mean) |
StudentT(nu, mu, sigma) | Robust alternative to Normal (outlier-resistant) | nu: degrees of freedom (<10 = heavier tails) |
Beta(alpha, beta) | Probability or proportion in [0,1] | alpha=beta=2: weakly informative |
Gamma(alpha, beta) | Positive parameter (rate, concentration) | alpha: shape, beta: rate |
LogNormal(mu, sigma) | Positive parameter with multiplicative effects | mu, sigma: of underlying Normal |
LKJCorr(n, eta) | Correlation matrix prior | eta=1: uniform; eta>1: prefer identity |
Dirichlet(a) | Probability vector (sums to 1) | a: concentration; uniform if all equal |
Bernoulli(p / logit_p) | Binary outcome likelihood | Use logit_p for numerical stability |
Poisson(mu) | Count data (equidispersed) | mu: rate; use NegBinomial if overdispersed |
NegativeBinomial(mu, alpha) | Overdispersed count data | alpha: dispersion (smaller = more overdispersion) |
Diagnostic Thresholds
| Metric | Threshold | Interpretation | Action if Failed |
|---|---|---|---|
| R-hat | < 1.01 | Chains converged | Run longer chains; check multimodality |
| ESS bulk | > 400 | Sufficient independent samples | Increase draws; reparameterize |
| ESS tail | > 400 | Reliable tail estimates | Increase draws |
| Divergences | 0 | NUTS explored successfully | Increase target_accept; non-centered param. |
| Pareto-k (LOO) | < 0.7 | LOO estimate reliable | Use WAIC or k-fold CV |
| Max tree depth | < 10 | No trajectory truncation | Reparameterize or increase max_treedepth |
Model Variants Overview
| Problem Type | Recipe | Likelihood | Key Feature |
|---|---|---|---|
| Grouped/nested data | Hierarchical Model | Normal (varies) | Non-centered parameterization, partial pooling |
| Binary outcome | Logistic Regression | Bernoulli | logit_p link function |
| Nonlinear/spatial | Gaussian Process | Normal | Kernel-based covariance, flexible shape |
| Count data | (use Poisson in Workflow) | Poisson / NegBinomial | Log link; NegBinomial for overdispersion |
| Time series | (see references) | AR / GaussianRandomWalk | Autoregressive coefficients |
| Mixture/clustering | (see references) | Mixture / NormalMixture | Component weights via Dirichlet |
Common Recipes
Recipe: Hierarchical Model
When to use: data has natural grouping (patients within hospitals, students within schools). Non-centered parameterization avoids divergences from funnel geometry.
import pymc as pm
import arviz as az
import numpy as np
n_groups, n_per_group = 5, 30
group_idx = np.repeat(np.arange(n_groups), n_per_group)
group_names = [f"group_{i}" for i in range(n_groups)]
# Simulated grouped data
true_alphas = np.random.normal(3.0, 1.5, n_groups)
y_obs = np.random.normal(true_alphas[group_idx], 0.5)
with pm.Model(coords={"groups": group_names}) as hierarchical_model:
# Hyperpriors (population level)
mu_alpha = pm.Normal("mu_alpha", mu=0, sigma=5)
sigma_alpha = pm.HalfNormal("sigma_alpha", sigma=2)
# Non-centered parameterization
alpha_offset = pm.Normal("alpha_offset", mu=0, sigma=1, dims="groups")
alpha = pm.Deterministic(
"alpha", mu_alpha + sigma_alpha * alpha_offset, dims="groups"
)
# Observation level
sigma = pm.HalfNormal("sigma", sigma=1)
y = pm.Normal("y", mu=alpha[group_idx], sigma=sigma, observed=y_obs)
idata_hier = pm.sample(2000, tune=1000, target_accept=0.95, random_seed=42)
print(az.summary(idata_hier, var_names=["mu_alpha", "sigma_alpha", "alpha", "sigma"]))
Recipe: Logistic Regression
When to use: binary outcome variable (success/failure, disease/healthy). Use logit_p for numerical stability instead of computing probabilities directly.
import pymc as pm
import arviz as az
import numpy as np
# Simulated binary outcome
np.random.seed(42)
n, p = 200, 3
X_lr = np.random.randn(n, p)
true_beta = np.array([1.0, -0.5, 0.3])
prob = 1 / (1 + np.exp(-(0.2 + X_lr @ true_beta)))
y_binary = np.random.binomial(1, prob)
with pm.Model() as logistic_model:
alpha = pm.Normal("alpha", mu=0, sigma=2)
beta = pm.Normal("beta", mu=0, sigma=2, shape=p)
logit_p = alpha + pm.math.dot(X_lr, beta)
y = pm.Bernoulli("y", logit_p=logit_p, observed=y_binary)
idata_logit = pm.sample(2000, tune=1000, target_accept=0.9, random_seed=42)
summary = az.summary(idata_logit, var_names=["alpha", "beta"])
print(summary[["mean", "sd", "hdi_3%", "hdi_97%"]])
# Odds ratios
beta_samples = idata_logit.posterior["beta"].values.reshape(-1, p)
print(f"Odds ratios (median): {np.median(np.exp(beta_samples), axis=0)}")
Recipe: Gaussian Process
When to use: modeling unknown nonlinear functions, spatial data, or smooth latent processes. Computationally expensive for n > 1000; consider sparse approximations for larger datasets.
import pymc as pm
import arviz as az
import numpy as np
# Simulated nonlinear data
np.random.seed(42)
X_gp = np.sort(np.random.uniform(0, 10, 60))[:, None]
y_gp = np.sin(X_gp[:, 0]) + np.random.randn(60) * 0.3
with pm.Model() as gp_model:
# GP hyperparameters
ls = pm.Gamma("lengthscale", alpha=2, beta=1)
eta = pm.HalfNormal("amplitude", sigma=2)
sigma_noise = pm.HalfNormal("noise", sigma=0.5)
# Covariance function
cov = eta**2 * pm.gp.cov.Matern52(1, ls=ls)
gp = pm.gp.Marginal(cov_func=cov)
# Marginal likelihood
y_ = gp.marginal_likelihood("y", X=X_gp, y=y_gp, sigma=sigma_noise)
idata_gp = pm.sample(1000, tune=1000, target_accept=0.9, random_seed=42)
# Predict on new points
X_new_gp = np.linspace(0, 10, 100)[:, None]
with gp_model:
f_pred = gp.conditional("f_pred", X_new_gp)
pred_samples = pm.sample_posterior_predictive(idata_gp, var_names=["f_pred"])
print(f"GP predictions shape: {pred_samples.posterior_predictive['f_pred'].shape}")
Expected Outputs
- Trace plots (
az.plot_trace) -- chain mixing visualization; healthy chains look like "fuzzy caterpillars" - Posterior summary (
az.summary) -- mean, SD, HDI, R-hat, ESS per parameter - Prior/posterior predictive plots (
az.plot_ppc) -- simulated vs observed data distributions - Forest plots (
az.plot_forest) -- coefficient estimates with credible intervals - Model comparison table (
az.compare) -- ranked models with LOO/WAIC, weights, warnings - Rank plots (
az.plot_rank) -- uniform histograms indicate good mixing - Energy plot (
az.plot_energy) -- HMC energy transition diagnostics - InferenceData file (
idata.to_netcdf("results.nc")) -- serialized results for later analysis
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| Divergent transitions | Posterior geometry difficult for NUTS (funnels, ridges) | Increase target_accept=0.95; use non-centered parameterization; add stronger priors |
| Low ESS (< 400) | High autocorrelation between samples | Increase draws=5000; reparameterize to reduce correlation; try QR decomposition for correlated predictors |
| R-hat > 1.01 | Chains have not converged | Increase tune=2000, draws=5000; check for multimodality; initialize with ADVI |
| Slow sampling | Complex posterior or large dataset | Use ADVI for initialization; reduce model complexity; increase cores; try JAX backend |
| Pareto-k > 0.7 in LOO | Influential observations affecting LOO estimate | Use WAIC instead; investigate influential points; consider k-fold CV |
| Hit max tree depth | Model geometry requires long trajectories | Reparameterize model; increase max_treedepth parameter |
SamplingError at start | Poor initialization | Try init="adapt_diag" or init="jitter+adapt_diag"; provide initvals via MAP |
| Biased posterior (prior dominates) | Priors too strong relative to data | Weaken priors (increase sigma); check prior predictive; verify data is correctly passed |
ValueError: logp = -inf | Parameter hit impossible region | Check data for NaN/Inf; ensure positive params use HalfNormal/Gamma; verify likelihood matches data type |
Best Practices
- Use weakly informative priors, not flat priors. Flat priors (Uniform over large ranges) cause sampling difficulties and are rarely appropriate.
- Always run prior predictive checks before fitting. If prior-implied data is implausible, adjust priors before sampling.
- Standardize continuous predictors to mean=0, sd=1 for better NUTS geometry and faster sampling.
- Use non-centered parameterization for hierarchical models:
offset ~ Normal(0,1); param = mu + sigma * offsetinstead ofparam ~ Normal(mu, sigma). - Run at least 4 chains for reliable R-hat and ESS diagnostics. Never trust a single chain.
- Check all three diagnostics (R-hat < 1.01, ESS > 400, 0 divergences) before interpreting posteriors.
- Include
log_likelihood=Truewhen fitting if you plan to compare models via LOO or WAIC. - Use
dimsinstead ofshapefor named dimensions -- integrates with ArviZ for labeled summaries and subsetting. - Report credible intervals (HDI), not just point estimates. The posterior distribution is the result, not a single number.
- Save results with
idata.to_netcdf("results.nc")for reproducibility and later re-analysis.
Bundled Resources
Two reference files provide deeper detail for on-demand consultation:
-
references/distributions_inference.md-- Distribution catalog organized by category (continuous, discrete, multivariate, mixture, time series, special/modifiers) with full parameter signatures, support, and common uses. Sampling and inference methods (NUTS, Metropolis, Slice, CompoundStep, SMC, ADVI, fullrank ADVI, SVGD, MAP) with code examples and a method selection guide table. Reparameterization tricks (non-centered, QR decomposition) with code. Covers: consolidated from originalreferences/distributions.md(320 lines) andreferences/sampling_inference.md(424 lines). Relocated inline: core distribution selection guide table (Key Concepts), diagnostic threshold table (Key Concepts), basic ADVI usage (Recipe: Variational Inference in original, covered in references here). Omitted: shape broadcasting examples (trivial NumPy-styleshape=5), basicdimsusage (covered in Workflow Step 2), prior predictive sampling (covered in Workflow Step 3). -
references/advanced_workflows.md-- Model comparison workflow (LOO, WAIC, Pareto-k reliability checks, model averaging with weighted predictions), comprehensive diagnostic report generation (trace plots, rank plots, autocorrelation, energy, ESS evolution), prior-posterior comparison patterns, data preparation best practices (standardization, centering, missing data imputation), prior selection guidelines (weakly informative vs informative with domain knowledge), named dimensions (dims) usage with xarray subsetting, save/load patterns (NetCDF, pickle). Mixture model pattern. Covers: consolidated from originalreferences/workflows.md(526 lines). Relocated inline: core diagnostic checking logic (Workflow Step 5), model comparison basics (Workflow Step 7), prior predictive check (Workflow Step 3), linear/logistic/hierarchical model code (Recipes). Omitted: complete monolithic workflow template (redundant with 8-step Workflow section).
Per-reference-file disposition (all originals)
Original references/ (3 files):
distributions.md(320 lines) -> consolidated withsampling_inference.mdinto newreferences/distributions_inference.mdsampling_inference.md(424 lines) -> consolidated withdistributions.mdinto newreferences/distributions_inference.mdworkflows.md(526 lines) -> migrated as newreferences/advanced_workflows.md
Original scripts/ (2 files):
model_diagnostics.py(350 lines) ->check_diagnostics()logic (R-hat, ESS, divergences, tree depth checks) inlined in Workflow Step 5;create_diagnostic_report()plot generation patterns described in Expected Outputs and referenced inreferences/advanced_workflows.md;compare_prior_posterior()utility covered inreferences/advanced_workflows.mdmodel_comparison.py(387 lines) ->compare_models()andcheck_loo_reliability()patterns inlined in Workflow Step 7 (usingaz.compareandaz.loodirectly);model_averaging()pattern covered inreferences/advanced_workflows.md;cross_validation_comparison()guidance covered inreferences/advanced_workflows.md;plot_model_comparison()is a thin wrapper aroundaz.plot_compare(inlined in Step 7)
Related Skills
- arviz (planned) -- posterior visualization and diagnostics; PyMC returns ArviZ InferenceData objects
- bambi (planned) -- formula-based interface to PyMC (R-style
y ~ x1 + x2syntax); simpler API for standard GLMs - numpyro (planned) -- JAX-based probabilistic programming; faster NUTS via hardware acceleration
- statsmodels-statistical-modeling -- frequentist regression and GLMs; use when Bayesian framework is not needed
- scikit-learn-machine-learning -- ML classification/regression; use when prediction accuracy matters more than uncertainty quantification
References
- PyMC documentation -- official API reference and tutorials
- PyMC GitHub -- source code and issue tracker
- ArviZ documentation -- posterior analysis and visualization
- Bayesian Modeling and Computation in Python -- textbook using PyMC
- PyMC examples gallery -- curated notebook collection
scikit-survival -- Survival Analysis
Overview
scikit-survival is a Python library for time-to-event analysis built on scikit-learn. It handles right-censored data (observations where the event has not yet occurred) using Cox models, ensemble methods, survival SVMs, and non-parametric estimators. All models follow the scikit-learn fit/predict API and integrate with Pipelines, cross-validation, and GridSearchCV.
When to Use
- Modeling time-to-event outcomes with right-censored data (clinical trials, reliability)
- Fitting Cox proportional hazards models (standard or elastic net penalized)
- Building ensemble survival models (Random Survival Forest, Gradient Boosting)
- Training survival SVMs for margin-based learning on medium-sized datasets
- Evaluating survival predictions with censoring-aware metrics (C-index, Brier score, AUC)
- Estimating non-parametric survival curves (Kaplan-Meier, Nelson-Aalen)
- Analyzing competing risks with cumulative incidence functions
- High-dimensional survival data with automatic feature selection (CoxNet L1/L2)
- For simpler parametric models (Weibull, log-normal AFT) or statistical tests (log-rank), use
lifelines - For deep learning survival models, use
pycoxortorchlife
Prerequisites
pip install scikit-survival scikit-learn pandas numpy matplotlib
Python: >= 3.9. Dependencies: scikit-learn, numpy, scipy, pandas, joblib, osqp (for some SVM solvers).
Data format: Survival outcomes are NumPy structured arrays with (event, time) fields. Events are boolean (True = event occurred, False = censored). Times are positive floats.
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: timeAndEventColumns
kind: required
source: data
ask: "Which column holds follow-up time, which holds the event indicator, and does the indicator mark the event or censoring?"
default: null
- id: D2
param: eventDefinition
kind: required
source: user
depends_on: [D1]
ask: "Which event is being modelled, and are competing events treated as censoring?"
default: null
- id: D3
param: covariates
kind: required
source: data
ask: "Which variables enter the model, and are any measured after baseline?"
default: null
- id: D4
param: modelFamily
kind: required
source: user
ask: "A proportional-hazards model whose coefficients are interpretable, or an ensemble that predicts better but does not yield hazard ratios?"
default: "Cox proportional hazards"
- id: D5
param: regularization
kind: required
source: user
depends_on: [D3, D4]
ask: "With more covariates than events, should coefficients be penalized - and toward selection or toward shrinkage?"
default: "none"
- id: D6
param: validationScheme
kind: required
source: user
ask: "How should performance be estimated - cross-validation, a held-out split, or an external cohort?"
default: "cross-validation"
- id: D7
param: ensembleHyperparameters
kind: optional_conditional
source: user
depends_on: [D4]
ask: "How many trees, how deep, and at what learning rate?"
default: "100 estimators, depth 3, learning rate 0.1"
skip_if: "a proportional-hazards model was chosen"
D1 and D2 are asked before anything else because a flipped event indicator fits a model of the time to not having the event, and every hazard ratio comes back inverted without any warning. D3 carries the other silent error: a covariate measured after baseline leaks the outcome into the predictors and produces an impressive concordance index that will not replicate.
Quick Start
from sksurv.datasets import load_breast_cancer
from sksurv.ensemble import RandomSurvivalForest
from sksurv.metrics import concordance_index_ipcw
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
rsf = RandomSurvivalForest(n_estimators=100, random_state=42)
rsf.fit(X_train, y_train)
risk_scores = rsf.predict(X_test)
c_index = concordance_index_ipcw(y_train, y_test, risk_scores)[0]
print(f"C-index: {c_index:.3f}") # e.g., 0.68
# Individual survival curves
surv_fns = rsf.predict_survival_function(X_test[:2])
for fn in surv_fns:
print(f"5-year survival: {fn(365 * 5):.3f}")
Core API
Module 1: Data Preparation
Create structured survival arrays and preprocess features.
import numpy as np
import pandas as pd
from sksurv.util import Surv
from sksurv.preprocessing import OneHotEncoder, encode_categorical
from sksurv.datasets import load_gbsg2, load_breast_cancer
from sklearn.preprocessing import StandardScaler
# Create survival outcome from arrays
event = np.array([True, False, True, True, False])
time = np.array([120.0, 365.0, 200.0, 90.0, 400.0])
y = Surv.from_arrays(event=event, time=time)
print(y.dtype) # [('event', '?'), ('time', '<f8')]
# From DataFrame columns
# y = Surv.from_dataframe("event_col", "time_col", df)
# Load built-in datasets
# Available: load_gbsg2, load_breast_cancer, load_veterans_lung_cancer,
# load_whas500, load_aids, load_flchain
X, y = load_gbsg2()
print(f"Shape: {X.shape}, Events: {y['event'].sum()}, "
f"Censoring rate: {1 - y['event'].mean():.1%}")
# Encode categoricals (survival-aware one-hot)
X_encoded = encode_categorical(X) # auto-detect and encode all categorical cols
# Standardize (critical for Cox and SVM models)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X_encoded)
from sksurv.io import loadarff
# Load ARFF format (Weka format)
data = loadarff("survival_data.arff")
X_arff, y_arff = data[0], data[1] # DataFrame, structured array
Module 2: Cox Proportional Hazards
Semi-parametric model: h(t|x) = h_0(t) * exp(beta^T x). Interpretable coefficients as log hazard ratios.
from sksurv.linear_model import CoxPHSurvivalAnalysis, CoxnetSurvivalAnalysis, IPCRidge
# Standard Cox PH model
cox = CoxPHSurvivalAnalysis(alpha=0.0, ties="breslow")
cox.fit(X_train, y_train)
print(f"Coefficients: {cox.coef_}") # log hazard ratios
# Hazard ratio interpretation: exp(coef) = HR for 1-unit increase
risk_scores = cox.predict(X_test) # Higher = higher risk
# Survival function for individual patients
surv_funcs = cox.predict_survival_function(X_test[:3])
for fn in surv_funcs:
print(f"5-year survival: {fn(365 * 5):.3f}")
# Penalized Cox (elastic net) -- for high-dimensional data (p > n)
coxnet = CoxnetSurvivalAnalysis(
l1_ratio=0.9, # 0=Ridge, 1=Lasso, between=Elastic Net
alpha_min_ratio=0.01, # smallest alpha / largest alpha ratio
n_alphas=100, # steps in regularization path
)
coxnet.fit(X_train, y_train)
# Feature selection: non-zero coefficients
selected = np.where(coxnet.coef_ != 0)[0]
print(f"Selected {len(selected)} / {X_train.shape[1]} features")
# IPCRidge: accelerated failure time model (predicts log survival time)
ipcridge = IPCRidge(alpha=1.0)
ipcridge.fit(X_train, y_train)
log_survival_time = ipcridge.predict(X_test)
Module 3: Ensemble Methods
Non-parametric tree-based models for complex non-linear relationships.
from sksurv.ensemble import (
RandomSurvivalForest,
GradientBoostingSurvivalAnalysis,
ComponentwiseGradientBoostingSurvivalAnalysis,
ExtraSurvivalTrees,
)
# Random Survival Forest -- robust, minimal tuning
rsf = RandomSurvivalForest(
n_estimators=200, min_samples_split=10, min_samples_leaf=15,
max_features="sqrt", random_state=42, n_jobs=-1,
)
rsf.fit(X_train, y_train)
risk = rsf.predict(X_test)
# Gradient Boosting -- best performance, needs tuning
gbs = GradientBoostingSurvivalAnalysis(
loss="coxph", # "coxph" or "ipcwls" (AFT)
n_estimators=300, learning_rate=0.05, max_depth=3,
subsample=0.8, dropout_rate=0.1, random_state=42,
)
gbs.fit(X_train, y_train)
# ComponentwiseGB -- linear model with automatic feature selection
cgbs = ComponentwiseGradientBoostingSurvivalAnalysis(
n_estimators=100, learning_rate=0.1,
)
cgbs.fit(X_train, y_train)
print(f"Non-zero coefficients: {np.sum(cgbs.coef_ != 0)}")
# ExtraSurvivalTrees -- more regularized than RSF, faster training
est = ExtraSurvivalTrees(n_estimators=100, random_state=42)
est.fit(X_train, y_train)
# Survival curves from any ensemble model
surv_funcs = rsf.predict_survival_function(X_test[:1])
chf_funcs = rsf.predict_cumulative_hazard_function(X_test[:1])
Module 4: Survival SVMs
Margin-based learning for survival ranking. Always standardize features.
from sksurv.svm import FastSurvivalSVM, FastKernelSurvivalSVM, HingeLossSurvivalSVM
# Linear SVM -- fast, for linear relationships
lsvm = FastSurvivalSVM(alpha=1.0, rank_ratio=1.0, max_iter=100, random_state=42)
lsvm.fit(X_train_scaled, y_train)
risk = lsvm.predict(X_test_scaled)
# Kernel SVM -- for non-linear relationships (rbf, poly, sigmoid)
ksvm = FastKernelSurvivalSVM(
alpha=1.0, kernel="rbf", gamma="scale",
max_iter=50, random_state=42,
)
ksvm.fit(X_train_scaled, y_train)
# Hinge loss variant
hsvm = HingeLossSurvivalSVM(alpha=1.0, random_state=42)
hsvm.fit(X_train_scaled, y_train)
# NaiveSurvivalSVM also available but slower (O(n^3))
from sksurv.kernels import ClinicalKernelTransform
# Clinical kernel: combines clinical + molecular features
# Weighs clinical variables separately from high-dimensional molecular data
transform = ClinicalKernelTransform(fit_once=True)
transform.prepare(X_train) # auto-detect clinical features
X_kern = transform.fit_transform(X_train)
Module 5: Non-Parametric Estimation
Estimate survival and hazard curves without model assumptions.
from sksurv.nonparametric import kaplan_meier_estimator, nelson_aalen_estimator
import matplotlib.pyplot as plt
# Kaplan-Meier survival curve
time_km, surv_prob = kaplan_meier_estimator(y["event"], y["time"])
plt.step(time_km, surv_prob, where="post")
plt.xlabel("Time (days)")
plt.ylabel("Survival probability")
plt.title("Kaplan-Meier Estimate")
plt.savefig("km_curve.png", dpi=150)
# With confidence intervals
time_km, surv_prob, conf_int = kaplan_meier_estimator(
y["event"], y["time"], conf_type="log-log",
)
# Nelson-Aalen cumulative hazard
time_na, cum_hazard = nelson_aalen_estimator(y["event"], y["time"])
# Stratified KM by group
for group_name, mask in [("Treated", treated_mask), ("Control", control_mask)]:
t, s = kaplan_meier_estimator(y[mask]["event"], y[mask]["time"])
plt.step(t, s, where="post", label=group_name)
plt.legend()
Module 6: Evaluation Metrics
Censoring-aware metrics for discrimination and calibration.
from sksurv.metrics import (
concordance_index_censored,
concordance_index_ipcw,
cumulative_dynamic_auc,
integrated_brier_score,
brier_score,
as_concordance_index_ipcw_scorer,
as_integrated_brier_score_scorer,
)
import numpy as np
risk_scores = model.predict(X_test)
# Harrell's C-index (simple, biased with high censoring)
c_harrell = concordance_index_censored(
y_test["event"], y_test["time"], risk_scores
)[0]
# Uno's C-index (recommended -- robust to censoring)
c_uno = concordance_index_ipcw(y_train, y_test, risk_scores)[0]
print(f"C-index (Harrell): {c_harrell:.3f}, (Uno): {c_uno:.3f}")
# Time-dependent AUC at clinically relevant timepoints
times = np.array([365, 730, 1095]) # 1, 2, 3 years
auc, mean_auc = cumulative_dynamic_auc(y_train, y_test, risk_scores, times)
print(f"AUC at 1/2/3yr: {auc}, Mean: {mean_auc:.3f}")
# Integrated Brier Score (measures discrimination + calibration)
surv_funcs = model.predict_survival_function(X_test)
preds = np.row_stack([fn(times) for fn in surv_funcs])
ibs = integrated_brier_score(y_train, y_test, preds, times)
print(f"IBS: {ibs:.3f}") # Lower is better; compare vs KM baseline
Module 7: Competing Risks
Multiple mutually exclusive event types (e.g., death from cancer vs cardiovascular).
from sksurv.nonparametric import cumulative_incidence_competing_risks
from sksurv.linear_model import CoxPHSurvivalAnalysis
from sksurv.util import Surv
import numpy as np
# Cumulative Incidence Function (CIF) estimation
# y_competing: structured array with integer event codes
# (0=censored, 1=relapse, 2=death_in_remission)
time_pts, cif_relapse, cif_death = cumulative_incidence_competing_risks(y_competing)
import matplotlib.pyplot as plt
plt.step(time_pts, cif_relapse, where="post", label="Relapse")
plt.step(time_pts, cif_death, where="post", label="Death in remission")
plt.xlabel("Time")
plt.ylabel("Cumulative Incidence")
plt.legend()
# Cause-specific hazard models: fit separate Cox per event type
event_types = np.array([0, 1, 2, 1, 0, 2, 1])
times = np.array([10.2, 5.3, 8.1, 3.7, 12.5, 6.8, 4.2])
# Model for relapse (event type 1); treat type 2 as censored
y_relapse = Surv.from_arrays(event=(event_types == 1), time=times)
cox_relapse = CoxPHSurvivalAnalysis()
cox_relapse.fit(X, y_relapse)
# Model for death (event type 2); treat type 1 as censored
y_death = Surv.from_arrays(event=(event_types == 2), time=times)
cox_death = CoxPHSurvivalAnalysis()
cox_death.fit(X, y_death)
print(f"Relapse HR (age): {np.exp(cox_relapse.coef_[0]):.3f}")
print(f"Death HR (age): {np.exp(cox_death.coef_[0]):.3f}")
Key Concepts
Model Selection Guide
High-dimensional (p > n)?
├── Yes → CoxnetSurvivalAnalysis (elastic net)
└── No
├── Need interpretable coefficients?
│ ├── Yes → CoxPHSurvivalAnalysis or ComponentwiseGB
│ └── No
│ ├── Large data (n > 1000) → GradientBoostingSurvivalAnalysis
│ ├── Medium data → RandomSurvivalForest or FastKernelSurvivalSVM
│ └── Small data → RandomSurvivalForest
└── For max performance → compare multiple models
Censoring Types
scikit-survival primarily handles right-censoring (event not observed by end of study). Left-censoring and interval-censoring require other tools (e.g., lifelines). Censoring rate affects metric choice: >40% censoring favors Uno's C-index over Harrell's.
Concordance Index Interpretation
- 0.5 = random prediction (no discrimination)
- 0.6-0.7 = moderate discrimination
- 0.7-0.8 = good discrimination (typical for clinical models)
- 0.8+ = excellent (rare for clinical survival data)
- C-index measures ranking ability, not calibration. Always pair with Brier score.
Evaluation Metric Decision Table
| Metric | Measures | Best For |
|---|---|---|
| Harrell's C-index | Ranking only | Quick development, low censoring |
| Uno's C-index | Ranking (censoring-adjusted) | Publication, any censoring rate |
| Time-dependent AUC | Discrimination at fixed t | Clinical decision at specific horizons |
| Brier Score (single t) | Calibration at fixed t | Probability accuracy at timepoint |
| Integrated Brier Score | Overall calibration | Comprehensive model assessment |
Common Workflows
Workflow 1: Standard Survival Pipeline
Goal: End-to-end analysis with preprocessing, model fitting, and evaluation.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import GridSearchCV, train_test_split
from sksurv.datasets import load_breast_cancer
from sksurv.ensemble import GradientBoostingSurvivalAnalysis
from sksurv.metrics import (
concordance_index_ipcw, cumulative_dynamic_auc,
integrated_brier_score, as_concordance_index_ipcw_scorer,
)
import numpy as np
X, y = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Build pipeline
pipe = Pipeline([
("scaler", StandardScaler()),
("model", GradientBoostingSurvivalAnalysis(random_state=42)),
])
# Hyperparameter search
param_grid = {
"model__learning_rate": [0.01, 0.05, 0.1],
"model__n_estimators": [100, 200],
"model__max_depth": [3, 5],
}
cv = GridSearchCV(pipe, param_grid, cv=5,
scoring=as_concordance_index_ipcw_scorer(), n_jobs=-1)
cv.fit(X_train, y_train)
print(f"Best CV C-index: {cv.best_score_:.3f}, Params: {cv.best_params_}")
# Final evaluation on test set
best = cv.best_estimator_
risk = best.predict(X_test)
c_uno = concordance_index_ipcw(y_train, y_test, risk)[0]
times = np.percentile(y_test["time"][y_test["event"]], [25, 50, 75])
auc, mean_auc = cumulative_dynamic_auc(y_train, y_test, risk, times)
print(f"Test C-index: {c_uno:.3f}, Mean AUC: {mean_auc:.3f}")
Workflow 2: Model Comparison
Goal: Compare multiple model families and select the best.
from sksurv.linear_model import CoxPHSurvivalAnalysis
from sksurv.ensemble import RandomSurvivalForest, GradientBoostingSurvivalAnalysis
from sksurv.svm import FastSurvivalSVM
from sksurv.metrics import concordance_index_ipcw, integrated_brier_score
from sklearn.preprocessing import StandardScaler
import numpy as np
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)
models = {
"CoxPH": CoxPHSurvivalAnalysis(),
"RSF": RandomSurvivalForest(n_estimators=200, random_state=42),
"GBS": GradientBoostingSurvivalAnalysis(n_estimators=200, random_state=42),
"LinearSVM": FastSurvivalSVM(alpha=1.0, random_state=42),
}
times = np.percentile(y_test["time"][y_test["event"]], [25, 50, 75])
for name, model in models.items():
X_tr = X_train_s if name in ("CoxPH", "LinearSVM") else X_train
X_te = X_test_s if name in ("CoxPH", "LinearSVM") else X_test
model.fit(X_tr, y_train)
risk = model.predict(X_te)
c = concordance_index_ipcw(y_train, y_test, risk)[0]
print(f"{name:12s}: C-index = {c:.3f}")
Workflow 3: High-Dimensional Feature Selection
- Fit CoxnetSurvivalAnalysis with l1_ratio=0.9 (Lasso-dominant)
- Use GridSearchCV over alpha_min_ratio values with C-index scorer
- Extract non-zero coefficients from best model
- Refit standard CoxPH or RSF on selected features for final model
- Evaluate on held-out test set with Uno's C-index
Key Parameters
| Parameter | Module | Default | Range | Effect |
|---|---|---|---|---|
alpha | CoxPH | 0.0 | >= 0 | Regularization (0 = none) |
l1_ratio | CoxNet | 0.5 | 0-1 | L1 vs L2 penalty (1=Lasso, 0=Ridge) |
n_estimators | RSF, GBS | 100 | 50-1000 | Number of trees/iterations |
max_depth | RSF, GBS | None/3 | 1-20 | Tree depth (GBS: 3-5 recommended) |
learning_rate | GBS | 0.1 | 0.001-0.3 | Step size shrinkage per iteration |
subsample | GBS | 1.0 | 0.5-1.0 | Fraction of samples per tree |
dropout_rate | GBS | 0.0 | 0-0.3 | Drop previous learners during training |
max_features | RSF | None | "sqrt","log2",int | Features per split |
alpha | FastSurvivalSVM | 1.0 | 0.01-100 | SVM regularization |
kernel | FastKernelSVM | "rbf" | "linear","rbf","poly","sigmoid" | Kernel function |
gamma | FastKernelSVM | "scale" | "scale","auto",float | Kernel coefficient |
Best Practices
-
Always standardize features for Cox and SVM models: Coefficients and margins are scale-dependent. Use
StandardScalerinside a Pipeline to prevent leakage. -
Use Uno's C-index over Harrell's: More robust to censoring distribution differences; required for comparing models across datasets. Pass
y_trainas the first argument. -
Anti-pattern -- using built-in RSF feature importance: It is biased toward continuous and high-cardinality features. Use
sklearn.inspection.permutation_importance()instead. -
Report multiple metrics: C-index alone measures only discrimination. Add Brier score for calibration and time-dependent AUC for time-specific performance.
-
Anti-pattern -- ignoring proportional hazards assumption: Cox models assume constant hazard ratios over time. Validate with Schoenfeld residuals (available in lifelines) or use non-parametric alternatives (RSF, GBS).
-
Ensure sufficient events per feature: Rule of thumb: 10+ events per feature for Cox. With fewer events, use CoxNet (l1_ratio=0.9) or ensemble methods.
-
Use scikit-learn Pipelines: Wrapping preprocessing + model prevents data leakage and simplifies cross-validation and deployment.
-
Anti-pattern -- treating competing events as censored for KM: Kaplan-Meier overestimates event probability when competing risks exist. Use
cumulative_incidence_competing_risks()instead. -
GBS early stopping: For GradientBoosting, set
n_estimators=1000withn_iter_no_change=10to automatically stop when validation performance plateaus.
Common Recipes
Recipe: Hyperparameter Tuning with Cross-Validation
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sksurv.ensemble import RandomSurvivalForest
from sksurv.metrics import as_concordance_index_ipcw_scorer
pipe = Pipeline([
("scaler", StandardScaler()),
("model", RandomSurvivalForest(random_state=42)),
])
param_grid = {
"model__n_estimators": [100, 300, 500],
"model__min_samples_split": [6, 10, 20],
"model__max_depth": [None, 10, 20],
}
cv = GridSearchCV(pipe, param_grid, cv=5,
scoring=as_concordance_index_ipcw_scorer(), n_jobs=-1)
cv.fit(X, y)
print(f"Best: {cv.best_score_:.3f}, Params: {cv.best_params_}")
Recipe: Permutation Feature Importance
from sklearn.inspection import permutation_importance
from sksurv.metrics import as_concordance_index_ipcw_scorer
result = permutation_importance(
rsf, X_test, y_test, n_repeats=15,
scoring=as_concordance_index_ipcw_scorer(), random_state=42, n_jobs=-1,
)
sorted_idx = result.importances_mean.argsort()[::-1]
for i in sorted_idx[:10]:
print(f"{X.columns[i]:20s}: {result.importances_mean[i]:.4f} "
f"+/- {result.importances_std[i]:.4f}")
Recipe: Kaplan-Meier with Confidence Bands
from sksurv.nonparametric import kaplan_meier_estimator
import matplotlib.pyplot as plt
time, surv, conf_int = kaplan_meier_estimator(
y["event"], y["time"], conf_type="log-log",
)
plt.step(time, surv, where="post", label="KM estimate")
plt.fill_between(time, conf_int[0], conf_int[1], alpha=0.25, step="post")
plt.xlabel("Time")
plt.ylabel("Survival probability")
plt.legend()
plt.savefig("km_with_ci.png", dpi=150)
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
ValueError: y must be a structured array | Wrong outcome format | Use Surv.from_arrays(event, time) -- event must be bool, time must be float |
concordance_index_ipcw raises error | Missing y_train argument | IPCW requires training set for censoring distribution; pass y_train as first arg |
| Cox model won't converge | Correlated or unstandardized features | Standardize with StandardScaler; switch to CoxnetSurvivalAnalysis |
| Very low C-index (< 0.5) | Wrong sign or non-predictive features | Verify event/time coding; check feature engineering |
as_concordance_index_ipcw_scorer error in CV | Time range mismatch between folds | Ensure all folds contain events at evaluated timepoints |
| SVM extremely slow | Large dataset with kernel SVM | Use FastSurvivalSVM (linear, O(np)) for large n; subsample for kernel |
| RSF feature importance misleading | Built-in impurity importance biased | Use permutation_importance() from scikit-learn |
| Kaplan-Meier overestimates | Competing events treated as censoring | Use cumulative_incidence_competing_risks() instead |
Bundled Resources
references/models_evaluation.md
Covers: Detailed Cox model family (CoxPH, CoxNet, IPCRidge with parameter guidance and interpretation), ensemble methods (RSF, GBS, ComponentwiseGB, ExtraSurvivalTrees with hyperparameter tuning, early stopping, loss functions), SVM models (FastSurvivalSVM, FastKernelSurvivalSVM, HingeLossSurvivalSVM, NaiveSurvivalSVM, MinlipSurvivalAnalysis with kernel selection), evaluation metrics (Harrell vs Uno C-index selection, time-dependent AUC plotting, Brier score with null model comparison, comprehensive evaluation pipeline), model comparison tables.
Relocated inline: Core usage patterns for all model families consolidated into Core API Modules 2-4 and Module 6. Model selection decision tree relocated to Key Concepts.
Omitted: CoxNet cross-validation alpha selection via scoring string (not a valid sklearn scorer usage -- use as_concordance_index_ipcw_scorer() as shown in Core API). NaiveSurvivalSVM detailed usage (slow, primarily for benchmarking). MinlipSurvivalAnalysis detailed usage (research-only). Redundant "when to use" prose duplicated across original reference files.
Consolidates 4 originals: cox-models.md (182 lines), ensemble-models.md (327 lines), svm-models.md (411 lines), evaluation-metrics.md (378 lines). Total: 1,298 lines.
references/data_competing_risks.md
Covers: Complete data preprocessing workflows (Surv arrays, loading custom CSV, ARFF I/O, categorical encoding methods, standardization, missing data imputation strategies, feature selection), data validation checks (negative times, censoring rate, events-per-feature), train-test splitting (random and stratified by event/time), complete preprocessing pipeline with sklearn, competing risks analysis (CIF estimation, data format with event types, stratified group comparison, cause-specific hazard modeling, complete practical example with simulated data).
Relocated inline: Core Surv array creation, encode_categorical, StandardScaler usage consolidated into Core API Module 1. CIF estimation and cause-specific Cox models relocated to Core API Module 7.
Omitted: Custom preprocessing function with docstring (duplicates sklearn Pipeline pattern shown in Workflow 1). Time-varying covariates note (scikit-survival does not support them -- mentioned only as a 1-line note in original). Fine-Gray sub-distribution model (not available in scikit-survival; noted as lifelines alternative).
Consolidates 2 originals: data-handling.md (494 lines), competing-risks.md (397 lines). Total: 891 lines.
Related Skills
- scikit-learn-machine-learning -- general ML with same Pipeline/GridSearchCV API; non-survival tasks
- statsmodels-statistical-modeling -- frequentist regression (OLS, GLM); non-censored outcomes
- pymc-bayesian-modeling -- Bayesian survival models with prior specification
- matplotlib-scientific-plotting -- customize Kaplan-Meier curves and survival plots
- statistical-analysis -- test selection framework and reporting guidelines
References
- scikit-survival documentation -- official API reference
- scikit-survival GitHub -- source code and examples
- API reference -- complete class/function list
- Polsterl (2020) "scikit-survival: A Library for Time-to-Event Analysis Built on Top of scikit-learn" -- JMLR 21(212):1-6
Statistical Analysis
Overview
Statistical analysis is the systematic process of selecting appropriate tests, verifying assumptions, quantifying effect magnitudes, and reporting results. This knowhow guides test selection, assumption diagnostics, and APA-style reporting for frequentist and Bayesian analyses in academic research.
Key Concepts
Frequentist vs Bayesian Framework
| Aspect | Frequentist | Bayesian |
|---|---|---|
| Core output | p-value, confidence interval | Posterior distribution, credible interval |
| Interpretation | "How likely is this data if H0 is true?" | "How likely is H1 given the data?" |
| Null support | Cannot support H0 (only fail to reject) | Can quantify evidence for H0 via Bayes Factor |
| Prior info | Not used | Incorporated via prior distributions |
| Sample size | Requires adequate power | Works with any sample size |
| Best for | Standard analyses, large samples | Small samples, prior info, complex models |
Statistical vs Practical Significance
A statistically significant result (p < .05) may be trivially small in practice. Always report:
- Effect size: Magnitude of the effect (Cohen's d, eta-squared, r, R-squared)
- Confidence interval: Precision of the estimate
- Context: Clinical/practical relevance in the domain
Common Effect Sizes
| Test | Effect Size | Small | Medium | Large |
|---|---|---|---|---|
| t-test | Cohen's d | 0.20 | 0.50 | 0.80 |
| t-test (small n) | Hedges' g | 0.20 | 0.50 | 0.80 |
| ANOVA | eta-squared partial | 0.01 | 0.06 | 0.14 |
| ANOVA | omega-squared | 0.01 | 0.06 | 0.14 |
| Correlation | r | 0.10 | 0.30 | 0.50 |
| Regression | R-squared | 0.02 | 0.13 | 0.26 |
| Regression | f-squared | 0.02 | 0.15 | 0.35 |
| Chi-square | Cramer's V | 0.07 | 0.21 | 0.35 |
| Chi-square 2x2 | phi coefficient | 0.10 | 0.30 | 0.50 |
Cohen's benchmarks are guidelines, not rigid thresholds -- domain context always matters.
Assumptions Overview
Most parametric tests require:
- Independence: Observations are independent of each other
- Normality: Data (or residuals) are approximately normally distributed
- Homogeneity of variance: Groups have similar variances (for group comparisons)
- Linearity: Relationship between variables is linear (for regression)
When assumptions are violated:
- Normality violated, n > 30: Proceed -- parametric tests are robust with large samples
- Normality violated, n < 30: Use non-parametric alternative
- Variance heterogeneity: Use Welch's correction (t-test) or Welch's ANOVA
- Linearity violated: Add polynomial terms, transform variables, or use GAMs
Test-Specific Assumption Workflows
T-test assumptions: (1) Check normality per group with Shapiro-Wilk + Q-Q plots. (2) Check homogeneity with Levene's test. (3) If normality violated: Mann-Whitney U (independent) or Wilcoxon signed-rank (paired). If variance heterogeneity: use Welch's t-test.
ANOVA assumptions: (1) Normality per group. (2) Homogeneity via Levene's test. (3) For repeated measures: check sphericity (Mauchly's test); if violated, apply Greenhouse-Geisser (epsilon < 0.75) or Huynh-Feldt (epsilon > 0.75) correction. (4) If normality violated: Kruskal-Wallis (independent) or Friedman (repeated).
Linear regression assumptions: (1) Linearity via residuals-vs-fitted plot. (2) Independence via Durbin-Watson test (1.5-2.5 acceptable). (3) Homoscedasticity via Breusch-Pagan test + scale-location plot. (4) Normality of residuals via Q-Q plot + Shapiro-Wilk. (5) Multicollinearity via VIF (>10 = severe, >5 = moderate).
Logistic regression assumptions: (1) Independence. (2) Linearity of log-odds with continuous predictors (Box-Tidwell test). (3) No perfect multicollinearity (VIF). (4) Adequate sample size (10-20 events per predictor minimum).
Specialized Test Categories
Beyond the main decision flowchart, several specialized test families address specific data types:
Survival / time-to-event analysis:
- Log-rank test: Compares survival curves between groups (non-parametric)
- Cox proportional hazards: Models time-to-event with covariates; assumes proportional hazards
- Parametric survival models: Weibull, exponential, log-normal for known distributional forms
- Use when outcome is time until an event (death, relapse, failure) with possible censoring
Count outcome models:
- Poisson regression: For count data where mean approximately equals variance
- Negative binomial regression: For overdispersed counts (variance > mean)
- Zero-inflated models: For excess zeros beyond what Poisson/NB predicts
- Use when outcome is a count (number of events, incidents, occurrences)
Agreement and reliability:
- Cohen's kappa: Inter-rater agreement for categorical ratings (2 raters)
- Fleiss' kappa / Krippendorff's alpha: Agreement for >2 raters
- Intraclass correlation coefficient (ICC): Continuous ratings reliability
- Cronbach's alpha: Internal consistency of multi-item scales
- Bland-Altman analysis: Agreement between two measurement methods (continuous)
- Use when assessing measurement reliability or inter-rater consistency
Categorical data extensions:
- McNemar's test: Paired binary outcomes (2x2)
- Cochran's Q test: Paired binary outcomes (3+ conditions)
- Cochran-Armitage trend test: Ordered categories in contingency tables
Decision Framework
Test Selection Flowchart
What is your research question?
|
+-- Comparing GROUPS on a continuous outcome?
| |
| +-- How many groups?
| | +-- 2 groups
| | | +-- Independent -> Independent t-test (or Mann-Whitney U)
| | | +-- Paired/repeated -> Paired t-test (or Wilcoxon signed-rank)
| | +-- 3+ groups
| | +-- Independent -> One-way ANOVA (or Kruskal-Wallis)
| | +-- Repeated -> Repeated-measures ANOVA (or Friedman)
| |
| +-- Multiple factors? -> Factorial ANOVA / Mixed ANOVA
| +-- With covariates? -> ANCOVA
|
+-- Testing a RELATIONSHIP between variables?
| |
| +-- Both continuous?
| | +-- Normal -> Pearson correlation
| | +-- Non-normal or ordinal -> Spearman correlation
| |
| +-- Predicting continuous outcome?
| | +-- 1 predictor -> Simple linear regression
| | +-- Multiple predictors -> Multiple linear regression
| |
| +-- Predicting categorical outcome?
| | +-- Binary -> Logistic regression
| | +-- Ordinal -> Ordinal logistic regression
| |
| +-- Predicting count outcome?
| | +-- Equidispersed -> Poisson regression
| | +-- Overdispersed -> Negative binomial regression
| | +-- Excess zeros -> Zero-inflated Poisson/NB
| |
| +-- Time-to-event outcome?
| +-- Compare survival curves -> Log-rank test
| +-- With covariates -> Cox proportional hazards
|
+-- Testing ASSOCIATION between categorical variables?
| +-- Expected cell count >= 5 -> Chi-square test
| +-- Expected cell count < 5 -> Fisher's exact test
| +-- Ordered categories -> Cochran-Armitage trend test
| +-- Paired categories -> McNemar's test
|
+-- Assessing AGREEMENT / RELIABILITY?
+-- Categorical, 2 raters -> Cohen's kappa
+-- Categorical, >2 raters -> Fleiss' kappa
+-- Continuous ratings -> ICC
+-- Two measurement methods -> Bland-Altman analysis
+-- Internal consistency -> Cronbach's alpha
Quick Reference Table
| Research Question | Data Type | Normal? | Test | Non-parametric Alternative |
|---|---|---|---|---|
| 2 independent groups | Continuous | Yes | Independent t-test | Mann-Whitney U |
| 2 paired groups | Continuous | Yes | Paired t-test | Wilcoxon signed-rank |
| 3+ independent groups | Continuous | Yes | One-way ANOVA | Kruskal-Wallis |
| 3+ repeated groups | Continuous | Yes | Repeated-measures ANOVA | Friedman test |
| 2 variables | Continuous | Yes | Pearson r | Spearman rho |
| Predict continuous | Mixed | -- | Linear regression | -- |
| Predict binary | Mixed | -- | Logistic regression | -- |
| Predict counts | Count | -- | Poisson / Negative binomial | -- |
| Time-to-event | Survival | -- | Cox PH / Log-rank | -- |
| 2 categorical | Categorical | -- | Chi-square / Fisher's exact | -- |
| Rater agreement | Categorical | -- | Cohen's kappa / Fleiss' kappa | -- |
| Method agreement | Continuous | -- | Bland-Altman / ICC | -- |
Best Practices
- Pre-register analyses when possible to distinguish confirmatory from exploratory findings. Specify primary outcome, tests, and correction methods before data collection
- Always check assumptions before interpreting results. Run normality tests (Shapiro-Wilk), homogeneity tests (Levene's), and residual diagnostics. Document results even when assumptions are met
- Report effect sizes with confidence intervals for every test. p-values alone are insufficient -- effect sizes convey practical importance
- Report all planned analyses including non-significant findings. Selective reporting inflates false positive rates
- Use appropriate multiple comparison corrections. Bonferroni (conservative), Holm (step-down, less conservative), or FDR/Benjamini-Hochberg (for many tests). Choose based on the number of comparisons and acceptable error rate
- Visualize data before and after analysis. Box plots for group comparisons, scatter plots for correlations, residual plots for regression diagnostics
- Conduct sensitivity analyses to assess robustness: re-run with outliers removed, different transformations, or alternative tests
- Anti-pattern -- p-hacking: Testing multiple outcomes, subgroups, or model specifications until p < .05 inflates false positives. Pre-register to avoid
- Anti-pattern -- HARKing (Hypothesizing After Results are Known): Presenting exploratory findings as confirmatory undermines scientific integrity
- Anti-pattern -- misinterpreting non-significance: Failure to reject H0 does not mean H0 is true. Use Bayesian methods or equivalence testing to support null
Common Pitfalls
-
Misinterpreting p-values as probability of the hypothesis being true. p-values measure P(data | H0), not P(H0 | data). How to avoid: Use precise language: "If the null hypothesis were true, the probability of observing data this extreme is p = ..."
-
Confusing statistical significance with practical importance. A large sample can make trivially small effects significant. How to avoid: Always report and interpret effect sizes alongside p-values
-
Running post-hoc power analysis after a non-significant result. Post-hoc power is a mathematical function of the p-value and adds no new information. How to avoid: Use sensitivity analysis instead -- determine what effect size the study could detect at 80% power
-
Ignoring assumption violations and proceeding with parametric tests. How to avoid: Run assumption checks systematically. Use Welch's corrections, non-parametric alternatives, or transformations when violated
-
Multiple comparisons without correction. Running 20 tests at alpha = .05 gives ~64% chance of at least one false positive. How to avoid: Apply Bonferroni, Holm, or FDR correction. Report both corrected and uncorrected p-values
-
Treating ordinal data as continuous. Likert scales are ordinal -- means and standard deviations assume equal intervals. How to avoid: Use non-parametric tests (Mann-Whitney, Kruskal-Wallis) or ordinal regression
-
Ignoring missing data patterns. Listwise deletion assumes MCAR, which is rarely true. How to avoid: Assess missingness mechanism (MCAR, MAR, MNAR). Use multiple imputation for MAR data
-
Confusing correlation with causation. Observational studies cannot establish causal relationships regardless of effect size. How to avoid: Use causal language only for experimental designs with random assignment
-
Not reporting non-significant results. Publication bias and file-drawer effect distort the literature. How to avoid: Report all pre-registered analyses. Consider registered reports
-
Using one-tailed tests to "improve" significance. One-tailed tests should be pre-specified based on strong directional hypotheses. How to avoid: Default to two-tailed. Only use one-tailed when justified a priori
Workflow
Standard Analysis Pipeline
-
Define research question and hypotheses
- State H0 and H1 explicitly
- Specify primary outcome and covariates
-
Select statistical test (use Decision Framework above)
- Match test to data type, design, and assumptions
- Plan multiple comparison corrections if needed
-
Conduct a priori power analysis
- Specify target effect size (from literature or clinical relevance)
- Set alpha = .05, power = .80 (minimum), determine required n
- Libraries:
statsmodels.stats.power,pingouin
-
Inspect and clean data
- Check for missing data patterns (MCAR/MAR/MNAR)
- Identify outliers (IQR method: Q1 - 1.5IQR / Q3 + 1.5IQR, or z-scores > 3)
- Verify variable types and coding
- The original
assumption_checks.pyscript provides automated normality, homogeneity, and outlier detection with visualization
-
Check assumptions (see Test-Specific Assumption Workflows above)
- Normality: Shapiro-Wilk test + Q-Q plots (visual primary for n > 50)
- Homogeneity: Levene's test + box plots
- Linearity: Residual plots (for regression)
- Sphericity: Mauchly's test (for repeated measures)
- Document results and remedial actions
-
Run primary analysis
- Execute planned test with appropriate library (scipy.stats, pingouin, statsmodels)
- Calculate effect size and confidence interval
- For Bayesian analyses: specify priors, run MCMC, check convergence (see
references/bayesian_statistics.md)
-
Conduct post-hoc and secondary analyses
- Post-hoc pairwise comparisons (Tukey HSD, Bonferroni)
- Sensitivity analyses (remove outliers, alternative methods)
- Exploratory analyses (clearly labeled)
-
Report results in APA format
- Descriptive statistics (M, SD, n per group)
- Test statistic, degrees of freedom, exact p-value
- Effect size with confidence interval
- See
references/reporting_standards.mdfor templates
Bundled Resources
-
references/effect_sizes_and_power.md-- Detailed guide to calculating, interpreting, and reporting effect sizes (Cohen's d, Hedges' g, Glass's delta, eta-squared, omega-squared, partial eta-squared, phi coefficient, standardized beta, f-squared, Cramer's V, odds ratio); a priori, sensitivity, and correlation power analysis with code examples. Condensed from 582-line original. -
references/bayesian_statistics.md-- Comprehensive Bayesian analysis guide: Bayes' theorem, prior specification, ROPE (Region of Practical Equivalence), prior sensitivity analysis, Bayesian t-test/ANOVA/correlation/regression, hierarchical models, model comparison (WAIC/LOO), convergence diagnostics. Condensed from 662-line original. -
references/reporting_standards.md-- APA-style reporting templates for t-tests, ANOVA, regression, correlation, chi-square, non-parametric, and Bayesian analyses; pre-registration guidance; methods section templates (participants, design, measures); null results reporting; reporting checklist. Condensed from 470-line original.
Fully-Consolidated Files (no separate reference file)
-
test_selection_guide.md(130 lines original) -- Fully consolidated into Decision Framework (flowchart + Quick Reference Table) and Specialized Test Categories subsection in Key Concepts. Combined coverage: flowchart (~35 lines) + Quick Reference Table (~15 lines) + Specialized Test Categories (~35 lines) = ~85 lines covering all original capabilities. Original content on sample size considerations, multiple comparisons, and missing data was consolidated into Best Practices and Common Pitfalls. Omitted: study design considerations (RCTs, observational, clustered data) -- general guidance covered by statsmodels-statistical-modeling skill. -
assumptions_and_diagnostics.md(370 lines original) -- Fully consolidated into Key Concepts (Assumptions Overview + Test-Specific Assumption Workflows) and Workflow Steps 4-5. Combined coverage: Assumptions Overview (~12 lines) + Test-Specific Assumption Workflows (~20 lines) + Workflow Steps 4-5 (~16 lines) = ~48 lines. The original contained detailed code blocks for each assumption check; since this is Knowhow (not Skill), code is referenced rather than reproduced. Key diagnostic thresholds preserved (VIF > 10, Durbin-Watson 1.5-2.5, variance ratio < 2-3). Omitted: extensive Python code blocks for individual checks (normality, homogeneity, linearity, logistic regression diagnostics) -- available in scipy.stats and pingouin documentation. Sample size rules of thumb covered in Workflow Step 3.
Script Disposition
assumption_checks.py(540 lines) -- Contains 6 functions:check_normality(),check_normality_per_group(),check_homogeneity_of_variance(),check_linearity(),detect_outliers(),comprehensive_assumption_check(). As Knowhow entry, script functions are referenced in Workflow Step 4 rather than reproduced inline. Key capabilities (Shapiro-Wilk, Levene's, IQR/z-score outlier detection, Q-Q plots) are described in Assumptions Overview and Test-Specific Assumption Workflows. Users needing automated checking should use scipy.stats and pingouin directly following the patterns described.
Intentional Omissions
- Time series methods (ARIMA, ACF/PACF) -- specialized topic beyond core statistical testing scope
- Mixed-effects models / GEE -- covered by statsmodels-statistical-modeling skill
- Bootstrap and permutation tests -- mentioned in passing; detailed implementation deferred to computational statistics resources
Further Reading
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.)
- Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics (4th ed.)
- Gelman, A., & Hill, J. (2006). Data Analysis Using Regression and Multilevel/Hierarchical Models
- Kruschke, J. K. (2014). Doing Bayesian Data Analysis (2nd ed.)
- APA Publication Manual: https://apastyle.apa.org/
- Cross Validated (stats Q&A): https://stats.stackexchange.com/
Related Skills
- statsmodels-statistical-modeling -- Implementing OLS, GLM, Logit, time-series models programmatically
- pymc-bayesian-modeling -- Full Bayesian modeling with MCMC sampling
- scikit-learn-machine-learning -- Predictive modeling, cross-validation, classification
- matplotlib-scientific-plotting -- Creating publication-quality statistical figures
- hypothesis-generation -- Structured hypothesis formulation before statistical testing
statsmodels
Overview
Statsmodels provides classical statistical modeling with rigorous inference for Python. It covers linear models, generalized linear models, discrete choice, time series, and comprehensive diagnostics. Unlike scikit-learn (prediction-focused), statsmodels emphasizes coefficient interpretation, p-values, confidence intervals, and model diagnostics.
When to Use
- Fitting linear regression (OLS, WLS, GLS) with detailed coefficient tables and diagnostics
- Running logistic regression with odds ratios and marginal effects for clinical/epidemiological studies
- Analyzing count data with Poisson or negative binomial regression
- Time series forecasting with ARIMA, SARIMAX, or exponential smoothing
- Performing ANOVA, t-tests, or non-parametric tests with proper corrections
- Testing model assumptions (heteroskedasticity, autocorrelation, normality of residuals)
- Model comparison using AIC/BIC or likelihood ratio tests
- Using R-style formula interface (
y ~ x1 + x2 + C(group)) for intuitive model specification - For prediction-focused ML with cross-validation and hyperparameter tuning, use
scikit-learninstead - For Bayesian modeling with posterior inference, use
pymcinstead
Prerequisites
- Python packages:
statsmodels,numpy,pandas,scipy - Optional:
matplotlib(for diagnostic plots),patsy(for formula API, included with statsmodels) - Data: Tabular data as pandas DataFrames or NumPy arrays
pip install statsmodels numpy pandas matplotlib
Quick Start
import statsmodels.api as sm
import statsmodels.formula.api as smf
import pandas as pd
import numpy as np
# Generate sample data
np.random.seed(42)
n = 100
df = pd.DataFrame({
"x1": np.random.randn(n),
"x2": np.random.randn(n),
"group": np.random.choice(["A", "B"], n)
})
df["y"] = 2 + 3 * df["x1"] - 1.5 * df["x2"] + np.random.randn(n)
# OLS with formula API (R-style)
results = smf.ols("y ~ x1 + x2 + C(group)", data=df).fit()
print(results.summary())
print(f"R²: {results.rsquared:.3f}, AIC: {results.aic:.1f}")
Core API
Module 1: Linear Regression (OLS, WLS, GLS)
Standard linear models with comprehensive diagnostics.
import statsmodels.api as sm
import numpy as np
# Generate data
np.random.seed(42)
X = np.random.randn(200, 3)
y = 1 + 2*X[:, 0] - 0.5*X[:, 1] + np.random.randn(200)
# ALWAYS add constant for intercept
X_const = sm.add_constant(X)
results = sm.OLS(y, X_const).fit()
print(results.summary())
print(f"\nCoefficients: {results.params}")
print(f"P-values: {results.pvalues}")
print(f"R²: {results.rsquared:.4f}")
# Predictions with confidence intervals
pred = results.get_prediction(X_const[:5])
print(pred.summary_frame())
# Robust standard errors (heteroskedasticity-consistent)
results_robust = sm.OLS(y, X_const).fit(cov_type="HC3")
print("Robust SEs:", results_robust.bse)
# Weighted Least Squares
weights = 1 / np.abs(results.resid + 0.1) # Example weights
results_wls = sm.WLS(y, X_const, weights=weights).fit()
print(f"WLS R²: {results_wls.rsquared:.4f}")
Module 2: Generalized Linear Models (GLM)
Extend regression to non-normal outcomes (binary, count, continuous-positive).
import statsmodels.api as sm
import numpy as np
# Poisson regression for count data
np.random.seed(42)
X = np.random.randn(200, 2)
X_const = sm.add_constant(X)
y_counts = np.random.poisson(np.exp(0.5 + 0.3*X[:, 0]))
model = sm.GLM(y_counts, X_const, family=sm.families.Poisson())
results = model.fit()
print(results.summary())
# Rate ratios
rate_ratios = np.exp(results.params)
print(f"Rate ratios: {rate_ratios}")
# Check overdispersion
overdispersion = results.pearson_chi2 / results.df_resid
print(f"Overdispersion ratio: {overdispersion:.2f}")
if overdispersion > 1.5:
print("→ Consider Negative Binomial model")
Module 3: Discrete Choice Models (Logit, Probit, Count)
Binary, multinomial, and count outcome models.
import statsmodels.api as sm
import numpy as np
# Logistic regression
np.random.seed(42)
X = np.random.randn(300, 2)
X_const = sm.add_constant(X)
prob = 1 / (1 + np.exp(-(0.5 + X[:, 0] - 0.5*X[:, 1])))
y_binary = np.random.binomial(1, prob)
logit_results = sm.Logit(y_binary, X_const).fit()
print(logit_results.summary())
# Odds ratios
odds_ratios = np.exp(logit_results.params)
print(f"Odds ratios: {odds_ratios}")
# Marginal effects (at means)
margeff = logit_results.get_margeff()
print(margeff.summary())
# Predicted probabilities
probs = logit_results.predict(X_const[:5])
print(f"Predicted P(Y=1): {probs}")
Module 4: Time Series (ARIMA, SARIMAX)
Univariate and multivariate time series modeling and forecasting.
import statsmodels.api as sm
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.stattools import adfuller
import numpy as np
import pandas as pd
# Generate time series
np.random.seed(42)
dates = pd.date_range("2020-01-01", periods=200, freq="D")
y = np.cumsum(np.random.randn(200)) + 50
ts = pd.Series(y, index=dates)
# Stationarity test
adf_result = adfuller(ts)
print(f"ADF statistic: {adf_result[0]:.4f}, p-value: {adf_result[1]:.4f}")
print("Stationary" if adf_result[1] < 0.05 else "Non-stationary → difference")
# Fit ARIMA
model = ARIMA(ts, order=(1, 1, 1))
results = model.fit()
print(results.summary())
# Forecast with confidence intervals
forecast = results.get_forecast(steps=30)
forecast_df = forecast.summary_frame()
print(f"30-day forecast:\n{forecast_df.head()}")
# Seasonal ARIMA (SARIMAX)
from statsmodels.tsa.statespace.sarimax import SARIMAX
# Monthly data with yearly seasonality
model_sarima = SARIMAX(ts, order=(1, 1, 1), seasonal_order=(1, 1, 1, 12))
results_sarima = model_sarima.fit(disp=False)
print(f"AIC: {results_sarima.aic:.1f}")
# Diagnostic plots
results_sarima.plot_diagnostics(figsize=(12, 8))
Module 5: Statistical Tests and Diagnostics
Assumption tests, hypothesis tests, and model validation.
import statsmodels.api as sm
from statsmodels.stats.diagnostic import het_breuschpagan, acorr_ljungbox
from statsmodels.stats.stattools import jarque_bera
import numpy as np
# Fit a model first
np.random.seed(42)
X = sm.add_constant(np.random.randn(200, 2))
y = 1 + 2*X[:, 1] + np.random.randn(200) * X[:, 1] # Heteroskedastic
results = sm.OLS(y, X).fit()
# Heteroskedasticity test (Breusch-Pagan)
bp_stat, bp_p, _, _ = het_breuschpagan(results.resid, X)
print(f"Breusch-Pagan p-value: {bp_p:.4f} {'→ heteroskedastic' if bp_p < 0.05 else '→ OK'}")
# Normality test (Jarque-Bera)
jb_stat, jb_p, _, _ = jarque_bera(results.resid)
print(f"Jarque-Bera p-value: {jb_p:.4f} {'→ non-normal' if jb_p < 0.05 else '→ OK'}")
# Autocorrelation test (Ljung-Box)
lb_result = acorr_ljungbox(results.resid, lags=[10], return_df=True)
print(f"Ljung-Box p-value (lag 10): {lb_result['lb_pvalue'].values[0]:.4f}")
# Variance Inflation Factor (multicollinearity)
from statsmodels.stats.outliers_influence import variance_inflation_factor
vif_data = pd.DataFrame({
"Variable": [f"x{i}" for i in range(X.shape[1])],
"VIF": [variance_inflation_factor(X, i) for i in range(X.shape[1])]
})
print(vif_data) # VIF > 10 suggests multicollinearity
Module 6: Formula API (R-style)
Intuitive model specification using formulas with automatic dummy coding.
import statsmodels.formula.api as smf
import pandas as pd
import numpy as np
np.random.seed(42)
df = pd.DataFrame({
"y": np.random.randn(100),
"x1": np.random.randn(100),
"x2": np.random.randn(100),
"group": np.random.choice(["A", "B", "C"], 100),
})
# Formula with categoricals (auto dummy-coded)
res = smf.ols("y ~ x1 + x2 + C(group)", data=df).fit()
print(res.summary())
# Interactions
res2 = smf.ols("y ~ x1 * x2", data=df).fit() # x1 + x2 + x1:x2
print(f"Interaction term p-value: {res2.pvalues['x1:x2']:.4f}")
# Logit via formula
df["binary"] = (df["y"] > 0).astype(int)
logit_res = smf.logit("binary ~ x1 + x2 + C(group)", data=df).fit()
print(f"Logit AIC: {logit_res.aic:.1f}")
Common Workflows
Workflow 1: Complete Regression Analysis
Goal: Fit OLS, validate assumptions, use robust SEs if needed.
import statsmodels.api as sm
import statsmodels.formula.api as smf
from statsmodels.stats.diagnostic import het_breuschpagan
from statsmodels.stats.outliers_influence import variance_inflation_factor
import numpy as np
import pandas as pd
# 1. Fit initial model
np.random.seed(42)
df = pd.DataFrame({"y": np.random.randn(200), "x1": np.random.randn(200), "x2": np.random.randn(200)})
df["y"] = 2 + 3*df["x1"] - df["x2"] + np.random.randn(200)
results = smf.ols("y ~ x1 + x2", data=df).fit()
# 2. Check heteroskedasticity
bp_stat, bp_p, _, _ = het_breuschpagan(results.resid, results.model.exog)
print(f"Breusch-Pagan p: {bp_p:.4f}")
# 3. If heteroskedastic, use robust SEs
if bp_p < 0.05:
results = smf.ols("y ~ x1 + x2", data=df).fit(cov_type="HC3")
print("Using HC3 robust standard errors")
# 4. Check multicollinearity
X = results.model.exog
for i in range(1, X.shape[1]): # skip constant
print(f"VIF x{i}: {variance_inflation_factor(X, i):.2f}")
# 5. Final results
print(results.summary())
print(f"\nAIC: {results.aic:.1f}, BIC: {results.bic:.1f}")
Workflow 2: Model Comparison
Goal: Compare nested and non-nested models using appropriate criteria.
import statsmodels.formula.api as smf
from scipy import stats
import pandas as pd
import numpy as np
np.random.seed(42)
df = pd.DataFrame({"y": np.random.randn(200), "x1": np.random.randn(200),
"x2": np.random.randn(200), "x3": np.random.randn(200)})
df["y"] = 1 + 2*df["x1"] - df["x2"] + 0.1*df["x3"] + np.random.randn(200)
# Fit nested models
m1 = smf.ols("y ~ x1", data=df).fit()
m2 = smf.ols("y ~ x1 + x2", data=df).fit()
m3 = smf.ols("y ~ x1 + x2 + x3", data=df).fit()
# Compare via AIC/BIC (lower = better)
comparison = pd.DataFrame({
"R²": [m.rsquared for m in [m1, m2, m3]],
"AIC": [m.aic for m in [m1, m2, m3]],
"BIC": [m.bic for m in [m1, m2, m3]],
}, index=["y~x1", "y~x1+x2", "y~x1+x2+x3"])
print(comparison)
# Likelihood ratio test (nested: m2 vs m3)
lr_stat = 2 * (m3.llf - m2.llf)
p_val = 1 - stats.chi2.cdf(lr_stat, df=m3.df_model - m2.df_model)
print(f"\nLR test (m3 vs m2): stat={lr_stat:.2f}, p={p_val:.4f}")
Workflow 3: Time Series Forecasting Pipeline
Goal: Test stationarity, identify model order, forecast.
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.stattools import adfuller
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# Generate data
np.random.seed(42)
ts = pd.Series(np.cumsum(np.random.randn(200)) + 100,
index=pd.date_range("2020-01-01", periods=200, freq="D"))
# 1. Test stationarity
adf_p = adfuller(ts)[1]
print(f"ADF p-value: {adf_p:.4f} → {'stationary' if adf_p < 0.05 else 'non-stationary'}")
# 2. Identify order from ACF/PACF (on differenced series)
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 6))
plot_acf(ts.diff().dropna(), lags=20, ax=ax1)
plot_pacf(ts.diff().dropna(), lags=20, ax=ax2)
plt.savefig("acf_pacf.png", dpi=150, bbox_inches="tight")
# 3. Fit and forecast
model = ARIMA(ts[:180], order=(1, 1, 1))
results = model.fit()
forecast = results.get_forecast(steps=20)
fc_df = forecast.summary_frame()
print(f"ARIMA AIC: {results.aic:.1f}")
print(f"Forecast (first 5 days):\n{fc_df.head()}")
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
cov_type | OLS/WLS/GLM | "nonrobust" | "HC0"-"HC3", "HAC", "cluster" | Robust covariance estimator |
family | GLM | required | Poisson(), Binomial(), Gamma(), etc. | Distribution family |
order | ARIMA | required | (p, d, q) tuple | AR order, differencing, MA order |
seasonal_order | SARIMAX | (0,0,0,0) | (P, D, Q, s) tuple | Seasonal ARIMA parameters |
alpha | summary(), conf_int() | 0.05 | 0.01-0.10 | Significance level for CIs |
maxiter | All .fit() | 35-100 | 50-1000 | Max optimization iterations |
method | .fit() | model-dependent | "newton", "bfgs", "lbfgs", "powell" | Optimization algorithm |
lags | ACF/PACF | None | 10-50 | Number of lags to display |
Best Practices
-
Always add a constant for OLS/GLM:
sm.add_constant(X)or use formula API (adds intercept automatically) -
Match model to outcome type: Binary → Logit/Probit, Counts → Poisson/NegBin, Continuous → OLS/WLS, Time series → ARIMA
-
Check diagnostics before interpreting: Run Breusch-Pagan (heteroskedasticity), Jarque-Bera (normality), Ljung-Box (autocorrelation) on residuals
-
Use robust SEs when assumptions fail:
results = model.fit(cov_type="HC3")for heteroskedasticity-robust inference -
Report effect sizes, not just p-values: Include coefficients, confidence intervals, and R² alongside significance tests
-
Prefer formula API for exploratory work:
smf.ols("y ~ x1 * x2 + C(group)", data=df)is more readable and handles categoricals automatically -
Test stationarity before time series modeling: Use ADF test; difference if non-stationary
Common Recipes
Recipe: ANOVA with Post-hoc Tests
When to use: Comparing means across 3+ groups.
import statsmodels.formula.api as smf
from statsmodels.stats.multicomp import pairwise_tukeyhsd
import pandas as pd
import numpy as np
np.random.seed(42)
df = pd.DataFrame({"value": np.concatenate([np.random.normal(m, 1, 30) for m in [5, 6, 7]]),
"group": np.repeat(["A", "B", "C"], 30)})
# One-way ANOVA
anova = smf.ols("value ~ C(group)", data=df).fit()
print(sm.stats.anova_lm(anova))
# Post-hoc Tukey HSD
tukey = pairwise_tukeyhsd(df["value"], df["group"], alpha=0.05)
print(tukey)
Recipe: Power Analysis for Sample Size
When to use: Determining required sample size before a study.
from statsmodels.stats.power import TTestIndPower
analysis = TTestIndPower()
# What sample size for medium effect (d=0.5), 80% power, alpha=0.05?
n = analysis.solve_power(effect_size=0.5, alpha=0.05, power=0.8)
print(f"Required n per group: {n:.0f}")
# Power for given sample size
power = analysis.solve_power(effect_size=0.5, alpha=0.05, nobs1=50)
print(f"Power with n=50: {power:.3f}")
Recipe: Mixed Effects Model
When to use: Hierarchical/clustered data (patients within hospitals, students within schools).
import statsmodels.formula.api as smf
import pandas as pd
import numpy as np
np.random.seed(42)
df = pd.DataFrame({
"y": np.random.randn(100), "x": np.random.randn(100),
"group": np.repeat(range(10), 10)
})
# Random intercept model
model = smf.mixedlm("y ~ x", data=df, groups=df["group"])
results = model.fit()
print(results.summary())
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
MissingDataError | NaN values in data | Drop NAs: df.dropna() or impute before fitting |
| No intercept in results | Forgot sm.add_constant() | Always add constant, or use smf.ols() formula API |
ConvergenceWarning | Optimization failed | Increase maxiter, try different method, or scale variables |
| Overdispersion in Poisson | Variance > mean | Switch to NegativeBinomial or use GLM(family=NegativeBinomial()) |
| Non-stationary time series | Trend or unit root | Difference the series (ts.diff()) or increase d in ARIMA |
| Singular matrix error | Perfect multicollinearity | Remove redundant variables; check VIF > 10 |
| Different results from R | Default settings differ | Check: constant term, link function, optimizer, SE type |
PerfectSeparationError in Logit | Predictor perfectly separates classes | Use regularized logistic (penalized MLE) or Firth's method |
References
- statsmodels documentation — official docs
- statsmodels User Guide — tutorials
- statsmodels API Reference — complete API
- Seabold, S. & Perktold, J. (2010). Statsmodels: Econometric and Statistical Modeling with Python. Proceedings of the 9th Python in Science Conference.
Cellpose — Deep Learning Cell Segmentation
Overview
Cellpose uses a flow-based neural network to segment individual cells or nuclei in fluorescence microscopy images without manual parameter tuning. Pre-trained models (cyto3, nuclei, tissuenet) generalize across cell types, magnifications, and staining conditions — eliminating the need for manual threshold selection or watershed parameter optimization. Cellpose outputs integer label masks (each cell = unique integer) compatible with scikit-image regionprops for morphology measurement and with TrackPy for tracking. A built-in diameter estimator removes the need to specify cell size, though providing an approximate diameter improves accuracy.
When to Use
- Segmenting cells or nuclei in fluorescence microscopy images where rule-based thresholding fails due to varying intensity or cell touching
- Processing large microscopy datasets in batch without per-image parameter tuning
- Segmenting diverse cell types (adherent cells, blood cells, bacteria, organoids) with a single model
- Producing label masks for downstream region property measurement (area, intensity, shape) with scikit-image
- 3D volumetric segmentation of z-stack microscopy data with
do_3D=True - Use scikit-image watershed when cells are well-separated and rule-based thresholding is sufficient
- Use StarDist as an alternative deep learning segmenter optimized for star-convex cells (neurons, nuclei)
Prerequisites
- Python packages:
cellpose,numpy,matplotlib - Optional: GPU with CUDA for 10-50× speedup (
pip install cellpose[gui]for GUI) - Input: grayscale or multichannel TIFF/PNG images (2D or 3D arrays)
# Install Cellpose
pip install cellpose
# Install with GUI support
pip install cellpose[gui]
# Install with GPU (PyTorch CUDA)
pip install cellpose torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
# Verify
python -c "from cellpose import models; print('Cellpose ready')"
Quick Start
from cellpose import models
import numpy as np
from skimage import io
# Load image (grayscale or 2D array)
img = io.imread("cells.tif") # shape: (H, W) or (H, W, C)
# Initialize model and segment
model = models.Cellpose(model_type="cyto3", gpu=False)
masks, flows, styles, diams = model.eval(img, diameter=0, channels=[0, 0])
print(f"Cells segmented: {masks.max()}") # number of cells
print(f"Estimated diameter: {diams:.1f} px")
print(f"Mask shape: {masks.shape}")
Workflow
Step 1: Load and Inspect Images
Load microscopy images and inspect channel layout before segmentation.
import numpy as np
from skimage import io
import matplotlib.pyplot as plt
# Load single-channel fluorescence image
img_gray = io.imread("nucleus_dapi.tif") # shape: (H, W)
img_rgb = io.imread("cells_multichannel.tif") # shape: (H, W, C)
print(f"Grayscale shape: {img_gray.shape}, dtype: {img_gray.dtype}")
print(f"Multichannel shape: {img_rgb.shape}")
# Preview
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
axes[0].imshow(img_gray, cmap="gray")
axes[0].set_title("DAPI (nuclei)")
axes[1].imshow(img_rgb[..., 0], cmap="green")
axes[1].set_title("GFP channel")
plt.tight_layout()
plt.savefig("image_preview.png", dpi=100)
print("Saved: image_preview.png")
Step 2: Segment Cells with a Pre-trained Model
Run Cellpose with the appropriate pre-trained model.
from cellpose import models
import numpy as np
from skimage import io
# Available models: 'cyto3' (cells), 'nuclei', 'tissuenet', 'cyto2', 'CP'
model = models.Cellpose(model_type="cyto3", gpu=False)
img = io.imread("cells.tif")
# channels=[cytoplasm_channel, nucleus_channel]
# Use [0, 0] for grayscale; [1, 3] for green cytoplasm + blue nucleus (1-indexed)
masks, flows, styles, diams = model.eval(
img,
diameter=0, # 0 = auto-estimate; or provide px estimate
channels=[0, 0], # grayscale
flow_threshold=0.4, # lower = fewer false positives; range 0.1-1.0
cellprob_threshold=0.0, # lower = more cells detected; range -6 to 6
)
print(f"Cells found: {masks.max()}")
print(f"Estimated cell diameter: {diams:.1f} pixels")
np.save("masks.npy", masks)
Step 3: Segment Nuclei from DAPI Channel
Use the nuclei model for DAPI-stained nuclei.
from cellpose import models
from skimage import io
import numpy as np
model = models.Cellpose(model_type="nuclei", gpu=False)
dapi = io.imread("dapi.tif")
# Nucleus-only segmentation: channels=[0, 0] (single channel)
masks, flows, styles, diams = model.eval(
dapi,
diameter=30, # approximate nucleus diameter in pixels
channels=[0, 0],
flow_threshold=0.4,
cellprob_threshold=0.0,
)
print(f"Nuclei segmented: {masks.max()}")
# Save label mask as TIFF for ImageJ/FIJI compatibility
from skimage import io as skio
skio.imsave("nuclei_masks.tif", masks.astype(np.uint16))
print("Saved: nuclei_masks.tif")
Step 4: Visualize Segmentation Results
Overlay masks on original images for quality control.
from cellpose import plot as cpplot
import matplotlib.pyplot as plt
import numpy as np
from skimage import io
img = io.imread("cells.tif")
masks = np.load("masks.npy")
flows_data = None # load if you saved them: flows = np.load("flows.npy", allow_pickle=True)
# Cellpose built-in visualization
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
# Original image
axes[0].imshow(img, cmap="gray")
axes[0].set_title(f"Original image")
# Label mask (each cell = unique color)
axes[1].imshow(masks, cmap="tab20")
axes[1].set_title(f"Segmentation masks ({masks.max()} cells)")
# Overlay: outline on original
from skimage.segmentation import find_boundaries
boundaries = find_boundaries(masks, mode="inner")
overlay = np.stack([img / img.max()] * 3, axis=-1)
overlay[boundaries] = [1, 0, 0] # red outlines
axes[2].imshow(overlay)
axes[2].set_title("Outlines overlay")
plt.tight_layout()
plt.savefig("segmentation_result.png", dpi=150)
print("Saved: segmentation_result.png")
Step 5: Measure Cell Properties from Masks
Extract morphology and intensity measurements using scikit-image regionprops.
import numpy as np
import pandas as pd
from skimage.measure import regionprops_table
from skimage import io
masks = np.load("masks.npy")
img = io.imread("cells.tif")
# Measure morphology and intensity per cell
props = regionprops_table(
masks, intensity_image=img,
properties=["label", "area", "centroid", "eccentricity",
"mean_intensity", "max_intensity", "perimeter",
"equivalent_diameter_area"]
)
df = pd.DataFrame(props)
df.columns = ["cell_id", "area_px", "centroid_y", "centroid_x",
"eccentricity", "mean_intensity", "max_intensity",
"perimeter", "diameter_px"]
print(f"Cells measured: {len(df)}")
print(f"Median area: {df['area_px'].median():.0f} px²")
print(f"Median diameter: {df['diameter_px'].median():.1f} px")
print(df.head())
df.to_csv("cell_measurements.csv", index=False)
Step 6: Batch Segment Multiple Images
Process a directory of images and aggregate results.
from cellpose import models
from skimage import io
from skimage.measure import regionprops_table
import pandas as pd
import numpy as np
from pathlib import Path
model = models.Cellpose(model_type="cyto3", gpu=False)
image_dir = Path("images/")
output_dir = Path("results/")
output_dir.mkdir(exist_ok=True)
all_stats = []
for img_path in sorted(image_dir.glob("*.tif")):
img = io.imread(img_path)
masks, _, _, diams = model.eval(img, diameter=0, channels=[0, 0])
# Save mask
np.save(output_dir / f"{img_path.stem}_masks.npy", masks)
# Measure
if masks.max() > 0:
props = regionprops_table(masks, intensity_image=img,
properties=["label", "area", "mean_intensity"])
df = pd.DataFrame(props)
df["image"] = img_path.name
df["est_diameter"] = diams
all_stats.append(df)
print(f"{img_path.name}: {masks.max()} cells, diameter={diams:.0f}px")
summary = pd.concat(all_stats, ignore_index=True)
summary.to_csv(output_dir / "all_cells.csv", index=False)
print(f"\nTotal cells: {len(summary)} across {summary['image'].nunique()} images")
Key Parameters
| Parameter | Default | Range/Options | Effect |
|---|---|---|---|
model_type | "cyto3" | "cyto3", "cyto2", "nuclei", "tissuenet", "CP", custom path | Pre-trained model; cyto3 is most general; nuclei for DAPI-only |
diameter | 30 | 0–500 px | Approximate cell diameter in pixels; 0 = auto-estimate from image |
channels | [0, 0] | [cyto, nucleus] (0=gray, 1=R, 2=G, 3=B) | Channel indices for cytoplasm and nuclear stain |
flow_threshold | 0.4 | 0.1–1.0 | Cell probability threshold from flow field; lower = stricter |
cellprob_threshold | 0.0 | −6 to 6 | Cell probability cutoff; increase to find more cells |
gpu | False | True, False | Enable GPU inference (requires CUDA PyTorch) |
do_3D | False | True, False | Enable 3D volumetric segmentation of z-stacks |
min_size | 15 | integer px² | Minimum object size in pixels²; smaller objects discarded |
batch_size | 8 | integer | Number of image tiles processed per GPU batch |
normalize | True | True, False | Normalize image intensity before segmentation |
Common Recipes
Recipe 1: Segment Multichannel Image (GFP + DAPI)
from cellpose import models
from skimage import io
import numpy as np
model = models.Cellpose(model_type="cyto3", gpu=False)
# Multichannel image: channel 1 = GFP (cytoplasm), channel 3 = DAPI (nucleus)
img_multi = io.imread("cells_gfp_dapi.tif") # shape: (H, W, 3)
# channels=[cytoplasm_channel, nucleus_channel] (1-indexed for multichannel)
masks, flows, styles, diams = model.eval(
img_multi,
diameter=0,
channels=[2, 3], # GFP=channel2, DAPI=channel3 (1-indexed)
flow_threshold=0.4,
)
print(f"Cells segmented: {masks.max()}, diameter: {diams:.0f}px")
np.save("masks_multichannel.npy", masks)
Recipe 2: Use Cellpose CLI for Directory Batch Processing
# CLI batch segmentation of all TIFFs in a directory
cellpose \
--image_path images/ \
--pretrained_model cyto3 \
--diameter 0 \
--chan 0 \
--save_tif \
--no_npy
# With GPU
cellpose \
--image_path images/ \
--pretrained_model nuclei \
--diameter 30 \
--chan 0 \
--use_gpu \
--save_tif
# Results saved as: images/*_cp_masks.tif
echo "Done. Masks saved in images/ directory."
Recipe 3: Fine-tune Cellpose on Custom Cell Type
from cellpose import models, train
import numpy as np
from skimage import io
# Prepare training data: list of images and corresponding masks
train_images = [io.imread(f"train/img_{i}.tif") for i in range(10)]
train_masks = [np.load(f"train/mask_{i}.npy") for i in range(10)]
# Fine-tune starting from cyto3
model = models.CellposeModel(model_type="cyto3")
# Train: saves model to models/ directory
model_path = train.train_seg(
model.net,
train_data=train_images,
train_labels=train_masks,
channels=[0, 0],
save_path="models/",
n_epochs=100,
learning_rate=0.2,
weight_decay=1e-5,
)
print(f"Fine-tuned model saved: {model_path}")
Expected Outputs
| Output | Format | Description |
|---|---|---|
masks array | numpy int32 | Label mask: 0=background, 1..N=unique cell IDs |
flows list | numpy arrays | Flow field components: [XY flows, cell prob, gradient] |
styles array | numpy float | Style vector embedding (used for model similarity) |
diams float | scalar | Estimated average cell diameter in pixels |
*_masks.npy | NumPy | Saved mask array (from np.save) |
*_cp_masks.tif | TIFF uint16 | Mask TIFF (from CLI --save_tif); compatible with FIJI/ImageJ |
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| All cells merged into one mask | Diameter too large or cells too close | Reduce diameter; increase flow_threshold to 0.6–0.8 |
| Very few cells detected | Diameter too small or cellprob_threshold too high | Increase cellprob_threshold to −2; use diameter=0 for auto |
| Many false positives (background labeled) | Low flow_threshold | Increase flow_threshold to 0.6–0.9; increase min_size |
| GPU out of memory | Image too large for GPU batch | Process in tiles; reduce batch_size; crop image |
| Poor generalization on new cell type | Model not trained on similar cells | Try all pre-trained models; fine-tune with 10-20 annotated images |
| 3D segmentation very slow | Large z-stack on CPU | Enable GPU; reduce z-stack depth; use anisotropy parameter |
| Mask values overflow uint8 | More than 255 cells in image | Save with dtype=np.uint16 or np.int32 |
Import error: No module named 'cellpose' | Package not installed | pip install cellpose or conda install -c conda-forge cellpose |
References
- Cellpose GitHub: MouseLand/cellpose — source code, documentation, and model zoo
- Stringer C et al. (2021) "Cellpose: a generalist algorithm for cellular segmentation" — Nature Methods 18:100-106. DOI:10.1038/s41592-020-01018-x
- Pachitariu M & Stringer C (2022) "Cellpose 2.0: how to train your own model" — Nature Methods 19:1500-1508. DOI:10.1038/s41592-022-01663-4
- Cellpose documentation — official API reference and training guide
FlowIO — Flow Cytometry File Handler
Overview
FlowIO is a lightweight Python library for reading and writing Flow Cytometry Standard (FCS) files. It parses FCS metadata, extracts event data as NumPy arrays, and creates new FCS files. Supports FCS versions 2.0, 3.0, and 3.1. Minimal dependencies — ideal for data pipelines and preprocessing before advanced analysis.
When to Use
- Parsing FCS files to extract event data as NumPy arrays
- Reading channel metadata (names, ranges, types) from FCS files
- Converting flow cytometry data to pandas DataFrames or CSV
- Creating new FCS files from NumPy arrays or processed data
- Handling multi-dataset FCS files (separating combined datasets)
- Batch processing directories of FCS files
- Preprocessing flow cytometry data before downstream analysis
- For compensation, gating, and FlowJo workspace support, use FlowKit instead
- For advanced cytometry visualization (density plots, gating plots), use matplotlib or plotly
Prerequisites
pip install flowio numpy pandas
Requires Python 3.9+. No compiled dependencies — installs on any platform.
Quick Start
from flowio import FlowData
flow = FlowData("experiment.fcs")
print(f"Events: {flow.event_count}, Channels: {flow.channel_count}")
print(f"Channels: {flow.pnn_labels}")
events = flow.as_array() # Shape: (n_events, n_channels)
print(f"Data shape: {events.shape}")
Core API
1. Reading FCS Files
The FlowData class is the primary interface for reading FCS files.
from flowio import FlowData
# Standard reading
flow = FlowData("sample.fcs")
print(f"Version: {flow.version}") # '3.0', '3.1', etc.
print(f"Events: {flow.event_count}")
print(f"Channels: {flow.channel_count}")
# Event data
events = flow.as_array() # Preprocessed (gain, log scaling)
raw = flow.as_array(preprocess=False) # Raw values
print(f"Shape: {events.shape}") # (n_events, n_channels)
# Memory-efficient: metadata only (skip DATA segment)
flow_meta = FlowData("sample.fcs", only_text=True)
print(f"Instrument: {flow_meta.text.get('$CYT', 'Unknown')}")
# Handle problematic files
flow = FlowData("bad.fcs", ignore_offset_discrepancy=True)
flow = FlowData("bad.fcs", use_header_offsets=True)
# Exclude null channels
flow = FlowData("sample.fcs", null_channel_list=["Time", "Null"])
2. Channel Metadata
Extract channel names, types, and ranges from FCS files.
flow = FlowData("sample.fcs")
# Channel names
pnn = flow.pnn_labels # Short names: ['FSC-A', 'SSC-A', 'FL1-A', ...]
pns = flow.pns_labels # Descriptive: ['Forward Scatter', 'Side Scatter', 'FITC', ...]
pnr = flow.pnr_values # Range/max values per channel
# Channel type indices
scatter_idx = flow.scatter_indices # [0, 1] — FSC, SSC
fluoro_idx = flow.fluoro_indices # [2, 3, 4] — fluorescence channels
time_idx = flow.time_index # Time channel index (or None)
# Access by type
events = flow.as_array()
scatter_data = events[:, scatter_idx]
fluoro_data = events[:, fluoro_idx]
# Full metadata (TEXT segment dictionary)
text = flow.text
print(f"Date: {text.get('$DATE', 'N/A')}")
print(f"Instrument: {text.get('$CYT', 'N/A')}")
3. Creating FCS Files
Generate new FCS files from NumPy arrays.
import numpy as np
from flowio import create_fcs
# Basic creation
events = np.random.rand(10000, 5) * 1000
channels = ["FSC-A", "SSC-A", "FL1-A", "FL2-A", "Time"]
create_fcs("output.fcs", events, channels)
# With descriptive names and metadata
create_fcs(
"output.fcs",
events,
channels,
opt_channel_names=["Forward Scatter", "Side Scatter", "FITC", "PE", "Time"],
metadata={"$SRC": "Python pipeline", "$DATE": "17-FEB-2026", "$CYT": "Synthetic"},
)
# Output: FCS 3.1, single-precision float
4. Multi-Dataset FCS Files
Handle FCS files containing multiple datasets.
from flowio import FlowData, read_multiple_data_sets, MultipleDataSetsError
# Detect multi-dataset files
try:
flow = FlowData("sample.fcs")
except MultipleDataSetsError:
datasets = read_multiple_data_sets("sample.fcs")
print(f"Found {len(datasets)} datasets")
for i, ds in enumerate(datasets):
print(f"Dataset {i}: {ds.event_count} events, {ds.channel_count} channels")
events = ds.as_array()
# Read specific dataset by offset
first = FlowData("multi.fcs", nextdata_offset=0)
next_offset = int(first.text.get("$NEXTDATA", "0"))
if next_offset > 0:
second = FlowData("multi.fcs", nextdata_offset=next_offset)
5. Modifying and Re-Exporting
Read, modify, and save FCS data.
from flowio import FlowData, create_fcs
# Read original
flow = FlowData("original.fcs")
events = flow.as_array(preprocess=False) # Use raw for modification
# Filter events (e.g., threshold on FSC)
mask = events[:, 0] > 500
filtered = events[mask]
print(f"Before: {len(events)}, After: {len(filtered)}")
# Save filtered data as new FCS
create_fcs(
"filtered.fcs",
filtered,
flow.pnn_labels,
opt_channel_names=flow.pns_labels,
metadata={**flow.text, "$SRC": "Filtered"},
)
# Or write with updated metadata (no event modification)
flow.write_fcs("updated.fcs", metadata={"$SRC": "Updated"})
Key Concepts
FCS File Structure
FCS files consist of four segments:
| Segment | Content | FlowData attribute |
|---|---|---|
| HEADER | Version, byte offsets | flow.header |
| TEXT | Key-value metadata ($DATE, $CYT, channel names) | flow.text |
| DATA | Event data (binary/float) | flow.events (bytes), flow.as_array() |
| ANALYSIS | Optional processed results | flow.analysis |
Preprocessing (as_array)
When preprocess=True (default), FlowIO applies:
- Gain scaling: Multiply by PnG gain values
- Log transform: Apply PnE exponential transform if present (
value = a × 10^(b × raw)) - Time scaling: Convert time channel to proper units
Use preprocess=False when you need raw values for modification or custom transforms.
Common Workflows
Workflow: Batch FCS Summary
from pathlib import Path
from flowio import FlowData
import pandas as pd
fcs_files = list(Path("data/").glob("*.fcs"))
summaries = []
for f in fcs_files:
try:
flow = FlowData(str(f), only_text=True)
summaries.append({
"file": f.name, "version": flow.version,
"events": flow.event_count, "channels": flow.channel_count,
"date": flow.text.get("$DATE", "N/A"),
})
except Exception as e:
print(f"Error: {f.name}: {e}")
df = pd.DataFrame(summaries)
print(df)
Workflow: FCS to DataFrame with Channel Statistics
from flowio import FlowData
import pandas as pd
import numpy as np
flow = FlowData("sample.fcs")
df = pd.DataFrame(flow.as_array(), columns=flow.pnn_labels)
# Per-channel statistics
for col in df.columns:
print(f"{col}: mean={df[col].mean():.1f}, median={df[col].median():.1f}, std={df[col].std():.1f}")
# Export
df.to_csv("output.csv", index=False)
print(f"Exported {len(df)} events, {len(df.columns)} channels")
Key Parameters
| Parameter | Function | Default | Options | Effect |
|---|---|---|---|---|
preprocess | as_array() | True | True/False | Apply gain/log scaling |
only_text | FlowData() | False | True/False | Skip DATA segment (metadata only) |
ignore_offset_discrepancy | FlowData() | False | True/False | Tolerate HEADER/TEXT offset mismatch |
use_header_offsets | FlowData() | False | True/False | Prefer HEADER over TEXT offsets |
ignore_offset_error | FlowData() | False | True/False | Skip all offset validation |
null_channel_list | FlowData() | None | List of names | Exclude channels during parsing |
nextdata_offset | FlowData() | None | byte offset | Read specific dataset in multi-dataset files |
opt_channel_names | create_fcs() | None | List of names | Descriptive channel names (PnS) |
metadata | create_fcs() | None | Dict | Custom TEXT segment key-value pairs |
Best Practices
-
Use
only_text=Truefor metadata scanning: When processing many files, skip DATA segment parsing for 10-100x speedup. -
Use
preprocess=Falsefor data modification: Always work with raw values when filtering/modifying events, then re-export. Preprocessing is irreversible. -
Anti-pattern — modifying
flow.eventsdirectly: FlowIO does not support in-place event modification. Extract withas_array(), modify, thencreate_fcs()to save. -
Preserve metadata on re-export: Pass
flow.textas metadata tocreate_fcs()to retain original acquisition info. -
Check for multi-dataset files: Catch
MultipleDataSetsErrorand useread_multiple_data_sets()— some instruments write multiple acquisitions into one file.
Common Recipes
Recipe: Extract Fluorescence Channels Only
from flowio import FlowData
import numpy as np
flow = FlowData("sample.fcs")
events = flow.as_array()
fluoro = events[:, flow.fluoro_indices]
names = [flow.pnn_labels[i] for i in flow.fluoro_indices]
print(f"Fluorescence channels: {names}, shape: {fluoro.shape}")
Recipe: File Inspection Report
from flowio import FlowData
flow = FlowData("unknown.fcs")
print(f"Version: {flow.version} | Events: {flow.event_count:,} | Channels: {flow.channel_count}")
for i, (pnn, pns) in enumerate(zip(flow.pnn_labels, flow.pns_labels)):
ctype = "scatter" if i in flow.scatter_indices else "fluoro" if i in flow.fluoro_indices else "time" if i == flow.time_index else "other"
print(f" [{i}] {pnn:10s} | {pns:30s} | {ctype}")
for key in ["$DATE", "$CYT", "$INST", "$SRC"]:
print(f" {key}: {flow.text.get(key, 'N/A')}")
Recipe: Normalize Events to [0, 1] Range
When to use: Prepare fluorescence channels for machine learning or cross-sample comparison.
from flowio import FlowData
import numpy as np
flow = FlowData("sample.fcs")
events = flow.as_array()
# Normalize each fluorescence channel to [0, 1]
fluoro_idx = flow.fluoro_indices
fluoro = events[:, fluoro_idx]
pnr = np.array(flow.pnr_values)[fluoro_idx] # Per-channel max range
normalized = fluoro / pnr
print(f"Normalized shape: {normalized.shape}, range: [{normalized.min():.3f}, {normalized.max():.3f}]")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
DataOffsetDiscrepancyError | HEADER/TEXT offset mismatch | Use ignore_offset_discrepancy=True |
MultipleDataSetsError | File contains multiple datasets | Use read_multiple_data_sets() instead |
FCSParsingError | Corrupt or non-standard FCS file | Try ignore_offset_error=True; verify file is valid FCS |
| Out of memory on large files | Millions of events loaded at once | Use only_text=True for metadata; process in chunks by channel |
| Unexpected channel count | Null/padding channels in file | Use null_channel_list=["Time", "Null"] to exclude |
| Modified data has wrong values | Applied preprocessing before modification | Use preprocess=False for raw data when modifying events |
| Channel names missing (empty PnS) | Instrument didn't set descriptive names | Use pnn_labels (short names) instead; PnS is optional in FCS spec |
Related Skills
- matplotlib-scientific-plotting — create scatter plots, density plots, and histograms from extracted cytometry data
- scikit-learn-machine-learning — clustering and dimensionality reduction on cytometry event data
References
- FlowIO documentation — official GitHub repository and API
- FCS file format specification — ISAC data standards for flow cytometry
- Spidlen et al. (2010) "Data File Standard for Flow Cytometry, Version FCS 3.1" — Cytometry Part A
napari — Multi-dimensional Image Viewer
Overview
napari is a fast, interactive multi-dimensional viewer for scientific data built on PyQt5 and VisPy. It displays NumPy arrays and zarr arrays as layered visualizations — Image layers for raw data, Labels layers for segmentation masks, Points layers for cell centroids, and Shapes layers for ROI annotations. napari integrates with scikit-image, Cellpose, and StarDist via plugins, making it the standard visualization and annotation tool in Python bioimage analysis pipelines. For headless environments (HPC, CI), napari supports offscreen rendering and viewer.screenshot() for automated figure generation.
When to Use
- Visually inspecting and quality-checking microscopy images and segmentation masks before quantitative analysis
- Annotating training data for deep learning segmentation models (Cellpose, StarDist)
- Overlaying multiple image channels (DAPI, GFP, mCherry) with independent contrast and colormap control
- Reviewing 3D z-stacks and 4D time-lapse experiments with slider-based navigation
- Exporting annotated screenshots or label masks from GUI for publication figures
- Running plugin-based analysis (Cellpose napari plugin, StarDist plugin, n2v denoising) interactively
- Use ImageJ/FIJI for macro/batch scripting with minimal Python dependency
- Use ITK-SNAP as an alternative for medical imaging (DICOM, NIfTI) segmentation
Prerequisites
- Python packages:
napari,numpy,scikit-image - Qt backend: requires display server; for headless use
QT_QPA_PLATFORM=offscreen - Optional plugins:
napari-cellpose,napari-stardist,napari-animation
# Install with all backends
pip install "napari[all]"
# Or minimal install
pip install napari pyqt5
# Verify
python -c "import napari; print(napari.__version__)"
# 0.5.5
# Install useful plugins
pip install napari-cellpose napari-animation
Quick Start
import napari
import numpy as np
from skimage import data
# Open viewer with a sample image
viewer = napari.Viewer()
viewer.add_image(data.cells3d()[:, 1, :, :], name="DAPI", colormap="blue")
napari.run() # blocks until viewer closed (use in scripts)
Core API
Module 1: Image Layer — Display Raw Images
Add and configure multi-channel image layers.
import napari
import numpy as np
from skimage import io
viewer = napari.Viewer()
# Add single grayscale image
img = io.imread("cells.tif") # shape: (H, W)
viewer.add_image(img, name="phase contrast", colormap="gray",
contrast_limits=[0, img.max()])
# Add multichannel image (3 channels)
img_mc = io.imread("multichannel.tif") # shape: (H, W, 3)
viewer.add_image(img_mc[..., 0], name="DAPI", colormap="blue", blending="additive")
viewer.add_image(img_mc[..., 1], name="GFP", colormap="green", blending="additive")
viewer.add_image(img_mc[..., 2], name="mCherry", colormap="red", blending="additive")
print(f"Layers: {[l.name for l in viewer.layers]}")
Module 2: Labels Layer — Visualize Segmentation Masks
Display and edit integer label masks from Cellpose, StarDist, or scikit-image.
import napari
import numpy as np
from skimage import io
viewer = napari.Viewer()
img = io.imread("cells.tif")
masks = np.load("masks.npy") # integer label array: 0=background, 1..N=cells
# Add raw image
viewer.add_image(img, name="raw", colormap="gray")
# Add label mask (each cell gets a unique random color)
label_layer = viewer.add_labels(masks, name="cell_masks", opacity=0.5)
# Access labels for editing
print(f"Unique cells: {len(np.unique(masks)) - 1}")
print(f"Label layer data shape: {label_layer.data.shape}")
Module 3: Points Layer — Mark Cell Centroids
Add and style point markers for centroids, landmarks, or detected features.
import napari
import numpy as np
import pandas as pd
from skimage.measure import regionprops_table
viewer = napari.Viewer()
# Compute centroids from label mask
masks = np.load("masks.npy")
props = regionprops_table(masks, properties=["centroid", "label"])
centroids = np.column_stack([props["centroid-0"], props["centroid-1"]])
# Add centroids as Points layer
viewer.add_points(
centroids,
name=f"centroids ({len(centroids)} cells)",
size=8,
face_color="yellow",
edge_color="black",
edge_width=0.5,
)
print(f"Cells marked: {len(centroids)}")
Module 4: Shapes Layer — Draw ROIs and Annotations
Add bounding boxes, polygons, and line annotations.
import napari
import numpy as np
viewer = napari.Viewer()
# Add rectangles as ROIs (format: [[y1, x1], [y2, x2]])
rois = [
np.array([[50, 100], [200, 300]]), # ROI 1
np.array([[300, 150], [450, 350]]), # ROI 2
]
shapes_layer = viewer.add_shapes(
rois,
shape_type="rectangle",
name="ROIs",
edge_color="cyan",
face_color="transparent",
edge_width=2,
)
# Retrieve shapes data for analysis
for i, shape in enumerate(shapes_layer.data):
y_min, x_min = shape.min(axis=0)
y_max, x_max = shape.max(axis=0)
print(f"ROI {i+1}: y={y_min:.0f}-{y_max:.0f}, x={x_min:.0f}-{x_max:.0f}")
Module 5: 3D and Time-lapse Visualization
Display z-stacks and time series with sliders.
import napari
import numpy as np
from skimage import data
viewer = napari.Viewer()
# 3D z-stack: shape (Z, H, W)
zstack = data.cells3d()[:, 1, :, :] # nuclei channel
viewer.add_image(zstack, name="z-stack nuclei",
colormap="cyan", blending="additive")
# 4D time-lapse: shape (T, H, W) or (T, Z, H, W)
timelapse = np.random.randint(0, 65535, (10, 256, 256), dtype=np.uint16)
viewer.add_image(timelapse, name="timelapse", colormap="gray")
# napari shows axis sliders automatically for ndim > 2
print(f"z-stack shape: {zstack.shape} → slider for Z axis")
print(f"timelapse shape: {timelapse.shape} → sliders for T axis")
Module 6: Headless Screenshot Export
Export screenshots without a display (for HPC and CI environments).
import os
os.environ["QT_QPA_PLATFORM"] = "offscreen" # must be set BEFORE importing napari
import napari
import numpy as np
from skimage import io, data
import matplotlib
matplotlib.use("Agg") # also set matplotlib backend
viewer = napari.Viewer(show=False)
img = data.cells3d()[30, 1, :, :] # single z-slice
masks = (img > img.mean()).astype(int) # simple threshold mask
viewer.add_image(img, name="DAPI", colormap="blue", blending="additive")
viewer.add_labels(masks.astype(np.int32), name="masks", opacity=0.5)
# Export screenshot
screenshot = viewer.screenshot(path="napari_export.png", canvas_only=True)
print(f"Screenshot saved: napari_export.png ({screenshot.shape})")
viewer.close()
Key Parameters
| Parameter | Module | Default | Effect |
|---|---|---|---|
colormap | add_image | "gray" | Colormap name (matplotlib cmaps + napari built-ins: "green", "blue", "cyan") |
contrast_limits | add_image | auto | [min, max] intensity clipping for display |
blending | add_image | "translucent" | "additive" for multichannel overlay; "opaque" for solid |
opacity | add_labels | 0.7 | 0–1 transparency of label layer over image |
face_color | add_points | "white" | Point fill color (name, hex, or RGBA) |
size | add_points | 10 | Point radius in data coordinates (pixels) |
edge_width | add_shapes | 1 | Shape outline width in pixels |
show | Viewer() | True | False for headless/offscreen mode |
ndisplay | Viewer() | 2 | 3 for 3D OpenGL rendering mode |
canvas_only | screenshot() | False | True to exclude the napari toolbar from export |
Common Workflows
Workflow 1: Review Cellpose Segmentation Quality
import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"
import napari
import numpy as np
from cellpose import models
from skimage import io
from skimage.measure import regionprops_table
# Segment with Cellpose
img = io.imread("cells.tif")
model = models.Cellpose(model_type="cyto3", gpu=False)
masks, _, _, diams = model.eval(img, diameter=0, channels=[0, 0])
# Visualize in napari (headless for export)
viewer = napari.Viewer(show=False)
viewer.add_image(img, name="raw", colormap="gray")
viewer.add_labels(masks, name=f"masks ({masks.max()} cells)", opacity=0.6)
# Add centroids
props = regionprops_table(masks, properties=["centroid"])
centroids = np.column_stack([props["centroid-0"], props["centroid-1"]])
viewer.add_points(centroids, name="centroids", size=6, face_color="yellow")
viewer.screenshot(path="segmentation_review.png", canvas_only=True)
viewer.close()
print(f"QC export: segmentation_review.png — {masks.max()} cells detected")
Workflow 2: Multi-channel FISH Image Analysis
import napari
import numpy as np
from skimage import io
# Load 4-channel FISH image: DAPI + 3 RNA probes
img = io.imread("fish_4channel.tif") # shape: (H, W, 4)
viewer = napari.Viewer()
channels = [
("DAPI", "blue", img[..., 0]),
("probe_A_cy3", "yellow", img[..., 1]),
("probe_B_cy5", "red", img[..., 2]),
("probe_C_gfp", "green", img[..., 3]),
]
for name, colormap, channel in channels:
viewer.add_image(channel, name=name, colormap=colormap,
blending="additive",
contrast_limits=[channel.min(), np.percentile(channel, 99.5)])
napari.run()
Common Recipes
Recipe 1: Export All Layers as Annotated Figure
import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"
import napari
import numpy as np
from skimage import io
import matplotlib.pyplot as plt
viewer = napari.Viewer(show=False)
img = io.imread("cells.tif")
masks = np.load("masks.npy")
viewer.add_image(img, name="raw", colormap="gray")
viewer.add_labels(masks, name="segmentation", opacity=0.5)
# Set camera zoom and position
viewer.camera.zoom = 1.5
viewer.camera.center = (img.shape[0] // 2, img.shape[1] // 2)
screenshot = viewer.screenshot(path="figure_panel.png", canvas_only=True)
viewer.close()
# Add scalebar with matplotlib
fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(screenshot)
ax.axis("off")
plt.tight_layout()
plt.savefig("figure_final.pdf", dpi=300, bbox_inches="tight")
print("Exported: figure_final.pdf")
Recipe 2: Batch Export Z-stack Projections
import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"
import napari
import numpy as np
from skimage import io
from pathlib import Path
output_dir = Path("projections")
output_dir.mkdir(exist_ok=True)
for img_path in sorted(Path("zstacks").glob("*.tif")):
zstack = io.imread(img_path) # shape: (Z, H, W)
max_proj = zstack.max(axis=0)
viewer = napari.Viewer(show=False)
viewer.add_image(max_proj, name="max_projection", colormap="gray")
viewer.screenshot(path=str(output_dir / f"{img_path.stem}_maxproj.png"), canvas_only=True)
viewer.close()
print(f"Exported: {img_path.stem}_maxproj.png")
print("All z-stack projections exported.")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" | Missing display or Qt platform plugin | Set QT_QPA_PLATFORM=offscreen before importing napari; install libxcb-util-dev |
| napari window does not open | Running in SSH without X forwarding | Use viewer = napari.Viewer(show=False) and export via screenshot() |
| Slow rendering of large images | Image too large for GPU VRAM | Use viewer.add_image(img, multiscale=True) for pyramidal rendering |
| Labels layer shows wrong colors | Mask dtype overflow | Ensure masks are int32 not uint8 (overflow at 255 cells) |
napari.run() blocks Jupyter notebook | Qt event loop conflict | Use %gui qt magic in Jupyter; or use viewer.show() without napari.run() |
| Screenshot is black/empty | Viewer not fully rendered before screenshot | Add viewer.update() or slight delay before screenshot() |
| Plugin not appearing in menu | Plugin not installed or wrong napari version | pip install napari-<plugin>; check napari version compatibility on napari-hub |
| 3D rendering slow | Complex geometry or large volume | Switch viewer.dims.ndisplay = 2; reduce z-stack depth |
References
- napari documentation — official API reference and tutorials
- napari GitHub: napari/napari — source code and plugin development guide
- Sofroniew N et al. (2022) "napari: a multi-dimensional image viewer for Python" — Zenodo. DOI:10.5281/zenodo.3555620
- napari-hub — plugin registry with 300+ community analysis plugins
OpenCV — Bio-image Computer Vision
Overview
OpenCV (cv2) provides optimized C++-backed image processing routines for preprocessing, segmentation, feature extraction, and video analysis of biological images. In life sciences, OpenCV is used for fluorescence image enhancement (background subtraction, CLAHE), morphological segmentation (watershed, contour detection), brightfield cell detection, and real-time microscopy stream processing. Unlike scikit-image (which emphasizes scientific measurement), OpenCV prioritizes computational speed and video support — making it ideal for preprocessing pipelines and real-time imaging applications.
When to Use
- Preprocessing fluorescence or brightfield images: background subtraction, CLAHE, Gaussian/median blur
- Detecting cell contours, blobs, or edges without deep learning (classical methods)
- Processing video streams from live-cell imaging microscopes in real-time
- Template matching for finding repeated structures (organelles, crystals, patterns)
- Applying morphological operations (erosion, dilation, opening, closing) for mask refinement
- Computing optical flow between video frames for cell tracking
- Use scikit-image instead for scientific morphometry, regionprops, and scientific image I/O (TIFF metadata)
- Use Cellpose or StarDist instead for deep-learning cell segmentation on fluorescence images
Prerequisites
- Python packages:
opencv-python,numpy,matplotlib - Optional:
opencv-contrib-pythonfor extra modules (SIFT, SURF, optical flow)
# Install OpenCV
pip install opencv-python
# Install with extra contributed modules (SIFT, SURF, etc.)
pip install opencv-contrib-python
# Verify
python -c "import cv2; print(cv2.__version__)"
# 4.10.0
Quick Start
import cv2
import numpy as np
# Read and display image info
img = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)
print(f"Shape: {img.shape}, dtype: {img.dtype}")
print(f"Min: {img.min()}, Max: {img.max()}")
# Apply Gaussian blur and threshold
blurred = cv2.GaussianBlur(img, (5, 5), 0)
_, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
print(f"Cells detected (rough): {np.sum(binary > 0)} foreground pixels")
Core API
Module 1: Image I/O and Color Space Conversion
Read, write, and convert images between color spaces.
import cv2
import numpy as np
# Read image (GRAYSCALE, COLOR, or UNCHANGED for 16-bit)
img_gray = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE) # uint8
img_color = cv2.imread("rgb.tif", cv2.IMREAD_COLOR) # BGR order!
img_16bit = cv2.imread("16bit.tif", cv2.IMREAD_UNCHANGED) # uint16
print(f"Grayscale shape: {img_gray.shape}, dtype: {img_gray.dtype}")
print(f"Color shape: {img_color.shape}")
# Color space conversions
img_rgb = cv2.cvtColor(img_color, cv2.COLOR_BGR2RGB) # BGR → RGB
img_hsv = cv2.cvtColor(img_color, cv2.COLOR_BGR2HSV) # BGR → HSV
img_gray2 = cv2.cvtColor(img_color, cv2.COLOR_BGR2GRAY) # BGR → gray
# Write image
cv2.imwrite("output.png", img_gray)
cv2.imwrite("output_16bit.tif", img_16bit)
print("Images written.")
Module 2: Filtering and Enhancement
Apply filters and contrast enhancement for image preprocessing.
import cv2
import numpy as np
img = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)
# Gaussian blur (noise reduction)
blurred = cv2.GaussianBlur(img, (7, 7), sigmaX=1.5)
# Median blur (salt-and-pepper noise)
median = cv2.medianBlur(img, 5)
# CLAHE: Contrast Limited Adaptive Histogram Equalization (for microscopy)
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
clahe_img = clahe.apply(img)
# Top-hat filter for bright spots on dark background
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (15, 15))
tophat = cv2.morphologyEx(img, cv2.MORPH_TOPHAT, kernel)
print(f"CLAHE range: [{clahe_img.min()}, {clahe_img.max()}]")
cv2.imwrite("clahe_enhanced.tif", clahe_img)
Module 3: Thresholding and Binary Segmentation
Convert grayscale images to binary masks using various thresholding methods.
import cv2
import numpy as np
img = cv2.imread("nuclei.tif", cv2.IMREAD_GRAYSCALE)
# Otsu's thresholding (automatic threshold selection)
thresh_val, otsu_mask = cv2.threshold(img, 0, 255,
cv2.THRESH_BINARY + cv2.THRESH_OTSU)
print(f"Otsu threshold: {thresh_val:.0f}")
# Adaptive thresholding (handles uneven illumination)
adaptive = cv2.adaptiveThreshold(
img, 255,
cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
cv2.THRESH_BINARY,
blockSize=11, # neighborhood size (odd)
C=2, # constant subtracted from mean
)
# For 16-bit images: normalize first
img_16 = cv2.imread("16bit_nuclei.tif", cv2.IMREAD_UNCHANGED)
img_8 = cv2.normalize(img_16, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
_, mask_16 = cv2.threshold(img_8, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
print(f"Otsu mask foreground: {mask_16.sum() / 255} pixels")
Module 4: Contour Detection and Measurement
Find and measure cell contours from binary masks.
import cv2
import numpy as np
import pandas as pd
img = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)
blurred = cv2.GaussianBlur(img, (5, 5), 0)
_, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
# Remove small objects with morphological opening
kernel = np.ones((3, 3), np.uint8)
cleaned = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel, iterations=2)
# Find contours
contours, hierarchy = cv2.findContours(cleaned, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
print(f"Objects detected: {len(contours)}")
# Measure each contour
records = []
for i, cnt in enumerate(contours):
area = cv2.contourArea(cnt)
if area < 50: continue # skip tiny objects
perimeter = cv2.arcLength(cnt, True)
x, y, w, h = cv2.boundingRect(cnt)
(cx, cy), radius = cv2.minEnclosingCircle(cnt)
records.append({"cell_id": i, "area": area, "perimeter": perimeter,
"x": x, "y": y, "w": w, "h": h, "radius": radius})
df = pd.DataFrame(records)
print(f"Cells > 50 px²: {len(df)}")
print(df[["area", "perimeter", "radius"]].describe())
Module 5: Morphological Operations for Mask Refinement
Refine segmentation masks with morphological operations.
import cv2
import numpy as np
# Load binary mask (from thresholding or Cellpose)
mask = cv2.imread("rough_mask.png", cv2.IMREAD_GRAYSCALE)
_, mask = cv2.threshold(mask, 127, 255, cv2.THRESH_BINARY)
# Structural elements
ellipse = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (7, 7))
rect = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
# Opening: remove small bright noise
opened = cv2.morphologyEx(mask, cv2.MORPH_OPEN, ellipse, iterations=1)
# Closing: fill small holes inside cells
closed = cv2.morphologyEx(opened, cv2.MORPH_CLOSE, ellipse, iterations=2)
# Dilation: expand cell boundaries slightly
dilated = cv2.dilate(closed, ellipse, iterations=1)
# Distance transform for watershed seed generation
dist = cv2.distanceTransform(closed, cv2.DIST_L2, 5)
_, seeds = cv2.threshold(dist, 0.5 * dist.max(), 255, 0)
seeds = seeds.astype(np.uint8)
print(f"Potential cell centers: {cv2.connectedComponents(seeds)[0] - 1}")
Module 6: Video Processing for Live-Cell Imaging
Process video streams from time-lapse microscopy.
import cv2
import numpy as np
# Process a time-lapse video file
cap = cv2.VideoCapture("timelapse.avi")
fps = cap.get(cv2.CAP_PROP_FPS)
n_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
print(f"Video: {n_frames} frames at {fps} FPS")
# Background subtraction (remove static background)
bg_subtractor = cv2.createBackgroundSubtractorMOG2(
history=50, varThreshold=25, detectShadows=False
)
frame_counts = []
frame_idx = 0
while cap.isOpened():
ret, frame = cap.read()
if not ret: break
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
fg_mask = bg_subtractor.apply(gray)
# Count moving objects in this frame
contours, _ = cv2.findContours(fg_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
moving = [c for c in contours if cv2.contourArea(c) > 100]
frame_counts.append(len(moving))
frame_idx += 1
cap.release()
print(f"Processed {frame_idx} frames. Mean moving objects: {np.mean(frame_counts):.1f}")
Key Parameters
| Parameter | Module | Default | Effect |
|---|---|---|---|
sigmaX | GaussianBlur | auto from ksize | Gaussian standard deviation; larger = more smoothing |
clipLimit | createCLAHE | 40.0 | Maximum contrast amplification; 2.0–4.0 for microscopy |
tileGridSize | createCLAHE | (8,8) | Tile size for local histogram equalization |
blockSize | adaptiveThreshold | required | Neighborhood size for adaptive threshold (must be odd, ≥ 3) |
C | adaptiveThreshold | required | Constant subtracted from mean; positive to subtract |
iterations | morphologyEx | 1 | Number of erosion/dilation cycles; higher = stronger effect |
history | BackgroundSubtractorMOG2 | 500 | Frames to model background; lower = faster adaptation |
varThreshold | BackgroundSubtractorMOG2 | 16 | Pixel variance threshold; higher = less sensitive |
minArea | contour filter | — | Minimum cv2.contourArea(cnt) to keep; filter noise |
cv2.IMREAD_UNCHANGED | imread | — | Preserve bit-depth (16-bit, 32-bit); required for scientific images |
Common Workflows
Workflow 1: Fluorescence Nucleus Detection Pipeline
import cv2
import numpy as np
import pandas as pd
def detect_nuclei(image_path: str, min_area: int = 200) -> pd.DataFrame:
"""Detect DAPI-stained nuclei from a fluorescence image."""
img = cv2.imread(image_path, cv2.IMREAD_UNCHANGED)
# Normalize 16-bit to 8-bit
if img.dtype == np.uint16:
img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
# Preprocess: CLAHE → Gaussian blur
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
enhanced = clahe.apply(img)
blurred = cv2.GaussianBlur(enhanced, (5, 5), 1.5)
# Segment: Otsu threshold → morphological opening
_, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
cleaned = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel, iterations=1)
# Find and measure contours
contours, _ = cv2.findContours(cleaned, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
records = []
for cnt in contours:
area = cv2.contourArea(cnt)
if area < min_area: continue
M = cv2.moments(cnt)
if M["m00"] == 0: continue
cx = int(M["m10"] / M["m00"])
cy = int(M["m01"] / M["m00"])
records.append({"area": area, "cx": cx, "cy": cy,
"perimeter": cv2.arcLength(cnt, True)})
return pd.DataFrame(records)
df = detect_nuclei("dapi.tif", min_area=300)
print(f"Nuclei detected: {len(df)}")
print(df.describe())
Workflow 2: Batch Process Image Directory
import cv2
import numpy as np
import pandas as pd
from pathlib import Path
def process_image(path: str) -> dict:
img = cv2.imread(path, cv2.IMREAD_GRAYSCALE)
if img is None:
return {}
blurred = cv2.GaussianBlur(img, (5, 5), 0)
_, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cells = [c for c in contours if cv2.contourArea(c) > 200]
return {"file": Path(path).name, "cell_count": len(cells),
"mean_area": np.mean([cv2.contourArea(c) for c in cells]) if cells else 0}
results = [process_image(str(p)) for p in sorted(Path("images").glob("*.tif"))]
df = pd.DataFrame([r for r in results if r])
print(df)
df.to_csv("batch_results.csv", index=False)
print("Saved: batch_results.csv")
Common Recipes
Recipe 1: Annotate Detected Cells on Image
import cv2
import numpy as np
img = cv2.imread("cells.tif", cv2.IMREAD_GRAYSCALE)
img_color = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
blurred = cv2.GaussianBlur(img, (5, 5), 0)
_, binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for i, cnt in enumerate(contours):
if cv2.contourArea(cnt) < 200: continue
# Draw contour outline
cv2.drawContours(img_color, [cnt], -1, (0, 255, 0), 2)
# Label with cell number
M = cv2.moments(cnt)
if M["m00"] > 0:
cx, cy = int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"])
cv2.putText(img_color, str(i), (cx - 5, cy), cv2.FONT_HERSHEY_SIMPLEX,
0.4, (255, 255, 0), 1)
cv2.imwrite("annotated_cells.png", img_color)
print(f"Annotated {len(contours)} cells. Saved: annotated_cells.png")
Recipe 2: Background Subtraction with Rolling Ball
import cv2
import numpy as np
def rolling_ball_background(img: np.ndarray, radius: int = 50) -> np.ndarray:
"""Estimate and subtract background using a blur approximation."""
kernel_size = 2 * radius + 1
background = cv2.GaussianBlur(img, (kernel_size, kernel_size), radius / 3)
corrected = cv2.subtract(img, background)
return corrected
img = cv2.imread("uneven_fluorescence.tif", cv2.IMREAD_GRAYSCALE)
corrected = rolling_ball_background(img, radius=50)
cv2.imwrite("background_corrected.tif", corrected)
print(f"Background corrected. Range: [{corrected.min()}, {corrected.max()}]")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
imread returns None | File not found or unsupported format | Use absolute path; verify with Path(path).exists(); for TIFF use cv2.IMREAD_UNCHANGED |
| 16-bit image shows as black | IMREAD_GRAYSCALE clips to uint8 | Use cv2.IMREAD_UNCHANGED and normalize: cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX) |
| BGR vs RGB color mismatch | OpenCV uses BGR, matplotlib uses RGB | Convert: rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) before plt.imshow() |
| Contours split one cell into many | Binary mask has holes or noise | Apply cv2.MORPH_CLOSE before contour detection; increase Gaussian blur sigma |
GaussianBlur requires odd kernel | Even kernel size provided | Always use odd kernel sizes: 3, 5, 7, 9; ksize=(5,5) not (4,4) |
| CLAHE makes image worse | clipLimit too high | Reduce clipLimit to 1.5–2.0; increase tileGridSize to (16,16) |
| Background subtraction removes cells | History too short for MOG2 | Increase history parameter; use static frame subtraction for microscopy |
| Performance slow on large images | Python loop over pixels | Use vectorized NumPy operations or CUDA-accelerated cv2.cuda module |
References
- OpenCV Python documentation — full Python API reference and tutorials
- OpenCV GitHub: opencv/opencv — source code and Python bindings
- Bradski G (2000) "The OpenCV Library" — Dr. Dobb's Journal of Software Tools 25(11):120-125
- PyImageSearch tutorials — applied OpenCV for computer vision in biological imaging
PyImageJ — Python Bridge to ImageJ/Fiji
Overview
PyImageJ provides a Python interface to ImageJ2 and Fiji through PyJNIus and scyjava, embedding a full Java Virtual Machine inside a Python process. It enables bidirectional data exchange between NumPy arrays and ImageJ's ImagePlus/ImgLib2 data structures, so you can preprocess images in Python, pass them into Fiji plugins (Bio-Formats, TrackMate, Analyze Particles, Weka segmentation), and return results back to pandas DataFrames. The library supports headless operation for scripting and batch processing, as well as GUI mode for interactive Fiji sessions.
When to Use
- Running Fiji-specific plugins from Python: Bio-Formats multi-format I/O, TrackMate particle tracking, CLIJ2 GPU processing, or community Fiji update site plugins
- Automating ImageJ macro pipelines headlessly without opening the Fiji GUI, e.g., batch processing an entire experiment overnight
- Applying the ImageJ Ops framework (150+ image processing operations) with the full ImageJ type system
- Converting between NumPy arrays (SciPy ecosystem) and ImageJ hyperstacks (TZCYX channel order) for round-trip processing
- Parsing ImageJ Results tables and ROI Manager measurements into pandas DataFrames for downstream statistical analysis
- Executing existing
.ijmmacro files as part of a Python workflow without rewriting them - Use
scikit-imageinstead when you need pure Python processing without Fiji plugins — scikit-image is faster to install and avoids JVM overhead - Use
napariinstead for interactive multi-dimensional image visualization and annotation; PyImageJ does not replace a viewer
Prerequisites
- Python packages:
pyimagej,scyjava,numpy,pandas - Java: Java 8 or Java 11 (Java 17 is not supported); use conda for reliable Java management
- Fiji/ImageJ2: Downloaded automatically on first init, or specify a local Fiji installation path
- Environment: conda environment strongly recommended; pip-only installs often have JVM path issues
# Recommended: conda installation
conda create -n pyimagej -c conda-forge pyimagej openjdk=11
conda activate pyimagej
# Install additional dependencies
pip install pandas tifffile
# Verify
python -c "import imagej; ij = imagej.init('sc.fiji:fiji', mode='headless'); print(ij.getVersion())"
Quick Start
import imagej
import numpy as np
# Initialize Fiji in headless mode (downloads on first run, ~500 MB)
ij = imagej.init("sc.fiji:fiji", mode="headless")
print(f"ImageJ version: {ij.getVersion()}")
# Create a test image, process with Gaussian blur via Ops, convert back
arr = np.random.randint(0, 1000, (256, 256), dtype=np.uint16)
imp = ij.py.to_imageplus(arr)
blurred = ij.op().filter().gauss(imp.getProcessor(), 2.0)
result = ij.py.from_imageplus(imp)
print(f"Processed array shape: {result.shape}, dtype: {result.dtype}")
Core API
Module 1: Initialization
PyImageJ must be initialized once per Python session. The mode and endpoint determine which ImageJ distribution and GUI behavior to use.
import imagej
# Headless Fiji — most common for scripts and batch jobs
ij = imagej.init("sc.fiji:fiji", mode="headless")
# GUI mode — opens the Fiji window (requires a display)
ij = imagej.init("sc.fiji:fiji", mode="gui")
# Local Fiji installation — faster startup, no download
ij = imagej.init("/path/to/Fiji.app", mode="headless")
# Specific Fiji version
ij = imagej.init("sc.fiji:fiji:2.14.0", mode="headless")
# Bare ImageJ2 without Fiji plugins
ij = imagej.init("net.imagej:imagej", mode="headless")
print(f"ImageJ version: {ij.getVersion()}")
print(f"Headless: {ij.ui().isHeadless()}")
Module 2: Image I/O
Open and save images using ImageJ's I/O layer (which includes Bio-Formats for proprietary formats) and convert between ImageJ and NumPy representations.
import imagej
import numpy as np
ij = imagej.init("sc.fiji:fiji", mode="headless")
# Open any format Bio-Formats supports: CZI, LIF, ND2, ICS, TIFF, etc.
imp = ij.io().open("/data/experiment.czi")
print(f"Dimensions: {imp.getDimensions()}") # [W, H, C, Z, T]
print(f"nSlices: {imp.getNSlices()}, nFrames: {imp.getNFrames()}")
# Save image
ij.io().save(imp, "/data/output.tif")
print("Saved output.tif")
# NumPy ↔ ImageJ conversion
arr = np.zeros((100, 100), dtype=np.uint16)
arr[30:70, 30:70] = 1000 # bright square
# NumPy → ImagePlus
imp = ij.py.to_imageplus(arr)
print(f"ImagePlus: {imp.getWidth()}×{imp.getHeight()}, type={imp.getType()}")
# ImagePlus → NumPy (returns a view where possible)
arr_back = ij.py.from_imageplus(imp)
print(f"NumPy array: shape={arr_back.shape}, dtype={arr_back.dtype}")
# Multi-channel array: shape (C, H, W)
rgb = np.random.randint(0, 255, (3, 256, 256), dtype=np.uint8)
imp_rgb = ij.py.to_imageplus(rgb)
print(f"Channels: {imp_rgb.getNChannels()}")
Module 3: Macro Execution
Run ImageJ macro language (IJM) snippets or macro files. Macros execute inside the ImageJ environment and can call any built-in ImageJ command.
import imagej
ij = imagej.init("sc.fiji:fiji", mode="headless")
# Run an inline macro string
ij.macro.run("print('Hello from ImageJ macro');")
# Run a macro with options string (key=value pairs)
# Options string mirrors the dialog parameters of ImageJ commands
macro_code = """
run("Gaussian Blur...", "sigma=2");
run("Auto Threshold", "method=Otsu white");
"""
ij.macro.run(macro_code)
# Run a macro file from disk
ij.macro.runMacroFile("/scripts/my_analysis.ijm")
# Run macro that returns a value via getResult or output string
result = ij.macro.run("""
x = 42 * 2;
return x;
""")
print(f"Macro returned: {result}")
# Macro with current image: open → process → measure
ij.io().open("/data/cells.tif") # sets current active image
measure_macro = """
run("Set Measurements...", "area mean min integrated redirect=None decimal=3");
run("Analyze Particles...", "size=50-Infinity display clear summarize");
"""
ij.macro.run(measure_macro)
print("Analyze Particles complete; results in Results table")
Module 4: ImageJ Ops
ImageJ Ops is a framework of 150+ image processing operations with type-safe dispatch. Ops work on ImgLib2 Img objects and are the preferred way to call image processing algorithms programmatically.
import imagej
import numpy as np
ij = imagej.init("sc.fiji:fiji", mode="headless")
arr = np.random.randint(100, 900, (512, 512), dtype=np.uint16)
img = ij.py.to_java(arr) # converts to ImgLib2 RandomAccessibleInterval
# Gaussian blur
blurred = ij.op().filter().gauss(img, 2.0)
blurred_np = ij.py.from_java(blurred)
print(f"Blurred: {blurred_np.shape}")
# Otsu threshold → binary image
binary = ij.op().threshold().otsu(img)
binary_np = ij.py.from_java(binary)
print(f"Binary unique values: {np.unique(binary_np)}")
# Morphological operations
from jnius import autoclass
BitType = autoclass("net.imglib2.type.logic.BitType")
opened = ij.op().morphology().open(binary, [3, 3])
opened_np = ij.py.from_java(opened)
print(f"After opening: {opened_np.shape}")
# Statistics ops
mean_val = ij.op().stats().mean(img)
std_val = ij.op().stats().stdDev(img)
print(f"Mean intensity: {mean_val:.1f}, StdDev: {std_val:.1f}")
# Math ops: multiply image by scalar
scaled = ij.op().math().multiply(img, ij.py.to_java(2.0))
print(f"Scaled max: {ij.py.from_java(scaled).max()}")
Module 5: Plugin and Command Calls
SciJava commands are the primary way to invoke Fiji plugins programmatically. Commands accept a dict of named parameters mirroring the plugin dialog.
import imagej
ij = imagej.init("sc.fiji:fiji", mode="headless")
# Open a file using Bio-Formats opener command
future = ij.command().run(
"loci.plugins.LociImporter",
True,
{"id": "/data/image.lif", "open_files": True, "autoscale": True}
)
module = future.get()
imp = module.getOutput("imp")
print(f"Opened via Bio-Formats: {imp.getDimensions()}")
# Run Analyze Particles as a SciJava command
ij.io().open("/data/binary_mask.tif")
future = ij.command().run(
"ij.plugin.filter.ParticleAnalyzer",
True,
{
"minSize": 50.0,
"maxSize": float("inf"),
"options": 0, # SHOW_NONE
"measurements": 1, # AREA
}
)
future.get()
print("Analyze Particles command complete")
# Alternatively, run via macro string for simpler plugin invocation
ij.macro.run("""
run("Analyze Particles...", "size=50-Infinity display clear summarize");
""")
Module 6: Results Table and ROI Analysis
Retrieve measurement results from ImageJ's Results table and ROI Manager after running Analyze Particles or other measurement commands.
import imagej
import pandas as pd
ij = imagej.init("sc.fiji:fiji", mode="headless")
# After running Analyze Particles, read the Results table
def results_to_dataframe(ij) -> pd.DataFrame:
"""Convert ImageJ Results table to pandas DataFrame."""
rt = ij.ResultsTable.getResultsTable()
if rt is None or rt.size() == 0:
return pd.DataFrame()
headings = list(rt.getHeadings())
data = {col: [rt.getValue(col, i) for i in range(rt.size())]
for col in headings}
return pd.DataFrame(data)
# Run segmentation + measurement macro
ij.io().open("/data/cells.tif")
ij.macro.run("""
run("Gaussian Blur...", "sigma=1.5");
setAutoThreshold("Otsu dark");
run("Convert to Mask");
run("Analyze Particles...", "size=20-Infinity display clear");
""")
df = results_to_dataframe(ij)
print(f"Found {len(df)} objects")
print(df[["Area", "Mean", "IntDen"]].describe())
df.to_csv("particle_measurements.csv", index=False)
print("Saved particle_measurements.csv")
# Access the ROI Manager
def get_roi_manager(ij):
"""Return the ImageJ ROI Manager instance, creating if needed."""
RoiManager = ij.py.jclass("ij.plugin.frame.RoiManager")
rm = RoiManager.getInstance()
if rm is None:
rm = RoiManager(False) # headless=False means no GUI window
return rm
rm = get_roi_manager(ij)
roi_count = rm.getCount()
print(f"ROIs in manager: {roi_count}")
# Extract bounding boxes for all ROIs
rois = []
for i in range(roi_count):
roi = rm.getRoi(i)
bounds = roi.getBounds()
rois.append({"index": i, "x": bounds.x, "y": bounds.y,
"width": bounds.width, "height": bounds.height})
roi_df = pd.DataFrame(rois)
print(roi_df.head())
Common Workflows
Workflow 1: Automated Fluorescence Quantification
Goal: Open a multi-channel TIFF stack, apply Gaussian blur, threshold nuclei channel, run Analyze Particles, and export per-cell measurements as CSV.
import imagej
import pandas as pd
import numpy as np
from pathlib import Path
ij = imagej.init("sc.fiji:fiji", mode="headless")
def quantify_nuclei(tiff_path: str, output_csv: str,
channel: int = 1, sigma: float = 1.5,
min_size: int = 50) -> pd.DataFrame:
"""
Segment and measure nuclei in a fluorescence TIFF.
Parameters
----------
tiff_path : path to single- or multi-channel TIFF
output_csv : where to save results
channel : 1-based channel index for nuclear stain (e.g., DAPI)
sigma : Gaussian blur radius in pixels
min_size : minimum nucleus area in pixels
"""
# Step 1: Open image
imp = ij.io().open(tiff_path)
print(f"Loaded: {Path(tiff_path).name} dims={imp.getDimensions()}")
# Step 2: Extract channel if multi-channel
if imp.getNChannels() > 1:
imp.setC(channel)
# Step 3: Apply Gaussian blur and threshold via macro
ij.macro.run(f"""
selectWindow("{imp.getTitle()}");
run("Gaussian Blur...", "sigma={sigma}");
setAutoThreshold("Otsu dark");
run("Convert to Mask");
run("Fill Holes");
run("Watershed");
""")
# Step 4: Measure
ij.macro.run(f"""
run("Set Measurements...", "area mean min centroid integrated shape redirect=None decimal=3");
run("Analyze Particles...", "size={min_size}-Infinity display clear include summarize");
""")
# Step 5: Collect results
rt = ij.ResultsTable.getResultsTable()
if rt is None or rt.size() == 0:
print("No objects detected")
return pd.DataFrame()
headings = list(rt.getHeadings())
df = pd.DataFrame(
{col: [rt.getValue(col, i) for i in range(rt.size())]
for col in headings}
)
df["source_file"] = Path(tiff_path).stem
df.to_csv(output_csv, index=False)
print(f"Saved {len(df)} measurements → {output_csv}")
return df
df = quantify_nuclei(
tiff_path="/data/experiment_dapi.tif",
output_csv="nuclei_measurements.csv",
channel=1,
sigma=1.5,
min_size=50
)
print(df[["Area", "Mean", "Circ."]].describe())
Workflow 2: Batch Fiji Macro Processing
Goal: Process a folder of images with an existing Fiji .ijm macro file, collect the Results table from each image into a single DataFrame.
import imagej
import pandas as pd
from pathlib import Path
ij = imagej.init("sc.fiji:fiji", mode="headless")
def run_macro_on_image(ij, image_path: str, macro_file: str) -> pd.DataFrame:
"""Open one image, run a macro file, return its Results table."""
ij.io().open(image_path)
# Clear any previous results before running
ij.macro.run("run(\"Clear Results\");")
# Run macro file (macro must operate on the active image)
ij.macro.runMacroFile(macro_file)
rt = ij.ResultsTable.getResultsTable()
if rt is None or rt.size() == 0:
return pd.DataFrame()
headings = list(rt.getHeadings())
return pd.DataFrame(
{col: [rt.getValue(col, i) for i in range(rt.size())]
for col in headings}
)
input_dir = Path("/data/images")
macro_file = "/scripts/measure_cells.ijm"
output_csv = "batch_results.csv"
all_results = []
image_files = sorted(input_dir.glob("*.tif"))
for img_path in image_files:
print(f"Processing: {img_path.name}")
df = run_macro_on_image(ij, str(img_path), macro_file)
if not df.empty:
df["filename"] = img_path.name
all_results.append(df)
# Close all windows to free memory between images
ij.macro.run("close('*');")
if all_results:
combined = pd.concat(all_results, ignore_index=True)
combined.to_csv(output_csv, index=False)
print(f"Batch complete: {len(image_files)} images, "
f"{len(combined)} total measurements → {output_csv}")
else:
print("No results collected from any image")
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
mode | Initialization | "headless" | "headless", "gui", "interactive" | Controls whether Fiji GUI window opens; use "headless" for scripts |
endpoint | Initialization | "sc.fiji:fiji" | Maven coordinate or /path/to/Fiji.app | Selects ImageJ2 distribution; sc.fiji:fiji includes all Fiji plugins |
sigma (Gaussian blur) | Macro / Ops | 2.0 | 0.5–10.0 | Spatial smoothing radius in pixels; higher reduces noise but blurs edges |
minSize (Analyze Particles) | Plugin / Macro | 0 | pixels² or 0–Infinity | Smallest object area to include; eliminates noise particles |
method (Auto Threshold) | Macro / Ops | "Otsu" | "Otsu", "Triangle", "MaxEntropy", "Huang", "Li" | Threshold algorithm; Otsu works well for bimodal histograms |
measurements bitmask | Results / Macro | varies | OR combination of AREA=1, MEAN=2, CENTROID=4, etc. | Selects which columns appear in the Results table |
| Java heap size | Initialization (env) | JVM default (~25% RAM) | set via JAVA_TOOL_OPTIONS | Limits memory for large stacks; set -Xmx8g for big images |
Best Practices
-
Initialize once per session and reuse
ij: Starting a new JVM is expensive (5–15 seconds). Createijat module level or pass it as a parameter rather than callingimagej.init()inside a loop.# At module top level — initialized once import imagej ij = imagej.init("sc.fiji:fiji", mode="headless") def process(path): imp = ij.io().open(path) # reuse ij ... -
Use conda for Java management: pip-only installs frequently fail because
JAVA_HOMEis not set or the wrong JDK version is onPATH. A conda environment withopenjdk=11avoids 90% of JVM-not-found errors. -
Clear Results and ROI Manager between images in batch loops: ImageJ accumulates results across calls in the same session. Always call
run("Clear Results")andrm.reset()before each image to prevent row contamination.ij.macro.run("run('Clear Results');") rm = get_roi_manager(ij) rm.reset() -
Prefer
ij.macro.run()for simple commands overij.command().run(): Macro strings are shorter, easier to read, and use the same syntax as the Fiji Macro Recorder. Useij.command().run()only when you need programmatic access to command outputs (module return values). -
Convert to NumPy as late as possible:
ij.py.from_imageplus()copies data from the JVM to Python. For multi-step processing inside ImageJ, keep data in ImagePlus or ImgLib2 form and convert only at the end to minimize memory-copy overhead. -
Use
ij.py.to_java()/ij.py.from_java()for ImgLib2 Ops: Theto_imageplus()/from_imageplus()pair works with the classic ImageProcessor;to_java()/from_java()target the modern ImgLib2 type system required byij.op(). -
Set a Fiji update site plugins list during init for reproducibility: Specify a pinned Fiji version (
sc.fiji:fiji:2.14.0) rather thansc.fiji:fiji(latest) so your pipeline behavior does not change when Fiji releases new plugin updates.
Common Recipes
Recipe: TrackMate Headless Spot Detection
When to use: Run TrackMate particle tracking programmatically and retrieve detected spots as a DataFrame without opening the TrackMate GUI.
import imagej
import pandas as pd
ij = imagej.init("sc.fiji:fiji", mode="headless")
# TrackMate headless via macro (scripting interface)
trackmate_macro = """
run("TrackMate", "");
// For fully scripted TrackMate, use the Scripting Interface
// documented at https://imagej.net/plugins/trackmate/scripting
"""
# Scripted TrackMate via Jython-style Java interop
def run_trackmate_headless(ij, imp, radius=3.0, threshold=100.0):
"""Detect spots with LoG detector; return DataFrame of spot coordinates."""
from jnius import autoclass
# Import TrackMate classes
Model = autoclass("fiji.plugin.trackmate.Model")
Settings = autoclass("fiji.plugin.trackmate.Settings")
TrackMate = autoclass("fiji.plugin.trackmate.TrackMate")
LogDetectorFactory = autoclass(
"fiji.plugin.trackmate.detection.LogDetectorFactory")
model = Model()
settings = Settings(imp)
# Configure LoG spot detector
settings.detectorFactory = LogDetectorFactory()
settings.detectorSettings = {
"DO_SUBPIXEL_LOCALIZATION": True,
"RADIUS": radius,
"TARGET_CHANNEL": 1,
"THRESHOLD": threshold,
"DO_MEDIAN_FILTERING": False,
}
tm = TrackMate(model, settings)
tm.process()
# Extract spot table
spots = model.getSpots()
spots.setVisible(True)
records = []
for spot in spots.iterable(True):
records.append({
"id": spot.ID(),
"x": spot.getDoublePosition(0),
"y": spot.getDoublePosition(1),
"z": spot.getDoublePosition(2),
"frame": spot.getFeature("FRAME"),
"quality": spot.getFeature("QUALITY"),
})
return pd.DataFrame(records)
imp = ij.io().open("/data/timelapse.tif")
df = run_trackmate_headless(ij, imp, radius=3.0, threshold=50.0)
print(f"Detected {len(df)} spots across {df['frame'].nunique()} frames")
df.to_csv("spots.csv", index=False)
print(df.head())
Recipe: Convert ImageJ Hyperstack to NumPy 5D Array (TZCYX)
When to use: Import a multi-dimensional Fiji hyperstack into Python as a 5D NumPy array with the standard TZCYX axis order used by most scientific image analysis libraries.
import imagej
import numpy as np
ij = imagej.init("sc.fiji:fiji", mode="headless")
def hyperstack_to_numpy(ij, imp) -> np.ndarray:
"""
Convert an ImageJ hyperstack to a NumPy array with shape (T, Z, C, Y, X).
ImageJ internal order is C-Z-T (slowest to fastest in stack index).
This function reorders to the TZCYX convention used by tifffile, OME, etc.
"""
nC = imp.getNChannels()
nZ = imp.getNSlices()
nT = imp.getNFrames()
H = imp.getHeight()
W = imp.getWidth()
arr = np.zeros((nT, nZ, nC, H, W), dtype=np.uint16)
for t in range(1, nT + 1):
for z in range(1, nZ + 1):
for c in range(1, nC + 1):
idx = imp.getStackIndex(c, z, t)
imp.setSlice(idx)
arr[t-1, z-1, c-1] = ij.py.from_imageplus(imp)
return arr
imp = ij.io().open("/data/4d_experiment.tif")
print(f"ImageJ dims (W,H,C,Z,T): {imp.getDimensions()}")
stack = hyperstack_to_numpy(ij, imp)
print(f"NumPy TZCYX shape: {stack.shape}") # e.g., (10, 15, 2, 512, 512)
print(f"dtype: {stack.dtype}, max: {stack.max()}")
# Save as OME-TIFF with correct axis metadata
import tifffile
tifffile.imwrite(
"output_TZCYX.ome.tif",
stack,
imagej=True,
metadata={"axes": "TZCYX"},
photometric="minisblack"
)
print("Saved output_TZCYX.ome.tif")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
JVMNotFoundException on init | JAVA_HOME not set or wrong JDK version | Install via conda: conda install -c conda-forge openjdk=11; avoid Java 17 |
RuntimeError: Fiji download failed | No internet or corporate proxy | Download Fiji manually from fiji.sc, then use imagej.init("/path/to/Fiji.app") |
java.lang.OutOfMemoryError on large stacks | JVM default heap is too small | Set export JAVA_TOOL_OPTIONS="-Xmx8g" before importing imagej |
Macro run(...) silently does nothing | No active image when macro expects one | Call ij.io().open(path) before running processing macros; check ij.WindowManager.getImageCount() |
| Results table empty after Analyze Particles | Threshold not applied, or mask not binary | Verify mask is 8-bit binary (0/255) with imp.getType() == 0; run Convert to Mask before Analyze Particles |
AttributeError: 'NoneType' object on ij.ResultsTable.getResultsTable() | No measurements run yet in this session | Confirm Analyze Particles macro completed; run ij.macro.run("print(nResults);") to check count |
Plugin class not found (ClassNotFoundException) | Plugin not in this Fiji installation | Add the Fiji update site (e.g., TrackMate) or use sc.fiji:fiji endpoint which includes all default plugins |
gui mode crashes with HeadlessException | No display available (SSH/cluster) | Use mode="headless" for remote environments; GUI mode requires DISPLAY or X11 forwarding |
Related Skills
- scikit-image-processing — pure Python image processing without JVM; use when Fiji plugins are not needed
- napari-image-viewer — interactive multi-dimensional image viewer for Python; complement to PyImageJ for visualization
- trackpy-particle-tracking — Python-native Crocker-Grier SPT; alternative to TrackMate for simple 2D tracking
- cellpose-cell-segmentation — deep learning cell segmentation; can be run standalone or as a Fiji plugin
- omero-integration — OMERO server image management; PyImageJ can process images retrieved via omero-py
References
- PyImageJ documentation — official API reference, initialization guide, data conversion
- PyImageJ GitHub repository — source, issue tracker, notebooks, examples
- ImageJ Ops framework — Ops namespace reference and algorithm catalog
- TrackMate scripting guide — headless TrackMate Java API patterns
- Bio-Formats supported formats — list of proprietary microscopy formats openable via
ij.io().open() - ImageJ Macro Language reference — built-in functions for
ij.macro.run()strings
scikit-image — Scientific Image Processing
Overview
scikit-image is a Python library for image processing in the SciPy ecosystem. It provides algorithms for reading/writing images, filtering (noise reduction, edge detection), geometric transforms, segmentation (thresholding, watershed, active contours), object measurement (area, intensity, shape descriptors), and feature detection. Images are represented as NumPy arrays, enabling seamless integration with NumPy, SciPy, matplotlib, and pandas. Widely used for fluorescence microscopy, histology, and general bioimage analysis.
When to Use
- Preprocessing fluorescence microscopy images: background subtraction, denoising, illumination correction
- Segmenting cells, nuclei, or organelles using thresholding or watershed
- Measuring object properties: area, perimeter, intensity statistics, shape descriptors
- Applying morphological operations: erosion, dilation, opening, closing, fill holes
- Detecting keypoints or local features in biological images
- Converting between image formats and color spaces
- Use
OpenCVinstead for real-time video processing or GPU-accelerated operations - For deep-learning cell segmentation, use
CellPoseinstead (better accuracy for touching cells) - Use
napariinstead for interactive multi-dimensional image visualization and annotation - For whole-slide image tiling, use
PathMLorhistolabinstead
Prerequisites
- Python packages:
scikit-image,numpy,scipy,matplotlib - Input requirements: Images as files (TIFF, PNG, JPEG) or NumPy arrays; fluorescence images as 2D/3D grayscale arrays
- Environment: Python 3.9+
pip install scikit-image numpy scipy matplotlib
# For reading proprietary microscopy formats
pip install tifffile aicsimageio
# Verify
python -c "import skimage; print(skimage.__version__)"
Quick Start
from skimage import io, filters, measure
import numpy as np
# Load → denoise → threshold → measure
img = io.imread("cells.tif")
img_smooth = filters.gaussian(img, sigma=1.5)
threshold = filters.threshold_otsu(img_smooth)
binary = img_smooth > threshold
regions = measure.regionprops(measure.label(binary))
print(f"Found {len(regions)} objects")
print(f"Mean area: {np.mean([r.area for r in regions]):.1f} px²")
Core API
Module 1: Image I/O and Data Types
from skimage import io, img_as_float, img_as_uint
import numpy as np
# Read single image
img = io.imread("nuclei.tif")
print(f"Shape: {img.shape}, dtype: {img.dtype}") # (512, 512), uint16
# Read image collection from directory
from skimage import io as ski_io
images = ski_io.ImageCollection("data/*.tif")
print(f"Loaded {len(images)} images")
# Type conversions (critical for correct arithmetic)
img_f = img_as_float(img) # uint16 → float64, range [0, 1]
img_u8 = (img_f * 255).astype(np.uint8) # → 8-bit
# Save image
io.imsave("output.tif", img_u8)
# Multi-channel fluorescence (TIFF with CZYX or ZCYX dims)
import tifffile
stack = tifffile.imread("multichannel.tif") # shape: (C, Z, Y, X)
dapi = stack[0] # DAPI channel
gfp = stack[1] # GFP channel
print(f"DAPI: {dapi.shape}, GFP: {gfp.shape}")
# Maximum intensity projection along Z
mip = dapi.max(axis=0)
io.imsave("dapi_mip.tif", mip)
Module 2: Filters and Preprocessing
from skimage import filters, restoration
import numpy as np
# Gaussian blur (denoising, smoothing)
from skimage.filters import gaussian
smoothed = gaussian(img, sigma=2.0)
# Median filter (salt-and-pepper noise removal)
from skimage.filters import median
from skimage.morphology import disk
denoised = median(img, footprint=disk(3))
# Top-hat transform (background subtraction for uneven illumination)
from skimage.morphology import white_tophat, disk
background_removed = white_tophat(img, footprint=disk(50))
print(f"Background removed: range [{background_removed.min()}, {background_removed.max()}]")
# Edge detection
from skimage.filters import sobel, laplace, prewitt
edges_sobel = sobel(img_as_float(img))
edges_laplace = laplace(img_as_float(img))
# Difference of Gaussians (blob-like structure detection)
from skimage.filters import difference_of_gaussians
blob_enhanced = difference_of_gaussians(img_as_float(img), low_sigma=1, high_sigma=3)
# Contrast enhancement (CLAHE: local histogram equalization)
from skimage.exposure import equalize_adapthist
enhanced = equalize_adapthist(img_as_float(img), clip_limit=0.03)
Module 3: Thresholding and Segmentation
from skimage import filters, morphology, segmentation
from skimage.color import label2rgb
import numpy as np
# Automatic thresholding methods
from skimage.filters import (threshold_otsu, threshold_li,
threshold_triangle, threshold_yen)
img_f = img_as_float(img)
print(f"Otsu: {threshold_otsu(img_f):.3f}")
print(f"Li: {threshold_li(img_f):.3f}")
# Apply threshold and clean binary mask
binary = img_f > threshold_otsu(img_f)
binary_clean = morphology.remove_small_objects(binary, min_size=50)
binary_filled = morphology.remove_small_holes(binary_clean, area_threshold=100)
# Watershed segmentation (separate touching objects)
from skimage.segmentation import watershed
from skimage.feature import peak_local_max
from scipy import ndimage as ndi
# Distance transform → local maxima → watershed
distance = ndi.distance_transform_edt(binary_filled)
coords = peak_local_max(distance, min_distance=20, labels=binary_filled)
mask = np.zeros(distance.shape, dtype=bool)
mask[tuple(coords.T)] = True
markers = ndi.label(mask)[0]
labels = watershed(-distance, markers, mask=binary_filled)
print(f"Segmented objects: {labels.max()}")
overlay = label2rgb(labels, image=img_f, bg_label=0)
Module 4: Morphological Operations
from skimage.morphology import (erosion, dilation, opening, closing,
disk, ball, binary_erosion, binary_dilation)
# Erosion and dilation
eroded = erosion(binary, footprint=disk(3))
dilated = dilation(binary, footprint=disk(5))
# Opening: erosion then dilation (removes small objects, smooths edges)
opened = opening(binary, footprint=disk(3))
# Closing: dilation then erosion (fills small holes)
closed = closing(binary, footprint=disk(5))
# Skeletonization
from skimage.morphology import skeletonize
skeleton = skeletonize(binary)
print(f"Skeleton pixels: {skeleton.sum()}")
Module 5: Measurement and Region Properties
from skimage import measure
import pandas as pd
# Label connected components
labeled = measure.label(binary_filled)
# Extract region properties
props = measure.regionprops(labeled, intensity_image=img_as_float(img))
# Convert to DataFrame
data = []
for r in props:
data.append({
"label": r.label,
"area": r.area,
"perimeter": r.perimeter,
"eccentricity": r.eccentricity,
"mean_intensity": r.mean_intensity,
"max_intensity": r.max_intensity,
"centroid_y": r.centroid[0],
"centroid_x": r.centroid[1],
"bbox": r.bbox,
})
df = pd.DataFrame(data)
print(f"Objects: {len(df)}")
print(df[["area", "mean_intensity", "eccentricity"]].describe().round(2))
# Filter by property thresholds
cells = df[(df["area"] > 100) & (df["area"] < 5000) & (df["eccentricity"] < 0.9)]
print(f"Valid cells: {len(cells)}")
# Measure co-localization: fraction of channel-1 signal in channel-2 positive mask
from skimage.measure import regionprops_table
import numpy as np
# For multi-channel images
table = regionprops_table(
labeled, intensity_image=np.stack([dapi, gfp], axis=-1),
properties=["label", "area", "mean_intensity"]
)
Module 6: Feature Detection and Transforms
from skimage.feature import blob_log, blob_dog, corner_harris, corner_peaks
from skimage import transform
# Laplacian of Gaussian blob detection (nuclei, puncta)
blobs = blob_log(img_as_float(img), min_sigma=5, max_sigma=20,
num_sigma=5, threshold=0.05)
print(f"Blobs detected: {len(blobs)}")
# blobs columns: [y, x, sigma] where radius = sqrt(2) * sigma
# Difference of Gaussians (faster alternative)
blobs_dog = blob_dog(img_as_float(img), min_sigma=5, max_sigma=20, threshold=0.02)
# Geometric transforms
from skimage import transform
# Rescale
img_small = transform.rescale(img_as_float(img), 0.5)
# Rotate
img_rotated = transform.rotate(img_as_float(img), angle=15, resize=True)
# Affine registration (align two images)
from skimage.registration import phase_cross_correlation
shift, error, _ = phase_cross_correlation(ref_img, moving_img)
print(f"Alignment shift: {shift} px, error: {error:.4f}")
Key Concepts
Image Arrays and Conventions
scikit-image represents images as NumPy arrays. Shape conventions:
| Image Type | Shape | dtype |
|---|---|---|
| Grayscale 2D | (H, W) | uint8, uint16, float64 |
| RGB color | (H, W, 3) | uint8 |
| Multichannel | (H, W, C) | any |
| Z-stack | (Z, H, W) | any |
dtype matters: Most algorithms expect float64 in [0, 1]. Use img_as_float(img) before processing; convert back with img_as_uint(img) for saving.
Common Workflows
Workflow 1: Fluorescence Cell Segmentation and Measurement
Goal: Segment DAPI-stained nuclei and measure GFP fluorescence per nucleus.
from skimage import io, filters, morphology, measure, img_as_float
from skimage.segmentation import watershed
from skimage.feature import peak_local_max
from scipy import ndimage as ndi
import pandas as pd
import numpy as np
import tifffile
# Load 2-channel image (DAPI=ch0, GFP=ch1)
img = tifffile.imread("cells.tif")
dapi = img_as_float(img[0])
gfp = img_as_float(img[1])
# Segment nuclei from DAPI channel
dapi_smooth = filters.gaussian(dapi, sigma=2)
threshold = filters.threshold_otsu(dapi_smooth)
binary = dapi_smooth > threshold
binary = morphology.remove_small_objects(binary, min_size=200)
binary = morphology.remove_small_holes(binary, area_threshold=500)
# Watershed to separate touching nuclei
distance = ndi.distance_transform_edt(binary)
coords = peak_local_max(distance, min_distance=30, labels=binary)
mask = np.zeros_like(distance, dtype=bool)
mask[tuple(coords.T)] = True
markers = ndi.label(mask)[0]
labels = watershed(-distance, markers, mask=binary)
# Measure GFP per nucleus
props = measure.regionprops(labels, intensity_image=gfp)
df = pd.DataFrame([{
"nucleus_id": p.label,
"area_px2": p.area,
"gfp_mean": p.mean_intensity,
"gfp_max": p.max_intensity,
} for p in props])
df.to_csv("nucleus_measurements.csv", index=False)
print(f"Nuclei: {len(df)}, mean GFP: {df['gfp_mean'].mean():.3f}")
Workflow 2: Batch Image Processing
Goal: Apply the same preprocessing and measurement pipeline to a folder of images.
from pathlib import Path
from skimage import io, filters, measure, img_as_float, morphology
import pandas as pd
results = []
for img_path in sorted(Path("data/").glob("*.tif")):
img = img_as_float(io.imread(img_path))
if img.ndim == 3:
img = img.mean(axis=-1) # convert RGB to grayscale
# Preprocess
smooth = filters.gaussian(img, sigma=1.5)
thresh = filters.threshold_otsu(smooth)
binary = morphology.remove_small_objects(smooth > thresh, min_size=50)
# Measure
labeled = measure.label(binary)
props = measure.regionprops(labeled, intensity_image=img)
for p in props:
results.append({
"image": img_path.stem,
"object_id": p.label,
"area": p.area,
"mean_intensity": p.mean_intensity,
})
df = pd.DataFrame(results)
df.to_csv("batch_results.csv", index=False)
print(f"Processed {df['image'].nunique()} images, {len(df)} objects total")
Key Parameters
| Function | Parameter | Default | Range/Options | Effect |
|---|---|---|---|---|
gaussian | sigma | 1.0 | 0.5–10+ | Smoothing kernel size; larger = more blur |
median | footprint | disk(1) | disk(1–10) | Median filter neighborhood |
threshold_otsu | — | — | — | Returns automatic threshold (no tuning) |
remove_small_objects | min_size | 64 | any integer | Remove objects smaller than N pixels |
watershed | — | — | — | Uses marker positions from peak_local_max |
peak_local_max | min_distance | 1 | 5–50 | Min separation between detected peaks (px) |
blob_log | min_sigma / max_sigma | 1/50 | dependent | Expected blob radius range |
regionprops | intensity_image | None | array | Image for intensity measurements |
Best Practices
-
Always check dtype before processing: Operations like subtraction on
uint8silently clip to 0. Convert to float:img = img_as_float(img)as the first step. -
Visualize intermediate results: For every segmentation pipeline, plot the binary mask overlaid on the original before measuring. Silent segmentation errors are the most common failure mode.
-
Tune thresholds on representative samples: Otsu works well for bimodal histograms. For difficult images, compare Otsu/Li/Triangle with
try_all_threshold(img)from skimage. -
Use watershed for touching objects: Simple thresholding cannot separate touching nuclei. Always apply watershed with distance transform markers for densely packed cells.
-
Measure in physical units: Convert pixel measurements to microns using the pixel size from microscope metadata:
area_um2 = area_px * pixel_size_um**2. -
Validate with known samples: Before batch processing, verify the pipeline on 3–5 images with manually counted objects. Spot-check object counts against expectations.
Common Recipes
Recipe: Measure Spot Intensity in Fluorescence Images
from skimage import io, filters, measure, img_as_float
from skimage.feature import blob_log
import numpy as np
img = img_as_float(io.imread("spots.tif"))
blobs = blob_log(img, min_sigma=2, max_sigma=8, num_sigma=5, threshold=0.05)
intensities = []
for y, x, sigma in blobs:
r = int(np.ceil(np.sqrt(2) * sigma))
region = img[max(0,int(y-r)):int(y+r), max(0,int(x-r)):int(x+r)]
intensities.append({"y": y, "x": x, "radius": r, "mean_intensity": region.mean()})
import pandas as pd
df = pd.DataFrame(intensities)
print(f"Spots: {len(df)}, mean intensity: {df['mean_intensity'].mean():.4f}")
Recipe: Binary Mask from Multiple Channels
from skimage import filters, morphology, img_as_float
import numpy as np
# Segment objects positive in BOTH channels
ch1 = img_as_float(dapi)
ch2 = img_as_float(gfp)
mask1 = ch1 > filters.threshold_otsu(ch1)
mask2 = ch2 > filters.threshold_otsu(ch2)
combined_mask = mask1 & mask2 # objects in both channels
combined_mask = morphology.binary_closing(combined_mask)
print(f"Co-positive pixels: {combined_mask.sum()}")
Recipe: Save Labeled Overlay as Publication Figure
from skimage.color import label2rgb
from skimage import io, img_as_ubyte
import matplotlib.pyplot as plt
overlay = label2rgb(labels, image=img_as_float(dapi), bg_label=0, alpha=0.5)
fig, axes = plt.subplots(1, 2, figsize=(12, 6))
axes[0].imshow(dapi, cmap="gray")
axes[0].set_title(f"DAPI (raw)")
axes[1].imshow(overlay)
axes[1].set_title(f"Segmented ({labels.max()} nuclei)")
for ax in axes:
ax.axis("off")
plt.tight_layout()
plt.savefig("segmentation_overlay.png", dpi=300, bbox_inches="tight")
print("Saved segmentation_overlay.png")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
OverflowError in arithmetic | Integer overflow from uint8/uint16 ops | Convert to float first: img = img_as_float(img) |
| Otsu threshold finds one class | Bimodal distribution absent | Try threshold_li() or try_all_threshold(img) for comparison |
| Watershed over-segments | Markers too close together | Increase min_distance in peak_local_max; smooth distance map |
| All objects merged in binary | Threshold too low | Check plt.hist(img.ravel()) for histogram; manually adjust |
regionprops missing intensity stats | intensity_image not provided | Pass: regionprops(labels, intensity_image=img) |
| 3D images processed as 2D | Z-stack not detected | Check shape: img.shape; process per slice or use 3D functions |
| Tiny noise objects in binary | Threshold too aggressive | Apply morphology.remove_small_objects(binary, min_size=50) |
Related Skills
- pathml — whole-slide image processing using scikit-image under the hood
- matplotlib-scientific-plotting — visualizing segmentation results and measurement distributions
- histolab-wsi-processing — scikit-image-compatible preprocessing for H&E WSI tiles
References
- scikit-image documentation — API reference and tutorials
- GitHub: scikit-image/scikit-image — source code and examples
- van der Walt et al. (2014) "scikit-image: image processing in Python" — PeerJ 2:e453
- scikit-image examples gallery — annotated code examples by category
trackpy
Overview
trackpy is a Python library for single-particle tracking (SPT) in video microscopy. It implements the Crocker-Grier algorithm to locate bright spots in each frame with subpixel precision, then links those positions across frames into continuous trajectories. From trajectories, trackpy computes mean squared displacement (MSD), diffusion coefficients, and motion classifications (confined, normal, directed). It handles 2D fluorescence videos, 3D confocal z-stacks, and large image sequences via memory-efficient streaming through the pims image reader library.
When to Use
- You have a fluorescence microscopy video of labeled particles (quantum dots, fluorescent beads, vesicles, receptors) and need to extract individual trajectories and diffusion coefficients.
- You want to measure particle mobility: compute MSD curves and distinguish Brownian diffusion, directed motion, or confined motion from single-particle tracks.
- You are analyzing colloid dynamics, lipid membrane diffusion, intracellular cargo transport, or virus-cell interactions where you need per-particle trajectory data.
- You need 3D tracking from confocal z-stack time series to capture out-of-plane motion of particles or organelles.
- You want to apply drift correction to remove stage drift before computing intrinsic particle motion statistics.
- You need ensemble MSD averaged across hundreds of tracks to extract population-level diffusion behavior with statistical power.
- Use
TrackMate(Fiji/ImageJ plugin) instead when you need a graphical interface, manual curation of tracks, or integration with biological object segmenters (Cellpose, StarDist). - Use
napariwithnapari-trackpyinstead when you want interactive visualization and manual editing of trajectories alongside image data.
Prerequisites
- Python packages:
trackpy,pims,pandas,numpy,matplotlib,scipy - Data requirements: Grayscale or single-channel image sequence (TIF stack, AVI, or directory of PNG/TIF frames); particles should appear as bright Gaussian spots on a darker background (or use
invert=Truefor dark spots on bright background) - Environment: Works in Jupyter notebooks and scripts;
pimshandles most microscopy formats; for ND2 or CZI files installpims-nd2oraicsimageio
pip install trackpy pims pandas numpy matplotlib scipy
# For reading multi-channel or proprietary formats:
pip install pims[bioformats] # Bioformats via JPype
pip install aicsimageio # ND2, CZI, LIF via AICSImageIO
Quick Start
import trackpy as tp
import pims
# Load a TIF image stack (T frames × Y × X)
frames = pims.open("particles.tif") # shape: (T, Y, X)
# Locate particles in all frames
f = tp.batch(frames, diameter=11, minmass=500)
print(f"Found {len(f)} particle detections across {f['frame'].nunique()} frames")
# Link into trajectories
t = tp.link(f, search_range=5, memory=3)
# Remove short-lived tracks (fewer than 10 frames)
t = tp.filter_stubs(t, threshold=10)
print(f"Retained {t['particle'].nunique()} trajectories")
# Compute ensemble MSD
imsd = tp.imsd(t, mpp=0.16, fps=10) # mpp: microns per pixel, fps: frames per second
print(imsd.head())
Core API
Module 1: tp.locate() — Single-Frame Particle Detection
tp.locate() finds bright circular features in one image frame using a bandpass filter followed by local maximum detection. It returns a DataFrame with subpixel x/y positions, integrated mass, signal, and eccentricity for each detected particle.
import trackpy as tp
import pims
import matplotlib.pyplot as plt
frames = pims.open("particles.tif")
frame0 = frames[0] # single 2D array
# Locate particles: diameter must be odd integer, roughly matching spot size in pixels
f0 = tp.locate(frame0, diameter=11, minmass=300, maxsize=None, separation=None)
print(f"Detected {len(f0)} particles in frame 0")
print(f0[['x', 'y', 'mass', 'size', 'ecc']].head())
# x, y: subpixel centroid; mass: integrated brightness; size: Gaussian width; ecc: eccentricity (0=circular)
# Diagnostic plot: annotate detected particles on the raw frame
fig, ax = plt.subplots(figsize=(8, 8))
tp.annotate(f0, frame0, ax=ax, imshow_style={"cmap": "gray"})
ax.set_title(f"Frame 0: {len(f0)} particles detected")
plt.tight_layout()
plt.savefig("locate_diagnostic.png", dpi=150)
print("Saved locate_diagnostic.png")
Module 2: tp.batch() — Multi-Frame Detection
tp.batch() applies tp.locate() to every frame in an image sequence and concatenates results into a single DataFrame with a frame column. It accepts any pims-compatible image reader or a list of 2D arrays.
import trackpy as tp
import pims
frames = pims.open("particles.tif")
# Locate particles across all frames (same parameters as tp.locate)
f = tp.batch(frames, diameter=11, minmass=300, processes=1)
# processes=1 uses serial processing; set processes="auto" for multicore (requires joblib)
print(f"Total detections: {len(f)}")
print(f"Frames with data: {f['frame'].nunique()} / {len(frames)}")
print(f"Mean particles per frame: {len(f)/f['frame'].nunique():.1f}")
print(f.groupby('frame').size().describe())
# Mass histogram: use to choose minmass cutoff
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(6, 4))
f['mass'].hist(bins=40, ax=ax)
ax.axvline(300, color='red', linestyle='--', label='minmass=300')
ax.set_xlabel("Integrated mass")
ax.set_ylabel("Count")
ax.set_title("Mass distribution of detections")
ax.legend()
plt.tight_layout()
plt.savefig("mass_histogram.png", dpi=150)
print("Saved mass_histogram.png — use to refine minmass cutoff")
Module 3: tp.link() — Trajectory Linking
tp.link() connects particle detections across frames into trajectories by solving a bipartite assignment problem (Hungarian algorithm). It adds a particle column (integer trajectory ID) to the positions DataFrame. search_range (pixels) is the maximum displacement between frames; memory allows a particle to disappear for up to N frames before being dropped.
import trackpy as tp
import pims
frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=300)
# Link: search_range in pixels; memory handles brief disappearances (blinking, out-of-focus)
t = tp.link(f, search_range=5, memory=3)
print(f"Number of unique trajectories: {t['particle'].nunique()}")
print(f"Trajectory length distribution:")
print(t.groupby('particle').size().describe())
# Visualize all trajectories overlaid on the first frame
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(8, 8))
tp.plot_traj(t, superimpose=frames[0], ax=ax)
ax.set_title(f"{t['particle'].nunique()} trajectories")
plt.tight_layout()
plt.savefig("trajectories.png", dpi=150)
print("Saved trajectories.png")
Module 4: tp.filter_stubs() — Short-Track Removal
tp.filter_stubs() removes trajectories shorter than a given number of frames. Short tracks arise from noise detections, particles entering/leaving the field of view, or linking errors. Removing them improves MSD reliability because short tracks contribute high-variance MSD estimates at long lag times.
import trackpy as tp
import pims
frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=300)
t = tp.link(f, search_range=5, memory=3)
before = t['particle'].nunique()
t_filt = tp.filter_stubs(t, threshold=10) # keep only tracks with ≥10 frames
after = t_filt['particle'].nunique()
print(f"Tracks before filtering: {before}")
print(f"Tracks after filtering (≥10 frames): {after}")
print(f"Removed {before - after} short tracks ({100*(before-after)/before:.1f}%)")
Module 5: MSD Analysis — tp.imsd() and tp.emsd()
tp.imsd() computes per-particle mean squared displacement as a function of lag time, returning a DataFrame (lag time as index, particle ID as columns). tp.emsd() computes the ensemble-averaged MSD across all particles. Both require the physical scale (mpp, microns per pixel) and frame rate (fps).
import trackpy as tp
import pims
import matplotlib.pyplot as plt
frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=300)
t = tp.link(f, search_range=5, memory=3)
t = tp.filter_stubs(t, threshold=10)
mpp = 0.16 # microns per pixel (from microscope calibration)
fps = 10.0 # frames per second
# Individual MSD curves (one column per particle)
imsd = tp.imsd(t, mpp=mpp, fps=fps, max_lagtime=100)
print(f"IMSD shape: {imsd.shape}") # (lag times) × (particles)
# Ensemble MSD
emsd = tp.emsd(t, mpp=mpp, fps=fps, max_lagtime=100)
print(f"EMSD at lag 1 s: {emsd.iloc[0]:.4f} µm²")
# Plot ensemble MSD and fit diffusion coefficient
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import linregress
mpp = 0.16
fps = 10.0
# Fit MSD = 4*D*t (2D Brownian) over first 10 lag times
lag_s = emsd.index.values[:10] # lag times in seconds
msd_vals = emsd.values[:10]
slope, intercept, r, p, se = linregress(lag_s, msd_vals)
D = slope / 4 # diffusion coefficient in µm²/s
print(f"Diffusion coefficient D = {D:.4f} µm²/s (R²={r**2:.3f})")
fig, ax = plt.subplots(figsize=(6, 5))
ax.plot(emsd.index, emsd.values, 'o-', label='Ensemble MSD')
ax.plot(lag_s, slope * lag_s + intercept, 'r--', label=f'Fit: D={D:.4f} µm²/s')
ax.set_xlabel("Lag time (s)")
ax.set_ylabel("MSD (µm²)")
ax.set_title("Ensemble Mean Squared Displacement")
ax.legend()
plt.tight_layout()
plt.savefig("emsd.png", dpi=150)
print("Saved emsd.png")
Module 6: Motion Analysis — Characterize and Drift Correction
tp.motion.characterize() computes per-trajectory statistics (mean velocity, net displacement, straightness). tp.subtract_drift() removes bulk stage drift from trajectories before MSD analysis.
import trackpy as tp
import pims
frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=300)
t = tp.link(f, search_range=5, memory=3)
t = tp.filter_stubs(t, threshold=10)
# Estimate and subtract drift (bulk movement of the sample/stage)
drift = tp.compute_drift(t)
print("Drift (first 5 frames):")
print(drift.head())
t_corrected = tp.subtract_drift(t.copy(), drift)
print(f"Drift subtracted from {t_corrected['particle'].nunique()} trajectories")
import trackpy as tp
# Characterize individual trajectories (requires tp.motion module)
from trackpy import motion
# Per-particle summary statistics
char = motion.characterize(t, mpp=0.16, fps=10.0)
print(char.columns.tolist())
# Columns: 'alpha' (anomalous exponent), 'D_app' (apparent diffusion), 'r^2' (fit quality)
print(char[['alpha', 'D_app']].describe())
# alpha ~ 1.0: Brownian; alpha < 1: confined/subdiffusion; alpha > 1: directed/superdiffusion
Common Workflows
Workflow 1: Full 2D Tracking Pipeline with MSD and Diffusion Coefficient
Goal: Load a fluorescence video, locate and link particles across all frames, filter short tracks, compute MSD, and extract diffusion coefficients.
import trackpy as tp
import pims
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import linregress
# ── 1. Load image sequence ──────────────────────────────────────────────────
frames = pims.open("fluorescence_video.tif") # (T, Y, X) grayscale TIF stack
print(f"Loaded {len(frames)} frames, frame shape: {frames.frame_shape}")
# ── 2. Tune detection on a single frame ─────────────────────────────────────
f0 = tp.locate(frames[0], diameter=11, minmass=400)
print(f"Frame 0: {len(f0)} particles detected")
# Adjust diameter (odd integer ≥ spot size) and minmass until count looks right
# ── 3. Batch detect across all frames ───────────────────────────────────────
f = tp.batch(frames, diameter=11, minmass=400, processes=1)
print(f"Total detections: {len(f)} across {f['frame'].nunique()} frames")
# ── 4. Link into trajectories ────────────────────────────────────────────────
t = tp.link(f, search_range=6, memory=3)
print(f"Unique trajectories before filtering: {t['particle'].nunique()}")
# ── 5. Remove short trajectories ─────────────────────────────────────────────
t = tp.filter_stubs(t, threshold=15)
print(f"Trajectories after filtering (≥15 frames): {t['particle'].nunique()}")
# ── 6. Subtract stage drift ───────────────────────────────────────────────────
drift = tp.compute_drift(t)
t = tp.subtract_drift(t.copy(), drift)
# ── 7. Compute MSD ────────────────────────────────────────────────────────────
mpp = 0.16 # µm/pixel — from microscope calibration
fps = 10.0 # frames per second
emsd = tp.emsd(t, mpp=mpp, fps=fps, max_lagtime=50)
imsd = tp.imsd(t, mpp=mpp, fps=fps, max_lagtime=50)
# ── 8. Fit diffusion coefficient from linear regime (first 10 points) ─────────
n_fit = 10
lag_s = emsd.index.values[:n_fit]
msd_v = emsd.values[:n_fit]
slope, intercept, r, _, _ = linregress(lag_s, msd_v)
D = slope / 4 # MSD = 4Dt for 2D Brownian
print(f"Diffusion coefficient D = {D:.4f} µm²/s (R²={r**2:.3f})")
# ── 9. Plot ────────────────────────────────────────────────────────────────────
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
# Individual + ensemble MSD
axes[0].plot(imsd.index, imsd.values, alpha=0.2, color='steelblue', linewidth=0.8)
axes[0].plot(emsd.index, emsd.values, 'k-', linewidth=2, label='Ensemble MSD')
axes[0].plot(lag_s, slope * lag_s + intercept, 'r--', label=f'D={D:.4f} µm²/s')
axes[0].set_xlabel("Lag time (s)")
axes[0].set_ylabel("MSD (µm²)")
axes[0].set_title("MSD: individual (blue) + ensemble (black)")
axes[0].legend()
# Trajectory overlay
tp.plot_traj(t, superimpose=frames[0], ax=axes[1])
axes[1].set_title(f"{t['particle'].nunique()} trajectories")
plt.tight_layout()
plt.savefig("tracking_results.png", dpi=150, bbox_inches="tight")
print("Saved tracking_results.png")
# ── 10. Export trajectories ───────────────────────────────────────────────────
t.to_csv("trajectories.csv", index=False)
emsd.to_csv("ensemble_msd.csv")
print("Exported trajectories.csv and ensemble_msd.csv")
Workflow 2: 3D Particle Tracking from Confocal Z-Stacks
Goal: Track particles in 3D from a time-series of confocal z-stacks (T × Z × Y × X), link in 3D, and compute 3D MSD.
import trackpy as tp
import pims
import numpy as np
import matplotlib.pyplot as plt
# ── 1. Load 4D image stack (T × Z × Y × X) ─────────────────────────────────
# pims opens multi-page TIF; reshape into (T, Z, Y, X) as needed
raw = pims.open("confocal_3d_timeseries.tif")
# Assume each "frame" in pims is one Z-slice; reshape to (T, Z, Y, X)
T, n_z = 50, 20 # adjust to match acquisition
frames_4d = np.array(raw).reshape(T, n_z, raw.frame_shape[0], raw.frame_shape[1])
print(f"4D stack shape: {frames_4d.shape}") # (T, Z, Y, X)
# ── 2. Detect in 3D (tp.locate works on 3D arrays) ─────────────────────────
# For 3D, pass a single 3D volume; diameter can be (z_diam, y_diam, x_diam)
f0_3d = tp.locate(frames_4d[0], diameter=(7, 11, 11), minmass=2000)
print(f"3D detections in t=0: {len(f0_3d)}")
print(f0_3d[['x', 'y', 'z', 'mass']].head())
# ── 3. Batch detect across all time points ──────────────────────────────────
detections = []
for t_idx in range(T):
frame_3d = frames_4d[t_idx] # Z × Y × X
detected = tp.locate(frame_3d, diameter=(7, 11, 11), minmass=2000)
detected['frame'] = t_idx
detections.append(detected)
import pandas as pd
f3d = pd.concat(detections, ignore_index=True)
print(f"Total 3D detections: {len(f3d)}")
# ── 4. Link in 3D ────────────────────────────────────────────────────────────
# search_range in pixels; use a 3-tuple (z, y, x) for anisotropic voxels
t3d = tp.link(f3d, search_range=(3, 6, 6), memory=2)
t3d = tp.filter_stubs(t3d, threshold=10)
print(f"3D trajectories: {t3d['particle'].nunique()}")
# ── 5. Compute 3D MSD ────────────────────────────────────────────────────────
mpp_xy = 0.16 # µm/pixel in x, y
mpp_z = 0.30 # µm/pixel in z (z-step size)
fps = 1.0 # z-stack volume rate
# Scale z coordinates to µm
t3d_um = t3d.copy()
t3d_um['x'] *= mpp_xy
t3d_um['y'] *= mpp_xy
t3d_um['z'] *= mpp_z
emsd_3d = tp.emsd(t3d_um, mpp=1.0, fps=fps, max_lagtime=20) # mpp=1 since already in µm
print(f"3D ensemble MSD (lag=1 s): {emsd_3d.iloc[0]:.4f} µm²")
# ── 6. Fit 3D diffusion coefficient (MSD = 6Dt for 3D) ─────────────────────
from scipy.stats import linregress
lag_s = emsd_3d.index.values[:8]
slope, _, r, _, _ = linregress(lag_s, emsd_3d.values[:8])
D_3d = slope / 6
print(f"3D Diffusion coefficient D = {D_3d:.4f} µm²/s (R²={r**2:.3f})")
t3d.to_csv("trajectories_3d.csv", index=False)
print("Saved trajectories_3d.csv")
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
diameter | locate, batch | required | odd integer ≥ 3 (or tuple for 3D) | Approximate particle diameter in pixels; must be odd. Too small: split detections. Too large: merged detections |
minmass | locate, batch | 100 | 0 to ∞ | Minimum integrated brightness; primary filter against noise. Start at 0, plot mass histogram, set to separate noise peak |
search_range | link | required | 1–50 pixels | Max displacement between frames. Set to ~1.5× max expected per-frame movement |
memory | link | 0 | 0–10 frames | Frames a particle may be absent before track is broken; useful for blinking fluorophores |
threshold | filter_stubs | 1 | integer ≥ 1 | Minimum track length in frames; short tracks have unreliable MSD |
max_lagtime | imsd, emsd | 100 | integer | Maximum lag time in frames for MSD calculation; use ~10–20% of total frames for reliability |
mpp | imsd, emsd | 1 | float > 0 | Microns per pixel; converts pixel units to physical units (µm) |
fps | imsd, emsd | 1 | float > 0 | Frames per second; converts frame lag to seconds |
separation | locate, batch | diameter+1 | integer | Minimum center-to-center distance between features; prevents double-counting dense particles |
invert | locate, batch | False | True, False | Set True for dark particles on bright background (transmitted light imaging) |
Best Practices
-
Always tune
diameterandminmasson a single frame first: Runtp.locate()on one representative frame and usetp.annotate()to visually check detections before committing totp.batch(). Over-detection wastes time; under-detection misses particles.f0 = tp.locate(frames[0], diameter=11, minmass=200) tp.annotate(f0, frames[0]) # visual check in Jupyter -
Set
search_rangeconservatively: Too large a search range causes spurious links between unrelated particles in dense samples. Estimate typical per-frame displacement fromtp.locate()output scatter before linking. -
Subtract drift before computing MSD: Stage drift inflates MSD, causing overestimation of D. Always call
tp.compute_drift()+tp.subtract_drift()beforetp.emsd(). -
Use only the linear regime for diffusion coefficient fitting: MSD curves become noisy at long lag times (few track pairs contribute). Fit only the first 10–20% of available lag times. Use log-log slope to detect non-Brownian behavior before fitting.
-
Do not mix
mppunits between locate and MSD steps:tp.locate()returns positions in pixels.mppis applied only intp.imsd()/tp.emsd(). Avoid rescaling positions manually before linking, as this breaks the pixel-unit search_range. -
For 3D tracking with anisotropic voxels: Pass
diameterandsearch_rangeas tuples matching(z, y, x)axis order. The z-step is usually 2-5× coarser than xy pixel size; set the z component ofdiameterandsearch_rangeaccordingly.
Common Recipes
Recipe: Drift Correction and Corrected MSD Comparison
When to use: Compare raw vs drift-corrected MSD to assess stage drift contribution.
import trackpy as tp
import pims
import matplotlib.pyplot as plt
frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=400, processes=1)
t = tp.link(f, search_range=6, memory=3)
t = tp.filter_stubs(t, threshold=15)
mpp, fps = 0.16, 10.0
# MSD without drift correction
emsd_raw = tp.emsd(t, mpp=mpp, fps=fps, max_lagtime=50)
# Subtract drift
drift = tp.compute_drift(t)
t_corr = tp.subtract_drift(t.copy(), drift)
emsd_corr = tp.emsd(t_corr, mpp=mpp, fps=fps, max_lagtime=50)
fig, ax = plt.subplots(figsize=(6, 5))
ax.loglog(emsd_raw.index, emsd_raw.values, 'r--', label='Raw MSD')
ax.loglog(emsd_corr.index, emsd_corr.values, 'b-', label='Drift-corrected MSD')
ax.set_xlabel("Lag time (s)")
ax.set_ylabel("MSD (µm²)")
ax.set_title("Effect of drift correction on MSD")
ax.legend()
plt.tight_layout()
plt.savefig("drift_correction_comparison.png", dpi=150)
print("Saved drift_correction_comparison.png")
Recipe: Classify Particles by Diffusion Regime
When to use: Separate particle population into confined, normal (Brownian), and directed motion based on log-log MSD slope (anomalous exponent alpha).
import trackpy as tp
import pims
import numpy as np
import pandas as pd
from scipy.stats import linregress
frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=400, processes=1)
t = tp.link(f, search_range=6, memory=3)
t = tp.filter_stubs(t, threshold=20)
drift = tp.compute_drift(t)
t = tp.subtract_drift(t.copy(), drift)
mpp, fps = 0.16, 10.0
imsd = tp.imsd(t, mpp=mpp, fps=fps, max_lagtime=30)
# Fit log-log slope (anomalous exponent alpha) for each particle
results = []
for pid in imsd.columns:
curve = imsd[pid].dropna()
if len(curve) < 5:
continue
log_lag = np.log(curve.index.values)
log_msd = np.log(curve.values)
slope, intercept, r, _, _ = linregress(log_lag[:10], log_msd[:10])
D_app = np.exp(intercept) / 4 # apparent D from intercept
results.append({'particle': pid, 'alpha': slope, 'D_app': D_app, 'r2': r**2})
df_char = pd.DataFrame(results)
# Classify by alpha
df_char['regime'] = pd.cut(
df_char['alpha'],
bins=[-np.inf, 0.7, 1.3, np.inf],
labels=['confined', 'brownian', 'directed']
)
print(df_char['regime'].value_counts())
print(f"\nMean D by regime:\n{df_char.groupby('regime')['D_app'].mean()}")
df_char.to_csv("particle_classification.csv", index=False)
print("Saved particle_classification.csv")
Recipe: Filter by Eccentricity to Remove Aggregates
When to use: Exclude non-circular detections (doublets, aggregates, debris) that pass the mass threshold but are elongated.
import trackpy as tp
import pims
frames = pims.open("particles.tif")
f = tp.batch(frames, diameter=11, minmass=400, processes=1)
# Eccentricity: 0 = perfect circle, 1 = line
# Remove elongated features (likely aggregates or debris)
f_round = f[f['ecc'] < 0.3]
print(f"Before ecc filter: {len(f)} detections")
print(f"After ecc filter (ecc<0.3): {len(f_round)} detections")
print(f"Removed: {len(f)-len(f_round)} elongated features")
t = tp.link(f_round, search_range=6, memory=3)
t = tp.filter_stubs(t, threshold=10)
print(f"Trajectories after eccentricity filtering: {t['particle'].nunique()}")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| Too many spurious detections | minmass too low or diameter mismatched to spot size | Plot mass histogram; raise minmass to the valley between noise and signal peaks. Verify diameter matches actual spot width in pixels |
| Few or zero detections | minmass too high, or particles are dim / out of focus | Lower minmass; check image contrast; apply background subtraction before locate |
| Very short trajectories (all stubs filtered out) | search_range too small for particle velocity, or memory=0 with blinking | Increase search_range to 1.5–2× max per-frame displacement; set memory=2 or 3 for blinking dyes |
| MSD curves are noisy or non-monotonic at long lag times | Too few tracks or fitting too many lag points | Use only first 10–20% of lag times for fitting; ensure at least 50+ trajectories for ensemble MSD |
| Drift correction makes MSD worse | Too few immobile reference particles; drift estimated from mobile particles | Include fiducial beads or immobile particles; use tp.compute_drift() only on particles known to be immobile |
MemoryError during tp.batch() | All frames loaded into RAM at once | Use pims lazy reader (default); set processes=1; process frames in chunks using a loop over tp.locate() |
3D locate returns 2D positions only | Passed a 2D frame instead of a 3D volume | Confirm input array has 3 dimensions (Z, Y, X); check frames_4d[t_idx].ndim == 3 |
| Linked trajectories fragment into many short segments | Particles moving faster than search_range between frames | Increase search_range; increase memory; consider sub-sampling frames if frame rate is very high |
References
- trackpy documentation — official API reference, tutorials, and notebooks
- trackpy GitHub repository — source code, issue tracker, example notebooks
- Crocker, J.C. & Grier, D.G. (1996). Methods of Digital Video Microscopy for Colloidal Studies. J. Colloid Interface Sci. 179, 298–310 — original algorithm paper
- pims documentation — image sequence reader library that integrates with trackpy
- trackpy walkthrough notebook — step-by-step tutorial for 2D tracking
matplotlib
Overview
Matplotlib is Python's foundational visualization library for creating static, animated, and interactive plots. It provides both a MATLAB-style pyplot interface and an object-oriented API for full control over figures, axes, and artists. Essential for generating publication-quality scientific figures.
When to Use
- Creating publication-quality plots with precise control over every element (fonts, ticks, colors, spacing)
- Building multi-panel figures with complex subplot layouts for papers
- Generating standard scientific plot types: line, scatter, bar, histogram, heatmap, box, violin, contour
- Exporting figures to vector formats (PDF, SVG) for journal submission
- Creating 3D surface, scatter, or wireframe plots
- Customizing colormaps and color schemes for accessibility (colorblind-friendly)
- Integrating plots with NumPy arrays and pandas DataFrames
- For quick statistical visualizations (distributions, regressions), use
seaborninstead - For interactive/web-based plots with hover and zoom, use
plotlyinstead
Prerequisites
- Python packages:
matplotlib,numpy - Optional:
pandas(for DataFrame plotting),seaborn(for style presets) - Environment: Works in scripts, Jupyter notebooks (
%matplotlib inline), and GUI apps
pip install matplotlib numpy
Quick Start
import matplotlib.pyplot as plt
import numpy as np
# Publication-ready figure template: set size, plot, label, save as PDF
fig, ax = plt.subplots(figsize=(6, 4)) # single-column journal width ≈ 6 cm → set here in inches
x = np.linspace(0, 2 * np.pi, 200)
ax.plot(x, np.sin(x), color="steelblue", lw=1.5, label="sin(x)")
ax.plot(x, np.cos(x), color="coral", lw=1.5, label="cos(x)", linestyle="--")
ax.set_xlabel("x (radians)")
ax.set_ylabel("Amplitude")
ax.set_title("Sine and Cosine Waves")
ax.legend(frameon=False)
ax.spines[["top", "right"]].set_visible(False) # clean axis style
plt.tight_layout()
plt.savefig("quickstart.pdf", bbox_inches="tight", dpi=300)
print("Saved quickstart.pdf")
Core API
Module 1: Figure and Axes Creation
The fundamental objects: Figure (canvas) and Axes (plotting area).
import matplotlib.pyplot as plt
import numpy as np
# Single plot (recommended: OO interface)
fig, ax = plt.subplots(figsize=(8, 5))
x = np.linspace(0, 2 * np.pi, 100)
ax.plot(x, np.sin(x), label="sin(x)")
ax.plot(x, np.cos(x), label="cos(x)")
ax.set_xlabel("x"); ax.set_ylabel("y")
ax.set_title("Trigonometric Functions")
ax.legend(); ax.grid(True, alpha=0.3)
plt.savefig("basic_plot.png", dpi=300, bbox_inches="tight")
print("Saved basic_plot.png")
# Multi-panel subplots
fig, axes = plt.subplots(2, 2, figsize=(10, 8), constrained_layout=True)
axes[0, 0].plot(x, np.sin(x)); axes[0, 0].set_title("sin(x)")
axes[0, 1].scatter(x[::5], np.cos(x[::5])); axes[0, 1].set_title("cos(x)")
axes[1, 0].bar(["A", "B", "C"], [3, 7, 5]); axes[1, 0].set_title("Bar")
axes[1, 1].hist(np.random.randn(500), bins=30); axes[1, 1].set_title("Histogram")
plt.savefig("subplots.png", dpi=300, bbox_inches="tight")
print("Saved subplots.png with 4 panels")
Module 2: Plot Types
Standard scientific chart types.
import matplotlib.pyplot as plt
import numpy as np
fig, axes = plt.subplots(2, 3, figsize=(15, 9), constrained_layout=True)
# Line plot — trends over time
x = np.linspace(0, 10, 50)
axes[0, 0].plot(x, np.exp(-x/3) * np.sin(x), "b-", linewidth=2)
axes[0, 0].set_title("Line Plot")
# Scatter plot — correlations
np.random.seed(42)
axes[0, 1].scatter(np.random.randn(100), np.random.randn(100), alpha=0.6, c=np.random.rand(100), cmap="viridis")
axes[0, 1].set_title("Scatter Plot")
# Bar chart — categorical comparisons
categories = ["Gene A", "Gene B", "Gene C", "Gene D"]
axes[0, 2].bar(categories, [4.2, 7.1, 3.5, 6.8], color="steelblue", edgecolor="black")
axes[0, 2].set_title("Bar Chart")
# Histogram — distributions
axes[1, 0].hist(np.random.randn(1000), bins=40, edgecolor="black", alpha=0.7)
axes[1, 0].set_title("Histogram")
# Box plot — statistical distributions
data = [np.random.randn(50) + i for i in range(4)]
axes[1, 1].boxplot(data, labels=["Ctrl", "Drug A", "Drug B", "Drug C"])
axes[1, 1].set_title("Box Plot")
# Heatmap — matrix data
matrix = np.random.rand(8, 8)
im = axes[1, 2].imshow(matrix, cmap="coolwarm", aspect="auto")
plt.colorbar(im, ax=axes[1, 2])
axes[1, 2].set_title("Heatmap")
plt.savefig("plot_types.png", dpi=300, bbox_inches="tight")
print("Saved 6 plot types to plot_types.png")
Module 3: Styling and Customization
Colors, fonts, styles, annotations.
import matplotlib.pyplot as plt
import numpy as np
# Use style sheets
plt.style.use("seaborn-v0_8-whitegrid")
# Custom rcParams for publication
plt.rcParams.update({
"font.size": 12, "axes.labelsize": 14,
"axes.titlesize": 16, "xtick.labelsize": 10,
"ytick.labelsize": 10, "legend.fontsize": 11,
})
fig, ax = plt.subplots(figsize=(8, 5))
x = np.linspace(0, 5, 100)
ax.plot(x, np.exp(-x), "r--", linewidth=2, label="Exponential decay")
ax.fill_between(x, np.exp(-x) - 0.1, np.exp(-x) + 0.1, alpha=0.2, color="red")
# Annotations
ax.annotate("Half-life", xy=(0.693, 0.5), xytext=(2, 0.7),
arrowprops=dict(arrowstyle="->", color="black"),
fontsize=12, fontweight="bold")
ax.set_xlabel("Time (s)"); ax.set_ylabel("Signal")
ax.legend()
plt.savefig("styled_plot.png", dpi=300, bbox_inches="tight")
print("Saved styled_plot.png")
Module 4: Advanced Layouts
Mosaic layouts, GridSpec, insets.
import matplotlib.pyplot as plt
from matplotlib.gridspec import GridSpec
import numpy as np
# Mosaic layout — named axes
fig, axes = plt.subplot_mosaic(
[["main", "right"], ["main", "bottom_right"]],
figsize=(10, 7), constrained_layout=True,
gridspec_kw={"width_ratios": [2, 1]}
)
x = np.linspace(0, 10, 200)
axes["main"].plot(x, np.sin(x) * np.exp(-x/5), "b-", linewidth=2)
axes["main"].set_title("Main Panel")
axes["right"].hist(np.random.randn(300), bins=20, orientation="horizontal")
axes["right"].set_title("Distribution")
axes["bottom_right"].bar(["A", "B"], [3, 5])
axes["bottom_right"].set_title("Summary")
plt.savefig("mosaic_layout.png", dpi=300, bbox_inches="tight")
print("Saved mosaic_layout.png")
Module 5: 3D Visualization
Surface, scatter, and wireframe plots.
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
fig = plt.figure(figsize=(10, 7))
ax = fig.add_subplot(111, projection="3d")
# Surface plot
u = np.linspace(0, 2 * np.pi, 50)
v = np.linspace(0, np.pi, 50)
X = np.outer(np.cos(u), np.sin(v))
Y = np.outer(np.sin(u), np.sin(v))
Z = np.outer(np.ones_like(u), np.cos(v))
ax.plot_surface(X, Y, Z, cmap="viridis", alpha=0.8)
ax.set_xlabel("X"); ax.set_ylabel("Y"); ax.set_zlabel("Z")
ax.set_title("3D Surface Plot")
plt.savefig("surface_3d.png", dpi=300, bbox_inches="tight")
print("Saved surface_3d.png")
Module 6: Export and Saving
Output to various formats with publication settings.
import matplotlib.pyplot as plt
import numpy as np
fig, ax = plt.subplots(figsize=(6, 4))
ax.plot([1, 2, 3], [1, 4, 9], "ko-")
ax.set_title("Export Example")
# High-res PNG for presentations
fig.savefig("figure.png", dpi=300, bbox_inches="tight", facecolor="white")
# Vector PDF for journal submission
fig.savefig("figure.pdf", bbox_inches="tight")
# SVG for web
fig.savefig("figure.svg", bbox_inches="tight")
# Transparent background
fig.savefig("figure_transparent.png", dpi=300, bbox_inches="tight", transparent=True)
plt.close(fig) # Free memory
print("Exported to PNG, PDF, SVG, and transparent PNG")
Common Workflows
Workflow 1: Multi-Panel Figure for Publication
Goal: Create a 4-panel figure combining different plot types for a paper.
import matplotlib.pyplot as plt
import numpy as np
np.random.seed(42)
fig, axes = plt.subplots(2, 2, figsize=(10, 8), constrained_layout=True)
# Panel A: Time series
t = np.linspace(0, 24, 100)
axes[0, 0].plot(t, 50 + 10 * np.sin(t * np.pi / 12), "b-", linewidth=2)
axes[0, 0].set_xlabel("Time (h)"); axes[0, 0].set_ylabel("Expression")
axes[0, 0].set_title("A", loc="left", fontweight="bold")
# Panel B: Volcano plot
fc = np.random.randn(500)
pval = -np.log10(np.random.uniform(0.0001, 1, 500))
colors = ["red" if abs(f) > 1 and p > 2 else "grey" for f, p in zip(fc, pval)]
axes[0, 1].scatter(fc, pval, c=colors, s=10, alpha=0.7)
axes[0, 1].axhline(2, ls="--", color="black", alpha=0.5)
axes[0, 1].set_xlabel("log₂ FC"); axes[0, 1].set_ylabel("-log₁₀ p-value")
axes[0, 1].set_title("B", loc="left", fontweight="bold")
# Panel C: Bar chart with error bars
means = [3.2, 5.1, 4.7, 6.3]
sems = [0.4, 0.6, 0.3, 0.5]
axes[1, 0].bar(["Ctrl", "Drug A", "Drug B", "Combo"], means, yerr=sems,
capsize=5, color="steelblue", edgecolor="black")
axes[1, 0].set_ylabel("Response"); axes[1, 0].set_title("C", loc="left", fontweight="bold")
# Panel D: Heatmap
data = np.random.randn(6, 4)
im = axes[1, 1].imshow(data, cmap="RdBu_r", aspect="auto")
plt.colorbar(im, ax=axes[1, 1])
axes[1, 1].set_title("D", loc="left", fontweight="bold")
fig.savefig("publication_figure.pdf", bbox_inches="tight")
print("Saved publication_figure.pdf (4 panels)")
Workflow 2: Statistical Comparison Plot
Goal: Bar chart with individual data points and significance annotations.
import matplotlib.pyplot as plt
import numpy as np
np.random.seed(42)
groups = {"Control": np.random.normal(5, 1.2, 20),
"Treatment A": np.random.normal(7, 1.5, 20),
"Treatment B": np.random.normal(6, 1.0, 20)}
fig, ax = plt.subplots(figsize=(6, 5))
positions = range(len(groups))
for i, (name, data) in enumerate(groups.items()):
ax.bar(i, np.mean(data), yerr=np.std(data)/np.sqrt(len(data)),
capsize=5, color=["#4C72B0", "#DD8452", "#55A868"][i],
edgecolor="black", alpha=0.8, width=0.6)
# Overlay individual data points
ax.scatter(np.full_like(data, i) + np.random.uniform(-0.15, 0.15, len(data)),
data, color="black", s=15, alpha=0.5, zorder=5)
ax.set_xticks(positions); ax.set_xticklabels(groups.keys())
ax.set_ylabel("Measurement")
# Add significance bracket
y_max = max(max(d) for d in groups.values()) + 1
ax.plot([0, 0, 1, 1], [y_max, y_max + 0.2, y_max + 0.2, y_max], "k-", linewidth=1)
ax.text(0.5, y_max + 0.3, "**", ha="center", fontsize=14)
fig.savefig("comparison_plot.png", dpi=300, bbox_inches="tight")
print("Saved comparison_plot.png")
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
figsize | Figure creation | (6.4, 4.8) | (w, h) in inches | Figure dimensions |
dpi | savefig | 100 | 72-600 | Resolution: 300 for print, 150 for web |
bbox_inches | savefig | None | "tight", None | Crop whitespace around figure |
constrained_layout | subplots | False | True/False | Auto-adjust spacing to prevent overlap |
cmap | Heatmap/scatter | "viridis" | "viridis", "coolwarm", "RdBu_r", etc. | Colormap for data mapping |
alpha | All plot types | 1.0 | 0.0-1.0 | Transparency (0=invisible, 1=opaque) |
linewidth | Line plots | 1.5 | 0.5-5.0 | Line thickness in points |
s | Scatter | 20 | 1-500 | Marker size in points² |
bins | Histogram | 10 | 5-100 or array | Number of histogram bins |
projection | add_subplot | None | "3d", "polar" | Axes projection type |
Best Practices
-
Always use the OO interface (
fig, ax = plt.subplots()) for production code. Reserveplt.plot()for quick interactive exploration only -
Use
constrained_layout=Trueto prevent overlapping labels and titles:fig, ax = plt.subplots(figsize=(8, 5), constrained_layout=True) -
Choose accessible colormaps: Use
viridis,cividis, orplasma(perceptually uniform, colorblind-safe). Avoidjetandrainbow -
Close figures after saving to prevent memory leaks in loops:
plt.close(fig) # After savefig -
Set DPI appropriately: 300 for print/journal, 150 for web/slides, 72 for screen-only
-
Use
rasterized=Truefor large datasets to reduce PDF/SVG file size:ax.scatter(x, y, rasterized=True) # Vector labels + rasterized data -
Label panels consistently: Use bold letters (A, B, C, D) at top-left of each subplot for multi-panel figures
Common Recipes
Recipe: Custom Color Palette
When to use: Consistent colors across multiple figures in a paper.
import matplotlib.pyplot as plt
# Define a custom palette
palette = {"control": "#4C72B0", "treatment": "#DD8452", "combo": "#55A868"}
fig, ax = plt.subplots()
for group, color in palette.items():
ax.bar(group, [5, 7, 6][list(palette.keys()).index(group)], color=color)
plt.savefig("custom_palette.png", dpi=300, bbox_inches="tight")
Recipe: Twin Y-Axes
When to use: Plotting two variables with different scales on the same figure.
import matplotlib.pyplot as plt
import numpy as np
fig, ax1 = plt.subplots(figsize=(8, 5))
x = np.arange(10)
ax1.bar(x, np.random.randint(10, 100, 10), alpha=0.7, color="steelblue", label="Count")
ax1.set_ylabel("Count", color="steelblue")
ax2 = ax1.twinx()
ax2.plot(x, np.cumsum(np.random.rand(10)), "r-o", linewidth=2, label="Cumulative")
ax2.set_ylabel("Cumulative", color="red")
fig.legend(loc="upper left", bbox_to_anchor=(0.15, 0.95))
plt.savefig("twin_axes.png", dpi=300, bbox_inches="tight")
Recipe: Inset Zoom Plot
When to use: Showing a zoomed-in region of a larger plot.
import matplotlib.pyplot as plt
import numpy as np
fig, ax = plt.subplots(figsize=(8, 5))
x = np.linspace(0, 10, 500)
y = np.sin(x) * np.exp(-x / 5)
ax.plot(x, y, "b-", linewidth=2)
# Inset
axins = ax.inset_axes([0.5, 0.5, 0.4, 0.4])
axins.plot(x, y, "b-", linewidth=2)
axins.set_xlim(1, 3); axins.set_ylim(0.2, 0.8)
ax.indicate_inset_zoom(axins, edgecolor="black")
plt.savefig("inset_zoom.png", dpi=300, bbox_inches="tight")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| Overlapping labels/titles | No layout management | Add constrained_layout=True to plt.subplots() |
UserWarning: tight_layout | Incompatible with constrained_layout | Use only one: constrained_layout OR tight_layout(), not both |
| Blurry figures in Jupyter | Low default DPI | Set %config InlineBackend.figure_format = 'retina' |
| Memory grows in loops | Figures not closed | Add plt.close(fig) after each savefig() |
| Font not found warning | Missing system font | Use plt.rcParams["font.sans-serif"] = ["DejaVu Sans"] |
| Large PDF/SVG file size | Many data points in vector | Use rasterized=True on heavy artists |
| 3D plot rotation stuck | Interactive backend issue | Use %matplotlib widget in Jupyter or plt.show() in scripts |
| Colorbar wrong size | Default sizing doesn't match axes | Use fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04) |
References
- matplotlib documentation — official docs
- matplotlib gallery — examples by plot type
- matplotlib cheatsheets — quick reference (PDF)
- Hunter, J.D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering 9(3):90-95.
3Dmol.js molecular visualization
Overview
3Dmol.js is a WebGL molecular viewer that runs entirely in the browser. This skill emits
self-contained HTML files that load 3Dmol from a CDN and render a structure, a trajectory,
or a vibrational mode — no server, no build step, no Python runtime to view. The bundled
scripts/mol_viewer.py generates that HTML from any .xyz/.trj/.pdb/.sdf/.mol2/.cube file;
the Core API below shows the underlying 3Dmol.js calls so you can hand-write or customize a
viewer.
When to Use
- Animate a transition-state imaginary vibrational mode (from a mode trajectory or dx/dy/dz vectors)
- Play back a reaction path (IRC/NEB) or an MD trajectory with a speed control
- Show a protein–ligand docking pose with cartoon + ligand sticks + a binding-site surface
- Display an orbital or electron-density isosurface from a Gaussian
.cubefile - Hand a colleague one HTML file that opens in any browser, no install
- Use py3Dmol instead for inline viewers inside a Jupyter notebook (same engine, Python API)
- Use PyMOL/ChimeraX instead for publication ray-traced stills or heavy structural editing
- Use rdkit-chemdraw-cdxml for 2D chemical structures, plotly/matplotlib for 2D plots
Prerequisites
- Viewing: any modern browser with network access (the HTML pulls 3Dmol.js from a CDN)
- Generator script:
scripts/mol_viewer.py— Python 3 standard library only, no install - Optional:
pip install py3Dmolfor notebook use (wraps the same library)
No package is needed to produce or open the HTML. The generator lives in this skill's scripts/
folder (next to this SKILL.md). It can't be run in place from the skill directory, so use your
file tools to read scripts/mol_viewer.py and save it into your working directory before running.
Quick Start
# animate a mode/trajectory file with play/pause + speed slider, in one call
python3 mol_viewer.py ts_imaginary_mode_000.trj --mode trajectory \
--title "TS mode" --subtitle "-621.8 cm-1" --out ts_mode.html
# static structure: python3 mol_viewer.py mol.xyz --out mol.html
Core API
All snippets assume <script src="https://3Dmol.org/build/3Dmol-min.js"></script> is loaded
and a <div id="v"></div> exists.
Create a viewer and load a structure
createViewer binds to a div; addModel(data, format) loads coordinates. Always zoomTo()
then render(). Supported format: xyz, pdb, sdf, mol2, cube, cif.
const viewer = $3Dmol.createViewer("v", {backgroundColor: "white"});
viewer.addModel(xyzString, "xyz"); // coordinates as a string, not a URL
viewer.setStyle({}, {stick: {radius: 0.15}, sphere: {scale: 0.28}});
viewer.zoomTo();
viewer.render();
Styles and coloring
setStyle(selection, styleSpec) — empty selection {} targets all atoms. Styles: stick,
sphere, line, cross, cartoon. Color by element (default), a scheme, or a fixed color.
viewer.setStyle({}, {stick: {}, sphere: {scale: 0.25}}); // ball-and-stick
viewer.setStyle({elem: "C"}, {stick: {color: "gray"}}); // per-element override
viewer.setStyle({chain: "A"}, {cartoon: {color: "spectrum"}}); // protein ribbon
viewer.render();
Animate a trajectory
Load every frame with addModelsAsFrames, then animate. interval is the delay between
frames in milliseconds (larger = slower) — do not use step, which skips frames and looks
jumpy. loop: "backAndForth" makes a one-way path oscillate; reps: 0 loops forever.
viewer.addModelsAsFrames(trjString, "xyz"); // multi-frame .trj or multi-model .xyz/.pdb
viewer.setStyle({}, {stick: {radius: 0.14}, sphere: {scale: 0.28}});
viewer.zoomTo();
viewer.render();
viewer.animate({loop: "backAndForth", interval: 120, reps: 0});
Animate a vibrational normal mode
If a model's atoms carry displacement vectors (dx, dy, dz — extra columns on each XYZ line:
elem x y z dx dy dz), model.vibrate(numFrames, amplitude, bothWays, arrowSpec) builds the
oscillation frames. bothWays: true swings symmetrically about equilibrium; arrowSpec draws
motion arrows.
const m = viewer.addModel(modeXyz, "xyz"); // each atom line: elem x y z dx dy dz
m.vibrate(10, 1.0, true, {radius: 0.08, color: "black"}); // 10 frames, full amplitude, arrows
viewer.setStyle({}, {stick: {radius: 0.14}, sphere: {scale: 0.28}});
viewer.zoomTo();
viewer.render();
viewer.animate({loop: "backAndForth", interval: 120, reps: 0});
If you only have a precomputed frame trajectory (e.g. pysisyphus ts_imaginary_mode_000.trj),
use the trajectory path above instead — no dx/dy/dz needed.
Surfaces and volumetric isosurfaces
addSurface(type, style, atomsel) builds a molecular surface (VDW, SAS, SES, MS).
For an orbital/density isosurface, load the .cube and call addVolumetricData.
viewer.addSurface($3Dmol.SurfaceType.VDW, {opacity: 0.75, color: "lightblue"}, {chain: "A"});
// isosurface from a Gaussian cube (positive and negative lobes):
viewer.addVolumetricData(cubeString, "cube", {isoval: 0.02, color: "blue", opacity: 0.85});
viewer.addVolumetricData(cubeString, "cube", {isoval: -0.02, color: "red", opacity: 0.85});
viewer.render();
Labels and interactive speed control
addLabel(text, spec) annotates. For animations, a slider bound to interval (restarting via
stopAnimate() + animate()) lets the viewer set the speed — the fix for "sometimes too fast".
viewer.addLabel("TS", {position: {x: 0, y: 0, z: 0}, backgroundColor: "black", fontSize: 14});
let interval = 140;
const play = () => viewer.animate({loop: "backAndForth", interval});
document.getElementById("spd").oninput = e => { interval = +e.target.value; viewer.stopAnimate(); play(); };
play();
Key Concepts
interval vs step. interval (ms) sets playback speed; every frame is shown. step
plays every Nth frame — it skips motion and is the usual cause of a "too fast"/jumpy animation.
Control speed with interval, never step.
Coordinates are strings, not URLs. addModel/addModelsAsFrames take the file contents.
Embed them in the HTML as a JSON-encoded string so quotes and newlines survive
(scripts/mol_viewer.py uses json.dumps; a raw backtick template breaks on backticks in data).
CDN and CSP. The page fetches 3Dmol.js from a CDN, so it needs network access when opened, and a strict Content-Security-Policy (e.g. inside some artifact sandboxes) will blank it. Open it as a normal local/hosted file.
Common Workflows
TS imaginary-mode animation (quantum-chemistry)
End-to-end HTML from a precomputed mode trajectory, with play/pause and a speed slider — the
deliverable the neb-irc-activation-energy skill hands off.
python3 mol_viewer.py ts_imaginary_mode_000.trj --mode trajectory \
--title "Transition-state mode" --subtitle "-621.8 cm-1" --out ts_mode.html
# open ts_mode.html; drag the slider if the oscillation is too fast
Reaction-path / MD playback
python3 mol_viewer.py trajectory.pdb --mode trajectory --style ballstick --out md.html
# any multi-model .xyz/.pdb works; backAndForth loop + interval control are built in
Docking pose: protein ribbon + ligand sticks + pocket surface
const viewer = $3Dmol.createViewer("v", {backgroundColor: "white"});
viewer.addModel(complexPdb, "pdb");
viewer.setStyle({}, {cartoon: {color: "spectrum"}}); // protein
viewer.setStyle({resn: "LIG"}, {stick: {radius: 0.2}}); // ligand
viewer.addSurface($3Dmol.SurfaceType.VDW, {opacity: 0.6}, {resn: "LIG", byres: true, expand: 5});
viewer.zoomTo({resn: "LIG"});
viewer.render();
Key Parameters
| Parameter | Method | Default | Range / Options | Effect |
|---|---|---|---|---|
interval | animate | 50 | 40–400 ms | Frame delay; larger = slower playback |
loop | animate | forward | forward/backward/backAndForth | backAndForth oscillates a one-way path |
reps | animate | 0 | 0=∞, n | Number of loops |
radius | stick | 0.3 | 0.1–0.3 | Bond cylinder thickness |
scale | sphere | 1.0 (vdW) | 0.2–0.4 for ball-and-stick | Atom sphere size |
amplitude | vibrate | 1.0 | 0.5–2.0 | Normal-mode distortion size |
numFrames | vibrate | 10 | 8–20 | Frames per half-cycle |
isoval | addVolumetricData | — | e.g. ±0.02 | Isosurface contour value (sign = lobe) |
opacity | addSurface | 1.0 | 0–1 | Surface transparency |
Best Practices
- Control animation speed with
interval(ms), neverstep. - Embed coordinates as a JSON-encoded string (
json.dumps), not a raw backtick template. - Call
zoomTo()beforerender(), and again after adding a large model. - Keep default element colors unless the analysis needs a specific scheme — don't bake a palette.
- For large trajectories (>500 frames or >5k atoms), subsample frames; WebGL redraw is the limit.
- Ship one CDN
<script>tag; only vendor the ~1 MB3Dmol-min.jsinline if offline use is required.
Common Recipes
Recipe: generate a viewer in one call
python3 mol_viewer.py mode.xyz --mode vibrate --amplitude 1.2 --title "mode" --out mode.html
python3 mol_viewer.py mol.sdf --style stick --out mol.html # static
Recipe: inline viewer in a Jupyter notebook (py3Dmol)
import py3Dmol
view = py3Dmol.view(width=500, height=400)
view.addModel(open("mol.xyz").read(), "xyz")
view.setStyle({}, {"stick": {}, "sphere": {"scale": 0.25}})
view.zoomTo(); view.show()
Recipe: side-by-side viewers
const viewer = $3Dmol.createViewerGrid("v", {rows: 1, cols: 2});
viewer[0][0].addModel(reactantXyz, "xyz"); viewer[0][0].setStyle({}, {stick: {}});
viewer[0][1].addModel(productXyz, "xyz"); viewer[0][1].setStyle({}, {stick: {}});
viewer[0][0].zoomTo(); viewer[0][1].zoomTo(); viewer[0][0].render(); viewer[0][1].render();
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| Blank white page | 3Dmol.js not loaded (offline / strict CSP) | Open with network access; check the CDN <script> resolves |
| Animation too fast / jumpy | Using step, or a tiny interval | Use interval (ms); raise it; never set step |
| Vibration shows no motion | Model lacks dx/dy/dz vectors | Add mode vectors as extra XYZ columns, or use a precomputed frame .trj |
| Nothing rendered | Wrong format string or bad data | Match format to the file; coordinates must be the file contents, not a path |
| JS syntax error in page | Backtick/quote in embedded data | Embed via json.dumps (the generator does this) |
| Structure loads but no bonds | XYZ without connectivity + line style | Use stick/sphere; 3Dmol infers bonds by distance |
| Surface slow or hangs | Large SES/MS on a big system | Use VDW, restrict the atomsel, or lower resolution |
Bundled Resources
scripts/mol_viewer.py— emit a standalone 3Dmol HTML (static / trajectory / vibrate) from a structure file, with built-in play/pause + speed slider for animations
Related Skills
- neb-irc-activation-energy — produces TS imaginary-mode trajectories and IRC paths that this skill animates
- rdkit-chemdraw-cdxml — 2D chemical structure and reaction-scheme drawing
- plotly-interactive-plots — interactive 2D scientific plots and dashboards
References
- 3Dmol.js documentation — GLViewer / GLModel API
- Rego & Koes, Bioinformatics 2015, 31(8):1322–1324 — 3Dmol.js: molecular visualization with WebGL
- GLViewer.animate —
interval,loop,reps - GLModel.vibrate — normal-mode animation from
dx/dy/dz - py3Dmol on PyPI — Python/Jupyter wrapper
multipanel
Overview
A multi-panel figure is one figure, built one of two ways depending on what you have:
- Option 1 — redraw every panel (you have the data or plotting code): draw
each data panel with a python script into its own
subfigureso it packs to its own labels — no empty bands, and axes need NOT align across the grid. Follow the discipline below so legends stay inside their panels, panel letters sit at each panel's own top-left, and text never overlaps. - Option 2 — composite finished images (you only have rendered PNG/PDF panels):
paste them onto a
plt.subplot_mosaiccanvas — fine here, since images carry no tick labels to misalign — add panel letters, and export.
A mix is allowed: if one or two panels are image-only (no data/code), imshow
them onto their own subfigure axes and redraw the rest into the same figure. Both
modes export a vector PDF and a high-DPI PNG.
Always export the individual panels AND the composite. Every run outputs both:
one standalone figure per panel (figure1A.png, figure1B.png, …) and the combined
figure (combined_figure1.pdf + .png) — not just the composite. Because a
matplotlib subfigure cannot be saved on its own, factor every data panel's plotting
body into a draw_<letter>(ax) function (option 1); the same function then draws onto
the composite's subfigure axis AND onto a fresh standalone figure, so the panels stay
identical across both outputs with no duplicated drawing code. See "Exporting
individual panels" below.
This skill covers composition. For how to draw each individual plot type
(volcano, GSEA bar, heatmap, box/violin, PCA, Kaplan–Meier, …), use the sibling
omics-plotting skill — copy each recipe's body onto a subfigure's axis rather than
calling it as a standalone figure. Everything you need here (shared style,
composite recipe, panel-label helper) is in this document.
When to use
- The user asks for a multi-panel / composite / journal figure (panels A, B, C…) combining two or more plots into one page of image.
- The user hands you or points out already-rendered panels (PNG/PDF) and wants them combined into one figure (image assembly — see "Assembling user-provided panels").
- You are assembling a figure for a report, a paper submission, or a presentation and want all panels to read as one consistent system.
Do NOT use for
- A single plot from a data table — use the sibling
omics-plottingskill. - Interactive dashboards or web charts (this is static matplotlib output).
- 3D molecular structure rendering (that is the structure viewer, not a plot).
Key Concepts
Redraw vs composite — two composition modes
There are two fundamentally different ways to build a composite, and the user
chooses. Redraw (option 1) rebuilds every panel from data or
code in one script, giving uniform style, fonts, colors, and panel letters — best
when you hold the underlying data/DataFrame or the plotting code. Composite
(option 2) pastes already-rendered PNG/PDF panels onto a canvas and only adds
panel letters — image assembly, not plotting — best when you have only the
finished images. A mix is allowed: image-only panels are imshow-pasted while
data panels are redrawn, all into one figure.
Independent subfigures vs shared mosaic
The central layout decision. Giving each panel its own subfigure lets it run
its own constrained_layout and pack tightly to its OWN labels — panels sit flush
with no empty bands, and axes deliberately do NOT align across the grid. A single
shared subplot_mosaic gridspec instead equalizes every column's margin to its
widest y-label, leaving wide empty bands beside short-label panels. Independent
subfigures are the default here because composites usually mix heterogeneous plot
types; a shared mosaic is correct only when panels genuinely share a scale and are
meant to be read against each other.
Panel letters in the subfigure frame
Panel letters (bold A, B, C…) must sit at each panel's OWN outer top-left, left
of that panel's y-axis labels — never merged into the title and never snapped to a
shared column x-position. Placing each letter at (0, 1) in its subfigure's
coordinate frame (transform=sf.transSubfigure) guarantees it hugs its panel
regardless of neighbors' label widths.
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: panelSelection
kind: required
source: user
ask: "Which results earn a panel, and what claim should the assembled figure make?"
default: null
- id: D2
param: panelOrder
kind: required
source: user
depends_on: [D1]
ask: "In what order should the panels be read, and which belong on the same row as a group?"
default: "analysis order"
- id: D3
param: sharedScales
kind: required
source: user
depends_on: [D1]
ask: "Should panels that show the same quantity share one axis range, or each get its own?"
default: "shared within a row of comparable panels"
- id: D4
param: figureDimensions
kind: required
source: user
ask: "What column width and page format is this figure for?"
default: "single column, portrait"
- id: D5
param: exportFormat
kind: optional
source: user
ask: "Vector for submission, raster for a slide, or both?"
default: "vector PDF plus a raster preview"
D3 is the only one here that changes what a reader concludes rather than how it looks. Independent y-axes make a small effect and a large one occupy the same visual height, and side-by-side panels invite exactly that comparison.
Decision Framework
Start from what you have, then how panels relate:
What sources do you have?
├─ Data / code for every panel .................. Option 1: redraw all
├─ Only finished PNG/PDF images ................. Option 2: composite images
└─ Mix (some data, some image-only) ............. Option 1 + imshow the image-only panels
│
▼
How do the panels relate?
├─ Heterogeneous plot types (default) ........... Independent subfigures (tight pack, axes need NOT align)
└─ Same scale, read against each other .......... Shared subplot_mosaic (aligned axes)
│
▼
Layout: sketch the grid [[...]], nest subfigures for spanning panels, fill every cell
| Situation | Approach | Layout primitive | Panel letters |
|---|---|---|---|
| Have data/code for all panels | Redraw (option 1) | fig.subfigures(...) per panel | subfigure frame (0,1) |
| Only rendered images | Composite (option 2) | plt.subplot_mosaic + imshow | mosaic axes top-left |
| Some data, some image-only | Redraw + paste | subfigures + imshow leaf | subfigure frame (0,1) |
| Panels share a common scale | Shared mosaic | subplot_mosaic aligned | axes top-left |
| Spanning panel (e.g. bottom row) | Nested subfigures | top[0].subfigures(1, 2) | leaf subfigure frame |
Workflow
-
Ask which approach first — ask the user, then wait. Both approaches below are usually viable and the choice is the user's, so before drawing or writing any script, ask the user to choose between these two concrete options:
- Option 1 — Redraw every panel into one unified figure (from data/code): consistent style, fonts, colors, and panel letters across all panels. Best when you have the underlying data (CSV/TSV/DataFrame) or the plotting code.
- Option 2 — Composite already-rendered images: paste the finished PNG/PDF panels onto a canvas and add panel letters — image assembly, not plotting. Best when you only have the finished images (no data/code) or the user wants to keep the originals as-is.
Skip the question only when one option is impossible (e.g. only images and no data/code → option 2 is forced; or a data table with no rendered images → option 1) and say why. If a mix (some panels have data, one or two are images-only), tell the user that the image-only panels will be pasted regardless (discipline in the intro).
-
Decide the layout (the grid
[[...]]sketch is just to plan the tiling; you build it with nestedsubfigures, notsubplot_mosaic— see discipline #1). Fill every cell. — e.g. two on top, one spanning the bottom →[["A", "B"], ["C", "C"]]→top = fig.subfigures(2, 1); tc = top[0].subfigures(1, 2)(A,B intc; C intop[1]). — e.g. three on top, two on the bottom →[["A", "B", "C"], ["D", "E", "E"]].
- e.g. one big panel on the left, two stacked on the right →
[["A", "B"], ["A", "C"]]→lr = fig.subfigures(1, 2); A = lr[0]; rr = lr[1].subfigures(2, 1).
- Gather each panel's source — a workspace-relative CSV/TSV (or DataFrame) for data panels, or a user-supplied PNG/PDF for image panels.
- Write one python script: paste the style block, factor each data panel's
plotting body into a
draw_<letter>(ax)function (so it can render onto both a subfigure axis and a standalone figure), build the subfigures (nest for spanning panels), call eachdraw_<letter>onto its axis (data) orimshowthe image, collect the subfigures into apanelsdict, and add panel letters with the helper. Then always save both outputs to workspace-relative paths underfigures/:- the composite as
figures/combined_figure1.pdf+figures/combined_figure1.png, and - each individual panel as
figures/figure1A.png,figures/figure1B.png, … (plus matching.pdf) by rendering everydraw_<letter>onto a fresh standalone figure. See "Exporting individual panels" for the exact loop.
- the composite as
- Report the saved paths back to the user — the combined figure and every individual panel file.
Shared style — paste at the top of the script
import matplotlib.pyplot as plt
# Publication style (colorblind-friendly, editable vector text, no top/right spines)
PUB_STYLE = {
"figure.dpi": 110, "savefig.dpi": 300, "savefig.bbox": "tight",
"font.family": "sans-serif",
"font.sans-serif": ["Arial", "Liberation Sans", "Nimbus Sans", "Helvetica", "DejaVu Sans"],
"font.size": 11, "axes.titlesize": 13, "axes.titleweight": "bold",
"figure.titlesize": 13, "figure.titleweight": "bold",
"axes.labelsize": 12, "axes.linewidth": 1.0,
"axes.spines.top": False, "axes.spines.right": False,
"xtick.labelsize": 10, "ytick.labelsize": 10,
"xtick.direction": "out", "ytick.direction": "out",
"legend.frameon": False, "legend.fontsize": 9,
"svg.fonttype": "none", "pdf.fonttype": 42, "ps.fonttype": 42,
}
plt.rcParams.update(PUB_STYLE)
# Palette — reuse the SAME colors across every panel
UP, DOWN, NS = "#d73721", "#204897", "#d9d9d9" # up / down / not-significant
PALETTE = ["#2a78d6", "#eb6834", "#1baf7a", "#eda100", "#e87ba4",
"#008300", "#4a3aa7", "#e34948", "#12a4c0", "#a66a2e"] # categorical (CVD-safe)
DIVERGING_CMAP = "RdBu_r" # z-score / log2FC — set center=0, vmin=-vmax
SEQUENTIAL_CMAP = "viridis" # magnitude / -log10 p / density
For a dense composite, lower the font: plt.rcParams.update({"font.size": 7, "axes.titlesize": 8, "axes.labelsize": 7, "legend.fontsize": 6}).
Multi-panel discipline
This is what keeps a composite clean — every rule prevents a specific failure.
- One figure, independent subfigures, constrained layout. Give each panel its
own subfigure so it packs to its OWN labels:
fig = plt.figure(layout="constrained", figsize=(width_mm/25.4, height_mm/25.4)), thensfs = fig.subfigures(nrows, ncols, width_ratios=..., height_ratios=...)andax = sfs[r, c].subplots()per panel. Each subfigure runs its ownconstrained_layout, so a panel with long y-tick labels no longer shoves its column-neighbors' plots sideways — axes deliberately do NOT align across the grid; panels sit flush with no empty bands (a single sharedsubplot_mosaicgridspec, by contrast, equalizes each column's margin to its widest y-label and leaves a wide gap beside the short-label panels). Reserve a hair of margin so panel letters never clip:fig.get_layout_engine().set(rect=(0.012, 0, 0.988, 0.985)). Never addtight_layout()or manualsubplots_adjust. Size in mm (single column = 88 mm, double = 180 mm).- Spanning panels: nest subfigures — e.g. two panels on top, one spanning the
bottom →
top = fig.subfigures(2, 1); tc = top[0].subfigures(1, 2)(A, B intc[0],tc[1]; C intop[1]). One.subplots()per leaf subfigure. - Match each panel to its plot's shape via the subfigures'
width_ratios/height_ratios(pin withax.set_box_aspect(...)if it still deforms): scatter panels (volcano/PCA) near-square; for bar / box / histogram, protect the value axis in both orientations — horizontal (barh, horizontal box) kept wide, vertical (bar, box, hist) kept tall. Never let a neighbor squeeze that axis flat. - When panels genuinely share a scale (same y-range, meant to be read against each
other), a shared
subplot_mosaicwith aligned axes is the right choice instead — but this skill usually combines heterogeneous plot types, so independent subfigures are the default.
- Spanning panels: nest subfigures — e.g. two panels on top, one spanning the
bottom →
- Fill every cell. No empty grid slots. If a panel would be blank, span a
neighbor across it:
[["A", "B"], ["C", "C"]]. - Legends & colorbars belong to their own panel — a legend in that panel's
free corner (
ax.legend(loc="lower right", frameon=False)) or a colorbar on that one axis (fig.colorbar(im, ax=ax, fraction=0.025, pad=0.02)). Never float a figure-level legend in empty space or stack two in a margin; for a dot plot, keep only the colorbar and drop the size legend (count range → panel title).- Too wide? Thin a colorbar with
aspect=40+shrink=0.6(and smallfraction); tighten a legend withhandlelength=1.0,handletextpad=0.2,borderpad=0.2, or fold long legends intoncol.
- Too wide? Thin a colorbar with
- Panel letters at each panel's OWN outer top-left. Bold capitals
A, B, C…(lowercase fine; never numeric), placed to the LEFT of that panel's y-axis tick and axis labels — not merged into the title. Place each letter in its own subfigure's coordinate frame —sf.text(0.0, 1.0, letter, transform=sf.transSubfigure, ...)(the helper below). The subfigure's top-left corner is always left of that panel's y-labels and hugs that panel, so the letter never overlaps a wide label and never floats over an empty band. Do not snap letters to a shared column-x — with independent packing that would drag a short-label panel's letter far from its plot (the empty-gap failure the user sees). The reserved margin from discipline #1 keeps edge letters in-canvas. - Text must stay readable — the #1 way composites go wrong.
- Point labels (volcano/scatter): cap to ≤5 strongest hits in a small
panel, italic ~6 pt, and repel with
adjustText; if it is not installed, skip labels rather than dumping overlapping text. - Long category names (pathways/gene sets): put them on the y-axis
(horizontal, one per row), never crammed/rotated on a narrow x-axis. If they
must go on x, rotate (45° to save vertical space, or 90° when very long,
ha="right"so the tick end aligns under its bar), wrap to ≤26 chars, and give enough width per column — crammed x-labels otherwise collide into unreadable text. Shorten over-long names (common with MSigDB/GO/Reactome): strip the DB prefix (HALLMARK_,GO_,REACTOME_,KEGG_), swap_→space and title-case, and replace verbose terms with standard abbreviations (e.g.HALLMARK_INTERFERON_GAMMA_RESPONSE→IFN-γ response); truncate with an ellipsis only if still too long. Keep the full name in the underlying data/tooltip, not on the axis tick. - Size-encoded markers (dot plot): floor the size range (
sin ~[25,150]) so small dots stay visible. - In-cell heatmap numbers: annotate only when the values are needed (small font ~4–5 pt, no decimals); otherwise omit them and let the colorbar carry the values.
- Point labels (volcano/scatter): cap to ≤5 strongest hits in a small
panel, italic ~6 pt, and repel with
- Reuse one palette and axis convention across panels so the composite reads as a single system.
- Export vector PDF + PNG and report the relative path.
Panel-label helper
Place each letter at the top-left corner of its own subfigure. Because every panel lives in its own tightly-packed subfigure, that corner is always left of the panel's y-labels and hugs the panel — so letters never float over an empty band (the shared-column failure) and never overlap a wide y-label, no matter how the panels' label widths differ:
def add_panel_labels(panels, size=11):
"""Bold letter at each panel's OWN outer top-left, in its subfigure frame.
panels : dict {letter: subfigure} — the subfigure that holds each panel's axes,
collected as you build them (for a spanning panel, its leaf subfigure). Placing
the letter at (0, 1) in the subfigure's coordinates puts it at that cell's top-left
corner: always LEFT of the panel's y-labels and hugging the panel, with no
dependence on any neighbor's label width. Reserve a hair of figure margin first
(`fig.get_layout_engine().set(rect=(0.012, 0, 0.988, 0.985))`, discipline #1) so the
letters of edge panels are not clipped at the canvas edge.
"""
for letter, sf in panels.items():
sf.text(0.0, 1.0, letter, transform=sf.transSubfigure,
fontsize=size, fontweight="bold", va="top", ha="left")
Usage: collect the subfigures as you create them, e.g. panels = {"A": sfs[0, 0], "B": sfs[0, 1], "C": top[1]}, then call add_panel_labels(panels).
Exporting individual panels
Every run produces both the individual panels (figure1A.png, figure1B.png, …)
and the composite (combined_figure1.pdf + .png) — this is the default output,
not an extra. A matplotlib subfigure cannot be saved on its own, so put each panel's
plotting body in a draw_<letter>(ax) function and call it twice: once onto the
composite's subfigure axis, and once onto a fresh standalone figure. One source of
truth per panel — the panels stay identical across both outputs.
import os
os.makedirs("plots", exist_ok=True)
# 1) Factor each DATA panel's body into a function of a single Axes.
# (Copy the omics-plotting recipe body here, drawing onto `ax` instead of a new figure.)
def draw_A(ax):
ax.scatter(df["log2FC"], -np.log10(df["padj"]), s=8, c=NS) # volcano, etc.
ax.set_xlabel("log2 fold change"); ax.set_ylabel("-log10 FDR")
def draw_B(ax):
... # PCA / box / heatmap body onto ax
def draw_C(ax):
...
DATA_PANELS = {"A": draw_A, "B": draw_B, "C": draw_C}
# Per-panel standalone figure size (mm) — match each plot's shape (discipline #1).
PANEL_SIZE_MM = {"A": (88, 75), "B": (88, 75), "C": (180, 70)}
# Image-only panels stay separate: keep the PNG/PDF the user supplied as their
# standalone file, and only imshow them onto the composite axis (see intro).
# 2) Composite — draw each function onto its subfigure axis, add letters, save.
panels = {"A": sfs[0, 0], "B": sfs[0, 1], "C": top[1]}
for letter, sf in panels.items():
DATA_PANELS[letter](sf.subplots())
add_panel_labels(panels)
fig.savefig("figures/combined_figure1.pdf")
fig.savefig("figures/combined_figure1.png", dpi=300)
# 3) Individual panels — same functions onto fresh standalone figures (no letter).
for letter, draw in DATA_PANELS.items():
w_mm, h_mm = PANEL_SIZE_MM[letter]
fp = plt.figure(layout="constrained", figsize=(w_mm / 25.4, h_mm / 25.4))
draw(fp.subplots())
fp.savefig(f"figures/figure1{letter}.pdf")
fp.savefig(f"figures/figure1{letter}.png", dpi=300)
plt.close(fp)
Output files (Figure 1 with panels A, B, C):
figures/combined_figure1.pdf, figures/combined_figure1.png,
figures/figure1A.{pdf,png}, figures/figure1B.{pdf,png}, figures/figure1C.{pdf,png}.
Notes:
- No panel letter on standalones — the
A/B/Clabel belongs to the composite frame only; a lonefigure1A.pngneeds no letter baked in. - Size each standalone to its plot's shape (discipline #1) via
PANEL_SIZE_MM: scatter/PCA near-square,barh/horizontal-box wide, vertical bar/box/hist tall — don't reuse one size for all. - Legends/colorbars still belong to their own axis (discipline #3) — since the
body lives in
draw_<letter>, attach them inside that function so they appear in both the composite and the standalone. - Image-only panels are already standalone files (the user's PNG/PDF); don't re-export them — just reference the originals.
Best Practices
- One figure, one style. Never stitch separate PNGs or call standalone plot functions for a composite; copy their bodies onto each subfigure's axis.
- Workspace-relative paths only. Save under
figures/(create it if needed); never absolute paths like/tmpor/home/.... - Only plot data that exists. Never invent columns, groups, or values.
- Label every axis, keep every legend inside its panel, fill every cell.
- Always export both a vector
.pdfand a.png(dpi≥300) underfigures/.
Common Pitfalls
- Building the whole figure as one shared
subplot_mosaicgridspec. It equalizes each column's margin to its widest y-label, so a long-label panel shoves neighbors sideways, leaves empty bands, and strands letters snapped to the shared column edge. How to avoid: give each panel its ownsubfigureso it packs to its own labels (discipline #1); reserve a shared mosaic only for panels that genuinely share a scale. - Panel letters merged into titles, snapped to a shared column-x, or clipped at
the edge. They then sit right of the y-labels, float far from their plot, or
vanish off-canvas. How to avoid: place each letter at
(0, 1)in its own subfigure frame (transform=sf.transSubfigure) with theadd_panel_labelshelper, and reserve a hair of margin (rect=(0.012, 0, 0.988, 0.985), discipline #1) so edge letters stay in-canvas. - Floating or bulky legends and colorbars. Per-plot figure-level legends
collide in the margins, or a colorbar eats half the panel. How to avoid:
attach each legend/colorbar to its own panel's axis, drop a composite dot plot's
size legend, and thin a wide colorbar (
aspect=40,shrink=0.6, smallfraction) (discipline #3). - Value axis flattened — scatter dots merge or bars/boxes squash. A neighbor
steals the space the plot's value direction needs. How to avoid: widen or
heighten that cell via
width_ratios/height_ratios(or pin withax.set_box_aspect) instead of shrinking the plot; only then bump marker size. - Cramming long category names onto a narrow x-axis. Pathway/gene-set names
collide into unreadable text. How to avoid: put long names on the horizontal
y-axis, strip DB prefixes (
HALLMARK_,GO_) and abbreviate, or rotate 90° and wrap to ≤26 chars with enough panel width. - Fixing cramped panels with manual spacing. Adding
tight_layout()orsubplots_adjustfightsconstrained_layoutand makes it worse. How to avoid: instead increasefigsize(in mm), adjust the ratios, or lower the font, and let constrained layout re-space. - Inconsistent or unreadable text. Over-labeled points overlap, heatmap cell
numbers are too dense, and font sizes drift between panels. How to avoid: cap
point labels to ≤5 (repel with
adjustText, else skip); annotate heatmap cells only at ~4–5 pt with no decimals or drop them for the colorbar; keep the same font sizes across all panels, including any the user supplies (discipline #5).
Further Reading
- Matplotlib subfigures /
Figure.subfigures— https://matplotlib.org/stable/gallery/subplots_axes_and_figures/subfigures.html - Matplotlib constrained layout guide — https://matplotlib.org/stable/users/explain/axes/constrained_layout_guide.html
- Matplotlib
subplot_mosaictutorial — https://matplotlib.org/stable/users/explain/axes/mosaic.html - adjustText (label de-overlap) — https://github.com/Phlya/adjustText
omics-plotting
Overview
When the user wants a figure, generate it with matplotlib / seaborn, applying the shared style block below. The user can hand-tune colors, fonts, or spines per plot, but unless they ask for something specific, paste the style block and reuse the palette so a whole analysis reads as one figure system at a glance.
This skill is self-contained: everything you need (style, palette, recipes) is in this document.
When to use
- The user asks for a plot / figure / chart / visualization from a results table or an in-memory DataFrame (DEG table, enrichment result, expression matrix, long-form measurements, survival table…).
- You are preparing figures for a report, a paper submission or presentation and want a consistent publication style.
Combining several plots into one multi-panel composite, or assembling user-supplied PNG/PDF panels, is handled by the sibling
multipanelskill — use this skill to draw each individual panel.
Do NOT use for
- Interactive dashboards or web charts (this is static matplotlib output).
- 3D molecular structure rendering (that is the structure viewer, not a plot).
Key Concepts
One consistent figure system
The core idea is that every figure from a single analysis should look like it
came from the same publication. That is enforced by two shared objects: the
PUB_STYLE rcParams block (fonts, spines, DPI, editable vector text) and a fixed
PALETTE / directional color set (UP, DOWN, NS). Paste both at the top of
every plot script and map the same group or direction to the same color
across panels, so a reader can carry meaning from one figure to the next.
Diverging vs sequential colormaps
Color encoding is not free choice. Use the diverging colormap
(DIVERGING_CMAP = "RdBu_r", always center=0, vmin=-vmax) for signed
quantities where zero is meaningful — z-scores, log2 fold changes, correlations.
Use the sequential colormap (SEQUENTIAL_CMAP = "viridis") for unsigned
magnitudes — densities, -log10 p, counts. Mixing these (a sequential map on
signed data) hides the sign and misleads the reader.
Data shape drives figure type
Each recipe expects a specific table shape: a per-gene DEG table (volcano, MA), a genes × samples matrix (heatmap), a samples × features matrix (PCA/UMAP), or long-form tidy rows (box/violin/bar, ridgeline, Kaplan–Meier). Identifying the shape first — then reading the header to confirm the real column names — is what selects the recipe. The column names in each recipe are defaults to override, not fixed requirements.
Decision Framework
Pick the figure type from what the data represents and what question it answers:
What does the table hold?
├─ Per-gene stats (log2FC, padj)
│ ├─ emphasize significance ......... Volcano
│ └─ emphasize expression level ..... MA plot
├─ genes × samples matrix
│ ├─ show patterns/clusters ......... Clustered expression heatmap (z-score)
│ └─ show sample-sample QC .......... Correlation heatmap
├─ Enrichment / gene-set result
│ ├─ signed effect (NES) ............ GSEA bar
│ └─ ratio + size + significance .... GSEA dot plot
├─ Long-form measurements (x, y)
│ ├─ compare distributions .......... Box / Violin
│ ├─ compare means .................. Bar (with error bars)
│ └─ many groups, shape matters ..... Ridgeline
├─ samples × features (high-dim) ...... PCA / UMAP / t-SNE
└─ time-to-event + group ............. Kaplan–Meier
| Data you have | Question | Figure | Colormap / palette |
|---|---|---|---|
| DEG table | Which genes change, how significantly? | Volcano | UP/DOWN/NS |
| DEG table | Effect vs abundance | MA plot | UP/DOWN/NS |
| Expression matrix | Cluster structure | Clustered heatmap | diverging, center 0 |
| Expression matrix | Sample QC | Correlation heatmap | diverging, [-1, 1] |
| Enrichment result | Top pathways, direction | GSEA bar | UP/DOWN |
| Enrichment result | Ratio + significance + size | GSEA dot plot | sequential |
| Long-form | Group distributions | Box / Violin | categorical PALETTE |
| High-dim matrix | Global sample layout | PCA / UMAP / t-SNE | categorical PALETTE |
| Survival table | Group survival over time | Kaplan–Meier | categorical PALETTE |
Workflow
- Identify the data source — a workspace-relative CSV/TSV path or a DataFrame already in memory — and the figure type (pick from the table below). If the required columns are unclear, inspect the table's header first.
- Write one python script: paste the style block, load the data,
draw the plot with the matching recipe, and save to a workspace-relative
path under
figures/. - Report the saved path back to the user (and reference it in any report /
deck by that relative path, e.g.
).
Shared style — paste at the top of every plot script
import matplotlib.pyplot as plt
# Publication style (colorblind-friendly, editable vector text, no top/right spines)
PUB_STYLE = {
"figure.dpi": 110, "savefig.dpi": 300, "savefig.bbox": "tight",
"font.family": "sans-serif",
"font.sans-serif": ["Arial", "Liberation Sans", "Nimbus Sans", "Helvetica", "DejaVu Sans"],
"font.size": 11, "axes.titlesize": 13, "axes.titleweight": "bold",
"figure.titlesize": 13, "figure.titleweight": "bold",
"axes.labelsize": 12, "axes.linewidth": 1.0,
"axes.spines.top": False, "axes.spines.right": False,
"xtick.labelsize": 10, "ytick.labelsize": 10,
"xtick.direction": "out", "ytick.direction": "out",
"legend.frameon": False, "legend.fontsize": 9,
"svg.fonttype": "none", "pdf.fonttype": 42, "ps.fonttype": 42,
}
plt.rcParams.update(PUB_STYLE) # or: with plt.rc_context(PUB_STYLE): ...
# Palette — reuse the SAME colors across every panel of an analysis
UP, DOWN, NS = "#d73721", "#204897", "#d9d9d9" # up / down / not-significant
PALETTE = ["#2a78d6", "#eb6834", "#1baf7a", "#eda100", "#e87ba4",
"#008300", "#4a3aa7", "#e34948", "#12a4c0", "#a66a2e"] # categorical (CVD-safe order)
GROUP_COLORS = {"Group1": "#204897", "Group2": "#e34948", "Group3": "#E7B800"}
DIVERGING_CMAP = "RdBu_r" # z-score / log2FC heatmaps — set center=0, vmin=-vmax
SEQUENTIAL_CMAP = "viridis" # magnitude / -log10 p / density
Multi-panel / composite figures
For combining several plots into one multi-panel journal figure (panels A,
B, C…), or assembling already-rendered PNG/PDF panels the user supplies, use the
sibling multipanel skill — it owns the composition discipline
(one subplot_mosaic canvas, per-panel legends, correctly placed panel letters,
text-legibility rules, image assembly). Draw each panel with the single-panel
recipes below, then compose per that skill. The recipes here each build their
own figure, so do not call them directly for a composite — copy the recipe
body onto a mosaic axis as multipanel describes.
Plot catalogue
Pick the recipe by figure type. Columns listed are the defaults — override the column-name variables to match the actual table.
| Figure | Input shape | Key columns (defaults) |
|---|---|---|
| Volcano | DEG table | log2FoldChange, padj; optional label column |
| MA plot | DEG table | baseMean, log2FoldChange, padj |
| Expression heatmap | genes × samples matrix | numeric matrix, optional index_col |
| Correlation heatmap | samples × features (numeric) | all numeric columns |
| GSEA bar plot | enrichment result | Term, NES, FDR q-val |
| GSEA dot plot | enrichment result | Term, GeneRatio, Count, Adjusted P-value |
| Box / Violin / Bar | long-form | x (category), y (numeric), optional hue |
| Ridgeline | long-form | numeric x, categorical group |
| PCA / UMAP / t-SNE | samples × features | numeric features + optional group |
| Kaplan–Meier | survival table | time, event, group |
| Manhattan | GWAS summary stats | CHR, BP, P |
| QQ plot | p-value vector | P |
| Forest | effect + CI table | label, estimate, ci_low, ci_high |
Recipes
Each is a full python script body. Adjust column names, thresholds, and the
save path. All save under figures/.
Volcano (-log10 p vs log2 fold change):
import numpy as np, pandas as pd
df = pd.read_csv("deg_results.csv").dropna(subset=["log2FoldChange", "padj"])
fc, p = df["log2FoldChange"].to_numpy(float), df["padj"].to_numpy(float)
nlp = -np.log10(np.clip(p, 1e-300, None))
fc_t, p_t = 0.58, 0.05
up, down = (fc >= fc_t) & (p < p_t), (fc <= -fc_t) & (p < p_t)
ns = ~(up | down)
fig, ax = plt.subplots(figsize=(7, 6))
ax.scatter(fc[ns], nlp[ns], c=NS, s=12, alpha=0.5, edgecolors="none", rasterized=True, label=f"NS ({ns.sum()})")
ax.scatter(fc[down], nlp[down], c=DOWN, s=18, alpha=0.85, edgecolors="none", label=f"Down ({down.sum()})")
ax.scatter(fc[up], nlp[up], c=UP, s=18, alpha=0.85, edgecolors="none", label=f"Up ({up.sum()})")
for v in (fc_t, -fc_t): ax.axvline(v, ls="--", lw=0.8, color="0.5")
ax.axhline(-np.log10(p_t), ls="--", lw=0.8, color="0.5")
lab = (df["gene"] if "gene" in df else pd.Series(df.index)).astype(str).to_numpy()
sig = np.where(up | down)[0]
top = sig[np.argsort(nlp[sig])[::-1][:10]] # standalone: top ~10; composite panel: cut to <=5
try: # repel labels so they never overlap
from adjustText import adjust_text
texts = [ax.text(fc[i], nlp[i], lab[i], fontsize=7) for i in top]
adjust_text(texts, ax=ax, expand=(1.3, 1.6),
arrowprops=dict(arrowstyle="-", color="0.6", lw=0.5))
except ImportError: # no adjustText -> label fewer, with an offset
for i in top[:5]:
ax.annotate(lab[i], (fc[i], nlp[i]), xytext=(6, 6), textcoords="offset points",
fontsize=7, ha="left", va="bottom")
ax.set_xlabel(r"$\log_{2}$ fold change"); ax.set_ylabel(r"$-\log_{10}$ padj")
ax.set_title("Volcano plot"); ax.legend(loc="upper right", markerscale=1.4)
fig.tight_layout(); fig.savefig("figures/volcano.png")
MA plot (log2 fold change vs mean expression; columns baseMean, log2FoldChange, optional padj):
import numpy as np, pandas as pd
df = pd.read_csv("deg_results.csv").dropna(subset=["baseMean", "log2FoldChange"])
x = np.log10(df["baseMean"].to_numpy(float) + 1)
fc = df["log2FoldChange"].to_numpy(float)
sig = (df["padj"].to_numpy(float) < 0.05) if "padj" in df else np.zeros(len(df), bool)
fig, ax = plt.subplots(figsize=(7, 5))
ax.scatter(x[~sig], fc[~sig], c=NS, s=10, alpha=0.5, edgecolors="none", rasterized=True, label="NS")
ax.scatter(x[sig], fc[sig], c=UP, s=14, alpha=0.85, edgecolors="none", label=f"padj<0.05 ({int(sig.sum())})")
ax.axhline(0, color="0.4", lw=0.8)
ax.set_xlabel(r"$\log_{10}$(mean norm. count + 1)"); ax.set_ylabel(r"$\log_{2}$ fold change")
ax.set_title("MA plot"); ax.legend(loc="upper right")
fig.tight_layout(); fig.savefig("figures/ma.png")
Clustered expression heatmap (z-scored, seaborn clustermap):
import seaborn as sns, pandas as pd
mat = pd.read_csv("expression.csv", index_col=0).select_dtypes("number")
g = sns.clustermap(mat, z_score=0, cmap=DIVERGING_CMAP, center=0,
figsize=(8, 8), xticklabels=True, yticklabels=mat.shape[0] <= 60,
cbar_kws={"label": "z-score"})
g.ax_col_dendrogram.set_title("Expression heatmap", pad=12)
g.figure.savefig("figures/heatmap.png")
Correlation heatmap (sample QC):
import numpy as np, seaborn as sns, pandas as pd
corr = pd.read_csv("expr.csv", index_col=0).select_dtypes("number").corr(method="pearson")
mask = np.triu(np.ones_like(corr, dtype=bool), k=1)
fig, ax = plt.subplots(figsize=(7, 6))
sns.heatmap(corr, mask=mask, cmap=DIVERGING_CMAP, vmin=-1, vmax=1, center=0,
annot=True, # if the n>20 samples then set annot=False
fmt=".2f", annot_kws={"size": 7}, square=True,
linewidths=0.5,
cbar_kws={"label": "Pearson r", "shrink": 0.7},
ax=ax)
ax.set_title("Sample correlation"); fig.tight_layout(); fig.savefig("figures/corr.png")
GSEA bar (top gene sets, colored by direction):
import pandas as pd
df = pd.read_csv("gsea.csv").dropna(subset=["Term", "NES"]).copy()
df = df.assign(_a=df["NES"].abs()).nlargest(15, "_a").sort_values("NES")
colors = [UP if v >= 0 else DOWN for v in df["NES"]]
fig, ax = plt.subplots(figsize=(7, 6))
ax.barh(df["Term"].astype(str), df["NES"], color=colors)
ax.axvline(0, color="0.4", lw=0.8); ax.set_xlabel("NES"); ax.set_title("GSEA")
fig.tight_layout(); fig.savefig("figures/gsea_bar.png")
GSEA dot plot (clusterProfiler-style; GeneRatio may be "k/n"):
import numpy as np, pandas as pd
from matplotlib.cm import ScalarMappable
from matplotlib.colors import Normalize
df = pd.read_csv("enrichment.csv").dropna(subset=["Term", "GeneRatio", "Adjusted P-value"]).copy()
df["_ratio"] = df["GeneRatio"].map(lambda v: float(v.split("/")[0]) / float(v.split("/")[1]) if isinstance(v, str) and "/" in v else float(v))
df["_nlp"] = -np.log10(np.clip(df["Adjusted P-value"].astype(float), 1e-300, None))
df = df.nlargest(15, "_nlp").sort_values("_ratio")
cnt = df["Count"].to_numpy(float); sizes = 40 + 220 * (cnt - cnt.min()) / (np.ptp(cnt) + 1e-9)
norm = Normalize(df["_nlp"].min(), df["_nlp"].max())
fig, ax = plt.subplots(figsize=(7, 6))
ax.scatter(df["_ratio"], df["Term"].astype(str), s=sizes, c=df["_nlp"], cmap=SEQUENTIAL_CMAP, norm=norm, edgecolors="0.3", linewidths=0.5, zorder=3)
ax.grid(axis="y", ls=":", color="0.8", zorder=0); ax.set_xlabel("Gene ratio"); ax.set_title("Enrichment")
cb = fig.colorbar(ScalarMappable(norm=norm, cmap=SEQUENTIAL_CMAP), ax=ax, shrink=0.6, pad=0.02)
cb.set_label(r"$-\log_{10}$ adj. $p$"); fig.tight_layout(); fig.savefig("figures/dotplot.png")
Box plot (long-form, jittered points, optional 2-group test):
import seaborn as sns, pandas as pd
from scipy import stats
df = pd.read_csv("measurements.csv"); x, y = "group", "value"
fig, ax = plt.subplots(figsize=(6, 5))
sns.boxplot(data=df, x=x, y=y, hue=x, palette=PALETTE[:df[x].nunique()], legend=False, fliersize=0, width=0.6, ax=ax)
sns.stripplot(data=df, x=x, y=y, color="0.25", size=3, alpha=0.6, ax=ax)
lv = list(df[x].dropna().unique())
if len(lv) == 2: # optional significance star
a, b = (df.loc[df[x] == l, y].dropna() for l in lv)
p = stats.mannwhitneyu(a, b, alternative="two-sided").pvalue
star = "****" if p < 1e-4 else "***" if p < 1e-3 else "**" if p < 1e-2 else "*" if p < 0.05 else "ns"
ymax = df[y].max(); h = (ymax - df[y].min()) * 0.08
ax.plot([0, 0, 1, 1], [ymax + h, ymax + 2*h, ymax + 2*h, ymax + h], lw=1.0, c="0.2")
ax.text(0.5, ymax + 2*h, star, ha="center", va="bottom")
ax.set_title(f"{y} by {x}"); fig.tight_layout(); fig.savefig("figures/box.png")
For violin swap sns.boxplot → sns.violinplot(..., inner="box", cut=0);
for bar of the mean use sns.barplot(..., errorbar="se", capsize=0.15).
Grouped embedding scatter (PCA / UMAP / t-SNE / factor scatter): (samples × features + group column):
import numpy as np, pandas as pd
def scatter_emb(ax, emb, grp, xi=0, yi=1, ev=None, name="PC", centered=True):
"""emb: (n_samples, n_comp). ev: variance explained % or None. UMAP/t-SNE -> centered=False."""
emb, grp = np.asarray(emb), pd.Series(grp).astype(str)
for g in [g for g in GROUP_COLORS if g in set(grp)]:
m = (grp == g).to_numpy()
ax.scatter(emb[m, xi], emb[m, yi], c=GROUP_COLORS[g], s=45, alpha=.9,
edgecolors="white", linewidths=.4, label=g)
if centered: # PC/factor are 0-centered and equally spaced
ax.axhline(0, lw=.5, ls="--", c="0.75"); ax.axvline(0, lw=.5, ls="--", c="0.75")
ax.set_aspect("equal", adjustable="datalim")
lab = lambda i: f"{name}{i+1}" + (f" ({ev[i]:.1f}%)" if ev is not None else "")
ax.set_xlabel(lab(xi)); ax.set_ylabel(lab(yi))
ax.spines[["top", "right"]].set_visible(False); ax.legend(frameon=False, fontsize=7)
FOR PCA: samples × features + 'group' column
from sklearn.decomposition import PCA
df = pd.read_csv("samples_features.csv", index_col=0)
grp = df.pop("group") if "group" in df else pd.Series("all", index=df.index)
X = df.select_dtypes("number")
p = PCA(n_components=min(5, *X.shape)).fit(X)
emb, ev = p.transform(X), p.explained_variance_ratio_ * 100
fig, ax = plt.subplots(figsize=(70/25.4, 70/25.4), constrained_layout=True)
scatter_emb(ax, emb, grp, 0, 1, ev, "PC")
fig.savefig("figures/pca.png", dpi=400)
FOR UMAP/t-SNE: no ev
import umap # t-SNE: from sklearn.manifold import TSNE
emb = umap.UMAP(n_neighbors=15, min_dist=0.1).fit_transform(X) # TSNE(perplexity=30).fit_transform(X)
fig, ax = plt.subplots(figsize=(70/25.4, 70/25.4), constrained_layout=True)
scatter_emb(ax, emb, grp, name="UMAP", centered=False) # 원점·축 스케일에 의미 없음
fig.savefig("figures/umap.png", dpi=400)
FOR factor scatter:
Z = pd.DataFrame(model.get_factors()["group1"], index=meta.index)
r2 = model.get_variance_explained()["r2_per_factor"]["group1"] # view × factor
ev = r2.sum(axis=0).to_numpy()
keep = [i for i, v in enumerate(ev) if v >= 5.0]
fig, ax = plt.subplots(figsize=(70/25.4, 70/25.4), constrained_layout=True)
scatter_emb(ax, Z, meta["condition"], keep[0], keep[1], ev, "Factor")
fig.savefig("figures/factor_scatter.png", dpi=400)
Kaplan–Meier (survival by group):
import pandas as pd
from lifelines import KaplanMeierFitter
df = pd.read_csv("survival.csv"); kmf = KaplanMeierFitter()
fig, ax = plt.subplots(figsize=(6, 5))
for i, g in enumerate(sorted(df["group"].unique())):
m = df["group"] == g
kmf.fit(df.loc[m, "time"], df.loc[m, "event"], label=str(g))
kmf.plot_survival_function(ax=ax, color=PALETTE[i % len(PALETTE)], ci_show=True)
ax.set_xlabel("Time"); ax.set_ylabel("Survival probability"); ax.set_ylim(0, 1.02)
ax.set_title("Kaplan–Meier"); fig.tight_layout(); fig.savefig("figures/km.png")
Manhattan (GWAS summary stats; columns CHR, BP, P):
import numpy as np, pandas as pd
df = pd.read_csv("gwas.csv").dropna(subset=["CHR", "BP", "P"]).copy()
df["CHR"] = df["CHR"].astype(str)
chrom_order = sorted(df["CHR"].unique(), key=lambda c: (len(c), c))
df["_nlp"] = -np.log10(np.clip(df["P"].astype(float), 1e-300, None))
offset, xticks, ticklabels, xpos = 0.0, [], [], np.zeros(len(df)) # lay chromosomes end-to-end
for c in chrom_order:
m = (df["CHR"] == c).to_numpy()
bp = df.loc[m, "BP"].astype(float)
xpos[m] = bp + offset
xticks.append(offset + bp.median()); ticklabels.append(c)
offset = xpos[m].max() + 1
fig, ax = plt.subplots(figsize=(180/25.4, 70/25.4), constrained_layout=True)
for i, c in enumerate(chrom_order):
m = (df["CHR"] == c).to_numpy()
ax.scatter(xpos[m], df["_nlp"].to_numpy()[m], s=6, c=["#2a78d6", "#9fb3c8"][i % 2], edgecolors="none")
ax.axhline(-np.log10(5e-8), color=UP, lw=0.8, ls="--") # genome-wide significance
ax.set_xticks(xticks); ax.set_xticklabels(ticklabels, fontsize=6)
ax.set_xlabel("Chromosome"); ax.set_ylabel(r"$-\log_{10} p$"); ax.set_title("Manhattan")
fig.savefig("figures/manhattan.png", dpi=400)
QQ plot (p-value calibration; reports genomic inflation λ):
import numpy as np, pandas as pd
from scipy import stats
p = pd.read_csv("gwas.csv")["P"].dropna().astype(float).to_numpy()
p = np.clip(np.sort(p), 1e-300, 1.0)
obs = -np.log10(p)
exp = -np.log10((np.arange(1, len(p) + 1) - 0.5) / len(p))
lam = np.median(stats.chi2.isf(p, 1)) / stats.chi2.isf(0.5, 1) # genomic inflation
fig, ax = plt.subplots(figsize=(70/25.4, 70/25.4), constrained_layout=True)
ax.scatter(exp, obs, s=6, c="#2a78d6", edgecolors="none")
lim = float(max(exp.max(), obs.max()))
ax.plot([0, lim], [0, lim], color="0.4", lw=0.8, ls="--")
ax.set_xlabel(r"Expected $-\log_{10} p$"); ax.set_ylabel(r"Observed $-\log_{10} p$")
ax.set_title(f"QQ (λ = {lam:.3f})"); fig.savefig("figures/qq.png", dpi=400)
Forest (effect size ± 95% CI per study/variant; columns label, estimate, ci_low, ci_high):
import pandas as pd
df = pd.read_csv("effects.csv").iloc[::-1].reset_index(drop=True) # first row at top
y = list(range(len(df)))
fig, ax = plt.subplots(figsize=(90/25.4, max(60, 14 * len(df)) / 25.4), constrained_layout=True)
ax.errorbar(df["estimate"], y,
xerr=[df["estimate"] - df["ci_low"], df["ci_high"] - df["estimate"]],
fmt="s", color="#204897", ecolor="0.4", capsize=2, markersize=4, lw=0.8)
ax.axvline(1.0, color=UP, lw=0.8, ls="--") # null: OR/HR=1 (use 0.0 for log-scale beta)
ax.set_yticks(y); ax.set_yticklabels(df["label"])
ax.set_xlabel("Effect size (95% CI)"); ax.set_title("Forest")
fig.savefig("figures/forest.png", dpi=400)
Best Practices
- Only plot data that exists. Never invent columns, groups, or values; if a needed column is missing, inspect the header and ask or adapt — do not fabricate.
- Workspace-relative paths only. Save under
figures/(create it if needed); never write to absolute paths like/tmpor/home/.... - Reuse the palette across a figure set so related panels share colors for
the same group / direction. Use the diverging colormap (centered at 0) for
z-scores and log2FC; the sequential colormap for magnitudes and
-log10 p. However, if the user explicitly requests a different color, use it. - Label axes and give a real title. Include units, group
n, and thresholds where relevant (e.g. volcano cutoff lines). - For vector / print output, also save a
.pdf(fig.savefig("figures/x.pdf")); text stays editable becausepdf.fonttype=42/svg.fonttype="none".
Common Pitfalls
- Wrong colormap family for the data. A sequential map on signed values
(log2FC, z-score) hides the sign. How to avoid: use the diverging colormap
with
center=0, vmin=-vmaxfor signed data, and reserve the sequential colormap for unsigned magnitudes only. - Columns don't match the recipe (
KeyError). The recipe's default column names rarely match the real table. How to avoid: read the header first and override the*_col/x/yvariables to the real names; never invent a missing column — pick a figure the data supports instead. - Overcrowded / overlapping labels. Annotating every volcano point, or
cramming long pathway names on an x-axis, produces unreadable overlap. How to
avoid: label only the top ~10 by
|log2FC| × -log10 p(≤5 inside a composite panel) and repel withadjustText; move long category names to a horizontal y-axis, or rotate 90° and wrap to ≤26 chars; skip labels rather than dumping colliding text. - Dot / bubble markers all one size or too small. A size encoding that maps to
invisible dots carries no information. How to avoid: floor the size mapping
(
sin ~[25, 220]) so the smallest stays visible, and confirm the size column actually varies. - Legend or clustermap title collides. A legend sits on top of the data, or a
clustermaptitle is cropped. How to avoid: move the legend to a free corner or outside the axes (or shrink its font); put a clustermap title ong.ax_col_dendrogram, not a figuresuptitle. - Empty / all-NS plot. Nothing passes the threshold because the columns or scale are wrong. How to avoid: confirm the p-value and fold-change columns are numeric and that thresholds match the data scale (adjusted vs raw p).
For multi-panel composition issues (panel letters misplaced, legends floating in
margins, panels colliding), see the multipanel skill.
Further Reading
- Matplotlib documentation — https://matplotlib.org/stable/
- Seaborn documentation — https://seaborn.pydata.org/
- Wong, "Points of view: Color blindness", Nature Methods (2011) — https://www.nature.com/articles/nmeth.1618
- ColorBrewer diverging/sequential palettes — https://colorbrewer2.org/
Plotly Interactive Plots
Overview
Plotly is a Python library for producing interactive, web-ready figures backed by HTML and JavaScript. It exposes two complementary APIs: plotly.express (px) provides a high-level, DataFrame-oriented interface for generating common chart types in one line, while plotly.graph_objects (go) offers fine-grained control over every trace, axis, and layout property. Figures are fully interactive by default — supporting hover tooltips, zoom, pan, and click events — and can be embedded in web pages, Jupyter notebooks, or built into web applications using the Dash framework.
When to Use
- You need hover tooltips that display gene names, p-values, or sample metadata without cluttering the static figure.
- You are building a multi-panel interactive dashboard for dose-response curves, patient cohorts, or multi-condition comparisons.
- You want to share figures as self-contained HTML files that non-programmers can explore in a browser.
- You need 3D scatter or surface plots for structural biology, conformational landscapes, or PCA of high-dimensional data.
- You are creating heatmaps of gene expression or correlation matrices where users need to zoom into specific gene clusters.
- You require animation frames to show time-series or treatment-response trajectories.
- Use
seaborninstead when you need automatic statistical aggregation (confidence intervals, regression fits) with minimal code. - Use
matplotlibwhen you need fine-grained control over every axis element for print-ready publication figures at exact journal specifications.
Prerequisites
- Python packages:
plotly,kaleido(static image export),pandas,numpy - Data requirements: pandas DataFrames or NumPy arrays; long-form (tidy) data works best with
px - Environment: Jupyter Lab/Notebook (inline rendering), or save as HTML for browser display
pip install plotly kaleido pandas numpy
For Jupyter Lab inline rendering (if not automatic):
pip install "jupyterlab>=3" ipywidgets
Quick Start
import plotly.express as px
import pandas as pd
# Gene expression scatter with hover info
df = pd.DataFrame({
"log2FC": [-3.1, 0.2, 1.8, 2.5, -0.5, 4.1],
"neg_log10_padj": [8.2, 0.4, 2.1, 6.8, 0.1, 9.3],
"gene": ["BRCA1", "MYC", "TP53", "EGFR", "CDKN1A", "KRAS"],
"significance": ["sig", "ns", "ns", "sig", "ns", "sig"],
})
fig = px.scatter(
df, x="log2FC", y="neg_log10_padj",
color="significance", hover_name="gene",
title="Volcano Plot — Treatment vs Control",
)
fig.show()
Core API
Module 1: px Scatter and Line — Relational Plots
px.scatter() and px.line() map DataFrame columns to visual encodings (color, symbol, size) and automatically populate hover tooltips from hover_data.
import plotly.express as px
import pandas as pd
import numpy as np
# Dose-response scatter: color by drug, symbol by cell line
np.random.seed(42)
df = pd.DataFrame({
"dose_uM": np.tile([0.01, 0.1, 1, 10, 100], 4),
"viability": np.clip(np.random.normal(
[100, 90, 70, 40, 10] * 4, 5), 0, 110),
"drug": ["DrugA"] * 5 + ["DrugA"] * 5 + ["DrugB"] * 5 + ["DrugB"] * 5,
"cell_line": ["HCT116"] * 10 + ["MCF7"] * 10,
"replicate": np.tile([1, 2, 3, 4, 5], 4),
})
fig = px.scatter(
df, x="dose_uM", y="viability",
color="drug", symbol="cell_line",
log_x=True,
hover_data={"replicate": True, "dose_uM": ":.2f"},
labels={"viability": "Cell Viability (%)", "dose_uM": "Dose (µM)"},
title="Dose-Response by Drug and Cell Line",
)
fig.show()
print(f"Figure has {len(fig.data)} traces")
# Time-course gene expression line plot
time_df = pd.DataFrame({
"hour": list(range(0, 25, 4)) * 3,
"expression": [1.0, 1.8, 3.2, 4.5, 3.8, 2.1, 1.2,
1.0, 2.5, 5.1, 6.8, 5.5, 3.2, 1.8,
1.0, 1.1, 1.0, 1.2, 1.1, 1.0, 0.9],
"gene": ["MYC"] * 7 + ["EGFR"] * 7 + ["GAPDH"] * 7,
})
fig = px.line(
time_df, x="hour", y="expression",
color="gene", markers=True,
labels={"expression": "Relative Expression (log2)", "hour": "Time (h)"},
title="Time-Course Gene Expression",
)
fig.update_traces(line=dict(width=2.5), marker=dict(size=8))
fig.show()
Module 2: px Statistical Plots — Distributions and Categories
px.box(), px.violin(), px.histogram(), and px.strip() produce publication-ready distribution summaries with built-in grouping.
import plotly.express as px
import pandas as pd
import numpy as np
# Violin + strip overlay: expression by cell type
np.random.seed(7)
n = 60
cell_data = pd.DataFrame({
"expression": np.concatenate([
np.random.normal(4.2, 0.8, n),
np.random.normal(6.5, 1.2, n),
np.random.normal(2.8, 0.6, n),
]),
"cell_type": ["T cell"] * n + ["B cell"] * n + ["NK cell"] * n,
"patient_id": np.tile([f"P{i:02d}" for i in range(1, 11)], 18),
})
fig = px.violin(
cell_data, x="cell_type", y="expression",
color="cell_type", box=True, points="all",
hover_data=["patient_id"],
labels={"expression": "CD3E Expression (log2 CPM)"},
title="CD3E Expression Across Cell Types",
)
fig.update_traces(jitter=0.3, pointpos=-1.5)
fig.show()
print(f"Cells per type: {cell_data.groupby('cell_type').size().to_dict()}")
# Histogram with rug: distribution of fold changes
fc_df = pd.DataFrame({
"log2FC": np.concatenate([
np.random.normal(0.1, 0.8, 500), # not DE genes
np.random.normal(2.5, 0.4, 50), # upregulated
np.random.normal(-2.3, 0.4, 40), # downregulated
]),
"category": ["background"] * 500 + ["up"] * 50 + ["down"] * 40,
})
fig = px.histogram(
fc_df, x="log2FC", color="category",
nbins=60, barmode="overlay", opacity=0.7,
marginal="rug",
labels={"log2FC": "log2 Fold Change", "count": "Gene Count"},
title="Distribution of Fold Changes (DESeq2 Results)",
color_discrete_map={"background": "gray", "up": "crimson", "down": "steelblue"},
)
fig.show()
Module 3: px Heatmap and Matrix — Gene Expression and Correlations
px.imshow() renders 2D arrays or DataFrames as color-encoded matrices, ideal for expression heatmaps and correlation matrices.
import plotly.express as px
import pandas as pd
import numpy as np
# Gene expression heatmap (genes × samples)
np.random.seed(12)
genes = [f"Gene_{g}" for g in ["BRCA1", "TP53", "EGFR", "MYC", "KRAS",
"CDKN1A", "RB1", "PTEN", "VHL", "APC"]]
samples = [f"S{i:02d}" for i in range(1, 9)]
expr_matrix = pd.DataFrame(
np.random.normal(0, 1.5, (10, 8)) +
np.array([2, -1, 3, -2, 1, -3, 0, 2, -1, 3]).reshape(-1, 1),
index=genes, columns=samples,
)
fig = px.imshow(
expr_matrix,
color_continuous_scale="RdBu_r",
color_continuous_midpoint=0,
aspect="auto",
labels={"color": "log2 Expression (z-score)"},
title="Gene Expression Heatmap",
)
fig.update_xaxes(side="top")
fig.update_layout(width=600, height=500)
fig.show()
print(f"Heatmap shape: {expr_matrix.shape} (genes × samples)")
# Correlation matrix heatmap
from itertools import combinations
markers = ["IL6", "TNF", "CXCL10", "IFNg", "IL10", "IL1B", "CCL2", "IL17A"]
np.random.seed(3)
raw = np.random.multivariate_normal(
mean=np.zeros(8),
cov=np.eye(8) * 0.3 + 0.7,
size=80,
)
corr_df = pd.DataFrame(raw, columns=markers).corr()
fig = px.imshow(
corr_df,
color_continuous_scale="RdBu_r",
color_continuous_midpoint=0,
zmin=-1, zmax=1,
text_auto=".2f",
title="Cytokine Correlation Matrix (n=80 patients)",
)
fig.update_traces(textfont_size=10)
fig.show()
Module 4: go Graph Objects — Full Trace Control
plotly.graph_objects provides fine-grained access to every trace property: marker symbols, error bars, fill areas, and multi-trace layouts. Essential when px lacks the flexibility you need.
import plotly.graph_objects as go
import numpy as np
# Volcano plot built from scratch with go.Scatter
np.random.seed(99)
n_genes = 5000
log2fc = np.random.normal(0, 1.2, n_genes)
pval = np.random.uniform(0, 1, n_genes) ** 2 # skew toward low p-values
neg_log10_p = -np.log10(pval + 1e-300)
gene_names = [f"Gene_{i:04d}" for i in range(n_genes)]
# Classify genes
sig_mask = (np.abs(log2fc) > 1.5) & (neg_log10_p > 3)
up_mask = sig_mask & (log2fc > 0)
down_mask = sig_mask & (log2fc < 0)
ns_mask = ~sig_mask
fig = go.Figure()
# Non-significant background
fig.add_trace(go.Scatter(
x=log2fc[ns_mask], y=neg_log10_p[ns_mask],
mode="markers",
name="Not significant",
marker=dict(color="lightgray", size=4, opacity=0.5),
text=[gene_names[i] for i in np.where(ns_mask)[0]],
hovertemplate="<b>%{text}</b><br>log2FC: %{x:.2f}<br>-log10(p): %{y:.2f}<extra></extra>",
))
# Upregulated
fig.add_trace(go.Scatter(
x=log2fc[up_mask], y=neg_log10_p[up_mask],
mode="markers",
name=f"Up ({up_mask.sum()} genes)",
marker=dict(color="crimson", size=7, opacity=0.8),
text=[gene_names[i] for i in np.where(up_mask)[0]],
hovertemplate="<b>%{text}</b><br>log2FC: %{x:.2f}<br>-log10(p): %{y:.2f}<extra></extra>",
))
# Downregulated
fig.add_trace(go.Scatter(
x=log2fc[down_mask], y=neg_log10_p[down_mask],
mode="markers",
name=f"Down ({down_mask.sum()} genes)",
marker=dict(color="steelblue", size=7, opacity=0.8),
text=[gene_names[i] for i in np.where(down_mask)[0]],
hovertemplate="<b>%{text}</b><br>log2FC: %{x:.2f}<br>-log10(p): %{y:.2f}<extra></extra>",
))
# Threshold lines
fig.add_hline(y=3, line_dash="dash", line_color="black", line_width=1)
fig.add_vline(x=1.5, line_dash="dash", line_color="black", line_width=1)
fig.add_vline(x=-1.5, line_dash="dash", line_color="black", line_width=1)
fig.update_layout(
title="Volcano Plot (Treatment vs Control, n=5000 genes)",
xaxis_title="log2 Fold Change",
yaxis_title="-log10(adjusted p-value)",
legend=dict(x=0.01, y=0.99),
width=750, height=550,
)
fig.show()
print(f"Up: {up_mask.sum()}, Down: {down_mask.sum()}, NS: {ns_mask.sum()}")
# Bar chart with error bars: mean ± SEM per treatment group
groups = ["Vehicle", "DrugA 1µM", "DrugA 10µM", "DrugB 1µM", "DrugB 10µM"]
means = [100.0, 82.3, 54.7, 91.2, 68.5]
sems = [3.2, 4.1, 3.8, 3.5, 4.7]
fig = go.Figure(go.Bar(
x=groups, y=means,
error_y=dict(type="data", array=sems, visible=True),
marker_color=["gray", "lightsalmon", "crimson", "lightblue", "steelblue"],
hovertemplate="%{x}<br>Mean: %{y:.1f}%<br>SEM: ±%{error_y.array:.1f}%<extra></extra>",
))
fig.update_layout(
title="Cell Viability by Treatment (Mean ± SEM, n=6)",
yaxis_title="Viability (%)", yaxis_range=[0, 120],
xaxis_title="Treatment Group",
showlegend=False,
)
fig.show()
Module 5: 3D and Specialized Charts
Plotly supports 3D scatter, surface plots, parallel coordinates, and treemaps — chart types unavailable in seaborn or standard matplotlib.
import plotly.express as px
import numpy as np
import pandas as pd
# 3D PCA scatter: cell clusters in embedding space
np.random.seed(42)
n_per_cluster = 80
cluster_centers = {"T cell": [3, 2, 1], "B cell": [-3, 1, 2], "Monocyte": [0, -3, -1]}
records = []
for ctype, center in cluster_centers.items():
coords = np.random.normal(center, 0.8, (n_per_cluster, 3))
for row in coords:
records.append({
"PC1": row[0], "PC2": row[1], "PC3": row[2],
"cell_type": ctype,
"score": np.random.uniform(0.5, 1.0),
})
pca_df = pd.DataFrame(records)
fig = px.scatter_3d(
pca_df, x="PC1", y="PC2", z="PC3",
color="cell_type", size="score", opacity=0.7,
hover_data={"score": ":.3f"},
title="3D PCA — Single-Cell Transcriptomics",
)
fig.update_traces(marker=dict(sizeref=0.04))
fig.show()
print(f"Total cells: {len(pca_df)}, clusters: {pca_df['cell_type'].nunique()}")
import plotly.graph_objects as go
import numpy as np
import pandas as pd
# Parallel coordinates: multi-parameter drug screen
np.random.seed(5)
n_compounds = 200
drug_df = pd.DataFrame({
"MW": np.random.normal(380, 60, n_compounds),
"logP": np.random.uniform(-1, 6, n_compounds),
"HBA": np.random.randint(2, 10, n_compounds),
"HBD": np.random.randint(0, 6, n_compounds),
"IC50_nM": np.random.lognormal(4, 1.5, n_compounds),
"selectivity": np.random.uniform(1, 100, n_compounds),
})
fig = px.parallel_coordinates(
drug_df,
color="IC50_nM",
color_continuous_scale="RdYlGn_r",
dimensions=["MW", "logP", "HBA", "HBD", "IC50_nM", "selectivity"],
labels={
"MW": "MW (Da)", "logP": "logP",
"HBA": "H-Bond Acceptors", "HBD": "H-Bond Donors",
"IC50_nM": "IC50 (nM)", "selectivity": "Selectivity Index",
},
title="Drug Candidate Properties — Parallel Coordinates",
)
fig.show()
Module 6: Subplots and Export
make_subplots() creates multi-panel layouts with shared axes, mixed chart types, and independent traces per panel. fig.write_html() exports interactive figures; fig.write_image() exports static files via kaleido.
from plotly.subplots import make_subplots
import plotly.graph_objects as go
import numpy as np
# Two-panel: raw data + summary statistics
np.random.seed(77)
doses = [0.01, 0.1, 1, 10, 100]
drugs = {"DrugA": {"EC50": 1.0, "hill": 1.5}, "DrugB": {"EC50": 8.0, "hill": 0.9}}
def hill_curve(dose, ec50, hill, top=100, bottom=0):
return bottom + (top - bottom) / (1 + (ec50 / dose) ** hill)
fig = make_subplots(
rows=1, cols=2,
subplot_titles=["Dose-Response Curves", "IC50 Comparison"],
shared_yaxes=False,
)
colors = {"DrugA": "crimson", "DrugB": "steelblue"}
ic50_values = []
for drug, params in drugs.items():
# Smooth fit curve
x_fit = np.logspace(-2, 2, 200)
y_fit = hill_curve(x_fit, params["EC50"], params["hill"])
fig.add_trace(go.Scatter(
x=x_fit, y=y_fit, mode="lines",
name=f"{drug} fit", line=dict(color=colors[drug], width=2.5),
), row=1, col=1)
# Noisy data points
y_data = [hill_curve(d, params["EC50"], params["hill"]) +
np.random.normal(0, 4) for d in doses]
fig.add_trace(go.Scatter(
x=doses, y=y_data, mode="markers",
name=f"{drug} data", marker=dict(color=colors[drug], size=9),
showlegend=False,
), row=1, col=1)
ic50_values.append(params["EC50"])
# Bar chart of IC50 values
fig.add_trace(go.Bar(
x=list(drugs.keys()), y=ic50_values,
marker_color=list(colors.values()),
showlegend=False,
hovertemplate="%{x}<br>IC50: %{y:.2f} µM<extra></extra>",
), row=1, col=2)
fig.update_xaxes(type="log", title_text="Dose (µM)", row=1, col=1)
fig.update_yaxes(title_text="Viability (%)", row=1, col=1)
fig.update_xaxes(title_text="Drug", row=1, col=2)
fig.update_yaxes(title_text="IC50 (µM)", row=1, col=2)
fig.update_layout(title="Dose-Response Dashboard", height=450, width=850)
fig.show()
print(f"Subplots: {len(fig.data)} traces across 2 panels")
# Export to HTML (interactive) and PNG (static)
# Requires: pip install kaleido
fig.write_html("dose_response_dashboard.html")
print("Saved: dose_response_dashboard.html (interactive, shareable)")
fig.write_image("dose_response_dashboard.png", width=1200, height=600, scale=2)
print("Saved: dose_response_dashboard.png (300 DPI equivalent with scale=2)")
fig.write_image("dose_response_dashboard.svg")
print("Saved: dose_response_dashboard.svg (vector, editable in Inkscape/Illustrator)")
Common Workflows
Workflow 1: Interactive Volcano Plot with Gene Annotations
Goal: Build a fully annotated volcano plot from DESeq2 results, with gene-name hover tooltips, threshold lines, and highlighted hit labels for sharing as HTML.
import plotly.graph_objects as go
import pandas as pd
import numpy as np
# Simulate DESeq2 output (replace with pd.read_csv("deseq2_results.csv"))
np.random.seed(42)
n = 3000
df = pd.DataFrame({
"gene": [f"GENE_{i:04d}" for i in range(n)],
"log2FC": np.random.normal(0, 1.0, n),
"padj": np.clip(np.random.exponential(0.1, n), 1e-20, 1.0),
"baseMean": np.random.lognormal(5, 1.5, n),
})
# Inject some hits
df.loc[:20, "log2FC"] = np.random.uniform(2.5, 5, 21)
df.loc[:20, "padj"] = np.random.uniform(1e-15, 1e-5, 21)
df.loc[21:35, "log2FC"] = np.random.uniform(-4, -2, 15)
df.loc[21:35, "padj"] = np.random.uniform(1e-12, 1e-4, 15)
df["neg_log10_padj"] = -np.log10(df["padj"].clip(1e-300))
# Classify
FC_THRESH, P_THRESH = 1.5, 2.0 # |log2FC| > 1.5, -log10(padj) > 2
df["category"] = "NS"
df.loc[(df["log2FC"] > FC_THRESH) & (df["neg_log10_padj"] > P_THRESH), "category"] = "Up"
df.loc[(df["log2FC"] < -FC_THRESH) & (df["neg_log10_padj"] > P_THRESH), "category"] = "Down"
color_map = {"NS": "lightgray", "Up": "crimson", "Down": "steelblue"}
size_map = {"NS": 4, "Up": 7, "Down": 7}
opacity_map = {"NS": 0.4, "Up": 0.85, "Down": 0.85}
fig = go.Figure()
for cat in ["NS", "Up", "Down"]:
sub = df[df["category"] == cat]
fig.add_trace(go.Scatter(
x=sub["log2FC"], y=sub["neg_log10_padj"],
mode="markers",
name=f"{cat} (n={len(sub)})",
marker=dict(
color=color_map[cat],
size=size_map[cat],
opacity=opacity_map[cat],
),
customdata=sub[["gene", "padj", "baseMean"]].values,
hovertemplate=(
"<b>%{customdata[0]}</b><br>"
"log2FC: %{x:.3f}<br>"
"padj: %{customdata[1]:.2e}<br>"
"baseMean: %{customdata[2]:.1f}<extra></extra>"
),
))
# Threshold lines
fig.add_hline(y=P_THRESH, line_dash="dot", line_color="black", line_width=1.2,
annotation_text=f"padj=0.01", annotation_position="right")
fig.add_vline(x=FC_THRESH, line_dash="dot", line_color="black", line_width=1.2)
fig.add_vline(x=-FC_THRESH, line_dash="dot", line_color="black", line_width=1.2)
# Label top 5 upregulated hits by significance
top_up = df[df["category"] == "Up"].nlargest(5, "neg_log10_padj")
for _, row in top_up.iterrows():
fig.add_annotation(
x=row["log2FC"], y=row["neg_log10_padj"],
text=row["gene"], showarrow=True,
arrowhead=2, arrowsize=1, arrowcolor="crimson",
font=dict(size=9, color="crimson"),
xshift=8, yshift=5,
)
fig.update_layout(
title="Volcano Plot — Treatment vs Control (DESeq2)",
xaxis_title="log2 Fold Change",
yaxis_title="-log10(adjusted p-value)",
legend=dict(x=0.01, y=0.99, bordercolor="lightgray", borderwidth=1),
width=800, height=560,
plot_bgcolor="white",
)
fig.update_xaxes(showgrid=True, gridcolor="lightgray", zeroline=True, zerolinecolor="darkgray")
fig.update_yaxes(showgrid=True, gridcolor="lightgray")
fig.write_html("volcano_interactive.html")
print(f"Up: {(df.category=='Up').sum()}, Down: {(df.category=='Down').sum()}")
print("Saved: volcano_interactive.html")
Workflow 2: Multi-Panel Dose-Response Dashboard with make_subplots
Goal: Display dose-response curves for multiple drugs across cell lines in a grid layout with a shared color scale and consistent formatting.
from plotly.subplots import make_subplots
import plotly.graph_objects as go
import numpy as np
import pandas as pd
# Simulated IC50 data for 3 drugs × 3 cell lines
np.random.seed(10)
drugs = ["DrugA", "DrugB", "DrugC"]
cell_lines = ["HCT116", "MCF7", "A549"]
doses = np.logspace(-2, 2, 7) # 0.01 to 100 µM
def hill(x, ec50, hill_n, top=100, bottom=0):
return bottom + (top - bottom) / (1 + (ec50 / x) ** hill_n)
ec50_table = {
("DrugA", "HCT116"): 0.5, ("DrugA", "MCF7"): 2.0, ("DrugA", "A549"): 8.0,
("DrugB", "HCT116"): 5.0, ("DrugB", "MCF7"): 0.8, ("DrugB", "A549"): 15.0,
("DrugC", "HCT116"): 12.0, ("DrugC", "MCF7"): 6.0, ("DrugC", "A549"): 1.2,
}
palette = px_colors = ["#EF553B", "#636EFA", "#00CC96", "#AB63FA", "#FFA15A",
"#19D3F3", "#FF6692", "#B6E880", "#FF97FF"]
fig = make_subplots(
rows=len(drugs), cols=len(cell_lines),
subplot_titles=[f"{d} / {c}" for d in drugs for c in cell_lines],
shared_xaxes=True, shared_yaxes=True,
vertical_spacing=0.08, horizontal_spacing=0.04,
)
for r, drug in enumerate(drugs, start=1):
for c, cell_line in enumerate(cell_lines, start=1):
ec50 = ec50_table[(drug, cell_line)]
x_fit = np.logspace(-2, 2, 200)
y_fit = hill(x_fit, ec50, hill_n=1.5)
# Noisy replicate data
y_data = np.array([hill(d, ec50, 1.5) + np.random.normal(0, 5) for d in doses])
color = palette[(r - 1) * len(cell_lines) + (c - 1)]
show_legend = (c == 1 and r == 1)
fig.add_trace(go.Scatter(
x=x_fit, y=y_fit, mode="lines",
line=dict(color=color, width=2),
name=f"{drug}/{cell_line}",
showlegend=False,
hovertemplate=f"{drug} in {cell_line}<br>Dose: %{{x:.2f}} µM<br>Viability: %{{y:.1f}}%<extra></extra>",
), row=r, col=c)
fig.add_trace(go.Scatter(
x=doses, y=np.clip(y_data, 0, 110), mode="markers",
marker=dict(color=color, size=7, opacity=0.8),
showlegend=False,
hovertemplate=f"Measured<br>Dose: %{{x:.2f}} µM<br>Viability: %{{y:.1f}}%<extra></extra>",
), row=r, col=c)
# IC50 annotation
fig.add_annotation(
x=np.log10(ec50), y=50,
text=f"IC50={ec50:.1f}µM",
font=dict(size=8), showarrow=False,
xref=f"x{(r-1)*len(cell_lines)+c if (r-1)*len(cell_lines)+c > 1 else ''}",
yref=f"y{(r-1)*len(cell_lines)+c if (r-1)*len(cell_lines)+c > 1 else ''}",
)
# Apply log scale to all x-axes
for i in range(1, len(drugs) * len(cell_lines) + 1):
axis_key = f"xaxis{i if i > 1 else ''}"
fig.layout[axis_key].update(type="log", title_text="Dose (µM)" if i > 6 else "")
for i in range(1, len(drugs) * len(cell_lines) + 1):
axis_key = f"yaxis{i if i > 1 else ''}"
fig.layout[axis_key].update(range=[-5, 115],
title_text="Viability (%)" if i in [1, 4, 7] else "")
fig.update_layout(
title="Dose-Response Dashboard — 3 Drugs × 3 Cell Lines",
height=700, width=900,
)
fig.write_html("dose_response_dashboard.html")
print("Saved: dose_response_dashboard.html")
print(f"Grid: {len(drugs)} drugs × {len(cell_lines)} cell lines = {len(drugs)*len(cell_lines)} panels")
Key Parameters
| Parameter | Module / Function | Default | Range / Options | Effect |
|---|---|---|---|---|
color | px.* | None | Column name | Maps a DataFrame column to trace color; auto-assigns palette |
hover_data | px.* | {} | Dict or list of column names | Extra columns shown in hover tooltip |
log_x / log_y | px.* | False | True, False | Apply log10 scale to x or y axis |
facet_col / facet_row | px.* | None | Column name | Split into subplot grid by a categorical variable |
color_continuous_scale | px.imshow, px.scatter | "plasma" | "RdBu_r", "Viridis", "Hot", etc. | Colormap for continuous color mapping |
color_continuous_midpoint | px.imshow | None | Any numeric | Centers the diverging colormap at this value (use 0 for z-scores) |
barmode | px.histogram, px.bar | "relative" | "relative", "overlay", "group" | How multiple bar traces are displayed |
opacity | go.Scatter, px.* | 1.0 | 0.0–1.0 | Point/bar transparency |
size / sizeref | go.Scatter | 6 / auto | Positive numeric | Marker size; sizeref normalizes sizes across traces |
line_dash | fig.add_hline, go.Scatter | "solid" | "solid", "dash", "dot", "dashdot" | Line style for reference lines and traces |
shared_xaxes / shared_yaxes | make_subplots | False | True, False, "rows", "cols" | Link axes across subplot panels |
scale | fig.write_image | 1 | 1–4 | Resolution multiplier for PNG export (use 2 for ~150 DPI) |
Best Practices
-
Prefer
pxfor DataFrame data, fall back togofor multi-trace composition. Usepx.scatter()and its siblings for 80% of plots. Switch togowhen you need traces with different types in the same figure (e.g., scatter + filled area) or need fine-grained per-trace control.# Correct: px for simple grouped plots fig = px.box(df, x="treatment", y="expression", color="genotype") # Correct: go when px cannot express the structure fig = go.Figure() fig.add_trace(go.Scatter(x=x, y=y_upper, fill="tonexty", ...)) fig.add_trace(go.Scatter(x=x, y=y_lower, ...)) -
Always include
hovertemplatefor scientific figures. The default tooltip shows raw coordinates without units or gene names. A custom template withcustomdataprovides full biological context.fig.add_trace(go.Scatter( customdata=df[["gene", "padj"]].values, hovertemplate="<b>%{customdata[0]}</b><br>padj: %{customdata[1]:.2e}<extra></extra>", )) -
Export HTML for sharing, PNG/SVG for journals.
fig.write_html()produces a self-contained file with no external dependencies. Usescale=2or higher withwrite_image()to achieve sufficient resolution for print. -
Don't use
fig.show()in batch scripts. In non-interactive contexts (CI, HPC, cron jobs),fig.show()may open a browser window or fail. Usewrite_html()orwrite_image()exclusively. -
Use
color_continuous_midpoint=0for diverging palettes on z-score data. Without it, the midpoint color defaults to the data midpoint, not zero, misrepresenting symmetric fold changes or correlations.fig = px.imshow(corr_matrix, color_continuous_scale="RdBu_r", color_continuous_midpoint=0, zmin=-1, zmax=1) -
Set
plot_bgcolor="white"for publication figures. Plotly defaults to a light-gray grid background. White background with subtle gridlines is cleaner for most scientific contexts.fig.update_layout(plot_bgcolor="white") fig.update_xaxes(showgrid=True, gridcolor="lightgray") fig.update_yaxes(showgrid=True, gridcolor="lightgray")
Common Recipes
Recipe: Dropdown Menu to Toggle Between Conditions
When to use: Overlay multiple conditions in one figure with a dropdown button to show/hide individual traces cleanly.
import plotly.graph_objects as go
import numpy as np
conditions = ["Untreated", "DrugA", "DrugB"]
colors = ["gray", "crimson", "steelblue"]
np.random.seed(1)
x = np.linspace(0, 24, 49)
fig = go.Figure()
for i, (cond, color) in enumerate(zip(conditions, colors)):
y = np.sin(x / 4 + i * 0.5) * (1 - i * 0.2) + np.random.normal(0, 0.05, len(x))
fig.add_trace(go.Scatter(
x=x, y=y, mode="lines+markers",
name=cond, line=dict(color=color, width=2),
visible=(i == 0), # only first trace visible initially
))
# One button per condition (shows only that trace)
buttons = []
for i, cond in enumerate(conditions):
visibility = [j == i for j in range(len(conditions))]
buttons.append(dict(label=cond, method="update",
args=[{"visible": visibility}, {"title": f"Gene Expression — {cond}"}]))
# "Show All" button
buttons.append(dict(label="Show All", method="update",
args=[{"visible": [True] * len(conditions)}, {"title": "Gene Expression — All Conditions"}]))
fig.update_layout(
updatemenus=[dict(type="dropdown", x=0.01, y=1.15, showactive=True, buttons=buttons)],
title="Gene Expression — Untreated",
xaxis_title="Time (h)", yaxis_title="Relative Expression",
)
fig.show()
Recipe: Annotating Specific Hits with Arrows
When to use: Label outliers, drug hits, or significant genes directly on the figure without cluttering non-annotated points.
import plotly.graph_objects as go
import numpy as np
import pandas as pd
np.random.seed(33)
df = pd.DataFrame({
"x": np.random.normal(0, 1.5, 300),
"y": np.random.normal(0, 1.5, 300),
"gene": [f"G{i:03d}" for i in range(300)],
})
# Inject top hits
hits = pd.DataFrame({
"x": [3.2, -2.8, 2.5, -3.5],
"y": [4.1, 3.8, -3.2, -2.9],
"gene": ["BRCA1", "TP53", "EGFR", "KRAS"],
})
fig = go.Figure()
fig.add_trace(go.Scatter(
x=df["x"], y=df["y"], mode="markers",
marker=dict(color="lightgray", size=5, opacity=0.6),
text=df["gene"],
hovertemplate="<b>%{text}</b><br>x: %{x:.2f}, y: %{y:.2f}<extra></extra>",
name="Background",
))
fig.add_trace(go.Scatter(
x=hits["x"], y=hits["y"], mode="markers",
marker=dict(color="crimson", size=10, symbol="diamond"),
text=hits["gene"],
hovertemplate="<b>%{text}</b> [HIT]<br>x: %{x:.2f}, y: %{y:.2f}<extra></extra>",
name="Hits",
))
for _, row in hits.iterrows():
fig.add_annotation(
x=row["x"], y=row["y"],
text=f"<b>{row['gene']}</b>",
showarrow=True, arrowhead=2, arrowwidth=1.5,
arrowcolor="crimson", font=dict(size=11, color="crimson"),
ax=25, ay=-30, # arrow offset in pixels
bgcolor="rgba(255,255,255,0.7)", bordercolor="crimson", borderwidth=1,
)
fig.update_layout(
title="Hit Identification with Arrow Annotations",
xaxis_title="Score A", yaxis_title="Score B",
plot_bgcolor="white",
)
fig.show()
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
ValueError: kaleido is required for static image export | kaleido not installed | pip install kaleido; verify with import kaleido |
| Blank figure in Jupyter Notebook | Renderer not configured | Run import plotly.io as pio; pio.renderers.default = "notebook" or upgrade JupyterLab to ≥3 |
fig.show() opens blank browser tab | No data in figure or offline renderer issue | Check len(fig.data) > 0; use pio.renderers.default = "browser" |
| Hover tooltips show wrong values | customdata index mismatch in hovertemplate | Verify customdata column order matches %{customdata[N]} indices in template |
| Colors not assigned consistently across traces | px re-orders palette when category counts differ | Use color_discrete_map={"Cat1": "#color1", ...} to pin colors explicitly |
write_image produces blurry PNG | Default scale=1 is too low for print | Use fig.write_image("fig.png", scale=2) for 150 DPI or scale=4 for 300 DPI |
Subplots x-axes not all log-scaled after shared_xaxes=True | Shared axis only synchronizes range, not type | Iterate over all xaxis keys in fig.layout and set type="log" explicitly |
| Large datasets slow to render in browser | Too many individual points in a single scatter trace | Downsample background noise points; keep labeled hits as a separate, smaller trace |
fig.update_layout does not apply to subplot axes | Multi-panel figures use indexed axes (xaxis2, xaxis3) | Use fig.update_xaxes() (applies to all) or target fig.layout["xaxis2"] explicitly |
Related Skills
- seaborn-statistical-plots — use for statistical aggregation (confidence intervals, regression), publication-quality static figures with minimal code, and when matplotlib-level output is required
- matplotlib-scientific-plotting — use for full control over every figure element, custom layouts, embedded text rendering, and journal-specification figure preparation
- pydeseq2-differential-expression — volcano plot outputs from DESeq2 results are a primary input for the interactive volcano workflow above
- scanpy-scrna-seq — Scanpy's UMAP embeddings can be visualized interactively in 3D with
px.scatter_3d
References
- Plotly Python Documentation — official API reference, examples gallery, and getting-started guides
- Plotly Express API Reference — complete
pxfunction signatures and parameters - Plotly Graph Objects Reference — full
gotrace and layout attribute reference - Plotly GitHub Repository — source code, issue tracker, changelog
- Kaleido Static Image Export — dependency for
write_image()PNG/SVG/PDF export
scientific-visualization
Overview
Effective scientific visualization communicates data clearly, honestly, and accessibly. Poor chart choices, misleading axes, or inaccessible color palettes can obscure findings or introduce bias. This guide covers the full workflow of scientific figure preparation: from selecting the right chart type for your data structure through color theory, accessibility, and journal submission formatting requirements.
Key Concepts
Chart Type and Data Type Alignment
Every chart type is optimized for a specific data structure. Mismatches (e.g., pie charts for continuous distributions, bar charts for time series) hide structure and distort perception.
| Data Type | Recommended Chart | Avoid |
|---|---|---|
| Continuous distribution (1 group) | Histogram, violin plot, ridge plot | Bar chart with mean only |
| Continuous distribution (2–5 groups) | Violin + boxplot overlay, beeswarm | Grouped bar chart |
| Two continuous variables, correlation | Scatter plot, hexbin (large N) | Line chart without temporal order |
| Categorical counts / proportions | Bar chart (horizontal for long labels) | Pie chart (>4 categories) |
| Change over time (continuous) | Line chart | Bar chart |
| Change over time (sparse events) | Step chart, event raster | Connected scatter |
| Part-to-whole (≤5 parts) | Stacked bar, waffle chart | 3D pie chart |
| High-dimensional (>5 variables) | Heatmap (clustered), parallel coordinates | 3D scatter |
| Spatial data | Map, spatial heatmap | Bubble chart |
| Survival / time-to-event | Kaplan-Meier curve | Bar chart of median survival |
Color Theory for Science
Color encodes information. Misused color introduces artifacts and fails readers with color vision deficiency (CVD; ~8% of males).
Sequential palettes encode ordered numeric data from low to high (e.g., expression level, concentration). Use perceptually uniform palettes: viridis, magma, cividis. These also print in grayscale.
Diverging palettes encode data with a meaningful midpoint (e.g., fold-change centered at 0, correlation from -1 to +1). Use RdBu, coolwarm, or vlag. Always ensure the midpoint maps to white/neutral.
Qualitative palettes encode unordered categories. Use Okabe-Ito (CVD-safe), tab10 (matplotlib default), or ColorBrewer qualitative palettes. Limit to ≤8 distinguishable colors; use shape or pattern as redundant encoding beyond that.
Color don'ts:
- Rainbow/jet colormap: not perceptually uniform; creates false contours
- Red vs. green encoding: fails deuteranopia (~6% males)
- Saturated color for background or large areas
Figure Composition and Layout
Scientific figures are typically multi-panel. Panel layout and labeling affect how readers parse information.
- Panel labels: Bold uppercase letters (A, B, C) in the top-left corner; use 8–12 pt in the figure, larger in the caption reference.
- Alignment: Align panel edges on a grid. Unaligned panels signal lack of attention to detail.
- White space: Leave adequate margins; crowded panels reduce readability.
- Figure size: Design for the target column width — single column (~85 mm / 3.35 in), 1.5 column (~114 mm / 4.5 in), or double column (~170 mm / 6.7 in) for Nature-family journals.
- Font: Sans-serif (Arial, Helvetica) at 6–8 pt minimum in the final figure at publication resolution.
Journal Formatting Requirements
Major journals specify exact figure requirements for submission. Violating these causes desk-rejection delays.
| Journal/Style | Max Width | Resolution | Color Mode | Font | File Format |
|---|---|---|---|---|---|
| Nature family | 89 mm (1-col), 183 mm (2-col) | 300 dpi (photos), 600 dpi (line art) | RGB or CMYK | Arial 5–7 pt | PDF, TIFF, EPS |
| Cell/iScience | 85 mm (1-col), 170 mm (2-col) | 300 dpi raster, 600 dpi halftone | RGB | Helvetica 6–8 pt | PDF, EPS, TIFF |
| ACS journals | 3.25 in (1-col), 7 in (2-col) | 600 dpi (color), 1200 dpi (b&w line art) | RGB (screen), CMYK (print) | Arial/Helvetica 4.5–7 pt | TIFF, EPS, PDF |
| PLOS ONE | No strict width | 300 dpi (raster), 600–1200 dpi (line art) | RGB | Any | TIFF, EPS, PDF |
Decision Framework
Use this tree to select the right visualization for your analysis goal:
What is the primary message of this figure?
|
+-- Show a distribution or spread of values
| +-- One group --> Histogram or violin plot
| +-- 2-5 groups --> Violin + jitter (show all points if N < 100)
| +-- Many groups --> Ridge plot (joy plot)
|
+-- Compare quantities between categories
| +-- Few categories (2-5) --> Bar chart with error bars + individual points
| +-- Many categories (>8) --> Lollipop chart or dot plot (horizontal)
| +-- Paired measurements --> Slopegraph or paired dot plot
|
+-- Show a relationship between two continuous variables
| +-- N < 1000 --> Scatter plot
| +-- N > 1000 --> Hexbin or 2D density plot
| +-- Time ordered --> Line chart
|
+-- Show composition or part-to-whole
| +-- 2-4 parts --> Stacked bar or waffle chart
| +-- Over time --> Stacked area chart
| +-- Avoid pie chart unless <= 3 parts and proportions are obvious
|
+-- Show high-dimensional data
| +-- Genes x samples --> Clustered heatmap (seaborn.clustermap)
| +-- Embeddings (UMAP, PCA) --> Scatter colored by metadata
| +-- Feature importance --> Horizontal bar chart (sorted)
|
+-- Show spatial or geographic data
| +-- Microscopy --> Image overlay with colorbar
| +-- Geographic --> Choropleth map
| Analysis Goal | Chart Type | Library | Key Consideration |
|---|---|---|---|
| Gene expression across groups | Violin + jitter | seaborn, plotnine | Show all points if N < 50; never bar+SEM only |
| Differential expression | Volcano plot | matplotlib | Log2FC on x-axis, -log10(p) on y-axis |
| Clustering results | UMAP scatter | scanpy, matplotlib | One plot per annotation variable |
| Correlation matrix | Clustered heatmap | seaborn.clustermap | Use diverging palette centered at 0 |
| Protein structure | Ribbon diagram | PyMOL, ChimeraX | Not covered here — use dedicated molecular graphics tools |
| Survival analysis | Kaplan-Meier | lifelines | Include confidence bands and at-risk table |
| Time course | Line chart with CI | matplotlib | Show uncertainty; connect group means, not individual points |
Best Practices
-
Show the data, not just summaries: For N < 100, overlay individual data points on violin or box plots using jitter or beeswarm. Bar charts with only mean ± SEM conceal distribution shape, outliers, and bimodality.
-
Choose CVD-safe color palettes by default: Use Okabe-Ito or
viridis/cividisfor sequential data. Test your figure with a CVD simulator (e.g., Coblis) before submission. -
Design at final publication size from the start: Set your figure canvas to the exact column width of the target journal (e.g., 89 mm for Nature single-column). Rescaling after the fact makes fonts too small or too large, and changes aspect ratios.
-
Label axes with units and use descriptive titles: Every axis must have a label with units in parentheses (e.g., "Expression level (log2 CPM)"). Avoid cryptic abbreviations without legend entries.
-
Use vector formats for line art and text: Save figures as PDF or SVG when they contain text and lines. Rasterize only when submitting to a journal that requires TIFF. Vector figures scale without pixelation and remain editable.
-
Match statistical annotations to the test performed: If you annotate significance stars (*), state in the caption which test was used, the exact p-value, and the sample size. "n.s." should still report the p-value.
-
Avoid dual y-axes: Two different y-axes on one plot are almost always misleading — the apparent relationship depends on scale choices. Use two separate panels instead.
Common Pitfalls
-
Bar chart for continuous distributions (dynamite plot)
- What goes wrong: Mean ± SEM bars hide whether data is normally distributed, multimodal, or has outliers. Two datasets with identical bars can have completely different distributions.
- How to avoid: Use violin plots, box-and-whisker with jitter, or beeswarm plots. Show all data points when N < 50.
-
Truncated or broken y-axis that exaggerates differences
- What goes wrong: Starting the y-axis at a non-zero value makes small differences appear large and misleads readers about effect size.
- How to avoid: Start y-axis at zero for ratio/count data. If zooming in is genuinely needed, use an inset or a separate panel with a clear label indicating the truncation.
-
Using rainbow/jet colormap for heatmaps
- What goes wrong: Jet is not perceptually uniform — it creates false contour lines and fails under CVD and grayscale printing.
- How to avoid: Use
viridis,magma, orinfernofor sequential;RdBuorcoolwarmfor diverging data. These are the defaults in seaborn >= 0.12.
-
Overlapping data points without jitter or transparency
- What goes wrong: Points stack on top of each other (overplotting), making dense regions appear sparse and hiding the true data density.
- How to avoid: Add jitter (
seaborn.stripplot(jitter=True)), use transparency (alpha=0.3), or switch to a hexbin / 2D density plot for large N.
-
P-value annotations without effect size or sample size
- What goes wrong: A statistically significant result (p < 0.05) with N = 10,000 may be scientifically trivial. Stars alone are uninformative.
- How to avoid: Report exact p-values, effect sizes (Cohen's d, odds ratio), and sample sizes in the figure caption or directly on the plot.
-
Figure text too small at publication size
- What goes wrong: Designing at screen size and then shrinking to journal column width results in 3–4 pt text that fails minimum readability standards.
- How to avoid: Design at final print size from the start. Check that axis labels are ≥ 6 pt at the final dimensions.
-
Inconsistent style across panels in a multi-panel figure
- What goes wrong: Different fonts, line widths, or color schemes across panels make the figure look unprofessional and harder to compare.
- How to avoid: Define a shared
matplotlib.rcParamsstyle dictionary at the top of your figure script and apply it to all panels.
Workflow
-
Define the message first
- Write one sentence describing what you want the reader to take away from this figure.
- Choose the chart type that most directly communicates that message using the Decision Framework above.
-
Prepare the data
- Aggregate, tidy, and validate your data before plotting.
- Check for outliers that should be investigated, not silently dropped.
-
Prototype at screen resolution
- Build a draft figure in your preferred library (matplotlib, seaborn, ggplot2, plotnine).
- Focus on getting data and chart type right; ignore styling at this stage.
-
Apply journal-specific styling
- Set canvas size to target journal column width.
- Set font family and minimum size.
- Choose CVD-safe palette.
- Standardize line widths (0.5–1.0 pt for axes and ticks, 1.0–2.0 pt for data lines).
-
Add annotations and labels
- Panel labels (A, B, C) in bold.
- Axis labels with units.
- Statistical annotations with test name and exact p-value.
- Legend with descriptive entries.
-
Export at correct resolution
- Vector: PDF or SVG for line art and text.
- Raster: TIFF at 300 dpi (halftone/photos) or 600 dpi (line art).
- Use
plt.savefig("fig1.pdf", bbox_inches="tight")in matplotlib.
-
Accessibility check
- Simulate CVD using an online tool or GIMP color blindness filter.
- Print in grayscale and verify panels remain distinguishable.
Protocol Guidelines
- For matplotlib/seaborn: Set
rcParamsat the top of every figure script; usefig.set_size_inches()to enforce journal dimensions; export withdpi=300minimum. - For R/ggplot2: Use
theme_classic()or a custom theme; setggsave(width=..., units="mm", dpi=300). - For multi-panel assembly: Use
matplotlib.gridspec,patchworklib(Python), orcowplot/patchwork(R) for aligned panel grids. - For interactive figures: Plotly or Bokeh for HTML outputs; Vega-Lite for web embedding. Always provide a static fallback for publication.
- For poster presentations: Scale all fonts up by 1.5–2×; use larger markers (pt size 8–12) and thicker lines (2–3 pt); prefer high-contrast, dark background palettes.
Further Reading
- Fundamentals of Data Visualization — Claus O. Wilke (open access) — comprehensive guide covering chart selection, color, and figure design
- Ten Simple Rules for Better Figures (Rougier et al., 2014, PLOS Computational Biology) — practical peer-reviewed guidelines
- ColorBrewer 2.0 — browser tool for selecting print-safe, CVD-safe color palettes
- Coblis CVD Simulator — test figure accessibility under 8 types of color vision deficiency
- Nature Methods — Points of View column — monthly visualization advice from Bang Wong and colleagues
- ACS Publications Figure Preparation Guide — journal-specific technical requirements
Related Skills
matplotlib-figures— Python implementation of publication-quality figures with matplotlib and seaborndata-visualization— general Python plotting recipesbiostatistics— statistical test selection to accompany figure annotations
Seaborn — Statistical Plots
Overview
Seaborn is a Python library for statistical data visualization built on top of matplotlib. It works directly with pandas DataFrames, automatically handles grouping by categorical variables, computes confidence intervals and kernel density estimates, and produces attractive publication-ready figures with minimal configuration. Seaborn separates axes-level functions (embeddable in custom layouts) from figure-level functions (with built-in faceting), enabling both quick exploratory analysis and structured multi-panel figures.
When to Use
- Comparing gene expression, protein abundance, or measurement distributions across experimental conditions (treatment vs. control, cell lines, time points)
- Generating grouped box plots, violin plots, or strip plots to show both summary statistics and individual data points simultaneously
- Visualizing pairwise correlations in multi-gene or multi-feature datasets as annotated heatmaps
- Plotting regression fits with confidence bands between continuous variables (e.g., cell viability vs. drug concentration)
- Faceting a single plot type across multiple sample subsets, tissue types, or experimental batches in one call
- Rapid exploratory analysis of a new dataset using
pairplotto survey all pairwise relationships at once - Use
matplotlibdirectly when you need pixel-level control over figure elements, complex mixed-type layouts, or non-statistical custom plots - Use
plotlywhen the output must be interactive (hover tooltips, zoom, pan) or embedded in a web application
Prerequisites
- Python packages:
seaborn>=0.13,matplotlib,pandas,numpy - Data requirements: Pandas DataFrame in long-form (tidy) format; each observation is a row, each variable is a column
- Environment: Standard Python environment; no GPU or special hardware required
pip install "seaborn>=0.13" matplotlib pandas numpy scipy
Quick Start
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
# Simulate gene expression across conditions
rng = np.random.default_rng(42)
df = pd.DataFrame({
"gene": ["BRCA1"] * 60 + ["TP53"] * 60,
"condition": ["control", "treated"] * 60,
"log2_expr": np.concatenate([
rng.normal(5.2, 0.8, 60),
rng.normal(6.1, 0.9, 60),
])
})
sns.set_theme(style="ticks", context="notebook")
sns.boxplot(data=df, x="gene", y="log2_expr", hue="condition", palette="Set2")
plt.ylabel("log2 Expression")
plt.title("Gene Expression by Condition")
plt.tight_layout()
plt.savefig("quickstart_boxplot.png", dpi=150)
print("Saved quickstart_boxplot.png")
Core API
1. Distribution Plots
Visualize univariate distributions and compare them across groups. histplot bins data; kdeplot fits a smooth density estimate; displot is the figure-level wrapper that adds faceting.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(0)
n = 200
df = pd.DataFrame({
"log2_tpm": np.concatenate([rng.normal(4.5, 1.1, n), rng.normal(6.0, 1.3, n)]),
"sample": ["tumor"] * n + ["normal"] * n,
})
fig, axes = plt.subplots(1, 3, figsize=(15, 4))
# Histogram with density normalization and stacked hue groups
sns.histplot(data=df, x="log2_tpm", hue="sample", stat="density",
multiple="stack", bins=30, ax=axes[0])
axes[0].set_title("Histogram (stacked)")
# KDE with fill — bandwidth controlled by bw_adjust
sns.kdeplot(data=df, x="log2_tpm", hue="sample", fill=True,
bw_adjust=0.8, alpha=0.4, ax=axes[1])
axes[1].set_title("KDE (filled)")
# ECDF — useful for comparing cumulative distributions
sns.ecdfplot(data=df, x="log2_tpm", hue="sample", ax=axes[2])
axes[2].set_title("ECDF")
plt.tight_layout()
plt.savefig("distributions.png", dpi=150)
print("Saved distributions.png")
# Bivariate KDE: joint distribution of two continuous variables
rng = np.random.default_rng(1)
df2 = pd.DataFrame({
"log2_rna": rng.normal(5.5, 1.2, 300),
"log2_prot": rng.normal(4.8, 1.0, 300) + 0.6 * rng.normal(5.5, 1.2, 300),
})
sns.kdeplot(data=df2, x="log2_rna", y="log2_prot",
fill=True, levels=8, thresh=0.05, cmap="Blues")
plt.xlabel("log2 RNA (TPM)")
plt.ylabel("log2 Protein (iBAQ)")
plt.title("RNA–Protein Correlation Density")
plt.tight_layout()
plt.savefig("bivariate_kde.png", dpi=150)
print("Saved bivariate_kde.png")
2. Categorical Plots
Compare distributions or aggregated statistics across categorical groups. Axes-level functions (boxplot, violinplot, stripplot, swarmplot, barplot) accept an ax= parameter for embedding in custom layouts.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(2)
conditions = ["DMSO", "Drug A 1uM", "Drug A 10uM", "Drug B 1uM", "Drug B 10uM"]
df = pd.DataFrame({
"condition": np.repeat(conditions, 30),
"viability": np.concatenate([
rng.normal(100, 5, 30),
rng.normal(92, 7, 30),
rng.normal(65, 10, 30),
rng.normal(88, 8, 30),
rng.normal(45, 12, 30),
])
})
fig, axes = plt.subplots(1, 3, figsize=(18, 5))
# Box plot — shows quartiles and outliers
sns.boxplot(data=df, x="condition", y="viability",
palette="husl", width=0.5, ax=axes[0])
axes[0].set_xticklabels(axes[0].get_xticklabels(), rotation=30, ha="right")
axes[0].set_title("Box Plot")
# Violin — KDE shape + inner quartile lines
sns.violinplot(data=df, x="condition", y="viability",
inner="quart", palette="muted", ax=axes[1])
axes[1].set_xticklabels(axes[1].get_xticklabels(), rotation=30, ha="right")
axes[1].set_title("Violin Plot")
# Strip plot overlaid on box — shows all individual points
sns.boxplot(data=df, x="condition", y="viability",
palette="pastel", width=0.5, ax=axes[2])
sns.stripplot(data=df, x="condition", y="viability",
color="black", alpha=0.4, size=3, jitter=True, ax=axes[2])
axes[2].set_xticklabels(axes[2].get_xticklabels(), rotation=30, ha="right")
axes[2].set_title("Box + Strip")
plt.tight_layout()
plt.savefig("categorical.png", dpi=150)
print("Saved categorical.png")
# Bar plot with mean ± 95% CI and individual points (swarm)
fig, ax = plt.subplots(figsize=(8, 5))
sns.barplot(data=df, x="condition", y="viability",
estimator="mean", errorbar="ci", palette="Set3", ax=ax)
sns.swarmplot(data=df, x="condition", y="viability",
color="black", size=3, alpha=0.5, ax=ax)
ax.set_ylabel("Cell Viability (%)")
ax.set_xticklabels(ax.get_xticklabels(), rotation=30, ha="right")
plt.tight_layout()
plt.savefig("barswarm.png", dpi=150)
print("Saved barswarm.png")
3. Relational Plots
Visualize relationships between continuous variables. scatterplot and lineplot are axes-level; relplot is the figure-level wrapper that supports col and row faceting.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(3)
n = 150
df = pd.DataFrame({
"molecular_weight": rng.uniform(200, 800, n),
"logP": rng.uniform(-2, 6, n),
"pIC50": rng.normal(6.5, 1.2, n),
"target_class": rng.choice(["kinase", "GPCR", "protease"], n),
"pass_lipinski": rng.choice(["yes", "no"], n, p=[0.7, 0.3]),
})
# Scatter with hue (categorical color) + size (continuous) + style (marker)
sns.scatterplot(data=df, x="molecular_weight", y="pIC50",
hue="target_class", size="logP", style="pass_lipinski",
sizes=(30, 120), alpha=0.7)
plt.xlabel("Molecular Weight (Da)")
plt.ylabel("pIC50")
plt.title("Compound Bioactivity by Target Class")
plt.tight_layout()
plt.savefig("relational_scatter.png", dpi=150)
print("Saved relational_scatter.png")
# Line plot with automatic mean aggregation and SD error band across replicates
timepoints = [0, 1, 2, 4, 8, 24]
groups = ["untreated", "low_dose", "high_dose"]
rows = []
for grp, base in zip(groups, [100.0, 95.0, 80.0]):
for tp in timepoints:
for _ in range(5): # 5 replicates
rows.append({"timepoint_h": tp, "group": grp,
"confluency": base * np.exp(-0.02 * tp * (1 + rng.normal(0, 0.1)))})
time_df = pd.DataFrame(rows)
sns.lineplot(data=time_df, x="timepoint_h", y="confluency",
hue="group", style="group", errorbar="sd", markers=True, dashes=False)
plt.xlabel("Time (h)")
plt.ylabel("Confluency (%)")
plt.title("Cell Growth Inhibition (mean ± SD, n=5)")
plt.tight_layout()
plt.savefig("lineplot.png", dpi=150)
print("Saved lineplot.png")
4. Regression Plots
Fit linear (or polynomial/lowess) models and visualize them with confidence bands. regplot is axes-level; lmplot is figure-level with faceting support.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(4)
n = 120
tumor_size = rng.uniform(0.5, 6.0, n)
survival_months = 40 - 5 * tumor_size + rng.normal(0, 4, n)
grade = rng.choice(["low", "high"], n, p=[0.5, 0.5])
df = pd.DataFrame({"tumor_size_cm": tumor_size,
"survival_months": survival_months,
"grade": grade})
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
# Linear regression with 95% CI band
sns.regplot(data=df, x="tumor_size_cm", y="survival_months",
ci=95, scatter_kws={"alpha": 0.4, "s": 25}, ax=axes[0])
axes[0].set_title("Linear Regression (95% CI)")
# Residuals plot — check for homoscedasticity
sns.residplot(data=df, x="tumor_size_cm", y="survival_months",
scatter_kws={"alpha": 0.4, "s": 25}, ax=axes[1])
axes[1].axhline(0, color="red", linestyle="--", linewidth=1)
axes[1].set_title("Residuals vs Fitted")
plt.tight_layout()
plt.savefig("regression.png", dpi=150)
print("Saved regression.png")
# lmplot — figure-level: separate regression lines per grade (hue) + facets
g = sns.lmplot(data=df, x="tumor_size_cm", y="survival_months",
hue="grade", col="grade", ci=95,
scatter_kws={"alpha": 0.4}, height=4, aspect=1.1)
g.set_axis_labels("Tumor Size (cm)", "Survival (months)")
g.set_titles("{col_name} grade")
g.savefig("lmplot_faceted.png", dpi=150)
print("Saved lmplot_faceted.png")
5. Matrix Plots
Visualize rectangular data as color-encoded matrices. heatmap is axes-level; clustermap is figure-level and applies hierarchical clustering to rows and columns.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(5)
genes = [f"GENE{i}" for i in range(1, 9)]
samples = [f"S{i}" for i in range(1, 7)]
# Simulate log2 fold-change matrix (rows=genes, cols=samples)
lfc = pd.DataFrame(
rng.normal(0, 1.5, size=(8, 6)),
index=genes, columns=samples
)
# Inject a pattern: first 3 genes up in samples 1-3, down in 4-6
lfc.iloc[:3, :3] += 2.5
lfc.iloc[:3, 3:] -= 2.5
# Correlation heatmap of numeric features
df_num = pd.DataFrame(
rng.standard_normal((80, 5)),
columns=["GeneA", "GeneB", "GeneC", "GeneD", "GeneE"]
)
df_num["GeneB"] = df_num["GeneA"] * 0.85 + rng.normal(0, 0.3, 80)
corr = df_num.corr()
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
sns.heatmap(corr, annot=True, fmt=".2f", cmap="coolwarm",
center=0, square=True, linewidths=0.5, ax=axes[0])
axes[0].set_title("Pearson Correlation Heatmap")
sns.heatmap(lfc, cmap="RdBu_r", center=0, annot=True, fmt=".1f",
linewidths=0.3, cbar_kws={"label": "log2FC"}, ax=axes[1])
axes[1].set_title("log2 Fold Change Matrix")
plt.tight_layout()
plt.savefig("heatmaps.png", dpi=150)
print("Saved heatmaps.png")
# Clustermap with hierarchical clustering and row/column color annotations
rng = np.random.default_rng(6)
n_genes, n_samples = 30, 16
expr = pd.DataFrame(
rng.lognormal(mean=2.0, sigma=1.2, size=(n_genes, n_samples)),
index=[f"GENE{i:03d}" for i in range(n_genes)],
columns=[f"{'T' if i < 8 else 'N'}{i:02d}" for i in range(n_samples)]
)
# Column annotation colors (tumor vs normal)
col_colors = ["#D32F2F" if c.startswith("T") else "#1976D2" for c in expr.columns]
g = sns.clustermap(
np.log2(expr + 1),
cmap="viridis",
standard_scale=0, # z-score across rows (genes)
method="ward",
metric="euclidean",
col_colors=col_colors,
figsize=(12, 10),
linewidths=0,
cbar_pos=(0.02, 0.8, 0.03, 0.15),
cbar_kws={"label": "Row z-score"},
)
g.ax_heatmap.set_xlabel("Sample")
g.ax_heatmap.set_ylabel("Gene")
plt.savefig("clustermap.png", dpi=150, bbox_inches="tight")
print("Saved clustermap.png")
6. Multi-Variable Grids
Survey all pairwise relationships with pairplot or display a bivariate distribution with marginals using jointplot. For fully custom grid layouts, use FacetGrid directly.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(7)
n = 60
df = pd.DataFrame({
"cell_area": rng.normal(350, 60, n * 3),
"nucleus_area": rng.normal(90, 15, n * 3),
"mean_intensity": rng.exponential(500, n * 3),
"aspect_ratio": np.abs(rng.normal(1.3, 0.3, n * 3)),
"cell_type": (["HeLa"] * n + ["MCF7"] * n + ["A549"] * n),
})
# Pairplot — matrix of pairwise scatter + KDE on diagonal
g = sns.pairplot(df, hue="cell_type", corner=True,
diag_kind="kde", plot_kws={"alpha": 0.5, "s": 20})
g.savefig("pairplot.png", dpi=150)
print("Saved pairplot.png")
# Jointplot — bivariate KDE with marginal histograms
g = sns.jointplot(data=df, x="cell_area", y="nucleus_area",
hue="cell_type", kind="scatter",
marginal_kws={"fill": True, "alpha": 0.3})
g.set_axis_labels("Cell Area (µm²)", "Nucleus Area (µm²)")
g.savefig("jointplot.png", dpi=150)
print("Saved jointplot.png")
# FacetGrid — custom layout: KDE of mean_intensity per cell type
g = sns.FacetGrid(df, col="cell_type", height=3.5, aspect=1.1,
sharey=False)
g.map(sns.histplot, "mean_intensity", bins=20, kde=True, color="steelblue")
g.set_axis_labels("Mean Intensity (AU)", "Count")
g.set_titles("{col_name}")
g.tight_layout()
g.savefig("facetgrid_intensity.png", dpi=150)
print("Saved facetgrid_intensity.png")
Key Concepts
Figure-Level vs Axes-Level Functions
Seaborn has two tiers of functions with different return types and composability:
| Feature | Axes-Level | Figure-Level |
|---|---|---|
| Examples | scatterplot, histplot, boxplot, heatmap, regplot | relplot, displot, catplot, lmplot |
| Returns | matplotlib.axes.Axes | FacetGrid / JointGrid / PairGrid |
| Faceting | Manual (create subplots yourself) | Built-in (col=, row= params) |
| Sizing | figsize= on parent figure | height= + aspect= per facet panel |
| Placement | ax= parameter | Cannot be placed in an existing axes |
| Saving | plt.savefig(...) | g.savefig(...) |
| Use when | Combining different plot types in one figure | Quick multi-panel exploratory views |
# Axes-level: place in a pre-allocated subplot grid
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
sns.violinplot(data=df, x="cell_type", y="cell_area", ax=axes[0])
sns.scatterplot(data=df, x="cell_area", y="nucleus_area", hue="cell_type", ax=axes[1])
Long-Form vs Wide-Form Data
Seaborn semantic mappings (hue, size, style) require long-form (tidy) data where each variable is a column and each observation is a row. Some functions (heatmap, clustermap, lineplot) also accept wide-form.
# Wide-form: unsuitable for hue/style mappings
# sample_A sample_B sample_C
# 0 5.1 6.2 4.8
# Long-form (preferred): melt wide → long
wide = pd.DataFrame({"sampleA": [5.1, 4.3], "sampleB": [6.2, 5.9]})
long = wide.melt(var_name="sample", value_name="log2_expr")
# → columns: sample, log2_expr
Common Workflows
Workflow 1: Differential Expression Scatter with Significance Thresholds
Goal: Visualize log2 fold-change vs -log10 p-value (volcano-style) with significance annotations, colored by regulation status, and labeled top hits.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(42)
n = 500
lfc = rng.normal(0, 1.5, n)
pvals = 10 ** (-rng.exponential(1.5, n)) # skewed toward low significance
pvals = np.clip(pvals, 1e-20, 1.0)
genes = [f"GENE{i:04d}" for i in range(n)]
df_de = pd.DataFrame({"gene": genes, "log2fc": lfc, "pvalue": pvals})
df_de["neg_log10_p"] = -np.log10(df_de["pvalue"])
# Classify regulation status
lfc_thresh = 1.0
padj_thresh = 0.05
df_de["sig"] = "NS"
df_de.loc[(df_de["log2fc"] > lfc_thresh) & (df_de["pvalue"] < padj_thresh), "sig"] = "Up"
df_de.loc[(df_de["log2fc"] < -lfc_thresh) & (df_de["pvalue"] < padj_thresh), "sig"] = "Down"
palette = {"NS": "#AAAAAA", "Up": "#D32F2F", "Down": "#1976D2"}
sns.set_theme(style="ticks", context="paper", font_scale=1.1)
fig, ax = plt.subplots(figsize=(8, 6))
sns.scatterplot(data=df_de, x="log2fc", y="neg_log10_p",
hue="sig", palette=palette,
alpha=0.6, s=18, linewidth=0, ax=ax)
# Threshold lines
ax.axhline(-np.log10(padj_thresh), color="black", linestyle="--", linewidth=0.8)
ax.axvline( lfc_thresh, color="black", linestyle="--", linewidth=0.8)
ax.axvline(-lfc_thresh, color="black", linestyle="--", linewidth=0.8)
# Label top 5 most significant genes per direction
for direction in ["Up", "Down"]:
top = df_de[df_de["sig"] == direction].nlargest(5, "neg_log10_p")
for _, row in top.iterrows():
ax.text(row["log2fc"], row["neg_log10_p"] + 0.3, row["gene"],
fontsize=6, ha="center", va="bottom",
color=palette[direction])
# Annotation counts
n_up = (df_de["sig"] == "Up").sum()
n_down = (df_de["sig"] == "Down").sum()
ax.set_title(f"Volcano Plot | Up: {n_up} Down: {n_down}")
ax.set_xlabel("log2 Fold Change")
ax.set_ylabel("-log10 p-value")
sns.despine(trim=True)
plt.tight_layout()
plt.savefig("volcano_plot.png", dpi=300, bbox_inches="tight")
print(f"Volcano: {n_up} up, {n_down} down — saved volcano_plot.png")
Workflow 2: Multi-Condition Comparison with Grouped Violin + Strip Plots
Goal: Compare gene expression (or any continuous measurement) across multiple treatments and time points, showing full distributions plus individual replicates.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(99)
genes = ["BRCA1", "TP53", "EGFR"]
treats = ["DMSO", "Drug A", "Drug B"]
timepoints = ["6h", "24h", "48h"]
rows = []
for gene in genes:
base_expr = {"BRCA1": 7.5, "TP53": 6.2, "EGFR": 8.1}[gene]
for treat in treats:
treat_shift = {"DMSO": 0.0, "Drug A": -0.8, "Drug B": 0.6}[treat]
for tp in timepoints:
tp_shift = {"6h": 0.0, "24h": 0.3, "48h": 0.6}[tp]
for _ in range(12):
rows.append({
"gene": gene,
"treatment": treat,
"timepoint": tp,
"log2_expr": base_expr + treat_shift + tp_shift + rng.normal(0, 0.5),
})
df_mc = pd.DataFrame(rows)
sns.set_theme(style="whitegrid", context="paper", font_scale=1.0)
g = sns.catplot(
data=df_mc,
x="timepoint", y="log2_expr",
hue="treatment",
col="gene",
kind="violin",
inner="quart",
dodge=True,
palette="Set2",
height=4, aspect=0.9,
col_order=genes,
order=timepoints,
)
# Overlay individual points
for ax in g.axes.flat:
gene_label = ax.get_title()
gene_name = gene_label.split(" = ")[-1] if " = " in gene_label else gene_label
subset = df_mc[df_mc["gene"] == gene_name]
sns.stripplot(
data=subset,
x="timepoint", y="log2_expr",
hue="treatment",
dodge=True,
jitter=True,
size=2.5,
alpha=0.4,
palette="dark:black",
order=timepoints,
legend=False,
ax=ax,
)
g.set_axis_labels("Timepoint", "log2 Expression")
g.set_titles("{col_name}")
g.add_legend(title="Treatment")
sns.despine(trim=True)
g.tight_layout()
g.savefig("multigroup_violin.png", dpi=300, bbox_inches="tight")
print("Saved multigroup_violin.png")
Workflow 3: Pairwise Feature Exploration for Cell Morphology
Goal: Quickly survey pairwise relationships in a multi-feature cell morphology dataset using pairplot, then examine one key pair with a jointplot.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
rng = np.random.default_rng(12)
n_per_type = 80
df_morph = pd.DataFrame({
"cell_area_um2": np.concatenate([rng.normal(320, 50, n_per_type),
rng.normal(420, 70, n_per_type),
rng.normal(280, 40, n_per_type)]),
"nucleus_area_um2": np.concatenate([rng.normal(85, 12, n_per_type),
rng.normal(110, 18, n_per_type),
rng.normal(75, 10, n_per_type)]),
"eccentricity": np.abs(np.concatenate([rng.normal(0.6, 0.12, n_per_type),
rng.normal(0.8, 0.10, n_per_type),
rng.normal(0.5, 0.09, n_per_type)])),
"mean_dapi": np.concatenate([rng.exponential(400, n_per_type),
rng.exponential(600, n_per_type),
rng.exponential(350, n_per_type)]),
"cell_line": ["HeLa"] * n_per_type + ["MCF7"] * n_per_type + ["U2OS"] * n_per_type,
})
# 1. Pairplot survey
g = sns.pairplot(df_morph, hue="cell_line", corner=True,
diag_kind="kde", plot_kws={"alpha": 0.5, "s": 15},
palette="Dark2")
g.savefig("morphology_pairplot.png", dpi=150)
print("Saved morphology_pairplot.png")
# 2. Focused jointplot for the most informative pair
g2 = sns.jointplot(data=df_morph, x="cell_area_um2", y="nucleus_area_um2",
hue="cell_line", kind="scatter",
marginal_kws={"fill": True, "alpha": 0.25},
palette="Dark2", alpha=0.6)
g2.set_axis_labels("Cell Area (µm²)", "Nucleus Area (µm²)")
g2.savefig("morphology_jointplot.png", dpi=150)
print("Saved morphology_jointplot.png")
Key Parameters
| Parameter | Function(s) | Default | Range / Options | Effect |
|---|---|---|---|---|
hue | All plot functions | None | Column name (categorical or continuous) | Color-encodes a variable; triggers automatic legend |
style | scatterplot, lineplot | None | Categorical column name | Encodes variable with marker shape or line dash pattern |
size | scatterplot, lineplot | None | Categorical or continuous column | Encodes variable via point or line size |
col / row | Figure-level only (relplot, displot, catplot, lmplot) | None | Categorical column name | Creates one subplot panel per unique value |
col_wrap | Figure-level only | None | int | Wraps columns onto a new row after N panels |
estimator | barplot, pointplot | "mean" | "mean", "median", any callable | Aggregation function applied within each category |
errorbar | barplot, lineplot, pointplot | ("ci", 95) | "ci", "sd", "se", "pi", None | Error bar type displayed around the estimate |
stat | histplot | "count" | "count", "frequency", "density", "probability" | Normalization applied to histogram bar heights |
bw_adjust | kdeplot, violinplot | 1.0 | 0.1–3.0 | KDE bandwidth multiplier; lower=spikier, higher=smoother |
multiple | histplot, kdeplot | "layer" | "layer", "stack", "dodge", "fill" | How overlapping hue groups are drawn |
inner | violinplot | "box" | "box", "quart", "point", "stick", None | Interior annotation inside the violin body |
standard_scale | clustermap | None | 0 (rows), 1 (columns) | Z-score normalization axis before clustering |
dodge | boxplot, violinplot, stripplot | Varies | True, False | Separate hue-grouped elements along the axis |
context | set_theme() | "notebook" | "paper", "notebook", "talk", "poster" | Scales font and line widths for output medium |
Best Practices
-
Prefer long-form DataFrames with named columns: Seaborn's semantic mapping (
hue,style,size) reads variable names directly from column names. Passing raw arrays loses axis labels and legends. Usepd.melt()to convert wide-form data. -
Call
set_theme()once at the top of a script: This sets the global style, context, and palette for all subsequent plots, ensuring consistency. Reset to defaults withsns.set_theme().sns.set_theme(style="ticks", context="paper", font_scale=1.1, rc={"axes.spines.right": False, "axes.spines.top": False}) -
Use axes-level functions for mixed-type custom layouts: Figure-level functions (
relplot,catplot) create their own figure and cannot be placed in an existingAxes. When combining different plot types (e.g., scatter + violin + heatmap), allocate aplt.subplots()grid and use axes-level functions withax=. -
Use colorblind-safe palettes:
sns.set_palette("colorblind")orpalette="colorblind"produces a palette distinguishable by readers with common color vision deficiencies. For diverging data, use"RdBu_r"or"coolwarm"withcenter=0. -
Overlay individual data points on summary plots: Violin and bar plots hide distribution shape and sample size. Overlaying a
stripplotorswarmplotwithalpha=0.4and smallsizeconveys data density without obscuring the summary statistic. -
Size figure-level plots with
heightandaspect, notfigsize: Figure-level functions ignorefigsize. Useheight=(inches per panel) andaspect=(width-to-height ratio per panel). For axes-level, setfigsizeon theplt.subplots()call. -
Anti-pattern — calling
plt.savefig()on a figure-level grid: Figure-level functions return aFacetGrid/JointGridobject. Save it withg.savefig("out.png", dpi=300, bbox_inches="tight"), notplt.savefig(), which may capture a blank figure.
Common Recipes
Recipe: Publication-Ready Figure with Custom Palette and 300 DPI Export
When to use: Preparing a multi-panel figure for journal submission or a slide deck.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
sns.set_theme(style="ticks", context="paper", font_scale=1.2,
rc={"pdf.fonttype": 42, "ps.fonttype": 42})
rng = np.random.default_rng(7)
df = pd.DataFrame({
"condition": np.repeat(["Control", "Treated"], 40),
"ki67_pct": np.concatenate([rng.normal(18, 4, 40), rng.normal(32, 6, 40)]),
"apoptosis": np.concatenate([rng.normal(5, 1.5, 40), rng.normal(12, 2.5, 40)]),
})
custom_palette = {"Control": "#4575B4", "Treated": "#D73027"}
fig, axes = plt.subplots(1, 2, figsize=(8, 4))
# Panel A
sns.boxplot(data=df, x="condition", y="ki67_pct",
palette=custom_palette, width=0.45, linewidth=1.2, ax=axes[0])
sns.stripplot(data=df, x="condition", y="ki67_pct",
color="black", alpha=0.35, size=3, jitter=True, ax=axes[0])
axes[0].set_ylabel("Ki67 Positive Cells (%)")
axes[0].set_xlabel("")
axes[0].set_title("A", loc="left", fontweight="bold")
# Panel B
sns.boxplot(data=df, x="condition", y="apoptosis",
palette=custom_palette, width=0.45, linewidth=1.2, ax=axes[1])
sns.stripplot(data=df, x="condition", y="apoptosis",
color="black", alpha=0.35, size=3, jitter=True, ax=axes[1])
axes[1].set_ylabel("Apoptotic Cells (%)")
axes[1].set_xlabel("")
axes[1].set_title("B", loc="left", fontweight="bold")
sns.despine(trim=True)
plt.tight_layout()
plt.savefig("figure1.pdf", dpi=300, bbox_inches="tight")
plt.savefig("figure1.png", dpi=300, bbox_inches="tight")
print("Saved figure1.pdf and figure1.png at 300 DPI")
Recipe: Clustered Heatmap with Row and Column Color Annotations
When to use: Displaying a gene expression matrix with sample group annotations and hierarchical clustering to reveal co-expression modules.
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import numpy as np
import pandas as pd
rng = np.random.default_rng(21)
n_genes, n_samples = 40, 20
conditions = ["tumor"] * 10 + ["normal"] * 10
# Simulate expression: 3 co-expression modules
expr = pd.DataFrame(
rng.lognormal(2.5, 0.8, (n_genes, n_samples)),
index=[f"GENE{i:03d}" for i in range(n_genes)],
columns=[f"{c[0].upper()}{i:02d}" for i, c in enumerate(conditions)],
)
# Module 1: genes 0-13 up in tumor
expr.iloc[:14, :10] *= 3.0
# Module 2: genes 14-27 down in tumor
expr.iloc[14:28, :10] *= 0.3
# Module 3: genes 28-39 unchanged
log_expr = np.log2(expr + 1)
# Column colors: tumor=red, normal=blue
cond_pal = {"tumor": "#C62828", "normal": "#1565C0"}
col_colors = [cond_pal[c] for c in conditions]
# Row colors: module membership
module_pal = {"up": "#EF9A9A", "down": "#90CAF9", "stable": "#C8E6C9"}
row_modules = (["up"] * 14) + (["down"] * 14) + (["stable"] * 12)
row_colors = [module_pal[m] for m in row_modules]
g = sns.clustermap(
log_expr,
cmap="RdYlBu_r",
center=log_expr.values.mean(),
standard_scale=0, # z-score per gene (row)
method="ward",
metric="euclidean",
col_colors=col_colors,
row_colors=row_colors,
figsize=(14, 12),
linewidths=0,
cbar_pos=(0.02, 0.85, 0.03, 0.12),
cbar_kws={"label": "Row z-score"},
dendrogram_ratio=(0.12, 0.08),
)
g.ax_heatmap.set_xlabel("Sample", fontsize=10)
g.ax_heatmap.set_ylabel("Gene", fontsize=10)
g.ax_heatmap.set_title("Gene Expression Clustermap", fontsize=12, pad=80)
# Manual legend for column/row annotations
legend_handles = [
mpatches.Patch(color="#C62828", label="Tumor"),
mpatches.Patch(color="#1565C0", label="Normal"),
mpatches.Patch(color="#EF9A9A", label="Up in tumor"),
mpatches.Patch(color="#90CAF9", label="Down in tumor"),
mpatches.Patch(color="#C8E6C9", label="Stable"),
]
g.ax_heatmap.legend(handles=legend_handles, bbox_to_anchor=(1.25, 1.05),
loc="upper left", frameon=False, fontsize=9)
plt.savefig("clustermap_annotated.png", dpi=300, bbox_inches="tight")
print("Saved clustermap_annotated.png")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| Legend placed outside plot, clipped in saved file | Figure-level functions place the legend outside by default | Add bbox_inches="tight" to savefig(): g.savefig("out.png", dpi=300, bbox_inches="tight") |
TypeError: FacetGrid.savefig() or blank figure saved | Called plt.savefig() on a figure-level grid that owns its own figure | Use g.savefig(...) instead of plt.savefig(...) |
| Overlapping x-axis category labels | Long label strings overlap at default rotation | Add plt.xticks(rotation=45, ha="right") and plt.tight_layout() after the plot call |
ValueError: Could not interpret value ... for parameter 'hue' | Data is in wide-form; hue mapping requires long-form | Convert with df.melt(id_vars=[...], var_name="sample", value_name="expr") |
| KDE bandwidth too smooth (loses bimodality) | Default bw_adjust=1.0 over-smooths small datasets | Lower to bw_adjust=0.5; confirm peaks with histplot |
clustermap ignores figsize | Figure-level functions do not accept figsize as a kwarg in older seaborn | Pass figsize as a direct argument: sns.clustermap(..., figsize=(12, 10)) |
| Violin plot is a thin line (no shape) | Too few observations for KDE estimation | Switch to kind="box" or kind="strip"; or use cut=0 to restrict KDE to data range |
| Colors not distinguishable for many groups | Default palette repeats with >6 categories | Use sns.color_palette("husl", n_colors=N) or "tab20" for up to 20 distinct colors |
Figure-level function ignores ax= parameter | Axes-level distinction: figure-level functions create their own figure | Use the corresponding axes-level function (scatterplot, histplot, etc.) with ax= |
Related Skills
- matplotlib-scientific-plotting — low-level figure building, custom annotations, non-statistical plot types, and multi-panel layouts that mix seaborn with raw matplotlib
- plotly-interactive-plots — interactive charts with hover, zoom, and HTML/Dash export
- pydeseq2-differential-expression — produces the log2FC and p-values that feed into volcano-style scatter plots
- scikit-image-processing — generates cell morphology measurements visualized with seaborn categorical/distribution plots
- scientific-visualization — decision guide for selecting the right chart type and color scheme before coding
References
- Seaborn official documentation — API reference, tutorial, and gallery
- Seaborn example gallery — visual index of all plot types
- Seaborn GitHub — source code and issue tracker
- Waskom ML (2021). "seaborn: statistical data visualization." Journal of Open Source Software, 6(60), 3021. https://doi.org/10.21105/joss.03021
Statistical Significance Annotation on Plots
Overview
Statistical significance annotations (asterisk notation) are visual markers placed on comparison plots to indicate the results of hypothesis tests between groups. They consist of brackets connecting two groups and asterisk symbols denoting the p-value range. Proper annotation ensures that the visual claims in a figure match the quantitative evidence, making plots publication-ready and scientifically rigorous. This guide covers the standard conventions, when and how to annotate, and a reusable matplotlib implementation.
Key Concepts
Standard Asterisk Notation
The widely adopted convention maps p-value ranges to asterisk symbols:
| Symbol | P-value Range | Meaning |
|---|---|---|
| ns | p > 0.05 | Not significant |
| * | p <= 0.05 | Significant |
| ** | p <= 0.01 | Highly significant |
| *** | p <= 0.001 | Very highly significant |
| **** | p <= 0.0001 | Extremely significant |
The conversion function:
def pvalue_to_asterisk(p: float) -> str:
"""Convert a p-value to standard asterisk notation."""
if p <= 0.0001:
return "****"
elif p <= 0.001:
return "***"
elif p <= 0.01:
return "**"
elif p <= 0.05:
return "*"
else:
return "ns"
Adjusted vs Raw P-values
- Single comparison (one t-test): Use raw p-value.
- Multiple comparisons (pairwise tests across 3+ groups, multiple genes): Use adjusted p-values (FDR/Benjamini-Hochberg or Bonferroni). Annotating with raw p-values inflates significance.
- Pre-computed results (DESeq2
padj, ANOVA post-hoc): Use the adjusted values already provided.
Comparison Selection
Not every pair of groups needs annotation. Select comparisons that:
- Directly support the claim made in the analysis text
- Are biologically meaningful (e.g., treatment vs control, not control-A vs control-B)
- Are limited in number to keep the figure readable (typically 1-5 per panel)
Decision Framework
Does the plot compare groups?
├── No (scatter, heatmap, PCA, line trend) → Do NOT annotate
└── Yes (box, violin, bar, strip)
├── Does the analysis claim significance? → Annotate the claimed comparisons
├── Exploratory (no specific claim) → Annotate vs control only, or skip
└── Too many groups (>6 pairwise) → Annotate key comparisons only
| Scenario | Annotate? | Which pairs |
|---|---|---|
| DEG box plot: treatment vs control | Yes | Treatment vs Control |
| Multi-group ANOVA with post-hoc | Yes | Significant post-hoc pairs only |
| Gene expression across 10 cell types | Selectively | vs reference cell type only |
| PCA or UMAP | No | N/A |
| Heatmap or volcano plot | No | N/A |
| Correlation scatter | No | Report r and p in text/legend |
| Exploratory bar plot, no hypothesis | Optional | vs control if applicable |
Best Practices
- Match annotations to text claims: Every asterisk on the plot must correspond to a statistical test described in the analysis. Never annotate without having computed the test.
- Use adjusted p-values for multiple comparisons: When testing more than one pair, always use FDR-corrected or Bonferroni-corrected p-values. State the correction method in the figure legend.
- Limit annotated pairs: Annotate only comparisons relevant to the analysis conclusion. Over-annotating clutters the figure and dilutes focus.
- Position brackets clearly: Place brackets above the data range with enough vertical offset to avoid overlapping with data points, error bars, or other brackets. Stack multiple brackets with consistent spacing.
- State the statistical test: Always note the test used (t-test, Mann-Whitney U, Wilcoxon, ANOVA + Tukey HSD, etc.) in the figure title, caption, or legend.
- Include sample sizes: Show n per group in the axis labels (e.g., "Control (n=30)") or figure legend.
- Use bold titles: Set
fontweight='bold'on figure titles for publication readiness.
Common Pitfalls
-
Annotating all pairwise comparisons in a multi-group plot
- How to avoid: Select only hypothesis-driven pairs. For k groups, k*(k-1)/2 pairs quickly becomes unreadable. Show vs control or specific contrasts only.
-
Using raw p-values when multiple comparisons were performed
- How to avoid: Apply
statsmodels.stats.multitest.multipletests(pvals, method='fdr_bh')or use adjusted p-values from upstream tools (DESeq2padj).
- How to avoid: Apply
-
Bracket overlap with data or other brackets
- How to avoid: Use incremental vertical offset for stacked brackets. Start the first bracket above the maximum data value + error bar, then add a fixed offset for each additional bracket.
-
Asterisks without stating which test was used
- How to avoid: Always include the test name in the plot title or annotation (e.g., "Mann-Whitney U test" or "Tukey HSD post-hoc").
-
Inconsistent notation across figures
- How to avoid: Use the same
pvalue_to_asterisk()function throughout the analysis. Define it once and reuse.
- How to avoid: Use the same
-
Annotating "ns" on every non-significant pair
- How to avoid: Only show "ns" when the non-significance itself is a notable finding (e.g., showing no difference between two treatments). Omit ns annotations for pairs not being compared.
-
Placing annotations below the data
- How to avoid: Always place brackets and asterisks above the compared groups, never below.
Workflow
Step 1: Compute Statistical Tests
Run the appropriate test and collect p-values before plotting:
from scipy import stats
# Two-group comparison
stat, pval = stats.mannwhitneyu(group_a, group_b, alternative='two-sided')
# or for normal data:
stat, pval = stats.ttest_ind(group_a, group_b)
# Multi-group: ANOVA + post-hoc
from scipy.stats import f_oneway
stat, pval_anova = f_oneway(group_a, group_b, group_c)
# Post-hoc pairwise (if ANOVA significant)
from itertools import combinations
from statsmodels.stats.multitest import multipletests
pairs = list(combinations(["A", "B", "C"], 2))
groups = {"A": group_a, "B": group_b, "C": group_c}
raw_pvals = []
for g1, g2 in pairs:
_, p = stats.mannwhitneyu(groups[g1], groups[g2], alternative='two-sided')
raw_pvals.append(p)
# Adjust for multiple comparisons
rejected, adj_pvals, _, _ = multipletests(raw_pvals, method='fdr_bh')
Step 2: Add Bracket Annotations to the Plot
Use this helper function to draw brackets with asterisks on any matplotlib axes:
def add_significance_bracket(ax, x1, x2, y, p_value, dh=0.02, barh=0.015, fontsize=11):
"""Draw a significance bracket with asterisk notation between two x positions.
Args:
ax: matplotlib Axes object.
x1, x2: x-axis positions of the two groups (0-indexed).
y: y-coordinate for the bracket (top of bracket line).
p_value: p-value for the comparison.
dh: vertical offset above bracket for the text (in axes fraction).
barh: height of the bracket tips (in axes fraction).
fontsize: font size for the asterisk text.
"""
asterisk = pvalue_to_asterisk(p_value)
# Draw bracket: two tips and a connecting line
ax.plot([x1, x1, x2, x2], [y - barh, y, y, y - barh],
lw=1.2, color='black')
# Place asterisk text centered above the bracket
ax.text((x1 + x2) / 2, y + dh, asterisk,
ha='center', va='bottom', fontsize=fontsize, fontweight='bold')
Step 3: Integrate with Seaborn Plots
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
# Example: box plot with significance annotation
fig, ax = plt.subplots(figsize=(6, 5))
sns.boxplot(data=df, x="group", y="value", ax=ax, palette="Set2")
sns.stripplot(data=df, x="group", y="value", ax=ax,
color="black", alpha=0.4, size=3, jitter=True)
# Determine bracket y-position from data
y_max = df["value"].max()
y_range = df["value"].max() - df["value"].min()
offset = y_range * 0.08 # spacing between brackets
# Add brackets for each significant comparison
# pairs_with_pvals: list of (group1_idx, group2_idx, p_value)
pairs_with_pvals = [(0, 1, 0.003), (0, 2, 0.042)]
for i, (x1, x2, pval) in enumerate(pairs_with_pvals):
bracket_y = y_max + offset * (i + 1)
add_significance_bracket(ax, x1, x2, bracket_y, pval)
ax.set_title("Gene Expression by Treatment", fontweight='bold', fontsize=14)
ax.set_ylabel("Expression (log2 CPM)")
# Extend y-axis to fit brackets
ax.set_ylim(top=y_max + offset * (len(pairs_with_pvals) + 1.5))
plt.tight_layout()
plt.savefig("expression_comparison.png", dpi=150, bbox_inches='tight')
Step 4: Annotating Grouped Bar Plots
For bar plots with error bars, position brackets above the error bars:
fig, ax = plt.subplots(figsize=(7, 5))
bar_plot = sns.barplot(data=df, x="gene", y="fold_change", hue="condition",
ax=ax, palette="Set2", ci="sd", capsize=0.05)
# For grouped bars, calculate x positions manually
# Each gene has multiple bars offset by group
n_groups = df["condition"].nunique()
n_genes = df["gene"].nunique()
bar_width = 0.8 / n_groups
for gene_idx in range(n_genes):
# x positions of the two bars within this gene group
x1 = gene_idx - bar_width / 2
x2 = gene_idx + bar_width / 2
# Get the max value + error for this gene
gene_data = df[df["gene"] == df["gene"].unique()[gene_idx]]
y_top = gene_data["fold_change"].mean() + gene_data["fold_change"].std()
p_val = pvals_per_gene[gene_idx] # pre-computed
if p_val <= 0.05: # only annotate significant results
add_significance_bracket(ax, x1, x2, y_top + 0.1, p_val)
ax.set_title("Fold Change by Condition", fontweight='bold', fontsize=14)
plt.tight_layout()
plt.savefig("fold_change_comparison.png", dpi=150, bbox_inches='tight')
Protocol Guidelines
- Always compute tests before plotting: The statistical test should be run and results stored before any plotting code. Do not compute p-values inside the plotting block.
- Use consistent style: Use the same
add_significance_bracketfunction andpvalue_to_asteriskconversion across all figures in an analysis. - Report test details in solution text: When presenting the figure, state: the test used, number of samples per group, and whether p-values are adjusted.
- Adjust y-axis limits: After adding brackets, extend the y-axis upper limit to prevent clipping. Use
ax.set_ylim(top=...)orax.margins(y=0.15). - For DESeq2/edgeR results: Use
padj(adjusted p-value) directly. Do not re-test the raw counts.
Further Reading
- Graphpad: How to report statistical significance — Asterisk notation conventions
- Nature Methods: Points of Significance — Statistical best practices for figures
- Weissgerber et al. (2015) Beyond Bar and Line Graphs — Why to show individual data points alongside statistical annotations
Related Skills
seaborn-statistical-plots— Seaborn plotting fundamentals; use this guide's annotation workflow on top of seaborn figuresscientific-visualization— General scientific figure design principles
BWA-MEM2 — DNA Short-Read Aligner
Overview
BWA-MEM2 aligns short DNA reads (Illumina, 50–250 bp) to a reference genome using the BWT-FM index. It is the standard aligner for whole-genome sequencing (WGS), whole-exome sequencing (WES), ChIP-seq, and ATAC-seq DNA alignment. BWA-MEM2 is 2× faster than the original BWA-MEM while producing identical results. It outputs SAM format with proper read group (@RG) headers required by GATK HaplotypeCaller and Picard tools. For paired-end reads, it marks proper pairs and resolves chimeric/split reads into supplementary alignments.
When to Use
- Aligning WGS or WES Illumina reads to a reference genome for variant calling (SNP, indel, SV)
- ChIP-seq or ATAC-seq DNA alignment to produce BAM files for peak calling with MACS3
- Producing GATK-compatible BAM files with
@RGread group tags - Aligning reads ≥ 50 bp; for shorter reads (< 50 bp), BWA-backtrack may be more appropriate
- Re-aligning legacy FASTQ files to an updated reference genome assembly
- Use STAR instead for RNA-seq reads that span splice junctions
- Use Bowtie2 as an alternative for local alignment or when index size must be minimized
Prerequisites
- Software: bwa-mem2 (conda or pre-compiled binary), samtools
- Reference: genome FASTA (e.g., GRCh38, hg19, mm10)
- RAM: ~28 GB for human genome index; 6–8 GB for mouse
Check before installing: The tool may already be available in the current environment (e.g., inside a
pixi/condaenv). Runcommand -v bwa-mem2first and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool viapixi run bwa-mem2rather than barebwa-mem2.
# Install with conda (recommended)
conda install -c bioconda bwa-mem2 samtools
# Or download pre-compiled binary
wget https://github.com/bwa-mem2/bwa-mem2/releases/download/v2.2.1/bwa-mem2-2.2.1_x64-linux.tar.bz2
tar -jxf bwa-mem2-2.2.1_x64-linux.tar.bz2
export PATH="$PWD/bwa-mem2-2.2.1_x64-linux:$PATH"
# Verify
bwa-mem2 version
# 2.2.1
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: reference_bundle
kind: required
source: user
ask: "Which species and genome assembly should reads be aligned against?"
default: null
- id: D2
param: read_group
kind: required
source: user
ask: "What sample name and library identity should be stamped into each alignment?"
default: null
- id: D3
param: downstream_compatibility
kind: derived
source: upstream
ask: "Will GATK or Picard consume this BAM, requiring supplementary alignments flagged and soft-clipped?"
default: "-M -Y when a GATK/Picard step follows"
- id: D4
param: seedLength
kind: optional
source: user
ask: "Do these reads need a shorter seed than the default to map sensitively?"
default: 19
- id: D5
param: minAlignmentScore
kind: optional
source: user
ask: "How weak an alignment is still worth reporting?"
default: 30
- id: D6
param: interleavedInput
kind: derived
source: data
ask: "Are both mates held in a single interleaved FASTQ?"
default: false
skip_if: "reads supplied as separate R1/R2 files"
- id: D7
param: threads
kind: never_ask
source: data
reason: "Affects runtime only, not the alignments"
default: "min(8, available_cores)"
D2 is not cosmetic. The sample name in the read group is what joint genotyping groups by; an omitted or duplicated one silently merges or splits samples downstream, long after the alignment itself looks fine.
Quick Start
# 1. Build genome index (~30 min, run once)
bwa-mem2 index GRCh38.fa
# 2. Align paired-end reads and sort
bwa-mem2 mem -t 16 -R "@RG\tID:sample1\tSM:sample1\tPL:ILLUMINA" \
GRCh38.fa sample1_R1.fastq.gz sample1_R2.fastq.gz \
| samtools sort -@ 8 -o sample1.sorted.bam
# 3. Index the BAM
samtools index sample1.sorted.bam
echo "Aligned reads: $(samtools view -c -F 4 sample1.sorted.bam)"
Workflow
Step 1: Download Reference Genome
Obtain the reference genome FASTA file matching the target assembly.
# Download GRCh38 primary assembly (human)
wget https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/405/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna.gz
gunzip GCA_000001405.15_GRCh38_no_alt_analysis_set.fna.gz
mv GCA_000001405.15_GRCh38_no_alt_analysis_set.fna GRCh38.fa
# Or use ENSEMBL/GENCODE
wget https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_47/GRCh38.primary_assembly.genome.fa.gz
gunzip GRCh38.primary_assembly.genome.fa.gz
echo "Reference size: $(du -sh GRCh38.fa)"
Step 2: Build BWA-MEM2 Index
Index the reference genome — required once per genome, takes ~25-35 min for human.
# Build index (~28 GB RAM required for human genome)
bwa-mem2 index GRCh38.fa
# This creates: GRCh38.fa.0123, GRCh38.fa.amb, GRCh38.fa.ann,
# GRCh38.fa.bwt.2bit.64, GRCh38.fa.pac
echo "Index files: $(ls GRCh38.fa.* | wc -l) created"
ls -lh GRCh38.fa.*
Step 3: Align Paired-End Reads
Align FASTQ reads with a read group header required for GATK compatibility.
# Align with read group (required for GATK)
# @RG fields: ID (run ID), SM (sample name), PL (platform), LB (library), PU (flowcell)
bwa-mem2 mem \
-t 16 \
-R "@RG\tID:sample1_run1\tSM:sample1\tPL:ILLUMINA\tLB:lib1\tPU:flowcell1" \
GRCh38.fa \
sample1_R1.fastq.gz \
sample1_R2.fastq.gz \
| samtools sort -@ 8 -m 2G -o sample1.sorted.bam
samtools index sample1.sorted.bam
echo "Alignment complete."
echo "Total reads: $(samtools view -c sample1.sorted.bam)"
echo "Mapped reads: $(samtools view -c -F 4 sample1.sorted.bam)"
Step 4: Mark PCR Duplicates
Remove or mark optical and PCR duplicates before variant calling.
# Option A: samtools markdup (fast)
samtools fixmate -m sample1.sorted.bam sample1.fixmate.bam
samtools sort -@ 8 -o sample1.fixmate.sorted.bam sample1.fixmate.bam
samtools markdup -@ 8 sample1.fixmate.sorted.bam sample1.markdup.bam
samtools index sample1.markdup.bam
echo "Duplication rate:"
samtools flagstat sample1.markdup.bam | grep "duplicate"
# Option B: Picard MarkDuplicates (GATK best practices)
picard MarkDuplicates \
INPUT=sample1.sorted.bam \
OUTPUT=sample1.markdup.bam \
METRICS_FILE=sample1.dupmetrics.txt \
REMOVE_DUPLICATES=false \
CREATE_INDEX=true
cat sample1.dupmetrics.txt | grep -A2 "ESTIMATED"
Step 5: Assess Alignment Quality
Generate alignment statistics and check key quality metrics.
# Full alignment statistics
samtools flagstat sample1.markdup.bam > sample1.flagstat.txt
cat sample1.flagstat.txt
# Coverage statistics
samtools coverage sample1.markdup.bam | head -30
# Parse key metrics with Python
python3 - << 'EOF'
from pathlib import Path
flagstat = Path("sample1.flagstat.txt").read_text()
for line in flagstat.splitlines():
if any(kw in line for kw in ["total", "mapped", "properly paired", "duplicate"]):
print(line)
EOF
Step 6: Complete WGS/WES Pipeline → Variant Calling
Pipe BWA-MEM2 output directly into GATK HaplotypeCaller.
#!/bin/bash
# Complete WGS alignment → variant calling pipeline
GENOME="GRCh38.fa"
SAMPLE="sample1"
R1="data/${SAMPLE}_R1.fastq.gz"
R2="data/${SAMPLE}_R2.fastq.gz"
THREADS=16
OUTDIR="results/${SAMPLE}"
mkdir -p "$OUTDIR"
# Step 1: Align + sort
bwa-mem2 mem -t $THREADS \
-R "@RG\tID:${SAMPLE}\tSM:${SAMPLE}\tPL:ILLUMINA\tLB:lib1\tPU:run1" \
$GENOME $R1 $R2 \
| samtools sort -@ 8 -o $OUTDIR/${SAMPLE}.sorted.bam
samtools index $OUTDIR/${SAMPLE}.sorted.bam
# Step 2: Mark duplicates
samtools fixmate -m $OUTDIR/${SAMPLE}.sorted.bam - \
| samtools sort -@ 8 \
| samtools markdup -@ 8 - $OUTDIR/${SAMPLE}.markdup.bam
samtools index $OUTDIR/${SAMPLE}.markdup.bam
# Step 3: GATK variant calling
gatk HaplotypeCaller \
-R $GENOME \
-I $OUTDIR/${SAMPLE}.markdup.bam \
-O $OUTDIR/${SAMPLE}.g.vcf.gz \
-ERC GVCF \
--native-pair-hmm-threads 4
echo "Pipeline complete: $OUTDIR/${SAMPLE}.g.vcf.gz"
Key Parameters
| Parameter | Default | Range/Options | Effect |
|---|---|---|---|
-t | 1 | 1–64 | CPU threads; use 8–16 for production runs |
-R | — | @RG\tID:...\tSM:... | Read group string; required for GATK compatibility |
-k | 19 | 10–28 | Minimum seed length; lower = more sensitive for shorter reads |
-w | 100 | 50–500 | Band width for Smith-Waterman alignment |
-M | off | flag | Mark split/supplementary reads as secondary (BWA-MEM style); needed for Picard compatibility |
-a | off | flag | Output all alignments for single-end reads (for seeding; increases file size) |
-p | off | flag | Treat input as interleaved paired-end FASTQ |
-c | 500 | 100–10000 | Skip alignment for MEM count > threshold (reduces multi-mapper noise) |
-T | 30 | 20–60 | Minimum alignment score threshold; lower = report more low-quality alignments |
-Y | off | flag | Use soft clipping for supplementary alignments (recommended for GATK) |
Common Recipes
Recipe 1: Batch Align Multiple Samples
#!/bin/bash
# Align all samples in parallel using GNU parallel or sequential loop
GENOME="GRCh38.fa"
SAMPLES=(ctrl_1 ctrl_2 treat_1 treat_2)
THREADS=12
for sample in "${SAMPLES[@]}"; do
echo "=== Aligning $sample ==="
bwa-mem2 mem -t $THREADS \
-R "@RG\tID:${sample}\tSM:${sample}\tPL:ILLUMINA\tLB:lib1\tPU:run1" \
$GENOME \
data/${sample}_R1.fastq.gz \
data/${sample}_R2.fastq.gz \
| samtools sort -@ 4 -m 2G -o results/${sample}.sorted.bam
samtools index results/${sample}.sorted.bam
MAPPED=$(samtools view -c -F 4 results/${sample}.sorted.bam)
TOTAL=$(samtools view -c results/${sample}.sorted.bam)
echo "$sample: $MAPPED / $TOTAL reads mapped"
done
Recipe 2: Parse Alignment Metrics with Python
import subprocess
import pandas as pd
from pathlib import Path
samples = ["ctrl_1", "ctrl_2", "treat_1", "treat_2"]
metrics = []
for sample in samples:
bam = f"results/{sample}.sorted.bam"
result = subprocess.run(
["samtools", "flagstat", bam], capture_output=True, text=True
)
stats = {}
for line in result.stdout.splitlines():
if "total" in line:
stats["total"] = int(line.split()[0])
elif "mapped" in line and "%" in line:
stats["mapped"] = int(line.split()[0])
stats["pct_mapped"] = float(line.split("(")[1].split("%")[0])
elif "properly paired" in line:
stats["properly_paired"] = int(line.split()[0])
stats["sample"] = sample
metrics.append(stats)
df = pd.DataFrame(metrics).set_index("sample")
print(df[["total", "mapped", "pct_mapped", "properly_paired"]])
df.to_csv("alignment_metrics.tsv", sep="\t")
Expected Outputs
| Output | Format | Description |
|---|---|---|
*.sorted.bam | BAM | Coordinate-sorted aligned reads; index with samtools index |
*.sorted.bam.bai | BAI | BAM index; required for random access by GATK and IGV |
*.flagstat.txt | Text | Alignment summary: total/mapped/paired/duplicate counts and percentages |
*.markdup.bam | BAM | Duplicate-marked BAM; use as input to GATK HaplotypeCaller |
*.dupmetrics.txt | Text | Picard duplication metrics with estimated library size |
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| Low mapping rate (< 85%) | Genome mismatch, contamination, or low quality reads | Verify genome assembly matches sample; run FastQC; trim adapters with Trim Galore |
@RG header missing error in GATK | -R flag not specified during alignment | Re-align with -R "@RG\tID:...\tSM:...\tPL:ILLUMINA" |
| Out of memory during indexing | Insufficient RAM for genome index | BWA-MEM2 requires ~28 GB for human; use classic bwa with less RAM if needed |
| Unbalanced paired-end counts | Interleaved FASTQ or file mismatch | Add -p for interleaved; verify R1/R2 read counts with zcat r1.fq.gz | wc -l |
[E::bwa_idx_load_from_disk] error | Index files missing or wrong prefix | Re-run bwa-mem2 index genome.fa; ensure all .0123, .bwt.2bit.64 files exist |
| Slow alignment speed | Low thread count or slow I/O | Use -t 16 or more; store data on SSD; pipe directly to samtools sort |
| Supplementary alignments causing issues | Split reads in downstream tools | Add -M flag to mark split reads as secondary (Picard compatibility) |
| GATK base quality score recalibration fails | Missing known variant VCF | Download dbSNP VCF for your genome assembly from NCBI or GATK resource bundle |
References
- BWA-MEM2 GitHub: bwa-mem2/bwa-mem2 — source code, pre-compiled binaries, and benchmarks
- Vasimuddin M et al. (2019) "Efficient Architecture-Aware Acceleration of BWA-MEM for Multicore Systems" — IPDPS 2019. DOI:10.1109/IPDPS.2019.00041
- Li H & Durbin R (2009) "Fast and accurate short read alignment with Burrows-Wheeler Aligner" — Bioinformatics 25(14):1754-1760. DOI:10.1093/bioinformatics/btp324
- GATK Best Practices for germline short variant discovery — GATK workflow recommending BWA-MEM2 alignment
Pysam — Genomic File Toolkit
Overview
Pysam provides a Pythonic interface to htslib for reading, manipulating, and writing genomic data files. It handles SAM/BAM/CRAM alignments, VCF/BCF variants, and FASTA/FASTQ sequences with efficient region-based random access. Also exposes samtools and bcftools as callable Python functions.
When to Use
- Reading and querying BAM/CRAM alignment files (region extraction, read filtering)
- Analyzing VCF/BCF variant files (genotype access, variant filtering, annotation)
- Extracting reference sequences from indexed FASTA files
- Calculating per-base coverage and pileup statistics
- Building custom bioinformatics pipelines that combine alignment + variant + sequence data
- Quality control of NGS data (mapping quality, flag filtering, coverage)
- For alignment from FASTQ (read mapping), use STAR, BWA, or minimap2 instead
- For variant calling from BAM, use GATK or DeepVariant instead
Prerequisites
pip install pysam
Note: Requires htslib C library (bundled with pip install on most platforms). On some Linux systems, may need libhts-dev or equivalent. Index files (.bai, .tbi, .fai) required for random access — create with pysam.index(), pysam.tabix_index(), or pysam.faidx().
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: mappingQualityFloor
kind: required
source: user
ask: "Below what mapping confidence should reads be ignored when piling up or counting?"
default: "0 - no filter"
- id: D2
param: baseQualityFloor
kind: required
source: user
ask: "How confident must an individual base call be to be counted at a position?"
default: 15
- id: D3
param: regionTruncation
kind: required
source: user
ask: "Should a pileup report only positions inside the requested region, or every position touched by an overlapping read?"
default: "not truncated - overlapping reads extend the reported range"
- id: D4
param: indexRequirement
kind: derived
source: data
ask: "Is the file indexed, or must records be streamed sequentially?"
default: "indexed random access"
- id: D5
param: iteratorSafety
kind: never_ask
source: user
reason: "Whether simultaneous iterators are allowed is an API-usage concern with a small overhead cost, not a result-changing choice"
default: false
D3 is the one that surprises people: an untruncated pileup returns positions outside the region that was asked for, so per-region statistics computed straight off the result cover a wider span than intended.
Quick Start
import pysam
# Read BAM file, fetch reads in a region
with pysam.AlignmentFile("sample.bam", "rb") as bam:
for read in bam.fetch("chr1", 1000, 2000):
print(f"{read.query_name}: pos={read.reference_start}, mapq={read.mapping_quality}")
print(f"Total reads in region: {bam.count('chr1', 1000, 2000)}")
Core API
1. Alignment Files (SAM/BAM/CRAM)
Read, query, and write aligned sequencing reads.
import pysam
# Open BAM (binary) or SAM (text) file
bam = pysam.AlignmentFile("sample.bam", "rb") # rb=read BAM, r=read SAM, rc=read CRAM
# Fetch reads overlapping a region (requires .bai index)
for read in bam.fetch("chr1", 10000, 20000):
print(f"Name: {read.query_name}")
print(f" Position: {read.reference_start}-{read.reference_end}")
print(f" MAPQ: {read.mapping_quality}")
print(f" CIGAR: {read.cigarstring}")
print(f" Sequence: {read.query_sequence[:30]}...")
break
# Count reads in region (fast, no iteration needed)
n_reads = bam.count("chr1", 10000, 20000)
print(f"Reads in region: {n_reads}")
# Filter reads by quality and flags
for read in bam.fetch("chr1", 10000, 20000):
if read.mapping_quality >= 30 and not read.is_unmapped and not read.is_duplicate:
pass # Process high-quality, mapped, non-duplicate reads
bam.close()
# Write filtered reads to a new BAM file
with pysam.AlignmentFile("input.bam", "rb") as inbam:
with pysam.AlignmentFile("filtered.bam", "wb", header=inbam.header) as outbam:
for read in inbam.fetch("chr1", 10000, 20000):
if read.mapping_quality >= 30:
outbam.write(read)
# Index the output
pysam.index("filtered.bam")
print("Created filtered.bam + filtered.bam.bai")
2. Coverage and Pileup Analysis
Calculate per-base coverage statistics.
import pysam
import numpy as np
bam = pysam.AlignmentFile("sample.bam", "rb")
# Pileup: per-base coverage with read-level detail
for pileup_col in bam.pileup("chr1", 10000, 10100, min_mapping_quality=30):
bases = [p.alignment.query_sequence[p.query_position]
for p in pileup_col.pileups if not p.is_del and p.query_position is not None]
print(f"Pos {pileup_col.reference_pos}: depth={pileup_col.nsegments}, bases={''.join(bases[:5])}")
# Quick coverage count per region (faster than pileup)
coverage = bam.count_coverage("chr1", 10000, 10100, quality_threshold=20)
# Returns tuple of 4 arrays (A, C, G, T counts per position)
total_cov = np.array(coverage).sum(axis=0)
print(f"Mean coverage: {total_cov.mean():.1f}x")
bam.close()
3. Variant Files (VCF/BCF)
Read, query, and filter genetic variants.
import pysam
# Open VCF/BCF file
vcf = pysam.VariantFile("variants.vcf.gz")
# Iterate all variants
for record in vcf.fetch("chr1", 10000, 50000):
print(f"{record.chrom}:{record.pos} {record.ref}>{','.join(record.alts or [])}")
print(f" QUAL={record.qual}, FILTER={list(record.filter)}")
print(f" INFO: {dict(record.info)}")
# Access genotypes per sample
for sample in record.samples:
gt = record.samples[sample]["GT"]
print(f" {sample}: GT={gt}")
break
vcf.close()
# Filter variants and write to new VCF
with pysam.VariantFile("variants.vcf.gz") as vcf_in:
with pysam.VariantFile("filtered.vcf.gz", "wz", header=vcf_in.header) as vcf_out:
for record in vcf_in:
if record.qual and record.qual >= 30 and "PASS" in record.filter:
vcf_out.write(record)
pysam.tabix_index("filtered.vcf.gz", preset="vcf")
print("Created filtered.vcf.gz + filtered.vcf.gz.tbi")
4. Sequence Files (FASTA/FASTQ)
Random access to reference sequences and sequential reading of raw reads.
import pysam
# FASTA: random access (requires .fai index)
fasta = pysam.FastaFile("reference.fasta")
seq = fasta.fetch("chr1", 10000, 10050)
print(f"Sequence ({len(seq)} bp): {seq}")
print(f"Available contigs: {fasta.references[:5]}")
print(f"Contig lengths: {dict(zip(fasta.references[:3], fasta.lengths[:3]))}")
fasta.close()
# Create FASTA index if needed
# pysam.faidx("reference.fasta")
# FASTQ: sequential reading
with pysam.FastxFile("reads.fastq.gz") as fq:
for i, entry in enumerate(fq):
print(f"Read {entry.name}: {len(entry.sequence)} bp, mean_qual={sum(entry.get_quality_array())/len(entry.sequence):.1f}")
if i >= 2:
break
5. Read Groups and Sample Information
Extract and filter reads by read group (essential for multi-sample BAM files).
import pysam
bam = pysam.AlignmentFile("multisample.bam", "rb")
# Access read group information from BAM header
print("Read groups in file:")
for rg_dict in bam.header.get("RG", []):
print(f" ID: {rg_dict['ID']}, Sample: {rg_dict.get('SM', 'N/A')}, Library: {rg_dict.get('LB', 'N/A')}, Platform: {rg_dict.get('PL', 'N/A')}")
# Get all samples in the BAM (from RG headers)
samples = set()
for rg_dict in bam.header.get("RG", []):
if "SM" in rg_dict:
samples.add(rg_dict["SM"])
print(f"Samples in BAM: {sorted(samples)}")
bam.close()
# Filter reads by read group ID
def extract_reads_by_rg(bam_path, rg_id, output_path):
"""Extract all reads from a specific read group.
WARNING: Uses fetch(until_eof=True), which scans the entire BAM sequentially.
Multi-sample BAMs can be tens to hundreds of GB — this may be slow.
For large files, prefer region-based filtering:
for read in bam.fetch("chr1", start, end): ...
Or use the samtools CLI equivalent (faster for one-off extractions):
samtools view -b -r <rg_id> input.bam -o output.bam
"""
with pysam.AlignmentFile(bam_path, "rb") as bam_in:
with pysam.AlignmentFile(output_path, "wb", header=bam_in.header) as bam_out:
for read in bam_in.fetch(until_eof=True):
if read.has_tag("RG") and read.get_tag("RG") == rg_id:
bam_out.write(read)
pysam.index(output_path)
print(f"Extracted reads from RG:{rg_id} → {output_path}")
extract_reads_by_rg("multisample.bam", "SAMPLE_001_LaneA", "sample001_laneA.bam")
from collections import defaultdict
import pysam
# Count reads per sample
def reads_per_sample(bam_path):
"""Count reads per sample from read group information.
Two distinct "unknown" cases are tracked separately:
- "no_sm_field": RG header entry exists but is missing the SM (sample name) field.
- "undefined_rg": A read carries an RG tag not declared in the BAM header.
"""
counts = defaultdict(int)
rg_to_sample = {}
with pysam.AlignmentFile(bam_path, "rb") as bam:
# Build RG → sample mapping from header
for rg_dict in bam.header.get("RG", []):
rg_id = rg_dict["ID"]
# (a) RG header entry lacks SM field
rg_to_sample[rg_id] = rg_dict.get("SM", "no_sm_field")
# Count reads per resolved sample name
for read in bam.fetch(until_eof=True):
if read.has_tag("RG"):
rg_id = read.get_tag("RG")
# (b) Read's RG tag is not declared in the header
sample = rg_to_sample.get(rg_id, "undefined_rg")
counts[sample] += 1
return dict(counts)
sample_counts = reads_per_sample("multisample.bam")
for sample, count in sorted(sample_counts.items()):
print(f" {sample}: {count:,} reads")
6. Samtools/Bcftools CLI Access
Call samtools and bcftools commands from Python.
import pysam
# Sort BAM file
pysam.sort("-o", "sorted.bam", "input.bam")
# Index BAM
pysam.index("sorted.bam")
# View region as BAM
pysam.view("-b", "-o", "region.bam", "sorted.bam", "chr1:1000-2000")
# BCFtools: compress and index VCF
pysam.bcftools.view("-O", "z", "-o", "output.vcf.gz", "input.vcf")
pysam.tabix_index("output.vcf.gz", preset="vcf")
# Error handling
try:
pysam.sort("-o", "output.bam", "nonexistent.bam")
except pysam.SamtoolsError as e:
print(f"samtools error: {e}")
CLI equivalents (for reference — use Python API in automated pipelines):
# These are equivalent to the Python calls above:
samtools sort -o sorted.bam input.bam
samtools index sorted.bam
samtools view -b -o region.bam sorted.bam chr1:1000-2000
bcftools view -O z -o output.vcf.gz input.vcf
Key Concepts
Coordinate Systems
Critical: pysam uses 0-based, half-open coordinates (Python convention):
| System | Start | End | Example: "bases 1000-2000" |
|---|---|---|---|
| pysam Python API | 0-based | exclusive | fetch("chr1", 999, 2000) |
| samtools region string | 1-based | inclusive | fetch("chr1:1000-2000") |
| VCF file format | 1-based | — | record.pos = 1-based, record.start = 0-based |
| BED format | 0-based | exclusive | chr1\t999\t2000 |
Index File Requirements
| File Type | Index Extension | Create With |
|---|---|---|
| BAM | .bai | pysam.index("file.bam") |
| CRAM | .crai | pysam.index("file.cram") |
| FASTA | .fai | pysam.faidx("file.fasta") |
| VCF.gz | .tbi | pysam.tabix_index("file.vcf.gz", preset="vcf") |
| BCF | .csi | pysam.tabix_index("file.bcf", preset="bcf") |
Without an index, use fetch(until_eof=True) for sequential reading.
File Mode Strings
| Mode | Format | Direction |
|---|---|---|
"rb" | BAM (binary) | Read |
"r" | SAM (text) | Read |
"rc" | CRAM | Read |
"wb" | BAM | Write |
"w" | SAM | Write |
"wz" | VCF.gz (compressed) | Write |
Common Workflows
Workflow 1: Coverage Analysis for Target Regions
Goal: Calculate coverage statistics for a set of target regions (e.g., exome capture targets).
import pysam
import numpy as np
def coverage_for_regions(bam_path, regions, min_mapq=30):
"""Calculate coverage stats for a list of (chrom, start, end) regions."""
results = []
with pysam.AlignmentFile(bam_path, "rb") as bam:
for chrom, start, end in regions:
cov = np.array(bam.count_coverage(chrom, start, end,
quality_threshold=min_mapq))
total = cov.sum(axis=0)
results.append({
"region": f"{chrom}:{start}-{end}",
"mean_cov": total.mean(),
"min_cov": total.min(),
"pct_above_20x": (total >= 20).mean() * 100,
})
return results
regions = [("chr1", 10000, 10500), ("chr1", 20000, 20500), ("chr2", 5000, 5500)]
stats = coverage_for_regions("sample.bam", regions)
for s in stats:
print(f"{s['region']}: mean={s['mean_cov']:.1f}x, min={s['min_cov']}x, ≥20x={s['pct_above_20x']:.1f}%")
Workflow 2: Variant Annotation with Read Support
Goal: For each variant in a VCF, count supporting reads from the BAM.
import pysam
def annotate_variants_with_reads(vcf_path, bam_path, output_path):
"""Add read support counts to each variant."""
with pysam.VariantFile(vcf_path) as vcf_in:
# Add INFO field to header
vcf_in.header.add_line(
'##INFO=<ID=READ_SUPPORT,Number=1,Type=Integer,Description="Reads supporting alt allele">'
)
with pysam.VariantFile(output_path, "w", header=vcf_in.header) as vcf_out:
with pysam.AlignmentFile(bam_path, "rb") as bam:
for record in vcf_in:
alt_count = 0
for col in bam.pileup(record.chrom, record.start, record.stop,
min_mapping_quality=30, truncate=True):
if col.reference_pos == record.start:
for p in col.pileups:
if (not p.is_del and p.query_position is not None and
p.alignment.query_sequence[p.query_position] in (record.alts or [])):
alt_count += 1
record.info["READ_SUPPORT"] = alt_count
vcf_out.write(record)
annotate_variants_with_reads("variants.vcf", "sample.bam", "annotated.vcf")
print("Created annotated.vcf with READ_SUPPORT field")
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
| mode string | AlignmentFile, VariantFile | — | "rb", "r", "rc", "wb", "w", "wz" | File format and read/write direction |
min_mapping_quality | pileup() | 0 | 0–60 | Filter reads below this MAPQ |
quality_threshold | count_coverage() | 15 | 0–40 | Minimum base quality to count |
truncate | pileup() | False | True/False | Truncate pileup to exact region (True) vs include overlapping reads (False) |
until_eof | fetch() | False | True/False | Read all records sequentially without index |
multiple_iterators | fetch() | False | True/False | Allow multiple simultaneous iterators (slight overhead) |
preset | tabix_index() | — | "vcf", "bed", "gff", "sam" | File format for tabix indexing |
Best Practices
-
Always use context managers (
withstatement) for automatic file cleanup. Unclosed files can leak file descriptors. -
Create and verify index files first: Most random-access operations fail silently or raise cryptic errors without indexes. Check for
.bai/.tbi/.faifiles before queries. -
Use
count()instead of iterating to count reads:bam.count("chr1", 1000, 2000)is much faster thansum(1 for _ in bam.fetch(...)). -
Use
count_coverage()for coverage,pileup()for base-level detail:count_coverage()is faster when you only need depth numbers. Usepileup()only when you need per-read, per-base information. -
Anti-pattern — mixing 0-based and 1-based coordinates: Always double-check coordinate systems when combining pysam (0-based) with VCF files (1-based POS), BED files (0-based), or region strings (1-based). See Key Concepts table.
-
Anti-pattern — forgetting
truncate=Truein pileup: Withouttruncate=True,pileup()extends to the full extent of overlapping reads, which can be much larger than the requested region.
Common Recipes
Recipe: Extract Gene Sequences from Reference
import pysam
def get_gene_sequence(fasta_path, chrom, start, end, strand="+"):
"""Extract gene sequence, reverse-complement if on minus strand."""
with pysam.FastaFile(fasta_path) as fasta:
seq = fasta.fetch(chrom, start, end)
if strand == "-":
complement = str.maketrans("ACGTacgt", "TGCAtgca")
seq = seq.translate(complement)[::-1]
return seq
seq = get_gene_sequence("reference.fasta", "chr1", 10000, 11000, strand="-")
print(f"Gene sequence ({len(seq)} bp): {seq[:50]}...")
Recipe: BAM Statistics Summary
import pysam
def bam_summary(bam_path):
"""Quick summary statistics for a BAM file."""
with pysam.AlignmentFile(bam_path, "rb") as bam:
stats = {"total": 0, "mapped": 0, "unmapped": 0, "duplicates": 0, "mapq_ge30": 0}
for read in bam.fetch(until_eof=True):
stats["total"] += 1
if read.is_unmapped:
stats["unmapped"] += 1
else:
stats["mapped"] += 1
if read.is_duplicate:
stats["duplicates"] += 1
if read.mapping_quality >= 30:
stats["mapq_ge30"] += 1
return stats
summary = bam_summary("sample.bam")
for k, v in summary.items():
print(f" {k}: {v:,}")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
ValueError: could not open alignment file | Missing file or wrong mode string | Check file path; use "rb" for BAM, "r" for SAM |
ValueError: fetch called on bamfile without index | No .bai index file | Run pysam.index("file.bam") first |
| Region returns unexpected reads | Reads overlapping boundaries are included | Use truncate=True in pileup() or filter by read.reference_start >= start |
| Coordinate off-by-one errors | Mixing 0-based (pysam) with 1-based (VCF, samtools) | See Key Concepts coordinate table; record.pos is 1-based, record.start is 0-based |
PileupProxy accessed after iterator finished | Pileup iterator went out of scope | Store needed data from pileup columns immediately, don't save PileupProxy references |
SamtoolsError from CLI calls | Invalid arguments or missing input | Wrap in try/except pysam.SamtoolsError; check samtools docs for argument syntax |
| Very slow iteration | Iterating all reads without region query | Use fetch("chr1", start, end) for targeted queries; use indexed files |
| Read group filter returns 0 reads | RG tag missing or wrong ID specified | Verify RG tag exists: read.has_tag("RG"); list available RGs from bam.header.get("RG", []) |
Related Skills
- biopython-molecular-biology — sequence I/O and alignment; complementary for non-BAM sequence formats
- pydeseq2-differential-expression — downstream analysis of read counts from BAM coverage data
- scanpy-scrna-seq — single-cell analysis; pysam handles the upstream BAM processing
References
- Pysam documentation — official API reference
- htslib — underlying C library for genomic file formats
- SAM format specification — SAM/BAM format details
- Li et al. (2009) "The Sequence Alignment/Map format and SAMtools" — Bioinformatics
samtools — SAM/BAM/CRAM Alignment Toolkit
Overview
samtools is the standard command-line toolkit for processing sequence alignment files in SAM, BAM, and CRAM formats. It handles the complete alignment file lifecycle: format conversion, coordinate sorting, index creation, quality control statistics, read filtering, duplicate marking, and multi-file merging. samtools is a near-universal component of NGS pipelines between alignment (STAR, BWA) and downstream analysis (variant calling, peak calling, coverage).
When to Use
- Sorting BAM files by coordinate after alignment (required before indexing)
- Indexing sorted BAM files for random access and region queries
- Converting between SAM, BAM, and CRAM formats to save storage
- Generating alignment QC metrics: mapping rates, insert sizes, per-chromosome stats
- Filtering reads by mapping quality, FLAG bits, or genomic regions
- Marking or removing PCR duplicates before variant calling
- Merging multiple BAM files from different lanes or samples
- Calculating per-base depth or coverage breadth for target regions
- Use
pysaminstead for Python-native BAM manipulation in custom scripts - Use
deeptools bamCoverageinstead when you need normalized bigWig coverage tracks - Use
mosdepthinstead for whole-genome per-base depth (faster, parallelized)
Prerequisites
- Installation: samtools 1.17+ recommended
- Input requirements: SAM/BAM/CRAM files; CRAM requires FASTA reference
- Companion tools:
samtools faidxfor FASTA indexing;samtools sortbeforesamtools index
Check before installing: The tool may already be available in the current environment (e.g., inside a
pixi/condaenv). Runcommand -v samtoolsfirst and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool viapixi run samtoolsrather than baresamtools.
# Bioconda (recommended)
conda install -c bioconda samtools
# Homebrew (macOS)
brew install samtools
# Verify
samtools --version | head -1
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: mappingQualityFloor
kind: required
source: user
ask: "Below what mapping confidence should reads be dropped? Everything counted downstream inherits this cut."
default: "0 - keep every alignment, including ambiguously placed ones"
- id: D2
param: flagFilter
kind: required
source: user
ask: "Which read classes should be excluded - unmapped, secondary, supplementary, failed-QC, duplicates?"
default: "keep all classes"
- id: D3
param: duplicateHandling
kind: required
source: user
ask: "Should duplicates be marked and left in place, or physically removed from the file?"
default: "marked, not removed - downstream tools can then choose"
- id: D4
param: sortOrder
kind: derived
source: upstream
ask: "Does the next step need coordinate order, or name order for mate pairing?"
default: "coordinate"
- id: D5
param: outputFormat
kind: optional
source: user
ask: "Write BAM, or CRAM against a reference to save space?"
default: "BAM"
- id: D6
param: opticalDuplicateDistance
kind: optional_conditional
source: data
ask: "Should optical duplicates be distinguished from PCR duplicates, using this platform's pixel distance?"
default: "not distinguished"
- id: D7
param: threadsAndMemory
kind: never_ask
source: data
reason: "Compression threads and per-thread sort memory affect runtime and peak RAM, not the records"
default: "available cores, 768M per thread"
D1 and D2 look like plumbing and are not. A MAPQ floor silently removes multi-mapping regions - paralogs, recent duplications, repeat-adjacent genes - from everything computed afterwards, and a run that keeps secondary alignments counts the same fragment several times. Neither is visible in the output file.
Quick Start
# Typical post-alignment workflow: sort → index → QC
samtools sort -@ 8 -o sorted.bam input.bam
samtools index sorted.bam
samtools flagstat sorted.bam
Core API
Module 1: BAM/SAM I/O and Format Conversion
Convert between SAM/BAM/CRAM formats and extract subsets.
# SAM → BAM (saves ~75% disk space)
samtools view -b -h input.sam -o output.bam
# BAM → CRAM (saves additional 40-50%)
samtools view -C -T reference.fa input.bam -o output.cram
# Filter: mapping quality ≥20, exclude unmapped (-F 4)
samtools view -q 20 -F 4 input.bam -o filtered.bam
# Extract specific region (requires index)
samtools view -h sorted.bam "chr1:1000000-2000000" -o region.bam
# Count reads matching filter
samtools view -c -F 4 input.bam
# Output: 45231923 (number of mapped reads)
# Extract reads as FASTQ (for realignment or de novo assembly)
samtools fastq -@ 4 -1 R1.fastq.gz -2 R2.fastq.gz -0 unpaired.fastq.gz input.bam
# Extract reads as FASTA
samtools fasta input.bam > reads.fasta
# Filter by read group
samtools view -r SAMPLE_001 multi_rg.bam -o sample001.bam
Module 2: Sorting and Indexing
Organize BAM files for efficient random access.
# Sort by coordinate (required before indexing)
samtools sort -@ 8 -m 2G input.bam -o sorted.bam
# Sort by read name (required for fixmate/markdup)
samtools sort -n -@ 8 input.bam -o namesorted.bam
# Index sorted BAM (creates sorted.bam.bai)
samtools index sorted.bam
# For chromosomes > 512 Mbp: use CSI index instead
samtools index -c sorted.bam
# Group reads by name (fast, for fixmate — no full sort needed)
samtools collate -o collated.bam input.bam
Module 3: Quality Control and Statistics
Generate alignment QC metrics and coverage reports.
# Quick summary: total, mapped, paired, properly paired
samtools flagstat sorted.bam
# Example output:
# 50000000 + 0 in total (QC-passed reads + QC-failed reads)
# 48523111 + 0 mapped (97.05% : N/A)
# 50000000 + 0 paired in sequencing
# 48490234 + 0 properly paired (96.98% : N/A)
# Per-chromosome mapped/unmapped read counts
samtools idxstats sorted.bam
# chr1 248956422 12345678 0
# chr2 242193529 11234567 0
# Comprehensive stats (insert sizes, GC content, base quality)
samtools stats -r reference.fa sorted.bam > full_stats.txt
grep "^SN" full_stats.txt | cut -f2,3 # Summary Numbers only
# Coverage report (min/max/mean per region/chromosome)
samtools coverage sorted.bam
# Per-base read depth for specific regions
samtools depth -b target_regions.bed sorted.bam > depth.txt
# Output: chr pos depth (e.g., chr1 1000 45)
# Statistics split by read group
samtools stats -S RG sorted.bam > per_rg_stats.txt
Module 4: Read Filtering and FLAG Operations
Filter reads using SAM FLAG bits for specific subsets.
# FLAG reference — common masks:
# 1 = paired 4 = unmapped
# 2 = proper pair 8 = mate unmapped
# 16 = reverse strand 64 = R1 (first in pair)
# 128 = R2 256= secondary alignment
# 1024 = PCR duplicate 2048= supplementary
# Extract properly paired, mapped reads (FLAG 2 set, 4 unset)
samtools view -f 2 -F 4 sorted.bam -o proper_pairs.bam
# Extract R1 reads only
samtools view -f 64 sorted.bam -o R1.bam
# Remove secondary and supplementary alignments
samtools view -F 2304 sorted.bam -o primary.bam
# Extract reads from BED file regions
samtools view -L regions.bed -b sorted.bam -o regions.bam
Module 5: Duplicate Handling
Mark or remove PCR duplicates before variant calling.
# Full duplicate marking workflow (collate → fixmate → sort → markdup)
samtools collate -@ 8 -o collated.bam input.bam
samtools fixmate -m -@ 8 collated.bam fixmated.bam
samtools sort -@ 8 -o sorted.bam fixmated.bam
samtools markdup -@ 8 sorted.bam marked.bam
samtools index marked.bam
# Check duplication rate
samtools flagstat marked.bam | grep "duplicates"
# Output: 2345678 + 0 duplicates (4.83%)
# NovaSeq optical duplicate detection (2500 pixel distance)
samtools markdup -d 2500 sorted.bam marked_novaseq.bam
# Remove duplicates instead of marking
samtools markdup -r sorted.bam deduped.bam
# Get duplication stats without writing output
samtools markdup -s sorted.bam /dev/null
Module 6: Multi-file Operations and Region Analysis
Merge BAM files and perform region-level analysis.
# Merge multiple BAM files (all must be sorted)
samtools merge -@ 8 merged.bam lane1.bam lane2.bam lane3.bam
# Merge files listed in a text file (one per line)
samtools merge -b bam_list.txt -@ 8 merged.bam
# Merge with read group tags from filenames
samtools merge -r merged.bam sample1.bam sample2.bam
# Extract specific chromosome region from merged output
samtools view -h merged.bam chr1 -b -o chr1.bam
Key Concepts
SAM FLAG Bits
FLAGS encode read properties as a sum of bit values. Common filtering patterns:
| Common Filter | -f (require) | -F (exclude) | Selects |
|---|---|---|---|
| Mapped reads | — | 4 | All aligned reads |
| Proper pairs | 2 | — | Properly paired, both mapped |
| Unique primary | — | 2308 | No secondary/supplementary/duplicate |
| R1 only | 64 | — | First-in-pair reads |
| Unmapped | 4 | — | Failed to align |
CRAM vs BAM vs SAM
| Format | Size | Speed | Requires |
|---|---|---|---|
| SAM | ~10× BAM | Slow I/O | Nothing |
| BAM | 1× | Fast | .bai index for random access |
| CRAM | ~0.6× BAM | Slightly slower | Reference FASTA + index |
Use CRAM for long-term storage; BAM for active analysis.
Common Workflows
Workflow 1: Post-Alignment QC and Preparation
Goal: Convert aligner output to analysis-ready BAM with QC metrics.
#!/bin/bash
SAMPLE="sample_001"
REF="reference.fa"
THREADS=8
# 1. Sort and index (aligner often outputs unsorted SAM/BAM)
samtools sort -@ $THREADS -o ${SAMPLE}.sorted.bam ${SAMPLE}.bam
samtools index ${SAMPLE}.sorted.bam
# 2. QC metrics
samtools flagstat ${SAMPLE}.sorted.bam > ${SAMPLE}.flagstat.txt
samtools stats -r $REF ${SAMPLE}.sorted.bam > ${SAMPLE}.stats.txt
samtools coverage ${SAMPLE}.sorted.bam > ${SAMPLE}.coverage.txt
# 3. Per-chromosome stats
samtools idxstats ${SAMPLE}.sorted.bam > ${SAMPLE}.idxstats.txt
echo "QC complete: $(grep 'mapped (' ${SAMPLE}.flagstat.txt | head -1)"
Workflow 2: Full Duplicate-Marking Pipeline
Goal: Prepare BAM for GATK or other variant callers requiring deduplicated input.
#!/bin/bash
INPUT="aligned.bam"
FINAL="deduped.bam"
THREADS=8
# Collate → fixmate → sort → markdup
samtools collate -@ $THREADS -o collated.bam $INPUT
samtools fixmate -m -@ $THREADS collated.bam fixmated.bam
samtools sort -@ $THREADS -o sorted.bam fixmated.bam
samtools markdup -@ $THREADS -s sorted.bam $FINAL
# Clean up intermediates
rm collated.bam fixmated.bam sorted.bam
# Index and verify
samtools index $FINAL
samtools flagstat $FINAL | grep "duplic"
# Expected: 3-15% duplicates (WGS); 10-30% for amplicon
Key Parameters
| Parameter | Command | Default | Range/Options | Effect |
|---|---|---|---|---|
-@ | Most | 0 | 1–N cores | Additional compression/I/O threads |
-m | sort | 768M | e.g., 2G, 4G | Memory per thread for sorting |
-q | view | 0 | 0–60 | Minimum mapping quality filter |
-f | view | 0 | FLAG bits | Include reads with ALL bits set |
-F | view | 0 | FLAG bits | Exclude reads with ANY bit set |
-b | view | — | flag | Output BAM format |
-C | view | — | flag | Output CRAM (requires -T) |
-T | view | — | FASTA path | Reference for CRAM output |
-d | markdup | 0 | 0–2500 | Optical duplicate pixel distance |
-r | markdup | — | flag | Remove duplicates (vs just mark) |
-n | sort | — | flag | Sort by read name instead of position |
-c | index | — | flag | Create CSI index (needed for chr > 512 Mb) |
Best Practices
-
Always sort before indexing:
samtools indexrequires coordinate-sorted input. Attempting to index an unsorted BAM will fail or produce incorrect results. -
Use
-@for all production runs: Most samtools commands are I/O-bound. Adding-@ 8provides near-linear speedup for compression/decompression with minimal overhead. -
Run flagstat before any analysis:
samtools flagstatruns in seconds and catches alignment failures (low mapping rate, unexpected paired-end rates) before wasting time on downstream steps. -
Use the collate → fixmate → sort → markdup pipeline: Running
samtools markdupdirectly on coordinate-sorted BAM without fixmate produces incorrect duplicate detection. The mate information added byfixmate -mis essential. -
Prefer CRAM for archiving: CRAM reduces storage 40-50% vs BAM with no loss. Always store the reference FASTA alongside CRAM files.
-
Use
-L bed_filefor targeted analyses: Restrictingsamtools viewto BED-defined target regions (WES capture, amplicons) dramatically reduces I/O for downstream steps.
Common Recipes
Recipe: Batch Flagstat for Multiple Samples
# Process all BAM files in directory
for bam in *.sorted.bam; do
echo "=== $bam ==="
samtools flagstat $bam | grep -E "mapped|properly paired|duplicates"
done
Recipe: Extract Unmapped Reads for De Novo Assembly
# Pull both unmapped reads (useful for pathogen detection)
samtools view -f 4 -b input.bam -o unmapped.bam
samtools fastq -@ 4 -1 unmapped_R1.fastq -2 unmapped_R2.fastq unmapped.bam
echo "Unmapped pairs ready for de novo assembly"
Recipe: Downsample BAM to Target Coverage
# Estimate current depth, then subsample to ~30×
TOTAL=$(samtools flagstat input.bam | grep "mapped (" | head -1 | awk '{print $1}')
GENOME_SIZE=3100000000 # hg38
READ_LEN=150
CURRENT_COV=$(echo "scale=1; $TOTAL * $READ_LEN / $GENOME_SIZE" | bc)
TARGET_FRAC=$(echo "scale=3; 30 / $CURRENT_COV" | bc)
echo "Current: ${CURRENT_COV}×; subsample fraction: $TARGET_FRAC"
samtools view -b -s $TARGET_FRAC input.bam -o downsampled.bam
samtools index downsampled.bam
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
[bam_index_build2] fail to index | BAM not sorted by coordinate | Sort first: samtools sort -o sorted.bam input.bam |
BAI index too large for chromosome | Chromosome > 512 Mbp | Use CSI index: samtools index -c input.bam |
CRAM: reference not found | Missing or wrong reference FASTA | Set REF_PATH env var or use -T ref.fa |
| Duplicate marking incorrect | fixmate step skipped | Run full pipeline: collate → fixmate → sort → markdup |
flagstat shows 0% properly paired | Paired-end BAM missing mate info | Run samtools fixmate to populate mate coordinates |
| Very slow sorting | Low memory per thread | Increase -m 4G; reduce -@ if memory-limited |
| Region query returns nothing | BAM not indexed or wrong coords | Run samtools index; use 1-based coords: chr1:1000-2000 |
[E::hts_open_format] fail to open | File path wrong or BAM corrupt | Verify path; test with samtools quickcheck file.bam |
Related Skills
- deeptools-ngs-analysis — normalized bigWig coverage tracks and ChIP-seq visualization downstream of samtools
- pysam-genomic-files — Python API for BAM manipulation in custom scripts
- bedtools-genomic-intervals — genomic interval operations on BAM/BED files produced by samtools
References
- samtools documentation — official man pages and command reference
- GitHub: samtools/samtools — source, releases, issue tracker
- Danecek et al. (2021) "Twelve years of SAMtools and BCFtools" — GigaScience 10(2)
- SAM format specification — FLAG bits, CIGAR strings, optional tags
STAR — Spliced RNA-seq Aligner
Overview
STAR (Spliced Transcripts Alignment to a Reference) aligns RNA-seq reads to a genome in a splice-aware manner, identifying novel and annotated splice junctions in a single pass. It generates coordinate-sorted BAM files compatible with samtools, IGV, deeptools, and GATK. STAR's 2-pass mode re-aligns reads using junctions discovered in the first pass, improving sensitivity for novel splice sites. With --quantMode GeneCounts, STAR simultaneously produces gene-level read count tables without requiring a separate featureCounts or HTSeq step.
When to Use
- Aligning bulk RNA-seq reads to a reference genome when downstream tools require a BAM file (variant calling, visualization, deeptools)
- Running ENCODE-compliant RNA-seq pipelines that mandate genome alignment
- Discovering novel splice junctions and alternative splicing events in the dataset
- Generating gene count tables alongside BAM alignment in a single step with
--quantMode GeneCounts - Processing long reads or reads with high mismatch rates by tuning
--outFilterMismatchNmax - Use Salmon instead when you only need transcript/gene quantification and do not need a BAM file — Salmon is 20-50× faster
Prerequisites
- Software: STAR ≥ 2.7.0 (conda or compiled binary)
- Reference files: genome FASTA + GTF annotation (same assembly)
- RAM: 30–32 GB for human/mouse genome index; 8–16 GB for smaller genomes
- Disk: ~25 GB for human genome index, ~5–10 GB per sample BAM
Check before installing: The tool may already be available in the current environment (e.g., inside a
pixi/condaenv). Runcommand -v STARfirst and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool viapixi run STARrather than bareSTAR.
# Install with conda (recommended)
conda install -c bioconda star
# Verify
STAR --version
# STAR_2.7.11a
# Or compile from source
git clone https://github.com/alexdobin/STAR
cd STAR/source && make STAR
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: reference_bundle
kind: required
source: user
ask: "Which species, genome assembly, and matching GTF annotation release should be used?"
default: null
- id: D2
param: --sjdbOverhang
kind: derived
source: data
ask: "What is the longest trimmed read length observed in the FASTQ files?"
default: "maximum trimmed read length minus 1"
- id: D3
param: --quantMode
kind: derived
source: upstream
ask: "Does the downstream workflow require STAR gene counts, transcriptome BAM for RSEM, or only the genomic BAM?"
default: "no quantMode; emit a coordinate-sorted genomic BAM"
- id: D4
param: --twopassMode
kind: optional
source: user
ask: "Should alignment prioritize sensitive discovery of novel splice junctions?"
default: "None"
- id: D5
param: --outFilterMultimapNmax
kind: optional
source: user
ask: "How many genomic loci may a read match before it is treated as too ambiguous to align?"
default: 10
- id: D6
param: --outFilterMismatchNmax
kind: optional
source: user
ask: "Does this assay require a non-default maximum number of alignment mismatches?"
default: 10
- id: D7
param: --alignIntronMax
kind: optional_conditional
source: literature
ask: "Does the organism or assay require an intron-length limit different from the reference bundle's standard setting?"
default: 1000000
- id: D8
param: --genomeSAindexNbases
kind: never_ask
source: data
reason: "Calculated from genome length to size the index; it affects memory use, not the biological result."
default: "min(14, floor(log2(genome_length) / 2 - 1))"
- id: D9
param: --runThreadN
kind: never_ask
source: data
reason: "Affects runtime only, not the alignments"
default: "min(8, available_cores)"
D2 is measured from the input FASTQ rather than chosen by the user. The GTF,
genome FASTA, and STAR index must all come from the same reference release.
GeneCounts produces STAR's own counts; omit it when featureCounts will produce
the count matrix. TranscriptomeSAM is for transcriptome-BAM consumers such as
RSEM, not ordinary Salmon quantification.
Quick Start
# 1. Generate genome index (~30 min, run once)
STAR --runMode genomeGenerate \
--runThreadN 8 \
--genomeDir genome/star_index \
--genomeFastaFiles genome/GRCh38.fa \
--sjdbGTFfile genome/gencode.v47.gtf \
--sjdbOverhang 100 # ReadLength - 1
# 2. Align paired-end reads (~10-20 min)
STAR --runThreadN 8 \
--genomeDir genome/star_index \
--readFilesIn sample_R1.fastq.gz sample_R2.fastq.gz \
--readFilesCommand zcat \
--outSAMtype BAM SortedByCoordinate \
--outFileNamePrefix results/sample/
# 3. Index the BAM
samtools index results/sample/Aligned.sortedByCoord.out.bam
Workflow
Step 1: Prepare Reference Files
Download a genome FASTA and matching GTF annotation (same assembly version).
# Download GRCh38 genome and GENCODE annotation
wget https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_47/GRCh38.primary_assembly.genome.fa.gz
wget https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_47/gencode.v47.primary_assembly.annotation.gtf.gz
gunzip GRCh38.primary_assembly.genome.fa.gz gencode.v47.primary_assembly.annotation.gtf.gz
mkdir -p genome/star_index
echo "Genome and GTF ready."
ls -lh GRCh38.primary_assembly.genome.fa gencode.v47.primary_assembly.annotation.gtf
Step 2: Generate Genome Index
Build the STAR genome index — required once per genome/read-length combination.
# Standard human genome index (requires ~32 GB RAM)
STAR --runMode genomeGenerate \
--runThreadN 16 \
--genomeDir genome/star_index/ \
--genomeFastaFiles GRCh38.primary_assembly.genome.fa \
--sjdbGTFfile gencode.v47.primary_assembly.annotation.gtf \
--sjdbOverhang 100
# For small genomes (e.g., E. coli ~4.6 Mb), reduce genomeSAindexNbases
# STAR --runMode genomeGenerate \
# --genomeSAindexNbases 11 \
# --genomeDir genome/ecoli_index/ ...
echo "Index complete: $(ls genome/star_index/ | wc -l) files"
Step 3: Align RNA-seq Reads
Align single-end or paired-end FASTQ files to the indexed genome.
# Single-end alignment
STAR --runThreadN 8 \
--genomeDir genome/star_index/ \
--readFilesIn sample1.fastq.gz \
--readFilesCommand zcat \
--outSAMtype BAM SortedByCoordinate \
--outSAMattributes NH HI AS NM MD \
--outFileNamePrefix results/sample1/
# Paired-end alignment
STAR --runThreadN 8 \
--genomeDir genome/star_index/ \
--readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
--readFilesCommand zcat \
--outSAMtype BAM SortedByCoordinate \
--outSAMattributes NH HI AS NM MD \
--outFileNamePrefix results/sample1/
echo "BAM: results/sample1/Aligned.sortedByCoord.out.bam"
Step 4: Run 2-Pass Alignment for Improved Sensitivity
Two-pass mode collects splice junctions from the first pass and uses them as annotation for the second pass.
# First pass — collect splice junctions
STAR --runThreadN 8 \
--genomeDir genome/star_index/ \
--readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
--readFilesCommand zcat \
--outSAMtype None \
--outFileNamePrefix pass1/sample1/
# Second pass — realign with all junctions from pass 1
SJ_FILES=$(ls pass1/*/SJ.out.tab | tr '\n' ' ')
STAR --runThreadN 8 \
--genomeDir genome/star_index/ \
--readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
--readFilesCommand zcat \
--sjdbFileChrStartEnd $SJ_FILES \
--outSAMtype BAM SortedByCoordinate \
--outFileNamePrefix results/sample1/
# Alternative: single-command 2-pass
STAR --runThreadN 8 \
--genomeDir genome/star_index/ \
--readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
--readFilesCommand zcat \
--twopassMode Basic \
--outSAMtype BAM SortedByCoordinate \
--outFileNamePrefix results/sample1/
Step 5: Check Alignment Statistics
Parse the alignment log to assess mapping rate and read quality.
# View the alignment summary
cat results/sample1/Log.final.out
# Parse key metrics with python
python3 - << 'EOF'
import re, sys
from pathlib import Path
log = Path("results/sample1/Log.final.out").read_text()
metrics = {}
for line in log.splitlines():
if "|" in line:
key, _, val = line.partition("|")
metrics[key.strip()] = val.strip()
print(f"Unique mapping: {metrics.get('Uniquely mapped reads %', 'N/A')}")
print(f"Multi-mapping: {metrics.get('% of reads mapped to multiple loci', 'N/A')}")
print(f"Too many mismatches:{metrics.get('% of reads unmapped: too many mismatches', 'N/A')}")
print(f"Total input reads: {metrics.get('Number of input reads', 'N/A')}")
EOF
Step 6: Generate Gene Count Tables
Enable simultaneous gene counting during alignment using --quantMode GeneCounts.
# Align and count simultaneously
STAR --runThreadN 8 \
--genomeDir genome/star_index/ \
--readFilesIn sample1_R1.fastq.gz sample1_R2.fastq.gz \
--readFilesCommand zcat \
--outSAMtype BAM SortedByCoordinate \
--quantMode GeneCounts \
--outFileNamePrefix results/sample1/
# ReadsPerGene.out.tab has 4 columns:
# gene_id unstranded stranded_fwd stranded_rev
head results/sample1/ReadsPerGene.out.tab
# Load into pandas (select column based on library strandedness)
python3 - << 'EOF'
import pandas as pd
df = pd.read_csv("results/sample1/ReadsPerGene.out.tab",
sep="\t", header=None, skiprows=4,
names=["gene_id", "unstranded", "fwd", "rev"])
# For unstranded library: use column 2 (unstranded)
counts = df.set_index("gene_id")["unstranded"]
print(f"Genes with counts > 0: {(counts > 0).sum()}")
print(counts[counts > 0].sort_values(ascending=False).head())
EOF
Key Parameters
| Parameter | Default | Range/Options | Effect |
|---|---|---|---|
--runThreadN | 1 | 1–64 | CPU threads for alignment |
--sjdbOverhang | 99 | ReadLength-1 | Splice junction overhang; set to ReadLength-1 |
--outSAMtype | SAM | BAM SortedByCoordinate, BAM Unsorted | Output format and sort order |
--outFilterMismatchNmax | 10 | 0–33 | Max mismatches per read; lower for stricter mapping |
--outFilterMultimapNmax | 10 | 1–9999 | Max genomic loci per read; reads exceeding limit marked unmapped |
--quantMode | – | GeneCounts, TranscriptomeSAM | Enable gene counting or transcriptome BAM |
--twopassMode | None | None, Basic | Enable 2-pass alignment for novel junction discovery |
--alignIntronMax | 1000000 | 1–1e9 | Maximum intron length; reduce for bacterial genomes |
--outReadsUnmapped | None | Fastx | Write unmapped reads to FASTQ |
--genomeSAindexNbases | 14 | 10–14 | SA index size; set log2(GenomeSize)/2 − 1 for small genomes |
Common Recipes
Recipe 1: Batch Align All Samples
#!/bin/bash
# Align all paired-end samples in a directory
SAMPLES=(ctrl_1 ctrl_2 treat_1 treat_2)
INDEX="genome/star_index"
DATA="data"
OUT="results"
THREADS=12
mkdir -p "$OUT"
for sample in "${SAMPLES[@]}"; do
echo "Aligning: $sample"
mkdir -p "$OUT/$sample"
STAR --runThreadN "$THREADS" \
--genomeDir "$INDEX" \
--readFilesIn "$DATA/${sample}_R1.fastq.gz" "$DATA/${sample}_R2.fastq.gz" \
--readFilesCommand zcat \
--outSAMtype BAM SortedByCoordinate \
--quantMode GeneCounts \
--twopassMode Basic \
--outFileNamePrefix "$OUT/$sample/"
samtools index "$OUT/$sample/Aligned.sortedByCoord.out.bam"
echo "Done: $sample — $(grep 'Uniquely mapped reads %' $OUT/$sample/Log.final.out | awk '{print $NF}')"
done
Recipe 2: Build Gene Count Matrix Across Samples
import pandas as pd
from pathlib import Path
results_dir = Path("results")
samples = ["ctrl_1", "ctrl_2", "treat_1", "treat_2"]
strandedness = "unstranded" # or "fwd" / "rev"
col_map = {"unstranded": 1, "fwd": 2, "rev": 3}
col = col_map[strandedness]
counts = {}
for sample in samples:
count_file = results_dir / sample / "ReadsPerGene.out.tab"
df = pd.read_csv(count_file, sep="\t", header=None, skiprows=4)
counts[sample] = df.set_index(0)[col]
matrix = pd.DataFrame(counts)
matrix = matrix[matrix.sum(axis=1) > 0] # drop zero-count genes
matrix.to_csv("gene_count_matrix.tsv", sep="\t")
print(f"Count matrix: {matrix.shape} (genes × samples)")
print(matrix.head())
Recipe 3: Integrate with DESeq2 via pydeseq2
import pandas as pd
from pydeseq2.dds import DeseqDataSet
from pydeseq2.default_inference import DefaultInference
# Load count matrix from STAR output
counts = pd.read_csv("gene_count_matrix.tsv", sep="\t", index_col=0).T
metadata = pd.DataFrame({
"condition": ["control", "control", "treated", "treated"]
}, index=counts.index)
# Run DESeq2
dds = DeseqDataSet(counts=counts, metadata=metadata,
design_factors="condition",
inference=DefaultInference(n_cpus=4))
dds.deseq2()
print("DESeq2 complete — see dds.varm['LFC'] for results")
Expected Outputs
| Output | Format | Description |
|---|---|---|
Aligned.sortedByCoord.out.bam | BAM | Coordinate-sorted aligned reads; index with samtools index |
SJ.out.tab | TSV | Splice junction table with coverage, motif, and novelty flags |
Log.final.out | Text | Alignment statistics: unique mapping %, multimappers %, etc. |
ReadsPerGene.out.tab | TSV | Gene counts (4 columns: unstranded/fwd/rev) when --quantMode GeneCounts |
Unmapped.out.mate1/2 | FASTQ | Unmapped reads (when --outReadsUnmapped Fastx) |
Log.out | Text | Verbose run log; check for warnings and parameter echoes |
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| Unique mapping < 60% | Wrong genome assembly or species contamination | Verify genome FASTA matches sample species; run FastQC to check overrepresented sequences |
Fatal error: genome files not found | Wrong --genomeDir path or incomplete index | Re-run genomeGenerate; check genomeDir contains Genome, SA, SAindex files |
| Out of memory during genome generation | Not enough RAM for genome SA index | Add --genomeSAindexNbases 13 (or lower) for small genomes; request ≥32 GB RAM for human |
.gz files not decompressed | Missing --readFilesCommand zcat | Add --readFilesCommand zcat for gzip-compressed inputs |
Error: number of input files differ | R1/R2 read count mismatch | Verify FASTQ files with `zcat file.fastq.gz |
ReadsPerGene.out.tab missing | --quantMode GeneCounts not set | Re-run with --quantMode GeneCounts or use featureCounts on BAM |
| Very high multimapping (>20%) | Highly repetitive genome or wrong --outFilterMultimapNmax | Reduce --outFilterMultimapNmax; use --outSAMmultNmax 1 to output only one alignment per read |
| Genome index takes too long | Large genome + slow disk | Use SSD storage; pre-built indices available from ENCODE and Ensembl |
References
- STAR GitHub repository — source code, releases, and STAR manual PDF
- Dobin A et al. (2013) "STAR: ultrafast universal RNA-seq aligner" — Bioinformatics 29(1):15-21. DOI:10.1093/bioinformatics/bts635
- ENCODE RNA-seq alignment standards — ENCODE project guidelines using STAR
- GENCODE genome annotations — recommended GTF source for human/mouse STAR indices
Bakta Genome Annotation
Overview
Bakta is a command-line pipeline for rapid, standardized annotation of bacterial and archaeal genomes and plasmids. It combines Prodigal for CDS prediction, tRNAscan-SE/Aragorn/Barrnap/Infernal for non-coding RNA, PILER-CR/PILERCR for CRISPR detection, and a tiered DIAMOND/HMM search against a curated UniRef100 + IPS/UPS database to assign gene names, EC numbers, GO terms, and COG categories. Bakta produces NCBI-compatible outputs (GFF3, GenBank, EMBL, INSDC-formatted FASTA, plus a JSON summary and a circular Circos plot) for a typical 5 Mb genome in 5–15 minutes on 8 CPUs.
When to Use
- Annotating bacterial or archaeal genome assemblies (Illumina, PacBio, Nanopore) with NCBI-compatible locus tags and product names
- Annotating plasmids and other circular replicons separately with
--plasmidand--completeflags - Producing JSON-structured annotation outputs that can be parsed without GenBank or GFF3 detours
- Generating a publication-ready circular genome plot via the bundled
bakta_plotcommand - Annotating MAGs (metagenome-assembled genomes) with
--metato disable Prodigal training - Use Prokka instead when you need viral/mitochondrial kingdoms or when you must reproduce a legacy Prokka pipeline exactly
- Use PGAP instead when submitting to NCBI GenBank with full standards compliance
- Use Bakta when you want faster runs, regularly updated UniRef-derived databases, AMRFinderPlus integration, and a JSON summary out of the box
Prerequisites
- Software: Bakta ≥ 1.9, Python 3.8+, Prodigal, tRNAscan-SE, Aragorn, Barrnap, Infernal, DIAMOND, HMMER3, PILER-CR, BLAST+, AMRFinderPlus
- Database: Bakta DB (full ~70 GB, or light ~3 GB) downloaded once with
bakta_db download - Python packages (for output parsing):
biopython,pandas,matplotlib - Input: assembled genome in FASTA format (one or more contigs)
- Hardware: ≥ 16 GB RAM for full DB, ≥ 4 GB RAM for light DB; ≥ 8 CPUs recommended
Check before installing: The tool may already be available in the current environment (e.g., inside a
pixi/condaenv). Runcommand -v baktafirst and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool viapixi run baktarather than barebakta.
# Install Bakta via conda/mamba (recommended)
mamba install -c conda-forge -c bioconda bakta
# Verify installation
bakta --version
# bakta 1.9.4
# Download the light database (~3 GB, faster, fewer functional hits)
bakta_db download --output db/ --type light
# Or full database (~70 GB, comprehensive UniRef100 coverage)
# bakta_db download --output db/ --type full
# Install Python parsing dependencies
pip install biopython pandas matplotlib
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: database
kind: required
source: user
ask: "Annotate against the light database or the full one? The light database leaves more genes described only generically."
default: null
- id: D2
param: organismMetadata
kind: required
source: user
ask: "Which genus, species, and strain should be recorded in the output?"
default: null
- id: D3
param: repliconTopology
kind: required
source: data
ask: "Are these contigs complete circular replicons, plasmids, or draft fragments?"
default: "draft fragments - no circularity assumed"
- id: D4
param: assemblyOrigin
kind: required
source: data
ask: "Is this an isolate genome, or a bin recovered from a metagenome?"
default: "isolate - gene prediction trains on the assembly itself"
- id: D5
param: translationTable
kind: required
source: user
ask: "Which genetic code does this organism use?"
default: "11, the bacterial and archaeal code"
- id: D6
param: locusTagPrefix
kind: optional
source: user
ask: "Which prefix should locus tags carry, if they will be submitted or cross-referenced?"
default: "generated automatically"
- id: D7
param: minContigLength
kind: optional_conditional
source: data
ask: "Should short contigs be skipped?"
default: "no minimum"
- id: D8
param: threadsAndPlotting
kind: never_ask
source: data
reason: "Thread count and the circular plot affect runtime and output files, not the annotation"
default: "available cores, plot enabled"
D5 is asked rather than defaulted because the exceptions are real and silent: Mycoplasma and relatives reassign a stop codon, so annotating them with the standard bacterial code truncates a large fraction of their genes into fragments that still look like ordinary short CDS calls.
Quick Start
# Annotate a bacterial genome — results in results/ directory
bakta genome.fasta \
--db db/bakta_db_light \
--output results/ \
--prefix sample1 \
--threads 8
# Inspect the JSON summary for feature counts
python -c "
import json
with open('results/sample1.json') as f:
d = json.load(f)
print('Genus:', d['genome'].get('genus'))
print('Length:', d['genome']['size'], 'bp')
print('CDS:', sum(1 for f in d['features'] if f['type'] == 'cds'))
print('tRNA:', sum(1 for f in d['features'] if f['type'] == 'tRNA'))
"
Workflow
Step 1: Install Bakta and Download the Database
Install Bakta and prepare the reference database. The database download is one-time and reused across runs.
# Create a dedicated conda environment (avoids dependency conflicts)
mamba create -n bakta_env -c conda-forge -c bioconda bakta python=3.11 -y
mamba activate bakta_env
# Verify Bakta and its dependencies
bakta --version
# bakta 1.9.4
bakta --help | head -20
# Download the light database (sufficient for routine annotation)
mkdir -p db/
bakta_db download --output db/ --type light
# Downloads ~3 GB; expands to ~5 GB on disk
# Verify the database was extracted correctly
ls db/bakta_db_light/
# antifam.h3f bakta.db expert oric.fna pfam.h3f rfam-go.tsv ...
# (Optional) Update AMRFinderPlus DB used by Bakta for AMR gene calling
amrfinder -u
# Install Python parsing tools
pip install biopython pandas matplotlib
Step 2: Prepare the Input Assembly
Bakta requires clean FASTA headers without spaces or special characters. Pre-clean and optionally filter short contigs.
from Bio import SeqIO
import re
input_fasta = "genome.fasta"
records = list(SeqIO.parse(input_fasta, "fasta"))
print(f"Input assembly: {len(records)} contigs")
total_bases = sum(len(r) for r in records)
print(f"Total bases: {total_bases:,}")
print(f"Largest contig: {max(len(r) for r in records):,} bp")
# Bakta preferred: short, alphanumeric, unique IDs
cleaned = []
for i, rec in enumerate(records, 1):
new_id = f"contig_{i:04d}"
new_rec = rec.__class__(rec.seq, id=new_id, description="")
cleaned.append(new_rec)
SeqIO.write(cleaned, "genome_clean.fasta", "fasta")
print(f"Wrote genome_clean.fasta with {len(cleaned)} contigs")
# Filter out short contigs (<200 bp) which contribute little to annotation
awk 'BEGIN{RS=">"; ORS=""} NR>1 {n=split($0, a, "\n"); seq=""; for(i=2;i<=n;i++) seq=seq a[i]; if (length(seq) >= 200) print ">" $0}' \
genome_clean.fasta > genome_filtered.fasta
echo "Filtered assembly: $(grep -c '>' genome_filtered.fasta) contigs"
Step 3: Run Standard Bakta Annotation
Run Bakta with genus/species hints. Locus tags are auto-generated from the strain field.
# Standard annotation for a draft bacterial genome
bakta genome_clean.fasta \
--db db/bakta_db_light \
--output annotation/ \
--prefix E_coli_K12 \
--genus Escherichia \
--species coli \
--strain K12 \
--locus-tag ECOLI \
--threads 8 \
--keep-contig-headers
# Expected runtime: 5–15 min for ~5 Mb genome on 8 CPUs (light DB)
echo "Bakta annotation outputs:"
ls annotation/
# E_coli_K12.embl E_coli_K12.faa E_coli_K12.ffn
# E_coli_K12.fna E_coli_K12.gbff E_coli_K12.gff3
# E_coli_K12.hypotheticals.faa E_coli_K12.hypotheticals.tsv
# E_coli_K12.json E_coli_K12.log E_coli_K12.png
# E_coli_K12.svg E_coli_K12.tsv E_coli_K12.txt
Step 4: Parse the JSON Summary
Bakta's JSON output is the canonical, machine-readable annotation. Parse it directly for downstream pipelines.
import json
import pandas as pd
from collections import Counter
with open("annotation/E_coli_K12.json") as f:
bakta = json.load(f)
# Genome-level metadata
genome = bakta["genome"]
print(f"Organism: {genome.get('genus')} {genome.get('species')} {genome.get('strain')}")
print(f"Size: {genome['size']:,} bp across {len(bakta['sequences'])} sequences")
print(f"GC content: {genome['gc']:.2%}")
# Feature type counts
features = bakta["features"]
type_counts = Counter(f["type"] for f in features)
print("\nFeature counts:")
for ftype, n in sorted(type_counts.items(), key=lambda x: -x[1]):
print(f" {ftype:>10}: {n}")
# Build a tidy CDS DataFrame
cds_rows = []
for f in features:
if f["type"] != "cds":
continue
cds_rows.append({
"locus_tag": f.get("locus", ""),
"contig": f.get("contig", ""),
"start": f.get("start"),
"stop": f.get("stop"),
"strand": f.get("strand"),
"gene": f.get("gene", ""),
"product": f.get("product", ""),
"length_aa": len(f.get("aa", "")),
})
cds_df = pd.DataFrame(cds_rows)
print(f"\nTotal CDS: {len(cds_df)}")
print(cds_df.head(5).to_string(index=False))
Step 5: Parse the TSV Feature Table
The TSV output is convenient for spreadsheet workflows and quick filtering.
import pandas as pd
# Bakta TSV begins with comment lines starting with '#'
df = pd.read_csv("annotation/E_coli_K12.tsv", sep="\t", comment="#",
names=["sequence_id", "type", "start", "stop", "strand",
"locus_tag", "gene", "product", "dbxrefs"])
print(f"Total features: {len(df)}")
print(f"Feature types: {df['type'].value_counts().to_dict()}")
# Hypothetical vs annotated CDS
cds = df[df["type"] == "cds"].copy()
hypothetical = cds["product"].str.contains("hypothetical", case=False, na=True)
print(f"\nCDS with assigned function: {(~hypothetical).sum()} / {len(cds)}")
print(f"Hypothetical proteins: {hypothetical.sum()}")
# Cross-references (UniRef, KEGG, EC, GO, etc.) parsed from the dbxrefs column
def split_xrefs(xref_str):
if not isinstance(xref_str, str) or xref_str in ("", "-"):
return []
return [x.strip() for x in xref_str.split(",")]
cds["dbxref_list"] = cds["dbxrefs"].apply(split_xrefs)
ec_hits = cds[cds["dbxref_list"].apply(lambda xs: any(x.startswith("EC:") for x in xs))]
print(f"CDS with EC numbers: {len(ec_hits)}")
print(ec_hits[["locus_tag", "gene", "product"]].head(5).to_string(index=False))
Step 6: Render the Circular Genome Plot
Bakta emits a Circos-style PNG/SVG by default. Regenerate with custom styling using bakta_plot.
# Re-render the plot from the existing JSON with a different style
bakta_plot --output annotation/ \
--prefix E_coli_K12_replot \
--type cog \
--dpi 300 \
annotation/E_coli_K12.json
ls annotation/E_coli_K12_replot.*
# E_coli_K12_replot.png E_coli_K12_replot.svg
# Display the rendered PNG inline in a Jupyter notebook
python <<'PY'
from pathlib import Path
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
img = mpimg.imread("annotation/E_coli_K12.png")
fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(img)
ax.axis("off")
ax.set_title("Bakta circular genome annotation")
plt.savefig("bakta_circular_thumbnail.png", dpi=120, bbox_inches="tight")
print("Saved bakta_circular_thumbnail.png")
PY
Step 7: Compute Annotation Quality Statistics
Summarize hypothetical-protein rate, gene density, and feature coverage to assess annotation quality.
import json
import pandas as pd
import matplotlib.pyplot as plt
with open("annotation/E_coli_K12.json") as f:
bakta = json.load(f)
genome_size = bakta["genome"]["size"]
features = bakta["features"]
# Per-feature-type counts and density (per Mb)
counts = {}
for f in features:
counts[f["type"]] = counts.get(f["type"], 0) + 1
density = {k: v / (genome_size / 1e6) for k, v in counts.items()}
cds = [f for f in features if f["type"] == "cds"]
n_cds = len(cds)
n_hypo = sum(1 for f in cds if "hypothetical" in (f.get("product") or "").lower())
n_known = n_cds - n_hypo
coding_density = sum(abs(f["stop"] - f["start"] + 1) for f in cds) / genome_size
print(f"Genome: {genome_size:,} bp")
print(f"CDS: {n_cds} ({density.get('cds', 0):.1f} per Mb)")
print(f" Known function: {n_known} Hypothetical: {n_hypo}")
print(f"Coding density: {coding_density:.1%}")
print(f"tRNA: {counts.get('tRNA', 0)} rRNA: {counts.get('rRNA', 0)}")
print(f"ncRNA: {counts.get('ncRNA', 0)} CRISPR: {counts.get('crispr', 0)}")
# Bar plot of feature type counts
fig, ax = plt.subplots(figsize=(8, 4))
items = sorted(counts.items(), key=lambda x: -x[1])
labels = [k for k, _ in items]
values = [v for _, v in items]
bars = ax.bar(labels, values, color="#2980B9", edgecolor="white")
for bar, v in zip(bars, values):
ax.text(bar.get_x() + bar.get_width() / 2, v + max(values) * 0.01,
str(v), ha="center", fontsize=9)
ax.set_ylabel("Count")
ax.set_title("Bakta feature counts by type")
plt.xticks(rotation=20)
plt.tight_layout()
plt.savefig("bakta_feature_counts.png", dpi=150, bbox_inches="tight")
print("Saved bakta_feature_counts.png")
Step 8: Batch Annotation Across Multiple Genomes
Run Bakta over a directory of assemblies and aggregate per-sample summary statistics.
#!/bin/bash
# batch_bakta.sh — annotate all FASTA files in a directory
INPUT_DIR="genomes/"
OUTPUT_DIR="annotations/"
DB="db/bakta_db_light"
mkdir -p "$OUTPUT_DIR"
for FASTA in "$INPUT_DIR"/*.fasta; do
SAMPLE=$(basename "$FASTA" .fasta)
echo "Annotating: $SAMPLE"
bakta "$FASTA" \
--db "$DB" \
--output "${OUTPUT_DIR}/${SAMPLE}" \
--prefix "$SAMPLE" \
--threads 4 \
--skip-plot \
--force
done
echo "Batch annotation complete."
# Aggregate per-genome summaries from each sample's JSON file
from pathlib import Path
import json
import pandas as pd
annotation_dir = Path("annotations/")
rows = []
for json_file in sorted(annotation_dir.glob("*/*.json")):
with open(json_file) as f:
bakta = json.load(f)
sample = json_file.stem
counts = {}
for feat in bakta["features"]:
counts[feat["type"]] = counts.get(feat["type"], 0) + 1
rows.append({
"sample": sample,
"size": bakta["genome"]["size"],
"gc": bakta["genome"]["gc"],
"CDS": counts.get("cds", 0),
"tRNA": counts.get("tRNA", 0),
"rRNA": counts.get("rRNA", 0),
"ncRNA": counts.get("ncRNA", 0),
"crispr": counts.get("crispr", 0),
})
summary_df = pd.DataFrame(rows)
print(f"Annotated genomes: {len(summary_df)}")
print(summary_df.to_string(index=False))
summary_df.to_csv("batch_bakta_summary.csv", index=False)
print("Saved: batch_bakta_summary.csv")
Key Parameters
| Parameter | Default | Range / Options | Effect |
|---|---|---|---|
--db | — | path to Bakta DB directory | Required; selects light or full reference DB |
--genus | — | any genus name string | Sets organism metadata in GenBank/EMBL output |
--species | — | any species name string | Combined with --genus for organism qualifier |
--strain | — | any string | Adds strain qualifier to organism metadata |
--locus-tag | auto | 3–24 alphanumeric chars | Prefix for locus tags (e.g., ECOLI_00001) |
--complete | off | flag | Treat all input contigs as complete circular replicons |
--plasmid | off | flag | Treat all input contigs as plasmids (circular) |
--meta | off | flag | Disable Prodigal training (use for MAGs / metagenomic bins) |
--translation-table | 11 | NCBI table IDs (1, 4, 11, 25, …) | Genetic code used by Prodigal for CDS translation |
--min-contig-length | 1 | any integer (bp) | Skip contigs shorter than this length |
--threads | 1 | 1–CPU count | Parallel threads for DIAMOND/HMMER searches |
--skip-plot | off | flag | Skip the slow circular plot rendering step |
--keep-contig-headers | off | flag | Preserve original contig IDs instead of renaming |
--proteins | — | path to GenBank or FASTA file | Custom expert protein DB used before UniRef search |
Common Recipes
Recipe: Plasmid-only Annotation
When to use: A finished circular plasmid sequence that should be annotated without chromosome assumptions.
bakta plasmid.fasta \
--db db/bakta_db_light \
--output plasmid_annotation/ \
--prefix pBR322 \
--plasmid \
--complete \
--threads 4
# Inspect plasmid-typing results (incompatibility group, replication initiator)
grep -E "rep|inc" plasmid_annotation/pBR322.tsv | head -10
Recipe: MAG (Metagenome-Assembled Genome) Annotation
When to use: Annotating a bin from metagenomic assembly where Prodigal cannot train on the full sequence.
bakta MAG_bin_42.fasta \
--db db/bakta_db_light \
--output mag_annotation/ \
--prefix MAG_bin_42 \
--meta \
--min-contig-length 500 \
--skip-plot \
--threads 8
cat mag_annotation/MAG_bin_42.txt | head -20
Recipe: Annotation with Custom Expert Protein Database
When to use: You have curated reference proteins (e.g., a virulence factor catalog) that should take priority over UniRef.
# Custom proteins are searched before the bundled UniRef DB
bakta genome.fasta \
--db db/bakta_db_light \
--proteins virulence_factors.faa \
--output annotation_custom/ \
--prefix novel_strain \
--threads 8
echo "Annotations referencing custom DB:"
grep "User-provided" annotation_custom/novel_strain.tsv | head
Recipe: Convert Bakta GFF3 to a Pandas DataFrame
When to use: You need feature coordinates and qualifiers for downstream coordinate-overlap analyses.
import pandas as pd
def parse_gff3(gff_file):
rows = []
with open(gff_file) as f:
for line in f:
if line.startswith("##FASTA"):
break
if line.startswith("#") or not line.strip():
continue
parts = line.rstrip("\n").split("\t")
if len(parts) < 9:
continue
attrs = {}
for item in parts[8].split(";"):
if "=" in item:
k, v = item.split("=", 1)
attrs[k] = v
rows.append({
"seqname": parts[0],
"feature": parts[2],
"start": int(parts[3]),
"end": int(parts[4]),
"strand": parts[6],
"locus_tag": attrs.get("locus_tag", attrs.get("ID", "")),
"gene": attrs.get("gene", ""),
"product": attrs.get("product", ""),
})
return pd.DataFrame(rows)
gff_df = parse_gff3("annotation/E_coli_K12.gff3")
print(f"Features parsed: {len(gff_df)}")
print(gff_df[gff_df["feature"] == "CDS"].head(5).to_string(index=False))
Expected Outputs
| Output File | Format | Description |
|---|---|---|
{prefix}.gff3 | GFF3 | Genome annotation with feature coordinates and attributes; INSDC-compliant |
{prefix}.gbff | GenBank Flat File | Annotated GenBank record for use in Geneious, BioPython, NCBI submission prep |
{prefix}.embl | EMBL | EMBL-format annotation for ENA submission |
{prefix}.fna | FASTA | Nucleotide sequences of the input contigs |
{prefix}.ffn | FASTA | Nucleotide sequences of all annotated features |
{prefix}.faa | FASTA | Protein sequences for all CDS features |
{prefix}.hypotheticals.faa | FASTA | Protein sequences flagged as hypothetical (for further investigation) |
{prefix}.hypotheticals.tsv | TSV | Detailed table for hypothetical CDS with low-confidence hits |
{prefix}.tsv | TSV | Full feature table: seqid, type, start, stop, strand, locus_tag, gene, product, dbxrefs |
{prefix}.json | JSON | Machine-readable summary with genome metadata + every feature with full qualifiers |
{prefix}.png / .svg | Image | Circos circular genome plot colored by COG category |
{prefix}.txt | Text | Plain-text summary of feature counts and per-replicon stats |
{prefix}.log | Text | Full Bakta run log including timing per step |
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
Error: database not found at <path> | DB path incorrect, or DB not extracted | Re-run bakta_db download --output db/ --type light; pass full path to --db |
KILLED during DIAMOND step | Out of memory on full DB | Switch to --type light, reduce --threads, or run on a host with ≥ 32 GB RAM |
| Very high hypothetical-protein rate (>60 %) | Divergent strain or light DB without close hits | Re-run with the full DB or supply a curated --proteins reference set |
tRNAscan-SE: Error opening file | Missing tRNAscan-SE installation in conda env | mamba install -c bioconda trnascan-se and rerun |
| Bakta complains about contig headers | Spaces or special characters in FASTA IDs | Pre-clean headers with awk '/^>/{print ">contig_"++i; next}{print}' |
| Plot generation hangs or fails | Circos / matplotlib backend issue | Re-run with --skip-plot; render later via bakta_plot from the JSON file |
AMRFinderPlus: database not up-to-date warning | AMRFinderPlus DB stale | amrfinder -u to refresh, or pass --skip-amr to bypass |
| Different locus tag prefix than expected | --locus-tag not set, default uses random prefix | Explicitly pass --locus-tag MYORG to control prefix |
| Run is much slower than reported | Default --threads 1 | Set --threads to physical core count; use --skip-plot for batch jobs |
References
- Bakta GitHub: oschwengers/bakta — source code, installation, parameter reference, and changelog
- Schwengers et al. (2021) Microb Genom 7(11):000685 — original Bakta publication with benchmarks vs. Prokka and PGAP
- Bakta database releases on Zenodo — versioned light/full database archives
- Prodigal documentation — gene prediction engine used by Bakta for CDS calling
- BioPython GenBank parsing tutorial — reference for parsing Bakta
.gbffoutput in Python - DIAMOND aligner — protein search engine used for the UniRef-derived database lookups
Prokka Genome Annotation
Overview
Prokka is a command-line pipeline for rapid annotation of prokaryotic genomes (bacteria, archaea, and viruses). It uses a tiered search strategy: protein-coding genes (CDS) are predicted with Prodigal and searched first against a genus-specific database, then RefSeq proteins, then Pfam/TIGRFAMs HMMs. Non-coding RNA genes (rRNA, tRNA, tmRNA) are identified with Barrnap, Aragorn, and Infernal. Prokka processes a single FASTA assembly in minutes and outputs a comprehensive annotation in GFF3, GenBank, FASTA, and tabular formats.
When to Use
- Annotating a newly assembled bacterial or archaeal genome from Illumina, PacBio, or Nanopore assemblies
- Getting functional protein annotations (CDS with product names, EC numbers, GO terms) from a draft or complete genome
- Preparing annotation files for downstream comparative genomics (Roary pan-genome, OrthoFinder)
- Annotating viral or phage genomes when kingdom-specific databases are important
- Performing metagenome-assembled genome (MAG) annotation with the
--metagenomeflag - Parsing annotated outputs in Python with BioPython for downstream sequence or feature analysis
- Use PGAP (NCBI Prokaryotic Genome Annotation Pipeline) instead when the goal is NCBI GenBank submission with standards compliance
- Use Bakta instead for faster annotation with built-in NCBI-compatible outputs and a more regularly updated database
Prerequisites
- Software: Prokka ≥ 1.14, Perl 5, Prodigal, Barrnap, HMMER3, BLAST+, Aragorn, Infernal, tbl2asn
- Python packages (for output parsing):
biopython,pandas,matplotlib - Input: assembled genome in FASTA format (complete or draft with multiple contigs)
- Environment: conda strongly recommended to handle the Perl and C dependency stack
Check before installing: The tool may already be available in the current environment (e.g., inside a
pixi/condaenv). Runcommand -v prokkafirst and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool viapixi run prokkarather than bareprokka.
# Install Prokka via conda/mamba (recommended)
conda install -c conda-forge -c bioconda prokka
# Or with mamba (faster)
mamba install -c conda-forge -c bioconda prokka
# Verify installation and database setup
prokka --version
# prokka 1.14.6
# Check that required tools are on PATH
prokka --depends
# prokka needs: awk, sed, grep, makeblastdb, blastp, hmmscan, ...
# Install Python parsing dependencies
pip install biopython pandas matplotlib
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: kingdom
kind: required
source: user
ask: "Is this a bacterial, archaeal, viral, or mitochondrial assembly?"
default: "Bacteria"
- id: D2
param: organismMetadata
kind: required
source: user
ask: "Which genus, species, and strain should be recorded, and used to prioritise a genus-specific protein database?"
default: null
- id: D3
param: assemblyOrigin
kind: required
source: data
ask: "Is this an isolate genome, or a bin recovered from a metagenome?"
default: "isolate - gene prediction trains on the assembly itself"
- id: D4
param: rnaPrediction
kind: optional
source: user
ask: "Should rRNA, tRNA, and other non-coding RNA be predicted alongside coding genes?"
default: "tRNA and rRNA predicted; Rfam search off as it is slow"
- id: D5
param: customProteins
kind: optional
source: user
ask: "Is there a curated protein set that should be searched before the general reference database?"
default: "none"
- id: D6
param: hitEvalue
kind: optional
source: user
ask: "How confident must a database hit be before it names a gene?"
default: "1e-6"
- id: D7
param: minContigLength
kind: optional_conditional
source: data
ask: "Should short contigs be skipped - and does the assembly have enough of them to matter?"
default: "no minimum"
- id: D8
param: cpus
kind: never_ask
source: data
reason: "Affects runtime only, not the annotation"
default: "min(8, available_cores)"
D3 changes how genes are predicted rather than how they are named. Prodigal trains its model on the input; a metagenomic bin holding more than one organism trains a blended model, and the resulting gene boundaries are wrong in a way that no downstream step flags.
Quick Start
# Annotate a bacterial genome assembly — results in results/ directory
prokka genome.fasta \
--outdir results/ \
--prefix sample1 \
--kingdom Bacteria \
--cpus 4
# Check output summary
cat results/sample1.txt
# Organism: Genus species strain
# Contigs: 1
# Bases: 4639675
# CDS: 4140
# rRNA: 22
# tRNA: 86
echo "Annotation complete. Key output files:"
ls results/sample1.{gff,gbk,faa,ffn,tsv}
Workflow
Step 1: Install and Verify Prokka
Install Prokka and confirm all dependent tools are accessible in the current environment.
# Create a dedicated conda environment
conda create -n prokka_env -c conda-forge -c bioconda prokka python=3.10 -y
conda activate prokka_env
# Verify Prokka version and all tool dependencies
prokka --version
# prokka 1.14.6
prokka --depends
# Checking that required tools are installed...
# OK: makeblastdb is installed (2.13.0+)
# OK: blastp is installed (2.13.0+)
# OK: hmmscan is installed (3.3.2)
# OK: prodigal is installed (2.6.3)
# OK: barrnap is installed (0.9)
# Check available genus-specific databases bundled with Prokka
ls $(conda info --base)/envs/prokka_env/db/genus/
# Archaea Bacteria Mitochondria Viruses
# Install Python parsing tools
pip install biopython pandas matplotlib
Step 2: Prepare the Input Genome
Clean and rename contigs to comply with Prokka's header requirements before annotation.
from Bio import SeqIO
import re
# Load and inspect assembly
input_fasta = "genome.fasta"
records = list(SeqIO.parse(input_fasta, "fasta"))
print(f"Input assembly: {len(records)} contigs")
total_bases = sum(len(r) for r in records)
print(f"Total bases: {total_bases:,}")
print(f"Largest contig: {max(len(r) for r in records):,} bp")
print(f"N50 approx: see assembly stats tool")
# Rename contigs to short IDs compatible with Prokka (max 37 chars)
# Prokka requires: no spaces, no special characters in header
cleaned = []
for i, rec in enumerate(records, 1):
new_id = f"contig_{i:04d}"
new_rec = rec.__class__(rec.seq, id=new_id, description=f"len={len(rec.seq)}")
cleaned.append(new_rec)
SeqIO.write(cleaned, "genome_clean.fasta", "fasta")
print(f"\nWrote genome_clean.fasta with {len(cleaned)} renamed contigs")
# genome_clean.fasta: contig_0001 through contig_NNNN
# Alternatively, clean headers with a simple bash one-liner
awk '/^>/{print ">contig_" ++i; next}{print}' genome.fasta > genome_clean.fasta
# Filter out short contigs (< 200 bp) to reduce annotation noise
awk '/^>/{header=$0; next} length($0) >= 200 {print header; print}' \
genome_clean.fasta > genome_filtered.fasta
echo "Filtered assembly ready: $(grep -c '>' genome_filtered.fasta) contigs"
Step 3: Run Basic Prokka Annotation
Run Prokka with standard options for a bacterial genome, specifying genus/species for database selection.
# Basic annotation with genus/species hint (uses genus-specific protein database first)
prokka genome_clean.fasta \
--outdir annotation/ \
--prefix E_coli_K12 \
--kingdom Bacteria \
--genus Escherichia \
--species coli \
--strain K12 \
--cpus 8 \
--mincontiglen 200
# Expected runtime: 2–10 minutes for a typical 4–6 Mb bacterial genome
echo "Prokka annotation output files:"
ls annotation/
# E_coli_K12.err E_coli_K12.faa E_coli_K12.ffn
# E_coli_K12.fna E_coli_K12.gbk E_coli_K12.gff
# E_coli_K12.log E_coli_K12.sqn E_coli_K12.tbl
# E_coli_K12.tsv E_coli_K12.txt
Step 4: Parse Annotation Summary (TSV)
Load the TSV output for a quick overview of annotated features and their functional assignments.
import pandas as pd
# Load the annotation TSV (tab-delimited feature table)
tsv_file = "annotation/E_coli_K12.tsv"
df = pd.read_csv(tsv_file, sep="\t")
print(f"Total features: {len(df)}")
print(f"Columns: {list(df.columns)}")
# Columns: [locus_tag, ftype, length_bp, gene, EC_number, COG, product]
# Feature type summary
print("\nFeature type counts:")
print(df["ftype"].value_counts().to_string())
# CDS 4140
# tRNA 86
# rRNA 22
# tmRNA 1
# Functional gene annotations (non-hypothetical CDS)
cds_df = df[df["ftype"] == "CDS"].copy()
hypothetical = cds_df["product"].str.contains("hypothetical", case=False, na=True)
print(f"\nCDS with known function: {(~hypothetical).sum()}")
print(f"Hypothetical proteins: {hypothetical.sum()}")
# Genes with EC numbers (enzymes)
ec_annotated = cds_df[cds_df["EC_number"].notna() & (cds_df["EC_number"] != "")]
print(f"CDS with EC numbers: {len(ec_annotated)}")
print(ec_annotated[["locus_tag", "gene", "EC_number", "product"]].head(5).to_string(index=False))
Step 5: Parse GenBank Output with BioPython
Read the GenBank file to access per-gene sequences, qualifiers, and feature coordinates.
from Bio import SeqIO
import pandas as pd
# Parse GenBank file
gbk_file = "annotation/E_coli_K12.gbk"
records = list(SeqIO.parse(gbk_file, "genbank"))
print(f"Contigs in GenBank: {len(records)}")
# Iterate over CDS features and extract details
rows = []
for rec in records:
for feat in rec.features:
if feat.type != "CDS":
continue
qualifiers = feat.qualifiers
rows.append({
"contig": rec.id,
"locus_tag": qualifiers.get("locus_tag", ["?"])[0],
"gene": qualifiers.get("gene", [""])[0],
"product": qualifiers.get("product", ["hypothetical protein"])[0],
"EC_number": qualifiers.get("EC_number", [""])[0],
"protein_id": qualifiers.get("protein_id", [""])[0],
"start": int(feat.location.start),
"end": int(feat.location.end),
"strand": feat.location.strand,
"aa_length": len(qualifiers.get("translation", [""])[0]),
})
features_df = pd.DataFrame(rows)
print(f"CDS features extracted: {len(features_df)}")
print(features_df.head(3).to_string(index=False))
# Retrieve protein sequence for a specific gene
gene_name = "dnaA"
gene_feat = features_df[features_df["gene"] == gene_name]
if not gene_feat.empty:
idx = gene_feat.index[0]
print(f"\n{gene_name}: {features_df.loc[idx, 'aa_length']} aa")
Step 6: Visualize Annotation Statistics
Generate a summary barplot of feature types and functional annotation coverage.
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
tsv_file = "annotation/E_coli_K12.tsv"
df = pd.read_csv(tsv_file, sep="\t")
cds = df[df["ftype"] == "CDS"].copy()
n_cds = len(cds)
n_known = (~cds["product"].str.contains("hypothetical", case=False, na=True)).sum()
n_hypo = n_cds - n_known
n_ec = cds["EC_number"].notna().sum()
n_rrna = (df["ftype"] == "rRNA").sum()
n_trna = (df["ftype"] == "tRNA").sum()
fig, axes = plt.subplots(1, 2, figsize=(11, 4))
# Left panel: feature type counts
types = df["ftype"].value_counts()
colors_left = ["#2980B9", "#27AE60", "#E74C3C", "#F39C12"] + ["#95A5A6"] * len(types)
axes[0].bar(types.index, types.values, color=colors_left[:len(types)], edgecolor="white")
axes[0].set_title("Annotated Feature Counts")
axes[0].set_xlabel("Feature Type")
axes[0].set_ylabel("Count")
for i, (label, val) in enumerate(types.items()):
axes[0].text(i, val + 5, str(val), ha="center", fontsize=9)
# Right panel: CDS functional annotation breakdown
labels = ["Known function", "Hypothetical protein", "Enzyme (EC number)"]
sizes = [n_known, n_hypo, n_ec]
colors_right = ["#2980B9", "#BDC3C7", "#E74C3C"]
bars = axes[1].bar(labels, sizes, color=colors_right, edgecolor="white")
axes[1].set_title(f"CDS Functional Annotation\n(n={n_cds} total CDS)")
axes[1].set_ylabel("Count")
axes[1].tick_params(axis="x", rotation=15)
for bar, val in zip(bars, sizes):
axes[1].text(bar.get_x() + bar.get_width() / 2,
val + 10, str(val), ha="center", fontsize=9)
plt.suptitle("Prokka Genome Annotation Summary", fontsize=12, fontweight="bold")
plt.tight_layout()
plt.savefig("prokka_annotation_summary.png", dpi=150, bbox_inches="tight")
print(f"Saved prokka_annotation_summary.png")
print(f"CDS: {n_cds} | Known function: {n_known} | Hypothetical: {n_hypo}")
print(f"rRNA: {n_rrna} | tRNA: {n_trna} | EC-annotated CDS: {n_ec}")
Step 7: Batch Annotation Across Multiple Genomes
Annotate multiple genome assemblies sequentially and collect summary statistics.
#!/bin/bash
# batch_prokka.sh — annotate all FASTA files in a directory
INPUT_DIR="genomes/"
OUTPUT_DIR="annotations/"
mkdir -p "$OUTPUT_DIR"
for FASTA in "$INPUT_DIR"/*.fasta; do
SAMPLE=$(basename "$FASTA" .fasta)
echo "Annotating: $SAMPLE"
prokka "$FASTA" \
--outdir "${OUTPUT_DIR}/${SAMPLE}" \
--prefix "$SAMPLE" \
--kingdom Bacteria \
--cpus 4 \
--mincontiglen 200 \
--quiet
echo " Done: ${OUTPUT_DIR}/${SAMPLE}/${SAMPLE}.txt"
done
echo "Batch annotation complete."
# Collect summary statistics from all Prokka .txt files
from pathlib import Path
import pandas as pd
annotation_dir = Path("annotations/")
rows = []
for txt_file in sorted(annotation_dir.glob("*/*.txt")):
sample = txt_file.stem
stats = {}
with open(txt_file) as f:
for line in f:
line = line.strip()
if ": " in line:
key, val = line.split(": ", 1)
stats[key.strip()] = val.strip()
rows.append({
"sample": sample,
"contigs": int(stats.get("Contigs", 0)),
"bases": int(stats.get("Bases", 0)),
"CDS": int(stats.get("CDS", 0)),
"rRNA": int(stats.get("rRNA", 0)),
"tRNA": int(stats.get("tRNA", 0)),
})
summary_df = pd.DataFrame(rows)
print(f"Annotated genomes: {len(summary_df)}")
print(summary_df.to_string(index=False))
summary_df.to_csv("batch_annotation_summary.csv", index=False)
print("\nSaved: batch_annotation_summary.csv")
Step 8: Compare Annotations Between Strains
Use the protein FASTA outputs for cross-strain comparison with identity-based clustering.
from Bio import SeqIO, pairwise2
import pandas as pd
# Load protein sequences from two strains
def load_proteins(faa_file):
"""Return dict of locus_tag -> protein sequence."""
return {rec.id: str(rec.seq)
for rec in SeqIO.parse(faa_file, "fasta")}
strain_a = load_proteins("annotation_A/strain_A.faa")
strain_b = load_proteins("annotation_B/strain_B.faa")
print(f"Strain A proteins: {len(strain_a)}")
print(f"Strain B proteins: {len(strain_b)}")
# Compare gene counts and size distributions
import matplotlib.pyplot as plt
import numpy as np
len_a = [len(seq) for seq in strain_a.values()]
len_b = [len(seq) for seq in strain_b.values()]
fig, ax = plt.subplots(figsize=(8, 4))
bins = np.linspace(0, 1500, 50)
ax.hist(len_a, bins=bins, alpha=0.6, label=f"Strain A (n={len(len_a)})", color="#2980B9")
ax.hist(len_b, bins=bins, alpha=0.6, label=f"Strain B (n={len(len_b)})", color="#E74C3C")
ax.set_xlabel("Protein Length (aa)")
ax.set_ylabel("Count")
ax.set_title("Protein Length Distribution by Strain")
ax.legend()
plt.tight_layout()
plt.savefig("strain_protein_length_comparison.png", dpi=150, bbox_inches="tight")
print("Saved strain_protein_length_comparison.png")
# Find proteins unique to each strain by size/count difference
print(f"\nSize difference: {abs(len(strain_a) - len(strain_b))} proteins")
print("→ Use Roary or OrthoFinder for formal pan-genome analysis")
Key Parameters
| Parameter | Default | Range / Options | Effect |
|---|---|---|---|
--kingdom | Bacteria | Bacteria, Archaea, Viruses, Mitochondria | Selects gene prediction model and default databases |
--genus | — | any genus name string | Prioritizes genus-specific protein database for CDS annotation |
--species | — | any species name string | Combined with --genus for locus_tag prefix and organism metadata |
--strain | — | any string | Added to organism metadata in GenBank output |
--proteins | — | path to FASTA file | Custom protein database prepended before RefSeq search |
--hmms | — | path to HMM file | Additional custom HMM database for specialized annotation |
--evalue | 1e-6 | 1e-4–1e-9 | E-value cutoff for BLAST and HMMER hits |
--cpus | 8 | 1–CPU count | Parallel processes for BLAST and HMMER searches |
--mincontiglen | 1 | any integer | Skip contigs shorter than this length (bp) |
--metagenome | off | flag | Disables Prodigal training on this genome (for MAGs) |
--rfam | off | flag | Enable Infernal rRNA/ncRNA search against Rfam (slower) |
--norrna | off | flag | Skip rRNA prediction (use when assembly has no rRNA genes) |
--notrna | off | flag | Skip tRNA prediction |
Common Recipes
Recipe: Annotation with Custom Protein Database
When to use: You have closely related reference proteins (e.g., characterized isolate) to improve annotation accuracy.
# Use a custom protein database to enhance annotation of a novel strain
# Custom proteins are searched first, before internal Prokka databases
prokka genome.fasta \
--proteins reference_proteins.faa \
--outdir custom_annotation/ \
--prefix novel_strain \
--kingdom Bacteria \
--cpus 4
echo "Custom DB annotation complete:"
grep "CDS" custom_annotation/novel_strain.txt
Recipe: Metagenome-Assembled Genome (MAG) Annotation
When to use: Annotating a MAG where Prodigal cannot train on the full genome sequence.
# --metagenome disables Prodigal model training (uses meta mode)
# --mincontiglen 500 discards short, potentially chimeric contigs
prokka MAG_bin_42.fasta \
--metagenome \
--kingdom Bacteria \
--outdir mag_annotation/ \
--prefix MAG_bin_42 \
--mincontiglen 500 \
--cpus 4
echo "MAG annotation complete:"
cat mag_annotation/MAG_bin_42.txt
Recipe: Extract Specific Gene Sequences
When to use: Retrieve nucleotide or protein sequences for a target gene or pathway from the annotation.
from Bio import SeqIO
# Extract all sequences for a specific gene name from protein FASTA
faa_file = "annotation/E_coli_K12.faa"
target_gene = "dnaA"
matches = []
for rec in SeqIO.parse(faa_file, "fasta"):
# Prokka FASTA header: >locus_tag gene product
if target_gene in rec.description:
matches.append(rec)
print(f"Proteins matching '{target_gene}': {len(matches)}")
for rec in matches:
print(f" {rec.id}: {len(rec.seq)} aa — {rec.description}")
# Save matching sequences
if matches:
SeqIO.write(matches, f"{target_gene}_proteins.faa", "fasta")
print(f"Saved: {target_gene}_proteins.faa")
Recipe: Convert GFF to Pandas DataFrame
When to use: Work with genomic coordinates for feature overlap analysis or visualization.
import pandas as pd
def parse_gff(gff_file):
"""Parse Prokka GFF3 file into a DataFrame (skips sequence section)."""
rows = []
with open(gff_file) as f:
for line in f:
if line.startswith("##FASTA"):
break
if line.startswith("#") or not line.strip():
continue
parts = line.rstrip("\n").split("\t")
if len(parts) < 9:
continue
attr_dict = {}
for item in parts[8].split(";"):
if "=" in item:
k, v = item.split("=", 1)
attr_dict[k] = v
rows.append({
"seqname": parts[0],
"source": parts[1],
"feature": parts[2],
"start": int(parts[3]),
"end": int(parts[4]),
"score": parts[5],
"strand": parts[6],
"frame": parts[7],
"locus_tag": attr_dict.get("ID", ""),
"gene": attr_dict.get("gene", ""),
"product": attr_dict.get("product", ""),
})
return pd.DataFrame(rows)
gff_df = parse_gff("annotation/E_coli_K12.gff")
print(f"GFF features: {len(gff_df)}")
print(gff_df[gff_df["feature"] == "CDS"].head(5)[
["seqname", "start", "end", "strand", "gene", "product"]
].to_string(index=False))
Expected Outputs
| Output File | Format | Description |
|---|---|---|
{prefix}.gff | GFF3 | Genome annotation with feature coordinates and attributes; includes FASTA sequence at the end |
{prefix}.gbk | GenBank | Full GenBank-format annotation for use in Geneious, BioPython, and submission prep |
{prefix}.faa | FASTA | Predicted protein sequences for all CDS features |
{prefix}.ffn | FASTA | Nucleotide sequences for all annotated features (CDS, rRNA, tRNA) |
{prefix}.fna | FASTA | Nucleotide FASTA of the complete assembly (contigs) |
{prefix}.tsv | TSV | Tab-delimited summary table: locus_tag, ftype, length_bp, gene, EC_number, COG, product |
{prefix}.txt | Text | One-line summary counts: Contigs, Bases, CDS, rRNA, tRNA, tmRNA |
{prefix}.tbl | TBL | Feature table format for tbl2asn GenBank submission |
{prefix}.sqn | SQN | ASN.1 format for NCBI submission (generated by tbl2asn) |
{prefix}.err | Text | Warnings and errors from tbl2asn validation step |
{prefix}.log | Text | Full Prokka run log with timing per step |
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
FATAL: Can't find any tRNA genes | No tRNA detected on short or highly fragmented assembly | Add --notrna flag; check if assembly is too fragmented (N50 < 1 kb) |
Can't exec "makeblastdb" | BLAST+ not on PATH | conda install -c bioconda blast; ensure prokka env is activated |
Contig ID too long (>37 chars) | Assembler produced long contig headers | Pre-process FASTA to shorten headers: awk '/^>/{print ">contig_"++i; next}{print}' |
| Very high hypothetical protein rate (>60%) | Divergent organism with few database matches | Add --proteins with closely related strain FAA; consider --genus flag |
ERROR: Argument --proteins: file does not exist | Path to custom protein file is incorrect | Use absolute path; verify file exists with ls -la custom.faa |
tbl2asn error: multiple /product | Duplicate product qualifiers in annotation | Ignore if exporting for local use; for NCBI submission, use PGAP instead |
| Annotation is very slow (>30 min for ~5 Mb) | --cpus not set or set to 1; --rfam enabled | Set --cpus to available thread count; disable --rfam for faster runs |
| rRNA count is 0 for complete genome | --norrna flag was set, or barrnap threshold too strict | Remove --norrna; check barrnap is installed with barrnap --version |
References
- Prokka GitHub: tseemann/prokka — source code, installation instructions, and detailed parameter reference
- Seemann T (2014) Bioinformatics 30(14):2068–2069 — original Prokka paper with benchmarks vs. RAST
- Prodigal documentation — gene prediction engine used by Prokka for CDS calling
- Barrnap GitHub: tseemann/barrnap — rRNA prediction tool bundled with Prokka
- BioPython GenBank parsing tutorial — reference for parsing Prokka GenBank output in Python
Roary Pan-Genome Pipeline
Overview
Roary is a high-throughput pan-genome pipeline for prokaryotes that takes per-sample GFF3 annotations (typically from Prokka or Bakta) and produces a clustered gene presence/absence matrix across the entire input set. It first reduces redundancy with CD-HIT iterative clustering, then performs an all-vs-all BLASTP within each pre-cluster, and finally applies MCL graph clustering to define orthologous gene families. The output partitions the gene space into core (≥ 99 %), soft-core (95–99 %), shell (15–95 %), and cloud (< 15 %) genes and optionally builds a concatenated core-gene alignment suitable for phylogenetic inference.
When to Use
- Computing a pan-genome from a set of bacterial isolate annotations (10–10,000 genomes)
- Producing a
gene_presence_absence.csvmatrix for downstream GWAS, accessory-gene mining, or core-gene phylogenetics - Building a concatenated core-gene multi-FASTA alignment for ML/Bayesian phylogenetic trees
- Generating a pan-genome reference FASTA to use as a non-redundant gene catalog
- Comparative genomics across closely related strains where >95 % nucleotide identity is expected
- Use Panaroo instead when assemblies are highly fragmented or annotations are noisy (Panaroo aggressively cleans annotation errors)
- Use PIRATE instead when paralog-aware clustering with multiple identity thresholds is needed
- Use PPanGGOLiN instead when graph-based, statistically grounded gene-family partitioning is preferred over fixed-frequency cutoffs
Prerequisites
- Software: Roary ≥ 3.13, Perl 5, BLAST+, CD-HIT, MCL, BEDTools, PRANK or MAFFT (for
-ecore alignment), FastTree (optional) - Python packages (for output parsing):
pandas,matplotlib,seaborn,biopython,dendropy - Input: per-sample GFF3 files with embedded FASTA at the end (Prokka/Bakta default output)
- Hardware: ≥ 8 GB RAM for ~50 genomes; 32 GB+ recommended for ~500 genomes
- Naming: each GFF3 filename becomes the sample column header in the output matrix; use
sample_id.gffstyle
Check before installing: The tool may already be available in the current environment (e.g., inside a
pixi/condaenv). Runcommand -v roaryfirst and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool viapixi run roaryrather than bareroary.
# Install Roary via conda/mamba (recommended)
mamba install -c conda-forge -c bioconda roary
# Verify installation
roary --version
# 3.13.0
# Verify dependent tools
which cd-hit blastp mcl bedtools mafft
# /opt/conda/bin/cd-hit
# /opt/conda/bin/blastp
# /opt/conda/bin/mcl
# /opt/conda/bin/bedtools
# /opt/conda/bin/mafft
# Install Python parsing dependencies
pip install pandas matplotlib seaborn biopython dendropy
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: inputAnnotations
kind: derived
source: upstream
ask: "Which GFF3 files, and were they all produced by the same annotation tool and version?"
default: "the annotation stage output"
- id: D2
param: identityThreshold
kind: required
source: user
ask: "How similar must two proteins be to count as the same gene? Lower values merge diverged orthologs; higher values split them."
default: "95%"
- id: D3
param: coreDefinition
kind: required
source: user
ask: "In what fraction of isolates must a gene appear before it counts as core?"
default: "99%"
- id: D4
param: paralogSplitting
kind: required
source: user
ask: "Should duplicated genes within a genome be split into separate families, or kept together?"
default: "split"
- id: D5
param: coreAlignment
kind: required
source: upstream
ask: "Is a concatenated core-gene alignment needed for a downstream phylogeny?"
default: "not produced - it is the slowest stage by a wide margin"
- id: D6
param: alignmentSpeed
kind: optional
source: user
depends_on: [D5]
ask: "Align every gene accurately, or use the fast route sufficient for tree building?"
default: "fast alignment"
skip_if: "no core alignment requested"
- id: D7
param: clusterInflation
kind: optional
source: user
ask: "How tightly should the clustering step group families?"
default: 1.5
- id: D8
param: maxClusters
kind: optional_conditional
source: data
ask: "Is the expected number of gene families above the default cap?"
default: 50000
- id: D9
param: processes
kind: never_ask
source: data
reason: "Affects runtime only, not the gene families"
default: "min(8, available_cores)"
D1 is derived and carries a consistency requirement rather than a choice: Roary compares annotations, so GFF3 files from different annotation tools or database versions produce apparent presence/absence differences that are annotation artefacts, not biology. D2 and D3 together decide the size of the core genome, which is usually the headline number of the analysis.
Quick Start
# Run Roary on all GFF3 files in current directory; emit core gene alignment
roary -e --mafft -p 8 -o pangenome -f roary_out/ *.gff
# Inspect summary statistics
cat roary_out/summary_statistics.txt
# Core genes (99% <= strains <= 100%) 2823
# Soft core genes (95% <= strains < 99%) 78
# Shell genes (15% <= strains < 95%) 1542
# Cloud genes ( 0% <= strains < 15%) 3104
# Total genes 7547
Workflow
Step 1: Install Roary and Verify Dependencies
Install Roary in a dedicated environment to avoid Perl module conflicts.
# Create a dedicated conda environment
mamba create -n roary_env -c conda-forge -c bioconda roary mafft fasttree python=3.11 -y
mamba activate roary_env
# Verify Roary
roary --version
# 3.13.0
# Confirm pipeline dependencies are reachable
for tool in cd-hit blastp mcl bedtools mafft FastTree; do
if command -v $tool >/dev/null; then
echo "OK: $tool"
else
echo "MISSING: $tool"
fi
done
# Install Python parsing tools
pip install pandas matplotlib seaborn biopython dendropy
Step 2: Prepare Per-Sample GFF3 Files from Prokka or Bakta
Roary requires GFF3 files with an embedded FASTA section (the ##FASTA block). Both Prokka .gff and Bakta .gff3 outputs satisfy this format.
# Verify GFF3 files include the embedded FASTA section
mkdir -p roary_input/
cp annotations/*/sample*.gff roary_input/
for GFF in roary_input/*.gff; do
if grep -q "^##FASTA" "$GFF"; then
echo "OK: $GFF"
else
echo "MISSING ##FASTA in: $GFF"
fi
done
# Roary uses GFF filename (without .gff) as the sample column name.
# Rename or symlink to ensure unique, descriptive names.
cd roary_input/
ls *.gff | head
# strain_A.gff strain_B.gff strain_C.gff
# Sanity-check sample size and unique names
from pathlib import Path
gff_files = sorted(Path("roary_input/").glob("*.gff"))
print(f"GFF files: {len(gff_files)}")
names = [g.stem for g in gff_files]
duplicates = [n for n in names if names.count(n) > 1]
assert not duplicates, f"Duplicate sample names: {set(duplicates)}"
# Show first 5
for g in gff_files[:5]:
size_mb = g.stat().st_size / 1e6
print(f" {g.name}: {size_mb:.1f} MB")
Step 3: Run Roary with Core Gene Alignment
The -e --mafft combination produces the concatenated core-gene alignment used downstream for phylogenetics.
# Run Roary on the prepared GFF directory
# -e: extract core gene alignment per gene, then concatenate
# --mafft: use MAFFT for alignment (faster than default PRANK)
# -p 8: 8 parallel processes
# -i 95: min BLASTP percentage identity (default 95)
# -cd 99: % isolates a gene must be in to be "core" (default 99)
roary -e --mafft \
-p 8 \
-i 95 \
-cd 99 \
-o pangenome \
-f roary_out/ \
roary_input/*.gff
# Expected runtime:
# ~50 genomes: 10–20 min
# ~500 genomes: 4–8 hours
ls roary_out/
# accessory_binary_genes.fa gene_presence_absence.csv
# accessory_binary_genes.fa.newick gene_presence_absence.Rtab
# accessory.header.embl number_of_conserved_genes.Rtab
# accessory.tab number_of_genes_in_pan_genome.Rtab
# blast_identity_frequency.Rtab number_of_new_genes.Rtab
# clustered_proteins number_of_unique_genes.Rtab
# core_accessory.header.embl pan_genome_reference.fa
# core_accessory.tab summary_statistics.txt
# core_gene_alignment.aln
Step 4: Parse the Gene Presence/Absence Matrix
The CSV output is the canonical pan-genome matrix: rows are gene families, columns include metadata and one column per sample (filled with locus tags or empty).
import pandas as pd
import numpy as np
df = pd.read_csv("roary_out/gene_presence_absence.csv", low_memory=False)
print(f"Gene families: {len(df)}")
print(f"Columns: {len(df.columns)}")
# Standard metadata columns end with 'Inference', then sample columns follow
metadata_cols = list(df.columns[:14])
sample_cols = list(df.columns[14:])
print(f"Samples: {len(sample_cols)}")
# Build a binary presence/absence matrix (1 = locus tag present, 0 = empty)
binary = df[sample_cols].notna().astype(int)
binary.index = df["Gene"]
print(f"Binary matrix shape: {binary.shape}")
# Pan-genome partition by frequency
n_samples = len(sample_cols)
freq = binary.sum(axis=1) / n_samples
core = freq[freq >= 0.99]
soft_core = freq[(freq >= 0.95) & (freq < 0.99)]
shell = freq[(freq >= 0.15) & (freq < 0.95)]
cloud = freq[freq < 0.15]
print(f"\nPan-genome partition (n={n_samples} genomes):")
print(f" Core (>=99 %): {len(core):>5}")
print(f" Soft-core (95–99 %): {len(soft_core):>5}")
print(f" Shell (15–95 %): {len(shell):>5}")
print(f" Cloud (<15 %): {len(cloud):>5}")
print(f" Total : {len(freq):>5}")
Step 5: Visualize the Pan-Genome Frequency Distribution
A frequency histogram exposes whether the input set has a U-shaped (open) or core-skewed (closed) pan-genome.
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
df = pd.read_csv("roary_out/gene_presence_absence.csv", low_memory=False)
sample_cols = list(df.columns[14:])
n = len(sample_cols)
binary = df[sample_cols].notna().astype(int)
freq = binary.sum(axis=1)
fig, axes = plt.subplots(1, 2, figsize=(11, 4))
# Histogram of gene frequencies
axes[0].hist(freq, bins=range(1, n + 2), color="#2980B9", edgecolor="white")
axes[0].axvline(0.99 * n, color="red", linestyle="--", label="99 % core cutoff")
axes[0].axvline(0.15 * n, color="orange", linestyle="--", label="15 % shell cutoff")
axes[0].set_xlabel("Number of genomes containing the gene")
axes[0].set_ylabel("Number of gene families")
axes[0].set_title("Pan-genome frequency distribution")
axes[0].legend()
# Pie chart of pan-genome partition
freqp = freq / n
core_n = (freqp >= 0.99).sum()
soft_n = ((freqp >= 0.95) & (freqp < 0.99)).sum()
shell_n = ((freqp >= 0.15) & (freqp < 0.95)).sum()
cloud_n = (freqp < 0.15).sum()
axes[1].pie([core_n, soft_n, shell_n, cloud_n],
labels=[f"Core ({core_n})", f"Soft-core ({soft_n})",
f"Shell ({shell_n})", f"Cloud ({cloud_n})"],
colors=["#27AE60", "#3498DB", "#F39C12", "#95A5A6"],
autopct="%1.1f%%", startangle=90)
axes[1].set_title(f"Pan-genome partition (n={n} genomes)")
plt.tight_layout()
plt.savefig("pangenome_distribution.png", dpi=150, bbox_inches="tight")
print(f"Saved pangenome_distribution.png (total families: {len(freq)})")
Step 6: Build a Phylogenetic Tree from the Core Gene Alignment
Use FastTree on the concatenated core-gene alignment to recover strain relationships.
# FastTree expects an aligned multi-FASTA; Roary produces this with `-e`
FastTree -nt -gtr -nosupport \
< roary_out/core_gene_alignment.aln \
> roary_out/core_gene_tree.nwk
echo "Tree built: roary_out/core_gene_tree.nwk"
# (Optional) IQ-TREE for support values:
# iqtree -s core_gene_alignment.aln -m GTR+G -bb 1000 -nt 8
# Visualize the FastTree result alongside the gene presence/absence matrix
import dendropy
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
tree = dendropy.Tree.get(path="roary_out/core_gene_tree.nwk", schema="newick")
leaves = [leaf.taxon.label for leaf in tree.leaf_node_iter()]
print(f"Tree leaves: {len(leaves)}")
print(f"Tree length: {tree.length():.4f} substitutions/site")
# Print first 10 leaves with their root-to-tip distances
for leaf in list(tree.leaf_node_iter())[:10]:
dist = leaf.distance_from_root()
print(f" {leaf.taxon.label:<25} d={dist:.4f}")
Step 7: Produce Roary's Built-in Summary Plots
Roary ships with a roary_plots.py companion that builds a tree-anchored matrix figure.
# Use the bundled python helper from the Roary install
# (download from: https://github.com/sanger-pathogens/Roary/blob/master/contrib/roary_plots/roary_plots.py)
python roary_plots.py \
roary_out/core_gene_tree.nwk \
roary_out/gene_presence_absence.csv \
--format png \
--labels
ls *.png
# pangenome_frequency.png
# pangenome_matrix.png
# pangenome_pie.png
Step 8: Compute Per-Genome Accessory Gene Counts
Identify which genomes carry the most unique (cloud) genes — useful for strain-specific gene investigation.
import pandas as pd
df = pd.read_csv("roary_out/gene_presence_absence.csv", low_memory=False)
sample_cols = list(df.columns[14:])
n = len(sample_cols)
binary = df[sample_cols].notna().astype(int)
binary.index = df["Gene"]
freq = binary.sum(axis=1)
cloud_mask = freq < (0.15 * n)
shell_mask = (freq >= 0.15 * n) & (freq < 0.95 * n)
per_genome = pd.DataFrame({
"genes_total": binary.sum(axis=0),
"cloud_genes": binary.loc[cloud_mask].sum(axis=0),
"shell_genes": binary.loc[shell_mask].sum(axis=0),
})
per_genome = per_genome.sort_values("cloud_genes", ascending=False)
print(per_genome.head(10).to_string())
per_genome.to_csv("per_genome_accessory_counts.csv")
print("\nSaved per_genome_accessory_counts.csv")
Key Parameters
| Parameter | Default | Range / Options | Effect |
|---|---|---|---|
-i | 95 | 70–99 (% identity) | Minimum BLASTP percentage identity for clustering |
-cd | 99 | 90–100 (%) | Minimum % of isolates a gene must be in to be called "core" |
-p | 1 | 1–CPU count | Number of parallel processes for BLAST and MCL stages |
-e | off | flag | Extract concatenated core gene multi-FASTA alignment (per-gene MSA + concat) |
--mafft | off | flag (with -e) | Use MAFFT for alignment (faster than default PRANK) |
-s | off | flag | Do not split paralogs into separate gene families |
-n | off | flag | Use fast core gene alignment with MAFFT (skips per-gene PRANK) |
-g | 50000 | any integer | Maximum number of clusters expected (cap on output size) |
-iv | 1.5 | 1.2–5.0 | MCL inflation value; higher = tighter clusters |
-r | off | flag | Generate R plots (presence/absence heatmap, pan-genome curves) |
-o | clustered_proteins | string | Output prefix for clustered proteins file |
-f | . | path | Output directory |
-z | off | flag | Don't delete intermediate files (useful for debugging) |
Common Recipes
Recipe: Core Gene Alignment from Existing Roary Output
When to use: You already ran Roary without -e and now need a phylogenetic alignment.
# Re-run only the alignment step using the extracted core genes
# query_pan_genome from the Roary suite extracts gene-family multi-FASTAs
query_pan_genome -a intersection \
-g roary_out/clustered_proteins \
-o core_gene_list.txt \
roary_input/*.gff
echo "Core genes: $(wc -l < core_gene_list.txt)"
# Then re-run Roary with -e to materialize the alignment:
# roary -e --mafft -p 8 -f roary_out2/ roary_input/*.gff
Recipe: Lower Identity Threshold for Diverse Genus-Level Sets
When to use: Comparing genomes across multiple closely related species (e.g., genus-level pan-genome).
# Drop identity to 70 % to capture orthologs across species boundaries
roary -e --mafft \
-p 8 \
-i 70 \
-cd 99 \
-f roary_genus/ \
roary_input/*.gff
cat roary_genus/summary_statistics.txt
# Expect a smaller core and a much larger shell + cloud
Recipe: Roary on a Subset of Strains
When to use: You want a pan-genome of a specific subclade without re-annotating all genomes.
# Stage GFF files for the subset
mkdir -p subset/
for SAMPLE in strain_A strain_B strain_E strain_K; do
cp roary_input/${SAMPLE}.gff subset/
done
roary -e --mafft -p 4 -f roary_subset/ subset/*.gff
echo "Subset pan-genome:"
cat roary_subset/summary_statistics.txt
Recipe: Cross-Tabulate Accessory Genes Against Strain Metadata
When to use: You want to find genes enriched in a specific phenotype or geographic group.
import pandas as pd
from scipy.stats import fisher_exact
# Load presence/absence and strain metadata
pa = pd.read_csv("roary_out/gene_presence_absence.csv", low_memory=False)
sample_cols = list(pa.columns[14:])
binary = pa[sample_cols].notna().astype(int)
binary.index = pa["Gene"]
binary.columns = sample_cols
# Metadata: sample_id, phenotype (e.g., "resistant" / "susceptible")
meta = pd.read_csv("strain_metadata.csv").set_index("sample_id")
group_resistant = meta[meta["phenotype"] == "resistant"].index
group_susceptible = meta[meta["phenotype"] == "susceptible"].index
group_resistant = [g for g in group_resistant if g in binary.columns]
group_susceptible = [g for g in group_susceptible if g in binary.columns]
# Fisher's exact test per gene family
results = []
for gene, row in binary.iterrows():
a = int(row[group_resistant].sum())
b = len(group_resistant) - a
c = int(row[group_susceptible].sum())
d = len(group_susceptible) - c
if a + c == 0 or a + c == len(group_resistant) + len(group_susceptible):
continue # skip invariant genes
odds, pval = fisher_exact([[a, b], [c, d]])
results.append({"gene": gene, "n_resistant": a, "n_susceptible": c,
"odds_ratio": odds, "pvalue": pval})
results_df = pd.DataFrame(results).sort_values("pvalue")
print(f"Gene-phenotype associations (top 10):")
print(results_df.head(10).to_string(index=False))
results_df.to_csv("phenotype_associated_genes.csv", index=False)
Expected Outputs
| Output File | Format | Description |
|---|---|---|
gene_presence_absence.csv | CSV | Pan-genome matrix; rows = gene families, columns = sample locus tags |
gene_presence_absence.Rtab | TSV | Binary 0/1 matrix in R-friendly format |
summary_statistics.txt | Text | Counts of core / soft-core / shell / cloud / total gene families |
pan_genome_reference.fa | FASTA | Non-redundant nucleotide reference of one representative per gene family |
clustered_proteins | Text | Cluster definitions: cluster name → constituent locus tags |
core_gene_alignment.aln | FASTA (aligned) | Concatenated multi-FASTA of core gene alignments (only with -e) |
accessory_binary_genes.fa | FASTA | Binary 0/1 sequences of accessory presence (input to FastTree) |
accessory_binary_genes.fa.newick | Newick | Accessory gene phylogeny (built from binary FASTA) |
number_of_conserved_genes.Rtab | TSV | Rarefaction curve of core genes vs. genome count |
number_of_genes_in_pan_genome.Rtab | TSV | Rarefaction curve of pan-genome growth |
number_of_new_genes.Rtab | TSV | New genes added per genome |
number_of_unique_genes.Rtab | TSV | Per-genome unique gene counts |
blast_identity_frequency.Rtab | TSV | Distribution of BLAST identities used during clustering |
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
GFF file does not contain ##FASTA | Annotation file is not a Prokka/Bakta-style GFF3 with embedded sequences | Re-export from Prokka/Bakta; do not strip the FASTA tail |
Could not run blastp | BLAST+ missing from the conda env | mamba install -c bioconda blast and re-activate the env |
| Roary produces 0 core genes | One sample has very different annotations or is misidentified | Inspect gene_presence_absence.csv; remove outlier with roary --check_input style sanity checks |
| Job runs out of memory on > 200 genomes | Default identity (95 %) creates many BLAST jobs | Increase -i to 90 if accuracy permits; split into batches; use Panaroo on RAM-tight machines |
| Paralogs split into many tiny clusters | MCL inflation too high or duplicate locus tags across samples | Add -s to keep paralogs together, or rerun Prokka/Bakta with unique --locus-tag per sample |
FastTree: bad alignment on core_gene_alignment.aln | Alignment built with PRANK was truncated | Re-run with --mafft for MAFFT-based alignment |
| Output column names look mangled or missing | GFF filenames had spaces or special characters | Rename input GFFs to [a-zA-Z0-9_]+.gff before running |
| Same genome counted twice | Duplicate GFF filenames (e.g., copied symlinks) | Run `ls roary_input/*.gff |
| Pan-genome size grows linearly without saturation | Truly open pan-genome (expected for many bacteria) | This is biological, not a bug; report rarefaction curves and use Heaps' law for confirmation |
References
- Roary GitHub: sanger-pathogens/Roary — source code, parameter reference, and roary_plots.py helper
- Page et al. (2015) Bioinformatics 31(22):3691–3693 — original Roary publication describing the CD-HIT + MCL pipeline
- Roary tutorial — step-by-step worked example with sample data
- Panaroo (alternative tool) — graph-based pan-genome with stricter annotation cleaning
- PIRATE (alternative tool) — paralog-aware pan-genome at multiple identity thresholds
- PPanGGOLiN (alternative tool) — graph-based statistical partitioning of pan-genomes
- FastTree documentation — reference for ML phylogenetic inference from core gene alignments
Arboreto GRN Inference
Overview
Arboreto infers gene regulatory networks (GRNs) from gene expression data using parallelized tree-based regression. For each target gene, it trains a regression model with all other genes (or a specified TF list) as features and emits TF-target-importance triplets. It provides two interchangeable algorithms -- GRNBoost2 (gradient boosting, fast) and GENIE3 (Random Forest, classic) -- sharing identical input/output formats. Computation is Dask-parallelized, scaling from laptop cores to HPC clusters.
When to Use
- Inferring transcription factor-to-target gene regulatory relationships from bulk RNA-seq expression data
- Building gene regulatory networks from single-cell RNA-seq count matrices (cells as rows, genes as columns)
- Generating the adjacency matrix (Step 1) of the pySCENIC regulatory analysis pipeline
- Comparing regulatory network structure across experimental conditions (e.g., control vs treatment)
- Producing consensus regulatory networks by running inference across multiple random seeds
- Validating GRN results by comparing GRNBoost2 and GENIE3 outputs on the same dataset
- For downstream regulon identification and activity scoring, use arboreto output with pySCENIC
- For single-cell preprocessing (QC, normalization, clustering) before GRN inference, use scanpy-scrna-seq
Prerequisites
- Python packages:
arboreto,pandas,numpy,dask,distributed,scikit-learn,scipy - Data requirements: Gene expression matrix (genes as columns, observations as rows) in TSV/CSV; optionally a TF list file (one gene name per line)
- Environment: Python 3.8+; optional
networkx,matplotlibfor visualization
pip install arboreto distributed networkx matplotlib
Pre-flight Interview
Settle these with the user before writing any analysis code.
decisions:
- id: D1
param: algorithm
kind: required
source: user
ask: "Infer the network with gradient boosting (faster) or random forests (the original GENIE3 formulation)?"
default: "GRNBoost2"
- id: D2
param: expressionMatrix
kind: required
source: upstream
ask: "Which processed expression matrix, and which cells or samples within it, should the network be inferred from?"
default: null
- id: D3
param: regulatorList
kind: required
source: literature
ask: "Restrict candidate regulators to a curated transcription-factor list, or let every gene be a candidate regulator?"
default: "a species-matched TF list"
- id: D4
param: edgeThreshold
kind: required
source: user
ask: "The output ranks every regulator-target pair - how many of the top edges should be kept as the network?"
default: null
- id: D5
param: seed
kind: never_ask
source: data
reason: "Fixes the draw for reproducibility; does not change what the data supports"
default: 0
- id: D6
param: daskClient
kind: never_ask
source: data
reason: "Execution backend; affects runtime only"
default: "local"
D3 is the decision that separates a usable network from a hairball. Leaving regulators unrestricted lets any correlated gene appear as a regulator, and the result still returns a fully populated ranked table. D4 has no safe default: the algorithm scores all pairs, so where the list is cut is the user's call about the network they intend to interpret.
Quick Start
Complete GRN inference in a single block. The if __name__ == '__main__': guard is required because Dask spawns worker processes via multiprocessing.
import pandas as pd
from arboreto.algo import grnboost2
from arboreto.utils import load_tf_names
if __name__ == '__main__':
# Load expression matrix (observations x genes)
expression_matrix = pd.read_csv('expression_data.tsv', sep='\t')
tf_names = load_tf_names('tf_list.txt') # optional TF filter
# Infer GRN (uses all local cores by default)
network = grnboost2(expression_data=expression_matrix,
tf_names=tf_names, seed=777)
# Filter top links and save
top_network = network[network['importance'] > 1.0]
top_network.to_csv('grn_output.tsv', sep='\t', index=False, header=False)
print(f"Inferred {len(network)} links, kept {len(top_network)} above threshold")
# Example: Inferred 185432 links, kept 12876 above threshold
Workflow
Step 1: Load Expression Data
Arboreto accepts a pandas DataFrame (recommended) or NumPy array. Rows are observations (cells or samples), columns are genes. Gene names must be column headers.
import pandas as pd
# From TSV (genes as columns, observations as rows)
expression_matrix = pd.read_csv('expression_data.tsv', sep='\t')
print(f"Shape: {expression_matrix.shape} "
f"({expression_matrix.shape[0]} observations x {expression_matrix.shape[1]} genes)")
# Example: Shape: (5000, 18654) (5000 observations x 18654 genes)
# From AnnData (e.g., after scanpy preprocessing)
import anndata as ad
adata = ad.read_h5ad('preprocessed.h5ad')
expression_matrix = pd.DataFrame(
adata.X.toarray() if hasattr(adata.X, 'toarray') else adata.X,
columns=adata.var_names.tolist()
)
print(f"Converted AnnData: {expression_matrix.shape}")
# Example: Converted AnnData: (5000, 18654)
Step 2: Load Transcription Factor List
Providing a TF list restricts regulators to known transcription factors, reducing computation time and improving biological relevance. If omitted, all genes are treated as potential regulators.
from arboreto.utils import load_tf_names
# From file (one TF name per line)
tf_names = load_tf_names('human_tfs.txt')
print(f"Loaded {len(tf_names)} transcription factors")
# Example: Loaded 1639 transcription factors
# Or define directly
tf_names = ['MYC', 'TP53', 'SOX2', 'NANOG', 'POU5F1']
# Verify TFs exist in expression matrix columns
tf_in_data = [tf for tf in tf_names if tf in expression_matrix.columns]
print(f"TFs found in expression data: {len(tf_in_data)}/{len(tf_names)}")
# Example: TFs found in expression data: 1583/1639
Step 3: Configure Dask Client (Optional)
By default arboreto creates an internal Dask client using all local cores. Create an explicit client for resource control, monitoring, or cluster deployment.
from distributed import LocalCluster, Client
# Custom local client with resource limits
local_cluster = LocalCluster(
n_workers=8,
threads_per_worker=1, # avoid GIL contention in scikit-learn
memory_limit='4GB' # per worker
)
client = Client(local_cluster)
print(f"Dashboard: {client.dashboard_link}")
# Example: Dashboard: http://127.0.0.1:8787/status
Step 4: Run GRN Inference
Call grnboost2() (recommended) or genie3(). Both share the same signature and output format.
from arboreto.algo import grnboost2
if __name__ == '__main__':
network = grnboost2(
expression_data=expression_matrix,
tf_names=tf_names, # 'all' if no TF list
client_or_address=client, # omit to use default local scheduler
seed=777, # for reproducibility
verbose=True # print progress
)
print(f"Inferred {len(network)} regulatory links")
print(network.head())
# Example:
# TF target importance
# 0 MYC CDK4 3.214
# 1 MYC CCND1 2.871
# 2 TP53 CDKN1A 2.654
Step 5: Filter Results
Raw output contains links for every TF-target pair with non-zero importance. Filter to retain high-confidence regulatory interactions.
# Strategy 1: Importance threshold
threshold = 1.0
filtered = network[network['importance'] > threshold]
print(f"Threshold {threshold}: {len(filtered)} links "
f"({len(filtered)/len(network)*100:.1f}% retained)")
# Example: Threshold 1.0: 12876 links (6.9% retained)
# Strategy 2: Top N links per target gene
top_n = 10
top_per_target = (network.groupby('target')
.apply(lambda g: g.nlargest(top_n, 'importance'))
.reset_index(drop=True))
print(f"Top {top_n} per target: {len(top_per_target)} links")
# Example: Top 10 per target: 186540 links
Step 6: Save Network
Save as a TSV file with three columns: TF, target, importance.
# Save full network
network.to_csv('full_network.tsv', sep='\t', index=False)
print(f"Saved full network: {len(network)} links")
# Save filtered network (without header for pySCENIC compatibility)
filtered.to_csv('filtered_network.tsv', sep='\t', index=False, header=False)
print(f"Saved filtered network: {len(filtered)} links")
# Clean up Dask client if explicitly created
client.close()
local_cluster.close()
Step 7: Visualize Results (Optional)
Plot the top regulatory interactions as a directed network graph.
import networkx as nx
import matplotlib.pyplot as plt
# Build directed graph from top links
top_links = network.nlargest(50, 'importance')
G = nx.from_pandas_edgelist(
top_links, source='TF', target='target',
edge_attr='importance', create_using=nx.DiGraph()
)
# Draw network
fig, ax = plt.subplots(figsize=(12, 10))
pos = nx.spring_layout(G, k=2, seed=42)
nx.draw_networkx_nodes(G, pos, node_size=300, node_color='lightblue', ax=ax)
nx.draw_networkx_labels(G, pos, font_size=7, ax=ax)
edges = nx.draw_networkx_edges(
G, pos, edge_color=[G[u][v]['importance'] for u, v in G.edges()],
edge_cmap=plt.cm.Reds, width=1.5, arrows=True, ax=ax
)
plt.colorbar(edges, ax=ax, label='Importance')
ax.set_title('Top 50 Regulatory Interactions')
plt.tight_layout()
plt.savefig('grn_network.png', dpi=300, bbox_inches='tight')
print("Saved grn_network.png")
Key Parameters
| Parameter | Default | Range / Options | Effect |
|---|---|---|---|
expression_data | (required) | DataFrame or ndarray | Expression matrix, observations x genes |
tf_names | 'all' | list of str or 'all' | Restrict regulators to known TFs; 'all' uses every gene |
gene_names | None | list of str | Required when expression_data is a NumPy array |
client_or_address | 'local' | Client, str, or 'local' | Dask client instance or scheduler address |
seed | None | int | Random seed for reproducibility; always set for replicable results |
verbose | False | bool | Print progress messages during inference |
Key Concepts
Algorithm Selection Guide
Both algorithms follow the same multiple-regression strategy (train one model per target gene, extract feature importances) but differ in the underlying regressor.
| Feature | GRNBoost2 | GENIE3 |
|---|---|---|
| Method | Stochastic gradient boosting with early stopping | Random Forest (or ExtraTrees) |
| Speed | Fast -- optimized for large datasets | Slower -- higher per-model cost |
| Memory | Lower (early stopping limits tree depth) | Higher (full forests per target) |
| Best for | Default choice; 10k+ observations, single-cell scale | Comparison with published GENIE3 results; validation |
| Import | from arboreto.algo import grnboost2 | from arboreto.algo import genie3 |
| Output format | TF, target, importance (identical) | TF, target, importance (identical) |
Decision rule: Start with GRNBoost2. Use GENIE3 only when reproducing published GENIE3 analyses or as an independent validation of GRNBoost2 results.
Output Format
The network DataFrame has three columns, sorted by descending importance:
| Column | Type | Description |
|---|---|---|
TF | str | Transcription factor (regulator) gene name |
target | str | Target gene name |
importance | float | Regulatory importance score (higher = stronger predicted regulation) |
Importance scores are not p-values. They represent feature importance from the tree-based model. Filter by absolute threshold or top-N per target for downstream analysis.
The if __name__ == '__main__': Requirement
Dask uses Python's multiprocessing module to spawn worker processes. Without the main guard, each spawned process re-executes the script, leading to infinite process spawning. Always wrap arboreto calls in if __name__ == '__main__': when running as a script. This is not needed inside Jupyter notebooks.
Common Recipes
Recipe: Distributed Computing on a Cluster
When to use: datasets too large for a single machine (e.g., 50k+ cells, 20k+ genes). Requires a Dask distributed scheduler running on the cluster.
# On head node: start scheduler
dask-scheduler
# Output: Scheduler at tcp://10.118.224.134:8786
# On each compute node: start workers
dask-worker tcp://10.118.224.134:8786 --nprocs 4 --nthreads 1 --memory-limit 16GB
from distributed import Client
from arboreto.algo import grnboost2
import pandas as pd
if __name__ == '__main__':
client = Client('tcp://10.118.224.134:8786')
print(f"Connected to cluster: {client.scheduler_info()['workers'].__len__()} workers")
# Example: Connected to cluster: 16 workers
expression_data = pd.read_csv('large_scrna.tsv', sep='\t')
tf_names = pd.read_csv('tf_list.txt', header=None)[0].tolist()
network = grnboost2(
expression_data=expression_data,
tf_names=tf_names,
client_or_address=client,
seed=42, verbose=True
)
network.to_csv('cluster_grn.tsv', sep='\t', index=False)
print(f"Cluster inference complete: {len(network)} links")
client.close()
Recipe: Consensus Network from Multiple Seeds
When to use: improve robustness by aggregating networks from multiple random seeds. Links appearing consistently across runs are more reliable.
import pandas as pd
from distributed import LocalCluster, Client
from arboreto.algo import grnboost2
if __name__ == '__main__':
client = Client(LocalCluster(n_workers=8, threads_per_worker=1))
expression_data = pd.read_csv('expression_data.tsv', sep='\t')
tf_names = pd.read_csv('tf_list.txt', header=None)[0].tolist()
seeds = [42, 123, 456, 789, 1001]
all_networks = []
for seed in seeds:
net = grnboost2(expression_data=expression_data,
tf_names=tf_names,
client_or_address=client, seed=seed)
net['seed'] = seed
all_networks.append(net)
print(f"Seed {seed}: {len(net)} links")
combined = pd.concat(all_networks, ignore_index=True)
# Consensus: mean importance across seeds, keep links found in 3+ runs
consensus = (combined.groupby(['TF', 'target'])
.agg(mean_importance=('importance', 'mean'),
n_seeds=('seed', 'nunique'))
.reset_index())
consensus = consensus[consensus['n_seeds'] >= 3]
consensus = consensus.sort_values('mean_importance', ascending=False)
consensus.to_csv('consensus_network.tsv', sep='\t', index=False)
print(f"Consensus network: {len(consensus)} links (found in 3+ of {len(seeds)} runs)")
# Example: Consensus network: 28453 links (found in 3+ of 5 runs)
client.close()
Recipe: pySCENIC Integration
When to use: downstream regulatory analysis after arboreto GRN inference. Arboreto produces the adjacency matrix (Step 1 of SCENIC); pySCENIC performs regulon identification (Step 2) and activity scoring (Step 3).
from arboreto.algo import grnboost2
from arboreto.utils import load_tf_names
import pandas as pd
if __name__ == '__main__':
# Step 1 (arboreto): GRN inference
expression_matrix = pd.read_csv('scrna_expression.tsv', sep='\t')
tf_names = load_tf_names('allTFs_hg38.txt')
adjacencies = grnboost2(
expression_data=expression_matrix,
tf_names=tf_names, seed=777
)
adjacencies.to_csv('adjacencies.tsv', sep='\t', index=False, header=False)
print(f"SCENIC Step 1 complete: {len(adjacencies)} adjacencies")
# Example: SCENIC Step 1 complete: 234567 adjacencies
# Steps 2-3 use pySCENIC CLI (requires pyscenic installation):
# pyscenic ctx adjacencies.tsv hg38_ranking.feather \
# --annotations_fname motifs-v10nr.hgnc-m0.001-o0.0.tbl \
# --output regulons.csv
#
# pyscenic aucell scrna_expression.tsv regulons.csv \
# --output auc_matrix.csv
print("Next: run pyscenic ctx (Step 2) and pyscenic aucell (Step 3)")
Expected Outputs
full_network.tsv-- complete TF-target-importance table; columns: TF, target, importancefiltered_network.tsv-- high-confidence subset after threshold or top-N filteringconsensus_network.tsv-- aggregated network from multi-seed runs; columns: TF, target, mean_importance, n_seedsadjacencies.tsv-- arboreto output formatted for pySCENIC input (no header)grn_network.png-- directed graph visualization of top regulatory interactions
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
RuntimeError: freeze_support() or infinite process spawning | Missing if __name__ == '__main__': guard | Wrap all arboreto calls in main guard; not needed in Jupyter |
MemoryError during inference | Expression matrix too large for available RAM | Filter low-variance genes first, reduce to top 5-10k genes, or use distributed cluster |
| Empty or near-empty network | TF names do not match expression matrix column names | Verify TF names overlap with expression_matrix.columns; check case sensitivity |
| Very slow inference | Using GENIE3 on large dataset, or too few Dask workers | Switch to GRNBoost2; create explicit Dask client with more workers |
ModuleNotFoundError: No module named 'arboreto' | Package not installed | pip install arboreto |
| All importance scores are 0 or identical | Expression matrix has no variance (e.g., all zeros after filtering) | Check data quality; ensure matrix contains non-zero expression values with variance |
TypeError with NumPy array input | Missing gene_names parameter | Provide gene_names= list matching the column count of the array |
Bundled Resources
This entry is fully self-contained with no references/ directory.
Original reference file dispositions:
- references/algorithms.md (139 lines) -- Algorithm comparison, parameter details, custom regressor kwargs. Consolidated into Key Concepts (Algorithm Selection Guide table) and Key Parameters. Custom regressor kwargs (
regressor_type,regressor_kwargs) omitted: advanced tuning rarely needed; users can pass scikit-learn kwargs directly per the arboreto API docs. - references/basic_inference.md (152 lines) -- Input formats, TF loading, basic workflow, output format. Fully consolidated into Workflow Steps 1-2, Key Concepts (Output Format table), and Quick Start.
- references/distributed_computing.md (242 lines) -- Local client, cluster setup, monitoring, performance tips. Consolidated into Workflow Step 3 and Recipe: Distributed Computing. Omitted: Dask dashboard panel descriptions (monitoring UI detail beyond scope), detailed network/storage tuning (cluster-admin concern).
- scripts/basic_grn_inference.py (98 lines) -- CLI wrapper with argparse. Core load-infer-save logic absorbed into Workflow Steps 1-6 and Quick Start. Argparse/CLI boilerplate stripped.
Narrative use-case dispositions (from original Common Use Cases):
- Single-cell RNA-seq analysis -- absorbed into When to Use bullets and Workflow Steps 1/4
- Bulk RNA-seq with TF filtering -- absorbed into Workflow Step 2 and Quick Start
- Comparative analysis (multiple conditions) -- omitted as standalone recipe: trivial loop over conditions, pattern shown in Consensus Network recipe
Related Skills
- scanpy-scrna-seq -- upstream single-cell preprocessing (QC, normalization, HVG selection) before GRN inference
- scvi-tools-single-cell -- deep generative models for batch correction and imputation prior to network inference
- pyscenic (planned) -- downstream regulon identification and cellular activity scoring using arboreto adjacencies
- networkx-graph-analysis -- graph-theoretic analysis of inferred regulatory networks (centrality, communities)
References
- Arboreto GitHub repository -- source code, README, examples
- Arboreto PyPI -- installation and version history
- Moerman et al. (2019) "GRNBoost2 and Arboreto: efficient and scalable inference of gene regulatory networks" Bioinformatics 35(12):2159-2161 -- algorithm paper
- Aibar et al. (2017) "SCENIC: single-cell regulatory network inference and clustering" Nature Methods 14:1083-1086 -- SCENIC pipeline paper
- Dask distributed documentation -- Dask client, cluster setup, dashboard
Biopython: Computational Molecular Biology Toolkit
Overview
Biopython is the standard open-source Python library for computational molecular biology, providing modular APIs for sequence handling, biological file parsing, NCBI database access, BLAST searches, protein structure analysis, and phylogenetics. It supports Python 3 and requires NumPy.
When to Use
- Parse and convert biological file formats (FASTA, GenBank, FASTQ, PDB, mmCIF, PHYLIP)
- Fetch sequences or publications from NCBI databases (GenBank, PubMed, Protein) programmatically
- Run and parse BLAST searches (remote NCBI or local BLAST+)
- Perform pairwise or multiple sequence alignments with custom scoring
- Analyze 3D protein structures — distances, angles, DSSP, superimposition
- Build and visualize phylogenetic trees from sequence alignments
- Calculate sequence statistics (GC content, molecular weight, melting temperature)
- Batch-process thousands of sequences with custom filtering logic
- Use
pysaminstead for reading SAM/BAM/CRAM alignment files and working with mapped reads; usescikit-bioinstead for advanced ecological diversity metrics
Prerequisites
- Python packages:
biopython,numpy,matplotlib(for tree visualization) - Data requirements: Sequence files (FASTA, GenBank, FASTQ) or accession IDs for NCBI access
- Environment: Python 3.8+; NCBI Entrez requires email registration
pip install biopython numpy matplotlib
Quick Start
from Bio import SeqIO
from Bio.Seq import Seq
from Bio.SeqUtils import gc_fraction
# Parse a FASTA file and compute basic statistics
records = list(SeqIO.parse("sequences.fasta", "fasta"))
print(f"Sequences loaded: {len(records)}")
seq = records[0].seq
print(f"ID: {records[0].id}")
print(f"Length: {len(seq)} bp")
print(f"GC content: {gc_fraction(seq)*100:.1f}%")
print(f"Reverse complement: {seq.reverse_complement()[:30]}...")
print(f"Protein translation: {seq.translate()[:10]}...")
Core API
Module 1: Sequence Objects (Bio.Seq)
Create and manipulate DNA, RNA, and protein sequences.
from Bio.Seq import Seq
# Create sequence and perform standard operations
dna = Seq("ATGGCCATTGTAATGGGCCGCTGAAAGGGTGCCCGATAG")
print(f"Length: {len(dna)} bp")
print(f"Complement: {dna.complement()}")
print(f"Reverse complement: {dna.reverse_complement()}")
print(f"Transcription: {dna.transcribe()}")
print(f"Translation: {dna.translate()}")
print(f"Translation (to stop): {dna.translate(to_stop=True)}")
# Length: 39 bp
# Translation: MAIVMGR*KGAR*
# Translation (to stop): MAIVMGR
from Bio.Seq import Seq
# Alternative genetic codes (e.g., mitochondrial)
mito_dna = Seq("ATGGCCATTGTAATGGGCCGCTGA")
std_protein = mito_dna.translate(table=1) # Standard
mito_protein = mito_dna.translate(table=2) # Vertebrate mitochondrial
print(f"Standard: {std_protein}")
print(f"Mitochondrial: {mito_protein}")
# Find all start codons
coding_dna = Seq("ATGAAACCCATGGGGTTTAAATAG")
positions = [i for i in range(len(coding_dna) - 2) if coding_dna[i:i+3] == "ATG"]
print(f"ATG positions: {positions}")
# ATG positions: [0, 9]
Module 2: Sequence I/O (Bio.SeqIO)
Read, write, and convert biological file formats.
from Bio import SeqIO
# Parse FASTA file — returns SeqRecord iterator
records = list(SeqIO.parse("sequences.fasta", "fasta"))
print(f"Loaded {len(records)} sequences")
for rec in records[:3]:
print(f" {rec.id}: {len(rec.seq)} bp — {rec.description}")
# Parse GenBank — rich annotation access
for rec in SeqIO.parse("genome.gb", "genbank"):
print(f"{rec.id}: {len(rec.features)} features, {len(rec.seq)} bp")
for feat in rec.features[:5]:
print(f" {feat.type}: {feat.location}")
# Convert between formats
count = SeqIO.convert("input.gb", "genbank", "output.fasta", "fasta")
print(f"Converted {count} records: GenBank → FASTA")
from Bio import SeqIO
from Bio.SeqRecord import SeqRecord
from Bio.Seq import Seq
# Write sequences to file
records = [
SeqRecord(Seq("ATCGATCG"), id="seq1", description="test sequence 1"),
SeqRecord(Seq("GCTAGCTA"), id="seq2", description="test sequence 2"),
]
count = SeqIO.write(records, "output.fasta", "fasta")
print(f"Wrote {count} records to output.fasta")
# Filter sequences by length (streaming — memory efficient)
long_seqs = (rec for rec in SeqIO.parse("large_file.fasta", "fasta") if len(rec.seq) >= 200)
count = SeqIO.write(long_seqs, "filtered.fasta", "fasta")
print(f"Kept {count} sequences >= 200 bp")
# Index large FASTA for random access
idx = SeqIO.index("large_file.fasta", "fasta")
print(f"Indexed {len(idx)} sequences")
rec = idx["target_sequence_id"]
print(f"Retrieved: {rec.id}, {len(rec.seq)} bp")
Module 3: NCBI Database Access (Bio.Entrez)
Programmatic search and download from NCBI databases.
from Bio import Entrez, SeqIO
Entrez.email = "your.email@example.com"
# Entrez.api_key = "your_key" # Optional: 10 req/s instead of 3 req/s
# Search PubMed
handle = Entrez.esearch(db="pubmed", term="CRISPR Cas9 2024", retmax=5)
results = Entrez.read(handle)
handle.close()
print(f"Found {results['Count']} articles, retrieved {len(results['IdList'])} IDs")
print(f"IDs: {results['IdList']}")
# Fetch GenBank record by accession
handle = Entrez.efetch(db="nucleotide", id="EU490707", rettype="gb", retmode="text")
record = SeqIO.read(handle, "genbank")
handle.close()
print(f"{record.id}: {record.description}")
print(f"Length: {len(record.seq)} bp, Features: {len(record.features)}")
from Bio import Entrez
import time
Entrez.email = "your.email@example.com"
# Batch download with rate limiting
handle = Entrez.esearch(db="protein", term="insulin[Protein Name] AND human[Organism]", retmax=20)
results = Entrez.read(handle)
handle.close()
# Fetch summaries in batch
ids = results["IdList"][:10]
handle = Entrez.esummary(db="protein", id=",".join(ids))
summaries = Entrez.read(handle)
handle.close()
for doc in summaries:
print(f" {doc['AccessionVersion']}: {doc['Title'][:60]}...")
print(f"\nFetched {len(summaries)} protein summaries")
Module 4: BLAST Operations (Bio.Blast)
Run and parse BLAST searches against NCBI or local databases.
from Bio.Blast import NCBIWWW, NCBIXML
# Remote BLAST search (nucleotide)
query_seq = "ATCGATCGATCGATCGATCGATCGATCGATCG"
result_handle = NCBIWWW.qblast("blastn", "nt", query_seq, hitlist_size=5)
blast_record = NCBIXML.read(result_handle)
result_handle.close()
print(f"Query: {blast_record.query[:50]}")
print(f"Database: {blast_record.database}")
print(f"Hits: {len(blast_record.alignments)}")
for aln in blast_record.alignments[:3]:
hsp = aln.hsps[0]
print(f"\n {aln.title[:60]}...")
print(f" E-value: {hsp.expect:.2e}, Identity: {hsp.identities}/{hsp.align_length}")
print(f" Score: {hsp.score}, Bits: {hsp.bits:.1f}")
from Bio.Blast.Applications import NcbiblastpCommandline
from Bio.Blast import NCBIXML
# Local BLAST (requires BLAST+ installed)
blastp_cline = NcbiblastpCommandline(
query="query.fasta",
db="swissprot",
evalue=1e-5,
outfmt=5, # XML output
out="blast_results.xml",
num_threads=4,
)
print(f"Command: {blastp_cline}")
# stdout, stderr = blastp_cline() # Execute
# Parse local BLAST XML results
with open("blast_results.xml") as f:
for record in NCBIXML.parse(f):
print(f"Query: {record.query}")
for aln in record.alignments[:3]:
print(f" Hit: {aln.hit_def[:50]}, E={aln.hsps[0].expect:.2e}")
Module 5: Pairwise Alignment (Bio.Align)
Global and local pairwise sequence alignment with customizable scoring.
from Bio import Align
# Global alignment
aligner = Align.PairwiseAligner()
aligner.mode = "global"
aligner.match_score = 2
aligner.mismatch_score = -1
aligner.open_gap_score = -5
aligner.extend_gap_score = -0.5
alignments = aligner.align("ACCGGTAACG", "ACGGTAAC")
print(f"Score: {alignments.score}")
print(f"Number of alignments: {len(alignments)}")
print(f"Best alignment:\n{alignments[0]}")
from Bio import Align
from Bio.Align import substitution_matrices
# Protein alignment with BLOSUM62
aligner = Align.PairwiseAligner()
aligner.mode = "local"
aligner.substitution_matrix = substitution_matrices.load("BLOSUM62")
aligner.open_gap_score = -10
aligner.extend_gap_score = -0.5
seq1 = "MVLSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTKTYFPHFDLSH"
seq2 = "MVHLTPEEKSAVTALWGKVNVDEVGGEALGRLLVVYPWTQRFFESFGDLST"
alignments = aligner.align(seq1, seq2)
print(f"Score: {alignments.score}")
print(f"Best alignment:\n{alignments[0]}")
Module 6: Protein Structure Analysis (Bio.PDB)
Parse PDB/mmCIF files and analyze 3D protein structures.
from Bio.PDB import PDBParser, PPBuilder
# Parse PDB structure
parser = PDBParser(QUIET=True)
structure = parser.get_structure("1CRN", "1crn.pdb")
# Navigate SMCRA hierarchy: Structure > Model > Chain > Residue > Atom
model = structure[0]
for chain in model:
residues = list(chain.get_residues())
print(f"Chain {chain.id}: {len(residues)} residues")
# Extract sequence from structure
ppb = PPBuilder()
for pp in ppb.build_peptides(structure):
print(f"Peptide: {pp.get_sequence()[:50]}... ({len(pp.get_sequence())} aa)")
# Calculate CA-CA distance
chain_a = model["A"]
ca1 = chain_a[10]["CA"]
ca2 = chain_a[20]["CA"]
distance = ca1 - ca2
print(f"CA distance (res 10-20): {distance:.2f} Angstrom")
from Bio.PDB import PDBParser, Superimposer
import numpy as np
# Structure superimposition (RMSD calculation)
parser = PDBParser(QUIET=True)
struct1 = parser.get_structure("s1", "structure1.pdb")
struct2 = parser.get_structure("s2", "structure2.pdb")
# Get CA atoms for alignment
atoms1 = [res["CA"] for res in struct1[0]["A"].get_residues() if "CA" in res]
atoms2 = [res["CA"] for res in struct2[0]["A"].get_residues() if "CA" in res]
# Superimpose (requires same number of atoms)
n = min(len(atoms1), len(atoms2))
sup = Superimposer()
sup.set_atoms(atoms1[:n], atoms2[:n])
sup.apply(struct2.get_atoms())
print(f"RMSD: {sup.rms:.3f} Angstrom over {n} CA atoms")
Module 7: Phylogenetics (Bio.Phylo)
Build, manipulate, and visualize phylogenetic trees.
from Bio import Phylo, AlignIO
from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor
import io
# Build tree from multiple sequence alignment
alignment = AlignIO.read("aligned_sequences.fasta", "fasta")
print(f"Alignment: {len(alignment)} sequences, {alignment.get_alignment_length()} positions")
# Calculate distance matrix and build NJ tree
calculator = DistanceCalculator("identity")
dm = calculator.get_distance(alignment)
print(f"Distance matrix:\n{dm}")
constructor = DistanceTreeConstructor()
nj_tree = constructor.nj(dm)
upgma_tree = constructor.upgma(dm)
# Visualize
Phylo.draw_ascii(nj_tree)
# Save tree
Phylo.write(nj_tree, "tree.nwk", "newick")
print("Saved tree.nwk")
Module 8: Sequence Utilities (Bio.SeqUtils)
Compute sequence statistics and physicochemical properties.
from Bio.Seq import Seq
from Bio.SeqUtils import gc_fraction, molecular_weight, MeltingTemp
dna = Seq("ATCGATCGATCGATCGATCG")
print(f"GC content: {gc_fraction(dna):.2%}")
print(f"Molecular weight: {molecular_weight(dna, seq_type='DNA'):.2f} Da")
print(f"Melting temp (basic): {MeltingTemp.Tm_Wallace(dna):.1f} C")
print(f"Melting temp (NN): {MeltingTemp.Tm_NN(dna):.1f} C")
# Protein analysis
from Bio.SeqUtils.ProtParam import ProteinAnalysis
protein = ProteinAnalysis("MVLSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTK")
print(f"\nProtein MW: {protein.molecular_weight():.2f} Da")
print(f"Isoelectric point: {protein.isoelectric_point():.2f}")
print(f"Aromaticity: {protein.aromaticity():.4f}")
print(f"Instability index: {protein.instability_index():.2f}")
print(f"GRAVY: {protein.gravy():.4f}")
aa_pct = protein.get_amino_acids_percent()
print(f"Top 3 amino acids: {sorted(aa_pct.items(), key=lambda x: -x[1])[:3]}")
Common Workflows
Workflow 1: Gene Sequence Retrieval and Analysis Pipeline
Goal: Fetch a gene from NCBI, analyze its properties, translate to protein, and compute statistics.
from Bio import Entrez, SeqIO
from Bio.Seq import Seq
from Bio.SeqUtils import gc_fraction
from Bio.SeqUtils.ProtParam import ProteinAnalysis
Entrez.email = "your.email@example.com"
# Step 1: Fetch gene from GenBank
handle = Entrez.efetch(db="nucleotide", id="NM_007294.4", rettype="gb", retmode="text")
record = SeqIO.read(handle, "genbank")
handle.close()
print(f"Gene: {record.description}")
print(f"Length: {len(record.seq)} bp")
# Step 2: Extract CDS
cds_features = [f for f in record.features if f.type == "CDS"]
if cds_features:
cds = cds_features[0]
cds_seq = cds.location.extract(record).seq
print(f"CDS: {len(cds_seq)} bp, GC: {gc_fraction(cds_seq):.2%}")
# Step 3: Translate
protein_seq = cds_seq.translate(to_stop=True)
print(f"Protein: {len(protein_seq)} aa")
print(f"First 30 aa: {protein_seq[:30]}...")
# Step 4: Protein properties
analysis = ProteinAnalysis(str(protein_seq))
print(f"MW: {analysis.molecular_weight():.0f} Da")
print(f"pI: {analysis.isoelectric_point():.2f}")
print(f"Instability: {analysis.instability_index():.1f}")
print(f"GRAVY: {analysis.gravy():.3f}")
Workflow 2: Comparative Sequence Analysis with BLAST and Phylogeny
Goal: BLAST a protein sequence, fetch homologs, align, and build a phylogenetic tree.
from Bio.Blast import NCBIWWW, NCBIXML
from Bio import Entrez, SeqIO, AlignIO, Phylo
from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor
from Bio.Align.Applications import MuscleCommandline
import time
Entrez.email = "your.email@example.com"
# Step 1: BLAST search
query = "MVLSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTKTYFPHFDLSH"
result_handle = NCBIWWW.qblast("blastp", "swissprot", query, hitlist_size=10)
blast_record = NCBIXML.read(result_handle)
print(f"BLAST hits: {len(blast_record.alignments)}")
# Step 2: Collect homolog accessions
accessions = []
for aln in blast_record.alignments[:8]:
acc = aln.accession
accessions.append(acc)
print(f" {acc}: E={aln.hsps[0].expect:.2e}, {aln.hit_def[:50]}...")
# Step 3: Fetch sequences and save for alignment
handle = Entrez.efetch(db="protein", id=",".join(accessions), rettype="fasta", retmode="text")
records = list(SeqIO.parse(handle, "fasta"))
handle.close()
SeqIO.write(records, "homologs.fasta", "fasta")
print(f"Saved {len(records)} homolog sequences")
# Step 4: Align (requires MUSCLE installed)
# muscle_cline = MuscleCommandline(input="homologs.fasta", out="aligned.fasta")
# muscle_cline()
# Step 5: Build phylogenetic tree from alignment
alignment = AlignIO.read("aligned.fasta", "fasta")
calculator = DistanceCalculator("blosum62")
dm = calculator.get_distance(alignment)
tree = DistanceTreeConstructor().nj(dm)
Phylo.draw_ascii(tree)
Phylo.write(tree, "homologs.nwk", "newick")
print("Saved phylogenetic tree to homologs.nwk")
Workflow 3: Batch Sequence Processing and Quality Filtering
Goal: Process a large FASTQ/FASTA dataset — filter by quality/length, compute statistics, and export.
from Bio import SeqIO
from Bio.SeqUtils import gc_fraction
import pandas as pd
# Step 1: Stream through large file and collect stats
stats = []
passed = []
for rec in SeqIO.parse("reads.fastq", "fastq"):
seq_len = len(rec.seq)
gc = gc_fraction(rec.seq)
avg_qual = sum(rec.letter_annotations["phred_quality"]) / seq_len
stats.append({"id": rec.id, "length": seq_len, "gc": gc, "avg_qual": avg_qual})
# Filter: length >= 100 and avg quality >= 20
if seq_len >= 100 and avg_qual >= 20:
passed.append(rec)
# Step 2: Summary statistics
df = pd.DataFrame(stats)
print(f"Total reads: {len(df)}")
print(f"Passed QC: {len(passed)} ({len(passed)/len(df)*100:.1f}%)")
print(f"\nLength: mean={df['length'].mean():.0f}, median={df['length'].median():.0f}")
print(f"GC: mean={df['gc'].mean():.2%}, std={df['gc'].std():.2%}")
print(f"Quality: mean={df['avg_qual'].mean():.1f}, min={df['avg_qual'].min():.1f}")
# Step 3: Export filtered reads
count = SeqIO.write(passed, "filtered_reads.fastq", "fastq")
print(f"\nExported {count} filtered reads to filtered_reads.fastq")
# Step 4: Save statistics
df.to_csv("read_statistics.csv", index=False)
print(f"Saved statistics to read_statistics.csv")
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
Seq.translate(table=) | Bio.Seq | 1 (Standard) | 1-33 | NCBI genetic code table for translation |
Seq.translate(to_stop=) | Bio.Seq | False | True, False | Stop at first stop codon vs translate entire sequence |
SeqIO.parse(format=) | Bio.SeqIO | — | "fasta", "genbank", "fastq", "phylip" | File format for parsing |
Entrez.email | Bio.Entrez | (required) | Valid email | NCBI requires email for tracking; set before any Entrez call |
Entrez.api_key | Bio.Entrez | None | NCBI API key string | Increases rate limit from 3 to 10 requests/second |
PairwiseAligner.mode | Bio.Align | "global" | "global", "local" | Needleman-Wunsch vs Smith-Waterman algorithm |
PairwiseAligner.open_gap_score | Bio.Align | -1 | -20 to 0 | Penalty for opening a gap; more negative = fewer gaps |
PairwiseAligner.substitution_matrix | Bio.Align | None | "BLOSUM62", "BLOSUM45", "PAM250" | Scoring matrix for protein alignment |
PDBParser(QUIET=) | Bio.PDB | False | True, False | Suppress parser warnings for non-standard PDB files |
DistanceCalculator(model=) | Bio.Phylo | "identity" | "identity", "blosum62" | Distance model for tree construction |
Common Recipes
Recipe: Restriction Enzyme Analysis
When to use: Find restriction sites in a DNA sequence for cloning design.
from Bio.Seq import Seq
from Bio.Restriction import EcoRI, BamHI, HindIII, RestrictionBatch, Analysis
seq = Seq("GAATTCAAAGGATCCTTTTAAGCTTGGGAATTC")
# Single enzyme
print(f"EcoRI cuts at: {EcoRI.search(seq)}")
print(f"BamHI cuts at: {BamHI.search(seq)}")
# Batch analysis
batch = RestrictionBatch([EcoRI, BamHI, HindIII])
analysis = Analysis(batch, seq)
result = analysis.full()
for enzyme, sites in result.items():
if sites:
print(f"{enzyme}: cuts at positions {sites}")
Recipe: Motif Discovery and Position Weight Matrix
When to use: Analyze transcription factor binding sites or consensus patterns.
from Bio import motifs
from Bio.Seq import Seq
# Create motif from observed binding sites
instances = [
Seq("TACGAT"),
Seq("TAGCAT"),
Seq("TACGGT"),
Seq("TAGCAT"),
Seq("TACGAT"),
]
m = motifs.create(instances)
print(f"Consensus: {m.consensus}")
print(f"Degenerate: {m.degenerate_consensus}")
print(f"\nPosition Weight Matrix:")
print(m.counts)
# Score a new sequence against the motif
pwm = m.counts.normalize(pseudocounts=0.5)
pssm = pwm.log_odds()
test_seq = Seq("AATACGATCCC")
for pos, score in pssm.search(test_seq, threshold=0.0):
print(f" Position {pos}: score={score:.2f}")
Recipe: GenomeDiagram — Visualize Genomic Features
When to use: Create a circular or linear genome map from a GenBank file.
from Bio import SeqIO
from Bio.Graphics import GenomeDiagram
from reportlab.lib import colors
from reportlab.lib.units import cm
# Load annotated genome
record = SeqIO.read("plasmid.gb", "genbank")
# Create diagram
diagram = GenomeDiagram.Diagram(record.name)
track = diagram.new_track(1, name="Annotated Features", greytrack=True)
feature_set = track.new_set()
# Color features by type
color_map = {"CDS": colors.blue, "gene": colors.green, "promoter": colors.red}
for feature in record.features:
if feature.type in color_map:
feature_set.add_feature(feature, color=color_map[feature.type], label=True, label_size=8)
# Draw circular map
diagram.draw(format="circular", circular=True, pagesize=(20*cm, 20*cm), start=0, end=len(record))
diagram.write("genome_map.pdf", "PDF")
print(f"Saved genome_map.pdf ({len(record.features)} features)")
Recipe: Batch PubMed Literature Search
When to use: Programmatically search and download publication metadata.
from Bio import Entrez
import time
Entrez.email = "your.email@example.com"
# Search PubMed with complex query
query = "(CRISPR[Title]) AND (2024[Date - Publication]) AND (review[Publication Type])"
handle = Entrez.esearch(db="pubmed", term=query, retmax=100)
results = Entrez.read(handle)
handle.close()
print(f"Found {results['Count']} articles")
# Fetch abstracts in batches
ids = results["IdList"]
batch_size = 20
articles = []
for i in range(0, len(ids), batch_size):
batch = ids[i:i+batch_size]
handle = Entrez.efetch(db="pubmed", id=",".join(batch), rettype="xml")
records = Entrez.read(handle)
handle.close()
for article in records["PubmedArticle"]:
info = article["MedlineCitation"]["Article"]
title = info.get("ArticleTitle", "N/A")
abstract = info.get("Abstract", {}).get("AbstractText", ["N/A"])[0]
articles.append({"title": title, "abstract": str(abstract)[:200]})
time.sleep(0.4) # Rate limiting
for a in articles[:5]:
print(f" {a['title'][:70]}...")
print(f"\nRetrieved {len(articles)} articles")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
HTTPError 400 from Entrez | Invalid accession/ID or malformed query | Validate accessions; check query syntax with NCBI web interface first |
HTTPError 429 (Too Many Requests) | Exceeding NCBI rate limit (3 req/s) | Set Entrez.api_key for 10 req/s; add time.sleep(0.4) between calls |
ValueError: No records found in SeqIO.read() | Empty file or wrong format string | Use SeqIO.parse() to check if file has records; verify format matches actual content |
Bio.PDB.PDBExceptions.PDBConstructionWarning | Non-standard atoms or occupancy issues | Use PDBParser(QUIET=True) or fix PDB with pdb-tools; check for alternate conformations |
| BLAST search times out | Large query or busy NCBI servers | Use local BLAST+ for large-scale searches; set NCBIWWW.qblast(hitlist_size=N) to limit results |
Alignment has sequences of different lengths | Unaligned sequences passed to AlignIO | Align sequences first with MUSCLE/Clustal before loading as alignment |
SeqIO.index() raises ValueError | Duplicate IDs in FASTA file | Deduplicate IDs with SeqIO.to_dict() or pre-process with awk |
ImportError: No module named Bio | Biopython not installed in active environment | pip install biopython; verify with python -c "import Bio; print(Bio.__version__)" |
translate() gives unexpected * | Stop codons in middle of sequence | Check reading frame; use Seq.translate(table=N) with correct genetic code |
| GenomeDiagram blank output | No features matched filter criteria | Check feature.type values in your GenBank file; print types to debug |
References
- Biopython Tutorial and Cookbook — Official comprehensive tutorial
- Biopython API Documentation — Complete API reference
- Cock, P.J.A. et al. (2009) Bioinformatics 25:1422-1423 — Original Biopython paper
- NCBI Entrez Programming Utilities — E-utilities documentation for programmatic NCBI access
- Biopython GitHub Repository — Source code, issues, and release notes
Biopython: Sequence Analysis Toolkit
Overview
Biopython provides a comprehensive suite of modules for sequence-centric bioinformatics: reading and writing every major biological file format (FASTA, FASTQ, GenBank, GFF), querying NCBI databases programmatically, running BLAST searches and parsing results, aligning sequences pairwise or in multiple-sequence alignments, and building and visualizing phylogenetic trees. This skill focuses on analysis workflows — from NCBI data retrieval through alignment to phylogenetic inference.
For PCR primer design, restriction enzyme digestion, cloning simulation, protein structure analysis (Bio.PDB), and molecular weight/Tm calculations, see biopython-molecular-biology.
When to Use
- Download a gene family from NCBI Nucleotide/Protein, align sequences, and construct a phylogenetic tree
- Parse GenBank or GFF3 annotation files and extract CDS sequences for a set of features
- Run a BLAST search against NCBI
ntornr, filter significant hits, and fetch their full sequences - Compute pairwise sequence identities or score alignments with BLOSUM62/PAM250 matrices
- Index a large multi-FASTA or FASTQ file with
SeqIO.index()for random-access retrieval without loading all sequences into RAM - Convert between sequence formats (FASTA ↔ GenBank ↔ FASTQ ↔ PHYLIP) in a single call
- Traverse, root, prune, and annotate a Newick or Nexus phylogenetic tree programmatically
- Use pysam instead when working with SAM/BAM/CRAM alignment files and mapped reads
- Use scikit-bio instead when you need ecological diversity metrics (UniFrac, beta diversity) or ordination methods such as PCoA
- Use gget instead for quick gene lookups and one-liner NCBI queries without writing an Entrez pipeline
Prerequisites
- Python packages:
biopython,numpy,matplotlib - Optional tools: MUSCLE or ClustalW installed locally (for
Bio.Align.Applicationswrappers) - NCBI access: Set
Entrez.emailbefore any E-utilities call; obtain a free API key at https://www.ncbi.nlm.nih.gov/account/ for 10 req/s (default is 3 req/s) - Local BLAST: BLAST+ installed separately (
conda install -c bioconda blast) for offline searches
Check before installing: The tool may already be available in the current environment (e.g., inside a
pixi/condaenv). Runcommand -v pythonfirst and skip the install commands below if it returns a path. When running inside a pixi project, invoke the tool viapixi run pythonrather than barepython.
pip install biopython numpy matplotlib
conda install -c bioconda blast # optional, for local BLAST
Quick Start
from Bio import SeqIO, Entrez
from Bio.SeqUtils import gc_fraction
# Fetch a GenBank record and display basic stats
Entrez.email = "your.email@example.com"
handle = Entrez.efetch(db="nucleotide", id="NM_007294", rettype="gb", retmode="text")
record = SeqIO.read(handle, "genbank")
handle.close()
print(f"ID: {record.id}")
print(f"Length: {len(record.seq)} bp")
print(f"GC content: {gc_fraction(record.seq)*100:.1f}%")
print(f"Features: {len(record.features)}")
print(f"First feature: {record.features[0].type} at {record.features[0].location}")
# ID: NM_007294.4
# Length: 7207 bp
# GC content: 47.3%
# Features: 9
Core API
Module 1: SeqIO — File Parsing and Format Conversion
Bio.SeqIO reads and writes every major sequence format (FASTA, FASTQ, GenBank, EMBL, PHYLIP, Nexus). SeqIO.parse() returns an iterator of SeqRecord objects; SeqIO.index() builds an on-disk or in-memory dictionary for large files.
from Bio import SeqIO
# Parse FASTA and FASTQ
fasta_records = list(SeqIO.parse("sequences.fasta", "fasta"))
print(f"FASTA: {len(fasta_records)} sequences")
# FASTQ: access quality scores
for rec in SeqIO.parse("reads.fastq", "fastq"):
quals = rec.letter_annotations["phred_quality"]
avg_q = sum(quals) / len(quals)
print(f" {rec.id}: {len(rec.seq)} bp, mean Q={avg_q:.1f}")
break # show one example
# Parse GenBank with feature access
for rec in SeqIO.parse("chromosome.gb", "genbank"):
cdss = [f for f in rec.features if f.type == "CDS"]
print(f"{rec.id}: {len(cdss)} CDS features")
for cds in cdss[:3]:
gene = cds.qualifiers.get("gene", ["unknown"])[0]
print(f" {gene}: {cds.location}")
from Bio import SeqIO
# SeqIO.index() for random access without loading all records
# Useful for large reference FASTA files (genomes, nr database subsets)
idx = SeqIO.index("large_genome.fasta", "fasta")
print(f"Index contains {len(idx)} sequences")
# Retrieve specific sequences by ID in O(1)
target = idx["chr1"]
print(f"chr1: {len(target.seq):,} bp")
region = target.seq[1_000_000:1_001_000]
print(f"Region [1M-1M+1kb]: {region[:60]}...")
# Format conversion: GenBank → FASTA in one call
n = SeqIO.convert("annotation.gb", "genbank", "sequences.fasta", "fasta")
print(f"Converted {n} records to FASTA")
Module 2: Seq and SeqRecord — Sequence Objects and Feature Annotations
Bio.Seq represents a biological sequence with in-place operations. Bio.SeqRecord wraps a Seq with an ID, description, and a dictionary of feature annotations. Features use FeatureLocation with strand (+1, -1).
from Bio.Seq import Seq
from Bio.SeqRecord import SeqRecord
from Bio.SeqFeature import SeqFeature, FeatureLocation
from Bio.SeqUtils import gc_fraction, MeltingTemp
dna = Seq("ATGAAACCCGGGTTTTAA")
print(f"GC content: {gc_fraction(dna)*100:.1f}%")
print(f"Reverse complement: {dna.reverse_complement()}")
print(f"Transcript: {dna.transcribe()}")
print(f"Translation: {dna.translate()}")
print(f"Translation (to stop): {dna.translate(to_stop=True)}")
# Tm calculation for a short primer
primer = Seq("ATGAAACCCGGG")
tm = MeltingTemp.Tm_Wallace(primer)
print(f"Tm (Wallace): {tm:.1f}°C")
from Bio.Seq import Seq
from Bio.SeqRecord import SeqRecord
from Bio.SeqFeature import SeqFeature, FeatureLocation
# Build a SeqRecord with annotated features
gene_seq = Seq("ATGAAACCCGGGTTTTAAATCGATCG" * 10)
record = SeqRecord(gene_seq, id="MY_GENE_001", description="synthetic example gene")
# Annotate a CDS feature
cds = SeqFeature(
FeatureLocation(0, 18, strand=+1),
type="CDS",
qualifiers={"gene": ["myGene"], "product": ["hypothetical protein"]}
)
record.features.append(cds)
# Extract and translate the feature
cds_seq = cds.location.extract(record.seq)
protein = cds_seq.translate(to_stop=True)
print(f"CDS: {cds_seq}")
print(f"Protein: {protein}")
# Slice record preserving feature annotations
sub = record[0:60]
print(f"Subrecord: {len(sub.seq)} bp, {len(sub.features)} features")
Module 3: Entrez — Programmatic NCBI Database Access
Bio.Entrez wraps the NCBI E-utilities API: esearch finds records matching a query, efetch retrieves full records, elink finds related records across databases, and esummary returns document summaries. Always set Entrez.email and respect the 3 req/s rate limit (10 req/s with API key).
from Bio import Entrez, SeqIO
import time
Entrez.email = "your.email@example.com"
# Entrez.api_key = "YOUR_API_KEY" # for 10 req/s
# esearch: find IDs matching a query
handle = Entrez.esearch(db="nucleotide", term="BRCA1[Gene] AND Homo sapiens[Organism] AND mRNA[Filter]", retmax=10)
search_results = Entrez.read(handle)
handle.close()
print(f"Total hits: {search_results['Count']}")
ids = search_results["IdList"]
print(f"Retrieved IDs: {ids}")
# efetch: download full GenBank records
for acc_id in ids[:3]:
handle = Entrez.efetch(db="nucleotide", id=acc_id, rettype="gb", retmode="text")
record = SeqIO.read(handle, "genbank")
handle.close()
print(f" {record.id}: {len(record.seq)} bp — {record.description[:60]}")
time.sleep(0.4) # stay within rate limit
from Bio import Entrez
import time
Entrez.email = "your.email@example.com"
# elink: cross-database links (e.g., PubMed article → related nucleotide sequences)
handle = Entrez.elink(dbfrom="pubmed", db="nucleotide", id="29087512")
link_results = Entrez.read(handle)
handle.close()
linked_ids = []
for linkset in link_results:
for db_links in linkset.get("LinkSetDb", []):
if db_links["DbTo"] == "nucleotide":
linked_ids = [lnk["Id"] for lnk in db_links["Link"]]
break
print(f"Nucleotide sequences linked to PubMed 29087512: {len(linked_ids)}")
print(f"First IDs: {linked_ids[:5]}")
# esummary: lightweight metadata without downloading full records
if linked_ids:
handle = Entrez.esummary(db="nucleotide", id=",".join(linked_ids[:5]))
summaries = Entrez.read(handle)
handle.close()
for doc in summaries:
print(f" {doc['AccessionVersion']}: {doc['Title'][:70]}")
Module 4: BLAST — Remote and Local Sequence Similarity Search
Bio.Blast.NCBIWWW.qblast() submits queries to NCBI BLAST servers and returns XML handles. Bio.Blast.NCBIXML.parse() (or .read() for single queries) yields Blast objects with alignments and hsps. For large-scale searches, use local BLAST+ via subprocess.
from Bio.Blast import NCBIWWW, NCBIXML
from Bio.Seq import Seq
# Remote BLASTP against Swiss-Prot (small, reviewed database)
query = Seq("MTEYKLVVVGAGGVGKSALTIQLIQNHFVDEYDPTIEDSY")
result_handle = NCBIWWW.qblast("blastp", "swissprot", str(query), hitlist_size=10)
# Parse results
blast_record = NCBIXML.read(result_handle)
print(f"Query: {blast_record.query}")
print(f"Database: {blast_record.database}")
print(f"Hits: {len(blast_record.alignments)}")
E_VALUE_THRESH = 1e-5
for alignment in blast_record.alignments[:5]:
for hsp in alignment.hsps:
if hsp.expect < E_VALUE_THRESH:
identity_pct = hsp.identities / hsp.align_length * 100
print(f"\n Hit: {alignment.title[:70]}")
print(f" E-value: {hsp.expect:.2e}, Identity: {identity_pct:.1f}%, Score: {hsp.score}")
print(f" Query: {hsp.query[:60]}")
print(f" Match: {hsp.match[:60]}")
print(f" Sbjct: {hsp.sbjct[:60]}")
import subprocess
from Bio.Blast import NCBIXML
from Bio import SeqIO
# Local BLAST+ via subprocess (faster for large batches)
# Requires: makeblastdb and blastp installed (conda install -c bioconda blast)
# Step 1: Build a local database from a FASTA file
subprocess.run(
["makeblastdb", "-in", "ref_proteins.fasta", "-dbtype", "prot", "-out", "ref_db"],
check=True
)
# Step 2: Run blastp against local database
result = subprocess.run(
["blastp", "-query", "query.fasta", "-db", "ref_db",
"-outfmt", "5", # XML output for NCBIXML parsing
"-evalue", "1e-5",
"-num_threads", "4",
"-out", "blast_results.xml"],
check=True
)
# Step 3: Parse XML results
with open("blast_results.xml") as fh:
for blast_rec in NCBIXML.parse(fh):
print(f"Query: {blast_rec.query_id}")
for aln in blast_rec.alignments[:3]:
hsp = aln.hsps[0]
print(f" Hit: {aln.hit_id}, E={hsp.expect:.2e}, Id={hsp.identities}/{hsp.align_length}")
Module 5: Phylo — Tree Parsing, Manipulation, and Visualization
Bio.Phylo reads Newick, Nexus, PhyloXML, and NeXML formats. Trees are represented as Clade objects with nested children. Key methods: find_clades() (generator over nodes), common_ancestor(), distance(), root_with_outgroup(), and prune(). Visualization uses matplotlib.
from Bio import Phylo
import io
# Parse a Newick tree from a string
newick = "((Homo_sapiens:0.01, Pan_troglodytes:0.012):0.08, (Mus_musculus:0.15, Rattus_norvegicus:0.14):0.12, Drosophila_melanogaster:0.85);"
tree = Phylo.read(io.StringIO(newick), "newick")
print(f"Number of terminals: {tree.count_terminals()}")
print(f"Total branch length: {tree.total_branch_length():.3f}")
print(f"Terminals: {[t.name for t in tree.get_terminals()]}")
# Root with outgroup and compute distances
tree.root_with_outgroup("Drosophila_melanogaster")
human = tree.find_any("Homo_sapiens")
chimp = tree.find_any("Pan_troglodytes")
mouse = tree.find_any("Mus_musculus")
print(f"\nDistance Human–Chimp: {tree.distance(human, chimp):.4f}")
print(f"Distance Human–Mouse: {tree.distance(human, mouse):.4f}")
# Traverse all internal nodes
for clade in tree.find_clades(order="level"):
if not clade.is_terminal():
children = [c.name or "internal" for c in clade.clades]
print(f" Internal node → {children}")
from Bio import Phylo
import matplotlib.pyplot as plt
import io
newick = "((Homo_sapiens:0.01, Pan_troglodytes:0.012):0.08, (Mus_musculus:0.15, Rattus_norvegicus:0.14):0.12, Drosophila_melanogaster:0.85);"
tree = Phylo.read(io.StringIO(newick), "newick")
tree.root_with_outgroup("Drosophila_melanogaster")
tree.ladderize() # sort clades by size for a clean layout
# Annotate bootstrap support (if present in node labels)
for clade in tree.find_clades():
if clade.confidence is not None:
clade.name = f"{clade.confidence:.0f}"
fig, ax = plt.subplots(figsize=(8, 5))
Phylo.draw(tree, axes=ax, do_show=False, show_confidence=True)
ax.set_title("Primate + Outgroup Phylogeny")
plt.tight_layout()
plt.savefig("phylogeny.png", dpi=150, bbox_inches="tight")
print("Saved phylogeny.png")
Module 6: PairwiseAligner — Pairwise Sequence Alignment
Bio.Align.PairwiseAligner replaces the legacy pairwise2 module (deprecated since Biopython 1.80). It supports local and global alignment, gap open/extend penalties, and any substitution matrix from Bio.Align.substitution_matrices.
from Bio.Align import PairwiseAligner, substitution_matrices
# Global protein alignment with BLOSUM62
aligner = PairwiseAligner()
aligner.mode = "global"
aligner.substitution_matrix = substitution_matrices.load("BLOSUM62")
aligner.open_gap_score = -11
aligner.extend_gap_score = -1
seq1 = "MTEYKLVVVGAGGVGKSALTIQLIQNHFVDEYDPTIEDSY"
seq2 = "MTEYKLVVVGAVGVGKSALTIQLIQNHFVDEYDPTIEDSY" # G12V variant
alignments = aligner.align(seq1, seq2)
best = alignments[0]
print(f"Score: {best.score:.1f}")
print(f"Number of alignments: {aligner.score(seq1, seq2):.0f} (score only, faster)")
print(best) # formatted alignment string
identity = sum(a == b for a, b in zip(*best) if a != "-") / best.shape[1]
print(f"Identity: {identity*100:.1f}%")
from Bio.Align import PairwiseAligner, substitution_matrices
# Local DNA alignment
aligner = PairwiseAligner()
aligner.mode = "local"
aligner.match_score = 2
aligner.mismatch_score = -1
aligner.open_gap_score = -5
aligner.extend_gap_score = -0.5
query = "ACGTACGTACGT"
subject = "TTTTACGTACGTACGTTTTT"
alignments = list(aligner.align(query, subject))
print(f"Local alignments found: {len(alignments)}")
best = alignments[0]
print(f"Score: {best.score}")
print(f"Aligned region in subject: [{best.coordinates[1][0]}:{best.coordinates[1][-1]}]")
print(best)
# Batch pairwise identity matrix
seqs = ["ACGTACGT", "ACGTATGT", "TTGTACGT", "ACGTACGG"]
n = len(seqs)
aligner2 = PairwiseAligner(mode="global", match_score=1, mismatch_score=-1)
print("\nPairwise identity matrix:")
for i in range(n):
row = []
for j in range(n):
score = aligner2.score(seqs[i], seqs[j])
max_len = max(len(seqs[i]), len(seqs[j]))
row.append(f"{score/max_len:.2f}")
print(" " + " ".join(row))
Key Concepts
SeqRecord and Feature Coordinates
SeqRecord stores sequence metadata and a list of SeqFeature objects. Features use zero-based, half-open FeatureLocation(start, end, strand) — the same convention as Python slicing. Use feature.location.extract(record.seq) to obtain the strand-correct subsequence. Compound locations (e.g., spliced exons) use CompoundLocation.
from Bio import SeqIO
# Extract all CDS protein translations from a GenBank record
for rec in SeqIO.parse("gene.gb", "genbank"):
for feat in rec.features:
if feat.type == "CDS":
gene = feat.qualifiers.get("gene", ["?"])[0]
product = feat.qualifiers.get("product", ["unknown"])[0]
cds_nt = feat.location.extract(rec.seq)
protein = cds_nt.translate(to_stop=True)
print(f"{gene} ({product}): {len(protein)} aa — {protein[:10]}...")
Phylo Clade Objects
A Clade is both a node and the subtree rooted at that node. Terminal clades (leaves) have .name set; internal clades may have .confidence (bootstrap) and .branch_length. Use tree.find_clades() with terminal=True/False to filter. The root is tree.root (also a Clade). Phylo.read() returns a Tree wrapper; tree.root gives the root Clade.
Common Workflows
Workflow 1: Gene Family Phylogeny — NCBI Download → MUSCLE Alignment → Tree
Goal: Download all BRCA1 orthologs from Vertebrata, align with MUSCLE, and build a neighbor-joining tree.
import subprocess
import time
from Bio import Entrez, SeqIO, Phylo, AlignIO
from Bio.Align import MultipleSeqAlignment
import matplotlib.pyplot as plt
Entrez.email = "your.email@example.com"
# Step 1: Search NCBI Protein for BRCA1 orthologs
handle = Entrez.esearch(
db="protein",
term="BRCA1[Gene] AND Vertebrata[Organism] AND RefSeq[Filter]",
retmax=20
)
results = Entrez.read(handle); handle.close()
ids = results["IdList"]
print(f"Found {results['Count']} sequences, using {len(ids)}")
# Step 2: Fetch sequences in FASTA format
handle = Entrez.efetch(db="protein", id=",".join(ids), rettype="fasta", retmode="text")
with open("brca1_orthologs.fasta", "w") as fh:
fh.write(handle.read())
handle.close()
print(f"Saved {len(ids)} sequences to brca1_orthologs.fasta")
time.sleep(1)
# Step 3: Align with MUSCLE (must be installed: conda install -c bioconda muscle)
subprocess.run(
["muscle", "-align", "brca1_orthologs.fasta", "-output", "brca1_aligned.fasta"],
check=True
)
alignment = AlignIO.read("brca1_aligned.fasta", "fasta")
print(f"Alignment: {len(alignment)} sequences × {alignment.get_alignment_length()} columns")
# Step 4: Build a neighbor-joining tree using Bio.Phylo + distance matrix
from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor
calculator = DistanceCalculator("identity")
dm = calculator.get_distance(alignment)
constructor = DistanceTreeConstructor(calculator, method="nj")
tree = constructor.build_tree(alignment)
# Step 5: Root and save
tree.root_at_midpoint()
tree.ladderize()
Phylo.write(tree, "brca1_nj.nwk", "newick")
print("Saved NJ tree to brca1_nj.nwk")
# Step 6: Visualize
fig, ax = plt.subplots(figsize=(10, 8))
Phylo.draw(tree, axes=ax, do_show=False)
ax.set_title("BRCA1 Ortholog NJ Tree")
plt.tight_layout()
plt.savefig("brca1_tree.png", dpi=150, bbox_inches="tight")
print("Saved brca1_tree.png")
Workflow 2: BLAST Pipeline — FASTA Query → Remote BLAST → Fetch Top Hits
Goal: Take a query FASTA, BLAST against NCBI nr, filter significant hits, retrieve full GenBank records for the top matches, and write a summary CSV.
import csv
import time
from Bio import SeqIO, Entrez
from Bio.Blast import NCBIWWW, NCBIXML
Entrez.email = "your.email@example.com"
# Step 1: Load query sequence
query_record = SeqIO.read("query.fasta", "fasta")
print(f"Query: {query_record.id} ({len(query_record.seq)} aa)")
# Step 2: Remote BLAST against nr protein database
print("Submitting BLAST job (may take 1-5 minutes)...")
result_handle = NCBIWWW.qblast(
"blastp", "nr", str(query_record.seq),
hitlist_size=20,
expect=1e-5,
word_size=6,
matrix_name="BLOSUM62"
)
# Step 3: Parse results and filter
E_THRESH = 1e-5
MIN_IDENTITY = 50.0
top_hits = []
blast_record = NCBIXML.read(result_handle)
for alignment in blast_record.alignments:
for hsp in alignment.hsps:
if hsp.expect > E_THRESH:
continue
identity_pct = hsp.identities / hsp.align_length * 100
if identity_pct < MIN_IDENTITY:
continue
accession = alignment.accession
top_hits.append({
"accession": accession,
"title": alignment.title[:80],
"length": alignment.length,
"score": hsp.score,
"evalue": hsp.expect,
"identity_pct": round(identity_pct, 1),
"coverage": round(hsp.align_length / blast_record.query_length * 100, 1),
})
print(f"Filtered to {len(top_hits)} significant hits")
# Step 4: Fetch full GenBank records for top 5 hits
accessions = [h["accession"] for h in top_hits[:5]]
time.sleep(1)
handle = Entrez.efetch(db="protein", id=",".join(accessions), rettype="gb", retmode="text")
gb_records = list(SeqIO.parse(handle, "genbank"))
handle.close()
SeqIO.write(gb_records, "top_blast_hits.gb", "genbank")
print(f"Saved {len(gb_records)} full GenBank records to top_blast_hits.gb")
# Step 5: Write CSV summary
with open("blast_summary.csv", "w", newline="") as fh:
writer = csv.DictWriter(fh, fieldnames=top_hits[0].keys())
writer.writeheader()
writer.writerows(top_hits)
print(f"Saved blast_summary.csv ({len(top_hits)} hits)")
Workflow 3: FASTQ Quality Filter and Format Pipeline
Goal: Read a FASTQ file, filter reads by mean quality and length, and output a FASTA for downstream alignment.
from Bio import SeqIO
from Bio.SeqUtils import gc_fraction
input_fastq = "raw_reads.fastq"
output_fasta = "filtered_reads.fasta"
MIN_QUAL = 20 # mean Phred quality
MIN_LEN = 100 # minimum read length
MAX_LEN = 300 # maximum read length
def passes_qc(record, min_q=MIN_QUAL, min_l=MIN_LEN, max_l=MAX_LEN):
quals = record.letter_annotations["phred_quality"]
mean_q = sum(quals) / len(quals)
length = len(record.seq)
return mean_q >= min_q and min_l <= length <= max_l
kept = 0
total = 0
with open(output_fasta, "w") as out_fh:
for record in SeqIO.parse(input_fastq, "fastq"):
total += 1
if passes_qc(record):
# Convert FASTQ → FASTA (drops quality scores)
SeqIO.write(record, out_fh, "fasta")
kept += 1
print(f"Total reads: {total:,}")
print(f"Passed QC: {kept:,} ({kept/total*100:.1f}%)")
print(f"Saved to: {output_fasta}")
Key Parameters
| Parameter | Module / Function | Default | Range / Options | Effect |
|---|---|---|---|---|
retmax | Entrez.esearch() | 20 | 1–10000 | Max IDs returned per search; use history server for >10000 |
hitlist_size | NCBIWWW.qblast() | 50 | 1–5000 | Max BLAST alignments returned |
expect | NCBIWWW.qblast() | 10.0 | 1e-100–1000 | E-value cutoff for BLAST hit reporting |
matrix_name | NCBIWWW.qblast() | "BLOSUM62" | "BLOSUM45", "BLOSUM80", "PAM250" | Substitution matrix; BLOSUM62 for general use |
mode | PairwiseAligner | "global" | "global", "local" | Needleman-Wunsch (global) vs Smith-Waterman (local) |
open_gap_score | PairwiseAligner | -1 | Negative float | Penalty for opening a gap; increase magnitude to penalize more |
extend_gap_score | PairwiseAligner | 0 | Negative float | Per-residue gap extension penalty |
method | DistanceTreeConstructor | "nj" | "nj", "upgma" | NJ (unrooted) vs UPGMA (rooted, assumes clock) |
format | Phylo.read/write() | required | "newick", "nexus", "phyloxml" | Tree file format |
rettype | Entrez.efetch() | required | "fasta", "gb", "xml" | Record format returned; pair with matching SeqIO format string |
Best Practices
-
Always set
Entrez.emailand respect rate limits. NCBI blocks IPs that exceed 3 req/s without a key. Addtime.sleep(0.4)between requests in loops, or useEntrez.api_keyfor 10 req/s.from Bio import Entrez import time Entrez.email = "your.email@institution.edu" Entrez.api_key = "YOUR_API_KEY" # from https://www.ncbi.nlm.nih.gov/account/ # In any batch loop: time.sleep(0.11) # ~9 req/s with key -
Use
SeqIO.index()for large FASTA files instead oflist(SeqIO.parse(...)). Loading all records into a list consumes O(N) memory;index()reads only the byte offsets and fetches records on demand.# Bad for large files: loads everything into RAM # records = {r.id: r for r in SeqIO.parse("genome.fasta", "fasta")} # Good: on-disk index, O(1) access from Bio import SeqIO idx = SeqIO.index("genome.fasta", "fasta") rec = idx["chr22"] # fetches only this record -
Use
PairwiseAlignernot the legacypairwise2.Bio.pairwise2is deprecated since Biopython 1.80 and will be removed.PairwiseAligneris faster, supports substitution matrices directly, and returnsAlignmentobjects with coordinate arrays. -
Close Entrez handles immediately after reading. Handles are HTTP connections; leaving them open risks timeouts and resource exhaustion.
handle = Entrez.efetch(db="nucleotide", id="NM_007294", rettype="gb", retmode="text") record = SeqIO.read(handle, "genbank") handle.close() # do not skip this -
Prefer local BLAST for batch searches. Remote
NCBIWWW.qblast()is suitable for ad hoc queries but can queue for minutes on NCBI servers. For screening >100 sequences, build a local BLAST+ database withmakeblastdband callblastp/blastnviasubprocesswith-outfmt 5(XML) forNCBIXMLparsing. -
Root trees before measuring distances.
Phylo.distance()measures the sum of branch lengths along the path between two nodes. On an unrooted tree, the path is still unique, but midpoint-rooting or outgroup-rooting makes biological sense for visualizations and clade assertions. -
Strip alignment gaps before building SeqRecord collections. BLAST and alignment results may include gap characters (
-).Seqoperations like.translate()will raise errors on gapped sequences; strip withseq.replace("-", "")or useungap().
Common Recipes
Recipe: Batch Entrez Fetch with History Server
When to use: Downloading more than 500 records from NCBI — avoids URL length limits and keeps search results server-side.
from Bio import Entrez, SeqIO
import time
Entrez.email = "your.email@example.com"
# Search and store results on NCBI history server
handle = Entrez.esearch(db="nucleotide", term="16S rRNA[Gene] AND Bacteria[Organism]",
retmax=500, usehistory="y")
results = Entrez.read(handle); handle.close()
webenv = results["WebEnv"]
query_key = results["QueryKey"]
count = int(results["Count"])
print(f"Total: {count} sequences")
# Fetch in batches of 200
batch_size = 200
all_records = []
for start in range(0, min(count, 1000), batch_size):
handle = Entrez.efetch(
db="nucleotide", rettype="fasta", retmode="text",
retstart=start, retmax=batch_size,
webenv=webenv, query_key=query_key
)
batch = list(SeqIO.parse(handle, "fasta"))
handle.close()
all_records.extend(batch)
print(f" Fetched {len(all_records)}/{min(count, 1000)}")
time.sleep(0.5)
SeqIO.write(all_records, "16S_sequences.fasta", "fasta")
print(f"Saved {len(all_records)} sequences")
Recipe: Parse GFF3 with Sequence Features
When to use: Extract gene/CDS sequences from a GFF3 annotation paired with a reference FASTA.
from Bio import SeqIO
# Load genome FASTA into indexed dict
genome = SeqIO.to_dict(SeqIO.parse("genome.fasta", "fasta"))
# Parse GFF3 manually (Biopython does not have a native GFF3 parser;
# use the gffutils package for complex queries, or parse directly for simple cases)
genes = []
with open("annotation.gff3") as fh:
for line in fh:
if line.startswith("#") or not line.strip():
continue
fields = line.strip().split("\t")
if len(fields) < 9 or fields[2] != "CDS":
continue
chrom, _, feat_type, start, end, _, strand, _, attrs = fields
start, end = int(start) - 1, int(end) # GFF3 is 1-based
if chrom not in genome:
continue
seq = genome[chrom].seq[start:end]
if strand == "-":
seq = seq.reverse_complement()
attr_dict = dict(kv.split("=") for kv in attrs.strip().split(";") if "=" in kv)
gene_id = attr_dict.get("gene_id", attr_dict.get("ID", "unknown"))
genes.append((gene_id, seq, strand))
print(f"Extracted {len(genes)} CDS features")
for gid, seq, strand in genes[:3]:
print(f" {gid} ({strand}): {len(seq)} bp — protein: {seq.translate(to_stop=True)[:8]}...")
Recipe: Compute a Pairwise Identity Matrix from a Multiple Alignment
When to use: Quickly assess sequence diversity within an alignment file (FASTA, PHYLIP, Clustal) and identify outlier sequences.
from Bio import AlignIO
import numpy as np
alignment = AlignIO.read("aligned_sequences.fasta", "fasta")
n = len(alignment)
names = [rec.id for rec in alignment]
length = alignment.get_alignment_length()
# Build identity matrix
identity_matrix = np.zeros((n, n))
for i in range(n):
for j in range(n):
if i == j:
identity_matrix[i, j] = 1.0
continue
matches = sum(
a == b and a != "-"
for a, b in zip(str(alignment[i].seq), str(alignment[j].seq))
)
aligned_cols = sum(a != "-" and b != "-"
for a, b in zip(str(alignment[i].seq), str(alignment[j].seq)))
identity_matrix[i, j] = matches / aligned_cols if aligned_cols else 0.0
print("Pairwise identity matrix:")
print(f"{'':20s} " + " ".join(f"{n[:8]:>8s}" for n in names))
for i, name in enumerate(names):
row = " ".join(f"{identity_matrix[i,j]*100:8.1f}" for j in range(n))
print(f"{name[:20]:20s} {row}")
# Find most divergent pair
min_id = np.min(identity_matrix[identity_matrix > 0])
idx = np.unravel_index(np.argmin(np.where(identity_matrix > 0, identity_matrix, 1)), identity_matrix.shape)
print(f"\nMost divergent pair: {names[idx[0]]} vs {names[idx[1]]} ({min_id*100:.1f}%)")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
urllib.error.HTTPError: 429 Too Many Requests | Exceeding NCBI rate limit (3 req/s) | Add time.sleep(0.4) between calls, or register a free API key at https://www.ncbi.nlm.nih.gov/account/ for 10 req/s |
RuntimeError: Too many requests were made without pausing | NCBI detects rapid-fire requests | Set Entrez.email (required), reduce loop frequency, batch IDs into comma-joined strings in a single efetch call |
Bio.Application.ApplicationError: blastall not found | Legacy blastall replaced by BLAST+ | Replace NcbiblastpCommandline with subprocess.run(["blastp", ...]) and BLAST+ tools |
AttributeError: 'PairwiseAligner' object has no attribute 'align' | Biopython < 1.78 | Upgrade: pip install --upgrade biopython (PairwiseAligner requires ≥ 1.72; stable from 1.78) |
Bio.pairwise2 deprecation warning | Using the legacy pairwise2 API | Replace with from Bio.Align import PairwiseAligner (removed in Biopython 1.84+) |
ValueError: Sequence contains letters not in the alphabet | Non-standard characters (e.g., N, ambiguity codes) in sequence before translate() | Strip or replace ambiguous bases; use seq.translate(table=1) which handles ambiguous codons |
Empty BLAST results / StopIteration from NCBIXML.read() | Empty or malformed XML; network timeout; query too short | Check query sequence length (>10 aa recommended); switch from .read() to .parse() and check blast_record.alignments length |
Phylo.draw() hangs or produces blank plot | Missing matplotlib backend | Call import matplotlib; matplotlib.use("Agg") before importing Phylo for headless environments |
TreeConstruction distance matrix dimension mismatch | Alignment contains duplicate IDs | Deduplicate SeqRecord IDs before constructing the alignment: {r.id: r for r in records}.values() |
Related Skills
- biopython-molecular-biology — restriction digestion, PCR primer design, protein structure analysis (Bio.PDB), molecular weight and Tm calculations; complement to this skill
- pysam-genomic-files — SAM/BAM/CRAM alignment file access, pileup, and region queries; use when working with read alignments
- scikit-bio — ecological diversity statistics (UniFrac, Bray-Curtis), ordination (PCoA), and microbiome analysis; more specialized than Biopython's phylogenetics for diversity analyses
- gget-genomic-databases — one-liner gene lookups, BLAST, and database queries without setting up an Entrez pipeline
- etetoolkit — advanced phylogenetic tree visualization, annotation with NCBI taxonomy, and publication-quality tree rendering
References
- Biopython Tutorial and Cookbook — comprehensive official tutorial (Chapters 2, 5, 6, 7, 9, 11)
- Biopython API Reference — complete module and class documentation
- GitHub: biopython/biopython — source, changelog, and issue tracker
- NCBI E-utilities Documentation — Entrez API reference for esearch, efetch, elink, esummary
- PairwiseAligner Migration Guide — replacing the deprecated
Bio.pairwise2module
ARCHS4 Database
Overview
ARCHS4 (All RNA-seq and ChIP-seq Sample and Signature Search) is a resource of uniformly aligned and processed human and mouse RNA-seq data from NCBI GEO and SRA, covering 1 million+ samples. The REST API at https://maayanlab.cloud/archs4/api/ provides gene-level expression profiles, z-score normalized tissue expression, co-expression networks, and sample metadata search — all without authentication. Large-scale bulk queries can also use the downloadable HDF5 expression matrices.
When to Use
- Retrieving tissue-specific or cell-type-specific expression z-scores for a gene of interest across hundreds of tissue types
- Finding genes co-expressed with a query gene (co-expression network construction or guilt-by-association analysis)
- Searching for RNA-seq samples by tissue, disease, or metadata keyword to identify candidate datasets for reanalysis
- Comparing expression profiles of multiple genes across tissues to prioritize candidates for wet-lab follow-up
- Accessing uniformly processed gene expression matrices (HDF5 format) for large-scale cross-study analysis
- Validating differential expression results by checking whether a gene's expression direction matches population-level tissue profiles
- Use omics-plotting SKILL to render bar charts and expression heatmaps from the query results
- For variant-level population allele frequencies use
gnomad-database; ARCHS4 provides expression evidence only - For Enrichr pathway enrichment from a gene list use
gget-genomic-databases(gget enrichr); ARCHS4 is for expression lookups
Prerequisites
- Python packages:
requests,pandas,matplotlib,seaborn - Data requirements: gene symbols (HGNC format, e.g.,
TP53,BRCA1); sample GEO/SRA IDs for direct sample queries - Environment: internet connection; no API key or account required
- Rate limits: ~10 requests/second; add
time.sleep(0.1)between sequential gene queries to avoid throttling
pip install requests pandas matplotlib seaborn
Quick Start
import requests
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def archs4_get(endpoint: str, params: dict = None) -> dict:
"""Send a GET request to the ARCHS4 API and return parsed JSON."""
r = requests.get(f"{ARCHS4_BASE}/{endpoint}", params=params, timeout=30)
r.raise_for_status()
return r.json()
# Quick check: top tissues expressing TP53
data = archs4_get("meta/genes/TP53/zscore")
tissues = data.get("values", [])
print(f"TP53 tissue expression entries: {len(tissues)}")
top5 = sorted(tissues, key=lambda x: x.get("zscore", 0), reverse=True)[:5]
for t in top5:
print(f" {t['tissue']:<40} z={t['zscore']:.2f}")
# TP53 tissue expression entries: 200
# thymus z=2.81
# testis z=2.44
Core API
Query 1: Gene Expression Z-Scores Across Tissues
Retrieve z-score normalized expression for a gene across all available tissue types. Z-scores are computed per-sample relative to the population distribution; positive values indicate above-average expression.
import requests
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def get_gene_tissue_zscore(gene_symbol: str, species: str = "human") -> pd.DataFrame:
"""Return tissue z-score expression profile for a gene.
Parameters
----------
gene_symbol : str
HGNC gene symbol (e.g., 'TP53').
species : str
'human' or 'mouse' (default: 'human').
"""
endpoint = f"meta/genes/{gene_symbol}/zscore"
r = requests.get(
f"{ARCHS4_BASE}/{endpoint}",
params={"species": species},
timeout=30
)
r.raise_for_status()
data = r.json()
records = data.get("values", [])
df = pd.DataFrame(records)
return df.sort_values("zscore", ascending=False).reset_index(drop=True)
df = get_gene_tissue_zscore("MYC")
print(f"MYC tissue z-scores: {len(df)} tissue types")
print(df[["tissue", "zscore"]].head(10).to_string(index=False))
# MYC tissue z-scores: 200
# tissue zscore
# colon 3.12
# small intestine 2.98
# placenta 2.74
# Query mouse tissues for a gene
df_mouse = get_gene_tissue_zscore("Myc", species="mouse")
print(f"Mouse Myc: top 5 tissues")
print(df_mouse[["tissue", "zscore"]].head(5).to_string(index=False))
Query 2: Co-expressed Genes
Find genes whose expression is most correlated with a query gene across all ARCHS4 samples. Useful for identifying pathway partners, regulators, or candidate targets.
import requests
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def get_coexpressed_genes(gene_symbol: str, top_n: int = 50,
species: str = "human") -> pd.DataFrame:
"""Return genes co-expressed with the query gene.
Parameters
----------
gene_symbol : str
HGNC gene symbol.
top_n : int
Number of correlated genes to return (default: 50).
species : str
'human' or 'mouse' (default: 'human').
"""
r = requests.get(
f"{ARCHS4_BASE}/meta/genes/{gene_symbol}/correlations",
params={"species": species, "limit": top_n},
timeout=30
)
r.raise_for_status()
data = r.json()
records = data.get("values", [])
df = pd.DataFrame(records)
return df.sort_values("correlation", ascending=False).reset_index(drop=True)
coexp = get_coexpressed_genes("PCNA", top_n=20)
print(f"Top co-expressed genes with PCNA (n={len(coexp)}):")
print(coexp[["gene", "correlation"]].head(10).to_string(index=False))
# Top co-expressed genes with PCNA (n=20):
# gene correlation
# RFC4 0.91
# RFC2 0.89
# MCM6 0.87
# Extract gene list for downstream enrichment
gene_list = coexp["gene"].tolist()
print(f"Co-expression gene list: {gene_list[:10]}")
# Pass gene_list to Enrichr or pathway analysis tools
Query 3: Sample Search
Search for RNA-seq samples by metadata keyword (tissue, disease condition, cell type, treatment). Returns GEO/SRA sample identifiers with metadata fields.
import requests
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def search_samples(keyword: str, species: str = "human",
limit: int = 100) -> pd.DataFrame:
"""Search ARCHS4 samples by metadata keyword.
Parameters
----------
keyword : str
Search term (e.g., 'breast cancer', 'liver', 'HeLa').
species : str
'human' or 'mouse'.
limit : int
Maximum number of samples to return.
"""
r = requests.get(
f"{ARCHS4_BASE}/samples/search",
params={"query": keyword, "species": species, "limit": limit},
timeout=30
)
r.raise_for_status()
data = r.json()
records = data.get("samples", [])
return pd.DataFrame(records)
samples = search_samples("pancreatic cancer", limit=50)
print(f"Samples matching 'pancreatic cancer': {len(samples)}")
if len(samples) > 0:
print(samples[["sample_id", "series_id", "title"]].head(5).to_string(index=False))
# Samples matching 'pancreatic cancer': 50
# sample_id series_id title
# GSM2345678 GSE123456 Pancreatic ductal adenocarcinoma - sample 1
Query 4: Gene-Level Metadata Summary
Retrieve summary statistics and metadata for a gene including the number of samples expressing it, expression percentile, and available annotation.
import requests
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def get_gene_metadata(gene_symbol: str, species: str = "human") -> dict:
"""Return metadata and expression summary for a gene."""
r = requests.get(
f"{ARCHS4_BASE}/meta/genes/{gene_symbol}",
params={"species": species},
timeout=30
)
r.raise_for_status()
return r.json()
meta = get_gene_metadata("GAPDH")
print(f"Gene: {meta.get('gene_symbol', 'N/A')}")
print(f"Species: {meta.get('species', 'N/A')}")
print(f"Ensembl ID: {meta.get('ensembl_gene_id', 'N/A')}")
print(f"Description: {meta.get('description', 'N/A')[:80]}")
# Compare metadata for a panel of housekeeping genes
import time
housekeeping = ["GAPDH", "ACTB", "B2M", "HPRT1", "RPLP0"]
for gene in housekeeping:
meta = get_gene_metadata(gene)
print(f" {gene:<8} {meta.get('ensembl_gene_id', 'N/A')}")
time.sleep(0.1)
Query 5: Visualization — Tissue Expression Barplot
Fetch the top-tissue z-scores, then read skills/data-visualization/omics-plotting/SKILL.md and follow its "Box / Violin / Bar" recipe on the exported CSV (→ figures/BRCA1_tissue_expression.png).
import requests
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def tissue_zscores(gene_symbol: str, top_n: int = 20, species: str = "human") -> pd.DataFrame:
"""Fetch top tissue z-scores for a gene and save for plotting."""
r = requests.get(
f"{ARCHS4_BASE}/meta/genes/{gene_symbol}/zscore",
params={"species": species},
timeout=30,
)
r.raise_for_status()
df = pd.DataFrame(r.json().get("values", [])).sort_values("zscore", ascending=False).head(top_n)
df.to_csv(f"{gene_symbol}_tissue_zscores.csv", index=False)
return df
df = tissue_zscores("BRCA1", top_n=15)
print(f"Top tissues for BRCA1: {len(df)} -> BRCA1_tissue_zscores.csv")
# Render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Box / Violin / Bar" recipe (horizontal bar of zscore by tissue).
Query 6: HDF5 Bulk Data Access
Download or stream from ARCHS4's precomputed HDF5 expression matrices for large-scale cross-sample analysis. The HDF5 files contain gene × sample count matrices for human and mouse.
import requests
# HDF5 files are available for bulk download from the ARCHS4 data portal
# URL pattern: https://maayanlab.cloud/archs4/download#expression
# Human gene-level: human_gene_v2.6.h5
# Mouse gene-level: mouse_gene_v2.6.h5
def get_h5_download_urls() -> dict:
"""Return download URLs for ARCHS4 HDF5 expression matrices."""
base = "https://maayanlab.cloud/archs4"
return {
"human_gene": f"{base}/files/human_gene_v2.6.h5",
"mouse_gene": f"{base}/files/mouse_gene_v2.6.h5",
"human_transcript": f"{base}/files/human_transcript_v2.6.h5",
"mouse_transcript": f"{base}/files/mouse_transcript_v2.6.h5",
}
urls = get_h5_download_urls()
for key, url in urls.items():
print(f" {key:<22} {url}")
# To work with a downloaded HDF5 file:
try:
import h5py
import numpy as np
h5_path = "human_gene_v2.6.h5" # after download
def extract_gene_from_h5(h5_path: str, gene_symbol: str,
n_samples: int = 1000) -> dict:
"""Extract expression values for a gene from the HDF5 matrix."""
with h5py.File(h5_path, "r") as f:
genes = [g.decode() for g in f["meta"]["genes"]["gene_symbol"][:]]
if gene_symbol not in genes:
raise ValueError(f"{gene_symbol} not found in HDF5")
idx = genes.index(gene_symbol)
expr = f["data"]["expression"][idx, :n_samples]
sample_ids = [s.decode() for s in f["meta"]["samples"]["geo_accession"][:n_samples]]
return {"gene": gene_symbol, "expression": expr, "sample_ids": sample_ids}
result = extract_gene_from_h5(h5_path, "TP53", n_samples=500)
print(f"TP53 expression: mean={result['expression'].mean():.2f},"
f" max={result['expression'].max():.2f} (n={len(result['expression'])} samples)")
except ImportError:
print("h5py not installed. Install with: pip install h5py")
except FileNotFoundError:
print("HDF5 file not downloaded yet. Use the URLs above to download first.")
Key Concepts
Z-Score Normalization
ARCHS4 reports gene expression as z-scores computed relative to all samples for that gene. A z-score of 0 means expression at the population mean; a z-score of 2.0 means expression 2 standard deviations above the mean. Z-scores are more interpretable across datasets than raw counts because they account for library size differences and batch effects introduced by uniform alignment across studies.
# Example: Positive z-score = above-average expression for that gene
# z > 2.0 → top ~2.5% of samples for that gene
# z < -2.0 → bottom ~2.5% of samples for that gene
# Use absolute z-score thresholds consistently when comparing across genes
HDF5 vs REST API
| Access method | Best for | Limitations |
|---|---|---|
REST API (/zscore, /correlations) | Quick single-gene queries, exploration | Aggregated profiles only, no per-sample access |
REST API (/samples/search) | Discovering relevant datasets | Returns metadata, not expression values |
| HDF5 download | Bulk analysis, custom co-expression, ML | Requires 30–60 GB disk; download once |
Species and Gene Symbol Conventions
ARCHS4 indexes human samples using HGNC gene symbols (uppercase, e.g., TP53) and mouse samples using MGI symbols (first letter uppercase, e.g., Trp53). The species parameter accepts "human" or "mouse". Mixed-case or ensemble IDs will return empty results.
Common Workflows
Workflow 1: Multi-Gene Tissue Expression Heatmap
Goal: Compare tissue expression profiles of a gene panel and visualize as a heatmap to identify tissue-specific vs ubiquitous expression patterns.
import requests, time
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
gene_panel = ["MYC", "TP53", "BRCA1", "EGFR", "KRAS", "CDK4"]
top_n_tissues = 25
def get_tissue_zscores(gene: str) -> pd.Series:
r = requests.get(
f"{ARCHS4_BASE}/meta/genes/{gene}/zscore",
params={"species": "human"},
timeout=30
)
r.raise_for_status()
records = r.json().get("values", [])
df = pd.DataFrame(records).set_index("tissue")["zscore"]
return df
# Build expression matrix (genes × tissues)
all_data = {}
for gene in gene_panel:
try:
all_data[gene] = get_tissue_zscores(gene)
print(f" Fetched {gene}")
except Exception as e:
print(f" Warning: {gene} failed — {e}")
time.sleep(0.1)
matrix = pd.DataFrame(all_data).T # genes × tissues
# Select top tissues by max absolute z-score
tissue_importance = matrix.abs().max(axis=0).sort_values(ascending=False)
top_tissues = tissue_importance.head(top_n_tissues).index
matrix_subset = matrix[top_tissues]
# Save the gene × tissue matrix; render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Expression heatmap"
# recipe (RdBu_r, center=0) -> figures/archs4_panel_heatmap.png
matrix_subset.to_csv("archs4_panel_matrix.csv")
print(f"Panel matrix ready: {matrix_subset.shape} -> archs4_panel_matrix.csv")
Workflow 2: Co-expression Network Seed Expansion
Goal: Start from a seed gene, retrieve co-expressed partners, then query their co-expressed genes in turn to build a two-hop co-expression neighborhood.
import requests, time
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def get_coexp(gene: str, top_n: int = 20, species: str = "human") -> list:
r = requests.get(
f"{ARCHS4_BASE}/meta/genes/{gene}/correlations",
params={"species": species, "limit": top_n},
timeout=30
)
r.raise_for_status()
return [rec["gene"] for rec in r.json().get("values", [])]
seed_gene = "PCNA"
min_correlation = 0.80
# Hop 1: direct co-expressed partners
hop1_genes = get_coexp(seed_gene, top_n=30)
print(f"Hop 1 partners of {seed_gene}: {len(hop1_genes)}")
time.sleep(0.1)
# Hop 2: co-expressed genes of each partner
edges = set()
for gene in hop1_genes[:10]: # limit for demonstration
partners = get_coexp(gene, top_n=20)
for partner in partners:
if partner != seed_gene:
edges.add((gene, partner))
time.sleep(0.1)
# Summarize the network
network_df = pd.DataFrame(list(edges), columns=["source", "target"])
hub_counts = network_df["source"].value_counts()
print(f"\nTwo-hop network: {len(edges)} edges")
print(f"Top hub genes:")
print(hub_counts.head(5))
network_df.to_csv(f"{seed_gene}_coexp_network.csv", index=False)
print(f"\nSaved {seed_gene}_coexp_network.csv")
Workflow 3: Sample Discovery and Dataset Summary
Goal: Search for samples by disease keyword, summarize how many GEO series are available, and export sample metadata for downstream reanalysis selection.
import requests, time
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def search_and_summarize(keyword: str, species: str = "human",
limit: int = 200) -> pd.DataFrame:
"""Search samples and return a tidy metadata DataFrame."""
r = requests.get(
f"{ARCHS4_BASE}/samples/search",
params={"query": keyword, "species": species, "limit": limit},
timeout=30
)
r.raise_for_status()
records = r.json().get("samples", [])
return pd.DataFrame(records)
keyword = "colorectal cancer"
df = search_and_summarize(keyword, limit=150)
print(f"Samples matching '{keyword}': {len(df)}")
if len(df) > 0:
# Summarize by GEO series
series_counts = df["series_id"].value_counts()
print(f"\nTop GEO series (by sample count):")
print(series_counts.head(8).to_string())
# Export sample list
df.to_csv(f"{keyword.replace(' ', '_')}_samples.csv", index=False)
print(f"\nSaved {keyword.replace(' ', '_')}_samples.csv ({len(df)} samples)")
print(f"Unique GEO series: {df['series_id'].nunique()}")
Key Parameters
| Parameter | Endpoint | Default | Range / Options | Effect |
|---|---|---|---|---|
species | All gene endpoints | "human" | "human", "mouse" | Selects the species-specific sample index |
limit | /correlations, /samples/search | 100 | 1–500 | Number of results returned |
gene_symbol (path) | /meta/genes/{gene}/zscore, /correlations | — | HGNC symbol (human) or MGI symbol (mouse) | Query gene; case-sensitive |
query | /samples/search | — | free-text string | Metadata keyword search across title, tissue, source fields |
offset | /samples/search | 0 | integer | Pagination offset for large result sets |
correlation (response field) | /correlations | — | -1.0–1.0 | Pearson correlation coefficient; filter > 0.7 for high co-expression |
zscore (response field) | /zscore | — | continuous float | Expression z-score; > 2.0 = high expression |
page_size (HDF5) | HDF5 slice | all | any integer | Number of samples to extract per read from HDF5 |
Best Practices
-
Use z-score thresholds consistently: Because z-scores are gene-specific, a z-score of 2.0 for a ubiquitous gene (GAPDH) and a tissue-restricted gene (TTR, liver) have different interpretive meaning. Always annotate which gene you are comparing and the tissue background.
-
Sleep between batch queries: ARCHS4 enforces a soft rate limit of ~10 requests/second. Add
time.sleep(0.1)between sequential gene queries to avoid429 Too Many Requestserrors. -
Download HDF5 for large-scale analyses: For queries covering 50+ genes or requiring per-sample expression values, the REST API is impractical. Download the HDF5 file once and use
h5pyslicing for fast matrix access; this avoids hitting rate limits and is 100× faster for bulk extraction. -
Match gene symbol conventions by species: Human queries require HGNC uppercase symbols (e.g.,
TP53); mouse queries require MGI-style symbols (e.g.,Trp53). Using the wrong case returns empty results without an error. -
Validate co-expression findings across datasets: ARCHS4 co-expression aggregates across all tissue types. A high correlation may be driven by a single tissue or study. Cross-check with tissue-specific queries or manually inspect the top contributing GEO series.
Common Recipes
Recipe: Quick Tissue Specificity Check
When to use: Rapidly determine whether a gene is broadly expressed (housekeeping) or tissue-restricted before designing experiments.
import requests
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def tissue_specificity_summary(gene_symbol: str) -> None:
"""Print a summary of high and low expression tissues for a gene."""
r = requests.get(
f"{ARCHS4_BASE}/meta/genes/{gene_symbol}/zscore",
params={"species": "human"},
timeout=30
)
r.raise_for_status()
records = r.json().get("values", [])
zscores = [rec["zscore"] for rec in records if rec.get("zscore") is not None]
top_high = sorted(records, key=lambda x: x.get("zscore", 0), reverse=True)[:5]
top_low = sorted(records, key=lambda x: x.get("zscore", float("inf")))[:3]
print(f"\n{gene_symbol} — {len(zscores)} tissues")
print(f" Range: [{min(zscores):.2f}, {max(zscores):.2f}] "
f"Mean: {sum(zscores)/len(zscores):.2f}")
print(" High expression:")
for t in top_high:
print(f" {t['tissue']:<35} z={t['zscore']:.2f}")
print(" Low expression:")
for t in top_low:
print(f" {t['tissue']:<35} z={t['zscore']:.2f}")
tissue_specificity_summary("TTR") # Transthyretin — liver-specific
Recipe: Batch Gene Co-Expression Table
When to use: Generate a pairwise correlation table for a gene panel from a list of differentially expressed genes.
import requests, time
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
def batch_coexpr_table(gene_list: list, top_n: int = 10) -> pd.DataFrame:
"""For each gene in gene_list, return its top co-expressed genes."""
rows = []
for gene in gene_list:
try:
r = requests.get(
f"{ARCHS4_BASE}/meta/genes/{gene}/correlations",
params={"species": "human", "limit": top_n},
timeout=30
)
r.raise_for_status()
for rec in r.json().get("values", []):
rows.append({
"query_gene": gene,
"coexp_gene": rec.get("gene"),
"correlation": rec.get("correlation"),
})
time.sleep(0.1)
except Exception as e:
print(f"Warning: {gene} skipped — {e}")
return pd.DataFrame(rows)
deg_list = ["MYC", "CCND1", "CDK4", "RB1", "E2F1"]
coexp_table = batch_coexpr_table(deg_list, top_n=10)
print(f"Co-expression entries: {len(coexp_table)}")
print(coexp_table.groupby("query_gene")["coexp_gene"].count())
coexp_table.to_csv("deg_coexpression_table.csv", index=False)
print("Saved deg_coexpression_table.csv")
Recipe: Export Sample IDs for GEO Download
When to use: Identify relevant GEO accessions to download raw count matrices for a meta-analysis.
import requests
import pandas as pd
ARCHS4_BASE = "https://maayanlab.cloud/archs4/api/v1"
keyword = "glioblastoma"
r = requests.get(
f"{ARCHS4_BASE}/samples/search",
params={"query": keyword, "species": "human", "limit": 200},
timeout=30
)
r.raise_for_status()
samples = pd.DataFrame(r.json().get("samples", []))
if len(samples) > 0:
# Get unique GEO series accessions
series = samples["series_id"].dropna().unique()
print(f"Unique GEO series for '{keyword}': {len(series)}")
for s in series[:10]:
n = (samples["series_id"] == s).sum()
print(f" {s} ({n} samples)")
# Export series list for GEO download script
pd.Series(series, name="geo_series").to_csv(
f"{keyword}_geo_series.txt", index=False
)
print(f"\nSaved {keyword}_geo_series.txt")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
HTTP 404 for gene query | Gene symbol not found in ARCHS4 index | Verify HGNC symbol spelling; check species parameter matches gene convention (human: uppercase, mouse: first-letter-upper) |
HTTP 429 Too Many Requests | Exceeded ~10 req/s rate limit | Add time.sleep(0.1) between requests; for batch queries use a 0.5 s delay |
Empty values list in z-score response | Gene is not expressed in any indexed tissue, or wrong species | Switch species; verify gene is protein-coding and has GEO coverage |
Empty samples list from search | Keyword not matched in metadata fields | Try broader or alternative keywords (e.g., "liver" instead of "hepatic") |
| HDF5 gene not found | Symbol mismatch between HDF5 version and query | Check available genes in f["meta"]["genes"]["gene_symbol"][:]; try Ensembl ID or alias |
requests.exceptions.Timeout | Slow API response under load | Increase timeout=60; retry with exponential backoff |
| Z-scores all near zero | Gene has very low or absent expression across tissues | Check the gene's expression in raw counts; the gene may be non-coding or very lowly expressed |
Related Skills
gnomad-database— Population variant frequencies; use after ARCHS4 to identify variants in highly expressed genesgget-genomic-databases— Enrichr pathway enrichment for ARCHS4 co-expression gene lists (gget enrichr)pydeseq2-differential-expression— Differential expression analysis on bulk RNA-seq; ARCHS4 HDF5 matrices can serve as reference cohorts
References
- ARCHS4 web portal — Interactive expression browser and dataset download
- ARCHS4 REST API documentation — Endpoint reference and parameters
- Lachmann et al., Nature Communications 2018 — ARCHS4 original publication describing uniform alignment pipeline
- ARCHS4 GitHub — Source code and HDF5 schema documentation
BioServices Multi-Database Access
Overview
BioServices provides a unified Python interface to 40+ bioinformatics web services including UniProt, KEGG, ChEMBL, ChEBI, PubChem, UniChem, PSICQUIC, QuickGO, and BLAST. Each service is accessed through a consistent object-oriented API with built-in caching, rate limiting, and output format handling.
When to Use
- Querying protein information from UniProt (search, retrieve, ID mapping)
- Discovering KEGG pathways and extracting gene/interaction networks
- Cross-referencing compounds across ChEMBL, ChEBI, PubChem, and KEGG
- Running BLAST sequence similarity searches against UniProtKB
- Mapping identifiers between biological databases (UniProt, Ensembl, KEGG, RefSeq, PDB)
- Retrieving Gene Ontology annotations via QuickGO
- Finding protein-protein interactions via PSICQUIC (IntAct, MINT, BioGRID)
- Batch converting thousands of biological identifiers with error handling
- For single-database deep queries → use gget (Ensembl), pubchempy (PubChem), or chembl-database-bioactivity skill
- For pathway visualization → use pathway analysis tools (Cytoscape, NetworkX) after retrieving data with bioservices
Prerequisites
pip install bioservices
# Optional: pandas for tabular output, matplotlib for visualization
pip install pandas matplotlib
API Rate Limits: Most services have rate limits. bioservices handles basic throttling internally, but for batch operations add explicit delays:
- UniProt mapping: ~1 request/second for batch jobs
- KEGG: 10 requests/second (be conservative with pathway parsing)
- ChEMBL/ChEBI: 5-10 requests/second
- BLAST: 1 job at a time (async polling, ~30-300s per job)
Quick Start
from bioservices import UniProt, KEGG
import time
# Protein lookup
u = UniProt(verbose=False)
result = u.search("ABL1_HUMAN", frmt="tsv", columns="accession,gene_names,organism_name,length")
print(result[:200])
# Pathway discovery
k = KEGG(verbose=False)
pathways = k.get_pathway_by_gene("hsa:25", "hsa") # ABL1
print(f"ABL1 participates in {len(pathways)} pathways")
for pid, name in list(pathways.items())[:3]:
print(f" {pid}: {name}")
Core API
1. Protein Analysis (UniProt)
from bioservices import UniProt
u = UniProt(verbose=False)
# Search by protein name or gene
result = u.search("BRCA1 AND organism_id:9606", frmt="tsv",
columns="accession,gene_names,protein_name,length,go_p")
print(result[:300])
# Retrieve full entry
entry = u.retrieve("P38398", frmt="txt") # Swiss-Prot flat file
fasta = u.retrieve("P38398", frmt="fasta")
print(fasta[:200])
# ID mapping: gene names → UniProt accessions
result = u.mapping(fr="Gene_Name", to="UniProtKB", query="BRCA1 TP53 ABL1", taxId=9606)
print(f"Mapped {len(result['results'])} entries")
for r in result['results']:
print(f" {r['from']} → {r['to']['primaryAccession']}")
2. Pathway Discovery (KEGG)
from bioservices import KEGG
k = KEGG(verbose=False)
# List pathways for an organism
pathways = k.pathwayIds # All reference pathways
human_pathways = k.list("pathway", "hsa")
print(f"Human pathways: {len(human_pathways.strip().splitlines())}")
# Get pathway details
pathway_data = k.get("hsa04110") # Cell cycle
parsed = k.parse(pathway_data)
print(f"Pathway: {parsed.get('NAME', 'Unknown')}")
print(f"Genes: {len(parsed.get('GENE', {}))}")
# KGML parsing for interaction networks
from bioservices import KEGG
k = KEGG(verbose=False)
kgml = k.get("hsa04110", "kgml") # XML pathway representation
# Parse KGML for entries and relations
import xml.etree.ElementTree as ET
root = ET.fromstring(kgml)
entries = root.findall("entry")
relations = root.findall("relation")
print(f"Entries: {len(entries)}, Relations: {len(relations)}")
# Extract interaction types
from collections import Counter
rel_types = Counter()
for rel in relations:
for subtype in rel.findall("subtype"):
rel_types[subtype.get("name")] += 1
print(f"Interaction types: {dict(rel_types)}")
3. Compound Databases (ChEMBL, ChEBI, UniChem, PubChem)
from bioservices import ChEMBL, ChEBI, UniChem
import time
# ChEMBL compound lookup
chembl = ChEMBL(verbose=False)
result = chembl.get_molecule("CHEMBL25") # Aspirin
print(f"Name: {result['pref_name']}")
print(f"MW: {result['molecule_properties']['full_mwt']}")
print(f"SMILES: {result['molecule_structures']['canonical_smiles']}")
time.sleep(0.2)
# ChEBI entity lookup
chebi = ChEBI(verbose=False)
entity = chebi.getCompleteEntity("CHEBI:15365") # Aspirin
print(f"ChEBI Name: {entity.chebiAsciiName}")
print(f"Formula: {entity.formulae[0].data if entity.formulae else 'N/A'}")
# Cross-database compound mapping via UniChem
from bioservices import UniChem
uc = UniChem()
# Map ChEMBL ID to other databases
# Source IDs: 1=ChEMBL, 2=DrugBank, 3=PDB, 4=IUPHAR, 7=ChEBI, 22=PubChem
mappings = uc.get_mapping("CHEMBL25", 1) # From ChEMBL
for m in mappings[:5]:
print(f" Source {m['src_id']}: {m['src_compound_id']}")
4. Sequence Analysis (BLAST)
from bioservices import NCBIblast
import time
blast = NCBIblast(verbose=False)
sequence = ">query\nMKTAYIAKQRQISFVKSHFSRQLE..." # Truncated for brevity
job_id = blast.run(
program="blastp",
database="uniprotkb_swissprot",
sequence=sequence,
stype="protein",
email="user@example.com" # Required by NCBI
)
print(f"Job submitted: {job_id}")
# Poll for results (async)
while blast.getStatus(job_id) == "RUNNING":
time.sleep(10)
print("Waiting...")
result_types = blast.getResultTypes(job_id)
alignment = blast.getResult(job_id, "out") # Text alignment
print(alignment[:500])
5. Identifier Mapping
from bioservices import UniProt
u = UniProt(verbose=False)
# Batch mapping: UniProt → multiple databases
accessions = "P00520 P12931 P04637 P38398"
# UniProt → PDB
result = u.mapping(fr="UniProtKB_AC-ID", to="PDB", query=accessions)
for r in result.get('results', []):
print(f" {r['from']} → {r['to']}")
# UniProt → Ensembl
result = u.mapping(fr="UniProtKB_AC-ID", to="Ensembl", query=accessions)
print(f"Mapped {len(result.get('results', []))} entries to Ensembl")
# KEGG-based identifier extraction
from bioservices import KEGG
k = KEGG(verbose=False)
# Get cross-references from a KEGG entry
entry = k.get("hsa:25") # ABL1
parsed = k.parse(entry)
# Extract database links
for key in ['DBLINKS', 'PATHWAY']:
if key in parsed:
print(f"{key}: {parsed[key]}")
6. Gene Ontology & Protein Interactions
from bioservices import QuickGO
go = QuickGO(verbose=False)
# Get GO annotations for a protein
annotations = go.Annotation(geneProductId="UniProtKB:P00520", taxonId="9606")
if hasattr(annotations, 'shape'): # DataFrame
print(f"Annotations: {len(annotations)}")
for aspect in ['biological_process', 'molecular_function', 'cellular_component']:
subset = annotations[annotations['goAspect'] == aspect]
print(f" {aspect}: {len(subset)} terms")
# Protein-protein interactions via PSICQUIC
from bioservices import PSICQUIC
psicquic = PSICQUIC(verbose=False)
# Query IntAct for interactions
interactions = psicquic.query("intact", "identifier:P00520", frmt="tab25")
lines = interactions.strip().split("\n") if interactions else []
print(f"Interactions found: {len(lines)}")
for line in lines[:3]:
fields = line.split("\t")
print(f" {fields[0]} <-> {fields[1]}") # Interactor A <-> Interactor B
Key Concepts
Service Initialization & Verbosity
All services share a common pattern: Service(verbose=False) suppresses HTTP request logging. Set verbose=True during debugging to see full request/response details.
Output Format Handling
| Service | Default Format | Available Formats | Notes |
|---|---|---|---|
| UniProt | XML | tsv, fasta, json, txt, xml, gff | Use tsv with columns= for tabular |
| KEGG | Text | kgml, image | Parse text with k.parse() |
| ChEMBL | JSON | JSON only | Dict access on response |
| ChEBI | SOAP XML | Object attributes | Access via .chebiAsciiName etc. |
| NCBIblast | Text | out, xml, json, ids | Async: submit → poll → retrieve |
| PSICQUIC | PSI-MI TAB | tab25, tab27, xml25, count | Tab-separated interaction records |
| QuickGO | DataFrame | tsv, json | Pandas DataFrame when available |
Common Organism Codes (KEGG)
| Code | Organism | Taxonomy ID |
|---|---|---|
hsa | Homo sapiens | 9606 |
mmu | Mus musculus | 10090 |
rno | Rattus norvegicus | 10116 |
dme | Drosophila melanogaster | 7227 |
sce | Saccharomyces cerevisiae | 559292 |
eco | Escherichia coli K-12 | 83333 |
ath | Arabidopsis thaliana | 3702 |
UniProt Mapping Database Codes
Common database pairs for u.mapping(fr=, to=):
| Category | Database Code | Example |
|---|---|---|
| Protein | UniProtKB_AC-ID | P00520 |
| Gene | Gene_Name, GeneID | ABL1, 25 |
| Structure | PDB | 2HYY |
| Sequence | RefSeq_Protein, Ensembl | NP_005148, ENSG00000097007 |
| Pathway | KEGG, Reactome | hsa:25, R-HSA-123 |
| Ontology | GO | GO:0006468 |
Full mapping database list: see references/identifier_mapping_guide.md.
Common Workflows
Workflow 1: Complete Protein Analysis Pipeline
from bioservices import UniProt, KEGG, QuickGO, PSICQUIC
import time
def analyze_protein(uniprot_id):
"""End-to-end protein analysis: metadata → pathways → GO → interactions."""
results = {}
# Step 1: UniProt metadata
u = UniProt(verbose=False)
entry = u.retrieve(uniprot_id, frmt="tsv",
columns="accession,gene_names,protein_name,length,organism_name")
results['uniprot'] = entry
print(f"[1] UniProt entry retrieved")
# Step 2: KEGG pathways
k = KEGG(verbose=False)
mapping = u.mapping(fr="UniProtKB_AC-ID", to="KEGG", query=uniprot_id)
kegg_ids = [r['to'] for r in mapping.get('results', [])]
if kegg_ids:
pathways = k.get_pathway_by_gene(kegg_ids[0], kegg_ids[0][:3])
results['pathways'] = pathways or {}
print(f"[2] Found {len(results['pathways'])} pathways")
time.sleep(0.3)
# Step 3: GO annotations
go = QuickGO(verbose=False)
annotations = go.Annotation(geneProductId=f"UniProtKB:{uniprot_id}")
results['go_annotations'] = annotations
print(f"[3] GO annotations retrieved")
time.sleep(0.3)
# Step 4: Protein interactions
psicquic = PSICQUIC(verbose=False)
interactions = psicquic.query("intact", f"identifier:{uniprot_id}", frmt="tab25")
lines = interactions.strip().split("\n") if interactions else []
results['interactions'] = len(lines)
print(f"[4] Found {len(lines)} interactions")
return results
# Usage
results = analyze_protein("P00520") # ABL1
Workflow 2: Cross-Database Compound Search
from bioservices import KEGG, ChEMBL, ChEBI, UniChem
import time
def search_compound(compound_name):
"""Search compound across KEGG → ChEBI → ChEMBL with cross-references."""
k = KEGG(verbose=False)
# Step 1: KEGG compound search
kegg_results = k.find("compound", compound_name)
if not kegg_results:
print(f"No KEGG results for '{compound_name}'")
return None
kegg_id = kegg_results.strip().split("\t")[0] # First hit
kegg_entry = k.parse(k.get(kegg_id))
print(f"[1] KEGG: {kegg_id} — {kegg_entry.get('NAME', ['Unknown'])[0]}")
print(f" Formula: {kegg_entry.get('FORMULA', 'N/A')}")
time.sleep(0.2)
# Step 2: ChEBI cross-reference
dblinks = kegg_entry.get('DBLINKS', {})
chebi_id = dblinks.get('ChEBI', [''])[0] if isinstance(dblinks, dict) else None
if chebi_id:
chebi = ChEBI(verbose=False)
entity = chebi.getCompleteEntity(f"CHEBI:{chebi_id}")
print(f"[2] ChEBI: {entity.chebiId} — {entity.chebiAsciiName}")
time.sleep(0.2)
# Step 3: ChEMBL via UniChem
uc = UniChem()
try:
chembl_mappings = uc.get_mapping(kegg_id.replace("cpd:", ""), 6) # 6=KEGG
if chembl_mappings:
chembl_id = [m for m in chembl_mappings if m['src_id'] == '1']
if chembl_id:
chembl = ChEMBL(verbose=False)
mol = chembl.get_molecule(chembl_id[0]['src_compound_id'])
print(f"[3] ChEMBL: {mol['molecule_chembl_id']} — MW: {mol['molecule_properties']['full_mwt']}")
except Exception as e:
print(f"[3] ChEMBL mapping failed: {e}")
return kegg_entry
result = search_compound("aspirin")
Workflow 3: Batch Identifier Conversion
from bioservices import UniProt
import time
def batch_convert_ids(ids, from_db, to_db, chunk_size=100):
"""Convert biological IDs in chunks with error handling."""
u = UniProt(verbose=False)
all_results = []
failed = []
for i in range(0, len(ids), chunk_size):
chunk = ids[i:i+chunk_size]
query = " ".join(chunk)
try:
result = u.mapping(fr=from_db, to=to_db, query=query)
mapped = result.get('results', [])
all_results.extend(mapped)
print(f" Chunk {i//chunk_size + 1}: {len(mapped)} mapped")
except Exception as e:
print(f" Chunk {i//chunk_size + 1} failed: {e}")
failed.extend(chunk)
time.sleep(1) # Rate limit between chunks
print(f"\nTotal mapped: {len(all_results)}, Failed: {len(failed)}")
return all_results, failed
# Usage: Gene names → UniProt accessions
gene_list = ["BRCA1", "TP53", "ABL1", "EGFR", "BRAF", "KRAS", "MYC", "PTEN"]
results, failed = batch_convert_ids(gene_list, "Gene_Name", "UniProtKB_AC-ID")
for r in results[:5]:
print(f" {r['from']} → {r['to']['primaryAccession']}")
Key Parameters
| Parameter | Function/Endpoint | Default | Range | Effect |
|---|---|---|---|---|
verbose | All services | True | bool | HTTP request logging |
frmt | UniProt search/retrieve | xml | tsv/fasta/json/txt | Output format |
columns | UniProt search | — | comma-separated | TSV columns to return |
taxId | UniProt mapping | — | NCBI taxonomy ID | Organism filter for mapping |
program | NCBIblast | — | blastp/blastn/blastx/tblastn | BLAST program type |
database | NCBIblast | — | uniprotkb_swissprot/nr/etc | Target database |
email | NCBIblast | — | email address | Required by NCBI policy |
chunk_size | Batch operations | 100 | 10-500 | IDs per batch request |
frmt | PSICQUIC query | tab25 | tab25/tab27/xml25/count | Interaction output format |
Best Practices
-
Always set
verbose=Falsein production — defaultTruefloods stdout with HTTP details. UseTrueonly for debugging failed requests -
Handle None/empty results gracefully — web services frequently return None, empty strings, or partial data. Always check before parsing:
result = k.get("hsa04110") if result and not isinstance(result, int): # KEGG returns 404 as int parsed = k.parse(result) -
Use explicit
time.sleep()between batch requests — even though bioservices has internal throttling, batch operations benefit from explicit delays (0.2-1.0s) to avoid 429 errors -
Cache service objects, not results — create one
UniProt()instance and reuse it across calls. Each constructor call initializes a new HTTP session -
Anti-pattern — parsing KEGG text manually: Always use
k.parse()to parse KEGG entry text into structured dicts. Manual string splitting breaks on multi-line fields -
Specify organism in UniProt searches — unfiltered searches return results across all species. Add
AND organism_id:9606for human-specific results
Common Recipes
Recipe 1: Pathway Network Statistics
from bioservices import KEGG
import xml.etree.ElementTree as ET
from collections import Counter
def pathway_stats(organism="hsa", limit=10):
"""Analyze pathway network statistics for an organism."""
k = KEGG(verbose=False)
pathway_list = k.list("pathway", organism).strip().split("\n")
stats = []
for line in pathway_list[:limit]:
pid = line.split("\t")[0]
kgml = k.get(pid, "kgml")
if not kgml or isinstance(kgml, int):
continue
root = ET.fromstring(kgml)
genes = len([e for e in root.findall("entry") if e.get("type") == "gene"])
rels = len(root.findall("relation"))
stats.append({"pathway": pid, "genes": genes, "relations": rels})
return sorted(stats, key=lambda x: x['relations'], reverse=True)
top = pathway_stats("hsa", limit=5)
for s in top:
print(f" {s['pathway']}: {s['genes']} genes, {s['relations']} interactions")
Recipe 2: GO Term Enrichment Preparation
from bioservices import QuickGO, UniProt
import time
def get_go_terms(uniprot_ids, aspect="biological_process"):
"""Collect GO annotations for a list of proteins."""
go = QuickGO(verbose=False)
all_terms = {}
for uid in uniprot_ids:
try:
annot = go.Annotation(geneProductId=f"UniProtKB:{uid}", taxonId="9606")
if hasattr(annot, 'shape') and len(annot) > 0:
subset = annot[annot['goAspect'] == aspect]
all_terms[uid] = list(subset['goId'].unique())
except Exception:
all_terms[uid] = []
time.sleep(0.3)
return all_terms
terms = get_go_terms(["P00520", "P12931", "P04637"])
for uid, go_ids in terms.items():
print(f" {uid}: {len(go_ids)} {go_ids[:3]}")
Recipe 3: Multi-Hop Identifier Mapping
from bioservices import UniProt, KEGG
import time
def gene_to_pathways(gene_name, organism="hsa"):
"""Map gene name → UniProt → KEGG → pathways (multi-hop)."""
u = UniProt(verbose=False)
# Hop 1: Gene name → UniProt
result = u.mapping(fr="Gene_Name", to="UniProtKB_AC-ID", query=gene_name, taxId=9606)
if not result.get('results'):
return None
uniprot_id = result['results'][0]['to']['primaryAccession']
print(f" {gene_name} → UniProt: {uniprot_id}")
time.sleep(0.3)
# Hop 2: UniProt → KEGG
kegg_result = u.mapping(fr="UniProtKB_AC-ID", to="KEGG", query=uniprot_id)
if not kegg_result.get('results'):
return None
kegg_id = kegg_result['results'][0]['to']
print(f" UniProt: {uniprot_id} → KEGG: {kegg_id}")
time.sleep(0.3)
# Hop 3: KEGG gene → pathways
k = KEGG(verbose=False)
pathways = k.get_pathway_by_gene(kegg_id, organism)
print(f" KEGG: {kegg_id} → {len(pathways or {})} pathways")
return pathways
pathways = gene_to_pathways("BRCA1")
if pathways:
for pid, name in list(pathways.items())[:5]:
print(f" {pid}: {name}")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
ConnectionError on service init | Network/firewall blocking | Check internet; some services need port 443 |
| UniProt search returns empty | Query syntax error | Use AND/OR operators; check field names in UniProt docs |
KEGG get() returns 404 (int) | Invalid pathway/gene ID | Verify ID format: hsa:25 (gene), hsa04110 (pathway), cpd:C00001 (compound) |
| BLAST job timeout | Large sequence or busy server | Increase polling interval; try uniprotkb_swissprot (smaller than nr) |
| UniProt mapping returns no results | Wrong database codes | Use exact codes: UniProtKB_AC-ID, Gene_Name, PDB, Ensembl (case-sensitive) |
AttributeError on ChEBI entity | SOAP response parsing | Check entity is not None; some ChEBI IDs lack certain fields |
| PSICQUIC returns empty string | No interactions in database | Try multiple databases: intact, mint, biogrid |
429 Too Many Requests | Rate limit exceeded | Add time.sleep(0.5) between requests; reduce batch size |
| Inconsistent mapping results | Multiple isoforms/entries | Filter by isReviewed=true for canonical UniProt entries |
Bundled Resources
references/services_catalog.md
Catalog of 20+ bioservices-accessible services organized by category: protein/gene resources (UniProt, KEGG, HGNC, MyGeneInfo), chemical compound resources (ChEBI, ChEMBL, UniChem, PubChem), sequence analysis (NCBIblast), pathway/interaction resources (Reactome, PSICQUIC, IntactComplex, OmniPath), gene ontology (QuickGO), genomic resources (BioMart, ArrayExpress, ENA), structural biology (PDB, Pfam), and specialized resources (BioModels, COG, BiGG). Includes method signatures, key parameters, and output formats for each service. General patterns (error handling, verbosity, caching) relocated to SKILL.md Best Practices. Original services_reference.md content partially relocated to Core API (UniProt, KEGG, ChEMBL, ChEBI, BLAST, PSICQUIC, QuickGO sections); remaining services retained in catalog.
references/identifier_mapping_guide.md
Comprehensive guide to cross-database identifier conversion: UniProt mapping service (100+ database pairs with database code tables), UniChem compound mapping (source database IDs), KEGG identifier conversions, 4 common mapping patterns with code, and troubleshooting. Original identifier_mapping.md content partially relocated to Core API section 5 (basic mapping examples); advanced patterns, full database code tables, and multi-hop strategies retained in guide.
Scripts disposition (4 original scripts, 1,444 lines total):
protein_analysis_workflow.py(409 lines, 7 functions: search_protein, retrieve_sequence, run_blast, discover_pathways, find_interactions, get_go_annotations, main): End-to-end pipeline → Common Workflow 1 "Complete Protein Analysis Pipeline"compound_cross_reference.py(379 lines, 7 functions: search_kegg_compound, get_kegg_info, get_chembl_id, get_chebi_info, get_chembl_info, save_results, main): Multi-DB compound search → Common Workflow 2 "Cross-Database Compound Search"batch_id_converter.py(348 lines, 8 functions: normalize_database_code, read_ids_from_file, batch_convert, save_mapping_csv, save_failed_ids, print_mapping_summary, list_common_databases, main): Batch conversion → Common Workflow 3 "Batch Identifier Conversion" + Core API section 5pathway_analysis.py(310 lines, 8 functions: get_all_pathways, analyze_pathway, analyze_all_pathways, save_pathway_summary, save_interactions_sif, save_detailed_pathway_info, print_statistics, main): KGML network analysis → Core API section 2 (KGML parsing) + Recipe 1 "Pathway Network Statistics"- CLI argument parsing, CSV/SIF file I/O, and summary printing utilities from all scripts: omitted — trivial boilerplate not essential for API understanding
Omitted original content: workflow_patterns.md (811 lines, 7 detailed workflows) — 4 workflows consolidated into Common Workflows 1-3 and Recipe 3; remaining 3 workflows (gene annotation pipeline, interaction network analysis, comparative species analysis) omitted as they combine patterns already shown in Core API modules without introducing new API calls.
Related Skills
- gget-genomic-databases — deeper Ensembl/BLAST/enrichment queries via gget (simpler API for single-database access)
- chembl-database-bioactivity — dedicated ChEMBL bioactivity queries beyond compound lookup
- pubchem-compound-search — dedicated PubChem compound/assay queries with similarity search
References
- BioServices documentation: https://bioservices.readthedocs.io/
- BioServices source: https://github.com/cokelaer/bioservices
- Cokelaer et al. (2013) BioServices: a common Python package to access biological Web Services programmatically. Bioinformatics 29(24): 3241-3242
- UniProt REST API: https://www.uniprot.org/help/api
- KEGG API: https://www.kegg.jp/kegg/rest/keggapi.html
- QuickGO API: https://www.ebi.ac.uk/QuickGO/api/index.html
cBioPortal Database
Overview
cBioPortal for Cancer Genomics is a public repository of cancer genomics data including TCGA, ICGC, and hundreds of curated studies spanning 100+ cancer types. It provides somatic mutation profiles, copy number alterations (CNA), gene expression, clinical data (survival, stage, treatment history), and methylation data for tens of thousands of patient samples. Data is accessible via a REST API at https://www.cbioportal.org/api/ with no authentication required.
When to Use
- Retrieving somatic mutation profiles (variant type, amino acid change) for a gene across TCGA studies
- Querying copy number alteration data (amplification, deep deletion) for candidate cancer driver genes
- Accessing clinical data — overall survival, disease-free survival, tumor stage — for survival curve analysis
- Identifying which cancer studies have molecular profiling data for a specific cancer type (e.g., breast, lung)
- Downloading gene expression (RNA-seq FPKM/RSEM) data from specific TCGA cohorts for differential expression analysis
- Correlating genomic alterations with clinical outcomes in a specific study
- Use omics-plotting SKILL to render bar charts, mutation-frequency heatmaps, and Kaplan–Meier curves from the query results
- Use
gnomad-databaseinstead when you need population-level variant allele frequencies in healthy individuals - For drug-gene interaction lookups use
opentargets-database; cBioPortal provides the genomic alteration data, not drug interaction annotations
Prerequisites
- Python packages:
requests,pandas,matplotlib - Data requirements: Entrez gene symbols (e.g.,
TP53), cBioPortal study IDs (e.g.,tcga_brca), molecular profile IDs - Environment: internet connection; no API key required
- Rate limits: no strict rate limits; use
time.sleep(0.2)between batch requests for polite access
pip install requests pandas matplotlib
Quick Start
import requests
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
"""GET request to cBioPortal REST API, returns parsed JSON."""
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
# List available cancer types
cancer_types = cbio_get("cancer-types")
print(f"Total cancer types: {len(cancer_types)}")
# Total cancer types: 87
# Find TCGA breast cancer study
studies = cbio_get("studies", params={"keyword": "breast"})
brca = [s for s in studies if "tcga_brca" in s["studyId"]]
if brca:
s = brca[0]
print(f"Study: {s['name']}")
print(f" studyId: {s['studyId']}")
print(f" Samples: {s['allSampleCount']}")
# Study: Breast Invasive Carcinoma (TCGA, PanCancer Atlas)
# studyId: brca_tcga_pan_can_atlas_2018
# Samples: 1084
Core API
Query 1: Cancer Types and Studies
List available cancer types and find studies by cancer type or keyword.
import requests
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
# Get all cancer types
cancer_types = cbio_get("cancer-types")
ct_df = pd.DataFrame(cancer_types)[["cancerTypeId", "name", "dedicatedColor"]]
print(f"Cancer types: {len(ct_df)}")
print(ct_df.head(5).to_string(index=False))
# Find all studies for a cancer type
lung_studies = cbio_get("studies", params={"keyword": "lung adenocarcinoma"})
print(f"\nLung adenocarcinoma studies: {len(lung_studies)}")
for s in lung_studies[:3]:
print(f" {s['studyId']:40s} n={s['allSampleCount']}")
# Get detailed study metadata including available data types
study_id = "brca_tcga_pan_can_atlas_2018"
study = cbio_get(f"studies/{study_id}")
print(f"Study: {study['name']}")
print(f" Reference genome: {study.get('referenceGenome', 'n/a')}")
print(f" All sample count: {study['allSampleCount']}")
# List molecular profiles for the study
profiles = cbio_get("molecular-profiles", params={"studyId": study_id})
print(f"\nMolecular profiles ({len(profiles)} total):")
for p in profiles:
print(f" {p['molecularProfileId']:55s} [{p['molecularAlterationType']}]")
Query 2: Somatic Mutations
Retrieve mutation data for a gene or set of genes in a study's mutation profile.
import requests, json
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_post(endpoint, body):
"""POST request to cBioPortal REST API."""
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=60)
r.raise_for_status()
return r.json()
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
# Get all samples for a study
study_id = "brca_tcga_pan_can_atlas_2018"
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]
print(f"Total samples: {len(sample_ids)}")
# Mutation profile ID follows pattern: {studyId}_mutations
profile_id = f"{study_id}_mutations"
# Fetch mutations for TP53 (Entrez gene ID = 7157)
body = {
"sampleIds": sample_ids[:200], # first 200 samples
"entrezGeneIds": [7157] # TP53
}
mutations = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch", body)
print(f"TP53 mutations in first 200 samples: {len(mutations)}")
# Summarize by mutation type
mut_df = pd.DataFrame(mutations)
print("\nMutation type distribution:")
print(mut_df["mutationType"].value_counts().head(8).to_string())
# Missense_Mutation 102
# Nonsense_Mutation 28
# Splice_Site 14
# Frame_Shift_Del 12
Query 3: Copy Number Alterations
Fetch discrete CNA data (amplification = 2, gain = 1, diploid = 0, loss = -1, deep deletion = -2).
import requests
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_post(endpoint, body):
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=60)
r.raise_for_status()
return r.json()
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
study_id = "brca_tcga_pan_can_atlas_2018"
# CNA profile: discrete copy number data
cna_profile_id = f"{study_id}_gistic" # GISTIC-derived discrete CNA
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples][:300]
# Fetch CNA for ERBB2 (Entrez 2064) and MYC (Entrez 4609)
body = {
"sampleIds": sample_ids,
"entrezGeneIds": [2064, 4609] # ERBB2, MYC
}
cna_data = cbio_post(
f"molecular-profiles/{cna_profile_id}/molecular-data/fetch", body
)
print(f"CNA records retrieved: {len(cna_data)}")
cna_df = pd.DataFrame(cna_data)
# CNA values: 2=amplification, 1=gain, 0=diploid, -1=loss, -2=deep deletion
cna_label = {2: "AMP", 1: "GAIN", 0: "DIPLOID", -1: "LOSS", -2: "HOMDEL"}
print("\nERBB2 CNA distribution:")
erbb2 = cna_df[cna_df["entrezGeneId"] == 2064]
erbb2_counts = erbb2["value"].map(lambda x: cna_label.get(int(x), str(x))).value_counts()
print(erbb2_counts.to_string())
# DIPLOID 210
# AMP 62
# GAIN 18
# LOSS 10
Query 4: Clinical Data
Retrieve per-sample or per-patient clinical attributes including survival, tumor stage, and treatment.
import requests
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
study_id = "brca_tcga_pan_can_atlas_2018"
# List available clinical attributes for this study
attrs = cbio_get(f"studies/{study_id}/clinical-attributes")
attr_df = pd.DataFrame(attrs)[["clinicalAttributeId", "displayName", "datatype", "patientAttribute"]]
print(f"Clinical attributes: {len(attr_df)}")
# Show survival-related attributes
survival_attrs = attr_df[attr_df["clinicalAttributeId"].str.contains("SURVIVAL|MONTHS|STATUS", na=False)]
print(survival_attrs[["clinicalAttributeId", "displayName"]].to_string(index=False))
# Fetch OS_STATUS and OS_MONTHS for all patients
clinical = cbio_get(f"studies/{study_id}/clinical-data",
params={"clinicalDataType": "PATIENT",
"projection": "DETAILED"})
clin_df = pd.DataFrame(clinical)
# Pivot to patient × attribute matrix
clin_pivot = clin_df.pivot_table(
index="patientId", columns="clinicalAttributeId",
values="value", aggfunc="first"
)
print(f"\nPatients: {len(clin_pivot)}")
if "OS_STATUS" in clin_pivot.columns:
print("OS status counts:")
print(clin_pivot["OS_STATUS"].value_counts().to_string())
# OS status counts:
# 0:LIVING 765
# 1:DECEASED 319
Query 5: Gene Expression Data
Retrieve mRNA expression values (RSEM or FPKM) from RNA-seq profiles.
import requests
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_post(endpoint, body):
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=60)
r.raise_for_status()
return r.json()
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
study_id = "brca_tcga_pan_can_atlas_2018"
# RNA-seq profile (RSEM normalized values)
rna_profile_id = f"{study_id}_rna_seq_v2_mrna_median_normed_log2"
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples][:100]
# Fetch expression for ESR1 (Entrez 2099), ERBB2 (2064), PGR (5241)
body = {
"sampleIds": sample_ids,
"entrezGeneIds": [2099, 2064, 5241] # ESR1, ERBB2, PGR
}
expr_data = cbio_post(
f"molecular-profiles/{rna_profile_id}/molecular-data/fetch", body
)
expr_df = pd.DataFrame(expr_data)
print(f"Expression records: {len(expr_df)}")
# Pivot to gene × sample matrix
expr_pivot = expr_df.pivot_table(
index="sampleId", columns="entrezGeneId", values="value"
)
expr_pivot.columns = ["ERBB2", "ESR1", "PGR"] # rename by gene symbol
print(f"\nExpression matrix: {expr_pivot.shape}")
print(expr_pivot.describe().round(2))
Query 6: Gene Details and Batch Lookup
Look up gene metadata (symbol, Entrez ID, type) required to construct mutation and CNA queries.
import requests
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
def cbio_post(endpoint, body):
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=30)
r.raise_for_status()
return r.json()
# Single gene lookup by Hugo symbol
gene = cbio_get("genes/TP53")
print(f"TP53: entrezGeneId={gene['entrezGeneId']}, type={gene['type']}")
# TP53: entrezGeneId=7157, type=protein-coding
# Batch gene lookup — convert Hugo symbols to Entrez IDs
gene_symbols = ["BRCA1", "BRCA2", "TP53", "PIK3CA", "PTEN", "KRAS", "EGFR"]
body = {"geneIds": gene_symbols, "geneIdType": "HUGO_GENE_SYMBOL"}
gene_list = cbio_post("genes/fetch", body)
gene_map = {g["hugoGeneSymbol"]: g["entrezGeneId"] for g in gene_list}
gene_df = pd.DataFrame(gene_list)[["hugoGeneSymbol", "entrezGeneId", "type"]]
print(f"\nResolved {len(gene_df)} genes:")
print(gene_df.to_string(index=False))
# hugoGeneSymbol entrezGeneId type
# BRCA1 672 protein-coding
# BRCA2 675 protein-coding
# TP53 7157 protein-coding
Query 7: Visualization — Mutation Frequency Barplot
Compute mutation frequency across TCGA studies for a cancer driver gene, then read skills/data-visualization/omics-plotting/SKILL.md and follow its "Box / Violin / Bar" recipe on the exported CSV (→ figures/TP53_mutation_frequency.png).
import requests, time
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
def cbio_post(endpoint, body):
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=60)
r.raise_for_status()
return r.json()
# Focus on a curated set of TCGA PanCancer Atlas studies
STUDIES = {
"brca_tcga_pan_can_atlas_2018": "BRCA",
"luad_tcga_pan_can_atlas_2018": "LUAD",
"coad_tcga_pan_can_atlas_2018": "COAD",
"prad_tcga_pan_can_atlas_2018": "PRAD",
"gbm_tcga_pan_can_atlas_2018": "GBM",
}
GENE_ENTREZ = 7157 # TP53
GENE_SYMBOL = "TP53"
rows = []
for study_id, label in STUDIES.items():
try:
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]
n_total = len(sample_ids)
profile_id = f"{study_id}_mutations"
body = {"sampleIds": sample_ids, "entrezGeneIds": [GENE_ENTREZ]}
muts = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch", body)
mutated_samples = len({m["sampleId"] for m in muts})
rows.append({"study": label, "n_mutated": mutated_samples,
"n_total": n_total,
"freq": mutated_samples / n_total * 100})
time.sleep(0.2)
except Exception as e:
print(f" Skipping {study_id}: {e}")
df = pd.DataFrame(rows).sort_values("freq", ascending=True)
df.to_csv(f"{GENE_SYMBOL}_mutation_frequency.csv", index=False)
print(df[["study", "n_mutated", "n_total", "freq"]].to_string(index=False))
# Render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Box / Violin / Bar" recipe (horizontal bar of freq by study).
Key Concepts
cBioPortal Data Model
cBioPortal organizes data in a three-tier hierarchy: Cancer Studies → Molecular Profiles → Sample-level data. A single study (e.g., brca_tcga_pan_can_atlas_2018) contains multiple molecular profiles, each covering one data type. Before querying mutation or expression data, always retrieve the molecular profile list with GET /molecular-profiles?studyId={studyId} to confirm the correct profile ID.
Molecular Profile ID Conventions
| Data Type | Typical Profile ID Suffix | Alteration Type |
|---|---|---|
| Somatic mutations | _mutations | MUTATION_EXTENDED |
| Discrete CNA (GISTIC) | _gistic | COPY_NUMBER_ALTERATION |
| Continuous CNA (log2) | _log2CNA | COPY_NUMBER_ALTERATION |
| RNA-seq (log2 RSEM) | _rna_seq_v2_mrna_median_normed_log2 | MRNA_EXPRESSION |
| Methylation | _methylation_hm27 or _hm450 | METHYLATION |
Not all studies have all profile types. Always verify with GET /molecular-profiles?studyId={studyId}.
Entrez Gene IDs
The REST API mutation and molecular data endpoints require Entrez Gene IDs (integers), not Hugo symbols. Use GET /genes/{hugoSymbol} or POST /genes/fetch to resolve symbols to IDs before batch queries.
Common Workflows
Workflow 1: Somatic Mutation Landscape for a Gene Panel
Goal: Retrieve mutations for multiple cancer driver genes across an entire TCGA study and export to CSV.
import requests, time
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
def cbio_post(endpoint, body):
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=120)
r.raise_for_status()
return r.json()
study_id = "luad_tcga_pan_can_atlas_2018"
profile_id = f"{study_id}_mutations"
# Resolve gene symbols to Entrez IDs
gene_symbols = ["KRAS", "EGFR", "TP53", "BRAF", "STK11", "KEAP1", "RB1"]
gene_list = cbio_post("genes/fetch",
{"geneIds": gene_symbols, "geneIdType": "HUGO_GENE_SYMBOL"})
gene_map = {g["entrezGeneId"]: g["hugoGeneSymbol"] for g in gene_list}
entrez_ids = list(gene_map.keys())
# Fetch all samples
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]
print(f"Study: {study_id} — {len(sample_ids)} samples")
# Batch mutations in chunks of 500 samples to avoid timeouts
chunk_size = 500
all_muts = []
for i in range(0, len(sample_ids), chunk_size):
chunk = sample_ids[i:i + chunk_size]
body = {"sampleIds": chunk, "entrezGeneIds": entrez_ids}
muts = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch", body)
all_muts.extend(muts)
time.sleep(0.1)
mut_df = pd.DataFrame(all_muts)
mut_df["hugoSymbol"] = mut_df["entrezGeneId"].map(gene_map)
print(f"Total mutations: {len(mut_df)}")
print("\nMutation counts per gene:")
print(mut_df.groupby("hugoSymbol")["sampleId"].nunique()
.sort_values(ascending=False).to_string())
mut_df.to_csv(f"{study_id}_driver_mutations.csv", index=False)
print(f"\nSaved: {study_id}_driver_mutations.csv")
Workflow 2: Survival Analysis — CNA Status vs. Overall Survival
Goal: Compare overall survival between patients with ERBB2 amplification vs. diploid/loss in TCGA BRCA.
import requests
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
def cbio_post(endpoint, body):
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=60)
r.raise_for_status()
return r.json()
study_id = "brca_tcga_pan_can_atlas_2018"
cna_profile_id = f"{study_id}_gistic"
# Get all samples
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]
# Fetch ERBB2 CNA (Entrez 2064)
cna_data = cbio_post(
f"molecular-profiles/{cna_profile_id}/molecular-data/fetch",
{"sampleIds": sample_ids, "entrezGeneIds": [2064]}
)
cna_df = pd.DataFrame(cna_data)[["sampleId", "value"]].rename(columns={"value": "erbb2_cna"})
cna_df["erbb2_cna"] = cna_df["erbb2_cna"].astype(int)
cna_df["erbb2_status"] = cna_df["erbb2_cna"].map(
{2: "Amplified", 1: "Gain", 0: "Diploid", -1: "Loss", -2: "Deep Deletion"})
# Fetch clinical data (OS_STATUS, OS_MONTHS)
clinical = cbio_get(f"studies/{study_id}/clinical-data",
params={"clinicalDataType": "PATIENT", "projection": "DETAILED"})
clin_df = pd.DataFrame(clinical)
clin_pivot = clin_df.pivot_table(
index="patientId", columns="clinicalAttributeId", values="value", aggfunc="first"
).reset_index()
# Map samples to patients
sample_patient = cbio_get(f"studies/{study_id}/samples", params={"projection": "DETAILED"})
sp_df = pd.DataFrame(sample_patient)[["sampleId", "patientId"]]
# Merge CNA + clinical via patient ID
merged = (cna_df
.merge(sp_df, on="sampleId")
.merge(clin_pivot[["patientId", "OS_STATUS", "OS_MONTHS"]],
on="patientId", how="inner"))
merged = merged.dropna(subset=["OS_STATUS", "OS_MONTHS"])
merged["OS_MONTHS"] = pd.to_numeric(merged["OS_MONTHS"], errors="coerce")
merged["event"] = (merged["OS_STATUS"] == "1:DECEASED").astype(int)
# Survival table for the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Kaplan–Meier" recipe (columns: time, event, group)
km_input = (merged.loc[merged["erbb2_status"].isin(["Amplified", "Diploid"]),
["OS_MONTHS", "event", "erbb2_status"]]
.rename(columns={"OS_MONTHS": "time", "erbb2_status": "group"}))
km_input.to_csv("erbb2_survival.csv", index=False)
print(f"Survival table: {len(km_input)} patients -> erbb2_survival.csv")
print(km_input["group"].value_counts().to_string())
# Render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Kaplan–Meier" recipe -> figures/erbb2_survival.png
Workflow 3: Multi-Study Alteration Frequency Heatmap
Goal: Build a gene × cancer-type alteration frequency matrix across TCGA studies.
import requests, time
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
def cbio_post(endpoint, body):
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=90)
r.raise_for_status()
return r.json()
STUDIES = {
"brca_tcga_pan_can_atlas_2018": "BRCA",
"luad_tcga_pan_can_atlas_2018": "LUAD",
"coad_tcga_pan_can_atlas_2018": "COAD",
"gbm_tcga_pan_can_atlas_2018": "GBM",
}
GENE_SYMBOLS = ["TP53", "KRAS", "PIK3CA", "EGFR", "PTEN"]
# Resolve genes
gene_list = cbio_post("genes/fetch",
{"geneIds": GENE_SYMBOLS, "geneIdType": "HUGO_GENE_SYMBOL"})
gene_map = {g["entrezGeneId"]: g["hugoGeneSymbol"] for g in gene_list}
entrez_ids = list(gene_map.keys())
freq_matrix = pd.DataFrame(index=GENE_SYMBOLS, columns=list(STUDIES.values()), dtype=float)
for study_id, label in STUDIES.items():
try:
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]
n_total = len(sample_ids)
profile_id = f"{study_id}_mutations"
body = {"sampleIds": sample_ids, "entrezGeneIds": entrez_ids}
muts = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch", body)
mut_df = pd.DataFrame(muts) if muts else pd.DataFrame()
for eid, symbol in gene_map.items():
if mut_df.empty:
freq_matrix.loc[symbol, label] = 0.0
else:
n_mut = mut_df[mut_df["entrezGeneId"] == eid]["sampleId"].nunique()
freq_matrix.loc[symbol, label] = n_mut / n_total * 100
time.sleep(0.2)
except Exception as e:
print(f" {label}: {e}")
freq_matrix = freq_matrix.fillna(0).astype(float)
freq_matrix.to_csv("mutation_frequency_matrix.csv")
print(freq_matrix.to_string())
# Render with the omics-plotting SKILL (`skills/data-visualization/omics-plotting/SKILL.md`) "Expression heatmap" recipe (gene x cancer-type, YlOrRd,
# annotated) -> figures/mutation_frequency_heatmap.png
Key Parameters
| Parameter | Function/Endpoint | Default | Range / Options | Effect |
|---|---|---|---|---|
studyId | All study endpoints | — | any valid cBioPortal study ID | Selects the cancer study |
molecularProfileId | mutations/fetch, molecular-data/fetch | — | {studyId}_mutations, {studyId}_gistic, etc. | Selects the data type profile |
entrezGeneIds | mutations/fetch, molecular-data/fetch | — | list of integer Entrez IDs | Genes to query; use POST /genes/fetch to resolve symbols |
sampleIds | mutations/fetch, molecular-data/fetch | — | list of sample ID strings | Samples to retrieve; use GET /studies/{id}/samples for all |
clinicalDataType | clinical-data | "SAMPLE" | "SAMPLE", "PATIENT" | Whether to return sample-level or patient-level clinical attributes |
projection | samples, clinical-data | "SUMMARY" | "ID", "SUMMARY", "DETAILED", "META" | Response verbosity; "ID" fastest for ID-only fetches |
keyword | studies | "" | free text | Filter studies by name/cancer type keyword |
Best Practices
-
Fetch sample IDs before data queries: All mutation and molecular data endpoints require explicit
sampleIds. Retrieve them withGET /studies/{studyId}/samples?projection=IDbefore each query. -
Verify profile IDs from the API: Profile IDs are not guaranteed to follow the
_mutations/_gisticpattern in every study. Always confirm withGET /molecular-profiles?studyId={studyId}rather than guessing. -
Chunk large sample sets: The API can time out on requests with thousands of sample IDs. Batch requests in chunks of 500 samples with
time.sleep(0.1)between chunks. -
Use Entrez IDs, not Hugo symbols, in data fetch endpoints: The mutation and molecular data endpoints accept
entrezGeneIds(integers). Resolve symbols first withPOST /genes/fetch. -
Don't hard-code Entrez IDs: Gene IDs can be looked up dynamically via the API. Hard-coded IDs become incorrect if the gene model changes. Use
POST /genes/fetchto resolve gene symbols at runtime.
Common Recipes
Recipe: List All Molecular Profiles for a Study
When to use: Before running any data query — verify which profile IDs are available.
import requests
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
study_id = "brca_tcga_pan_can_atlas_2018"
profiles = cbio_get("molecular-profiles", params={"studyId": study_id})
for p in profiles:
print(f"{p['molecularProfileId']:55s} {p['molecularAlterationType']}")
# brca_tcga_pan_can_atlas_2018_mutations MUTATION_EXTENDED
# brca_tcga_pan_can_atlas_2018_gistic COPY_NUMBER_ALTERATION
# brca_tcga_pan_can_atlas_2018_log2CNA COPY_NUMBER_ALTERATION
# brca_tcga_pan_can_atlas_2018_rna_seq_v2_mrna_median_normed_log2 MRNA_EXPRESSION
Recipe: Download Full Mutation MAF for a Study
When to use: Export all somatic mutations from a study into MAF-compatible format for downstream analysis.
import requests, time
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=60)
r.raise_for_status()
return r.json()
def cbio_post(endpoint, body):
r = requests.post(f"{BASE_URL}/{endpoint}", json=body,
headers={"Accept": "application/json",
"Content-Type": "application/json"},
timeout=120)
r.raise_for_status()
return r.json()
study_id = "coad_tcga_pan_can_atlas_2018"
profile_id = f"{study_id}_mutations"
samples = cbio_get(f"studies/{study_id}/samples", params={"projection": "ID"})
sample_ids = [s["sampleId"] for s in samples]
all_mutations = []
for i in range(0, len(sample_ids), 300):
chunk = sample_ids[i:i + 300]
muts = cbio_post(f"molecular-profiles/{profile_id}/mutations/fetch",
{"sampleIds": chunk, "entrezGeneIds": []}) # empty = all genes
all_mutations.extend(muts)
time.sleep(0.1)
mut_df = pd.DataFrame(all_mutations)
cols = ["hugoGeneSymbol", "sampleId", "chr", "startPosition", "endPosition",
"referenceAllele", "variantAllele", "mutationType",
"proteinChange", "variantType"]
available = [c for c in cols if c in mut_df.columns]
mut_df[available].to_csv(f"{study_id}_mutations.csv", index=False)
print(f"Saved {len(mut_df)} mutations → {study_id}_mutations.csv")
Recipe: Query Patient-Level Clinical Attribute
When to use: Extract a specific clinical variable (e.g., tumor stage, age at diagnosis) for all patients.
import requests
import pandas as pd
BASE_URL = "https://www.cbioportal.org/api"
def cbio_get(endpoint, params=None):
r = requests.get(f"{BASE_URL}/{endpoint}", params=params,
headers={"Accept": "application/json"}, timeout=30)
r.raise_for_status()
return r.json()
study_id = "brca_tcga_pan_can_atlas_2018"
# Fetch a specific clinical attribute for all patients
attr_id = "TUMOR_STAGE"
clinical = cbio_get(f"studies/{study_id}/clinical-data",
params={"clinicalDataType": "PATIENT",
"projection": "DETAILED"})
clin_df = pd.DataFrame(clinical)
if "clinicalAttributeId" in clin_df.columns:
stage_df = clin_df[clin_df["clinicalAttributeId"] == attr_id][["patientId", "value"]]
print(f"Patients with {attr_id} annotation: {len(stage_df)}")
print(stage_df["value"].value_counts().head(10).to_string())
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
404 Not Found on profile endpoint | Molecular profile does not exist for study | List profiles with GET /molecular-profiles?studyId={id}; confirm the profile ID |
| Empty mutations list | Gene has no mutations in the selected samples/profile | Verify study has a mutation profile; check sample IDs belong to the same study |
requests.exceptions.Timeout | Large sample set (>1000) in a single request | Chunk requests to 300–500 samples; increase timeout to 120s |
entrezGeneIds key error in response | Hugo symbol passed instead of Entrez ID | Use POST /genes/fetch to resolve symbols to integer Entrez IDs first |
| CNA values returned as strings | value field is string in JSON | Cast with pd.to_numeric() or int(value) before comparison |
| Expression profile not found | Study uses non-standard profile naming | Check profile list; look for MRNA_EXPRESSION alteration type in GET /molecular-profiles |
| Survival analysis has many NA values | Clinical attribute absent for some patients | Use dropna() on OS columns; check attribute availability with GET /studies/{id}/clinical-attributes |
Related Skills
gnomad-database— population variant allele frequencies for healthy cohorts (complement to cBioPortal somatic data)cnvkit-copy-number— CNVkit pipeline for generating SEG/CNA files that can be loaded into cBioPortalpydeseq2-differential-expression— differential expression analysis that can be applied to cBioPortal RNA-seq exports
References
- cBioPortal API Swagger UI — interactive REST API explorer with all endpoints
- cBioPortal GitHub — source code and issue tracker
- Cerami E et al., Cancer Discovery 2012 — original cBioPortal paper
- Gao J et al., Science Signaling 2013 — integrative analysis methods using cBioPortal
- TCGA PanCancer Atlas — primary data source for PanCancer Atlas studies
ClinPGx (PharmGKB) Pharmacogenomics Database
Overview
PharmGKB rebranded as ClinPGx in 2024 and the API moved from api.pharmgkb.org to api.clinpgx.org. The old host now returns 404/405; every example here uses the new endpoints. Two complementary APIs are used together:
- ClinPGx Data API (
api.clinpgx.org/v1) — record-style access to genes, drugs, variants, clinical annotations, guideline annotations, drug labels, and pathways. Responses wrap data as{"data": [...], "status": "success"}. Filters use dotted property paths (e.g.relatedChemicals.name=clopidogrel,levelOfEvidence.term=1A). - CPIC PostgREST API (
api.cpicpgx.org/v1) — relational lookup of genotype → drug recommendation rows. PostgREST filter syntax (column=eq.value, JSONcs.{...}for jsonb containment). Returns flat JSON arrays.
Use ClinPGx for what is known about a gene/drug/variant; use CPIC for how to prescribe given a phenotype. The pattern is ClinPGx for annotations, CPIC for recommendations.
When to Use
- Retrieving CPIC genotype-specific dosing recommendations for a gene-drug pair (e.g., CYP2C19 + clopidogrel) — use CPIC
- Looking up all pharmacogenomic clinical annotations for a drug or evidence level — use ClinPGx
data/clinicalAnnotation - Finding all CPIC/DPWG guideline annotations for a pharmacogene — use ClinPGx
data/guidelineAnnotation - Resolving a gene symbol, drug name, or rsID to ClinPGx PA identifiers — use
data/{gene,drug,variant} - Free-text search across all ClinPGx record types (genes, drugs, variants, annotations) — use
POST /site/search - Retrieving FDA/EMA pharmacogenomic drug label annotations — use ClinPGx
data/label - Building precision-medicine prescribing workflows that combine annotation evidence with phenotype-specific recommendations
- For germline disease pathogenicity (not PGx) use
clinvar-database - For somatic cancer pharmacogenomics use
cosmic-databaseoropentargets-database
Prerequisites
- Python packages:
requests,pandas— both already in standard environments - Data requirements: HGNC gene symbols, drug names (lowercase generic), dbSNP rsIDs, or PA identifiers
- Environment: internet connection; no authentication required for either host
- Rate limits: the ClinPGx host occasionally returns HTTP 429; insert
time.sleep(0.3–0.5)between sequential calls. CPIC is more permissive.
If you are inside a pixi/conda environment that already provides requests and pandas, skip the install — invoke scripts with pixi run python ....
pip install requests pandas
Quick Start
import requests
CLINPGX = "https://api.clinpgx.org/v1"
CPIC = "https://api.cpicpgx.org/v1"
# CPIC genotype → recommendation: clopidogrel + CYP2C19 Poor Metabolizer
drug = requests.get(f"{CPIC}/drug", params={"name": "eq.clopidogrel"}).json()[0]
recs = requests.get(f"{CPIC}/recommendation",
params={"drugid": f"eq.{drug['drugid']}",
"phenotypes": 'cs.{"CYP2C19":"Poor Metabolizer"}'}).json()
print(f"clopidogrel CYP2C19=PM: {len(recs)} recommendation(s)")
for rec in recs[:2]:
print(f" [{rec['classification']}] {rec['drugrecommendation'][:80]}…")
# ClinPGx side: how many CPIC guideline annotations cover CYP2C19?
glines = requests.get(f"{CLINPGX}/data/guidelineAnnotation",
params={"relatedGenes.symbol": "CYP2C19",
"source": "CPIC", "view": "base"}).json()["data"]
print(f"CYP2C19 CPIC guidelines: {len(glines)}")
Core API
Module 1: Free-text site search
POST /site/search with a JSON body {"query": "<term>"} is the canonical entry point when you don't know the PA ID. It searches across drugs, genes, variants, clinical annotations, guideline annotations, and labels in one shot.
import requests
CLINPGX = "https://api.clinpgx.org/v1"
r = requests.post(f"{CLINPGX}/site/search",
json={"query": "rs4149056"}, timeout=15)
r.raise_for_status()
hits = r.json()["data"]["hits"]
print(f"Total hits: {r.json()['data']['total']}")
for h in hits[:5]:
print(f" id={h.get('id')} name={h.get('name')[:80]}")
# Broader concept search
r = requests.post(f"{CLINPGX}/site/search",
json={"query": "TPMT azathioprine"}, timeout=15)
hits = r.json()["data"]["hits"]
print(f"TPMT+azathioprine hits: {len(hits)}")
for h in hits[:5]:
print(f" {h.get('id'):>15} {h.get('name','')[:80]}")
Module 2: Gene, drug, and variant record lookup
The /data/{type} endpoints accept simple property filters. All return {"data": [...], "status": "success"} — use view=base for summary, view=max for full nested objects.
import requests
CLINPGX = "https://api.clinpgx.org/v1"
# Gene by HGNC symbol
gene = requests.get(f"{CLINPGX}/data/gene",
params={"symbol": "CYP2D6", "view": "base"}).json()["data"][0]
print(f"{gene['symbol']} id={gene['id']} {gene['name']}")
# Drug by name (lowercase generic preferred)
drug = requests.get(f"{CLINPGX}/data/drug",
params={"name": "warfarin", "view": "base"}).json()["data"][0]
print(f"{drug['name']} id={drug['id']}")
# Variant by rsID
var = requests.get(f"{CLINPGX}/data/variant",
params={"name": "rs4149056", "view": "base"}).json()["data"][0]
print(f"{var['name']} id={var['id']} significance={var.get('clinicalSignificance')}")
# Direct record fetch when you already have a PA ID
r = requests.get(f"{CLINPGX}/data/drug/PA449088", params={"view": "max"}).json()
d = r["data"]
print(f"PA449088 → {d['name']} (objCls={d['objCls']})")
Module 3: Clinical annotations
data/clinicalAnnotation records associate a variant (location) with one or more drugs (relatedChemicals) and an evidence level (levelOfEvidence.term). The two supported filters are relatedChemicals.name= and levelOfEvidence.term=. There is no working gene= filter on this endpoint — see Module 4 for gene-driven access.
import requests, pandas as pd
CLINPGX = "https://api.clinpgx.org/v1"
# All clinical annotations for clopidogrel
data = requests.get(f"{CLINPGX}/data/clinicalAnnotation",
params={"relatedChemicals.name": "clopidogrel",
"view": "base"}).json()["data"]
print(f"clopidogrel annotations: {len(data)}")
rows = []
for ann in data[:10]:
loc = ann.get("location") or {}
drugs = ", ".join(c.get("name", "") for c in ann.get("relatedChemicals", []))
rows.append({
"id": ann["id"],
"variant": loc.get("displayName"),
"gene": (loc.get("genes") or [{}])[0].get("symbol"),
"drug": drugs,
"level": (ann.get("levelOfEvidence") or {}).get("term"),
"score": ann.get("score"),
})
print(pd.DataFrame(rows).to_string(index=False))
# All Level 1A clinical annotations (highest evidence)
data = requests.get(f"{CLINPGX}/data/clinicalAnnotation",
params={"levelOfEvidence.term": "1A",
"view": "base"}).json()["data"]
print(f"Level 1A annotations: {len(data)}")
drug_to_count = {}
for ann in data:
for c in ann.get("relatedChemicals") or []:
drug_to_count[c["name"]] = drug_to_count.get(c["name"], 0) + 1
top = sorted(drug_to_count.items(), key=lambda x: -x[1])[:10]
for d, n in top:
print(f" {n:3} {d}")
Module 4: Guideline annotations (gene-driven access)
data/guidelineAnnotation supports both relatedGenes.symbol= and relatedChemicals.name=, plus source= (CPIC, DPWG, CPNDS, RNPGx). This is the canonical way to get gene→guideline coverage.
import requests
CLINPGX = "https://api.clinpgx.org/v1"
# All CPIC guidelines mentioning CYP2C19
data = requests.get(f"{CLINPGX}/data/guidelineAnnotation",
params={"relatedGenes.symbol": "CYP2C19",
"source": "CPIC",
"view": "base"}).json()["data"]
print(f"CYP2C19 CPIC guidelines: {len(data)}")
for g in data[:5]:
print(f" PA{g['id']}: {g['name'][:80]}")
# Guidelines for a specific drug across all bodies (CPIC, DPWG, …)
data = requests.get(f"{CLINPGX}/data/guidelineAnnotation",
params={"relatedChemicals.name": "clopidogrel",
"view": "base"}).json()["data"]
by_source = {}
for g in data:
for s in (g.get("crossReferences") or []):
by_source.setdefault(s.get("resource", "?"), 0)
by_source[s["resource"]] = by_source.get(s["resource"], 0) + 1
print(f"clopidogrel guidelines: {len(data)} ({list({g.get('source') for g in data})})")
Module 5: Regulatory drug labels (FDA / EMA)
data/label records are PharmGKB-curated annotations of FDA/EMA pharmacogenomic labeling. Filter by relatedChemicals.name= and source= (FDA, EMA, HCSC, PMDA, Swissmedic).
import requests, pandas as pd
CLINPGX = "https://api.clinpgx.org/v1"
data = requests.get(f"{CLINPGX}/data/label",
params={"relatedChemicals.name": "warfarin",
"source": "FDA",
"view": "base"}).json()["data"]
print(f"warfarin FDA labels: {len(data)}")
rows = [{
"name": d["name"][:60],
"biomarker_status": d.get("biomarkerStatus"),
"testing_required": d.get("testingRequired"),
"alternate_drug": d.get("alternateDrugAvailable"),
} for d in data]
print(pd.DataFrame(rows).to_string(index=False))
Module 6: CPIC genotype → recommendation chain
CPIC's PostgREST API uses column=eq.value for equality and column=cs.{...} for JSONB containment. The standard lookup chain is drug → drugid → recommendation, optionally filtered by phenotype.
import requests
CPIC = "https://api.cpicpgx.org/v1"
# Resolve drug name to drugid (RxNorm-prefixed)
drug = requests.get(f"{CPIC}/drug",
params={"name": "eq.clopidogrel"}).json()[0]
print(f"clopidogrel drugid: {drug['drugid']}")
# All phenotype-specific recommendations for clopidogrel
recs = requests.get(f"{CPIC}/recommendation",
params={"drugid": f"eq.{drug['drugid']}"}).json()
print(f"Total recommendations: {len(recs)}")
for rec in recs[:3]:
print(f" {rec['phenotypes']} [{rec['classification']}]")
print(f" {rec['drugrecommendation'][:90]}…")
# Phenotype filter via jsonb containment (cs.{...})
# The phenotypes column is a jsonb dict; cs. checks that the query is a subset.
recs = requests.get(f"{CPIC}/recommendation",
params={"drugid": f"eq.{drug['drugid']}",
"phenotypes": 'cs.{"CYP2C19":"Poor Metabolizer"}'}
).json()
for rec in recs:
print(f" [{rec['classification']}] {rec['drugrecommendation'][:90]}…")
# Gene-driven: list every drug with a CPIC pair for CYP2C19
pairs = requests.get(f"{CPIC}/pair",
params={"genesymbol": "eq.CYP2C19"}).json()
print(f"\nCYP2C19 CPIC pairs: {len(pairs)}")
drug_ids = sorted({p["drugid"] for p in pairs})
print(f"Sample drug IDs: {drug_ids[:5]}")
Key Concepts
Two-host architecture
| Question | Use | Why |
|---|---|---|
| What clinical annotations exist for this drug? | ClinPGx data/clinicalAnnotation | Annotation-level evidence with curated levelOfEvidence.term |
| What CPIC guidelines cover this gene? | ClinPGx data/guidelineAnnotation | Filter by relatedGenes.symbol; no working gene= filter on clinicalAnnotation |
| Given phenotype X, what should I prescribe? | CPIC recommendation + phenotypes | Structured genotype→action rows; CPIC is the prescribing-rule oracle |
| What FDA labels mention this drug + gene? | ClinPGx data/label?source=FDA | Curated regulatory PGx labeling |
| Free-text "anything about X" | ClinPGx POST /site/search | Cross-record-type fan-out |
PharmGKB / ClinPGx evidence levels
Levels 1A → 4 in decreasing evidence quality:
- 1A — Annotation of a variant–drug pair in a clinical guideline or FDA label (strongest)
- 1B — Significant association replicated in multiple studies
- 2A — Variant in a known PGx gene, significant association
- 2B — Moderate evidence, often single study
- 3 — Limited evidence (single study or unreplicated)
- 4 — Case reports / biological plausibility only
Filter via levelOfEvidence.term on data/clinicalAnnotation. The term is a string, not an enum ("1A" not 1A).
ClinPGx response envelope and view modes
Every ClinPGx /data/... response is {"data": [...] | {...}, "status": "success" | "fail"}. On failure the body is {"status": "fail", "data": {"errors": [{"message": "..."}]}} — always read both keys.
view=base(default) — flat summary record; recommended for bulk filtersview=max— full nested objects (relatedDiseases, allelePhenotypes, scoreDetails, …). Larger payload, slower; use only for single-record details.
Common Workflows
Workflow 1: Pharmacogene panel CPIC coverage
Goal: Given a patient's pharmacogene panel, count how many CPIC guideline annotations cover each gene.
import requests, pandas as pd, time
CLINPGX = "https://api.clinpgx.org/v1"
pharmacogenes = ["CYP2D6", "CYP2C19", "CYP2C9", "DPYD", "TPMT", "SLCO1B1"]
rows = []
for g in pharmacogenes:
data = requests.get(f"{CLINPGX}/data/guidelineAnnotation",
params={"relatedGenes.symbol": g,
"source": "CPIC", "view": "base"},
timeout=20).json()["data"]
drugs = sorted({c["name"] for guideline in data
for c in (guideline.get("relatedChemicals") or [])})
rows.append({"gene": g, "cpic_guidelines": len(data),
"n_drugs": len(drugs), "sample": ", ".join(drugs[:3])})
time.sleep(0.3)
df = pd.DataFrame(rows).sort_values("cpic_guidelines", ascending=False)
print(df.to_string(index=False))
df.to_csv("pharmacogene_cpic_coverage.csv", index=False)
Workflow 2: Drug panel — CPIC prescribing rule lookup
Goal: Given a prescribed drug list, identify which have CPIC genotype-specific recommendations and surface the rule rows.
import requests, pandas as pd, time
CPIC = "https://api.cpicpgx.org/v1"
drugs = ["warfarin", "clopidogrel", "codeine", "simvastatin",
"metoprolol", "omeprazole", "azathioprine", "tacrolimus"]
rows = []
for name in drugs:
drug = requests.get(f"{CPIC}/drug", params={"name": f"eq.{name}"}, timeout=15).json()
if not drug:
rows.append({"drug": name, "in_cpic": False, "n_recs": 0, "phenotypes": ""}); continue
did = drug[0]["drugid"]
recs = requests.get(f"{CPIC}/recommendation",
params={"drugid": f"eq.{did}"}, timeout=15).json()
phens = sorted({f"{k}={v}" for rec in recs
for k, v in (rec.get("phenotypes") or {}).items()})
rows.append({"drug": name, "in_cpic": True, "n_recs": len(recs),
"phenotypes": "; ".join(phens[:3])})
time.sleep(0.3)
df = pd.DataFrame(rows).sort_values(["in_cpic", "n_recs"], ascending=[False, False])
print(df.to_string(index=False))
Workflow 3: Variant → drug interactions (rsID-driven)
Goal: Starting from a single rsID (e.g., SLCO1B1 *5 = rs4149056), find every clinical annotation that involves it.
The Data API does not accept rsID as a filter property. Use POST /site/search to discover related annotation IDs, then fetch each by ID.
import requests
CLINPGX = "https://api.clinpgx.org/v1"
rsid = "rs4149056"
hits = requests.post(f"{CLINPGX}/site/search",
json={"query": rsid}, timeout=15).json()["data"]["hits"]
print(f"{rsid}: {len(hits)} hits")
# Filter hits that look like clinical annotations
ann_hits = [h for h in hits if h.get("name", "").lower().startswith("clinical annotation")]
print(f"Clinical-annotation hits: {len(ann_hits)}")
for h in ann_hits[:5]:
print(f" id={h['id']} {h['name'][:90]}")
# Dereference one annotation by ID for full detail
if ann_hits:
ann = requests.get(f"{CLINPGX}/data/clinicalAnnotation/{ann_hits[0]['id']}",
params={"view": "max"}, timeout=15).json()["data"]
drugs = ", ".join(c["name"] for c in (ann.get("relatedChemicals") or []))
print(f"\nFirst annotation:")
print(f" drugs: {drugs}")
print(f" level: {(ann.get('levelOfEvidence') or {}).get('term')}")
Key Parameters
| Parameter | Module / Endpoint | Default | Range / Options | Effect |
|---|---|---|---|---|
view | all /data/... | base | base, min, max | Field detail level; max includes all nested arrays (slow but complete) |
relatedChemicals.name | clinicalAnnotation, variantAnnotation, guidelineAnnotation, label, pathway | — | lowercase generic drug name | Filter records related to a drug |
relatedGenes.symbol | guidelineAnnotation, pathway | — | HGNC gene symbol | Filter records related to a gene (not available on clinicalAnnotation) |
levelOfEvidence.term | clinicalAnnotation | — | "1A", "1B", "2A", "2B", "3", "4" | Minimum evidence level |
source | guidelineAnnotation, label | — | CPIC, DPWG, FDA, EMA, HCSC, PMDA, Swissmedic | Issuing body |
symbol | data/gene | — | HGNC gene symbol | Gene record lookup |
name | data/drug, data/variant | — | drug name or rsID | Record lookup by canonical name |
CPIC column=eq.value | all api.cpicpgx.org/v1/... | — | PostgREST equality | Filter by exact match |
CPIC phenotypes=cs.{json} | recommendation | — | JSON-encoded jsonb subset | Filter by phenotype containment (must URL-encode if special chars) |
Best Practices
-
Resolve PA identifiers once. Never hand-construct ClinPGx PA IDs. Call
data/{type}?{symbol|name}=...(orsite/search) once and cache the returnedidfor reuse —gene/PA128for CYP2D6,drug/PA449088for clopidogrel,variant/PA166154579for rs4149056. -
Pick the right host for the question. Use ClinPGx for
what is annotatedand CPIC forwhat to prescribe. Trying to derive genotype-specific recommendations from ClinPGx alone misses the structuredrecommendation.phenotypesrows. -
Filter by evidence level upfront when building clinical workflows.
levelOfEvidence.term=1Areturns 312 actionable annotations across all of ClinPGx; Level 3/4 records are exploratory and shouldn't drive prescribing. -
Don't filter
clinicalAnnotationby gene — filter byguidelineAnnotationwithrelatedGenes.symbol. TheclinicalAnnotationendpoint has no working gene property and returns HTTP 400 for any attempt. -
Use
view=basefor bulk filters,view=maxfor single-record drill-downs. A list query withview=maxcan time out or hit 429; the difference is roughly 5–10× payload size. -
Throttle the ClinPGx host. Insert
time.sleep(0.3)between sequential queries in loops; the API returns occasional HTTP 429s on tight loops. CPIC tolerates faster iteration. -
URL-encode
cs.{...}jsonb filters when phenotype values contain spaces or special characters.requests.get(..., params={"phenotypes": 'cs.{"CYP2C19":"Poor Metabolizer"}'})works becauserequestsdoes the encoding; a manual URL string needsurllib.parse.quote.
Common Recipes
Recipe 1 — Free-text discovery via site/search
When to use: you have an arbitrary string (rsID, drug name, gene, allele) and want to find related ClinPGx records without knowing which endpoint to hit.
import requests
r = requests.post("https://api.clinpgx.org/v1/site/search",
json={"query": "VKORC1 warfarin"}, timeout=15)
hits = r.json()["data"]["hits"]
for h in hits[:10]:
print(f" {h.get('id'):>15} {h.get('name','')[:80]}")
Recipe 2 — Top drugs by Level 1A annotation count
When to use: build a leaderboard of the most actionable PGx drugs.
import requests, pandas as pd
data = requests.get("https://api.clinpgx.org/v1/data/clinicalAnnotation",
params={"levelOfEvidence.term": "1A", "view": "base"},
timeout=30).json()["data"]
counts = {}
for ann in data:
for c in ann.get("relatedChemicals") or []:
counts[c["name"]] = counts.get(c["name"], 0) + 1
df = pd.DataFrame(sorted(counts.items(), key=lambda x: -x[1]),
columns=["drug", "n_1A_annotations"]).head(15)
print(df.to_string(index=False))
Recipe 3 — Patient genotype → drug recommendations
When to use: given a phenotype call from a PGx test, surface every CPIC recommendation row.
import requests
CPIC = "https://api.cpicpgx.org/v1"
genotype = {"CYP2C19": "Poor Metabolizer"}
drug = "clopidogrel"
did = requests.get(f"{CPIC}/drug", params={"name": f"eq.{drug}"}).json()[0]["drugid"]
import json
recs = requests.get(f"{CPIC}/recommendation",
params={"drugid": f"eq.{did}",
"phenotypes": f"cs.{json.dumps(genotype)}"}).json()
for rec in recs:
print(f"[{rec['classification']}] {rec['drugrecommendation']}")
print(f" implications: {rec['implications']}")
Recipe 4 — Robust session with retry
When to use: long-running loops over many genes / drugs / variants.
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
s = requests.Session()
s.headers.update({"Accept": "application/json"})
s.mount("https://", HTTPAdapter(max_retries=Retry(
total=4, backoff_factor=1.0,
status_forcelist=[429, 500, 502, 503, 504],
allowed_methods=["GET", "POST"])))
r = s.get("https://api.clinpgx.org/v1/data/gene",
params={"symbol": "CYP2D6", "view": "base"}, timeout=20)
r.raise_for_status()
print(r.json()["data"][0]["name"])
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
HTTP 404/405 on https://api.pharmgkb.org/v1/... | Old PharmGKB host is dead; the service rebranded to ClinPGx in 2024 | Migrate to https://api.clinpgx.org/v1/.... Old /clinicalAnnotation?gene=X is now data/clinicalAnnotation with different filters. |
{"status":"fail","data":{"errors":[{"message":"No such property: 'gene'"}]}} | data/clinicalAnnotation does not accept gene= or relatedGenes.symbol= | Use data/guidelineAnnotation?relatedGenes.symbol=X for gene-driven access, or ?relatedChemicals.name=Y for drug-driven. |
{"status":"fail","data":{"errors":[{"message":"Missing criteria."}]}} | A data/{type} list query has no filter and no ID | Add at least one filter (name=, symbol=, relatedChemicals.name=, …) or fetch by ID via data/{type}/{paId}. |
HTTP 405 on GET /site/search?query=... | site/search only accepts POST with a JSON body | Use requests.post(url, json={"query": "..."}). |
| HTTP 429 mid-loop | Hit ClinPGx rate limit | Insert time.sleep(0.3–0.5) between calls; use the Retry session in Recipe 4. |
HTTP 400 on https://api.cpicpgx.org/v1/recommendation?phenotypes=cs.{...} | The cs. JSON wasn't URL-encoded | Pass via requests params={"phenotypes": 'cs.{"CYP2C19":"Poor Metabolizer"}'} (auto-encoded) or urllib.parse.quote manually. |
Empty data list for an obviously-real drug | Drug name mismatch (brand vs. generic; capitalization) | Try lowercase generic name; fall back to POST /site/search to fan out and find the canonical PA ID. |
data/variant?name=rs... returns 1 record but data/clinicalAnnotation?location.name=rs... returns 404 | rsID is stored under location.displayName/location.rsid, not exposed as a filterable property | Use site/search to discover annotation IDs by rsID, then dereference each with data/clinicalAnnotation/{id}. (Workflow 3.) |
Related Skills
clinvar-database— germline pathogenicity / clinical significance for variants found in PharmGKB (complementary; ClinVar is disease-focused, ClinPGx is drug-response-focused)opentargets-database— drug-target associations and safety signals overlapping ClinPGx pharmacogene targetschembl-database-bioactivity— bioactivity and binding data for the drugs annotated in ClinPGxcosmic-database— somatic cancer mutations and tumor-specific PGx (orthogonal to germline PGx covered here)
References
- ClinPGx (formerly PharmGKB) website — Database front end; web links match the
data/{type}/{paId}URL shape - ClinPGx REST API documentation — Official endpoint reference,
data/...andsite/searchschemas - CPIC API documentation — Swagger UI for the PostgREST companion API
- CPIC guidelines — Source of
recommendationrows; canonical genotype-prescribing oracle - Relling & Klein (2011) Nature Reviews Drug Discovery — PharmGKB foundational paper
- Whirl-Carrillo et al. (2021) Clin Pharmacol Ther — PharmGKB data architecture update
ClinVar Clinical Variants Database
Overview
ClinVar is NCBI's public archive of interpretations of variants submitted by clinical laboratories, researchers, and expert panels. It contains 2M+ variants with clinical significance classifications (Pathogenic, Likely Pathogenic, VUS, Likely Benign, Benign) for over 6,000 conditions. Access is free and requires no authentication via NCBI E-utilities.
When to Use
- Checking whether a specific variant (rsID, HGVS, or genomic position) has a clinical significance classification
- Retrieving all pathogenic/likely-pathogenic variants in a gene of interest
- Identifying conflicting interpretations between submitting laboratories
- Pulling condition/phenotype associations for a variant (MIM, MeSH, HPO terms)
- Building variant filtering pipelines that prioritize clinically actionable variants
- For somatic cancer variants, also check
cosmic-database; for GWAS associations usegwas-database
Prerequisites
- Python packages:
requests,xml.etree.ElementTree(stdlib) - Data requirements: gene symbols, rsIDs, HGVS strings, or ClinVar Variation IDs
- Environment: internet connection; NCBI Entrez email required (set
emailparameter) - Rate limits: 3 requests/second unauthenticated; 10/second with API key (free at https://www.ncbi.nlm.nih.gov/account/)
pip install requests
# No additional packages required; xml.etree is part of Python stdlib
Quick Start
import requests
EMAIL = "your@email.com" # required by NCBI policy
def clinvar_search(query, retmax=10):
"""Search ClinVar and return a list of ClinVar Variation IDs."""
r = requests.get(
"https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi",
params={"db": "clinvar", "term": query, "retmax": retmax,
"retmode": "json", "email": EMAIL}
)
r.raise_for_status()
return r.json()["esearchresult"]["idlist"]
# Find pathogenic BRCA1 variants
ids = clinvar_search("BRCA1[gene] AND pathogenic[clinsig]", retmax=5)
print(f"Found variation IDs: {ids}")
Core API
Query 1: Search Variants by Gene and Clinical Significance
Use ESearch to find ClinVar Variation IDs matching a structured query.
import requests
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def esearch(query, retmax=200):
r = requests.get(f"{BASE}/esearch.fcgi",
params={"db": "clinvar", "term": query,
"retmax": retmax, "retmode": "json", "email": EMAIL})
r.raise_for_status()
result = r.json()["esearchresult"]
return result["idlist"], int(result["count"])
# Gene-specific pathogenic variants
ids, total = esearch("BRCA2[gene] AND (pathogenic[clinsig] OR likely pathogenic[clinsig])")
print(f"Pathogenic/LP BRCA2 variants: {total} total, retrieved {len(ids)}")
print(f"First 5 IDs: {ids[:5]}")
# By rsID
ids, _ = esearch("rs80357906[rs]")
print(f"Variant IDs for rs80357906: {ids}")
# By condition name
ids, total = esearch("breast cancer[dis] AND pathogenic[clinsig]")
print(f"Pathogenic variants for breast cancer: {total}")
Query 2: Fetch Variant Summary Records
Retrieve structured summary data (JSON) for a list of Variation IDs.
import requests, json
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def esummary(ids):
"""Fetch ESummary records for a list of ClinVar variation IDs."""
r = requests.post(f"{BASE}/esummary.fcgi",
data={"db": "clinvar", "id": ",".join(ids),
"retmode": "json", "email": EMAIL})
r.raise_for_status()
return r.json()["result"]
ids, _ = esearch_func = lambda q: requests.get(
f"{BASE}/esearch.fcgi",
params={"db": "clinvar", "term": q, "retmax": 5, "retmode": "json", "email": EMAIL}
).json()["esearchresult"]["idlist"]
# Manual example with known IDs
sample_ids = ["12375", "17684", "54270"]
result = esummary(sample_ids)
for vid in result.get("uids", []):
rec = result[vid]
# ClinVar 2024 schema: clinical_significance was replaced by germline_classification
# (also: clinical_impact_classification, oncogenicity_classification — same shape, often empty)
gc = rec.get("germline_classification", {})
print(f"\nVariation {vid}: {rec.get('title')}")
print(f" ClinSig : {gc.get('description')}")
print(f" Review : {gc.get('review_status')}")
print(f" Gene : {rec.get('genes', [{}])[0].get('symbol')}")
Query 3: Fetch Full XML Records
Retrieve the complete variant record in XML for detailed submitter and condition data.
import requests
import xml.etree.ElementTree as ET
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def efetch_xml(variation_ids):
# ClinVar 2024 XML overhaul: "clinvarset" rettype returns an empty stub.
# Use rettype="vcv" + is_variationid="true" to get the new <VariationArchive> records.
r = requests.post(f"{BASE}/efetch.fcgi",
data={"db": "clinvar", "id": ",".join(variation_ids),
"rettype": "vcv", "is_variationid": "true",
"retmode": "xml", "email": EMAIL})
r.raise_for_status()
return ET.fromstring(r.text)
root = efetch_xml(["17677"]) # BRCA1 c.5266dupC (rs80357906)
# Aggregate (germline) classification — one per VariationArchive
for va in root.iter("VariationArchive"):
name = va.get("VariationName")
gc = va.find("./ClassifiedRecord/Classifications/GermlineClassification")
desc = gc.find("Description") if gc is not None else None
rstat = gc.find("ReviewStatus") if gc is not None else None
print(f"{name}: {desc.text if desc is not None else 'n/a'} "
f"({rstat.text if rstat is not None else 'n/a'})")
# Per-submitter assertions
for ca in va.iter("ClinicalAssertion"):
acc = ca.find("ClinVarAccession")
cls = ca.find("Classification/GermlineClassification")
if acc is not None and cls is not None:
print(f" {acc.get('SubmitterName', '?')}: {cls.text}")
Query 4: ClinVar FTP Bulk Data
For large-scale queries, download and parse the full variant summary file.
import urllib.request
import gzip, csv, io
# Full summary (tab-separated, ~300 MB compressed)
URL = "https://ftp.ncbi.nlm.nih.gov/pub/clinvar/tab_delimited/variant_summary.txt.gz"
# Stream and parse without full download
with urllib.request.urlopen(URL) as resp:
with gzip.open(resp, "rt", encoding="utf-8") as f:
reader = csv.DictReader(f, delimiter="\t")
pathogenic_brca1 = []
for row in reader:
if row["GeneSymbol"] == "BRCA1" and "Pathogenic" in row["ClinicalSignificance"]:
pathogenic_brca1.append({
"name": row["Name"],
"clinsig": row["ClinicalSignificance"],
"condition": row["PhenotypeList"],
"rsid": row["RS# (dbSNP)"],
})
print(f"Pathogenic BRCA1 variants: {len(pathogenic_brca1)}")
for v in pathogenic_brca1[:3]:
print(f" {v['name']} | {v['clinsig']} | rs{v['rsid']}")
Query 5: Review Status and Conflicting Interpretations
Filter variants by review status (evidence quality) and find conflicts.
import requests
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
# Stars correspond to review levels:
# 0 = no assertion criteria, 1 = criteria provided (single),
# 2 = criteria provided (multiple), 3 = expert panel, 4 = practice guideline
def search_by_review_stars(gene, min_stars=2):
"""Search for variants with at least min_stars review status."""
star_terms = {1: "criteria provided, single submitter",
2: "criteria provided, multiple submitters, no conflicts",
3: "reviewed by expert panel",
4: "practice guideline"}
terms = [f'"{star_terms[s]}"[review status]' for s in range(min_stars, 5) if s in star_terms]
query = f"{gene}[gene] AND (" + " OR ".join(terms) + ")"
r = requests.get(f"{BASE}/esearch.fcgi",
params={"db": "clinvar", "term": query, "retmax": 100,
"retmode": "json", "email": EMAIL})
return r.json()["esearchresult"]
result = search_by_review_stars("BRCA1", min_stars=3)
print(f"Expert-reviewed BRCA1 variants: {result['count']}")
Query 6: Variant-to-Condition Mapping
Extract condition (phenotype) data from ClinVar records.
import requests, json
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def get_conditions(variation_ids):
"""Return condition data for a list of ClinVar variation IDs."""
r = requests.post(f"{BASE}/esummary.fcgi",
data={"db": "clinvar", "id": ",".join(variation_ids),
"retmode": "json", "email": EMAIL})
r.raise_for_status()
result = r.json()["result"]
conditions = {}
for vid in result.get("uids", []):
rec = result[vid]
# trait_set moved under germline_classification in the 2024 ClinVar JSON
trait_set = rec.get("germline_classification", {}).get("trait_set", [])
conditions[vid] = [t.get("trait_name") for t in trait_set]
return conditions
sample_ids = ["12375", "17684", "54270"]
cond_map = get_conditions(sample_ids)
for vid, conds in cond_map.items():
print(f"Variation {vid}: {', '.join(conds)}")
Key Concepts
ClinVar Variation ID vs. rsID
ClinVar assigns its own stable Variation ID (integer) to each interpreted variant record. This differs from dbSNP rsIDs. A single rsID can correspond to multiple ClinVar Variation IDs if different alleles or interpretations are submitted separately.
Review Stars and Evidence Quality
ClinVar's "review status" encodes the level of evidence:
- 0 stars: No assertion criteria provided
- 1 star: Criteria provided, single submitter
- 2 stars: Multiple submitters, no conflict
- 3 stars: Reviewed by expert panel (e.g., ENIGMA, ClinGen)
- 4 stars: Practice guideline
Common Workflows
Workflow 1: Gene Pathogenicity Report
Goal: Retrieve all high-confidence pathogenic variants in a gene and export to CSV.
import requests, json, time, pandas as pd
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def search_gene_pathogenic(gene, clinsig="pathogenic"):
query = f"{gene}[gene] AND {clinsig}[clinsig]"
r = requests.get(f"{BASE}/esearch.fcgi",
params={"db": "clinvar", "term": query, "retmax": 500,
"retmode": "json", "email": EMAIL})
return r.json()["esearchresult"]["idlist"]
def fetch_summaries(ids):
records = []
for i in range(0, len(ids), 100):
batch = ids[i:i+100]
r = requests.post(f"{BASE}/esummary.fcgi",
data={"db": "clinvar", "id": ",".join(batch),
"retmode": "json", "email": EMAIL})
result = r.json()["result"]
for vid in result.get("uids", []):
rec = result[vid]
# ClinVar 2024 schema: clinical_significance → germline_classification; trait_set nested inside it
gc = rec.get("germline_classification", {})
records.append({
"variation_id": vid,
"name": rec.get("title"),
"clinsig": gc.get("description"),
"review_status": gc.get("review_status"),
"gene": ",".join(g.get("symbol", "") for g in rec.get("genes", [])),
"conditions": "; ".join(t.get("trait_name", "") for t in gc.get("trait_set", [])),
})
time.sleep(0.15)
return records
gene = "BRCA1"
ids = search_gene_pathogenic(gene)
print(f"Found {len(ids)} pathogenic variants in {gene}")
records = fetch_summaries(ids)
df = pd.DataFrame(records)
df.to_csv(f"{gene}_pathogenic_variants.csv", index=False)
print(f"Saved {len(df)} records → {gene}_pathogenic_variants.csv")
print(df[["name", "clinsig", "review_status"]].head())
Workflow 2: Variant Classification Check
Goal: Check ClinVar status for a list of user-provided rsIDs or HGVS notations.
import requests, time, pandas as pd
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
variants = ["rs80357906", "rs80357220", "rs28897672"]
results = []
for rsid in variants:
r = requests.get(f"{BASE}/esearch.fcgi",
params={"db": "clinvar", "term": f"{rsid}[rs]",
"retmax": 5, "retmode": "json", "email": EMAIL})
ids = r.json()["esearchresult"]["idlist"]
if not ids:
results.append({"rsid": rsid, "variation_id": None, "clinsig": "Not in ClinVar"})
continue
r2 = requests.post(f"{BASE}/esummary.fcgi",
data={"db": "clinvar", "id": ",".join(ids[:1]),
"retmode": "json", "email": EMAIL})
rec = r2.json()["result"][ids[0]]
gc = rec.get("germline_classification", {}) # 2024 ClinVar JSON
results.append({
"rsid": rsid,
"variation_id": ids[0],
"clinsig": gc.get("description", "Unknown"),
"review_status": gc.get("review_status"),
})
time.sleep(0.15)
df = pd.DataFrame(results)
print(df.to_string(index=False))
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
retmax | ESearch | 20 | 1–10000 | Max records returned per query |
retmode | ESearch/ESummary | "xml" | "json", "xml" | Response format |
rettype | EFetch | "vcv" | "vcv" | Record type for XML fetch (legacy clinvarset returns empty stub since 2024) |
is_variationid | EFetch | "false" | "true"/"false" | Set to "true" when fetching by ClinVar Variation ID with rettype=vcv |
clinsig query field | ESearch | — | "pathogenic", "likely pathogenic", "VUS" | Filter by clinical significance |
review status query field | ESearch | — | 0–4 star terms | Filter by evidence quality |
email | All | required | valid email | NCBI policy; prevents blocking |
Best Practices
-
Always set
email: NCBI requires an email in all E-utility calls for rate-limit attribution and policy compliance. -
Use FTP bulk download for large queries: For more than ~1000 variants, download
variant_summary.txt.gzfrom the ClinVar FTP rather than looping over EFetch — it's faster and avoids rate limits. -
Filter by review status: Automated pipelines should filter to ≥2-star variants to reduce noise from single-submitter assertions without peer review.
-
Use API key for production: Register at https://www.ncbi.nlm.nih.gov/account/ to get a free API key (
api_keyparameter) and triple your rate limit (3 → 10 req/s). -
Handle VUS separately: "Conflicting interpretations of pathogenicity" is its own ClinSig category — don't combine it with "VUS" in filters; they have different implications for clinical decision-making.
Common Recipes
Recipe: Check if rsID Is in ClinVar
When to use: Quick lookup for a single known variant.
import requests
EMAIL = "your@email.com"
rsid = "rs80357906"
r = requests.get(
"https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi",
params={"db": "clinvar", "term": f"{rsid}[rs]",
"retmax": 1, "retmode": "json", "email": EMAIL}
)
count = int(r.json()["esearchresult"]["count"])
print(f"{rsid}: {'found' if count else 'NOT'} in ClinVar ({count} records)")
Recipe: Download Variant Summary TSV
When to use: Bulk analysis — load entire ClinVar into a pandas DataFrame.
import pandas as pd
url = "https://ftp.ncbi.nlm.nih.gov/pub/clinvar/tab_delimited/variant_summary.txt.gz"
# Only human GRCh38 pathogenic variants
df = pd.read_csv(url, sep="\t", compression="gzip",
usecols=["#AlleleID", "Name", "GeneSymbol", "ClinicalSignificance",
"ReviewStatus", "PhenotypeList", "Assembly", "RS# (dbSNP)"])
df = df[(df["Assembly"] == "GRCh38") & (df["ClinicalSignificance"].str.contains("Pathogenic", na=False))]
print(f"Pathogenic variants (GRCh38): {len(df)}")
df.to_csv("clinvar_pathogenic_grch38.csv", index=False)
Recipe: Search by OMIM Disease ID
When to use: Find all ClinVar variants associated with a specific OMIM condition.
import requests
EMAIL = "your@email.com"
omim_id = "604370" # BRCA1-associated breast-ovarian cancer
r = requests.get(
"https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi",
params={"db": "clinvar", "term": f"{omim_id}[MIM]",
"retmax": 20, "retmode": "json", "email": EMAIL}
)
result = r.json()["esearchresult"]
print(f"Variants for OMIM {omim_id}: {result['count']} total")
print(f"First IDs: {result['idlist'][:5]}")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
HTTP 429 or no response | Rate limit exceeded | Add time.sleep(0.35) between requests; use API key |
Empty idlist for rsID query | rsID not indexed in ClinVar | Try HGVS notation or gene+position query instead |
Missing clinsig in summary | Variant has no interpretation | Check review_status; "no interpretation for the single variant" means no ClinSig yet |
| XML parse error in EFetch | Incomplete response (timeout) | Set requests.get(..., timeout=30) and retry once |
<ClinVarResult-Set><set/></ClinVarResult-Set> empty stub | Using legacy rettype="clinvarset" (deprecated in 2024) | Switch to rettype="vcv" + is_variationid="true"; parse <VariationArchive> root |
KeyError: clinical_significance in ESummary parsing | Field renamed in 2024 ClinVar JSON | Use rec["germline_classification"] (also clinical_impact_classification, oncogenicity_classification); trait_set now nested inside germline_classification |
| Conflicting results for same rsID | Multiple submissions with different interpretations | Group by review_status and prefer higher-star entries |
| FTP download fails | Large file / slow connection | Use pandas.read_csv with chunksize=100000 or pre-filter with grep |
Related Skills
gwas-database— GWAS Catalog for population-level SNP-trait associations (complement to ClinVar's clinical assertions)ensembl-database— Ensembl VEP for predicting variant consequences without requiring prior clinical curationcosmic-database— Somatic cancer variant database (complementary to ClinVar's germline focus)pubmed-database— Retrieve supporting publications cited in ClinVar submissions
References
- ClinVar official site — Browse and download ClinVar data
- NCBI E-utilities documentation — Full E-utilities API reference
- ClinVar FTP downloads — Bulk data files (variant_summary.txt.gz, etc.)
- ClinVar data model — Understanding review status stars and ClinSig categories
COSMIC Somatic Cancer Mutations Database
Overview
COSMIC (Catalogue Of Somatic Mutations In Cancer) is the world's largest expert-curated database of somatic mutations in cancer, covering 6.7M+ coding mutations, 40,000+ cancer samples, 19,000+ genes across all cancer types. It includes the Cancer Gene Census (critical cancer genes), mutational signatures (SBS, DBS, ID), drug resistance variants, copy number data, gene expression, and methylation. The REST API v3.1 enables programmatic queries; most features are freely accessible after registration.
When to Use
- Checking whether a specific somatic variant in a cancer gene is annotated in COSMIC (frequency, cancer type distribution)
- Retrieving all somatic mutations in a gene of interest across COSMIC cancer samples
- Accessing COSMIC Cancer Gene Census classifications (Tier 1/2, role: oncogene/TSG/fusion)
- Looking up mutational signature attributions for samples or cancer types
- Identifying drug resistance variants (pharmacogenomic data) from COSMIC drug resistance database
- Building cancer driver gene lists for bioinformatic pipelines
- For germline/inherited variants use
clinvar-database; for drug-target associations useopentargets-database
Prerequisites
- Python packages:
requests,pandas - Data requirements: gene symbols (HGNC), COSMIC mutation IDs (COSM), sample IDs, or genomic coordinates
- Environment: internet connection; free account registration at https://cancer.sanger.ac.uk/cosmic/register
- Rate limits: authenticated requests only; 10 requests/second max; API key required
pip install requests pandas
# Register at https://cancer.sanger.ac.uk/cosmic/register to obtain API credentials
Quick Start
import requests
import base64
# COSMIC API requires base64-encoded email:password authentication
EMAIL = "your_registered@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
# Get mutations for KRAS gene
r = requests.get(f"{BASE}/mutations",
headers=HEADERS,
params={"gene_name": "KRAS", "limit": 5})
r.raise_for_status()
data = r.json()
print(f"Total KRAS mutations: {data['meta']['total']}")
for m in data["data"][:3]:
print(f" {m['mutation_id']:15s} AA: {m.get('mutation_aa')} | Cancer: {m.get('primary_site')}")
Core API
Query 1: Gene Mutations Search
Retrieve all COSMIC somatic mutations for a gene, with cancer type and amino acid change.
import requests, base64, pandas as pd
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
def get_gene_mutations(gene, limit=100, cancer_site=None):
params = {"gene_name": gene, "limit": limit}
if cancer_site:
params["primary_site"] = cancer_site
r = requests.get(f"{BASE}/mutations", headers=HEADERS, params=params)
r.raise_for_status()
return r.json()
data = get_gene_mutations("TP53", limit=20)
print(f"Total TP53 mutations in COSMIC: {data['meta']['total']}")
rows = []
for m in data["data"][:10]:
rows.append({
"mutation_id": m.get("mutation_id"),
"mutation_aa": m.get("mutation_aa"),
"mutation_cds": m.get("mutation_cds"),
"primary_site": m.get("primary_site"),
"histology": m.get("primary_histology"),
"count": m.get("count"),
})
df = pd.DataFrame(rows)
print(df.head())
# Filter by cancer site
data_lung = get_gene_mutations("TP53", cancer_site="lung", limit=20)
print(f"\nTP53 mutations in lung cancer: {data_lung['meta']['total']}")
Query 2: Cancer Gene Census
Retrieve the COSMIC Cancer Gene Census — classified cancer driver genes.
import requests, base64, pandas as pd
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
r = requests.get(f"{BASE}/genes", headers=HEADERS, params={"limit": 100})
r.raise_for_status()
data = r.json()
print(f"Total genes in COSMIC: {data['meta']['total']}")
# Get Cancer Gene Census genes
r_cgc = requests.get(f"{BASE}/genes",
headers=HEADERS,
params={"cgc_tier": "1", "limit": 50})
cgc_data = r_cgc.json()
print(f"\nCGC Tier 1 genes: {cgc_data['meta']['total']}")
rows = []
for g in cgc_data["data"][:15]:
rows.append({
"gene": g.get("gene_name"),
"tier": g.get("cgc_tier"),
"role": g.get("role_in_cancer"),
"mutation_types": g.get("mutation_types"),
"tumour_types": str(g.get("tumour_types_somatic", []))[:80],
})
df = pd.DataFrame(rows)
print(df.to_string(index=False))
Query 3: Specific Mutation Lookup
Retrieve details for a known COSMIC mutation ID (COSM…).
import requests, base64
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
# KRAS G12D mutation
mutation_id = "COSM521"
r = requests.get(f"{BASE}/mutations/{mutation_id}", headers=HEADERS)
r.raise_for_status()
m = r.json()
print(f"Mutation ID : {m.get('mutation_id')}")
print(f"Gene : {m.get('gene_name')}")
print(f"AA change : {m.get('mutation_aa')}")
print(f"CDS change : {m.get('mutation_cds')}")
print(f"Substitution: {m.get('mutation_description')}")
print(f"Count : {m.get('count')} samples")
print(f"Cancer types: {str(m.get('cancer_types', []))[:100]}")
Query 4: Sample-Level Mutation Data
Retrieve all somatic mutations for a specific cancer sample.
import requests, base64, pandas as pd
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
# Search for a specific sample
r = requests.get(f"{BASE}/samples",
headers=HEADERS,
params={"primary_site": "breast", "limit": 5})
r.raise_for_status()
samples = r.json()["data"]
print(f"Example breast cancer samples:")
for s in samples[:3]:
print(f" {s.get('sample_id')}: {s.get('sample_name')} | {s.get('primary_histology')}")
# Get mutations for a specific sample
if samples:
sample_id = samples[0]["sample_id"]
r2 = requests.get(f"{BASE}/samples/{sample_id}/mutations", headers=HEADERS)
if r2.ok:
muts = r2.json()["data"]
print(f"\nMutations in sample {sample_id}: {len(muts)}")
for m in muts[:5]:
print(f" {m.get('gene_name'):10s} {m.get('mutation_aa')}")
Query 5: Mutational Signatures
Retrieve COSMIC mutational signature data for cancer types.
import requests, base64, pandas as pd
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
# List available mutational signatures
r = requests.get(f"{BASE}/signatures", headers=HEADERS)
r.raise_for_status()
sigs = r.json()["data"]
print(f"COSMIC mutational signatures: {len(sigs)}")
for s in sigs[:5]:
print(f" {s.get('signature_name')}: {s.get('aetiology', '')[:80]}")
# Get signature attributions by cancer type
r2 = requests.get(f"{BASE}/signatures/attributions",
headers=HEADERS,
params={"cancer_type": "Breast", "limit": 10})
if r2.ok:
attributions = r2.json()["data"]
for a in attributions[:5]:
print(f" {a.get('signature_name')}: {a.get('attribution_proportion'):.2%} in breast cancer")
Query 6: Drug Resistance Variants
Query the COSMIC drug resistance database for variants conferring drug resistance.
import requests, base64, pandas as pd
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
# Get drug resistance variants
r = requests.get(f"{BASE}/resistance_mutations",
headers=HEADERS,
params={"gene": "EGFR", "limit": 20})
if r.ok:
data = r.json()
print(f"EGFR drug resistance variants: {data['meta'].get('total', 'n/a')}")
for v in data.get("data", [])[:5]:
print(f" {v.get('mutation_aa'):20s} Drug: {v.get('drug')} | Resistance: {v.get('resistance_type')}")
else:
print(f"Drug resistance API: {r.status_code} - endpoint may require specific access level")
Key Concepts
Cancer Gene Census Tiers
COSMIC's Cancer Gene Census classifies genes into:
- Tier 1: Well-established cancer drivers with documented mutations and molecular mechanisms in cancer
- Tier 2: Genes with strong evidence for roles in cancer but less functional characterization
Mutation ID Stability
COSMIC mutation IDs (COSM…) are stable identifiers for specific amino acid changes in a gene. The same COSM ID appears across all samples with that mutation, allowing cross-study comparison.
Common Workflows
Workflow 1: Gene Hotspot Mutation Analysis
Goal: Identify the most frequently occurring somatic mutations in a cancer gene.
import requests, base64, pandas as pd
from collections import Counter
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
def get_all_gene_mutations(gene, max_records=1000):
"""Paginate through all COSMIC mutations for a gene."""
all_muts = []
skip = 0
limit = 200
while len(all_muts) < max_records:
r = requests.get(f"{BASE}/mutations",
headers=HEADERS,
params={"gene_name": gene, "limit": limit, "skip": skip})
r.raise_for_status()
batch = r.json()["data"]
if not batch:
break
all_muts.extend(batch)
total = r.json()["meta"]["total"]
skip += limit
if skip >= total:
break
return all_muts
# Get hotspots for KRAS
mutations = get_all_gene_mutations("KRAS", max_records=500)
print(f"Retrieved {len(mutations)} KRAS somatic mutations")
# Rank by amino acid change frequency
aa_counter = Counter(m["mutation_aa"] for m in mutations if m.get("mutation_aa"))
hotspots = pd.DataFrame(aa_counter.most_common(15), columns=["mutation_aa", "sample_count"])
print("\nKRAS hotspot mutations:")
print(hotspots.head(10).to_string(index=False))
hotspots.to_csv("KRAS_hotspots.csv", index=False)
Workflow 2: Cancer Gene Census Export
Goal: Export the full Cancer Gene Census as a structured table for downstream pipeline use.
import requests, base64, pandas as pd, time
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
BASE = "https://cancer.sanger.ac.uk/cosmic/api"
HEADERS = {"Authorization": f"Basic {token}"}
all_genes = []
for tier in [1, 2]:
skip = 0
while True:
r = requests.get(f"{BASE}/genes",
headers=HEADERS,
params={"cgc_tier": str(tier), "limit": 100, "skip": skip})
r.raise_for_status()
batch = r.json()["data"]
if not batch:
break
all_genes.extend(batch)
if len(batch) < 100:
break
skip += 100
time.sleep(0.1)
rows = [{
"gene": g.get("gene_name"),
"tier": g.get("cgc_tier"),
"role_in_cancer": g.get("role_in_cancer"),
"mutation_types": g.get("mutation_types"),
"somatic_tumours": str(g.get("tumour_types_somatic", [])),
"germline_tumours": str(g.get("tumour_types_germline", [])),
"chr": g.get("chromosomal_location"),
} for g in all_genes]
df = pd.DataFrame(rows)
df.to_csv("COSMIC_cancer_gene_census.csv", index=False)
print(f"Exported {len(df)} Cancer Gene Census genes → COSMIC_cancer_gene_census.csv")
print(df.groupby("tier")["gene"].count())
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
gene_name | Mutations | — | HGNC symbol | Filter mutations by gene |
primary_site | Mutations/Samples | — | tissue type string | Filter by primary tumor site |
limit | All | 10 | 1–200 | Records per page |
skip | All | 0 | integer | Pagination offset |
cgc_tier | Genes | — | "1", "2" | Cancer Gene Census tier |
mutation_id | Mutations | — | COSM ID string | Lookup specific mutation |
Best Practices
-
Authenticate via Base64: COSMIC uses HTTP Basic Auth with base64-encoded
email:password. Store credentials in environment variables, not in code. -
Paginate large gene queries: Popular cancer genes (TP53, KRAS) have 100,000+ mutation records; use
skip/limitpagination and cache results locally. -
Use COSM IDs for cross-study comparison: Amino acid change strings may have formatting variations (p.G12D vs G12D); use COSMIC mutation IDs (COSM…) for unambiguous references.
-
Check data license for commercial use: COSMIC data is free for academic use but requires a commercial license for industry applications. Verify at https://cancer.sanger.ac.uk/cosmic/license.
-
Complement with clinical data: COSMIC captures somatic mutations from cancer sequencing; complement with
clinvar-databasefor germline pathogenicity andopentargets-databasefor therapeutic significance.
Common Recipes
Recipe: Top Mutated Genes in a Cancer Type
When to use: Identify frequently mutated genes in a specific cancer type.
import requests, base64, pandas as pd
from collections import Counter
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
HEADERS = {"Authorization": f"Basic {token}"}
r = requests.get("https://cancer.sanger.ac.uk/cosmic/api/mutations",
headers=HEADERS,
params={"primary_site": "lung", "limit": 200})
data = r.json()["data"]
gene_counts = Counter(m.get("gene_name") for m in data if m.get("gene_name"))
df = pd.DataFrame(gene_counts.most_common(10), columns=["gene", "mutations"])
print(df.to_string(index=False))
Recipe: Check if a Variant Is in COSMIC
When to use: Look up whether a specific amino acid change is recorded in COSMIC.
import requests, base64
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
HEADERS = {"Authorization": f"Basic {token}"}
gene = "KRAS"
aa_change = "p.G12D"
r = requests.get("https://cancer.sanger.ac.uk/cosmic/api/mutations",
headers=HEADERS,
params={"gene_name": gene, "limit": 200})
all_muts = r.json()["data"]
matches = [m for m in all_muts if aa_change in (m.get("mutation_aa") or "")]
print(f"{gene} {aa_change}: {'FOUND' if matches else 'NOT FOUND'} in COSMIC ({len(matches)} records)")
if matches:
print(f" Sample count: {sum(m.get('count', 0) for m in matches)}")
Recipe: Download CGC Tier 1 Gene List
When to use: Get a simple list of Tier 1 cancer driver genes for filtering.
import requests, base64
EMAIL = "your@email.com"
PASSWORD = "your_password"
token = base64.b64encode(f"{EMAIL}:{PASSWORD}".encode()).decode()
HEADERS = {"Authorization": f"Basic {token}"}
r = requests.get("https://cancer.sanger.ac.uk/cosmic/api/genes",
headers=HEADERS,
params={"cgc_tier": "1", "limit": 200})
genes = [g["gene_name"] for g in r.json()["data"]]
print(f"CGC Tier 1 genes ({len(genes)}): {', '.join(genes[:10])}...")
with open("cosmic_tier1_genes.txt", "w") as f:
f.write("\n".join(genes))
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
HTTP 401 Unauthorized | Missing or incorrect API credentials | Check base64 encoding: base64.b64encode(f"{email}:{password}".encode()) |
HTTP 403 Forbidden | Access requires different tier | Some endpoints need commercial license; check COSMIC license page |
Empty data array | No records match filter | Broaden query; check spelling of gene symbol or site name |
| Very slow for large genes | TP53/KRAS have 100K+ records | Paginate with small limit=200; cache results to local CSV |
| Rate limit errors | >10 req/s | Add time.sleep(0.15) between requests |
| Different AA notation format | Various mutation string formats | Normalize with RDKit or use COSM IDs for exact matching |
Related Skills
clinvar-database— Germline pathogenicity classifications complementing COSMIC's somatic focusopentargets-database— Drug-target associations for COSMIC cancer driver genesensembl-database— Variant consequence predictions (VEP) for COSMIC variantsgwas-database— Population-level SNP associations for cancer risk (vs. COSMIC's somatic mutations)
References
- COSMIC website — Official COSMIC database and downloads
- COSMIC REST API v3 documentation — API endpoint reference
- Cancer Gene Census — Curated cancer driver gene catalog
- COSMIC mutational signatures (Alexandrov et al. 2020) — Reference paper for COSMIC v3 signatures
dbSNP Database
Overview
NCBI dbSNP is the primary public repository for short human genetic variants, cataloguing over 1 billion SNPs, indels, and MNVs with allele frequencies, functional annotations, and cross-references to ClinVar, gnomAD, and 1000 Genomes. Variants are identified by stable rsIDs (reference SNP cluster IDs). Access is free via two APIs: the legacy NCBI E-utilities and the newer NCBI Variation Services REST API, which returns structured JSON.
When to Use
- Looking up allele frequencies and variant class for a known rsID
- Searching all dbSNP variants in a gene or chromosomal region by name or coordinates
- Resolving rsIDs to genomic coordinates (GRCh38/GRCh37) and HGVS notation
- Checking whether a variant of interest has clinical significance links to ClinVar entries
- Batch-fetching hundreds of rsIDs efficiently using epost+efetch history server
- Cross-referencing a list of variant positions to dbSNP rsIDs for downstream annotation
- For clinical pathogenicity classifications use
clinvar-database; dbSNP provides IDs and frequency but not curated clinical significance - For population frequency stratified by ancestry use
gnomad-database; dbSNP MAF is a single aggregate frequency
Prerequisites
- Python packages:
requests,pandas,matplotlib,xml.etree.ElementTree(stdlib) - Data requirements: rsIDs (
rs80357906), gene symbols, or chromosomal coordinates - Environment: internet connection; NCBI Entrez email required for E-utilities (set
emailparameter) - Rate limits: 3 requests/second without API key; 10 requests/second with free NCBI API key. Register at https://www.ncbi.nlm.nih.gov/account/ — add
&api_key=YOUR_KEYto all requests
pip install requests pandas matplotlib
# xml.etree.ElementTree is part of Python stdlib — no additional install needed
Quick Start
import requests
import json
EMAIL = "your@email.com" # required by NCBI policy
BASE_EUTILS = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
BASE_VARIATION = "https://api.ncbi.nlm.nih.gov/variation/v0"
def fetch_snp_by_rsid(rsid: str) -> dict:
"""Fetch a dbSNP record by rsID using the NCBI Variation Services API (structured JSON)."""
rs_num = str(rsid).lstrip("rs")
r = requests.get(f"{BASE_VARIATION}/refsnp/{rs_num}", timeout=15)
r.raise_for_status()
return r.json()
record = fetch_snp_by_rsid("rs1800497") # DRD2 Taq1A
print(f"rsID: rs{record['refsnp_id']}")
print(f"Variant type: {record['primary_snapshot_data'].get('variant_type')}")
# Top-level keys: citations, create_date, dbsnp1_merges, last_update_build_id,
# last_update_date, lost_obs_movements, mane_select_ids, present_obs_movements,
# primary_snapshot_data, refsnp_id. (No top-level `organism` field.)
# rsID: rs1800497
# Variant type: snv
Core API
Query 1: rsID Lookup via E-utilities
Fetch the full SNP record for a single rsID using efetch with db=snp. Returns an XML document with alleles, placements, and frequency data.
import requests
import xml.etree.ElementTree as ET
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def efetch_snp_xml(rsid: str) -> ET.Element:
"""Fetch dbSNP XML record for a single rsID via the docsum rettype.
Note: rettype="xml" returns a namespaced ExchangeSet; rettype="docsum"
returns the simpler eSummaryResult/DocumentSummary tree without namespaces."""
rs_num = str(rsid).lstrip("rs")
r = requests.get(f"{BASE}/efetch.fcgi",
params={"db": "snp", "id": rs_num,
"rettype": "docsum", "retmode": "xml",
"email": EMAIL},
timeout=20)
r.raise_for_status()
return ET.fromstring(r.text)
root = efetch_snp_xml("rs80357906")
# Parse the DocumentSummary record (MAF/MAFALLELE were removed in 2024;
# GLOBAL_MAFS is a sub-tree — use ESummary JSON below for easier access)
for docsum in root.iter("DocumentSummary"):
rs_id = docsum.get("uid")
snp_class = docsum.findtext("SNP_CLASS", "Unknown")
chr_pos = docsum.findtext("CHRPOS", "N/A")
clin_sig = docsum.findtext("CLINICAL_SIGNIFICANCE", "N/A")
print(f"rs{rs_id} | Class: {snp_class} | Position: {chr_pos}")
print(f" ClinSig: {clin_sig}")
# rs80357906 | Class: delins | Position: 17:43057062
# ClinSig: pathogenic,risk-factor,uncertain-significance
# Fetch using ESummary for structured JSON (preferred for batch)
def esummary_snp(rsid: str) -> dict:
rs_num = str(rsid).lstrip("rs")
r = requests.get(f"{BASE}/esummary.fcgi",
params={"db": "snp", "id": rs_num,
"retmode": "json", "email": EMAIL},
timeout=15)
r.raise_for_status()
result = r.json()["result"]
return result.get(rs_num, {})
rec = esummary_snp("rs80357906")
print(f"rs{rec.get('snp_id')}:")
print(f" Class : {rec.get('snp_class')}") # e.g., 'delins'
# `maf`/`mafallele` were removed from ESummary in 2024 — use `global_mafs`
# (list of {study, freq}) and pick a study (e.g., 'GnomAD_genomes') or the
# global aggregate ('TOPMED'/'1000Genomes').
for m in rec.get('global_mafs', [])[:4]:
print(f" MAF[{m['study']:18s}]: {m['freq']}")
print(f" ChrPos : {rec.get('chrpos')}") # 17:43057062
print(f" ClinSig : {rec.get('clinical_significance')}")
print(f" FxnClass : {rec.get('fxn_class')}")
Query 2: Gene Variant Search
Search dbSNP for all variants in a gene using esearch. Returns a list of rsIDs matching the gene.
import requests
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def esearch_snp(query: str, retmax: int = 100) -> tuple[list, int]:
"""Search dbSNP using a query string. Returns (id_list, total_count)."""
r = requests.get(f"{BASE}/esearch.fcgi",
params={"db": "snp", "term": query,
"retmax": retmax, "retmode": "json",
"email": EMAIL},
timeout=15)
r.raise_for_status()
result = r.json()["esearchresult"]
return result["idlist"], int(result["count"])
# All variants in BRCA1
ids, total = esearch_snp("BRCA1[gene] AND human[orgn]", retmax=20)
print(f"BRCA1 variants in dbSNP: {total:,} total")
print(f"First 5 rsIDs: {['rs' + i for i in ids[:5]]}")
# Only clinical variants (linked to ClinVar)
ids_clin, total_clin = esearch_snp(
"BRCA1[gene] AND human[orgn] AND clinsig[filter]", retmax=50)
print(f"BRCA1 variants with clinical significance: {total_clin:,}")
Query 3: Chromosomal Region Search
Search for all variants in a genomic region using chromosome coordinates.
import requests
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def search_region(chrom: str, start: int, stop: int,
assembly: str = "GRCh38", retmax: int = 200) -> tuple[list, int]:
"""Find all dbSNP variants in a chromosomal region."""
query = f"{chrom}[CHR] AND {start}:{stop}[CHRPOS37]" if assembly == "GRCh37" else \
f"{chrom}[CHR] AND {start}:{stop}[CHRPOS]"
r = requests.get(f"{BASE}/esearch.fcgi",
params={"db": "snp", "term": query,
"retmax": retmax, "retmode": "json",
"email": EMAIL},
timeout=20)
r.raise_for_status()
result = r.json()["esearchresult"]
return result["idlist"], int(result["count"])
# PCSK9 exon 4 region (GRCh38)
ids, total = search_region("1", 55039700, 55040200)
print(f"Variants in chr1:55039700-55040200: {total:,} total")
print(f"Retrieved {len(ids)} rsIDs: {['rs' + i for i in ids[:5]]}")
Query 4: Variant Summary — MAF, Alleles, Clinical Significance
Retrieve structured summary data for variant records using ESummary, extracting MAF, alleles, and database cross-links.
import requests, json
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def fetch_snp_summaries(rsids: list) -> dict:
"""Fetch ESummary records for a list of rsIDs. Returns dict keyed by rs number."""
ids_str = ",".join(str(r).lstrip("rs") for r in rsids)
r = requests.post(f"{BASE}/esummary.fcgi",
data={"db": "snp", "id": ids_str,
"retmode": "json", "email": EMAIL},
timeout=20)
r.raise_for_status()
return r.json()["result"]
rsids = ["rs80357906", "rs80357220", "rs28897672", "rs1801133"]
result = fetch_snp_summaries(rsids)
for rs_num in rsids:
uid = str(rs_num).lstrip("rs")
rec = result.get(uid, {})
# `global_mafs` is a list of {study, freq}; pick the first or filter by study
gmafs = rec.get("global_mafs", [])
maf_str = gmafs[0]["freq"] if gmafs else "N/A"
maf_study = gmafs[0]["study"] if gmafs else ""
print(f"\n{rs_num}:")
print(f" Class : {rec.get('snp_class', 'N/A')}")
print(f" MAF : {maf_str} (from {maf_study})")
print(f" Location : {rec.get('chrpos', 'N/A')}")
print(f" ClinSig : {rec.get('clinical_significance', 'N/A')}")
print(f" Function : {rec.get('fxn_class', 'N/A')}")
Query 5: Batch rsID Query with EPost+EFetch
Efficiently upload hundreds of rsIDs to the NCBI history server using EPost, then retrieve them in batches with EFetch.
import requests, time, pandas as pd
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def epost_ids(id_list: list) -> tuple[str, str]:
"""Upload rsIDs to NCBI history server. Returns (WebEnv, query_key)."""
ids_str = ",".join(str(i).lstrip("rs") for i in id_list)
r = requests.post(f"{BASE}/epost.fcgi",
data={"db": "snp", "id": ids_str, "email": EMAIL},
timeout=30)
r.raise_for_status()
import xml.etree.ElementTree as ET
root = ET.fromstring(r.text)
webenv = root.findtext("WebEnv")
query_key = root.findtext("QueryKey")
return webenv, query_key
def efetch_history(webenv: str, query_key: str,
retstart: int = 0, retmax: int = 100) -> dict:
"""Retrieve records from NCBI history server using ESummary."""
r = requests.get(f"{BASE}/esummary.fcgi",
params={"db": "snp", "WebEnv": webenv,
"query_key": query_key, "retstart": retstart,
"retmax": retmax, "retmode": "json",
"email": EMAIL},
timeout=30)
r.raise_for_status()
return r.json()["result"]
rsid_batch = ["rs80357906", "rs80357220", "rs28897672", "rs1801133",
"rs429358", "rs7412", "rs1800497"]
webenv, query_key = epost_ids(rsid_batch)
print(f"Posted {len(rsid_batch)} IDs | WebEnv: {webenv[:40]}...")
records = []
for start in range(0, len(rsid_batch), 100):
result = efetch_history(webenv, query_key, retstart=start, retmax=100)
for uid in result.get("uids", []):
rec = result[uid]
# global_mafs replaced maf/mafallele in 2024 — flatten the first entry
gmafs = rec.get("global_mafs", [])
records.append({
"rsid": f"rs{uid}",
"snp_class": rec.get("snp_class"),
"maf": gmafs[0]["freq"] if gmafs else None,
"maf_study": gmafs[0]["study"] if gmafs else None,
"chrpos": rec.get("chrpos"),
"clinical_sig": rec.get("clinical_significance"),
})
time.sleep(0.5)
df = pd.DataFrame(records)
print(f"\nRetrieved {len(df)} records:")
print(df.to_string(index=False))
Query 6: NCBI Variation Services API
The newer REST API returns structured JSON with detailed allele placements, frequencies, and variant type annotations. Preferred for programmatic rsID resolution.
import requests
import pandas as pd
BASE_VARIATION = "https://api.ncbi.nlm.nih.gov/variation/v0"
def fetch_refsnp(rsid: str) -> dict:
"""Fetch structured JSON from NCBI Variation Services API."""
rs_num = str(rsid).lstrip("rs")
r = requests.get(f"{BASE_VARIATION}/refsnp/{rs_num}", timeout=15)
r.raise_for_status()
return r.json()
def parse_allele_frequencies(record: dict) -> list:
"""Extract allele frequencies from a Variation Services record.
freq[i] keys: study_name, study_version, local_row_id, observation, allele_count, total_count.
The asserted allele identifier lives in freq.observation.inserted_sequence (NOT at ann.allele)."""
freqs = []
snapshot = record.get("primary_snapshot_data", {})
for ann in snapshot.get("allele_annotations", []):
for f in ann.get("frequency", []):
obs = f.get("observation", {})
freqs.append({
"allele": obs.get("inserted_sequence"),
"study": f.get("study_name"),
"allele_count": f.get("allele_count"),
"total_count": f.get("total_count"),
"freq": (f["allele_count"] / f["total_count"]
if f.get("total_count") else None),
})
return freqs
record = fetch_refsnp("rs1800497") # DRD2 Taq1A variant
print(f"rsID: rs{record['refsnp_id']}")
print(f"Variant type: {record['primary_snapshot_data'].get('variant_type')}") # 'snv'
placements = record["primary_snapshot_data"].get("placements_with_allele", [])
for placement in placements[:2]:
seq_id = placement.get("seq_id")
for allele in placement.get("alleles", []):
spdi = allele.get("allele", {}).get("spdi", {})
print(f" Placement: {seq_id} | SPDI: {spdi.get('inserted_sequence')}")
freqs = parse_allele_frequencies(record)
if freqs:
df_freq = pd.DataFrame(freqs[:5])
print(f"\nAllele frequencies ({len(freqs)} entries):")
print(df_freq.to_string(index=False))
Key Concepts
rsID vs. ss ID (Submitted SNP)
dbSNP uses two IDs: rs IDs (Reference SNP cluster IDs) are stable public identifiers assigned after clustering submitted variants. ss IDs (Submitted SNP IDs) are assigned to individual laboratory submissions before clustering. Use rs IDs for all queries — ss IDs are internal and submission-specific. A single rs ID may cluster multiple ss IDs from different submissions.
MAF vs. Clinical Significance
- MAF (Minor Allele Frequency): The aggregate frequency of the minor allele across all dbSNP submissions. This is a population-level statistic aggregated from 1000 Genomes, gnomAD, TOPMED, and other studies. It does not indicate whether the variant is pathogenic.
- Clinical significance: A link field pointing to ClinVar classifications (Pathogenic, VUS, Benign, etc.). A variant can have a very low MAF (rare) yet be classified as Benign, or a moderate MAF yet be Pathogenic in a specific context (e.g., founder variants). Use
clinvar-databasefor the full pathogenicity record.
Variant Classes in dbSNP
| Class | Description | Example |
|---|---|---|
snv | Single nucleotide variant (A>T) | rs80357906 |
indel | Insertion or deletion | rs786201005 |
mnv | Multi-nucleotide variant | rs1057519737 |
ins | Pure insertion | rs113993960 |
del | Pure deletion | rs66767301 |
microsatellite | STR (short tandem repeat) | rs5030655 |
Common Workflows
Workflow 1: Batch rsID Annotation from a Variant List
Goal: Given a list of rsIDs from a variant call pipeline, retrieve MAF, position, and clinical significance for all variants in one run.
import requests, time, pandas as pd
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def epost_snp(rsids):
ids = ",".join(str(r).lstrip("rs") for r in rsids)
r = requests.post(f"{BASE}/epost.fcgi",
data={"db": "snp", "id": ids, "email": EMAIL}, timeout=30)
r.raise_for_status()
import xml.etree.ElementTree as ET
root = ET.fromstring(r.text)
return root.findtext("WebEnv"), root.findtext("QueryKey")
def esummary_history(webenv, query_key, start, retmax=100):
r = requests.get(f"{BASE}/esummary.fcgi",
params={"db": "snp", "WebEnv": webenv,
"query_key": query_key, "retstart": start,
"retmax": retmax, "retmode": "json", "email": EMAIL},
timeout=30)
r.raise_for_status()
return r.json()["result"]
# Example: VCF post-processing — annotate a list of called variants
variant_rsids = [
"rs80357906", "rs80357220", "rs28897672", "rs1801133",
"rs429358", "rs7412", "rs1800497", "rs2230199",
]
print(f"Posting {len(variant_rsids)} rsIDs to NCBI history server...")
webenv, query_key = epost_snp(variant_rsids)
records = []
for start in range(0, len(variant_rsids), 100):
result = esummary_history(webenv, query_key, start=start, retmax=100)
for uid in result.get("uids", []):
rec = result[uid]
gmafs = rec.get("global_mafs", []) # 2024 schema (replaced maf/mafallele)
records.append({
"rsid": f"rs{uid}",
"snp_class": rec.get("snp_class"),
"maf": gmafs[0]["freq"] if gmafs else None,
"maf_study": gmafs[0]["study"] if gmafs else None,
"chrpos_grch38": rec.get("chrpos"),
"gene": rec.get("genes", [{}])[0].get("name") if rec.get("genes") else None,
"clinical_significance": rec.get("clinical_significance"),
"fxn_class": rec.get("fxn_class"),
})
time.sleep(0.5)
df = pd.DataFrame(records)
df.to_csv("variant_annotations.csv", index=False)
print(f"Annotated {len(df)} variants → variant_annotations.csv")
print(df[["rsid", "snp_class", "maf", "chrpos_grch38", "clinical_significance"]].to_string(index=False))
Workflow 2: Gene Variant Class Distribution Visualization
Goal: Search all dbSNP variants in a gene and plot their variant class distribution.
import requests, time
import pandas as pd
import matplotlib.pyplot as plt
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def search_gene_variants(gene: str, retmax: int = 500) -> list:
r = requests.get(f"{BASE}/esearch.fcgi",
params={"db": "snp", "term": f"{gene}[gene] AND human[orgn]",
"retmax": retmax, "retmode": "json", "email": EMAIL},
timeout=20)
r.raise_for_status()
return r.json()["esearchresult"]["idlist"]
def fetch_summaries_batch(ids: list, batch_size: int = 100) -> list:
records = []
for i in range(0, len(ids), batch_size):
batch = ids[i:i+batch_size]
ids_str = ",".join(batch)
r = requests.post(f"{BASE}/esummary.fcgi",
data={"db": "snp", "id": ids_str,
"retmode": "json", "email": EMAIL},
timeout=30)
r.raise_for_status()
result = r.json()["result"]
for uid in result.get("uids", []):
rec = result[uid]
records.append({"rsid": f"rs{uid}", "snp_class": rec.get("snp_class", "unknown")})
time.sleep(0.5)
return records
gene = "CFTR"
print(f"Searching dbSNP for {gene} variants...")
ids = search_gene_variants(gene, retmax=300)
print(f"Found {len(ids)} IDs; fetching summaries...")
records = fetch_summaries_batch(ids)
df = pd.DataFrame(records)
class_counts = df["snp_class"].value_counts()
# Plot variant class distribution
fig, ax = plt.subplots(figsize=(8, 5))
colors = ["#4472C4", "#ED7D31", "#A9D18E", "#FF0000", "#FFC000", "#7030A0"]
bars = ax.bar(class_counts.index, class_counts.values,
color=colors[:len(class_counts)], edgecolor="white")
ax.bar_label(bars, padding=3, fontsize=9)
ax.set_xlabel("Variant Class")
ax.set_ylabel("Count")
ax.set_title(f"dbSNP Variant Classes in {gene} (n={len(df)})")
plt.tight_layout()
plt.savefig(f"{gene}_variant_classes.png", dpi=150, bbox_inches="tight")
print(f"Saved {gene}_variant_classes.png")
print(class_counts.to_string())
# snv 241
# indel 38
# mnv 12
# del 7
# ins 2
Key Parameters
| Parameter | Function/Endpoint | Default | Range / Options | Effect |
|---|---|---|---|---|
db | All E-utilities | required | "snp" | Database selector; must be "snp" for dbSNP queries |
id | efetch, esummary, epost | required | rsID number(s) without rs prefix | Variant identifier(s) to fetch |
term | esearch | required | dbSNP query string | Search expression with field tags: [gene], [CHR], [CHRPOS], [rs], clinsig[filter] |
retmax | esearch | 20 | 1–10000 | Maximum records returned per search |
retmode | esearch, esummary | "xml" | "json", "xml" | Response format; use "json" for easy parsing |
rettype | efetch | "docsum" | "docsum", "xml" | Record type for efetch responses |
WebEnv + query_key | esummary, efetch | — | from epost response | History server tokens for batch retrieval |
email | All E-utilities | required | valid email string | NCBI policy; used for rate attribution |
api_key | All E-utilities | optional | NCBI API key string | Raises rate limit from 3 to 10 req/sec |
Best Practices
-
Register for a free NCBI API key: Adds
api_key=YOUR_KEYto requests and triples your rate limit (3 → 10 req/sec) with no other changes. Register at https://www.ncbi.nlm.nih.gov/account/. -
Use epost+esummary for batches of more than 10 rsIDs: Avoid looping individual efetch calls. EPost uploads all IDs in one request to the history server; subsequent ESummary calls retrieve them in configurable batches of up to 500.
-
Prefer NCBI Variation Services API for structured JSON: The
/variation/v0/refsnp/{rs_num}endpoint returns a fully structured JSON with SPDI allele representations, placements, and frequency tables. Easier to parse than E-utilities XML for modern applications. -
Check
snp_classbefore interpreting MAF: Indels and MNVs use different allele counting conventions than SNVs. Treat multi-allelic sites carefully — the reported MAF may refer to one allele among several. -
Combine dbSNP with ClinVar lookups: dbSNP records the
clinical_significancefield as a string (e.g.,"pathogenic") but does not contain submitter details, review status, or HGVS details. Useclinvar-databasefor the full pathogenicity record when clinical interpretation is required.
Common Recipes
Recipe: Quick rsID Existence and Class Check
When to use: Check whether a variant is registered in dbSNP before downstream annotation.
import requests
EMAIL = "your@email.com"
def check_rsid(rsid: str) -> dict:
"""Check if an rsID exists in dbSNP and return basic info."""
rs_num = str(rsid).lstrip("rs")
r = requests.get(
"https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi",
params={"db": "snp", "id": rs_num, "retmode": "json", "email": EMAIL},
timeout=10
)
result = r.json().get("result", {})
rec = result.get(rs_num, {})
if not rec or "error" in rec:
return {"rsid": rsid, "found": False}
gmafs = rec.get("global_mafs", [])
return {
"rsid": rsid,
"found": True,
"snp_class": rec.get("snp_class"),
"chrpos": rec.get("chrpos"),
"maf": gmafs[0]["freq"] if gmafs else None,
"maf_study": gmafs[0]["study"] if gmafs else None,
"clinical_significance": rec.get("clinical_significance"),
}
for rsid in ["rs80357906", "rs9999999999", "rs1800497"]:
info = check_rsid(rsid)
if info["found"]:
print(f"{rsid}: {info['snp_class']} | pos={info['chrpos']} | MAF={info['maf']} ({info['maf_study']})")
else:
print(f"{rsid}: NOT FOUND in dbSNP")
# rs80357906: delins | pos=17:43057062 | MAF=G=0.0008929/4 (Estonian)
# rs9999999999: NOT FOUND in dbSNP
# rs1800497: snv | pos=11:113400106 | MAF=G=0.42... (study varies)
Recipe: Resolve Gene Variants to rsIDs and Coordinates
When to use: Convert a gene name to a list of rsIDs for use in downstream tools (PLINK, ANNOVAR, etc.).
import requests, time, pandas as pd
EMAIL = "your@email.com"
BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
def gene_to_rsids(gene: str, max_variants: int = 200) -> pd.DataFrame:
"""Search dbSNP for a gene and return rsIDs with GRCh38 coordinates."""
# Step 1: search
r = requests.get(f"{BASE}/esearch.fcgi",
params={"db": "snp", "term": f"{gene}[gene] AND human[orgn]",
"retmax": max_variants, "retmode": "json", "email": EMAIL},
timeout=15)
r.raise_for_status()
ids = r.json()["esearchresult"]["idlist"]
if not ids:
return pd.DataFrame()
time.sleep(0.4)
# Step 2: fetch summaries
r2 = requests.post(f"{BASE}/esummary.fcgi",
data={"db": "snp", "id": ",".join(ids),
"retmode": "json", "email": EMAIL},
timeout=30)
r2.raise_for_status()
result = r2.json()["result"]
rows = []
for uid in result.get("uids", []):
rec = result[uid]
gmafs = rec.get("global_mafs", [])
rows.append({
"rsid": f"rs{uid}",
"chrpos_grch38": rec.get("chrpos"),
"snp_class": rec.get("snp_class"),
"maf": gmafs[0]["freq"] if gmafs else None,
})
return pd.DataFrame(rows)
df = gene_to_rsids("APOE", max_variants=50)
print(f"APOE variants retrieved: {len(df)}")
print(df.head(8).to_string(index=False))
df.to_csv("APOE_rsids.csv", index=False)
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
HTTP 429 or connection refused | Rate limit exceeded (3 req/sec) | Add time.sleep(0.35) between requests; register for API key to get 10 req/sec |
ESummary returns {"error": "Invalid uid"} | rsID does not exist in dbSNP | Check rsID spelling; verify with NCBI browser; variant may be a novel call not yet in dbSNP |
esearch returns 0 results for a gene | Gene symbol mismatch or missing human[orgn] filter | Try adding AND human[orgn]; check NCBI gene symbol at https://www.ncbi.nlm.nih.gov/gene |
KeyError: 'maf' / 'mafallele' in ESummary parsing | Fields removed in the 2024 dbSNP ESummary schema | Use rec["global_mafs"] — list of {"study": ..., "freq": "<allele>=<value>/<count>"}; pick a study (GnomAD_genomes, TOPMED, ALFA) and parse the freq string |
global_mafs empty | Variant has no aggregated population frequency in dbSNP | Use gnomAD via gnomad-database directly for population frequencies |
root.iter("DocumentSummary") returns 0 with rettype="xml" | EFetch XML root is namespaced ({https://www.ncbi.nlm.nih.gov/SNP/docsum}ExchangeSet) | Use rettype="docsum" (no namespace) or pass the full namespaced tag to iter() |
| Variation Services API returns 404 | rsID not found or wrong URL format | Confirm integer rs number (no rs prefix) in /refsnp/{rs_num} endpoint |
| EPost XML parsing fails | Non-XML response (rate limit HTML error page) | Check response status code first; add retry logic with time.sleep(1) |
| Batch efetch returns fewer records than posted | Some rsIDs were merged or retired | Cross-check against NCBI merge history; retired rsIDs redirect to current active rs |
Related Skills
clinvar-database— ClinVar pathogenicity classifications for variants identified by rsID (complement to dbSNP)gnomad-database— Population allele frequencies by ancestry group (more detailed than dbSNP MAF)gwas-database— GWAS Catalog for SNP-trait associations from published GWAS studiesensembl-database— Ensembl REST API for variant consequences and gene annotationssnpeff-variant-annotation— Annotate VCF files with SnpEff and SnpSift, which adds dbSNP rsIDs and functional predictions
References
- NCBI E-utilities for dbSNP — E-utilities field tags and query syntax specific to dbSNP
- NCBI Variation Services API — REST API documentation and interactive Swagger UI
- Sherry et al., Nucleic Acids Research 2001 — dbSNP original description paper
- NCBI E-utilities Reference — Full E-utilities API reference (applies to all NCBI databases)
DepMap CRISPR Gene Effect Analysis Guide
Overview
This guide covers the correct interpretation and analysis of DepMap CRISPR gene effect (Chronos) data. The most critical and common error in DepMap analyses is failing to negate the CRISPR scores when computing correlations with "essentiality." A secondary but equally damaging mistake is using bulk correlation shortcuts that mishandle per-gene NaN patterns. This guide provides the mandatory sign convention, the correct per-gene NaN-safe Spearman correlation implementation, and data loading/alignment procedures.
Key Concepts
DepMap CRISPR Score Convention
The CRISPR gene effect score (produced by the Chronos algorithm) quantifies how gene knockout affects cell viability:
- Negative score: gene knockout reduces cell viability -- the gene is essential for that cell line
- Zero score: no measurable effect on viability
- Positive score: gene knockout increases viability (rare, may indicate tumor-suppressive behavior)
The DepMap portal distributes these scores in the file CRISPRGeneEffect.csv. Each row is a cell line (DepMap ID, e.g., ACH-000001) and each column is a gene in the format GENE_NAME (ENTREZ_ID), e.g., A1BG (1).
Essentiality Sign Interpretation
Because negative raw scores indicate essentiality, any analysis that asks about "essentiality" or "dependency" requires negating the raw CRISPR scores:
- "Correlation with essentiality" = correlation with
-CRISPRGeneEffect(negated) - "Higher essentiality" = more negative raw score = more positive negated score
- "Most essential gene" = gene with the most negative raw score
If you correlate expression with raw CRISPR scores and find 3 genes with correlation <= -0.6 and 0 genes with correlation >= 0.6, then the correct answer for "genes with strong positive correlation with essentiality" is 3, not 0. The negative correlations with raw scores ARE the positive correlations with essentiality.
Data Structure: CRISPRGeneEffect Format
The standard DepMap data files use a consistent structure:
- Index: DepMap cell line identifiers (
ACH-XXXXXX) - Columns: Gene identifiers in
GENE_NAME (ENTREZ_ID)format - Values: Floating-point scores (may contain NaN for genes not screened in a given cell line)
- Companion files: Expression data (
OmicsExpressionProteinCodingGenesTPMLogp1BatchCorrected.csv) uses the same index/column format, enabling direct alignment
Different genes have different patterns of missing data across cell lines. This is because not all genes are screened in all cell lines, and quality control may remove specific gene-cell line combinations.
Decision Framework
Question: How should I compute correlations with DepMap CRISPR data?
├── Does the question mention "essentiality" or "dependency"?
│ ├── Yes → Negate CRISPR scores before correlating (see Best Practices #1)
│ └── No (raw gene effect) → Use raw scores directly
├── How should I compute correlations?
│ ├── Per-gene correlation → scipy.stats.spearmanr in a loop (see Best Practices #2)
│ └── Matrix-wide correlation → AVOID; use per-gene loop instead
└── How should I handle missing data?
├── Pairwise NaN removal → CORRECT (see Best Practices #3)
└── Global row/column dropping → INCORRECT; loses too much data
| Scenario | Recommended Approach | Rationale |
|---|---|---|
| Correlating expression with "essentiality" | Negate CRISPR scores, then per-gene Spearman | Sign convention requires negation; per-gene handles NaN correctly |
| Correlating expression with raw gene effect | Per-gene Spearman on raw scores | No negation needed, but NaN-safe per-gene loop still required |
| Ranking genes by essentiality across cell lines | Rank by most negative mean raw score | More negative = more essential across the panel |
| Identifying selectively essential genes | Compare score distributions across subgroups | Use per-subgroup mean/median of raw scores, then compare |
| Filtering genes before correlation | Require minimum 10 valid cell line pairs | Genes with too few observations yield unreliable correlations |
Best Practices
-
Always negate CRISPR scores when the analysis asks about "essentiality": The raw DepMap convention is that negative = essential. When a question or hypothesis refers to "essentiality," "dependency," or "gene importance," negate the scores so that higher values mean more essential. Explicitly state the sign convention in your results.
-
Use scipy.stats.spearmanr per gene in a loop: Bulk matrix shortcuts (
DataFrame.corrwith,DataFrame.rank().corrwith()) handle NaN inconsistently across columns. The only reliable method is to compute Spearman correlation gene by gene usingscipy.stats.spearmanrwith pairwise-complete observations. -
Apply pairwise NaN removal, not global dropping: Different genes have different missing-data patterns. Dropping rows globally (any NaN in any column) discards far too much data. Instead, for each gene, mask out only the cell lines where either the expression or CRISPR value is NaN.
-
Set a minimum valid-pair threshold: Genes with very few non-NaN cell line pairs produce unreliable correlation estimates. Require at least 10 (preferably 20+) valid pairs before computing a correlation. Skip genes below this threshold.
-
Report NaN summary before analysis: Before computing correlations, print the total NaN count per dataset, the number of common cell lines, and the number of common genes. This provides an audit trail and helps catch data loading errors early.
-
Verify dataset alignment before computation: Always intersect cell line IDs and gene columns between datasets before analysis. Misaligned indices produce silent errors -- correlations computed on mismatched rows are meaningless.
-
State the sign convention explicitly in results: When reporting correlation results, always include a statement like "CRISPR scores were negated so that positive values represent higher essentiality." This prevents downstream misinterpretation.
Common Pitfalls
-
Forgetting to negate CRISPR scores for essentiality analysis: The raw DepMap score convention (negative = essential) is counterintuitive. Omitting the negation reverses every correlation sign, leading to completely inverted conclusions.
- How to avoid: Always check whether the analysis question uses the word "essentiality" or "dependency." If so, negate the CRISPR scores. Add a comment in the code:
# Negate: in DepMap, negative = essential.
- How to avoid: Always check whether the analysis question uses the word "essentiality" or "dependency." If so, negate the CRISPR scores. Add a comment in the code:
-
Using bulk DataFrame correlation methods: Methods like
DataFrame.corrwith(method='spearman')orDataFrame.rank().corrwith()silently mishandle NaN values, potentially shifting correlations enough to push genes above or below significance thresholds.- How to avoid: Always use a per-gene loop with
scipy.stats.spearmanr. See the reference implementation in the Workflow section below.
- How to avoid: Always use a per-gene loop with
-
Dropping rows globally instead of pairwise: Calling
dropna()on the entire DataFrame before correlation removes all cell lines that have any NaN in any gene, drastically reducing sample size.- How to avoid: Apply NaN masking inside the per-gene loop:
mask = ~(np.isnan(x) | np.isnan(y)). This preserves the maximum number of observations per gene.
- How to avoid: Apply NaN masking inside the per-gene loop:
-
Not checking for sufficient valid pairs: Computing Spearman correlation on fewer than 10 observations produces unstable, unreliable estimates that may appear significant by chance.
- How to avoid: Add a guard clause:
if mask.sum() < 10: continue. Adjust the threshold upward (e.g., 20) for more conservative analysis.
- How to avoid: Add a guard clause:
-
Misinterpreting correlation signs without stated convention: Reporting "positive correlation" without stating whether CRISPR scores were negated leaves results ambiguous. Reviewers cannot tell if "positive" means "higher expression associates with more essential" or "less essential."
- How to avoid: Always include a sentence in the results section stating the sign convention used. For example: "CRISPR scores were negated so that positive correlation indicates higher expression is associated with greater essentiality."
-
Failing to align datasets before computation: Expression and CRISPR datasets may have different cell lines or different gene sets. Computing correlations without explicit alignment can silently match wrong rows or produce index errors.
- How to avoid: Always compute
common_lines = expr.index.intersection(crispr.index)andcommon_genes = expr.columns.intersection(crispr.columns), then subset both DataFrames before any computation.
- How to avoid: Always compute
-
Ignoring the gene column format: DepMap gene columns use the format
GENE_NAME (ENTREZ_ID). Attempting to match against plain gene symbols (e.g.,TP53instead ofTP53 (7157)) will produce empty intersections.- How to avoid: Inspect column formats with
df.columns[:5]before attempting any join or intersection. Parse gene names if needed:df.columns.str.extract(r'^(.+?)\s*\(')[0].
- How to avoid: Inspect column formats with
Workflow
-
Step 1: Load DepMap data
import pandas as pd # Load CRISPR gene effect data crispr = pd.read_csv('CRISPRGeneEffect.csv', index_col=0) # Load expression data expr = pd.read_csv( 'OmicsExpressionProteinCodingGenesTPMLogp1BatchCorrected.csv', index_col=0 ) # Column format: "GENE_NAME (ENTREZ_ID)" e.g., "A1BG (1)" # Index: DepMap cell line IDs e.g., "ACH-000001" -
Step 2: Align datasets
# Find common cell lines and genes common_lines = crispr.index.intersection(expr.index) common_genes = crispr.columns.intersection(expr.columns) print(f"Common cell lines: {len(common_lines)}") print(f"Common genes: {len(common_genes)}") # Subset to common crispr_aligned = crispr.loc[common_lines, common_genes] expr_aligned = expr.loc[common_lines, common_genes] -
Step 3: Report NaN summary
expr_nan = expr_aligned.isna().sum().sum() crispr_nan = crispr_aligned.isna().sum().sum() print(f"Expression NaN count: {expr_nan}") print(f"CRISPR NaN count: {crispr_nan}") -
Step 4: Negate CRISPR scores if computing essentiality correlations
# Negate: in DepMap, negative raw score = essential # After negation, positive = essential essentiality = -crispr_aligned -
Step 5: Compute per-gene NaN-safe Spearman correlation
from scipy.stats import spearmanr import numpy as np def compute_per_gene_spearman(expression_df, crispr_df, negate_crispr=True): """Compute Spearman correlation per gene with proper NaN handling. Args: expression_df: DataFrame (cell_lines x genes) crispr_df: DataFrame (cell_lines x genes) negate_crispr: If True, negate CRISPR scores to represent essentiality Returns: Series of Spearman correlations indexed by gene name """ # Align cell lines and genes common_lines = expression_df.index.intersection(crispr_df.index) common_genes = expression_df.columns.intersection(crispr_df.columns) expr = expression_df.loc[common_lines, common_genes] crispr = crispr_df.loc[common_lines, common_genes] if negate_crispr: crispr = -crispr # Print NaN summary BEFORE analysis expr_nan = expr.isna().sum().sum() crispr_nan = crispr.isna().sum().sum() print(f"Expression NaN count: {expr_nan}") print(f"CRISPR NaN count: {crispr_nan}") print(f"Common cell lines: {len(common_lines)}") print(f"Common genes: {len(common_genes)}") # Per-gene Spearman correlation with pairwise NaN removal correlations = {} for gene in common_genes: x = expr[gene].values y = crispr[gene].values # Remove pairs where either value is NaN mask = ~(np.isnan(x) | np.isnan(y)) if mask.sum() < 10: # Skip genes with too few valid pairs continue rho, pval = spearmanr(x[mask], y[mask]) correlations[gene] = rho return pd.Series(correlations).sort_values(ascending=False) -
Step 6: Apply threshold and report results
correlations = compute_per_gene_spearman(expr_aligned, crispr_aligned, negate_crispr=True) threshold = 0.6 strong_positive = correlations[correlations >= threshold] strong_negative = correlations[correlations <= -threshold] print(f"Genes with correlation >= {threshold}: {len(strong_positive)}") print(f"Genes with correlation <= -{threshold}: {len(strong_negative)}") print(f"\nNote: CRISPR scores were negated so that positive correlation") print(f"indicates higher expression associated with greater essentiality.") -
Step 7: Validate -- check for anti-patterns
Verify that none of these bulk shortcuts were used anywhere in the analysis:
# WRONG: Bulk rank-then-correlate shortcut ranked_expr = expression_df.rank() ranked_crispr = crispr_df.rank() correlations = ranked_expr.corrwith(ranked_crispr) # NaN handling is unreliable # WRONG: Bulk corrwith with method='spearman' correlations = expression_df.corrwith(crispr_df, method='spearman') # Same issueIf any of these patterns appear in the code, replace them with the per-gene loop from Step 5.
Further Reading
- DepMap Portal -- Primary data source for CRISPR gene effect scores, expression data, and other omics datasets across cancer cell lines
- Dempster et al. (2019) -- Chronos algorithm -- Original paper describing the Chronos computational method for estimating gene effect from CRISPR screen data
- DepMap Documentation and Data Downloads -- Detailed file format descriptions, release notes, and download links for all DepMap datasets
Related Skills
nan-safe-correlation-- General techniques for NaN-safe correlation computation across omics datasets; this guide applies those principles specifically to DepMap CRISPR datadegenerate-input-filtering-- Upstream data quality filtering to remove low-variance or degenerate features before correlation analysis; recommended as a preprocessing step before DepMap essentiality correlation
ENA Database — European Nucleotide Archive Programmatic Access
Overview
The European Nucleotide Archive (ENA) is EMBL-EBI's comprehensive nucleotide sequence database, encompassing raw sequencing reads, genome assemblies, annotated sequences, and associated metadata. It mirrors and extends INSDC data (GenBank, DDBJ). All access is via REST APIs with no authentication required.
When to Use
- Searching for sequencing studies, samples, or experiments by organism, project, or keyword
- Downloading raw FASTQ/BAM files for reanalysis of public sequencing datasets
- Retrieving genome assemblies with quality statistics (N50, contig count, genome size)
- Fetching nucleotide sequences in FASTA or EMBL flat-file format by accession
- Exploring taxonomic lineage and finding organisms by partial name
- Cross-referencing ENA records with external databases (ArrayExpress, UniProt, PDB)
- Building bulk download lists for large-scale sequencing projects
- For multi-database Python queries (ENA + UniProt + KEGG), prefer
bioservicesinstead - For NCBI-specific queries (PubMed literature, GenBank records), use
pubmed-databaseor Biopython Entrez
Prerequisites
pip install requests
API constraints:
- Rate limit: 50 requests per second across all ENA APIs
- No authentication required
- Large result sets: use pagination (
limit+offset) or streaming (limit=0for TSV download) - Portal API base:
https://www.ebi.ac.uk/ena/portal/api - Browser API base:
https://www.ebi.ac.uk/ena/browser/api - Taxonomy API base:
https://www.ebi.ac.uk/ena/taxonomy/rest - Cross-ref API base:
https://www.ebi.ac.uk/ena/xref/rest
Quick Start
import requests
import time
BASE_PORTAL = "https://www.ebi.ac.uk/ena/portal/api"
BASE_BROWSER = "https://www.ebi.ac.uk/ena/browser/api"
BASE_TAXONOMY = "https://www.ebi.ac.uk/ena/taxonomy/rest"
BASE_XREF = "https://www.ebi.ac.uk/ena/xref/rest"
def ena_query(endpoint, params=None, base=BASE_PORTAL):
"""Reusable ENA API caller with rate-limit compliance."""
resp = requests.get(f"{base}/{endpoint}", params=params)
resp.raise_for_status()
time.sleep(0.02) # 50 req/sec limit
return resp
# Search for human RNA-seq studies
resp = ena_query("search", params={
"result": "study",
"query": 'tax_tree(9606)', # `library_strategy` is a `read_run`/`read_experiment` field, not a `study` field
"fields": "study_accession,study_title",
"format": "json",
"limit": 3,
})
studies = resp.json()
for s in studies:
print(f"{s['study_accession']}: {s['study_title'][:60]}")
# PRJEB12345: Transcriptome analysis of human liver tissue...
Core API
Module 1: Portal API Search
The Portal API provides advanced metadata search across all ENA data types with boolean query syntax, field selection, and pagination.
# Search read runs for a specific study
resp = ena_query("search", params={
"result": "read_run",
"query": 'study_accession="PRJEB1787"',
"fields": "run_accession,sample_accession,instrument_model,read_count,base_count",
"format": "json",
"limit": 5,
})
runs = resp.json()
for r in runs:
print(f"{r['run_accession']} — {r.get('instrument_model', 'N/A')}, "
f"{int(r.get('read_count', 0)):,} reads")
# ERR123456 — Illumina HiSeq 2000, 45,231,890 reads
# Count total results without fetching data
count_resp = ena_query("count", params={
"result": "read_run",
"query": 'study_accession="PRJEB1787"',
})
print(f"Total runs: {count_resp.text.strip()}")
# Total runs: 142
Module 2: Browser API Retrieval
Fetch individual records by accession in multiple formats: XML, FASTA, EMBL flat-file, or plain text.
# Retrieve XML metadata for a study
resp = ena_query("xml/PRJEB1787", base=BASE_BROWSER)
print(resp.text[:300])
# <?xml version="1.0" encoding="UTF-8"?><PROJECT_SET>...
# Retrieve FASTA sequence for a coding sequence
resp = ena_query("fasta/M10051.1", base=BASE_BROWSER)
print(resp.text[:200])
# >ENA|M10051|M10051.1 Human insulin mRNA, complete cds.
# AGCCCTCCAGGACAGGCTGCAT...
# Retrieve EMBL flat-file format
resp = ena_query("embl/M10051.1", base=BASE_BROWSER)
print(resp.text[:300])
# ID M10051; SV 1; linear; mRNA; STD; HUM; 786 BP.
# ...
Module 3: File Reports and Downloads
Get download URLs for FASTQ, submitted, and analysis files. File reports return FTP and Aspera paths.
# Get FASTQ file URLs for specific runs
resp = ena_query("filereport", params={
"accession": "ERR000589",
"result": "read_run",
"fields": "run_accession,fastq_ftp,fastq_bytes,fastq_md5",
"format": "json",
})
files = resp.json()
for f in files:
ftp_urls = f.get("fastq_ftp", "").split(";")
sizes = f.get("fastq_bytes", "").split(";")
for url, size in zip(ftp_urls, sizes):
if url:
print(f"ftp://{url} ({int(size)/1e6:.1f} MB)")
# ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR000/ERR000589/ERR000589_1.fastq.gz (234.5 MB)
# ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR000/ERR000589/ERR000589_2.fastq.gz (241.2 MB)
Module 4: Taxonomy Queries
Look up organisms by taxonomy ID, scientific name, or partial name match.
# Lookup by taxonomy ID
resp = ena_query("tax-id/9606", base=BASE_TAXONOMY)
tax = resp.json()
print(f"{tax['scientificName']} (taxId: {tax['taxId']}, rank: {tax['rank']})")
# Homo sapiens (taxId: 9606, rank: species)
print(f"Lineage: {tax['lineage'][:80]}...")
# Search by scientific name — endpoint returns a list (one entry per matching taxon)
resp = ena_query("scientific-name/Arabidopsis thaliana", base=BASE_TAXONOMY)
matches = resp.json()
result = matches[0] if isinstance(matches, list) else matches
print(f"Tax ID: {result['taxId']}, Common: {result.get('commonName', 'N/A')}")
# Tax ID: 3702, Common: thale cress
# Suggest organisms by partial name
resp = ena_query("suggest-for-search/salmo", base=BASE_TAXONOMY)
suggestions = resp.json()
for s in suggestions[:3]:
print(f" {s['scientificName']} (taxId: {s['taxId']})")
# Salmo salar (taxId: 8030)
# Salmo trutta (taxId: 8032)
# Salmonella enterica (taxId: 28901)
Module 5: Cross-Reference Service
Find links between ENA records and external databases (ArrayExpress, UniProt, PDB, etc.).
# Find cross-references for an ENA accession
resp = ena_query("json/search", base=BASE_XREF, params={
"accession": "M10051",
})
xrefs = resp.json()
for x in xrefs[:5]:
print(f" {x['Source']} → {x['Source Primary Accession']} "
f"({x.get('Source Description', '')[:50]})")
# UniProt → P01308 (Insulin precursor)
# PDB → 1A7F (Crystal structure of human insulin)
# Search cross-references by external database
resp = ena_query("json/search", base=BASE_XREF, params={
"source": "UniProt",
"accession": "P01308",
})
xrefs = resp.json()
for x in xrefs[:3]:
print(f" ENA: {x['Target Primary Accession']} — {x.get('Target Description', '')[:60]}")
Module 6: CRAM Reference Registry
Retrieve reference sequences used in CRAM files by MD5 or SHA1 checksum. Essential for CRAM decompression.
# Look up reference by MD5 checksum
md5 = "aef131c3b4b05d8e2b3f907faba5af9b" # example
try:
resp = ena_query(
f"cram/md5/{md5}",
base="https://www.ebi.ac.uk/ena/cram"
)
print(f"Reference found: {len(resp.content)} bytes")
except requests.HTTPError as e:
if e.response.status_code == 404:
print("Reference not found — check MD5 checksum")
else:
raise
Key Concepts
ENA Data Hierarchy
| Level | Accession Prefix | Description | Contains |
|---|---|---|---|
| Study | PRJEB/ERP | Research project | Samples, Experiments |
| Sample | ERS/SAMEA | Biological sample | Metadata, taxonomy |
| Experiment | ERX | Library/sequencing setup | Runs |
| Run | ERR | Sequencing run | Raw read files (FASTQ) |
| Analysis | ERZ | Derived analysis | Assemblies, alignments |
| Assembly | GCA | Genome assembly | Contigs, scaffolds |
| Sequence | Accession.version | Annotated sequence | Features, coding seqs |
Query Syntax Operators
| Operator | Example | Description |
|---|---|---|
| Equality | instrument_model="Illumina NovaSeq 6000" | Exact match |
| Wildcard | study_title="*melanoma*" | Partial match |
| Range | base_count>=1000000 | Numeric comparison |
| Taxonomy tree | tax_tree(9606) | Taxon and all descendants |
| Exact taxon | tax_eq(9606) | Exact taxon only |
| Date range | first_public>=2023-01-01 | Date filtering |
| Boolean | AND, OR, NOT | Combine conditions |
| Grouping | (A OR B) AND C | Parenthetical grouping |
Result Types
| Result Type | Description | Key Fields |
|---|---|---|
study | Research projects | study_accession, study_title, center_name |
sample | Biological samples | sample_accession, tax_id, scientific_name |
read_run | Sequencing runs | run_accession, read_count, base_count, fastq_ftp |
read_experiment | Experiments | experiment_accession, library_strategy, instrument_model |
analysis | Derived analyses | analysis_accession, analysis_type |
assembly | Genome assemblies | assembly_accession, assembly_level, genome_representation |
sequence | Annotated sequences | accession, sequence_length, mol_type |
wgs_set | WGS scaffold sets | set_accession, set_size |
tsa_set | Transcriptome assemblies | set_accession, set_size |
coding | Coding sequences | accession, gene, product |
noncoding | Non-coding features | accession, description |
taxon | Taxonomy entries | tax_id, scientific_name, lineage |
Discoverable Fields
Use the returnFields endpoint to discover available fields for any result type:
resp = ena_query("returnFields", params={"result": "read_run"})
fields = resp.text.strip().split("\n")
print(f"Available fields for read_run: {len(fields)}")
print(fields[:10])
# ['accession', 'altitude', 'assembly_quality', 'assembly_software', ...]
Common Workflows
Workflow 1: Study Exploration Pipeline
Search for a study, list its samples, then retrieve run metadata.
import json
# Step 1: Find studies by organism — `study_title="*…*"` wildcards no longer match;
# use `tax_tree()` against the species tax ID (SARS-CoV-2 = 2697049).
resp = ena_query("search", params={
"result": "study",
"query": 'tax_tree(2697049) AND first_public>=2023-01-01',
"fields": "study_accession,study_title,center_name",
"format": "json",
"limit": 3,
})
studies = resp.json()
study_acc = studies[0]["study_accession"]
print(f"Selected: {study_acc} — {studies[0]['study_title'][:60]}")
# Step 2: List samples in the study
resp = ena_query("search", params={
"result": "sample",
"query": f'study_accession="{study_acc}"',
"fields": "sample_accession,scientific_name,collection_date",
"format": "json",
"limit": 5,
})
samples = resp.json()
print(f"Found {len(samples)} samples (showing first 5)")
for s in samples:
print(f" {s['sample_accession']} — {s.get('scientific_name', 'N/A')}")
# Step 3: Get run metadata for each sample
for s in samples[:2]:
resp = ena_query("search", params={
"result": "read_run",
"query": f'sample_accession="{s["sample_accession"]}"',
"fields": "run_accession,instrument_model,read_count,library_strategy",
"format": "json",
})
runs = resp.json()
for r in runs:
print(f" {r['run_accession']}: {r.get('library_strategy','N/A')}, "
f"{int(r.get('read_count',0)):,} reads")
time.sleep(0.02)
Workflow 2: Bulk FASTQ Download URL Collection
Search for runs matching criteria and collect download URLs.
# Step 1: Search for Illumina RNA-Seq runs from a specific organism
resp = ena_query("search", params={
"result": "read_run",
"query": ('tax_tree(10090) AND library_strategy="RNA-Seq" '
'AND instrument_platform="ILLUMINA" AND read_count>=10000000'),
"fields": "run_accession,study_accession,read_count",
"format": "json",
"limit": 10,
})
runs = resp.json()
print(f"Found {len(runs)} runs meeting criteria")
# Step 2: Get file reports with download URLs
download_list = []
for run in runs[:5]:
acc = run["run_accession"]
resp = ena_query("filereport", params={
"accession": acc,
"result": "read_run",
"fields": "run_accession,fastq_ftp,fastq_bytes,fastq_md5",
"format": "json",
})
for f in resp.json():
urls = f.get("fastq_ftp", "").split(";")
md5s = f.get("fastq_md5", "").split(";")
for url, md5 in zip(urls, md5s):
if url:
download_list.append({"url": f"ftp://{url}", "md5": md5, "run": acc})
time.sleep(0.02)
print(f"\nDownload list: {len(download_list)} files")
for d in download_list[:4]:
print(f" {d['run']}: {d['url'].split('/')[-1]}")
Workflow 3: Taxonomic Assembly Exploration
Find organisms, search their assemblies, and check quality statistics.
# Step 1: Resolve organism by exact scientific name (returns a list, take the first match).
# `suggest-for-search` only returns prefix matches and may not include the species you want.
resp = ena_query("scientific-name/Drosophila melanogaster", base=BASE_TAXONOMY)
matches = resp.json()
target_tax = matches[0]["taxId"]
print(f"Selected: {matches[0]['scientificName']} (taxId={target_tax})")
# Step 2: Search assemblies for this organism. `n50` is no longer a valid `assembly`
# field — use `base_count` and `program` for what's available.
resp = ena_query("search", params={
"result": "assembly",
"query": f'tax_eq({target_tax}) AND assembly_level="chromosome"',
"fields": ("assembly_accession,assembly_name,assembly_level,"
"genome_representation,base_count"),
"format": "json",
"limit": 5,
})
assemblies = resp.json()
for a in assemblies:
size = int(a.get("base_count", 0))
print(f" {a['assembly_accession']}: {a.get('assembly_name','N/A')}, "
f"Size={size/1e6:.1f} Mb")
# GCA_000001215.4: Release 6 plus ISO1 MT, Size=143.7 Mb
Key Parameters
| Endpoint | Parameter | Default | Description |
|---|---|---|---|
search | result | (required) | Result type: study, sample, read_run, assembly, etc. |
search | query | (required) | Boolean query string with field operators |
search | fields | all | Comma-separated field names to return |
search | format | tsv | Output format: json, tsv, xml |
search | limit | 100000 | Max results (0 = unlimited streaming) |
search | offset | 0 | Skip first N results (pagination) |
search | sortFields | — | Field(s) to sort by |
filereport | accession | (required) | Study, sample, or run accession |
filereport | result | (required) | read_run, analysis, etc. |
xml/ | accession path | (required) | Any ENA accession (Browser API) |
fasta/ | accession path | (required) | Sequence accession (Browser API) |
Best Practices
- Use
tax_tree()overtax_eq()for organism queries — it includes subspecies and strains automatically - Request only needed fields — reduces response size and server load significantly
- Prefer JSON format for programmatic access; TSV for large bulk exports (lower overhead)
- Use
limit=0for streaming large result sets directly to file, avoiding memory issues - Check
fastq_ftpandsubmitted_ftp— some runs have submitted files but no processed FASTQ - Verify downloads with MD5 — file reports include
fastq_md5for integrity checking - Anti-pattern: Do not fetch all fields then filter client-side — use query syntax server-side
Common Recipes
Recipe: Assembly Quality Filtering
# Find chromosome-level, full-representation assemblies.
# `n50` is no longer a valid `assembly` field — filter by base_count instead.
resp = ena_query("search", params={
"result": "assembly",
"query": ('tax_tree(7742) AND assembly_level="chromosome" '
'AND genome_representation="full"'),
"fields": "assembly_accession,scientific_name,base_count,assembly_level",
"format": "json",
"limit": 10,
})
for a in resp.json():
size = int(a.get("base_count", 0))
if size > 100_000_000: # >100 Mb (proxy for "well-assembled")
print(f"{a['assembly_accession']}: {a.get('scientific_name','?')}, Size={size/1e6:.0f} Mb")
Recipe: Cross-Database Linking
# Find UniProt/PDB cross-references for an ENA sequence
resp = ena_query("json/search", base=BASE_XREF, params={
"accession": "M10051",
})
xrefs = resp.json()
by_source = {}
for x in xrefs:
src = x.get("Source", "unknown")
by_source.setdefault(src, []).append(x["Source Primary Accession"])
for src, accs in by_source.items():
print(f" {src}: {', '.join(accs[:5])}")
# UniProt: P01308
# PDB: 1A7F, 1AI0, 1BEN
Recipe: Retry Session with Exponential Backoff
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
def ena_session():
"""Create a requests session with retry logic for ENA APIs."""
session = requests.Session()
retry = Retry(
total=3, backoff_factor=1.0,
status_forcelist=[429, 500, 502, 503, 504],
)
session.mount("https://", HTTPAdapter(max_retries=retry))
return session
session = ena_session()
resp = session.get(f"{BASE_PORTAL}/search", params={
"result": "study",
"query": 'tax_tree(9606)',
"fields": "study_accession",
"format": "json",
"limit": 5,
})
print(f"Status: {resp.status_code}, results: {len(resp.json())}")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
400 Bad Request on search | Invalid query syntax or unknown field name | Use returnFields endpoint to verify field names; check query operator syntax |
400 with result param | Invalid result type | Check result types table above; common: read_run not run |
| Empty results for known data | Wrong taxonomy operator | Use tax_tree() (includes descendants) not tax_eq() (exact only) |
fastq_ftp field is empty | Submitted files not processed to FASTQ | Check submitted_ftp field instead; some datasets only have BAM/CRAM |
429 Too Many Requests | Exceeded 50 req/sec rate limit | Add time.sleep(0.02) between requests; use retry session with backoff |
| Timeout on large queries | Result set too large for single request | Use limit + offset pagination, or limit=0 with streaming to file |
| XML parsing errors | Malformed XML for some records | Use JSON format instead (format=json) when available |
| Wrong sequence version | Accession without version suffix | Always use versioned accessions (e.g., M10051.1 not M10051) for Browser API |
| CRAM reference not found | MD5 checksum mismatch or non-INSDC reference | Verify MD5; check if reference is from a custom genome (not in registry) |
Bundled Resources
This skill is self-contained. The original entry had a separate references/api_reference.md (490 lines) covering all 6 API endpoints in detail. That content has been fully consolidated inline:
- Portal API (search, count, returnFields, filereport) — Core API Modules 1 and 3, Key Parameters table
- Browser API (XML, FASTA, EMBL retrieval) — Core API Module 2
- Taxonomy REST API (tax-id, scientific-name, suggest-for-search) — Core API Module 4
- Cross-Reference Service (json/search) — Core API Module 5
- CRAM Reference Registry (md5/sha1 lookup) — Core API Module 6
- Rate limiting and error handling — Prerequisites, Troubleshooting, Recipe 3 (retry session)
- Query syntax and result types — Key Concepts section (3 tables)
- Pagination and bulk download — Key Parameters, Best Practices, Workflow 2
- Omitted: detailed EMBL format field-by-field breakdown (rarely needed programmatically); Aspera download command examples (tool-specific, not requests-based)
Related Skills
- bioservices-multi-database — unified Python interface covering ENA via bioservices; prefer for multi-database workflows
- pubmed-database — PubMed literature search via NCBI E-utilities
- pysam-genomic-files — downstream processing of FASTQ/BAM/CRAM files retrieved from ENA
- biopython-molecular-biology — NCBI Entrez access and sequence parsing (GenBank/FASTA)
- ncbi-blast (planned) — BLAST sequence similarity search
References
- ENA Portal API docs: https://www.ebi.ac.uk/ena/portal/api/doc
- ENA Browser API docs: https://www.ebi.ac.uk/ena/browser/api/doc
- ENA Taxonomy REST API: https://www.ebi.ac.uk/ena/taxonomy/rest/
- ENA Cross-Reference Service: https://www.ebi.ac.uk/ena/xref/rest/
- ENA data model guide: https://ena-docs.readthedocs.io/en/latest/submit/general-guide/data-model.html
- INSDC standards: https://www.insdc.org/
ENCODE Database
Overview
The ENCODE (Encyclopedia of DNA Elements) Project has generated thousands of functional genomics experiments — TF ChIP-seq, ATAC-seq, DNase-seq, histone ChIP-seq, and RNA-seq — across 1000+ human and mouse cell types and tissues. The ENCODE Portal REST API provides structured JSON access to experiment metadata, file download URLs, and SCREEN cCRE (candidate cis-Regulatory Elements) annotations. All data is freely accessible without authentication for most endpoints.
When to Use
- Downloading TF ChIP-seq peak files (BED) for a specific transcription factor and cell type to annotate regulatory regions
- Finding ATAC-seq or DNase-seq peaks in a cell type to identify open chromatin regions near a gene of interest
- Retrieving cCREs (candidate cis-Regulatory Elements) overlapping a genomic region from ENCODE SCREEN
- Building reference regulatory tracks for variant annotation pipelines (e.g., annotating VCF variants against ENCODE peak sets)
- Exploring which experiments are available for a biosample (cell line, tissue, developmental stage) before planning a wet-lab experiment
- Querying all ChIP-seq experiments for a transcription factor across multiple cell types for comparative regulatory analysis
- Use
regulomedb-databaseinstead when you want pre-computed regulatory scores for specific SNPs — RegulomeDB integrates ENCODE data with eQTL and motif evidence into a single score - Use
deeptools-ngs-analysisinstead when you have your own BAM files and need to generate bigWig coverage tracks; ENCODE database is for retrieving existing deposited data
Prerequisites
- Python packages:
requests,pandas,matplotlib - Data requirements: experiment accessions (e.g.,
ENCSR000AKC), biosample names (e.g.,K562), TF target names (e.g.,CTCF,TP53), or genomic regions (chr7:117548628-117748628) - Environment: internet connection; no authentication required for public data; add
Authorization: Bearer {api_key}header for submitter access - Rate limits: no published hard limit; add
time.sleep(0.5)for large batch queries to avoid connection resets
pip install requests pandas matplotlib
Quick Start
import requests
BASE = "https://www.encodeproject.org"
def search_experiments(assay="TF ChIP-seq", target="CTCF", biosample="K562", limit=5):
"""Find ENCODE experiments matching assay type, target, and biosample."""
params = {
"type": "Experiment",
"assay_title": assay,
"target.label": target,
"biosample_ontology.term_name": biosample, # `biosample_summary` is a verbose freetext string; filter by ontology term name
"status": "released",
"format": "json",
"limit": limit,
}
r = requests.get(f"{BASE}/search/", params=params, timeout=30)
r.raise_for_status()
data = r.json()
experiments = data.get("@graph", [])
print(f"Found {data.get('total', 0)} experiments for {target} ChIP-seq in {biosample}")
for exp in experiments:
print(f" {exp['accession']} {exp.get('biosample_summary', '')} {exp.get('lab', {}).get('title', '')}")
return experiments
exps = search_experiments(assay="TF ChIP-seq", target="CTCF", biosample="K562")
Core API
Query 1: Experiment Search — Find Experiments by Assay, Biosample, Target
Search the ENCODE Portal for experiments matching structured criteria.
import requests, pandas as pd
BASE = "https://www.encodeproject.org"
def search_experiments(assay_title=None, target=None, biosample=None,
organism="Homo sapiens", status="released", limit=50):
"""
Search ENCODE experiments with flexible filters.
Returns: pd.DataFrame of matching experiments.
"""
params = {
"type": "Experiment",
"status": status,
"replicates.library.biosample.donor.organism.scientific_name": organism,
"format": "json",
"limit": limit,
}
if assay_title:
params["assay_title"] = assay_title
if target:
params["target.label"] = target
if biosample:
params["biosample_ontology.term_name"] = biosample # filter by ontology term, not the freetext `biosample_summary`
r = requests.get(f"{BASE}/search/", params=params, timeout=30)
r.raise_for_status()
data = r.json()
total = data.get("total", 0)
print(f"Total matching experiments: {total} (showing {min(limit, total)})")
records = []
for exp in data.get("@graph", []):
records.append({
"accession": exp.get("accession"),
"assay": exp.get("assay_title"),
"biosample": exp.get("biosample_summary"),
"target": exp.get("target", {}).get("label", ""),
"lab": exp.get("lab", {}).get("title", ""),
"date_released": exp.get("date_released", ""),
})
df = pd.DataFrame(records)
print(df.to_string(index=False))
return df
# CTCF ChIP-seq in HCT116 colon cancer cells
df = search_experiments(assay_title="TF ChIP-seq", target="CTCF", biosample="HCT116")
# ATAC-seq experiments in multiple cell types
df_atac = search_experiments(assay_title="ATAC-seq", limit=20)
print(f"\nUnique cell types: {df_atac['biosample'].nunique()}")
Query 2: File Download — Get Metadata and Download URLs for BED/bigWig Files
Retrieve file metadata for a specific experiment and obtain download URLs.
import requests, pandas as pd
BASE = "https://www.encodeproject.org"
def get_experiment_files(accession, file_format="bed", output_type="peaks",
assembly="GRCh38"):
"""
Get file download URLs for a specific ENCODE experiment.
accession: experiment accession, e.g. 'ENCSR000AKC'
file_format: 'bed', 'bigWig', 'fastq', 'bam'
output_type: 'peaks', 'signal', 'alignments', 'reads'
Returns: pd.DataFrame of matching files with download URLs.
"""
params = {
"type": "File",
"dataset": f"/experiments/{accession}/",
"file_format": file_format,
"output_type": output_type,
"assembly": assembly,
"status": "released",
"format": "json",
"limit": 50,
}
r = requests.get(f"{BASE}/search/", params=params, timeout=30)
r.raise_for_status()
data = r.json()
files = data.get("@graph", [])
print(f"Found {len(files)} {file_format} files ({output_type}) for {accession}")
records = []
for f in files:
records.append({
"file_accession": f.get("accession"),
"file_format": f.get("file_format"),
"output_type": f.get("output_type"),
"assembly": f.get("assembly"),
"file_size_mb": round(f.get("file_size", 0) / 1e6, 2),
"download_url": BASE + f.get("href", ""),
"biological_replicate": str(f.get("biological_replicates", [])),
})
df = pd.DataFrame(records)
print(df[["file_accession", "output_type", "assembly", "file_size_mb"]].to_string(index=False))
return df
# Get peak BED files for a CTCF ChIP-seq experiment
df_files = get_experiment_files("ENCSR000AKC", file_format="bed", output_type="peaks")
import urllib.request
def download_encode_file(download_url, output_path):
"""Download an ENCODE file from its href URL."""
print(f"Downloading {download_url} → {output_path}")
urllib.request.urlretrieve(download_url, output_path)
import os
size_mb = os.path.getsize(output_path) / 1e6
print(f"Downloaded {size_mb:.1f} MB → {output_path}")
# Download the first peak file
if len(df_files) > 0:
url = df_files.iloc[0]["download_url"]
download_encode_file(url, "CTCF_peaks.bed.gz")
Query 3: cCRE Region Query — ENCODE SCREEN Candidate cis-Regulatory Elements
Query ENCODE SCREEN for candidate cis-Regulatory Elements in a genomic region using the SCREEN API.
import requests, pandas as pd
SCREEN_BASE = "https://api.encodeproject.org/screen"
def search_ccres_by_region(chrom, start, end, assembly="GRCh38", limit=500):
"""
Find candidate cis-Regulatory Elements (cCREs) in a genomic region.
cCRE groups: PLS (promoter-like), pELS (proximal enhancer-like),
dELS (distal enhancer-like), DNase-H3K4me3, CTCF-only.
Returns: pd.DataFrame of cCREs with scores.
"""
payload = {
"assembly": assembly,
"coord_chrom": chrom,
"coord_start": start,
"coord_end": end,
"limit": limit,
}
r = requests.post(f"{SCREEN_BASE}/search/", json=payload, timeout=30)
r.raise_for_status()
data = r.json()
ccres = data.get("results", data.get("cCREs", []))
print(f"Found {len(ccres)} cCREs in {chrom}:{start}-{end}")
records = []
for c in ccres:
records.append({
"accession": c.get("accession"),
"chrom": c.get("chrom"),
"start": c.get("start"),
"end": c.get("end"),
"group": c.get("group"),
"dnase_zscore": round(c.get("dnase_zscore", 0), 2),
"h3k4me3_zscore": round(c.get("h3k4me3_zscore", 0), 2),
"h3k27ac_zscore": round(c.get("h3k27ac_zscore", 0), 2),
"ctcf_zscore": round(c.get("ctcf_zscore", 0), 2),
})
df = pd.DataFrame(records)
if len(df):
print(df["group"].value_counts().to_string())
return df
# cCREs in the TP53 locus (GRCh38)
df_ccres = search_ccres_by_region("chr17", 7_661_779, 7_887_538)
print(f"\nTotal cCREs: {len(df_ccres)}")
print(df_ccres.sort_values("h3k27ac_zscore", ascending=False).head(5).to_string(index=False))
Query 4: Gene cCRE Lookup — Find cCREs Near a Gene
Retrieve cCREs in the vicinity of a specific gene using SCREEN's gene-centric endpoint.
import requests, pandas as pd
SCREEN_BASE = "https://api.encodeproject.org/screen"
ENCODE_BASE = "https://www.encodeproject.org"
def get_gene_ccres(gene_symbol, assembly="GRCh38", window_kb=50):
"""
Get cCREs near a gene using ENCODE SCREEN gene search.
Returns: pd.DataFrame of nearby cCREs.
"""
# Step 1: resolve gene to genomic coordinates via ENCODE gene search
r = requests.get(
f"{ENCODE_BASE}/search/",
params={"type": "Gene", "symbol": gene_symbol, "assembly": assembly,
"format": "json", "limit": 1},
timeout=30
)
r.raise_for_status()
genes = r.json().get("@graph", [])
if not genes:
print(f"Gene {gene_symbol} not found in ENCODE")
return pd.DataFrame()
gene = genes[0]
chrom = gene.get("locations", [{}])[0].get("chromosome", "")
start = gene.get("locations", [{}])[0].get("start", 0)
end = gene.get("locations", [{}])[0].get("end", 0)
print(f"{gene_symbol}: {chrom}:{start}-{end}")
# Step 2: search cCREs in expanded window
window = window_kb * 1000
df = search_ccres_by_region(chrom, max(0, start - window), end + window, assembly=assembly)
df["distance_to_gene"] = df.apply(
lambda row: max(0, max(start - row["end"], row["start"] - end)), axis=1
)
return df.sort_values("distance_to_gene")
df_brca1_ccres = get_gene_ccres("BRCA1", window_kb=100)
print(f"\ncCREs near BRCA1: {len(df_brca1_ccres)}")
print(df_brca1_ccres[["accession", "group", "start", "end", "h3k27ac_zscore"]].head(10).to_string(index=False))
Query 5: Biosample Browser — List Available Cell Types and Tissues
Enumerate biosample terms (cell lines, primary tissues, developmental stages) available in ENCODE.
import requests, pandas as pd
BASE = "https://www.encodeproject.org"
def list_biosamples(organism="Homo sapiens", biosample_type=None, limit=200):
"""
List biosamples available in ENCODE.
biosample_type: 'cell line', 'primary cell', 'tissue', 'in vitro differentiated cells'
Returns: pd.DataFrame of biosample terms with experiment counts.
"""
params = {
"type": "Biosample",
"donor.organism.scientific_name": organism,
"format": "json",
"limit": limit,
}
if biosample_type:
params["biosample_ontology.classification"] = biosample_type
r = requests.get(f"{BASE}/search/", params=params, timeout=30)
r.raise_for_status()
data = r.json()
biosamples = data.get("@graph", [])
print(f"Found {data.get('total', 0)} biosamples (showing {len(biosamples)})")
records = []
for bs in biosamples:
records.append({
"biosample_name": bs.get("biosample_ontology", {}).get("term_name", bs.get("accession")),
"classification": bs.get("biosample_ontology", {}).get("classification"),
"tissue": bs.get("biosample_ontology", {}).get("organ_slims", [""])[0] if bs.get("biosample_ontology", {}).get("organ_slims") else "",
"accession": bs.get("accession"),
})
df = pd.DataFrame(records).drop_duplicates("biosample_name").reset_index(drop=True)
print(df["classification"].value_counts().to_string())
return df
df_bs = list_biosamples(organism="Homo sapiens", biosample_type="cell line")
print(f"\nUnique human cell lines: {len(df_bs)}")
print(df_bs.head(10).to_string(index=False))
Query 6: Target/TF Query — All ChIP-seq Experiments for a Transcription Factor
Retrieve all released ChIP-seq experiments for a given TF across all available cell types.
import requests, pandas as pd
BASE = "https://www.encodeproject.org"
def get_tf_experiments(tf_name, assay="TF ChIP-seq", assembly="GRCh38", limit=200):
"""
Find all ENCODE experiments for a given transcription factor.
Returns: pd.DataFrame of experiments sorted by biosample.
"""
params = {
"type": "Experiment",
"assay_title": assay,
"target.label": tf_name,
"files.assembly": assembly,
"status": "released",
"format": "json",
"limit": limit,
}
r = requests.get(f"{BASE}/search/", params=params, timeout=30)
r.raise_for_status()
data = r.json()
total = data.get("total", 0)
print(f"{tf_name} {assay}: {total} total experiments (assembly {assembly})")
records = []
for exp in data.get("@graph", []):
records.append({
"accession": exp.get("accession"),
"biosample": exp.get("biosample_summary"),
"lab": exp.get("lab", {}).get("title", ""),
"date_released": exp.get("date_released", ""),
})
df = pd.DataFrame(records).sort_values("biosample")
print(f"Showing {len(df)} experiments across {df['biosample'].nunique()} cell types/tissues")
print(df.head(10).to_string(index=False))
return df
df_tp53 = get_tf_experiments("TP53", assay="TF ChIP-seq")
Query 7: Peak Set Retrieval — Get Optimal Peak Set for an Experiment
Retrieve the optimal replicated peak set (IDR-filtered or pooled) from an experiment.
import requests, pandas as pd
BASE = "https://www.encodeproject.org"
def get_optimal_peaks(accession, assembly="GRCh38"):
"""
Retrieve the optimal peak file (IDR-filtered or pooled) from a ChIP-seq experiment.
Returns: dict with file accession, download URL, and file metadata.
"""
r = requests.get(f"{BASE}/experiments/{accession}/?format=json", timeout=30)
r.raise_for_status()
exp = r.json()
# Preferred output types in order of preference
preferred = ["IDR thresholded peaks", "optimal IDR thresholded peaks",
"peaks", "replicated peaks"]
matching = []
for f in exp.get("files", []):
if (f.get("file_format") == "bed"
and f.get("assembly") == assembly
and f.get("status") == "released"
and f.get("output_type") in preferred):
matching.append({
"file_accession": f.get("accession"),
"output_type": f.get("output_type"),
"file_size_mb": round(f.get("file_size", 0) / 1e6, 2),
"download_url": BASE + f.get("href", ""),
"biological_replicates": f.get("biological_replicates"),
})
if not matching:
print(f"No peak BED files found for {accession} ({assembly})")
return None
# Pick the highest-preference output type
for pref in preferred:
for f in matching:
if f["output_type"] == pref:
print(f"Selected: {f['file_accession']} ({f['output_type']}, {f['file_size_mb']} MB)")
print(f"URL: {f['download_url']}")
return f
best = matching[0]
print(f"Fallback: {best['file_accession']} ({best['output_type']})")
return best
peak_file = get_optimal_peaks("ENCSR000AKC", assembly="GRCh38")
Key Concepts
ENCODE Data Tiers and File Hierarchy
Each ENCODE experiment has multiple associated files in a hierarchy:
Experiment (e.g., ENCSR000AKC: CTCF ChIP-seq in K562)
├── Raw data: FASTQ files (reads, per replicate)
├── Alignments: BAM files (per replicate, GRCh38)
├── Signal tracks: bigWig (fold-change over control, p-value signal)
└── Peak calls (BED):
├── Replicate 1 peaks (narrow/broad)
├── Replicate 2 peaks
├── Pooled peaks
├── IDR thresholded peaks ← optimal for most analyses
└── Optimal IDR thresholded peaks ← preferred default
For most downstream analyses, use IDR thresholded peaks or optimal IDR thresholded peaks — these are the reproducibility-filtered, high-confidence peak sets.
cCRE Classification System
ENCODE SCREEN classifies cCREs into five functional groups based on epigenomic signal z-scores:
| Group | Full Name | Signals | Interpretation |
|---|---|---|---|
| PLS | Promoter-Like Sequence | High DNase + High H3K4me3 + High H3K27ac | Active promoter region |
| pELS | Proximal Enhancer-Like Sequence | High DNase + High H3K27ac, ≤2 kb from TSS | Proximal enhancer |
| dELS | Distal Enhancer-Like Sequence | High DNase + High H3K27ac, >2 kb from TSS | Distal enhancer |
| DNase-H3K4me3 | — | High DNase + High H3K4me3, low H3K27ac | Unusual promoter signature |
| CTCF-only | — | High CTCF, low H3K27ac | Insulator/boundary element |
Z-scores >1.64 (p < 0.05) are considered significant in each signal category.
Common Workflows
Workflow 1: Download TF Peak Files for a Cell Type
Goal: Retrieve CTCF ChIP-seq peak BED files for a specific cell line and load them into a DataFrame for variant annotation.
import requests, pandas as pd, time, gzip, io
BASE = "https://www.encodeproject.org"
def download_tf_peaks_for_cell_type(tf_name, biosample, assembly="GRCh38"):
"""Find and download IDR peak BED for a TF in a specific cell type."""
# Step 1: find experiments
params = {
"type": "Experiment",
"assay_title": "TF ChIP-seq",
"target.label": tf_name,
"biosample_ontology.term_name": biosample, # filter by ontology term, not the freetext `biosample_summary`
"files.assembly": assembly,
"status": "released",
"format": "json",
"limit": 5,
}
r = requests.get(f"{BASE}/search/", params=params, timeout=30)
r.raise_for_status()
exps = r.json().get("@graph", [])
if not exps:
print(f"No {tf_name} experiments found for {biosample}")
return None
accession = exps[0]["accession"]
print(f"Using experiment {accession}")
# Step 2: find optimal peak file
peak = get_optimal_peaks(accession, assembly=assembly)
if not peak:
return None
# Step 3: download and parse BED
r2 = requests.get(peak["download_url"], timeout=120, stream=True)
r2.raise_for_status()
content = r2.content
if peak["download_url"].endswith(".gz"):
content = gzip.decompress(content)
lines = content.decode("utf-8").strip().split("\n")
rows = [l.split("\t") for l in lines if l and not l.startswith("#")]
# Narrow peak BED6+4 columns
cols = ["chrom", "start", "end", "name", "score", "strand",
"signalValue", "pValue", "qValue", "peak"]
df = pd.DataFrame(rows, columns=cols[:len(rows[0])] if rows else cols)
df["start"] = pd.to_numeric(df["start"])
df["end"] = pd.to_numeric(df["end"])
print(f"Loaded {len(df):,} peaks for {tf_name} in {biosample} ({assembly})")
print(f"Genome coverage: {(df['end'] - df['start']).sum() / 1e6:.1f} Mb")
return df
df_peaks = download_tf_peaks_for_cell_type("CTCF", "K562")
if df_peaks is not None:
df_peaks.to_csv("CTCF_K562_peaks.bed", sep="\t", index=False, header=False)
print("Saved CTCF_K562_peaks.bed")
Workflow 2: cCRE Category Bar Chart for a Genomic Region
Goal: Query SCREEN for cCREs in a disease locus and visualize the distribution of regulatory element types.
import requests, pandas as pd
import matplotlib.pyplot as plt
SCREEN_BASE = "https://api.encodeproject.org/screen"
def ccre_category_chart(chrom, start, end, assembly="GRCh38", label="Region"):
"""Query SCREEN cCREs and plot category distribution."""
payload = {"assembly": assembly, "coord_chrom": chrom,
"coord_start": start, "coord_end": end, "limit": 1000}
r = requests.post(f"{SCREEN_BASE}/search/", json=payload, timeout=30)
r.raise_for_status()
data = r.json()
ccres = data.get("results", data.get("cCREs", []))
if not ccres:
print(f"No cCREs found in {chrom}:{start}-{end}")
return pd.DataFrame()
df = pd.DataFrame(ccres)
print(f"cCREs in {chrom}:{start}-{end}: {len(df)}")
# Count by group
group_counts = df["group"].value_counts()
group_colors = {
"PLS": "#d62728", # red — promoters
"pELS": "#ff7f0e", # orange — proximal enhancers
"dELS": "#1f77b4", # blue — distal enhancers
"DNase-H3K4me3": "#9467bd", # purple
"CTCF-only": "#2ca02c", # green — insulators
}
colors = [group_colors.get(g, "#aec7e8") for g in group_counts.index]
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
# Bar chart of category counts
axes[0].bar(group_counts.index, group_counts.values, color=colors, edgecolor="black")
axes[0].set_xlabel("cCRE Category")
axes[0].set_ylabel("Count")
axes[0].set_title(f"cCRE Categories — {label}\n({chrom}:{start}-{end})")
for i, (cat, cnt) in enumerate(group_counts.items()):
axes[0].text(i, cnt + 0.3, str(cnt), ha="center", va="bottom", fontsize=9)
# H3K27ac z-score distribution by category
if "h3k27ac_zscore" in df.columns:
groups_present = [g for g in ["PLS", "pELS", "dELS"] if g in df["group"].values]
data_to_plot = [df.loc[df["group"] == g, "h3k27ac_zscore"].dropna().tolist()
for g in groups_present]
axes[1].boxplot(data_to_plot, labels=groups_present,
patch_artist=True,
boxprops=dict(facecolor="lightsteelblue"))
axes[1].set_xlabel("cCRE Group")
axes[1].set_ylabel("H3K27ac Z-score")
axes[1].set_title("H3K27ac Signal Strength by cCRE Group")
axes[1].axhline(1.64, color="red", linestyle="--", label="p<0.05 threshold")
axes[1].legend()
plt.tight_layout()
plt.savefig("ccre_category_chart.png", dpi=150, bbox_inches="tight")
print(f"Saved ccre_category_chart.png")
print(group_counts.to_string())
return df
# BRCA1/BRCA2 locus — chr17 region
df_ccres = ccre_category_chart("chr17", 43_044_295, 43_170_000, label="BRCA1 locus")
Workflow 3: Build a TF Peak Atlas Across Multiple Cell Types
Goal: Systematically collect peak BED accessions for one TF across all available cell types for a comparative regulatory analysis.
import requests, pandas as pd, time
BASE = "https://www.encodeproject.org"
def build_tf_peak_atlas(tf_name, assembly="GRCh38", max_experiments=50):
"""
Collect peak file metadata for a TF across all available cell types.
Returns: pd.DataFrame with one row per experiment (download URL included).
"""
params = {
"type": "Experiment",
"assay_title": "TF ChIP-seq",
"target.label": tf_name,
"files.assembly": assembly,
"status": "released",
"format": "json",
"limit": max_experiments,
}
r = requests.get(f"{BASE}/search/", params=params, timeout=30)
r.raise_for_status()
exps = r.json().get("@graph", [])
print(f"{tf_name}: {len(exps)} experiments across {assembly}")
atlas_records = []
for exp in exps:
acc = exp.get("accession")
peak = get_optimal_peaks(acc, assembly=assembly)
if peak:
atlas_records.append({
"experiment": acc,
"biosample": exp.get("biosample_summary", ""),
"lab": exp.get("lab", {}).get("title", ""),
"file_accession": peak["file_accession"],
"output_type": peak["output_type"],
"file_size_mb": peak["file_size_mb"],
"download_url": peak["download_url"],
})
time.sleep(0.3)
df = pd.DataFrame(atlas_records)
print(f"Peak files found: {len(df)} / {len(exps)} experiments")
df.to_csv(f"{tf_name}_peak_atlas_{assembly}.csv", index=False)
print(f"Saved {tf_name}_peak_atlas_{assembly}.csv")
return df
# Collect CTCF peak atlas
df_atlas = build_tf_peak_atlas("CTCF", max_experiments=20)
print(f"\nCell types covered: {df_atlas['biosample'].nunique()}")
Key Parameters
| Parameter | Endpoint / Function | Default | Range / Options | Effect |
|---|---|---|---|---|
type | All search endpoints | required | "Experiment", "File", "Gene", "Biosample" | Result type to search |
assay_title | Experiment search | — | "TF ChIP-seq", "ATAC-seq", "DNase-seq", "Histone ChIP-seq", "RNA-seq" | Filter by assay type |
target.label | Experiment search | — | TF name string, e.g. "CTCF", "TP53" | Filter by target protein |
biosample_summary | Experiment search | — | Cell type string, e.g. "K562", "HeLa-S3" | Filter by biosample |
files.assembly | Experiment search | — | "GRCh38", "GRCh37", "mm10" | Filter by genome assembly |
output_type | File search | — | "peaks", "IDR thresholded peaks", "signal", "alignments" | Filter by file output type |
status | All search endpoints | — | "released", "in progress", "archived" | Filter by release status |
limit | All search endpoints | 25 | 1–500 | Max results per page |
coord_chrom | SCREEN cCRE search | required | chromosome string, e.g. "chr17" | Chromosome for region query |
coord_start | SCREEN cCRE search | required | integer | Region start coordinate |
coord_end | SCREEN cCRE search | required | integer | Region end coordinate |
Best Practices
-
Use IDR thresholded peaks for reproducible analyses: Raw replicate peaks contain many false positives. Always prefer
"output_type": "IDR thresholded peaks"or"optimal IDR thresholded peaks"for any variant annotation or motif analysis. -
Filter by
status=releasedand specifyassembly: ENCODE stores data for multiple genome builds. Always includefiles.assembly=GRCh38(or your target assembly) to avoid mixing coordinate systems from archived experiments. -
Add
time.sleep(0.5)in loops over experiments: The ENCODE Portal does not publish a hard rate limit, but aggressive sequential requests will trigger connection resets. Polite delays prevent this in atlas-building workflows. -
Prefer the SCREEN API for cCRE queries, not the Portal search: The ENCODE Portal search can find cCRE-related files, but the dedicated SCREEN API (
api.encodeproject.org/screen) is purpose-built for region-based cCRE lookup and returns structured z-score data. -
Check
biosample_summaryvsbiosample_ontology.term_name: Thebiosample_summaryfield (used in search) sometimes differs from the official ontology term name. If a biosample search returns zero results, try browsing ENCODE to find the exact summary string used.
Common Recipes
Recipe: Find All ENCODE Assay Types Available
When to use: Explore what assay types are available before planning a targeted search.
import requests
BASE = "https://www.encodeproject.org"
r = requests.get(
f"{BASE}/search/",
params={"type": "Experiment", "status": "released", "format": "json",
"limit": 0, "field": "assay_title"},
timeout=30
)
r.raise_for_status()
# Facets contain aggregated counts per assay type
for facet in r.json().get("facets", []):
if facet.get("field") == "assay_title":
for term in sorted(facet.get("terms", []), key=lambda x: -x["doc_count"])[:20]:
print(f" {term['doc_count']:6d} {term['key']}")
Recipe: Get Experiment Metadata by Accession
When to use: Retrieve full details for a known ENCODE accession (e.g., from a published paper).
import requests, json
BASE = "https://www.encodeproject.org"
def get_experiment_metadata(accession):
r = requests.get(f"{BASE}/experiments/{accession}/?format=json", timeout=30)
r.raise_for_status()
exp = r.json()
print(f"Accession : {exp['accession']}")
print(f"Assay : {exp.get('assay_title')}")
print(f"Biosample : {exp.get('biosample_summary')}")
print(f"Target : {exp.get('target', {}).get('label', 'N/A')}")
print(f"Lab : {exp.get('lab', {}).get('title')}")
print(f"Files : {len(exp.get('files', []))}")
print(f"Released : {exp.get('date_released')}")
return exp
exp = get_experiment_metadata("ENCSR000AKC")
Recipe: List bigWig Signal Files for Visualization
When to use: Download signal bigWig tracks to load into IGV or UCSC Genome Browser.
import requests, pandas as pd
BASE = "https://www.encodeproject.org"
def get_signal_bigwigs(accession, assembly="GRCh38"):
"""Get fold-change and p-value signal bigWig files for an experiment."""
params = {
"type": "File",
"dataset": f"/experiments/{accession}/",
"file_format": "bigWig",
"output_type": "fold change over control",
"assembly": assembly,
"status": "released",
"format": "json",
"limit": 10,
}
r = requests.get(f"{BASE}/search/", params=params, timeout=30)
r.raise_for_status()
files = r.json().get("@graph", [])
print(f"Found {len(files)} bigWig signal files for {accession}")
for f in files:
print(f" {f['accession']} {f['output_type']} {f.get('biological_replicates')} "
f"{round(f.get('file_size', 0)/1e6, 1)} MB")
print(f" URL: {BASE}{f['href']}")
return files
bigwigs = get_signal_bigwigs("ENCSR000AKC")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
HTTP 404 on experiment endpoint | Accession does not exist or was revoked | Verify accession on https://www.encodeproject.org; check status field in search results |
Empty @graph in search results | Overly restrictive filters or no matching data | Relax one filter at a time; check total field to see if data exists before pagination |
HTTP 503 or connection timeout | Server overload from rapid sequential requests | Add time.sleep(0.5) between requests; retry with exponential backoff |
cCRE search returns [] | SCREEN API endpoint path changed or region on non-canonical chromosome | Verify API URL; use canonical chromosomes (chr1–22, chrX, chrY) only |
| bigWig/BED files not found | Assembly mismatch (GRCh37 vs GRCh38) | Always include files.assembly in queries; verify the experiment has files in your target assembly |
target.label filter returns no results | Exact label mismatch (case-sensitive) | Browse ENCODE to find the exact string (e.g., "eGFP-TP53" vs "TP53") |
| File download fails mid-transfer | Large file + network timeout | Use stream=True in requests with chunked writing; increase timeout to 300 |
Related Skills
regulomedb-database— Pre-computed regulatory scores for variants integrating ENCODE ChIP-seq, DNase-seq, eQTL, and motif datagwas-database— NHGRI-EBI GWAS Catalog for published SNP-trait associations; combine with ENCODE peaks for functional annotation of GWAS hitsmacs3-peak-calling— Call peaks from your own ChIP-seq or ATAC-seq BAM files; use ENCODE peaks as reference comparison setsdeeptools-ngs-analysis— Generate bigWig coverage tracks, correlation heatmaps, and profile plots from your own aligned BAM files
References
- ENCODE Portal REST API Documentation — Full API reference with query examples
- ENCODE SCREEN Browser — Interactive cCRE browser and documentation
- ENCODE Project Consortium. "An integrated encyclopedia of DNA elements in the human genome." Nature 489: 57–74 (2012). https://doi.org/10.1038/nature11247
- ENCODE Project Consortium et al. "Perspectives on ENCODE." Nature 583: 693–698 (2020). https://doi.org/10.1038/s41586-020-2449-8
- Moore JE et al. "Expanded encyclopaedias of DNA elements in the human and mouse genomes." Nature 583: 699–710 (2020). https://doi.org/10.1038/s41586-020-2493-4
Ensembl Genome Database
Overview
Ensembl is a comprehensive genome annotation database covering 300+ vertebrate and non-vertebrate species. The Ensembl REST API provides programmatic access to gene models, transcript/protein sequences, variant annotations, cross-references, regulatory features, and comparative genomics without requiring any login or API key.
When to Use
- Retrieving official gene and transcript annotations (stable IDs, biotype, genomic coordinates) for human or model organism genes
- Converting between gene identifier namespaces (HGNC symbol ↔ Ensembl ID ↔ RefSeq ↔ UniProt)
- Fetching genomic or cDNA/CDS/protein sequences for a gene or transcript
- Looking up variant consequences and functional impact (VEP) for a list of SNPs
- Querying regulatory features (promoters, enhancers, CTCF sites) in a genomic region
- Performing comparative genomics queries (orthologs, paralogs, gene trees) across species
- For local offline access to large genomic annotations, use
pyensemblinstead - For pathway and metabolic annotations, use
kegg-databaseorreactome-databaseinstead
Prerequisites
- Python packages:
requests - Data requirements: gene symbols, Ensembl stable IDs (ENSG…/ENST…/ENSP…), or genomic coordinates
- Environment: internet connection required; no API key needed
- Rate limits: max ~15 requests/second; use
expand=1and batch endpoints to minimize calls
pip install requests
Quick Start
import requests
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
def ensembl_get(endpoint, params=None):
r = requests.get(f"{BASE}{endpoint}", headers=HEADERS, params=params)
r.raise_for_status()
return r.json()
# Look up human BRCA1
gene = ensembl_get("/lookup/symbol/homo_sapiens/BRCA1", params={"expand": 1})
print(f"ID: {gene['id']}, Chr: {gene['seq_region_name']}:{gene['start']}-{gene['end']}")
print(f"Transcripts: {len(gene.get('Transcript', []))}")
Core API
Query 1: Gene Lookup by Symbol or Stable ID
Retrieve gene metadata from a gene symbol or Ensembl stable ID.
import requests
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
# By gene symbol
r = requests.get(
f"{BASE}/lookup/symbol/homo_sapiens/TP53",
headers=HEADERS,
params={"expand": 1}
)
gene = r.json()
print(f"Ensembl ID : {gene['id']}")
print(f"Location : {gene['seq_region_name']}:{gene['start']}-{gene['end']} ({gene['strand']})")
print(f"Biotype : {gene['biotype']}")
print(f"Transcripts: {len(gene.get('Transcript', []))}")
# By stable ID (works for genes, transcripts, proteins)
r = requests.get(
f"{BASE}/lookup/id/ENSG00000141510",
headers=HEADERS,
params={"expand": 0}
)
obj = r.json()
print(f"Symbol: {obj.get('display_name')}, Species: {obj.get('species')}")
Query 2: Batch Lookup
Retrieve information for multiple IDs in one call (POST endpoint).
import requests, json
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
# Batch lookup by symbols
symbols = ["BRCA1", "BRCA2", "TP53", "EGFR", "MYC"]
r = requests.post(
f"{BASE}/lookup/symbol/homo_sapiens",
headers=HEADERS,
data=json.dumps({"symbols": symbols})
)
results = r.json()
for sym, data in results.items():
if data:
print(f"{sym}: {data['id']} ({data['seq_region_name']}:{data['start']}-{data['end']})")
Query 3: Sequence Retrieval
Fetch genomic, cDNA, CDS, or protein sequences.
import requests
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "text/plain"}
# Protein sequence for canonical transcript
r = requests.get(
f"{BASE}/sequence/id/ENST00000269305",
headers=HEADERS,
params={"type": "protein"}
)
seq = r.text
print(f"Protein sequence ({len(seq)} aa): {seq[:60]}...")
# Genomic region sequence
HEADERS_JSON = {"Content-Type": "application/json"}
r = requests.get(
f"{BASE}/sequence/region/human/17:43044295..43125364",
headers=HEADERS_JSON,
params={"coord_system_version": "GRCh38"}
)
result = r.json()
print(f"Retrieved {len(result['seq'])} bp of genomic sequence")
Query 4: Cross-References (ID Mapping)
Map Ensembl IDs to external database identifiers.
import requests
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
# All xrefs for a gene
r = requests.get(
f"{BASE}/xrefs/id/ENSG00000141510",
headers=HEADERS
)
xrefs = r.json()
# Group by database
from collections import defaultdict
by_db = defaultdict(list)
for x in xrefs:
by_db[x["dbname"]].append(x["primary_id"])
for db in ["HGNC", "RefSeq_gene_name", "Uniprot_gn", "MIM_gene"]:
if db in by_db:
print(f"{db}: {by_db[db]}")
Query 5: Variant Consequence Annotation (VEP)
Predict functional consequences of variants via REST VEP endpoint.
import requests, json
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
# Annotate a list of hgvs notations
variants = ["17:g.43094692C>T", "13:g.32929387C>T"]
r = requests.post(
f"{BASE}/vep/human/hgvs",
headers=HEADERS,
data=json.dumps({"hgvs_notations": variants})
)
for v in r.json():
print(f"\nVariant: {v.get('input')}")
for tc in v.get("transcript_consequences", [])[:2]:
print(f" Gene: {tc.get('gene_symbol')}, Impact: {tc.get('impact')}, Consequence: {tc.get('consequence_terms')}")
# Annotate by rsID
r = requests.get(
f"{BASE}/vep/human/id/rs699",
headers=HEADERS
)
v = r.json()[0]
print(f"rsID rs699 in gene: {v['transcript_consequences'][0]['gene_symbol']}")
print(f"Consequence: {v['transcript_consequences'][0]['consequence_terms']}")
Query 6: Regulatory Features
Query regulatory build features in a genomic region.
import requests
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
# Regulatory features in BRCA1 region
r = requests.get(
f"{BASE}/overlap/region/human/17:43044000-43126000",
headers=HEADERS,
params={"feature": "regulatory"}
)
features = r.json()
print(f"Found {len(features)} regulatory features")
for f in features[:5]:
print(f" {f.get('feature_type')}: {f.get('start')}-{f.get('end')} ({f.get('description', 'n/a')})")
Query 7: Comparative Genomics (Orthologs / Gene Trees)
Find orthologs and paralogs across species.
import requests
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
# Get mouse ortholog for human TP53
r = requests.get(
f"{BASE}/homology/symbol/human/TP53",
headers=HEADERS,
params={"target_species": "mus_musculus", "type": "orthologues"}
)
data = r.json()
for homo in data["data"][0]["homologies"][:3]:
tgt = homo["target"]
print(f"Mouse ortholog: {tgt['id']} ({tgt.get('perc_id', 'n/a')}% identity)")
Key Concepts
Stable IDs and Versioning
Ensembl uses stable IDs with optional version suffixes (e.g., ENSG00000141510.17). Genes (ENSG), transcripts (ENST), proteins (ENSP), and exons (ENSE) each have their own prefix. IDs are preserved across releases when possible; retired IDs can still be resolved via the archive API.
Assembly Versions
Human genome: GRCh38 (current) and GRCh37 (legacy, via grch37.rest.ensembl.org). Always specify which assembly your coordinates belong to when making region-based queries.
Common Workflows
Workflow 1: Gene-to-Protein Information Pipeline
Goal: Retrieve all key annotations for a gene list — coordinates, transcripts, xrefs, and canonical protein sequence.
import requests, json, time
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
def batch_lookup(symbols, species="homo_sapiens"):
r = requests.post(
f"{BASE}/lookup/symbol/{species}",
headers=HEADERS,
data=json.dumps({"symbols": symbols, "expand": 1})
)
return r.json()
def canonical_transcript(gene_data):
"""Return the ID of the canonical (longest CDS) transcript."""
transcripts = gene_data.get("Transcript", [])
coding = [t for t in transcripts if t.get("biotype") == "protein_coding"]
if not coding:
return None
return max(coding, key=lambda t: t.get("Translation", {}).get("length", 0))
genes = ["BRCA1", "BRCA2", "TP53"]
lookup = batch_lookup(genes)
for sym in genes:
g = lookup.get(sym)
if not g:
print(f"{sym}: not found")
continue
canon = canonical_transcript(g)
print(f"\n{sym} ({g['id']})")
print(f" Location: {g['seq_region_name']}:{g['start']}-{g['end']}")
if canon:
prot_len = canon.get("Translation", {}).get("length", "n/a")
print(f" Canonical transcript: {canon['id']} ({prot_len} aa)")
time.sleep(0.1) # be polite
Workflow 2: Variant Annotation Pipeline
Goal: Annotate a VCF-style variant list with gene, consequence, and impact.
import requests, json, pandas as pd
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
# Input: list of hgvs notations
hgvs_list = [
"17:g.43094692C>T",
"17:g.43063873A>G",
"13:g.32929387C>T",
]
# Annotate in batches of 200
def vep_batch(hgvs_batch):
r = requests.post(
f"{BASE}/vep/human/hgvs",
headers=HEADERS,
data=json.dumps({"hgvs_notations": hgvs_batch})
)
r.raise_for_status()
return r.json()
records = []
for ann in vep_batch(hgvs_list):
for tc in ann.get("transcript_consequences", []):
if tc.get("canonical") == 1:
records.append({
"variant": ann["input"],
"gene": tc.get("gene_symbol"),
"consequence": ",".join(tc.get("consequence_terms", [])),
"impact": tc.get("impact"),
"biotype": tc.get("biotype"),
})
df = pd.DataFrame(records)
print(df.to_string(index=False))
df.to_csv("vep_results.csv", index=False)
print(f"\nSaved {len(df)} variant annotations → vep_results.csv")
Key Parameters
| Parameter | Module | Default | Range / Options | Effect |
|---|---|---|---|---|
expand | Lookup | 0 | 0 or 1 | Include nested transcripts/translations |
type | Sequence | "genomic" | "genomic", "cDNA", "CDS", "protein" | Sequence type to return |
target_species | Homology | None | Species name or taxon ID | Filter homologs to target species |
feature | Overlap | required | "gene", "transcript", "regulatory", "variation" | Feature type to retrieve |
coord_system_version | Region | "GRCh38" | "GRCh38", "GRCh37" | Genome assembly |
content_type | All | via header | "application/json", "text/plain" | Response format |
Best Practices
-
Use batch endpoints: POST
/lookup/symbol/{species}and POST/vep/human/hgvsaccept up to 1000 IDs; single-ID GET requests in a loop will hit rate limits quickly. -
Pin assembly version: For region-based queries always specify
coord_system_version=GRCh38(or usegrch37.rest.ensembl.orgfor legacy coordinates) to avoid silent mismatch errors. -
Cache responses: Gene metadata rarely changes between Ensembl releases; cache results to disk (
joblib.Memory) to avoid redundant API calls during development.from joblib import Memory mem = Memory("cache/", verbose=0) cached_lookup = mem.cache(batch_lookup) -
Use
expand=0for metadata: When you only need gene coordinates and biotype (not transcript details), keepexpand=0for smaller payloads and faster responses. -
Check canonical flag in VEP: VEP returns consequences for all overlapping transcripts; filter on
tc.get("canonical") == 1to get the biologically most relevant consequence per variant.
Common Recipes
Recipe: Symbol → Ensembl ID Mapping Table
When to use: Build a lookup table from gene symbols to Ensembl IDs for downstream analysis.
import requests, json, pandas as pd
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
symbols = ["EGFR", "KRAS", "BRAF", "PIK3CA", "PTEN", "AKT1", "MYC", "RB1"]
r = requests.post(
f"{BASE}/lookup/symbol/homo_sapiens",
headers=HEADERS,
data=json.dumps({"symbols": symbols})
)
data = r.json()
rows = [{"symbol": s, "ensembl_id": d["id"] if d else None,
"chrom": d["seq_region_name"] if d else None} for s, d in data.items()]
df = pd.DataFrame(rows)
df.to_csv("symbol_to_ensembl.csv", index=False)
print(df.to_string(index=False))
Recipe: Region Gene Overlap
When to use: Find all genes overlapping a genomic interval (e.g., a GWAS locus).
import requests, pandas as pd
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
chrom, start, end = "17", 43044295, 43125364
r = requests.get(
f"{BASE}/overlap/region/human/{chrom}:{start}-{end}",
headers=HEADERS,
params={"feature": "gene", "biotype": "protein_coding"}
)
genes = r.json()
df = pd.DataFrame([{
"id": g["id"], "name": g.get("external_name"),
"start": g["start"], "end": g["end"], "strand": g["strand"]
} for g in genes])
print(df.to_string(index=False))
print(f"\n{len(df)} protein-coding genes in region")
Recipe: Species List
When to use: Check which species are available in Ensembl before querying.
import requests
BASE = "https://rest.ensembl.org"
HEADERS = {"Content-Type": "application/json"}
r = requests.get(f"{BASE}/info/species", headers=HEADERS)
species_list = r.json()["species"]
print(f"Total species: {len(species_list)}")
vertebrates = [s for s in species_list if s.get("division") == "EnsemblVertebrates"]
print(f"Vertebrates: {len(vertebrates)}")
for s in vertebrates[:5]:
print(f" {s['common_name']} ({s['name']}): {s['assembly']}")
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
HTTP 429 Too Many Requests | Exceeding ~15 req/s rate limit | Add time.sleep(0.1) between requests; use batch POST endpoints |
HTTP 400 Bad Request on VEP | Malformed HGVS notation | Verify format: chr:g.posREF>ALT (e.g., 17:g.43094692C>T) |
Gene not found | Gene symbol not in Ensembl | Try alternative symbol; check species name (use homo_sapiens not human for symbols) |
| Region query returns wrong genes | Assembly mismatch | Set coord_system_version=GRCh38 or use grch37.rest.ensembl.org |
| Old ID not resolving | Retired Ensembl ID | Query GET /archive/id/{id} to get current mapping |
HTTP 503 Service Unavailable | Server maintenance | Retry after a few minutes; check Ensembl status at status.ensembl.org |
Related Skills
gget-genomic-databases— CLI/Python wrapper covering Ensembl + 20 other databases; use for quick lookups without raw API codebiopython-molecular-biology— Biopython'sEntrezmodule for NCBI databases (alternative for RefSeq/GenBank queries)kegg-database— Pathway/metabolic annotations for the same gene setreactome-database— Pathway enrichment and hierarchy queries
References
- Ensembl REST API documentation — Interactive API explorer and endpoint reference
- Ensembl Help & Documentation — REST API overview
- Ensembl stable IDs guide — ID versioning policy
- VEP documentation — Variant Effect Predictor full reference
Sponsors
Details
- Marketplace
- jaechang-hits/SciAgent-Skills
- Author
- jaechang-hits
- Category
- Scientific Research
- License
- CC-BY-4.0
- Keywords
- life-sciences, bioinformatics, drug-discovery, genomics, scientific-computing