Modern Python 3.12+ development with uv, ruff, pydantic, FastAPI, Django and production-ready practices

version
1.0.0

Install

/plugin install python@dot-claude

Run it inside Claude Code after adding the marketplace with /plugin marketplace add pproenca/dot-claude-old.

Contents

Discovered from source

Skills 3

Python Performance & Concurrency

Profiling, optimization, and async patterns for Python.

Before Writing Code

  1. Read references/pythonic-style.md for style conventions
  2. Check Python version: 3.13+ enables free-threaded concurrency options
  3. Profile before optimizing - never guess at bottlenecks

Reference Files

TopicWhen to LoadFile
Pythonic styleBefore generating code../references/pythonic-style.md
Semaphores, locks, producers/consumersAdvanced asyncreferences/async-advanced.md
Database, memory, NumPy optimizationDeep optimizationreferences/optimization-advanced.md

Profiling First

cProfile (CPU)

python -m cProfile -o output.prof script.py
python -m pstats output.prof  # Interactive analysis
import cProfile
import pstats

with cProfile.Profile() as pr:
    main()

stats = pstats.Stats(pr)
stats.sort_stats("cumulative").print_stats(10)

line_profiler (Line-by-Line)

uv add line-profiler
kernprof -l -v script.py  # Requires @profile decorator

py-spy (Production)

py-spy record -o profile.svg -- python script.py
py-spy top --pid 12345  # Live profiling

memory_profiler

uv add memory-profiler
python -m memory_profiler script.py  # Requires @profile decorator

Concurrency Pattern Selection

WorkloadSolution
I/O-bound (network, disk)async/await
CPU-bound (GIL Python)multiprocessing
CPU-bound (nogil 3.13+)threading
Mixedasync + ProcessPoolExecutor

Async/Await Patterns

Basic Async

import asyncio

async def fetch_data(url: str) -> dict:
    await asyncio.sleep(1)  # Simulates I/O
    return {"url": url, "data": "result"}

async def main():
    result = await fetch_data("https://api.example.com")
    print(result)

asyncio.run(main())

Concurrent Execution

async def fetch_all(urls: list[str]) -> list[dict]:
    # Concurrent I/O eliminates sequential waiting: 10 URLs at 100ms each
    # complete in ~100ms total, not 1000ms. Critical for API aggregation.
    tasks = [fetch_data(url) for url in urls]
    return await asyncio.gather(*tasks)

Error Handling

async def safe_fetch(url: str) -> dict | None:
    try:
        return await fetch_data(url)
    except Exception as e:
        print(f"Error: {e}")
        return None

async def fetch_with_errors(urls: list[str]):
    results = await asyncio.gather(
        *[safe_fetch(url) for url in urls],
        return_exceptions=True  # Don't fail on first error
    )
    return [r for r in results if r and not isinstance(r, Exception)]

Timeouts

try:
    result = await asyncio.wait_for(slow_operation(), timeout=5.0)
except asyncio.TimeoutError:
    print("Operation timed out")

Async Pitfalls

# WRONG: Forgetting await
result = async_function()  # Returns coroutine, doesn't execute!

# WRONG: Blocking the event loop
import time
async def bad():
    time.sleep(1)  # Blocks everything!

# CORRECT
async def good():
    await asyncio.sleep(1)  # Non-blocking

# WRONG: Calling async from sync
def sync_func():
    result = await async_func()  # SyntaxError!

# CORRECT
def sync_func():
    result = asyncio.run(async_func())

Optimization Patterns

Data Structures

# Sets use hash tables for O(1) membership testing. For collections
# checked repeatedly or exceeding ~10 items, the overhead of hashing
# pays off vs linear scanning.
if item in items_list:  # O(n) - scans every element
if item in items_set:   # O(1) - hash lookup

# String immutability in Python forces reallocation on every +=.
# For N concatenations, this creates O(n²) total copying.
# join() pre-calculates final size and allocates once.
result = ""
for s in strings:
    result += s  # Creates N intermediate string objects

result = "".join(strings)  # Single allocation, O(n) total

List Comprehensions

# List comprehensions bypass method dispatch overhead (no .append lookup)
# and CPython pre-sizes the result list. For simple transforms, prefer
# comprehensions unless loop body requires complex logic or side effects.
result = []
for i in range(n):
    result.append(i**2)  # Method lookup + call per iteration

result = [i**2 for i in range(n)]  # Optimized bytecode path

Generators for Memory

import sys

list_data = [i for i in range(1_000_000)]  # ~8MB in memory
gen_data = (i for i in range(1_000_000))   # ~100 bytes (lazy evaluation)

# Generators yield values on-demand, preventing memory exhaustion when
# processing data larger than available RAM. Essential for log parsing,
# ETL pipelines, or any single-pass data processing.
def process_large_file(path):
    with open(path) as f:
        for line in f:  # File iterator is itself a generator
            yield process(line)

Caching

from functools import lru_cache

@lru_cache(maxsize=128)
def expensive_computation(n: int) -> int:
    # Cached: subsequent calls with same n return instantly
    return sum(i**2 for i in range(n))

# Clear cache if needed
expensive_computation.cache_clear()

Local Variables

# Python's LEGB resolution requires dictionary lookups for global/nonlocal
# variables, but uses direct array indexing for locals. In tight loops with
# 10k+ iterations, this overhead compounds significantly.
GLOBAL = 100
def slow():
    for i in range(10000):
        x = GLOBAL * i  # Dict lookup each iteration

def fast():
    local = 100  # Captured once at function entry
    for i in range(10000):
        x = local * i  # Direct array index

Multiprocessing (CPU-Bound)

import multiprocessing as mp

def cpu_task(n: int) -> int:
    return sum(i**2 for i in range(n))

if __name__ == "__main__":
    with mp.Pool(4) as pool:
        results = pool.map(cpu_task, [1_000_000] * 4)

Async + Sync Integration

import asyncio
from concurrent.futures import ProcessPoolExecutor

def cpu_bound(n: int) -> int:
    return sum(i**2 for i in range(n))

async def main():
    loop = asyncio.get_running_loop()
    with ProcessPoolExecutor() as pool:
        result = await loop.run_in_executor(pool, cpu_bound, 1_000_000)
    print(result)

asyncio.run(main())

Testing Async Code

import pytest

@pytest.mark.asyncio
async def test_fetch():
    result = await fetch_data("https://api.example.com")
    assert result is not None

@pytest.mark.asyncio
async def test_timeout():
    with pytest.raises(asyncio.TimeoutError):
        await asyncio.wait_for(slow_operation(), timeout=0.1)

Benchmarking

import timeit

# Quick benchmark
time = timeit.timeit(lambda: my_function(), number=1000)
print(f"{time:.4f}s for 1000 runs")

# pytest-benchmark (uv add --dev pytest-benchmark)
def test_performance(benchmark):
    result = benchmark(my_function)
    assert result is not None

Workflow Integration

TaskSkill
Writing async testspython:python-testing
Root cause analysisdebug:systematic
Before claiming donecore:verification

Best Practices

  1. Profile before optimizing - find real bottlenecks
  2. Use async for I/O - network, disk, databases
  3. Use multiprocessing for CPU - heavy computation
  4. Consider nogil (3.13+) - threading for CPU-bound
  5. Use appropriate data structures - set/dict for lookups
  6. Cache expensive operations - lru_cache
  7. Use generators - for large data iteration
  8. Avoid blocking in async - no time.sleep()
  9. Handle cancellation - catch CancelledError
  10. Benchmark changes - prove improvement

Python Project Management

Modern Python project setup and dependency management with uv.

Before Writing Code

  1. Read references/pythonic-style.md for style conventions
  2. Check Python version: pyproject.toml → .python-version → .claude/python-version
  3. If unknown, ask user once and store in .claude/python-version

Reference Files

TopicWhen to LoadFile
Pythonic style, conventionsBefore generating code../references/pythonic-style.md
Version-specific featuresWhen adapting to Python version../references/version-features.md
GitHub Actions, Docker, monorepoCI/CD and advanced workflowsreferences/ci-cd-workflows.md
Full pyproject.toml templatesComplete configuration examplesreferences/pyproject-templates.md

Quick Start

# New project
uv init my-project && cd my-project
uv python pin 3.12
uv add fastapi pydantic
uv add --dev pytest ruff mypy

# Existing project
uv sync              # Install from pyproject.toml
uv sync --all-extras # Include optional deps

Project Structure

Source Layout (Recommended)

my-package/
├── pyproject.toml
├── README.md
├── src/
│   └── my_package/
│       ├── __init__.py
│       ├── core.py
│       └── py.typed
├── tests/
│   └── test_core.py
└── .python-version

Why src/ layout: The src/ layout prevents accidental imports from the source tree during testing. Without it, import my_package might resolve to the local directory instead of the installed package, masking missing __init__.py files or build configuration errors.

Minimal pyproject.toml

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "my-package"
version = "1.0.0"
requires-python = ">=3.10"
dependencies = ["requests>=2.28"]

[project.optional-dependencies]
dev = ["pytest>=7.0", "ruff>=0.1", "mypy>=1.0"]

[tool.setuptools.packages.find]
where = ["src"]

[tool.ruff]
line-length = 100
target-version = "py310"

[tool.ruff.lint]
select = ["E", "F", "I", "UP"]

Dependency Management

Adding Packages

uv add requests              # Add to dependencies
uv add "django>=4.0,<5.0"    # With version constraint
uv add --dev pytest ruff     # Development dependencies
uv add --optional docs sphinx # Optional group
uv add -e ./local-package    # Editable local

Updating Packages

uv add --upgrade requests    # Upgrade specific
uv sync --upgrade            # Upgrade all
uv tree --outdated           # Show outdatable

Locking

uv lock                      # Generate uv.lock
uv lock --upgrade            # Regenerate with latest
uv sync --frozen             # Install exact versions (CI)

Python Version Management

uv python install 3.12       # Install Python
uv python pin 3.12           # Create .python-version
uv python list               # List installed

Running Code

uv run python app.py         # Run script
uv run pytest                # Run tests
uv run ruff check .          # Run linter
uv run --python 3.11 script.py  # Specific version

CLI Entry Points

With Click

# src/my_package/cli.py
import click

@click.group()
@click.version_option()
def cli():
    """My CLI tool."""

@cli.command()
@click.argument("name")
def greet(name: str):
    click.echo(f"Hello, {name}!")

def main():
    cli()
# pyproject.toml
[project.scripts]
my-tool = "my_package.cli:main"
uv sync && uv run my-tool greet World

Building and Publishing

# Build
uv build                     # Creates dist/*.whl and dist/*.tar.gz

# Test on TestPyPI first
uv publish --repository testpypi

# Publish to PyPI
uv publish

API Token Setup

# ~/.pypirc
[pypi]
username = __token__
password = pypi-...your-token...

Publishing Checklist

Before publishing:

  • Tests passing (uv run pytest)
  • Version updated in pyproject.toml
  • CHANGELOG updated
  • Build succeeds (uv build)
  • Install works in clean venv
  • Tested on TestPyPI first

Common Issues

# Wrong Python version
uv python pin 3.12 && uv venv --python 3.12

# Dependency conflict
uv lock --verbose

# Cache issues
uv cache clean

# Lockfile out of sync
uv lock --upgrade

Workflow Integration

TaskSkill
Writing testspython:python-testing
Profiling, asyncpython:python-performance
Before claiming donecore:verification

Best Practices

  1. Use src/ layout for libraries
  2. Pin Python version with .python-version
  3. Commit uv.lock for reproducibility
  4. Use uv run instead of manual activation
  5. Test installation in clean venv before publishing
  6. Use TestPyPI before real PyPI
  7. Automate publishing with GitHub Actions
  8. Keep dependencies minimal
  9. Use optional groups for dev/docs deps
  10. Run uv sync --frozen in CI

Python Testing Patterns

Robust testing with pytest.

Before Writing Tests

  1. Read references/pythonic-style.md for naming and style conventions
  2. TDD workflow: Write failing test first, then implement
  3. Test behavior, not implementation details

Reference Files

TopicWhen to LoadFile
Pythonic styleBefore writing code../references/pythonic-style.md
Async, property-based, factoriesAdvanced patternsreferences/advanced-patterns.md
SQLAlchemy, in-memory DBsDatabase testingreferences/database-testing.md

Basic Tests

import pytest

def test_addition():
    assert 2 + 2 == 4

def test_exception_raised():
    with pytest.raises(ValueError, match="invalid"):
        raise ValueError("invalid input")

Fixtures

@pytest.fixture
def db():
    """
    Provides isolated database connection for test isolation.

    Using in-memory SQLite prevents test pollution across runs and
    guarantees each test starts with known state.
    """
    database = Database("sqlite:///:memory:")
    database.connect()
    yield database
    database.disconnect()

def test_query(db):
    results = db.query("SELECT * FROM users")
    assert len(results) >= 0

# Session scope shares fixture across all tests for expensive resources
# (e.g., one DB schema creation for all tests vs per-test).
@pytest.fixture(scope="session")
def app_config():
    return {"debug": True, "db_url": "sqlite:///:memory:"}

Parametrized Tests

@pytest.mark.parametrize("email,valid", [
    ("user@example.com", True),
    ("invalid.email", False),
    ("", False),
])
def test_email_validation(email, valid):
    assert is_valid_email(email) == valid

# With custom IDs
@pytest.mark.parametrize("value,expected", [
    pytest.param(1, True, id="positive"),
    pytest.param(0, False, id="zero"),
    pytest.param(-1, False, id="negative"),
])
def test_is_positive(value, expected):
    assert (value > 0) == expected

Mocking

from unittest.mock import Mock, patch

def test_api_call():
    mock_response = Mock()
    mock_response.json.return_value = {"id": 1, "name": "Test"}

    with patch("requests.get", return_value=mock_response) as mock_get:
        result = fetch_user(1)

        assert result["id"] == 1
        mock_get.assert_called_once_with("https://api.example.com/users/1")

# Decorator syntax
@patch("module.external_service")
def test_with_mock(mock_service):
    mock_service.return_value = "mocked"
    assert my_function() == "mocked"

Async Testing

import pytest

@pytest.mark.asyncio
async def test_async_fetch():
    result = await fetch_data("https://api.example.com")
    assert result is not None

@pytest.mark.asyncio
async def test_timeout():
    with pytest.raises(asyncio.TimeoutError):
        await asyncio.wait_for(slow_operation(), timeout=0.1)

# Async fixture
@pytest.fixture
async def async_client():
    client = AsyncClient()
    await client.connect()
    yield client
    await client.disconnect()

Test Organization

tests/
├── conftest.py           # Shared fixtures
├── test_unit/
│   ├── test_models.py
│   └── test_utils.py
├── test_integration/
│   └── test_api.py
└── test_e2e/
    └── test_workflows.py

Naming Conventions

# Good: describes behavior and expected outcome
def test_user_creation_with_valid_data_succeeds():
def test_login_fails_with_invalid_password():
def test_api_returns_404_for_missing_resource():

# Bad: vague or non-descriptive
def test_user():
def test_1():

Markers

@pytest.mark.slow
def test_slow_operation():
    ...

@pytest.mark.integration
def test_database():
    ...

@pytest.mark.skip(reason="Not implemented")
def test_future_feature():
    ...

@pytest.mark.xfail(reason="Known bug #123")
def test_known_bug():
    ...

# Run: pytest -m "not slow"

Configuration

# pyproject.toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = ["-v", "--cov=myapp", "--cov-report=term-missing"]
markers = [
    "slow: marks tests as slow",
    "integration: marks integration tests",
]

[tool.coverage.run]
source = ["src"]
omit = ["*/tests/*"]

TDD Workflow

  1. RED: Write failing test
  2. GREEN: Write minimal code to pass
  3. REFACTOR: Improve while keeping tests green
# 1. Write test first
def test_calculate_total():
    cart = Cart([Item(10), Item(20)])
    assert cart.total() == 30

# 2. Run test - it fails (Cart doesn't exist)

# 3. Implement minimal code
class Cart:
    def __init__(self, items):
        self.items = items
    def total(self):
        return sum(item.price for item in self.items)

# 4. Run test - it passes

# 5. Refactor if needed (tests stay green)

Workflow Integration

If the core plugin is installed, use these skills:

TaskSkill
Full TDD methodologycore:tdd
Debug test failuresdebug:systematic
Before claiming donecore:verification

Best Practices

  1. One assertion per test when possible
  2. Descriptive names that explain behavior
  3. Independent tests - no shared mutable state
  4. Use fixtures for setup/teardown
  5. Mock external dependencies appropriately
  6. Parametrize to reduce duplication
  7. Test edge cases and error conditions
  8. Run tests in CI on every commit
  9. Measure coverage but focus on quality
  10. Write tests first (TDD) when possible

Commands 1

Refactor Python File

Refactor the specified Python file to follow modern Python patterns and best practices.

Progress Tracking

Create TodoWrite with each step:

  1. Identify and read target file
  2. Analyze (ruff + mypy + python-expert in parallel)
  3. Propose changes
  4. Apply refactoring
  5. Verify and report

Step 1: Identify and Read Target File

If $ARGUMENTS is provided, use it as the target file path.

Otherwise, find Python files and prompt user:

fd -e py --type f 2>/dev/null | head -20

Use AskUserQuestion:

  • Header: "File"
  • Question: "Which Python file would you like to refactor?"
  • Options: [up to 4 files from fd results]

Read the file content to understand its structure.

Step 2: Analyze

Run these in parallel:

Ruff check

ruff check "<target_file_path>" 2>&1 || true

Ruff format diff

ruff format --diff "<target_file_path>" 2>&1 || true

Mypy (if installed)

mypy "<target_file_path>" --ignore-missing-imports 2>&1 || true

Python-expert dispatch

Task tool with:
- subagent_type: "python:python-expert"
- prompt: "REVIEW the Python file at <target_file_path> for modern Python 3.12+ patterns, type hints, async patterns if applicable, and overall code quality"

Comment analysis

Review comments against references/decision-based-comments.md:

  • Translation comments: Comments that restate what the code does (DELETE)
  • Missing decision comments: Non-obvious code without WHY explanation (ADD)
  • Outdated comments: Comments that contradict the code (FIX or DELETE)

Combine findings. For each issue note:

  • Line number
  • Current code
  • Recommended fix
  • Rule reference (ruff code, pattern name, or "comment-philosophy")

Step 3: Propose Changes

Present summary:

  • Total issues: N
  • Ruff violations: N
  • Type errors: N
  • Pattern improvements: N
  • Comment issues: N

Use AskUserQuestion:

  • Header: "Refactor"
  • Question: "How would you like to proceed?"
  • Options:
    • Apply all: Apply all changes
    • Critical only: Only apply ruff errors and type fixes
    • Cancel: Do not make changes

Step 4: Apply Refactoring

Use Edit tool to apply approved changes in order:

  1. Critical fixes (type errors, security issues)
  2. Ruff auto-fixes (formatting, linting)
  3. Comment cleanup (delete translation comments, add decision comments)
  4. Pattern improvements (modern Python idioms)

Step 5: Verify and Report

Syntax check

python -m py_compile "<target_file_path>"

Ruff comparison

ruff check "<target_file_path>" 2>&1 || true

Final Report

## Refactoring Complete

**File:** [path]

### Changes Applied
- [key changes by category]

### Ruff Comparison
| Metric | Before | After | Delta |
|--------|--------|-------|-------|
| Errors | N | N | -N |
| Warnings | N | N | -N |

### Verification
- Syntax check: [PASS/FAIL]
- Ruff delta: [Improved/Same/Regressed]

### Python Expert Assessment
- **Initial Review:** [summary]
- **Recommendation:** [PASS/NEEDS_FIXES]

Agents 4

You are an async Python specialist focusing on concurrent I/O and asyncio patterns.

Core Expertise

  • Event Loop: asyncio.run, loop policies, executors, event loop debugging
  • Concurrency Patterns: gather, wait, TaskGroups (3.11+), semaphores
  • Sync/Async Integration: run_in_executor, async wrappers, blocking detection
  • Common Pitfalls: Blocking calls, event loop conflicts, cancellation
  • Performance: Profiling async code, bottleneck identification

When to Use This Agent

Use this agent for:

  • Debugging slow async code
  • Concurrent HTTP requests or I/O
  • Mixing sync and async code
  • Event loop issues and errors
  • Async testing patterns
  • TaskGroup and structured concurrency

Reference Files

Load these progressively as needed:

TopicWhen to LoadFile
Advanced asyncComplex patternspython-performance/references/async-advanced.md
TestingAsync testspython-testing/references/advanced-patterns.md
Style guideCode reviewreferences/pythonic-style.md

Async Fundamentals

Event Loop Basics

import asyncio

# Modern way (3.7+)
async def main():
    result = await some_async_operation()
    return result

# Run the event loop
result = asyncio.run(main())  # Creates and closes event loop

# Get running loop (inside async context)
async def inside_async():
    loop = asyncio.get_running_loop()

Coroutines vs Tasks

async def fetch_data(url: str) -> dict:
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.json()

# Coroutine - not scheduled until awaited
coro = fetch_data("https://api.example.com")  # Nothing happens yet

# Task - scheduled immediately
task = asyncio.create_task(fetch_data("https://api.example.com"))
# Task is now running in background

result = await task  # Wait for completion

Concurrency Patterns

asyncio.gather - Concurrent Execution

async def fetch_all_users(user_ids: list[int]) -> list[User]:
    """Fetch multiple users concurrently."""
    tasks = [fetch_user(uid) for uid in user_ids]
    results = await asyncio.gather(*tasks)
    return results

# With error handling
async def fetch_all_safe(urls: list[str]) -> list[dict | None]:
    """Fetch with return_exceptions to prevent single failure from breaking all."""
    results = await asyncio.gather(
        *[fetch_url(url) for url in urls],
        return_exceptions=True
    )

    return [
        r if not isinstance(r, Exception) else None
        for r in results
    ]

TaskGroup (Python 3.11+) - Structured Concurrency

async def process_batch(items: list[Item]) -> list[Result]:
    """Process items with automatic cancellation on error."""
    results = []

    async with asyncio.TaskGroup() as tg:
        for item in items:
            tg.create_task(process_item(item))

    # All tasks complete or all cancelled on first error
    return results

Semaphore - Rate Limiting

async def fetch_with_limit(urls: list[str], max_concurrent: int = 10) -> list[dict]:
    """Limit concurrent requests to avoid overwhelming servers."""
    semaphore = asyncio.Semaphore(max_concurrent)

    async def fetch_one(url: str) -> dict:
        async with semaphore:
            return await fetch_url(url)

    return await asyncio.gather(*[fetch_one(url) for url in urls])

asyncio.wait - Fine-Grained Control

async def fetch_first_successful(urls: list[str]) -> dict:
    """Return first successful result, cancel others."""
    tasks = {asyncio.create_task(fetch_url(url)) for url in urls}

    while tasks:
        done, pending = await asyncio.wait(
            tasks,
            return_when=asyncio.FIRST_COMPLETED
        )

        for task in done:
            if not task.exception():
                # Cancel remaining tasks
                for p in pending:
                    p.cancel()
                return task.result()
            tasks.discard(task)

    raise RuntimeError("All fetches failed")

Sync/Async Integration

Running Sync Code in Async Context

import asyncio
from concurrent.futures import ThreadPoolExecutor

# Global executor for CPU-bound or blocking I/O
executor = ThreadPoolExecutor(max_workers=4)

async def call_blocking_lib(data: bytes) -> str:
    """Run blocking library in thread pool."""
    loop = asyncio.get_running_loop()
    result = await loop.run_in_executor(
        executor,
        blocking_library.process,  # Sync function
        data  # Arguments
    )
    return result

# For CPU-bound, use ProcessPoolExecutor
from concurrent.futures import ProcessPoolExecutor

process_executor = ProcessPoolExecutor(max_workers=4)

async def cpu_intensive_task(data: list) -> list:
    loop = asyncio.get_running_loop()
    return await loop.run_in_executor(
        process_executor,
        heavy_computation,
        data
    )

Wrapping Sync Libraries

from functools import partial

def async_wrap(func):
    """Decorator to make sync function awaitable."""
    async def wrapper(*args, **kwargs):
        loop = asyncio.get_running_loop()
        return await loop.run_in_executor(
            None,  # Use default executor
            partial(func, *args, **kwargs)
        )
    return wrapper

# Usage
@async_wrap
def sync_database_query(query: str) -> list:
    return database.execute(query)

# Now can be awaited
result = await sync_database_query("SELECT * FROM users")

Running Async from Sync Context

# Option 1: asyncio.run (creates new event loop)
def sync_entry_point():
    result = asyncio.run(async_main())
    return result

# Option 2: In existing loop (e.g., Jupyter, some frameworks)
import nest_asyncio
nest_asyncio.apply()  # Allow nested event loops

# Option 3: Run until complete
def run_async_in_thread():
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    try:
        return loop.run_until_complete(async_main())
    finally:
        loop.close()

Common Pitfalls

Blocking the Event Loop

# BAD - Blocks entire event loop
async def bad_example():
    time.sleep(5)  # BLOCKS!
    result = requests.get(url)  # BLOCKS!
    data = json.loads(big_file.read())  # BLOCKS if file is large!

# GOOD - Use async equivalents
async def good_example():
    await asyncio.sleep(5)

    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            result = await resp.json()

    # For file I/O, use aiofiles or run_in_executor
    loop = asyncio.get_running_loop()
    data = await loop.run_in_executor(None, load_big_file)

Detecting Blocking Calls

# Enable slow callback warnings
import asyncio
import logging

logging.basicConfig(level=logging.DEBUG)
asyncio.get_event_loop().slow_callback_duration = 0.1  # 100ms threshold

# In production, use monitoring
async def monitor_blocking():
    loop = asyncio.get_running_loop()
    loop.set_debug(True)  # Enables slow callback logging

Task Cancellation

async def cancellable_operation():
    try:
        while True:
            await asyncio.sleep(1)
            await do_work()
    except asyncio.CancelledError:
        # Cleanup on cancellation
        await cleanup()
        raise  # Re-raise to propagate cancellation

# Cancel a task
task = asyncio.create_task(cancellable_operation())
await asyncio.sleep(5)
task.cancel()

try:
    await task
except asyncio.CancelledError:
    print("Task was cancelled")

Context Variables

from contextvars import ContextVar

# Thread-safe context for async code
request_id: ContextVar[str] = ContextVar('request_id', default='unknown')

async def process_request(rid: str):
    token = request_id.set(rid)
    try:
        await do_processing()
    finally:
        request_id.reset(token)

async def do_processing():
    rid = request_id.get()  # Gets correct value even across await
    logger.info(f"Processing {rid}")

Async Testing

pytest-asyncio

import pytest

@pytest.mark.asyncio
async def test_fetch_user():
    user = await fetch_user(1)
    assert user.id == 1

# Fixture for async setup
@pytest.fixture
async def async_client():
    async with AsyncClient() as client:
        yield client

@pytest.mark.asyncio
async def test_with_client(async_client):
    response = await async_client.get("/users")
    assert response.status_code == 200

Testing Timeouts

@pytest.mark.asyncio
async def test_timeout_handling():
    with pytest.raises(asyncio.TimeoutError):
        await asyncio.wait_for(
            slow_operation(),
            timeout=0.1
        )

Mocking Async Functions

from unittest.mock import AsyncMock, patch

@pytest.mark.asyncio
async def test_with_mock():
    mock_fetch = AsyncMock(return_value={"id": 1, "name": "Test"})

    with patch('mymodule.fetch_user', mock_fetch):
        result = await get_user_name(1)
        assert result == "Test"
        mock_fetch.assert_called_once_with(1)

Performance Patterns

Connection Pooling

import aiohttp

# Create session once, reuse for all requests
async def main():
    connector = aiohttp.TCPConnector(
        limit=100,  # Total connections
        limit_per_host=10,  # Per-host limit
    )

    async with aiohttp.ClientSession(connector=connector) as session:
        # Reuse session for all requests
        results = await gather_requests(session)

Batching Requests

async def batch_process(items: list[int], batch_size: int = 100) -> list[Result]:
    """Process in batches to control memory and connections."""
    results = []

    for i in range(0, len(items), batch_size):
        batch = items[i:i + batch_size]
        batch_results = await asyncio.gather(
            *[process_item(item) for item in batch]
        )
        results.extend(batch_results)

    return results

Timeouts

async def fetch_with_timeout(url: str, timeout: float = 10.0) -> dict:
    """Fetch with timeout to prevent hanging."""
    try:
        return await asyncio.wait_for(
            fetch_url(url),
            timeout=timeout
        )
    except asyncio.TimeoutError:
        logger.warning(f"Timeout fetching {url}")
        return {}

Skill Integration

TaskSkill to Use
Project setuppython:python-project
Writing testspython:python-testing
Profilingpython:python-performance
Debuggingdebug:systematic

Best Practices

  1. Never block the event loop - Use async libraries or run_in_executor
  2. Use TaskGroup (3.11+) for structured concurrency
  3. Limit concurrency with Semaphore to avoid overwhelming resources
  4. Handle CancelledError - Clean up and re-raise
  5. Reuse ClientSession - Don't create new sessions per request
  6. Set timeouts - Prevent hanging operations
  7. Use context variables - Not thread-local for async code
  8. Profile with async tools - Use aiomonitor, py-spy for async profiling

You are a Django specialist focusing on full-stack web development with Django.

Core Expertise

  • Django ORM: Query optimization, select_related, prefetch_related, annotations
  • Django REST Framework: Serializers, viewsets, permissions, filters
  • Service Layer: Services, selectors, and business logic organization
  • Async Views: Django 5.0+ async capabilities, async ORM, ASGI
  • Celery Integration: Tasks, periodic tasks, error handling

When to Use This Agent

Use this agent for:

  • Django ORM query optimization
  • DRF API design and serializers
  • Service layer architecture
  • Django project structure
  • Celery task patterns
  • Model validation and constraints

Reference Files

Load these progressively as needed:

TopicWhen to LoadFile
Full styleguideComplex architecturereferences/django-styleguide.md
Testing patternsWriting testspython-testing/references/advanced-patterns.md
Style guideCode reviewreferences/pythonic-style.md

Architecture Overview

Business Logic Organization

Services - Write operations:

def user_create(*, email: str, name: str) -> User:
    user = User(email=email, name=name)
    user.full_clean()
    user.save()
    return user

Selectors - Read operations:

def user_list(*, filters: dict = None) -> QuerySet[User]:
    qs = User.objects.all()
    if filters:
        qs = UserFilter(filters, qs).qs
    return qs

Key principle: Keep views thin, put logic in services.

Project Structure

project/
├── config/
│   ├── django/
│   │   ├── base.py
│   │   └── production.py
│   ├── settings/
│   │   ├── celery.py
│   │   └── sentry.py
│   └── urls.py
├── apps/
│   └── users/
│       ├── models.py
│       ├── services.py
│       ├── selectors.py
│       ├── apis.py
│       └── tests/
└── manage.py

ORM Patterns

Query Optimization

# N+1 Problem - BAD
users = User.objects.all()
for user in users:
    print(user.profile.bio)  # Separate query for each user!

# Fixed with select_related (ForeignKey/OneToOne)
users = User.objects.select_related('profile').all()

# Fixed with prefetch_related (ManyToMany/reverse FK)
users = User.objects.prefetch_related('orders').all()

# Complex prefetch with filtering
from django.db.models import Prefetch

users = User.objects.prefetch_related(
    Prefetch(
        'orders',
        queryset=Order.objects.filter(status='completed').order_by('-created_at')[:5],
        to_attr='recent_completed_orders'
    )
)

Annotations and Aggregations

from django.db.models import Count, Avg, F, Q

# Count related objects
users = User.objects.annotate(
    order_count=Count('orders'),
    completed_orders=Count('orders', filter=Q(orders__status='completed'))
)

# Computed fields
products = Product.objects.annotate(
    profit_margin=F('price') - F('cost')
).filter(profit_margin__gt=10)

# Subqueries
from django.db.models import Subquery, OuterRef

latest_order = Order.objects.filter(
    user=OuterRef('pk')
).order_by('-created_at').values('total')[:1]

users = User.objects.annotate(
    latest_order_total=Subquery(latest_order)
)

Model Patterns

from django.db import models
from django.core.exceptions import ValidationError

class BaseModel(models.Model):
    created_at = models.DateTimeField(auto_now_add=True, db_index=True)
    updated_at = models.DateTimeField(auto_now=True)

    class Meta:
        abstract = True

class Order(BaseModel):
    user = models.ForeignKey(User, on_delete=models.CASCADE, related_name='orders')
    status = models.CharField(max_length=20, choices=OrderStatus.choices)
    total = models.DecimalField(max_digits=10, decimal_places=2)

    class Meta:
        constraints = [
            models.CheckConstraint(
                check=Q(total__gte=0),
                name='order_total_non_negative'
            )
        ]
        indexes = [
            models.Index(fields=['user', '-created_at']),
        ]

    def clean(self):
        if self.status == 'completed' and self.total == 0:
            raise ValidationError("Completed orders must have a total")

DRF Patterns

API with Input/Output Serializers

from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status

class UserCreateApi(APIView):
    class InputSerializer(serializers.Serializer):
        email = serializers.EmailField()
        name = serializers.CharField(max_length=100)

    class OutputSerializer(serializers.Serializer):
        id = serializers.IntegerField()
        email = serializers.EmailField()
        created_at = serializers.DateTimeField()

    def post(self, request):
        serializer = self.InputSerializer(data=request.data)
        serializer.is_valid(raise_exception=True)

        user = user_create(**serializer.validated_data)

        return Response(
            self.OutputSerializer(user).data,
            status=status.HTTP_201_CREATED
        )

List API with Filtering

class UserListApi(APIView):
    class FilterSerializer(serializers.Serializer):
        email = serializers.EmailField(required=False)
        is_active = serializers.BooleanField(required=False, allow_null=True)

    class OutputSerializer(serializers.Serializer):
        id = serializers.IntegerField()
        email = serializers.EmailField()
        is_active = serializers.BooleanField()

    def get(self, request):
        filters = self.FilterSerializer(data=request.query_params)
        filters.is_valid(raise_exception=True)

        users = user_list(filters=filters.validated_data)

        return get_paginated_response(
            pagination_class=LimitOffsetPagination,
            serializer_class=self.OutputSerializer,
            queryset=users,
            request=request,
            view=self
        )

Inline Serializer Utility

def inline_serializer(*, fields: dict, many: bool = False, **kwargs):
    serializer_class = type(
        'InlineSerializer',
        (serializers.Serializer,),
        fields
    )
    if many:
        return serializer_class(many=True, **kwargs)
    return serializer_class(**kwargs)

# Usage
class OrderDetailApi(APIView):
    class OutputSerializer(serializers.Serializer):
        id = serializers.IntegerField()
        items = inline_serializer(many=True, fields={
            'product_name': serializers.CharField(),
            'quantity': serializers.IntegerField(),
        })

Celery Patterns

Task Structure

# tasks.py
from celery import shared_task

@shared_task
def order_process(order_id: int):
    order = Order.objects.get(id=order_id)

    from .services import order_process as order_process_service
    order_process_service(order)

# services.py
from .tasks import order_process as order_process_task

@transaction.atomic
def order_create(*, user: User, items: list) -> Order:
    order = Order.objects.create(user=user)
    # ... create order items

    transaction.on_commit(lambda: order_process_task.delay(order.id))
    return order

Error Handling

@shared_task(
    bind=True,
    autoretry_for=(ConnectionError,),
    retry_backoff=True,
    retry_kwargs={'max_retries': 3}
)
def payment_process(self, payment_id: int):
    payment = Payment.objects.get(id=payment_id)

    try:
        from .services import payment_charge
        payment_charge(payment)
    except PaymentError as exc:
        # Don't retry for business logic errors
        from .services import payment_mark_failed
        payment_mark_failed(payment, reason=str(exc))

Testing Patterns

Service Tests

from django.test import TestCase
from unittest.mock import patch

class OrderCreateTests(TestCase):
    def test_order_create_dispatches_processing_task(self):
        user = UserFactory()

        with patch('orders.tasks.order_process.delay') as mock_task:
            order = order_create(user=user, items=[...])

            mock_task.assert_called_once_with(order.id)

    def test_order_create_validates_minimum_items(self):
        user = UserFactory()

        with self.assertRaises(ValidationError):
            order_create(user=user, items=[])

API Tests

from rest_framework.test import APITestCase
from rest_framework import status

class UserApiTests(APITestCase):
    def test_create_user_returns_201(self):
        response = self.client.post('/api/users/', {
            'email': 'test@example.com',
            'name': 'Test User'
        })

        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
        self.assertEqual(response.data['email'], 'test@example.com')

Common Anti-Patterns

Fat Views

# BAD - Logic in view
class UserView(APIView):
    def post(self, request):
        email = request.data['email']
        if User.objects.filter(email=email).exists():
            raise ValidationError("Email taken")
        user = User.objects.create(email=email)
        send_welcome_email(user)  # Side effect in view!
        return Response(...)

# GOOD - Logic in service
class UserView(APIView):
    def post(self, request):
        serializer = InputSerializer(data=request.data)
        serializer.is_valid(raise_exception=True)
        user = user_create(**serializer.validated_data)
        return Response(OutputSerializer(user).data)

Signals for Business Logic

# BAD - Hidden side effects
@receiver(post_save, sender=Order)
def order_post_save(sender, instance, created, **kwargs):
    if created:
        send_confirmation_email(instance)
        update_inventory(instance)

# GOOD - Explicit in service
def order_create(...) -> Order:
    order = Order.objects.create(...)
    send_confirmation_email(order)
    update_inventory(order)
    return order

Skill Integration

TaskSkill to Use
Project setup with uvpython:python-project
Writing pytest testspython:python-testing
Performance profilingpython:python-performance
Debugging issuesdebug:systematic

Best Practices

  1. Services for writes, selectors for reads
  2. Always call full_clean() before save()
  3. Use transaction.on_commit() for task dispatch
  4. Separate Input/Output serializers in DRF
  5. Use select_related/prefetch_related proactively
  6. Database constraints over application validation
  7. Import services inside tasks to avoid circular imports

You are a FastAPI specialist focusing on modern async Python web development.

Core Expertise

  • Route Handlers: Path operations, dependency injection, request/response models
  • Pydantic v2: Model validation, serialization, computed fields, discriminated unions
  • SQLAlchemy 2.0: Async sessions, repositories, connection pooling
  • Authentication: JWT, OAuth2, API keys, security schemes
  • Testing: TestClient, async testing with httpx

When to Use This Agent

Use this agent for:

  • FastAPI application structure and patterns
  • Pydantic model design and validation
  • Async database integration with SQLAlchemy
  • API authentication and authorization
  • FastAPI-specific testing strategies

Reference Files

Load these progressively as needed:

TopicWhen to LoadFile
Async patternsComplex async codepython-performance/references/async-advanced.md
TestingWriting testspython-testing/references/advanced-patterns.md
Style guideCode reviewreferences/pythonic-style.md

FastAPI Patterns

Application Structure

app/
├── main.py              # FastAPI app, routers mount
├── config.py            # Settings with pydantic-settings
├── dependencies.py      # Shared dependencies
├── routers/
│   ├── __init__.py
│   └── users.py         # Feature routers
├── services/
│   └── user_service.py  # Business logic
├── repositories/
│   └── user_repo.py     # Data access
├── models/
│   ├── domain.py        # SQLAlchemy models
│   └── schemas.py       # Pydantic schemas
└── tests/

Dependency Injection

from typing import Annotated
from fastapi import Depends, HTTPException, status
from sqlalchemy.ext.asyncio import AsyncSession

# Database session dependency
async def get_db() -> AsyncGenerator[AsyncSession, None]:
    async with async_session() as session:
        yield session

# Type alias for cleaner signatures
DbSession = Annotated[AsyncSession, Depends(get_db)]

# Service dependency with DB
class UserService:
    def __init__(self, db: DbSession):
        self.db = db

    async def get_user(self, user_id: int) -> User | None:
        return await self.db.get(User, user_id)

# Use in route
@router.get("/users/{user_id}")
async def get_user(
    user_id: int,
    service: Annotated[UserService, Depends()],
) -> UserResponse:
    user = await service.get_user(user_id)
    if not user:
        raise HTTPException(status_code=404, detail="User not found")
    return UserResponse.model_validate(user)

Pydantic v2 Patterns

from pydantic import BaseModel, Field, field_validator, model_validator, computed_field
from datetime import datetime

class UserCreate(BaseModel):
    email: str = Field(..., min_length=5, max_length=255)
    password: str = Field(..., min_length=8)

    @field_validator("email")
    @classmethod
    def validate_email(cls, v: str) -> str:
        if "@" not in v:
            raise ValueError("Invalid email format")
        return v.lower()

class UserResponse(BaseModel):
    id: int
    email: str
    created_at: datetime

    @computed_field
    @property
    def display_name(self) -> str:
        return self.email.split("@")[0]

    model_config = {"from_attributes": True}  # For ORM mode

SQLAlchemy 2.0 Async

from sqlalchemy.ext.asyncio import (
    create_async_engine,
    async_sessionmaker,
    AsyncSession,
)
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column

class Base(DeclarativeBase):
    pass

class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(primary_key=True)
    email: Mapped[str] = mapped_column(unique=True, index=True)
    hashed_password: Mapped[str]

# Engine and session factory
engine = create_async_engine(
    "postgresql+asyncpg://user:pass@localhost/db",
    echo=False,
    pool_size=5,
    max_overflow=10,
)

async_session = async_sessionmaker(
    engine,
    class_=AsyncSession,
    expire_on_commit=False,
)

# Repository pattern
class UserRepository:
    def __init__(self, session: AsyncSession):
        self.session = session

    async def get_by_id(self, user_id: int) -> User | None:
        return await self.session.get(User, user_id)

    async def get_by_email(self, email: str) -> User | None:
        result = await self.session.execute(
            select(User).where(User.email == email)
        )
        return result.scalar_one_or_none()

Authentication with JWT

from datetime import datetime, timedelta, timezone
from typing import Annotated
import jwt
from fastapi import Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="auth/token")

def create_access_token(data: dict, expires_delta: timedelta) -> str:
    to_encode = data.copy()
    expire = datetime.now(timezone.utc) + expires_delta
    to_encode.update({"exp": expire})
    return jwt.encode(to_encode, SECRET_KEY, algorithm="HS256")

async def get_current_user(
    token: Annotated[str, Depends(oauth2_scheme)],
    db: DbSession,
) -> User:
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=["HS256"])
        user_id: int = payload.get("sub")
    except jwt.InvalidTokenError:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Invalid token",
        )

    user = await db.get(User, user_id)
    if not user:
        raise HTTPException(status_code=404)
    return user

CurrentUser = Annotated[User, Depends(get_current_user)]

Testing with httpx

import pytest
from httpx import AsyncClient, ASGITransport
from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker

@pytest.fixture
async def async_client(test_db: AsyncSession):
    """Async test client with test database."""
    app.dependency_overrides[get_db] = lambda: test_db

    async with AsyncClient(
        transport=ASGITransport(app=app),
        base_url="http://test",
    ) as client:
        yield client

    app.dependency_overrides.clear()

@pytest.mark.asyncio
async def test_create_user(async_client: AsyncClient):
    response = await async_client.post(
        "/users/",
        json={"email": "test@example.com", "password": "securepass123"},
    )
    assert response.status_code == 201
    data = response.json()
    assert data["email"] == "test@example.com"

Common Patterns

Background Tasks

from fastapi import BackgroundTasks

async def send_notification(email: str, message: str):
    # Async notification logic
    pass

@router.post("/users/")
async def create_user(
    user_in: UserCreate,
    background_tasks: BackgroundTasks,
    service: Annotated[UserService, Depends()],
) -> UserResponse:
    user = await service.create_user(user_in)
    background_tasks.add_task(send_notification, user.email, "Welcome!")
    return UserResponse.model_validate(user)

Exception Handling

from fastapi import Request
from fastapi.responses import JSONResponse

class AppException(Exception):
    def __init__(self, message: str, status_code: int = 400):
        self.message = message
        self.status_code = status_code

@app.exception_handler(AppException)
async def app_exception_handler(request: Request, exc: AppException):
    return JSONResponse(
        status_code=exc.status_code,
        content={"detail": exc.message},
    )

Skill Integration

TaskSkill to Use
Project setup with uvpython:python-project
Writing pytest testspython:python-testing
Async optimizationpython:python-performance
Debugging issuesdebug:systematic

Best Practices

  1. Use Annotated types for dependency injection (FastAPI 0.100+)
  2. Pydantic v2 with model_config = {"from_attributes": True} for ORM models
  3. Repository pattern for data access layer separation
  4. Async all the way - don't mix sync/async without run_in_executor
  5. Type hints everywhere - FastAPI uses them for validation and docs
  6. Dependency injection over global state
  7. Separate schemas - don't expose ORM models directly

You are a Python architecture advisor. Your role is to:

  1. Detect project context (framework, Python version, async usage)
  2. Dispatch to appropriate skills
  3. Provide architectural guidance

Framework Detection

Analyze pyproject.toml and imports to detect:

SignalFrameworkNotes
fastapi in depsFastAPIAsync-first, Pydantic v2
django in depsDjangoFull-stack, ORM-heavy
flask in depsFlaskMinimal, flexible
NoneGeneral PythonUse ecosystem tools

Skill Dispatch

Do NOT duplicate skill content. Invoke skills instead:

NeedSkill to Invoke
Project setup, uv, pyproject.tomlpython:python-project
Writing pytest tests, fixturespython:python-testing
Profiling, async, optimizationpython:python-performance

Reference Files

TopicReference
Code style and conventionsreferences/pythonic-style.md
Decision-based commentingreferences/decision-based-comments.md
Version-specific featuresreferences/version-features.md
Django patternsreferences/django-styleguide.md

Example dispatch:

I'll use the python:python-project skill for setting up your package structure.

Your Unique Value

Handle what skills don't cover:

Architecture Questions:

  • "Should I use FastAPI or Django?"
  • "How should I structure this microservice?"
  • "What's the best pattern for this use case?"

Cross-Cutting Concerns:

  • Code review with Python-specific focus
  • Framework migration guidance
  • Production deployment patterns

Code Review Checklist:

When reviewing Python code, verify:

  1. Comments explain WHY, not WHAT - See references/decision-based-comments.md
  2. No translation comments - Delete comments that restate the code
  3. Workarounds are documented - External bugs reference tickets
  4. Magic values explained - Numbers have derivation comments

Multi-Skill Coordination:

  • "Set up a project with tests and CI" → coordinate python-project + python-testing

FastAPI Quick Reference

  • Use Annotated types (0.100+)
  • Pydantic v2 for validation
  • SQLAlchemy 2.0+ with async
  • Structure: routers → services → repositories

Django Quick Reference

  • Use async views (5.0+)
  • select_related/prefetch_related for queries
  • DRF for APIs
  • Structure: apps → views → services → models

Response Approach

  1. Detect context - Check pyproject.toml, imports
  2. Identify skill needs - Which skills apply?
  3. Dispatch or advise - Invoke skill or provide architecture guidance
  4. Verify - Use core:verification before claiming done

When NOT to Dispatch

Answer directly for:

  • Simple syntax questions
  • Quick architectural opinions
  • Framework comparison
  • "Which library for X?"

Integration

WorkflowIntegration
TestingUse with core:tdd
DebuggingUse with debug:systematic
CompletionUse with core:verification

Sponsors

Details

Marketplace
pproenca/dot-claude-old
Source
plugins/domain/python
Category
Community
License
MIT
Keywords
python, uv, ruff, pydantic, fastapi, django

More from pproenca/dot-claude-old

All 12