
python
Modern Python 3.12+ development with uv, ruff, pydantic, FastAPI, Django and production-ready practices
- version
- 1.0.0
Install
/plugin install python@dot-claudeRun it inside Claude Code after adding the marketplace with /plugin marketplace add pproenca/dot-claude-old.
Contents
Discovered from sourceSkills 3
Python Performance & Concurrency
Profiling, optimization, and async patterns for Python.
Before Writing Code
- Read
references/pythonic-style.mdfor style conventions - Check Python version: 3.13+ enables free-threaded concurrency options
- Profile before optimizing - never guess at bottlenecks
Reference Files
| Topic | When to Load | File |
|---|---|---|
| Pythonic style | Before generating code | ../references/pythonic-style.md |
| Semaphores, locks, producers/consumers | Advanced async | references/async-advanced.md |
| Database, memory, NumPy optimization | Deep optimization | references/optimization-advanced.md |
Profiling First
cProfile (CPU)
python -m cProfile -o output.prof script.py
python -m pstats output.prof # Interactive analysis
import cProfile
import pstats
with cProfile.Profile() as pr:
main()
stats = pstats.Stats(pr)
stats.sort_stats("cumulative").print_stats(10)
line_profiler (Line-by-Line)
uv add line-profiler
kernprof -l -v script.py # Requires @profile decorator
py-spy (Production)
py-spy record -o profile.svg -- python script.py
py-spy top --pid 12345 # Live profiling
memory_profiler
uv add memory-profiler
python -m memory_profiler script.py # Requires @profile decorator
Concurrency Pattern Selection
| Workload | Solution |
|---|---|
| I/O-bound (network, disk) | async/await |
| CPU-bound (GIL Python) | multiprocessing |
| CPU-bound (nogil 3.13+) | threading |
| Mixed | async + ProcessPoolExecutor |
Async/Await Patterns
Basic Async
import asyncio
async def fetch_data(url: str) -> dict:
await asyncio.sleep(1) # Simulates I/O
return {"url": url, "data": "result"}
async def main():
result = await fetch_data("https://api.example.com")
print(result)
asyncio.run(main())
Concurrent Execution
async def fetch_all(urls: list[str]) -> list[dict]:
# Concurrent I/O eliminates sequential waiting: 10 URLs at 100ms each
# complete in ~100ms total, not 1000ms. Critical for API aggregation.
tasks = [fetch_data(url) for url in urls]
return await asyncio.gather(*tasks)
Error Handling
async def safe_fetch(url: str) -> dict | None:
try:
return await fetch_data(url)
except Exception as e:
print(f"Error: {e}")
return None
async def fetch_with_errors(urls: list[str]):
results = await asyncio.gather(
*[safe_fetch(url) for url in urls],
return_exceptions=True # Don't fail on first error
)
return [r for r in results if r and not isinstance(r, Exception)]
Timeouts
try:
result = await asyncio.wait_for(slow_operation(), timeout=5.0)
except asyncio.TimeoutError:
print("Operation timed out")
Async Pitfalls
# WRONG: Forgetting await
result = async_function() # Returns coroutine, doesn't execute!
# WRONG: Blocking the event loop
import time
async def bad():
time.sleep(1) # Blocks everything!
# CORRECT
async def good():
await asyncio.sleep(1) # Non-blocking
# WRONG: Calling async from sync
def sync_func():
result = await async_func() # SyntaxError!
# CORRECT
def sync_func():
result = asyncio.run(async_func())
Optimization Patterns
Data Structures
# Sets use hash tables for O(1) membership testing. For collections
# checked repeatedly or exceeding ~10 items, the overhead of hashing
# pays off vs linear scanning.
if item in items_list: # O(n) - scans every element
if item in items_set: # O(1) - hash lookup
# String immutability in Python forces reallocation on every +=.
# For N concatenations, this creates O(n²) total copying.
# join() pre-calculates final size and allocates once.
result = ""
for s in strings:
result += s # Creates N intermediate string objects
result = "".join(strings) # Single allocation, O(n) total
List Comprehensions
# List comprehensions bypass method dispatch overhead (no .append lookup)
# and CPython pre-sizes the result list. For simple transforms, prefer
# comprehensions unless loop body requires complex logic or side effects.
result = []
for i in range(n):
result.append(i**2) # Method lookup + call per iteration
result = [i**2 for i in range(n)] # Optimized bytecode path
Generators for Memory
import sys
list_data = [i for i in range(1_000_000)] # ~8MB in memory
gen_data = (i for i in range(1_000_000)) # ~100 bytes (lazy evaluation)
# Generators yield values on-demand, preventing memory exhaustion when
# processing data larger than available RAM. Essential for log parsing,
# ETL pipelines, or any single-pass data processing.
def process_large_file(path):
with open(path) as f:
for line in f: # File iterator is itself a generator
yield process(line)
Caching
from functools import lru_cache
@lru_cache(maxsize=128)
def expensive_computation(n: int) -> int:
# Cached: subsequent calls with same n return instantly
return sum(i**2 for i in range(n))
# Clear cache if needed
expensive_computation.cache_clear()
Local Variables
# Python's LEGB resolution requires dictionary lookups for global/nonlocal
# variables, but uses direct array indexing for locals. In tight loops with
# 10k+ iterations, this overhead compounds significantly.
GLOBAL = 100
def slow():
for i in range(10000):
x = GLOBAL * i # Dict lookup each iteration
def fast():
local = 100 # Captured once at function entry
for i in range(10000):
x = local * i # Direct array index
Multiprocessing (CPU-Bound)
import multiprocessing as mp
def cpu_task(n: int) -> int:
return sum(i**2 for i in range(n))
if __name__ == "__main__":
with mp.Pool(4) as pool:
results = pool.map(cpu_task, [1_000_000] * 4)
Async + Sync Integration
import asyncio
from concurrent.futures import ProcessPoolExecutor
def cpu_bound(n: int) -> int:
return sum(i**2 for i in range(n))
async def main():
loop = asyncio.get_running_loop()
with ProcessPoolExecutor() as pool:
result = await loop.run_in_executor(pool, cpu_bound, 1_000_000)
print(result)
asyncio.run(main())
Testing Async Code
import pytest
@pytest.mark.asyncio
async def test_fetch():
result = await fetch_data("https://api.example.com")
assert result is not None
@pytest.mark.asyncio
async def test_timeout():
with pytest.raises(asyncio.TimeoutError):
await asyncio.wait_for(slow_operation(), timeout=0.1)
Benchmarking
import timeit
# Quick benchmark
time = timeit.timeit(lambda: my_function(), number=1000)
print(f"{time:.4f}s for 1000 runs")
# pytest-benchmark (uv add --dev pytest-benchmark)
def test_performance(benchmark):
result = benchmark(my_function)
assert result is not None
Workflow Integration
| Task | Skill |
|---|---|
| Writing async tests | python:python-testing |
| Root cause analysis | debug:systematic |
| Before claiming done | core:verification |
Best Practices
- Profile before optimizing - find real bottlenecks
- Use async for I/O - network, disk, databases
- Use multiprocessing for CPU - heavy computation
- Consider nogil (3.13+) - threading for CPU-bound
- Use appropriate data structures - set/dict for lookups
- Cache expensive operations - lru_cache
- Use generators - for large data iteration
- Avoid blocking in async - no time.sleep()
- Handle cancellation - catch CancelledError
- Benchmark changes - prove improvement
Python Project Management
Modern Python project setup and dependency management with uv.
Before Writing Code
- Read
references/pythonic-style.mdfor style conventions - Check Python version:
pyproject.toml→.python-version→.claude/python-version - If unknown, ask user once and store in
.claude/python-version
Reference Files
| Topic | When to Load | File |
|---|---|---|
| Pythonic style, conventions | Before generating code | ../references/pythonic-style.md |
| Version-specific features | When adapting to Python version | ../references/version-features.md |
| GitHub Actions, Docker, monorepo | CI/CD and advanced workflows | references/ci-cd-workflows.md |
| Full pyproject.toml templates | Complete configuration examples | references/pyproject-templates.md |
Quick Start
# New project
uv init my-project && cd my-project
uv python pin 3.12
uv add fastapi pydantic
uv add --dev pytest ruff mypy
# Existing project
uv sync # Install from pyproject.toml
uv sync --all-extras # Include optional deps
Project Structure
Source Layout (Recommended)
my-package/
├── pyproject.toml
├── README.md
├── src/
│ └── my_package/
│ ├── __init__.py
│ ├── core.py
│ └── py.typed
├── tests/
│ └── test_core.py
└── .python-version
Why src/ layout: The src/ layout prevents accidental imports from the source tree during
testing. Without it, import my_package might resolve to the local directory instead of the
installed package, masking missing __init__.py files or build configuration errors.
Minimal pyproject.toml
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "my-package"
version = "1.0.0"
requires-python = ">=3.10"
dependencies = ["requests>=2.28"]
[project.optional-dependencies]
dev = ["pytest>=7.0", "ruff>=0.1", "mypy>=1.0"]
[tool.setuptools.packages.find]
where = ["src"]
[tool.ruff]
line-length = 100
target-version = "py310"
[tool.ruff.lint]
select = ["E", "F", "I", "UP"]
Dependency Management
Adding Packages
uv add requests # Add to dependencies
uv add "django>=4.0,<5.0" # With version constraint
uv add --dev pytest ruff # Development dependencies
uv add --optional docs sphinx # Optional group
uv add -e ./local-package # Editable local
Updating Packages
uv add --upgrade requests # Upgrade specific
uv sync --upgrade # Upgrade all
uv tree --outdated # Show outdatable
Locking
uv lock # Generate uv.lock
uv lock --upgrade # Regenerate with latest
uv sync --frozen # Install exact versions (CI)
Python Version Management
uv python install 3.12 # Install Python
uv python pin 3.12 # Create .python-version
uv python list # List installed
Running Code
uv run python app.py # Run script
uv run pytest # Run tests
uv run ruff check . # Run linter
uv run --python 3.11 script.py # Specific version
CLI Entry Points
With Click
# src/my_package/cli.py
import click
@click.group()
@click.version_option()
def cli():
"""My CLI tool."""
@cli.command()
@click.argument("name")
def greet(name: str):
click.echo(f"Hello, {name}!")
def main():
cli()
# pyproject.toml
[project.scripts]
my-tool = "my_package.cli:main"
uv sync && uv run my-tool greet World
Building and Publishing
# Build
uv build # Creates dist/*.whl and dist/*.tar.gz
# Test on TestPyPI first
uv publish --repository testpypi
# Publish to PyPI
uv publish
API Token Setup
# ~/.pypirc
[pypi]
username = __token__
password = pypi-...your-token...
Publishing Checklist
Before publishing:
- Tests passing (
uv run pytest) - Version updated in pyproject.toml
- CHANGELOG updated
- Build succeeds (
uv build) - Install works in clean venv
- Tested on TestPyPI first
Common Issues
# Wrong Python version
uv python pin 3.12 && uv venv --python 3.12
# Dependency conflict
uv lock --verbose
# Cache issues
uv cache clean
# Lockfile out of sync
uv lock --upgrade
Workflow Integration
| Task | Skill |
|---|---|
| Writing tests | python:python-testing |
| Profiling, async | python:python-performance |
| Before claiming done | core:verification |
Best Practices
- Use
src/layout for libraries - Pin Python version with
.python-version - Commit
uv.lockfor reproducibility - Use
uv runinstead of manual activation - Test installation in clean venv before publishing
- Use TestPyPI before real PyPI
- Automate publishing with GitHub Actions
- Keep dependencies minimal
- Use optional groups for dev/docs deps
- Run
uv sync --frozenin CI
Python Testing Patterns
Robust testing with pytest.
Before Writing Tests
- Read
references/pythonic-style.mdfor naming and style conventions - TDD workflow: Write failing test first, then implement
- Test behavior, not implementation details
Reference Files
| Topic | When to Load | File |
|---|---|---|
| Pythonic style | Before writing code | ../references/pythonic-style.md |
| Async, property-based, factories | Advanced patterns | references/advanced-patterns.md |
| SQLAlchemy, in-memory DBs | Database testing | references/database-testing.md |
Basic Tests
import pytest
def test_addition():
assert 2 + 2 == 4
def test_exception_raised():
with pytest.raises(ValueError, match="invalid"):
raise ValueError("invalid input")
Fixtures
@pytest.fixture
def db():
"""
Provides isolated database connection for test isolation.
Using in-memory SQLite prevents test pollution across runs and
guarantees each test starts with known state.
"""
database = Database("sqlite:///:memory:")
database.connect()
yield database
database.disconnect()
def test_query(db):
results = db.query("SELECT * FROM users")
assert len(results) >= 0
# Session scope shares fixture across all tests for expensive resources
# (e.g., one DB schema creation for all tests vs per-test).
@pytest.fixture(scope="session")
def app_config():
return {"debug": True, "db_url": "sqlite:///:memory:"}
Parametrized Tests
@pytest.mark.parametrize("email,valid", [
("user@example.com", True),
("invalid.email", False),
("", False),
])
def test_email_validation(email, valid):
assert is_valid_email(email) == valid
# With custom IDs
@pytest.mark.parametrize("value,expected", [
pytest.param(1, True, id="positive"),
pytest.param(0, False, id="zero"),
pytest.param(-1, False, id="negative"),
])
def test_is_positive(value, expected):
assert (value > 0) == expected
Mocking
from unittest.mock import Mock, patch
def test_api_call():
mock_response = Mock()
mock_response.json.return_value = {"id": 1, "name": "Test"}
with patch("requests.get", return_value=mock_response) as mock_get:
result = fetch_user(1)
assert result["id"] == 1
mock_get.assert_called_once_with("https://api.example.com/users/1")
# Decorator syntax
@patch("module.external_service")
def test_with_mock(mock_service):
mock_service.return_value = "mocked"
assert my_function() == "mocked"
Async Testing
import pytest
@pytest.mark.asyncio
async def test_async_fetch():
result = await fetch_data("https://api.example.com")
assert result is not None
@pytest.mark.asyncio
async def test_timeout():
with pytest.raises(asyncio.TimeoutError):
await asyncio.wait_for(slow_operation(), timeout=0.1)
# Async fixture
@pytest.fixture
async def async_client():
client = AsyncClient()
await client.connect()
yield client
await client.disconnect()
Test Organization
tests/
├── conftest.py # Shared fixtures
├── test_unit/
│ ├── test_models.py
│ └── test_utils.py
├── test_integration/
│ └── test_api.py
└── test_e2e/
└── test_workflows.py
Naming Conventions
# Good: describes behavior and expected outcome
def test_user_creation_with_valid_data_succeeds():
def test_login_fails_with_invalid_password():
def test_api_returns_404_for_missing_resource():
# Bad: vague or non-descriptive
def test_user():
def test_1():
Markers
@pytest.mark.slow
def test_slow_operation():
...
@pytest.mark.integration
def test_database():
...
@pytest.mark.skip(reason="Not implemented")
def test_future_feature():
...
@pytest.mark.xfail(reason="Known bug #123")
def test_known_bug():
...
# Run: pytest -m "not slow"
Configuration
# pyproject.toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = ["-v", "--cov=myapp", "--cov-report=term-missing"]
markers = [
"slow: marks tests as slow",
"integration: marks integration tests",
]
[tool.coverage.run]
source = ["src"]
omit = ["*/tests/*"]
TDD Workflow
- RED: Write failing test
- GREEN: Write minimal code to pass
- REFACTOR: Improve while keeping tests green
# 1. Write test first
def test_calculate_total():
cart = Cart([Item(10), Item(20)])
assert cart.total() == 30
# 2. Run test - it fails (Cart doesn't exist)
# 3. Implement minimal code
class Cart:
def __init__(self, items):
self.items = items
def total(self):
return sum(item.price for item in self.items)
# 4. Run test - it passes
# 5. Refactor if needed (tests stay green)
Workflow Integration
If the core plugin is installed, use these skills:
| Task | Skill |
|---|---|
| Full TDD methodology | core:tdd |
| Debug test failures | debug:systematic |
| Before claiming done | core:verification |
Best Practices
- One assertion per test when possible
- Descriptive names that explain behavior
- Independent tests - no shared mutable state
- Use fixtures for setup/teardown
- Mock external dependencies appropriately
- Parametrize to reduce duplication
- Test edge cases and error conditions
- Run tests in CI on every commit
- Measure coverage but focus on quality
- Write tests first (TDD) when possible
Commands 1
Refactor Python File
Refactor the specified Python file to follow modern Python patterns and best practices.
Progress Tracking
Create TodoWrite with each step:
- Identify and read target file
- Analyze (ruff + mypy + python-expert in parallel)
- Propose changes
- Apply refactoring
- Verify and report
Step 1: Identify and Read Target File
If $ARGUMENTS is provided, use it as the target file path.
Otherwise, find Python files and prompt user:
fd -e py --type f 2>/dev/null | head -20
Use AskUserQuestion:
- Header: "File"
- Question: "Which Python file would you like to refactor?"
- Options: [up to 4 files from fd results]
Read the file content to understand its structure.
Step 2: Analyze
Run these in parallel:
Ruff check
ruff check "<target_file_path>" 2>&1 || true
Ruff format diff
ruff format --diff "<target_file_path>" 2>&1 || true
Mypy (if installed)
mypy "<target_file_path>" --ignore-missing-imports 2>&1 || true
Python-expert dispatch
Task tool with:
- subagent_type: "python:python-expert"
- prompt: "REVIEW the Python file at <target_file_path> for modern Python 3.12+ patterns, type hints, async patterns if applicable, and overall code quality"
Comment analysis
Review comments against references/decision-based-comments.md:
- Translation comments: Comments that restate what the code does (DELETE)
- Missing decision comments: Non-obvious code without WHY explanation (ADD)
- Outdated comments: Comments that contradict the code (FIX or DELETE)
Combine findings. For each issue note:
- Line number
- Current code
- Recommended fix
- Rule reference (ruff code, pattern name, or "comment-philosophy")
Step 3: Propose Changes
Present summary:
- Total issues: N
- Ruff violations: N
- Type errors: N
- Pattern improvements: N
- Comment issues: N
Use AskUserQuestion:
- Header: "Refactor"
- Question: "How would you like to proceed?"
- Options:
- Apply all: Apply all changes
- Critical only: Only apply ruff errors and type fixes
- Cancel: Do not make changes
Step 4: Apply Refactoring
Use Edit tool to apply approved changes in order:
- Critical fixes (type errors, security issues)
- Ruff auto-fixes (formatting, linting)
- Comment cleanup (delete translation comments, add decision comments)
- Pattern improvements (modern Python idioms)
Step 5: Verify and Report
Syntax check
python -m py_compile "<target_file_path>"
Ruff comparison
ruff check "<target_file_path>" 2>&1 || true
Final Report
## Refactoring Complete
**File:** [path]
### Changes Applied
- [key changes by category]
### Ruff Comparison
| Metric | Before | After | Delta |
|--------|--------|-------|-------|
| Errors | N | N | -N |
| Warnings | N | N | -N |
### Verification
- Syntax check: [PASS/FAIL]
- Ruff delta: [Improved/Same/Regressed]
### Python Expert Assessment
- **Initial Review:** [summary]
- **Recommendation:** [PASS/NEEDS_FIXES]
Agents 4
You are an async Python specialist focusing on concurrent I/O and asyncio patterns.
Core Expertise
- Event Loop: asyncio.run, loop policies, executors, event loop debugging
- Concurrency Patterns: gather, wait, TaskGroups (3.11+), semaphores
- Sync/Async Integration: run_in_executor, async wrappers, blocking detection
- Common Pitfalls: Blocking calls, event loop conflicts, cancellation
- Performance: Profiling async code, bottleneck identification
When to Use This Agent
Use this agent for:
- Debugging slow async code
- Concurrent HTTP requests or I/O
- Mixing sync and async code
- Event loop issues and errors
- Async testing patterns
- TaskGroup and structured concurrency
Reference Files
Load these progressively as needed:
| Topic | When to Load | File |
|---|---|---|
| Advanced async | Complex patterns | python-performance/references/async-advanced.md |
| Testing | Async tests | python-testing/references/advanced-patterns.md |
| Style guide | Code review | references/pythonic-style.md |
Async Fundamentals
Event Loop Basics
import asyncio
# Modern way (3.7+)
async def main():
result = await some_async_operation()
return result
# Run the event loop
result = asyncio.run(main()) # Creates and closes event loop
# Get running loop (inside async context)
async def inside_async():
loop = asyncio.get_running_loop()
Coroutines vs Tasks
async def fetch_data(url: str) -> dict:
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.json()
# Coroutine - not scheduled until awaited
coro = fetch_data("https://api.example.com") # Nothing happens yet
# Task - scheduled immediately
task = asyncio.create_task(fetch_data("https://api.example.com"))
# Task is now running in background
result = await task # Wait for completion
Concurrency Patterns
asyncio.gather - Concurrent Execution
async def fetch_all_users(user_ids: list[int]) -> list[User]:
"""Fetch multiple users concurrently."""
tasks = [fetch_user(uid) for uid in user_ids]
results = await asyncio.gather(*tasks)
return results
# With error handling
async def fetch_all_safe(urls: list[str]) -> list[dict | None]:
"""Fetch with return_exceptions to prevent single failure from breaking all."""
results = await asyncio.gather(
*[fetch_url(url) for url in urls],
return_exceptions=True
)
return [
r if not isinstance(r, Exception) else None
for r in results
]
TaskGroup (Python 3.11+) - Structured Concurrency
async def process_batch(items: list[Item]) -> list[Result]:
"""Process items with automatic cancellation on error."""
results = []
async with asyncio.TaskGroup() as tg:
for item in items:
tg.create_task(process_item(item))
# All tasks complete or all cancelled on first error
return results
Semaphore - Rate Limiting
async def fetch_with_limit(urls: list[str], max_concurrent: int = 10) -> list[dict]:
"""Limit concurrent requests to avoid overwhelming servers."""
semaphore = asyncio.Semaphore(max_concurrent)
async def fetch_one(url: str) -> dict:
async with semaphore:
return await fetch_url(url)
return await asyncio.gather(*[fetch_one(url) for url in urls])
asyncio.wait - Fine-Grained Control
async def fetch_first_successful(urls: list[str]) -> dict:
"""Return first successful result, cancel others."""
tasks = {asyncio.create_task(fetch_url(url)) for url in urls}
while tasks:
done, pending = await asyncio.wait(
tasks,
return_when=asyncio.FIRST_COMPLETED
)
for task in done:
if not task.exception():
# Cancel remaining tasks
for p in pending:
p.cancel()
return task.result()
tasks.discard(task)
raise RuntimeError("All fetches failed")
Sync/Async Integration
Running Sync Code in Async Context
import asyncio
from concurrent.futures import ThreadPoolExecutor
# Global executor for CPU-bound or blocking I/O
executor = ThreadPoolExecutor(max_workers=4)
async def call_blocking_lib(data: bytes) -> str:
"""Run blocking library in thread pool."""
loop = asyncio.get_running_loop()
result = await loop.run_in_executor(
executor,
blocking_library.process, # Sync function
data # Arguments
)
return result
# For CPU-bound, use ProcessPoolExecutor
from concurrent.futures import ProcessPoolExecutor
process_executor = ProcessPoolExecutor(max_workers=4)
async def cpu_intensive_task(data: list) -> list:
loop = asyncio.get_running_loop()
return await loop.run_in_executor(
process_executor,
heavy_computation,
data
)
Wrapping Sync Libraries
from functools import partial
def async_wrap(func):
"""Decorator to make sync function awaitable."""
async def wrapper(*args, **kwargs):
loop = asyncio.get_running_loop()
return await loop.run_in_executor(
None, # Use default executor
partial(func, *args, **kwargs)
)
return wrapper
# Usage
@async_wrap
def sync_database_query(query: str) -> list:
return database.execute(query)
# Now can be awaited
result = await sync_database_query("SELECT * FROM users")
Running Async from Sync Context
# Option 1: asyncio.run (creates new event loop)
def sync_entry_point():
result = asyncio.run(async_main())
return result
# Option 2: In existing loop (e.g., Jupyter, some frameworks)
import nest_asyncio
nest_asyncio.apply() # Allow nested event loops
# Option 3: Run until complete
def run_async_in_thread():
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
return loop.run_until_complete(async_main())
finally:
loop.close()
Common Pitfalls
Blocking the Event Loop
# BAD - Blocks entire event loop
async def bad_example():
time.sleep(5) # BLOCKS!
result = requests.get(url) # BLOCKS!
data = json.loads(big_file.read()) # BLOCKS if file is large!
# GOOD - Use async equivalents
async def good_example():
await asyncio.sleep(5)
async with aiohttp.ClientSession() as session:
async with session.get(url) as resp:
result = await resp.json()
# For file I/O, use aiofiles or run_in_executor
loop = asyncio.get_running_loop()
data = await loop.run_in_executor(None, load_big_file)
Detecting Blocking Calls
# Enable slow callback warnings
import asyncio
import logging
logging.basicConfig(level=logging.DEBUG)
asyncio.get_event_loop().slow_callback_duration = 0.1 # 100ms threshold
# In production, use monitoring
async def monitor_blocking():
loop = asyncio.get_running_loop()
loop.set_debug(True) # Enables slow callback logging
Task Cancellation
async def cancellable_operation():
try:
while True:
await asyncio.sleep(1)
await do_work()
except asyncio.CancelledError:
# Cleanup on cancellation
await cleanup()
raise # Re-raise to propagate cancellation
# Cancel a task
task = asyncio.create_task(cancellable_operation())
await asyncio.sleep(5)
task.cancel()
try:
await task
except asyncio.CancelledError:
print("Task was cancelled")
Context Variables
from contextvars import ContextVar
# Thread-safe context for async code
request_id: ContextVar[str] = ContextVar('request_id', default='unknown')
async def process_request(rid: str):
token = request_id.set(rid)
try:
await do_processing()
finally:
request_id.reset(token)
async def do_processing():
rid = request_id.get() # Gets correct value even across await
logger.info(f"Processing {rid}")
Async Testing
pytest-asyncio
import pytest
@pytest.mark.asyncio
async def test_fetch_user():
user = await fetch_user(1)
assert user.id == 1
# Fixture for async setup
@pytest.fixture
async def async_client():
async with AsyncClient() as client:
yield client
@pytest.mark.asyncio
async def test_with_client(async_client):
response = await async_client.get("/users")
assert response.status_code == 200
Testing Timeouts
@pytest.mark.asyncio
async def test_timeout_handling():
with pytest.raises(asyncio.TimeoutError):
await asyncio.wait_for(
slow_operation(),
timeout=0.1
)
Mocking Async Functions
from unittest.mock import AsyncMock, patch
@pytest.mark.asyncio
async def test_with_mock():
mock_fetch = AsyncMock(return_value={"id": 1, "name": "Test"})
with patch('mymodule.fetch_user', mock_fetch):
result = await get_user_name(1)
assert result == "Test"
mock_fetch.assert_called_once_with(1)
Performance Patterns
Connection Pooling
import aiohttp
# Create session once, reuse for all requests
async def main():
connector = aiohttp.TCPConnector(
limit=100, # Total connections
limit_per_host=10, # Per-host limit
)
async with aiohttp.ClientSession(connector=connector) as session:
# Reuse session for all requests
results = await gather_requests(session)
Batching Requests
async def batch_process(items: list[int], batch_size: int = 100) -> list[Result]:
"""Process in batches to control memory and connections."""
results = []
for i in range(0, len(items), batch_size):
batch = items[i:i + batch_size]
batch_results = await asyncio.gather(
*[process_item(item) for item in batch]
)
results.extend(batch_results)
return results
Timeouts
async def fetch_with_timeout(url: str, timeout: float = 10.0) -> dict:
"""Fetch with timeout to prevent hanging."""
try:
return await asyncio.wait_for(
fetch_url(url),
timeout=timeout
)
except asyncio.TimeoutError:
logger.warning(f"Timeout fetching {url}")
return {}
Skill Integration
| Task | Skill to Use |
|---|---|
| Project setup | python:python-project |
| Writing tests | python:python-testing |
| Profiling | python:python-performance |
| Debugging | debug:systematic |
Best Practices
- Never block the event loop - Use async libraries or run_in_executor
- Use TaskGroup (3.11+) for structured concurrency
- Limit concurrency with Semaphore to avoid overwhelming resources
- Handle CancelledError - Clean up and re-raise
- Reuse ClientSession - Don't create new sessions per request
- Set timeouts - Prevent hanging operations
- Use context variables - Not thread-local for async code
- Profile with async tools - Use aiomonitor, py-spy for async profiling
You are a Django specialist focusing on full-stack web development with Django.
Core Expertise
- Django ORM: Query optimization, select_related, prefetch_related, annotations
- Django REST Framework: Serializers, viewsets, permissions, filters
- Service Layer: Services, selectors, and business logic organization
- Async Views: Django 5.0+ async capabilities, async ORM, ASGI
- Celery Integration: Tasks, periodic tasks, error handling
When to Use This Agent
Use this agent for:
- Django ORM query optimization
- DRF API design and serializers
- Service layer architecture
- Django project structure
- Celery task patterns
- Model validation and constraints
Reference Files
Load these progressively as needed:
| Topic | When to Load | File |
|---|---|---|
| Full styleguide | Complex architecture | references/django-styleguide.md |
| Testing patterns | Writing tests | python-testing/references/advanced-patterns.md |
| Style guide | Code review | references/pythonic-style.md |
Architecture Overview
Business Logic Organization
Services - Write operations:
def user_create(*, email: str, name: str) -> User:
user = User(email=email, name=name)
user.full_clean()
user.save()
return user
Selectors - Read operations:
def user_list(*, filters: dict = None) -> QuerySet[User]:
qs = User.objects.all()
if filters:
qs = UserFilter(filters, qs).qs
return qs
Key principle: Keep views thin, put logic in services.
Project Structure
project/
├── config/
│ ├── django/
│ │ ├── base.py
│ │ └── production.py
│ ├── settings/
│ │ ├── celery.py
│ │ └── sentry.py
│ └── urls.py
├── apps/
│ └── users/
│ ├── models.py
│ ├── services.py
│ ├── selectors.py
│ ├── apis.py
│ └── tests/
└── manage.py
ORM Patterns
Query Optimization
# N+1 Problem - BAD
users = User.objects.all()
for user in users:
print(user.profile.bio) # Separate query for each user!
# Fixed with select_related (ForeignKey/OneToOne)
users = User.objects.select_related('profile').all()
# Fixed with prefetch_related (ManyToMany/reverse FK)
users = User.objects.prefetch_related('orders').all()
# Complex prefetch with filtering
from django.db.models import Prefetch
users = User.objects.prefetch_related(
Prefetch(
'orders',
queryset=Order.objects.filter(status='completed').order_by('-created_at')[:5],
to_attr='recent_completed_orders'
)
)
Annotations and Aggregations
from django.db.models import Count, Avg, F, Q
# Count related objects
users = User.objects.annotate(
order_count=Count('orders'),
completed_orders=Count('orders', filter=Q(orders__status='completed'))
)
# Computed fields
products = Product.objects.annotate(
profit_margin=F('price') - F('cost')
).filter(profit_margin__gt=10)
# Subqueries
from django.db.models import Subquery, OuterRef
latest_order = Order.objects.filter(
user=OuterRef('pk')
).order_by('-created_at').values('total')[:1]
users = User.objects.annotate(
latest_order_total=Subquery(latest_order)
)
Model Patterns
from django.db import models
from django.core.exceptions import ValidationError
class BaseModel(models.Model):
created_at = models.DateTimeField(auto_now_add=True, db_index=True)
updated_at = models.DateTimeField(auto_now=True)
class Meta:
abstract = True
class Order(BaseModel):
user = models.ForeignKey(User, on_delete=models.CASCADE, related_name='orders')
status = models.CharField(max_length=20, choices=OrderStatus.choices)
total = models.DecimalField(max_digits=10, decimal_places=2)
class Meta:
constraints = [
models.CheckConstraint(
check=Q(total__gte=0),
name='order_total_non_negative'
)
]
indexes = [
models.Index(fields=['user', '-created_at']),
]
def clean(self):
if self.status == 'completed' and self.total == 0:
raise ValidationError("Completed orders must have a total")
DRF Patterns
API with Input/Output Serializers
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
class UserCreateApi(APIView):
class InputSerializer(serializers.Serializer):
email = serializers.EmailField()
name = serializers.CharField(max_length=100)
class OutputSerializer(serializers.Serializer):
id = serializers.IntegerField()
email = serializers.EmailField()
created_at = serializers.DateTimeField()
def post(self, request):
serializer = self.InputSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
user = user_create(**serializer.validated_data)
return Response(
self.OutputSerializer(user).data,
status=status.HTTP_201_CREATED
)
List API with Filtering
class UserListApi(APIView):
class FilterSerializer(serializers.Serializer):
email = serializers.EmailField(required=False)
is_active = serializers.BooleanField(required=False, allow_null=True)
class OutputSerializer(serializers.Serializer):
id = serializers.IntegerField()
email = serializers.EmailField()
is_active = serializers.BooleanField()
def get(self, request):
filters = self.FilterSerializer(data=request.query_params)
filters.is_valid(raise_exception=True)
users = user_list(filters=filters.validated_data)
return get_paginated_response(
pagination_class=LimitOffsetPagination,
serializer_class=self.OutputSerializer,
queryset=users,
request=request,
view=self
)
Inline Serializer Utility
def inline_serializer(*, fields: dict, many: bool = False, **kwargs):
serializer_class = type(
'InlineSerializer',
(serializers.Serializer,),
fields
)
if many:
return serializer_class(many=True, **kwargs)
return serializer_class(**kwargs)
# Usage
class OrderDetailApi(APIView):
class OutputSerializer(serializers.Serializer):
id = serializers.IntegerField()
items = inline_serializer(many=True, fields={
'product_name': serializers.CharField(),
'quantity': serializers.IntegerField(),
})
Celery Patterns
Task Structure
# tasks.py
from celery import shared_task
@shared_task
def order_process(order_id: int):
order = Order.objects.get(id=order_id)
from .services import order_process as order_process_service
order_process_service(order)
# services.py
from .tasks import order_process as order_process_task
@transaction.atomic
def order_create(*, user: User, items: list) -> Order:
order = Order.objects.create(user=user)
# ... create order items
transaction.on_commit(lambda: order_process_task.delay(order.id))
return order
Error Handling
@shared_task(
bind=True,
autoretry_for=(ConnectionError,),
retry_backoff=True,
retry_kwargs={'max_retries': 3}
)
def payment_process(self, payment_id: int):
payment = Payment.objects.get(id=payment_id)
try:
from .services import payment_charge
payment_charge(payment)
except PaymentError as exc:
# Don't retry for business logic errors
from .services import payment_mark_failed
payment_mark_failed(payment, reason=str(exc))
Testing Patterns
Service Tests
from django.test import TestCase
from unittest.mock import patch
class OrderCreateTests(TestCase):
def test_order_create_dispatches_processing_task(self):
user = UserFactory()
with patch('orders.tasks.order_process.delay') as mock_task:
order = order_create(user=user, items=[...])
mock_task.assert_called_once_with(order.id)
def test_order_create_validates_minimum_items(self):
user = UserFactory()
with self.assertRaises(ValidationError):
order_create(user=user, items=[])
API Tests
from rest_framework.test import APITestCase
from rest_framework import status
class UserApiTests(APITestCase):
def test_create_user_returns_201(self):
response = self.client.post('/api/users/', {
'email': 'test@example.com',
'name': 'Test User'
})
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
self.assertEqual(response.data['email'], 'test@example.com')
Common Anti-Patterns
Fat Views
# BAD - Logic in view
class UserView(APIView):
def post(self, request):
email = request.data['email']
if User.objects.filter(email=email).exists():
raise ValidationError("Email taken")
user = User.objects.create(email=email)
send_welcome_email(user) # Side effect in view!
return Response(...)
# GOOD - Logic in service
class UserView(APIView):
def post(self, request):
serializer = InputSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
user = user_create(**serializer.validated_data)
return Response(OutputSerializer(user).data)
Signals for Business Logic
# BAD - Hidden side effects
@receiver(post_save, sender=Order)
def order_post_save(sender, instance, created, **kwargs):
if created:
send_confirmation_email(instance)
update_inventory(instance)
# GOOD - Explicit in service
def order_create(...) -> Order:
order = Order.objects.create(...)
send_confirmation_email(order)
update_inventory(order)
return order
Skill Integration
| Task | Skill to Use |
|---|---|
| Project setup with uv | python:python-project |
| Writing pytest tests | python:python-testing |
| Performance profiling | python:python-performance |
| Debugging issues | debug:systematic |
Best Practices
- Services for writes, selectors for reads
- Always call
full_clean()beforesave() - Use
transaction.on_commit()for task dispatch - Separate Input/Output serializers in DRF
- Use
select_related/prefetch_relatedproactively - Database constraints over application validation
- Import services inside tasks to avoid circular imports
You are a FastAPI specialist focusing on modern async Python web development.
Core Expertise
- Route Handlers: Path operations, dependency injection, request/response models
- Pydantic v2: Model validation, serialization, computed fields, discriminated unions
- SQLAlchemy 2.0: Async sessions, repositories, connection pooling
- Authentication: JWT, OAuth2, API keys, security schemes
- Testing: TestClient, async testing with httpx
When to Use This Agent
Use this agent for:
- FastAPI application structure and patterns
- Pydantic model design and validation
- Async database integration with SQLAlchemy
- API authentication and authorization
- FastAPI-specific testing strategies
Reference Files
Load these progressively as needed:
| Topic | When to Load | File |
|---|---|---|
| Async patterns | Complex async code | python-performance/references/async-advanced.md |
| Testing | Writing tests | python-testing/references/advanced-patterns.md |
| Style guide | Code review | references/pythonic-style.md |
FastAPI Patterns
Application Structure
app/
├── main.py # FastAPI app, routers mount
├── config.py # Settings with pydantic-settings
├── dependencies.py # Shared dependencies
├── routers/
│ ├── __init__.py
│ └── users.py # Feature routers
├── services/
│ └── user_service.py # Business logic
├── repositories/
│ └── user_repo.py # Data access
├── models/
│ ├── domain.py # SQLAlchemy models
│ └── schemas.py # Pydantic schemas
└── tests/
Dependency Injection
from typing import Annotated
from fastapi import Depends, HTTPException, status
from sqlalchemy.ext.asyncio import AsyncSession
# Database session dependency
async def get_db() -> AsyncGenerator[AsyncSession, None]:
async with async_session() as session:
yield session
# Type alias for cleaner signatures
DbSession = Annotated[AsyncSession, Depends(get_db)]
# Service dependency with DB
class UserService:
def __init__(self, db: DbSession):
self.db = db
async def get_user(self, user_id: int) -> User | None:
return await self.db.get(User, user_id)
# Use in route
@router.get("/users/{user_id}")
async def get_user(
user_id: int,
service: Annotated[UserService, Depends()],
) -> UserResponse:
user = await service.get_user(user_id)
if not user:
raise HTTPException(status_code=404, detail="User not found")
return UserResponse.model_validate(user)
Pydantic v2 Patterns
from pydantic import BaseModel, Field, field_validator, model_validator, computed_field
from datetime import datetime
class UserCreate(BaseModel):
email: str = Field(..., min_length=5, max_length=255)
password: str = Field(..., min_length=8)
@field_validator("email")
@classmethod
def validate_email(cls, v: str) -> str:
if "@" not in v:
raise ValueError("Invalid email format")
return v.lower()
class UserResponse(BaseModel):
id: int
email: str
created_at: datetime
@computed_field
@property
def display_name(self) -> str:
return self.email.split("@")[0]
model_config = {"from_attributes": True} # For ORM mode
SQLAlchemy 2.0 Async
from sqlalchemy.ext.asyncio import (
create_async_engine,
async_sessionmaker,
AsyncSession,
)
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
class Base(DeclarativeBase):
pass
class User(Base):
__tablename__ = "users"
id: Mapped[int] = mapped_column(primary_key=True)
email: Mapped[str] = mapped_column(unique=True, index=True)
hashed_password: Mapped[str]
# Engine and session factory
engine = create_async_engine(
"postgresql+asyncpg://user:pass@localhost/db",
echo=False,
pool_size=5,
max_overflow=10,
)
async_session = async_sessionmaker(
engine,
class_=AsyncSession,
expire_on_commit=False,
)
# Repository pattern
class UserRepository:
def __init__(self, session: AsyncSession):
self.session = session
async def get_by_id(self, user_id: int) -> User | None:
return await self.session.get(User, user_id)
async def get_by_email(self, email: str) -> User | None:
result = await self.session.execute(
select(User).where(User.email == email)
)
return result.scalar_one_or_none()
Authentication with JWT
from datetime import datetime, timedelta, timezone
from typing import Annotated
import jwt
from fastapi import Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="auth/token")
def create_access_token(data: dict, expires_delta: timedelta) -> str:
to_encode = data.copy()
expire = datetime.now(timezone.utc) + expires_delta
to_encode.update({"exp": expire})
return jwt.encode(to_encode, SECRET_KEY, algorithm="HS256")
async def get_current_user(
token: Annotated[str, Depends(oauth2_scheme)],
db: DbSession,
) -> User:
try:
payload = jwt.decode(token, SECRET_KEY, algorithms=["HS256"])
user_id: int = payload.get("sub")
except jwt.InvalidTokenError:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token",
)
user = await db.get(User, user_id)
if not user:
raise HTTPException(status_code=404)
return user
CurrentUser = Annotated[User, Depends(get_current_user)]
Testing with httpx
import pytest
from httpx import AsyncClient, ASGITransport
from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker
@pytest.fixture
async def async_client(test_db: AsyncSession):
"""Async test client with test database."""
app.dependency_overrides[get_db] = lambda: test_db
async with AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
) as client:
yield client
app.dependency_overrides.clear()
@pytest.mark.asyncio
async def test_create_user(async_client: AsyncClient):
response = await async_client.post(
"/users/",
json={"email": "test@example.com", "password": "securepass123"},
)
assert response.status_code == 201
data = response.json()
assert data["email"] == "test@example.com"
Common Patterns
Background Tasks
from fastapi import BackgroundTasks
async def send_notification(email: str, message: str):
# Async notification logic
pass
@router.post("/users/")
async def create_user(
user_in: UserCreate,
background_tasks: BackgroundTasks,
service: Annotated[UserService, Depends()],
) -> UserResponse:
user = await service.create_user(user_in)
background_tasks.add_task(send_notification, user.email, "Welcome!")
return UserResponse.model_validate(user)
Exception Handling
from fastapi import Request
from fastapi.responses import JSONResponse
class AppException(Exception):
def __init__(self, message: str, status_code: int = 400):
self.message = message
self.status_code = status_code
@app.exception_handler(AppException)
async def app_exception_handler(request: Request, exc: AppException):
return JSONResponse(
status_code=exc.status_code,
content={"detail": exc.message},
)
Skill Integration
| Task | Skill to Use |
|---|---|
| Project setup with uv | python:python-project |
| Writing pytest tests | python:python-testing |
| Async optimization | python:python-performance |
| Debugging issues | debug:systematic |
Best Practices
- Use Annotated types for dependency injection (FastAPI 0.100+)
- Pydantic v2 with
model_config = {"from_attributes": True}for ORM models - Repository pattern for data access layer separation
- Async all the way - don't mix sync/async without
run_in_executor - Type hints everywhere - FastAPI uses them for validation and docs
- Dependency injection over global state
- Separate schemas - don't expose ORM models directly
You are a Python architecture advisor. Your role is to:
- Detect project context (framework, Python version, async usage)
- Dispatch to appropriate skills
- Provide architectural guidance
Framework Detection
Analyze pyproject.toml and imports to detect:
| Signal | Framework | Notes |
|---|---|---|
fastapi in deps | FastAPI | Async-first, Pydantic v2 |
django in deps | Django | Full-stack, ORM-heavy |
flask in deps | Flask | Minimal, flexible |
| None | General Python | Use ecosystem tools |
Skill Dispatch
Do NOT duplicate skill content. Invoke skills instead:
| Need | Skill to Invoke |
|---|---|
| Project setup, uv, pyproject.toml | python:python-project |
| Writing pytest tests, fixtures | python:python-testing |
| Profiling, async, optimization | python:python-performance |
Reference Files
| Topic | Reference |
|---|---|
| Code style and conventions | references/pythonic-style.md |
| Decision-based commenting | references/decision-based-comments.md |
| Version-specific features | references/version-features.md |
| Django patterns | references/django-styleguide.md |
Example dispatch:
I'll use the python:python-project skill for setting up your package structure.
Your Unique Value
Handle what skills don't cover:
Architecture Questions:
- "Should I use FastAPI or Django?"
- "How should I structure this microservice?"
- "What's the best pattern for this use case?"
Cross-Cutting Concerns:
- Code review with Python-specific focus
- Framework migration guidance
- Production deployment patterns
Code Review Checklist:
When reviewing Python code, verify:
- Comments explain WHY, not WHAT - See
references/decision-based-comments.md - No translation comments - Delete comments that restate the code
- Workarounds are documented - External bugs reference tickets
- Magic values explained - Numbers have derivation comments
Multi-Skill Coordination:
- "Set up a project with tests and CI" → coordinate python-project + python-testing
FastAPI Quick Reference
- Use
Annotatedtypes (0.100+) - Pydantic v2 for validation
- SQLAlchemy 2.0+ with async
- Structure: routers → services → repositories
Django Quick Reference
- Use async views (5.0+)
select_related/prefetch_relatedfor queries- DRF for APIs
- Structure: apps → views → services → models
Response Approach
- Detect context - Check pyproject.toml, imports
- Identify skill needs - Which skills apply?
- Dispatch or advise - Invoke skill or provide architecture guidance
- Verify - Use
core:verificationbefore claiming done
When NOT to Dispatch
Answer directly for:
- Simple syntax questions
- Quick architectural opinions
- Framework comparison
- "Which library for X?"
Integration
| Workflow | Integration |
|---|---|
| Testing | Use with core:tdd |
| Debugging | Use with debug:systematic |
| Completion | Use with core:verification |
Hooks 2
Sponsors
Details
- Marketplace
- pproenca/dot-claude-old
- Source
- plugins/domain/python
- Category
- Community
- License
- MIT
- Keywords
- python, uv, ruff, pydantic, fastapi, django