
open-python-skills
AI skill for Python backend development - FastAPI, SQLAlchemy, Redis, security patterns
- GitHub stars
- 10
Install
/plugin install open-python-skills@open-python-skillsRun it inside Claude Code after adding the marketplace with /plugin marketplace add jiatastic/open-python-skills.
Contents
Discovered from sourceSkills 8
commit-message
Analyze git changes and generate context-aware commit messages following Conventional Commits.
Quick Start
# Analyze all changes
python3 .shared/commit-message/scripts/analyze_changes.py --analyze
# Get batch commit suggestions
python3 .shared/commit-message/scripts/analyze_changes.py --batch
# Generate message for specific files
python3 .shared/commit-message/scripts/analyze_changes.py --generate "src/api/*.py"
Commands
| Command | Description |
|---|---|
--analyze | Show all changed files with status and categories |
--batch | Suggest how to split changes into multiple commits |
--generate [pattern] | Generate commit message for matching files |
--staged | Only analyze staged changes (default: all changes) |
Commit Types
| Type | Description | Example |
|---|---|---|
feat | New feature | feat(api): add user authentication |
fix | Bug fix | fix(db): resolve connection timeout |
refactor | Code restructuring | refactor(utils): simplify helper functions |
docs | Documentation | docs: update README |
test | Tests | test(api): add user endpoint tests |
chore | Maintenance | chore: update dependencies |
style | Formatting | style: fix linting errors |
Batch Commit Workflow
When you have multiple unrelated changes:
- Run
--batchto see suggested commit groups - Stage files for first commit:
git add <files> - Commit with suggested message
- Repeat for remaining groups
Grouping Strategy
Files are grouped by:
- Directory/Module:
src/api/,tests/,docs/ - Change Type: Added vs Modified vs Deleted
- Semantic Relationship: Related files together
Context-Aware Commit Messages
Note: The
analyze_changes.pyscript provides file grouping and basic suggestions. Use its output as a starting point, then readgit diffto understand the actual changes and generate context-aware messages following the examples below.
When generating commit messages, analyze the actual code changes to infer business context. Don't just describe files—describe what the changes accomplish.
Scope Guidelines
The scope should reflect the business module or feature, not just the directory:
| Scope Type | Example | When to Use |
|---|---|---|
| Feature/Module | companion, calendar, inbox | Changes to a specific product feature |
| Platform | ios, android, web | Platform-specific changes |
| Integration | outlook, gmail, slack | Third-party integration changes |
| Component | auth, api, db | Core infrastructure changes |
Input/Output Examples
Example 1: New Feature
Input (code changes):
+ src/companion/pages/AvailabilityDetailPage.tsx
+ src/companion/pages/AvailabilityActionsPage.tsx
+ src/companion/components/AvailabilityCard.tsx
M src/companion/navigation/routes.ts
Output:
feat(companion): add availability detail and actions pages for ios
- New AvailabilityDetailPage showing time slot details
- New AvailabilityActionsPage for booking/canceling
- AvailabilityCard component for list display
- Updated navigation routes
Example 2: Bug Fix
Input (code changes):
M src/integrations/outlook/email_sender.py
M src/integrations/outlook/auth.py
Output:
fix(outlook): resolve email sending failures due to token expiration
Refresh OAuth token before sending when close to expiry
Example 3: Multi-platform Change
Input (code changes):
M ios/Calendar/CalendarView.swift
M android/calendar/CalendarFragment.kt
M web/src/calendar/Calendar.tsx
Output:
feat(calendar): add week view across all platforms
Implement consistent week view UI for iOS, Android, and web
Example 4: Chore/Maintenance
Input (code changes):
M package.json
M yarn.lock
M requirements.txt
Output:
chore(deps): update dependencies to latest versions
Writing Good Descriptions
| Bad (Generic) | Good (Context-Aware) |
|---|---|
feat: add new file | feat(payments): add Stripe webhook handler |
fix: fix bug | fix(auth): prevent session timeout on mobile |
chore: update code | chore(ci): reduce build time with parallel jobs |
refactor: refactor utils | refactor(api): extract rate limiting to middleware |
Key Principles
- Read the code - Understand what the changes actually do
- Identify the feature - What user-facing or system capability is affected?
- Be specific - Include relevant details (platform, integration, component)
- Use active voice - "add", "fix", "update", not "added", "fixed", "updated"
- Keep it concise - First line under 72 characters
Error Handling
Production-ready error handling for Python APIs using the Let it crash philosophy.
Design Philosophy
Let it crash - Don't be defensive. Let exceptions propagate naturally and handle them at boundaries.
# BAD - Too defensive, obscures errors
@app.get("/users/{user_id}")
async def get_user(user_id: int):
try:
user = await user_service.get(user_id)
if not user:
raise HTTPException(404, "Not found")
return user
except DatabaseError as e:
raise HTTPException(500, "Database error")
except Exception as e:
logger.exception("Unexpected error")
raise HTTPException(500, "Internal error")
# GOOD - Let exceptions propagate, handle at boundary
@app.get("/users/{user_id}")
async def get_user(user_id: int):
user = await user_service.get(user_id)
if not user:
raise UserNotFoundError(user_id)
return user
Core Principles
- Raise low, catch high - Throw exceptions where errors occur, handle at API boundaries
- Domain exceptions - Create semantic exceptions, not generic ones
- Global handlers - Use
@app.exception_handler()for centralized error formatting - No bare except - Always catch specific exceptions
- Preserve context - Use
raise ... from errorto keep original traceback
Quick Start
1. Define Domain Exceptions
from enum import StrEnum
class ErrorCode(StrEnum):
USER_NOT_FOUND = "user_not_found"
INVALID_CREDENTIALS = "invalid_credentials"
RATE_LIMITED = "rate_limited"
class DomainError(Exception):
"""Base exception for all domain errors."""
def __init__(self, code: ErrorCode, message: str, status_code: int = 400):
self.code = code
self.message = message
self.status_code = status_code
super().__init__(message)
class UserNotFoundError(DomainError):
def __init__(self, user_id: int):
super().__init__(
code=ErrorCode.USER_NOT_FOUND,
message=f"User {user_id} not found",
status_code=404
)
2. Define Error Response Schema
from pydantic import BaseModel
class ErrorDetail(BaseModel):
code: str
message: str
request_id: str | None = None
class ErrorResponse(BaseModel):
error: ErrorDetail
3. Register Global Handlers
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from fastapi.exceptions import RequestValidationError
from starlette.exceptions import HTTPException as StarletteHTTPException
app = FastAPI()
@app.exception_handler(DomainError)
async def domain_error_handler(request: Request, exc: DomainError):
return JSONResponse(
status_code=exc.status_code,
content={"error": {"code": exc.code, "message": exc.message}}
)
@app.exception_handler(StarletteHTTPException)
async def http_exception_handler(request: Request, exc: StarletteHTTPException):
return JSONResponse(
status_code=exc.status_code,
content={"error": {"code": "http_error", "message": str(exc.detail)}}
)
@app.exception_handler(RequestValidationError)
async def validation_error_handler(request: Request, exc: RequestValidationError):
return JSONResponse(
status_code=422,
content={"error": {"code": "validation_error", "message": "Invalid request"}}
)
@app.exception_handler(Exception)
async def generic_error_handler(request: Request, exc: Exception):
# Log full error internally
logger.exception("Unhandled error")
# Return safe message to client
return JSONResponse(
status_code=500,
content={"error": {"code": "internal_error", "message": "Internal server error"}}
)
4. Use in Routes
@app.get("/users/{user_id}")
async def get_user(user_id: int):
user = await user_service.get(user_id)
if not user:
raise UserNotFoundError(user_id)
return user
When to Catch Exceptions
Only catch exceptions in these cases:
| Situation | Example |
|---|---|
| Need to retry | tenacity.retry() for transient failures |
| Need to transform | Wrap third-party SDK errors as domain errors |
| Need to clean up | Use finally or context managers |
| Need to add context | raise DomainError(...) from original |
Python + FastAPI Integration
| Layer | Responsibility |
|---|---|
| Service/Domain | Raise domain exceptions (UserNotFoundError) |
| Routes | Let exceptions propagate (no try/except) |
| Exception Handlers | Transform to HTTP responses |
| Middleware | Add request context (request_id, timing) |
Common Patterns
Third-Party SDK Wrapping
import httpx
from tenacity import retry, stop_after_attempt, wait_exponential
class ExternalServiceError(DomainError):
def __init__(self, service: str, original: Exception):
super().__init__(
code=ErrorCode.EXTERNAL_SERVICE_ERROR,
message=f"{service} unavailable",
status_code=503
)
self.__cause__ = original
@retry(stop=stop_after_attempt(3), wait=wait_exponential())
async def call_payment_api(data: dict):
try:
async with httpx.AsyncClient(timeout=10.0) as client:
response = await client.post("https://api.payment.com/charge", json=data)
response.raise_for_status()
return response.json()
except httpx.HTTPError as e:
raise ExternalServiceError("Payment API", e) from e
Background Task Error Handling
from fastapi import BackgroundTasks
async def safe_background_task(task_func, *args, **kwargs):
try:
await task_func(*args, **kwargs)
except Exception as e:
logger.exception(f"Background task failed: {e}")
# Optional: send to dead letter queue or alerting
@app.post("/orders")
async def create_order(order: Order, background_tasks: BackgroundTasks):
result = await order_service.create(order)
background_tasks.add_task(safe_background_task, send_confirmation_email, result.id)
return result
Troubleshooting
| Issue | Cause | Fix |
|---|---|---|
| Stack trace in response | No generic handler | Add @app.exception_handler(Exception) |
| Lost original error | Missing from | Use raise NewError() from original |
| Validation errors leak | Default handler | Override RequestValidationError handler |
| Silent failures | Swallowed exceptions | Let exceptions propagate, handle at boundary |
References
- Python Patterns - Exception design, when to catch, SDK wrapping
- FastAPI Patterns - HTTPException, global handlers, middleware
- Pydantic Patterns - ValidationError, raise in validators
- Asyncio Patterns - TaskGroup, timeout, background tasks
- FastAPI Docs: Handling Errors
- Pydantic Docs: Error Handling
excalidraw-ai
Generate Excalidraw diagrams by writing JSON directly. No templates needed - you have full control over every element.
When to Use This Skill
Use this skill when you need to:
- Create architecture diagrams, flowcharts, or mind maps
- Visualize system designs, data flows, or processes
- Generate diagrams from code analysis or documentation
- Create custom diagrams with precise control over layout and styling
Excalidraw JSON Schema
An Excalidraw file is a JSON object with this structure:
{
"type": "excalidraw",
"version": 2,
"source": "https://excalidraw.com",
"elements": [ /* array of element objects */ ],
"appState": {
"viewBackgroundColor": "#ffffff",
"gridSize": null
},
"files": {}
}
Element Types
Rectangle
{
"id": "unique-id-1",
"type": "rectangle",
"x": 100,
"y": 100,
"width": 200,
"height": 100,
"strokeColor": "#1971c2",
"backgroundColor": "#a5d8ff",
"fillStyle": "solid",
"strokeWidth": 2,
"strokeStyle": "solid",
"roughness": 1,
"opacity": 100,
"groupIds": [],
"angle": 0,
"seed": 12345,
"isDeleted": false
}
Ellipse
{
"id": "unique-id-2",
"type": "ellipse",
"x": 100,
"y": 100,
"width": 150,
"height": 100,
"strokeColor": "#7c3aed",
"backgroundColor": "#ede9fe",
"fillStyle": "solid",
"strokeWidth": 2,
"roughness": 1
}
Diamond
{
"id": "unique-id-3",
"type": "diamond",
"x": 100,
"y": 100,
"width": 120,
"height": 120,
"strokeColor": "#dc2626",
"backgroundColor": "#fee2e2",
"fillStyle": "solid"
}
Text
{
"id": "unique-id-4",
"type": "text",
"x": 110,
"y": 140,
"width": 180,
"height": 25,
"text": "Your Label Here",
"fontSize": 20,
"fontFamily": 1,
"textAlign": "center",
"verticalAlign": "middle",
"strokeColor": "#1e1e1e",
"backgroundColor": "transparent"
}
Arrow
{
"id": "unique-id-5",
"type": "arrow",
"x": 300,
"y": 150,
"width": 100,
"height": 50,
"strokeColor": "#1971c2",
"strokeWidth": 2,
"points": [[0, 0], [100, 50]],
"startBinding": {
"elementId": "source-element-id",
"focus": 0.5,
"gap": 5
},
"endBinding": {
"elementId": "target-element-id",
"focus": 0.5,
"gap": 5
},
"startArrowhead": null,
"endArrowhead": "arrow"
}
Line
{
"id": "unique-id-6",
"type": "line",
"x": 100,
"y": 100,
"width": 200,
"height": 0,
"strokeColor": "#868e96",
"strokeWidth": 1,
"points": [[0, 0], [200, 0]],
"startArrowhead": null,
"endArrowhead": null
}
Element Properties Reference
| Property | Type | Description |
|---|---|---|
id | string | Unique identifier (use UUID or similar) |
type | string | rectangle, ellipse, diamond, text, arrow, line |
x, y | number | Position coordinates |
width, height | number | Dimensions |
strokeColor | string | Border/stroke color (hex) |
backgroundColor | string | Fill color (hex) or "transparent" |
fillStyle | string | "solid", "hachure", "cross-hatch" |
strokeWidth | number | Line thickness (1, 2, 4) |
strokeStyle | string | "solid", "dashed", "dotted" |
roughness | number | 0 = sharp, 1 = normal, 2 = sketchy |
opacity | number | 0-100 |
angle | number | Rotation in radians |
groupIds | array | Group element IDs together |
seed | number | Random seed for hand-drawn effect |
Text Properties
| Property | Type | Description |
|---|---|---|
text | string | The text content |
fontSize | number | Font size in pixels |
fontFamily | number | 1 = Virgil (hand-drawn), 2 = Helvetica, 3 = Cascadia |
textAlign | string | "left", "center", "right" |
verticalAlign | string | "top", "middle" |
Arrow Properties
| Property | Type | Description |
|---|---|---|
points | array | Array of [x, y] coordinates relative to element origin |
startBinding | object | Connection to source element |
endBinding | object | Connection to target element |
startArrowhead | string | null, "arrow", "bar", "dot", "triangle" |
endArrowhead | string | null, "arrow", "bar", "dot", "triangle" |
Color Palette Reference
Component-Based Colors (Recommended for Architecture Diagrams)
| Component Type | Stroke | Background | Use For |
|---|---|---|---|
| Database | #7c3aed | #ede9fe | PostgreSQL, MySQL, MongoDB |
| Cache | #dc2626 | #fee2e2 | Redis, Memcached |
| Queue | #16a34a | #dcfce7 | Kafka, RabbitMQ, SQS |
| Load Balancer | #0891b2 | #cffafe | Nginx, HAProxy, ALB |
| Gateway | #475569 | #f1f5f9 | API Gateway, Kong |
| CDN | #06b6d4 | #e0f2fe | CloudFront, Fastly |
| Auth | #e11d48 | #ffe4e6 | OAuth, IAM, Cognito |
| Storage | #d97706 | #fef3c7 | S3, Blob Storage |
| Service | #2563eb | #dbeafe | Backend services, APIs |
| Container | #0284c7 | #bae6fd | Docker, Kubernetes |
| Function | #f59e0b | #fef3c7 | Lambda, Serverless |
| Monitoring | #84cc16 | #ecfccb | Prometheus, Grafana |
| Web App | #4f46e5 | #e0e7ff | React, Vue, Frontend |
| Mobile | #6366f1 | #eef2ff | iOS, Android |
Theme Colors
Modern (Default)
- Primary:
#1971c2/#e7f5ff - Neutral:
#64748b/#f1f5f9
Sketchy (Hand-drawn)
- Primary:
#495057/#f8f9fa - Use
roughness: 2andfontFamily: 3
Technical
- Primary:
#2f9e44/#ebfbee - Use
strokeStyle: "dashed"for connections
Icon Libraries (305 Components Total)
Professional icons are available in .excalidrawlib files in the assets/ directory:
| Library | Components | Content |
|---|---|---|
aws-architecture-icons.excalidrawlib | 248 | AWS service icons (Lambda, S3, EC2, RDS, DynamoDB, SQS, etc.) |
system-design.excalidrawlib | 24 | System design elements (server, database, storage, etc.) |
drwnio.excalidrawlib | 18 | Draw.io style icons |
system-design-template.excalidrawlib | 8 | Pre-built templates (steps, flow, db, etc.) |
software-architecture.excalidrawlib | 7 | Software architecture components |
Using Library Components
Library files contain element arrays that can be embedded directly into your diagram. Each library item is an array of Excalidraw elements that form a component.
To use: Parse the library JSON, find the component you need, adjust positions, and include in your diagram's elements array.
Layout Guidelines
Architecture Diagram Layout
Organize components in layers (top to bottom):
Layer 1: Client/External (y: 100)
Layer 2: Edge/CDN (y: 280)
Layer 3: Gateway/Load Balancer (y: 460)
Layer 4: Services (y: 640)
Layer 5: Data/Cache (y: 820)
Layer 6: Infrastructure (y: 1000)
Spacing recommendations:
- Horizontal spacing between components: 240px
- Vertical spacing between layers: 180px
- Component width: 180-220px
- Component height: 80-100px
Flowchart Layout
Arrange left-to-right or top-to-bottom:
- Step spacing: 200px
- Use rectangles for processes
- Use diamonds for decisions
- Use ellipses for start/end
Mind Map Layout
Radial layout from center:
- Center node at (400, 300)
- Child nodes at radius 250px
- Distribute evenly using angle calculations
Complete Example
{
"type": "excalidraw",
"version": 2,
"source": "https://excalidraw.com",
"elements": [
{
"id": "api-gateway",
"type": "rectangle",
"x": 100,
"y": 100,
"width": 180,
"height": 80,
"strokeColor": "#475569",
"backgroundColor": "#f1f5f9",
"fillStyle": "solid",
"strokeWidth": 2,
"roughness": 0,
"opacity": 100,
"groupIds": ["group-1"],
"seed": 1234
},
{
"id": "api-gateway-label",
"type": "text",
"x": 140,
"y": 130,
"width": 100,
"height": 25,
"text": "API Gateway",
"fontSize": 16,
"fontFamily": 1,
"textAlign": "center",
"strokeColor": "#1e1e1e",
"groupIds": ["group-1"]
},
{
"id": "redis",
"type": "diamond",
"x": 100,
"y": 280,
"width": 180,
"height": 100,
"strokeColor": "#dc2626",
"backgroundColor": "#fee2e2",
"fillStyle": "solid",
"strokeWidth": 2,
"roughness": 0,
"seed": 5678
},
{
"id": "redis-label",
"type": "text",
"x": 155,
"y": 320,
"width": 70,
"height": 20,
"text": "Redis",
"fontSize": 16,
"fontFamily": 1,
"textAlign": "center",
"strokeColor": "#1e1e1e"
},
{
"id": "arrow-1",
"type": "arrow",
"x": 190,
"y": 180,
"width": 0,
"height": 100,
"strokeColor": "#64748b",
"strokeWidth": 2,
"points": [[0, 0], [0, 100]],
"startBinding": {"elementId": "api-gateway", "focus": 0.5, "gap": 5},
"endBinding": {"elementId": "redis", "focus": 0.5, "gap": 5},
"endArrowhead": "arrow"
}
],
"appState": {
"viewBackgroundColor": "#ffffff",
"gridSize": null
},
"files": {}
}
Output
Save your diagram as a .excalidraw or .json file. It can be:
- Imported directly into excalidraw.com
- Embedded in documentation or web apps
- Converted to PNG/SVG using Excalidraw's export features
Tips for AI Agents
- Generate unique IDs: Use UUIDs or sequential IDs for each element
- Group related elements: Use
groupIdsto keep shapes and their labels together - Use bindings for arrows: Connect arrows to elements using
startBindingandendBinding - Apply consistent styling: Use the color palette for component types
- Calculate text positions: Center text within shapes by adjusting x/y based on text length
- Set seed values: Use
seed: hash(id)for consistent hand-drawn rendering
python-backend
Production-ready Python backend patterns for FastAPI, SQLAlchemy, and Upstash.
When to Use This Skill
- Building REST APIs with FastAPI
- Implementing JWT/OAuth2 authentication
- Setting up SQLAlchemy async databases
- Integrating Redis/Upstash caching and rate limiting
- Refactoring AI-generated Python code
- Designing API patterns and project structure
Core Principles
- Async-first - Use async/await for I/O operations
- Type everything - Pydantic models for validation
- Dependency injection - Use FastAPI's Depends()
- Fail fast - Validate early, use HTTPException
- Security by default - Never trust user input
Quick Patterns
Project Structure
src/
├── auth/
│ ├── router.py # endpoints
│ ├── schemas.py # pydantic models
│ ├── models.py # db models
│ ├── service.py # business logic
│ └── dependencies.py
├── posts/
│ └── ...
├── config.py
├── database.py
└── main.py
Async Routes
# BAD - blocks event loop
@router.get("/")
async def bad():
time.sleep(10) # Blocking!
# GOOD - runs in threadpool
@router.get("/")
def good():
time.sleep(10) # OK in sync function
# BEST - non-blocking
@router.get("/")
async def best():
await asyncio.sleep(10) # Non-blocking
Pydantic Validation
from pydantic import BaseModel, EmailStr, Field
class UserCreate(BaseModel):
email: EmailStr
username: str = Field(min_length=3, max_length=50, pattern="^[a-zA-Z0-9_]+$")
age: int = Field(ge=18)
Dependency Injection
async def get_current_user(token: str = Depends(oauth2_scheme)) -> User:
payload = decode_token(token)
user = await get_user(payload["sub"])
if not user:
raise HTTPException(401, "User not found")
return user
@router.get("/me")
async def get_me(user: User = Depends(get_current_user)):
return user
SQLAlchemy Async
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
engine = create_async_engine(DATABASE_URL, pool_pre_ping=True)
SessionLocal = async_sessionmaker(engine, expire_on_commit=False)
async def get_session() -> AsyncGenerator[AsyncSession, None]:
async with SessionLocal() as session:
yield session
Redis Caching
from upstash_redis import Redis
redis = Redis.from_env()
@app.get("/data/{id}")
def get_data(id: str):
cached = redis.get(f"data:{id}")
if cached:
return cached
data = fetch_from_db(id)
redis.setex(f"data:{id}", 600, data)
return data
Rate Limiting
from upstash_ratelimit import Ratelimit, SlidingWindow
ratelimit = Ratelimit(
redis=Redis.from_env(),
limiter=SlidingWindow(max_requests=10, window=60),
)
@app.get("/api/resource")
def protected(request: Request):
result = ratelimit.limit(request.client.host)
if not result.allowed:
raise HTTPException(429, "Rate limit exceeded")
return {"data": "..."}
Reference Documents
For detailed patterns, see:
| Document | Content |
|---|---|
references/fastapi_patterns.md | Project structure, async, Pydantic, dependencies, testing |
references/security_patterns.md | JWT, OAuth2, password hashing, CORS, API keys |
references/database_patterns.md | SQLAlchemy async, transactions, eager loading, migrations |
references/upstash_patterns.md | Redis, rate limiting, QStash background jobs |
Resources
Logfire
Structured observability for Python using Pydantic Logfire - fast setup, powerful features, OpenTelemetry-compatible.
Quick Start
uv pip install logfire
import logfire
logfire.configure(service_name="my-api", service_version="1.0.0")
logfire.info("Application started")
Core Patterns
1. Service Configuration
Always set service metadata at startup:
import logfire
logfire.configure(
service_name="backend",
service_version="1.0.0",
environment="production",
console=False, # Disable console output in production
send_to_logfire=True, # Send to Logfire platform
)
2. Framework Instrumentation
Instrument frameworks before creating clients/apps:
import logfire
from fastapi import FastAPI
# Configure FIRST
logfire.configure(service_name="backend")
# Then instrument
logfire.instrument_fastapi()
logfire.instrument_httpx()
logfire.instrument_sqlalchemy()
# Then create app
app = FastAPI()
3. Log Levels and Structured Logging
# All log levels (trace → fatal)
logfire.trace("Detailed trace", step=1)
logfire.debug("Debug context", variable=locals())
logfire.info("User action", action="login", success=True)
logfire.notice("Important event", event_type="milestone")
logfire.warn("Potential issue", threshold_exceeded=True)
logfire.error("Operation failed", error_code=500)
logfire.fatal("Critical failure", component="database")
# Python 3.11+ f-string magic (auto-extracts variables)
user_id = 123
status = "active"
logfire.info(f"User {user_id} status: {status}")
# Equivalent to: logfire.info("User {user_id}...", user_id=user_id, status=status)
# Exception logging with automatic traceback
try:
risky_operation()
except Exception:
logfire.exception("Operation failed", context="extra_info")
4. Manual Spans
# Spans for tracing operations
with logfire.span("Process order {order_id}", order_id="ORD-123"):
logfire.info("Validating cart")
# ... processing logic
logfire.info("Order complete")
# Dynamic span attributes
with logfire.span("Database query") as span:
results = execute_query()
span.set_attribute("result_count", len(results))
span.message = f"Query returned {len(results)} results"
5. Custom Metrics
# Counter - monotonically increasing
request_counter = logfire.metric_counter("http.requests", unit="1")
request_counter.add(1, {"endpoint": "/api/users", "method": "GET"})
# Gauge - current value
temperature = logfire.metric_gauge("temperature", unit="°C")
temperature.set(23.5)
# Histogram - distribution of values
latency = logfire.metric_histogram("request.duration", unit="ms")
latency.record(45.2, {"endpoint": "/api/data"})
6. LLM Observability
import logfire
from pydantic_ai import Agent
logfire.configure()
logfire.instrument_pydantic_ai() # Traces all agent interactions
agent = Agent("openai:gpt-4o", system_prompt="You are helpful.")
result = agent.run_sync("Hello!")
7. Suppress Noisy Instrumentation
# Suppress entire scope (e.g., noisy library)
logfire.suppress_scopes("google.cloud.bigquery.opentelemetry_tracing")
# Suppress specific code block
with logfire.suppress_instrumentation():
client.get("https://internal-healthcheck.local") # Not traced
8. Sensitive Data Scrubbing
import logfire
# Add custom patterns to scrub
logfire.configure(
scrubbing=logfire.ScrubbingOptions(
extra_patterns=["api_key", "secret", "token"]
)
)
# Custom callback for fine-grained control
def scrubbing_callback(match: logfire.ScrubMatch):
if match.path == ("attributes", "safe_field"):
return match.value # Don't scrub this field
return None # Use default scrubbing
logfire.configure(
scrubbing=logfire.ScrubbingOptions(callback=scrubbing_callback)
)
9. Sampling for High-Traffic Services
import logfire
# Sample 50% of traces
logfire.configure(sampling=logfire.SamplingOptions(head=0.5))
# Disable metrics to reduce volume
logfire.configure(metrics=False)
10. Testing
import logfire
from logfire.testing import CaptureLogfire
def test_user_creation(capfire: CaptureLogfire):
create_user("Alice", "alice@example.com")
spans = capfire.exporter.exported_spans
assert len(spans) >= 1
assert spans[0].attributes["user_name"] == "Alice"
capfire.exporter.clear() # Clean up for next test
Available Integrations
| Category | Integration | Method |
|---|---|---|
| Web | FastAPI | logfire.instrument_fastapi(app) |
| Starlette | logfire.instrument_starlette(app) | |
| Django | logfire.instrument_django() | |
| Flask | logfire.instrument_flask(app) | |
| AIOHTTP Server | logfire.instrument_aiohttp_server() | |
| ASGI | logfire.instrument_asgi(app) | |
| WSGI | logfire.instrument_wsgi(app) | |
| HTTP | HTTPX | logfire.instrument_httpx() |
| Requests | logfire.instrument_requests() | |
| AIOHTTP Client | logfire.instrument_aiohttp_client() | |
| Database | SQLAlchemy | logfire.instrument_sqlalchemy(engine) |
| Asyncpg | logfire.instrument_asyncpg() | |
| Psycopg | logfire.instrument_psycopg() | |
| Redis | logfire.instrument_redis() | |
| PyMongo | logfire.instrument_pymongo() | |
| LLM | Pydantic AI | logfire.instrument_pydantic_ai() |
| OpenAI | logfire.instrument_openai() | |
| Anthropic | logfire.instrument_anthropic() | |
| MCP | logfire.instrument_mcp() | |
| Tasks | Celery | logfire.instrument_celery() |
| AWS Lambda | logfire.instrument_aws_lambda() | |
| Logging | Standard logging | logfire.instrument_logging() |
| Structlog | logfire.instrument_structlog() | |
| Loguru | logfire.instrument_loguru() | |
logfire.instrument_print() | ||
| Other | Pydantic | logfire.instrument_pydantic() |
| System Metrics | logfire.instrument_system_metrics() |
Common Pitfalls
| Issue | Symptom | Fix |
|---|---|---|
| Missing service name | Spans hard to find in UI | Set service_name in configure() |
| Late instrumentation | No spans captured | Call configure() before creating clients |
| High-cardinality attrs | Storage explosion | Use IDs, not full payloads as attributes |
| Console noise | Logs pollute stdout | Set console=False in production |
References
- Configuration Options - All
configure()parameters - Integrations Guide - Framework-specific setup
- Metrics Guide - Counter, gauge, histogram, system metrics
- Advanced Patterns - Sampling, scrubbing, suppression, testing
- Pitfalls & Troubleshooting - Common issues and solutions
- Official Docs
pydantic
Type-driven validation and serialization using Pydantic models.
Overview
Pydantic validates data using Python type hints and provides rich serialization via model_dump() and JSON schema output.
When to Use
- Validating request/response payloads
- Normalizing untrusted input
- Generating JSON schema for docs
Quick Start
uv pip install pydantic
from pydantic import BaseModel
class User(BaseModel):
id: int
email: str
user = User(id=1, email="a@example.com")
Core Patterns
- Typed fields: strict schema definitions.
- Field validators: custom validation logic.
- Model validators: cross-field checks.
- Serialization:
model_dump()andmodel_dump_json(). - Settings: environment-driven config via
BaseSettings.
Example: field_validator
from pydantic import BaseModel, field_validator
class Model(BaseModel):
name: str
@field_validator("name")
@classmethod
def ensure_not_empty(cls, v: str):
if not v:
raise ValueError("name required")
return v
Example: model_validate + model_dump
from pydantic import BaseModel
class Model(BaseModel):
foo: int
model = Model.model_validate({"foo": 1})
print(model.model_dump())
Troubleshooting
- Coercion surprises: use strict types if needed
- Slow validators: keep them minimal
- Mutable defaults: use
default_factory
References
Ruff Linting
Ruff is an extremely fast Python linter designed as a drop-in replacement for Flake8 (plus dozens of plugins), isort, pydocstyle, pyupgrade, autoflake, and more. Written in Rust, it offers 10-100x performance improvements over traditional Python linters.
Overview
Ruff provides a single CLI for linting with optional auto-fix. It supports an extensive rule set with 800+ built-in rules and integrates cleanly with pre-commit, CI systems, and modern editors.
Key Features
- Extremely Fast: 10-100x faster than Flake8, Black, isort
- Drop-in Replacement: Compatible with existing Flake8 plugins and configurations
- Auto-fix Support: Automatically fix many common issues
- Comprehensive Rules: 800+ built-in rules from popular linters
- Single Tool: Replaces flake8, isort, pyupgrade, autoflake, pydocstyle, and more
When to Use
- Standardizing code quality across a project or team
- Enforcing consistent coding rules in CI/CD pipelines
- Replacing multiple linting tools with a single fast solution
- Auto-fixing common code style issues
- Migrating from Flake8, isort, or other legacy linters
Quick Start
# Install Ruff
uv pip install ruff
# or
pip install ruff
# Run linting on current directory
ruff check .
# Run linting with auto-fix
ruff check . --fix
# Watch mode for development
ruff check --watch
Core Patterns
- Start minimal: Enable
EandFrules first, then gradually expand - Auto-fix safely: Use
ruff check --fixfor safe fixes only - Per-file ignores: Use sparingly for generated code or special cases
- CI integration: Use
ruff check --output-format githubfor GitHub Actions - Single source of truth: Configure via
pyproject.tomlorruff.toml
Rule Selection
Ruff uses a code system where each rule consists of a 1-3 letter prefix followed by digits (e.g., F401). Rules are controlled via lint.select, lint.extend-select, and lint.ignore.
Recommended Rule Sets
Minimal (Start Here):
[tool.ruff.lint]
select = ["E", "F"] # pycodestyle errors + Pyflakes
Balanced (Recommended):
[tool.ruff.lint]
select = [
"E", # pycodestyle errors
"F", # Pyflakes
"UP", # pyupgrade
"B", # flake8-bugbear
"SIM", # flake8-simplify
"I", # isort
]
Comprehensive:
[tool.ruff.lint]
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # Pyflakes
"UP", # pyupgrade
"B", # flake8-bugbear
"SIM", # flake8-simplify
"I", # isort
"N", # pep8-naming
"S", # flake8-bandit (security)
"C4", # flake8-comprehensions
"DTZ", # flake8-datetimez
"T20", # flake8-print
"RUF", # Ruff-specific rules
]
ignore = ["E501"] # Line too long (handled by formatter)
Rule Priority
CLI options override pyproject.toml, which overrides inherited configs:
- CLI (
--select,--ignore) - highest priority - Current
pyproject.toml - Inherited
pyproject.tomlfiles
For detailed rule configuration, see references/rule_selection.md.
Configuration
pyproject.toml (Recommended)
[tool.ruff]
line-length = 88
target-version = "py311"
exclude = [".venv", "dist", "build", "*.pyi"]
[tool.ruff.lint]
select = ["E", "F", "UP", "B", "SIM", "I"]
ignore = ["E501"]
fixable = ["ALL"]
unfixable = []
[tool.ruff.lint.per-file-ignores]
"tests/**/*.py" = ["S101"] # Allow assert in tests
"__init__.py" = ["F401"] # Allow unused imports
"**/{tests,docs,tools}/*" = ["E402"] # Allow late imports
[tool.ruff.lint.isort]
known-first-party = ["myproject"]
[tool.ruff.lint.pydocstyle]
convention = "google"
ruff.toml Alternative
line-length = 88
target-version = "py311"
[lint]
select = ["E", "F", "UP", "B", "SIM", "I"]
ignore = ["E501"]
[lint.per-file-ignores]
"tests/**/*.py" = ["S101"]
Fix Safety
Ruff categorizes fixes as safe or unsafe:
| Type | Behavior | Default |
|---|---|---|
| Safe | Preserves code semantics | Enabled |
| Unsafe | May change runtime behavior | Disabled |
# Apply only safe fixes (default)
ruff check --fix
# Apply all fixes including unsafe
ruff check --fix --unsafe-fixes
# Show what unsafe fixes are available
ruff check --unsafe-fixes
Adjusting Fix Safety
[tool.ruff.lint]
# Promote unsafe fixes to safe
extend-safe-fixes = ["F601"]
# Demote safe fixes to unsafe
extend-unsafe-fixes = ["UP034"]
# Control which rules can be fixed
fixable = ["ALL"]
unfixable = ["F401"] # Never auto-fix unused imports
For detailed fix safety documentation, see references/fix_safety.md.
Error Suppression
Line-Level (noqa)
x = 1 # noqa: F841 # Ignore specific rule
i = 1 # noqa: E741, F841 # Ignore multiple rules
x = 1 # noqa # Ignore all rules (avoid this)
File-Level
# ruff: noqa # Ignore all rules in file
# ruff: noqa: F841 # Ignore specific rule in file
Block-Level (Preview Mode)
# ruff: disable[E501]
VALUE_1 = "Very long string..."
VALUE_2 = "Another long string..."
# ruff: enable[E501]
For detailed suppression patterns, see references/error_suppression.md.
CLI Commands
# Basic linting
ruff check . # Lint current directory
ruff check path/to/file.py # Lint specific file
ruff check . --fix # Lint and auto-fix
ruff check . --fix --unsafe-fixes # Include unsafe fixes
# Output formats
ruff check . --output-format text # Default human-readable
ruff check . --output-format github # GitHub Actions annotations
ruff check . --output-format json # JSON output
ruff check . --output-format sarif # SARIF format
# Inspection
ruff check . --diff # Show what would change
ruff check . --show-fixes # Show available fixes
ruff check . --statistics # Show rule statistics
ruff rule F401 # Explain a specific rule
# Development
ruff check --watch # Watch mode
ruff check . --add-noqa # Add noqa comments
ruff check . --extend-select RUF100 # Find unused noqa comments
Exit Codes
| Code | Meaning |
|---|---|
| 0 | No violations found, or all fixed |
| 1 | Violations found |
| 2 | Configuration error or internal error |
Modify exit behavior:
ruff check . --exit-zero # Always exit 0
ruff check . --exit-non-zero-on-fix # Exit 1 if any violations (even if fixed)
CI Integration
GitHub Actions
name: Lint
on: [push, pull_request]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/ruff-action@v3
with:
args: "check --output-format github"
Pre-commit
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.0
hooks:
- id: ruff
args: [--fix]
- id: ruff-format
Troubleshooting
| Issue | Solution |
|---|---|
| Rule conflicts with formatter | Ignore formatting rules (E501) when using ruff format |
| Too many violations | Start with minimal rules (E, F), expand gradually |
| Too many per-file ignores | Review rule selection, consider disabling noisy rules |
| Slow on large codebase | Ensure .venv excluded, check for recursive symlinks |
| noqa not working | Check syntax: # noqa: F401 (colon required) |
Common Rule Prefixes
| Prefix | Source | Description |
|---|---|---|
| E/W | pycodestyle | Style errors/warnings |
| F | Pyflakes | Logical errors |
| B | flake8-bugbear | Common bugs |
| I | isort | Import sorting |
| UP | pyupgrade | Python version upgrades |
| SIM | flake8-simplify | Code simplification |
| N | pep8-naming | Naming conventions |
| S | flake8-bandit | Security issues |
| C4 | flake8-comprehensions | Comprehension style |
| RUF | Ruff | Ruff-specific rules |
References
ty-skills
Master Python type checking with ty - the extremely fast type checker written in Rust by Astral (creators of uv and Ruff).
When to Use This Skill
- Adding type annotations to Python code
- Fixing type errors and diagnostics from ty
- Configuring ty rules and severity levels
- Migrating from mypy or pyright to ty
- Understanding advanced type patterns (intersection types, protocols, generics)
- Setting up ty language server in your editor
Quick Start
# Install
uv tool install ty
# or: pip install ty
# Check current directory
ty check
# Check specific files
ty check src/
# Full diagnostics
ty check --output-format full
Configuration
Configure via pyproject.toml:
[tool.ty.environment]
python-version = "3.12"
python = "./.venv"
python-platform = "linux"
root = ["./src"]
extra-paths = ["./typings"]
[tool.ty.rules]
# error: fail CI, warn: report, ignore: disable
possibly-unresolved-reference = "error"
invalid-argument-type = "error"
division-by-zero = "warn"
unused-ignore-comment = "warn"
[tool.ty.src]
include = ["src", "tests"]
exclude = ["src/migrations/"]
# Per-file overrides
[[tool.ty.overrides]]
include = ["tests/**"]
[tool.ty.overrides.rules]
possibly-unresolved-reference = "warn"
Rules Quick Reference
| Rule | Default | Description |
|---|---|---|
possibly-unresolved-reference | error | Variable might not be defined |
invalid-argument-type | error | Argument type mismatch |
incompatible-assignment | error | Assigned value incompatible |
missing-argument | error | Required argument missing |
unsupported-operator | error | Operator not supported for types |
invalid-return-type | error | Return type mismatch |
division-by-zero | warn | Potential division by zero |
unused-ignore-comment | warn | Suppression not needed |
redundant-cast | warn | Cast has no effect |
possibly-unbound-attribute | warn | Attribute might not exist |
index-out-of-bounds | warn | Index might be out of range |
Intersection Types (ty Exclusive)
ty has first-class intersection type support:
def output_as_json(obj: Serializable) -> str:
if isinstance(obj, Versioned):
reveal_type(obj) # reveals: Serializable & Versioned
return str({
"data": obj.serialize_json(), # From Serializable
"version": obj.version # From Versioned
})
return obj.serialize_json()
Suppression Comments
# Suppress single rule
x: int = "hello" # type: ignore[incompatible-assignment]
# Suppress multiple
y = risky() # type: ignore[possibly-unresolved-reference, invalid-argument-type]
Reference Documents
For detailed information, see:
| Document | Content |
|---|---|
references/ty_rules_reference.md | All rules with examples and fixes |
references/typing_cheatsheet.md | Python typing module quick reference |
references/advanced_patterns.md | Protocols, generics, type guards, variance |
references/migration_guide.md | mypy/pyright → ty migration |
references/common_errors.md | Error solutions with examples |
references/editor_setup/ | VS Code, Cursor, Neovim setup |
Resources
Sponsors
Details
- Marketplace
- jiatastic/open-python-skills
- Category
- Community