AI skill for Python backend development - FastAPI, SQLAlchemy, Redis, security patterns

GitHub stars
10

Install

/plugin install open-python-skills@open-python-skills

Run it inside Claude Code after adding the marketplace with /plugin marketplace add jiatastic/open-python-skills.

Contents

Discovered from source

Skills 8

commit-message

Analyze git changes and generate context-aware commit messages following Conventional Commits.

Quick Start

# Analyze all changes
python3 .shared/commit-message/scripts/analyze_changes.py --analyze

# Get batch commit suggestions
python3 .shared/commit-message/scripts/analyze_changes.py --batch

# Generate message for specific files
python3 .shared/commit-message/scripts/analyze_changes.py --generate "src/api/*.py"

Commands

CommandDescription
--analyzeShow all changed files with status and categories
--batchSuggest how to split changes into multiple commits
--generate [pattern]Generate commit message for matching files
--stagedOnly analyze staged changes (default: all changes)

Commit Types

TypeDescriptionExample
featNew featurefeat(api): add user authentication
fixBug fixfix(db): resolve connection timeout
refactorCode restructuringrefactor(utils): simplify helper functions
docsDocumentationdocs: update README
testTeststest(api): add user endpoint tests
choreMaintenancechore: update dependencies
styleFormattingstyle: fix linting errors

Batch Commit Workflow

When you have multiple unrelated changes:

  1. Run --batch to see suggested commit groups
  2. Stage files for first commit: git add <files>
  3. Commit with suggested message
  4. Repeat for remaining groups

Grouping Strategy

Files are grouped by:

  • Directory/Module: src/api/, tests/, docs/
  • Change Type: Added vs Modified vs Deleted
  • Semantic Relationship: Related files together

Context-Aware Commit Messages

Note: The analyze_changes.py script provides file grouping and basic suggestions. Use its output as a starting point, then read git diff to understand the actual changes and generate context-aware messages following the examples below.

When generating commit messages, analyze the actual code changes to infer business context. Don't just describe files—describe what the changes accomplish.

Scope Guidelines

The scope should reflect the business module or feature, not just the directory:

Scope TypeExampleWhen to Use
Feature/Modulecompanion, calendar, inboxChanges to a specific product feature
Platformios, android, webPlatform-specific changes
Integrationoutlook, gmail, slackThird-party integration changes
Componentauth, api, dbCore infrastructure changes

Input/Output Examples

Example 1: New Feature

Input (code changes):
  + src/companion/pages/AvailabilityDetailPage.tsx
  + src/companion/pages/AvailabilityActionsPage.tsx
  + src/companion/components/AvailabilityCard.tsx
  M src/companion/navigation/routes.ts

Output:
  feat(companion): add availability detail and actions pages for ios

  - New AvailabilityDetailPage showing time slot details
  - New AvailabilityActionsPage for booking/canceling
  - AvailabilityCard component for list display
  - Updated navigation routes

Example 2: Bug Fix

Input (code changes):
  M src/integrations/outlook/email_sender.py
  M src/integrations/outlook/auth.py

Output:
  fix(outlook): resolve email sending failures due to token expiration

  Refresh OAuth token before sending when close to expiry

Example 3: Multi-platform Change

Input (code changes):
  M ios/Calendar/CalendarView.swift
  M android/calendar/CalendarFragment.kt
  M web/src/calendar/Calendar.tsx

Output:
  feat(calendar): add week view across all platforms

  Implement consistent week view UI for iOS, Android, and web

Example 4: Chore/Maintenance

Input (code changes):
  M package.json
  M yarn.lock
  M requirements.txt

Output:
  chore(deps): update dependencies to latest versions

Writing Good Descriptions

Bad (Generic)Good (Context-Aware)
feat: add new filefeat(payments): add Stripe webhook handler
fix: fix bugfix(auth): prevent session timeout on mobile
chore: update codechore(ci): reduce build time with parallel jobs
refactor: refactor utilsrefactor(api): extract rate limiting to middleware

Key Principles

  1. Read the code - Understand what the changes actually do
  2. Identify the feature - What user-facing or system capability is affected?
  3. Be specific - Include relevant details (platform, integration, component)
  4. Use active voice - "add", "fix", "update", not "added", "fixed", "updated"
  5. Keep it concise - First line under 72 characters

Error Handling

Production-ready error handling for Python APIs using the Let it crash philosophy.

Design Philosophy

Let it crash - Don't be defensive. Let exceptions propagate naturally and handle them at boundaries.

# BAD - Too defensive, obscures errors
@app.get("/users/{user_id}")
async def get_user(user_id: int):
    try:
        user = await user_service.get(user_id)
        if not user:
            raise HTTPException(404, "Not found")
        return user
    except DatabaseError as e:
        raise HTTPException(500, "Database error")
    except Exception as e:
        logger.exception("Unexpected error")
        raise HTTPException(500, "Internal error")

# GOOD - Let exceptions propagate, handle at boundary
@app.get("/users/{user_id}")
async def get_user(user_id: int):
    user = await user_service.get(user_id)
    if not user:
        raise UserNotFoundError(user_id)
    return user

Core Principles

  1. Raise low, catch high - Throw exceptions where errors occur, handle at API boundaries
  2. Domain exceptions - Create semantic exceptions, not generic ones
  3. Global handlers - Use @app.exception_handler() for centralized error formatting
  4. No bare except - Always catch specific exceptions
  5. Preserve context - Use raise ... from error to keep original traceback

Quick Start

1. Define Domain Exceptions

from enum import StrEnum

class ErrorCode(StrEnum):
    USER_NOT_FOUND = "user_not_found"
    INVALID_CREDENTIALS = "invalid_credentials"
    RATE_LIMITED = "rate_limited"

class DomainError(Exception):
    """Base exception for all domain errors."""
    def __init__(self, code: ErrorCode, message: str, status_code: int = 400):
        self.code = code
        self.message = message
        self.status_code = status_code
        super().__init__(message)

class UserNotFoundError(DomainError):
    def __init__(self, user_id: int):
        super().__init__(
            code=ErrorCode.USER_NOT_FOUND,
            message=f"User {user_id} not found",
            status_code=404
        )

2. Define Error Response Schema

from pydantic import BaseModel

class ErrorDetail(BaseModel):
    code: str
    message: str
    request_id: str | None = None

class ErrorResponse(BaseModel):
    error: ErrorDetail

3. Register Global Handlers

from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from fastapi.exceptions import RequestValidationError
from starlette.exceptions import HTTPException as StarletteHTTPException

app = FastAPI()

@app.exception_handler(DomainError)
async def domain_error_handler(request: Request, exc: DomainError):
    return JSONResponse(
        status_code=exc.status_code,
        content={"error": {"code": exc.code, "message": exc.message}}
    )

@app.exception_handler(StarletteHTTPException)
async def http_exception_handler(request: Request, exc: StarletteHTTPException):
    return JSONResponse(
        status_code=exc.status_code,
        content={"error": {"code": "http_error", "message": str(exc.detail)}}
    )

@app.exception_handler(RequestValidationError)
async def validation_error_handler(request: Request, exc: RequestValidationError):
    return JSONResponse(
        status_code=422,
        content={"error": {"code": "validation_error", "message": "Invalid request"}}
    )

@app.exception_handler(Exception)
async def generic_error_handler(request: Request, exc: Exception):
    # Log full error internally
    logger.exception("Unhandled error")
    # Return safe message to client
    return JSONResponse(
        status_code=500,
        content={"error": {"code": "internal_error", "message": "Internal server error"}}
    )

4. Use in Routes

@app.get("/users/{user_id}")
async def get_user(user_id: int):
    user = await user_service.get(user_id)
    if not user:
        raise UserNotFoundError(user_id)
    return user

When to Catch Exceptions

Only catch exceptions in these cases:

SituationExample
Need to retrytenacity.retry() for transient failures
Need to transformWrap third-party SDK errors as domain errors
Need to clean upUse finally or context managers
Need to add contextraise DomainError(...) from original

Python + FastAPI Integration

LayerResponsibility
Service/DomainRaise domain exceptions (UserNotFoundError)
RoutesLet exceptions propagate (no try/except)
Exception HandlersTransform to HTTP responses
MiddlewareAdd request context (request_id, timing)

Common Patterns

Third-Party SDK Wrapping

import httpx
from tenacity import retry, stop_after_attempt, wait_exponential

class ExternalServiceError(DomainError):
    def __init__(self, service: str, original: Exception):
        super().__init__(
            code=ErrorCode.EXTERNAL_SERVICE_ERROR,
            message=f"{service} unavailable",
            status_code=503
        )
        self.__cause__ = original

@retry(stop=stop_after_attempt(3), wait=wait_exponential())
async def call_payment_api(data: dict):
    try:
        async with httpx.AsyncClient(timeout=10.0) as client:
            response = await client.post("https://api.payment.com/charge", json=data)
            response.raise_for_status()
            return response.json()
    except httpx.HTTPError as e:
        raise ExternalServiceError("Payment API", e) from e

Background Task Error Handling

from fastapi import BackgroundTasks

async def safe_background_task(task_func, *args, **kwargs):
    try:
        await task_func(*args, **kwargs)
    except Exception as e:
        logger.exception(f"Background task failed: {e}")
        # Optional: send to dead letter queue or alerting

@app.post("/orders")
async def create_order(order: Order, background_tasks: BackgroundTasks):
    result = await order_service.create(order)
    background_tasks.add_task(safe_background_task, send_confirmation_email, result.id)
    return result

Troubleshooting

IssueCauseFix
Stack trace in responseNo generic handlerAdd @app.exception_handler(Exception)
Lost original errorMissing fromUse raise NewError() from original
Validation errors leakDefault handlerOverride RequestValidationError handler
Silent failuresSwallowed exceptionsLet exceptions propagate, handle at boundary

References

excalidraw-ai

Generate Excalidraw diagrams by writing JSON directly. No templates needed - you have full control over every element.

When to Use This Skill

Use this skill when you need to:

  • Create architecture diagrams, flowcharts, or mind maps
  • Visualize system designs, data flows, or processes
  • Generate diagrams from code analysis or documentation
  • Create custom diagrams with precise control over layout and styling

Excalidraw JSON Schema

An Excalidraw file is a JSON object with this structure:

{
  "type": "excalidraw",
  "version": 2,
  "source": "https://excalidraw.com",
  "elements": [ /* array of element objects */ ],
  "appState": {
    "viewBackgroundColor": "#ffffff",
    "gridSize": null
  },
  "files": {}
}

Element Types

Rectangle
{
  "id": "unique-id-1",
  "type": "rectangle",
  "x": 100,
  "y": 100,
  "width": 200,
  "height": 100,
  "strokeColor": "#1971c2",
  "backgroundColor": "#a5d8ff",
  "fillStyle": "solid",
  "strokeWidth": 2,
  "strokeStyle": "solid",
  "roughness": 1,
  "opacity": 100,
  "groupIds": [],
  "angle": 0,
  "seed": 12345,
  "isDeleted": false
}
Ellipse
{
  "id": "unique-id-2",
  "type": "ellipse",
  "x": 100,
  "y": 100,
  "width": 150,
  "height": 100,
  "strokeColor": "#7c3aed",
  "backgroundColor": "#ede9fe",
  "fillStyle": "solid",
  "strokeWidth": 2,
  "roughness": 1
}
Diamond
{
  "id": "unique-id-3",
  "type": "diamond",
  "x": 100,
  "y": 100,
  "width": 120,
  "height": 120,
  "strokeColor": "#dc2626",
  "backgroundColor": "#fee2e2",
  "fillStyle": "solid"
}
Text
{
  "id": "unique-id-4",
  "type": "text",
  "x": 110,
  "y": 140,
  "width": 180,
  "height": 25,
  "text": "Your Label Here",
  "fontSize": 20,
  "fontFamily": 1,
  "textAlign": "center",
  "verticalAlign": "middle",
  "strokeColor": "#1e1e1e",
  "backgroundColor": "transparent"
}
Arrow
{
  "id": "unique-id-5",
  "type": "arrow",
  "x": 300,
  "y": 150,
  "width": 100,
  "height": 50,
  "strokeColor": "#1971c2",
  "strokeWidth": 2,
  "points": [[0, 0], [100, 50]],
  "startBinding": {
    "elementId": "source-element-id",
    "focus": 0.5,
    "gap": 5
  },
  "endBinding": {
    "elementId": "target-element-id",
    "focus": 0.5,
    "gap": 5
  },
  "startArrowhead": null,
  "endArrowhead": "arrow"
}
Line
{
  "id": "unique-id-6",
  "type": "line",
  "x": 100,
  "y": 100,
  "width": 200,
  "height": 0,
  "strokeColor": "#868e96",
  "strokeWidth": 1,
  "points": [[0, 0], [200, 0]],
  "startArrowhead": null,
  "endArrowhead": null
}

Element Properties Reference

PropertyTypeDescription
idstringUnique identifier (use UUID or similar)
typestringrectangle, ellipse, diamond, text, arrow, line
x, ynumberPosition coordinates
width, heightnumberDimensions
strokeColorstringBorder/stroke color (hex)
backgroundColorstringFill color (hex) or "transparent"
fillStylestring"solid", "hachure", "cross-hatch"
strokeWidthnumberLine thickness (1, 2, 4)
strokeStylestring"solid", "dashed", "dotted"
roughnessnumber0 = sharp, 1 = normal, 2 = sketchy
opacitynumber0-100
anglenumberRotation in radians
groupIdsarrayGroup element IDs together
seednumberRandom seed for hand-drawn effect

Text Properties

PropertyTypeDescription
textstringThe text content
fontSizenumberFont size in pixels
fontFamilynumber1 = Virgil (hand-drawn), 2 = Helvetica, 3 = Cascadia
textAlignstring"left", "center", "right"
verticalAlignstring"top", "middle"

Arrow Properties

PropertyTypeDescription
pointsarrayArray of [x, y] coordinates relative to element origin
startBindingobjectConnection to source element
endBindingobjectConnection to target element
startArrowheadstringnull, "arrow", "bar", "dot", "triangle"
endArrowheadstringnull, "arrow", "bar", "dot", "triangle"

Color Palette Reference

Component-Based Colors (Recommended for Architecture Diagrams)

Component TypeStrokeBackgroundUse For
Database#7c3aed#ede9fePostgreSQL, MySQL, MongoDB
Cache#dc2626#fee2e2Redis, Memcached
Queue#16a34a#dcfce7Kafka, RabbitMQ, SQS
Load Balancer#0891b2#cffafeNginx, HAProxy, ALB
Gateway#475569#f1f5f9API Gateway, Kong
CDN#06b6d4#e0f2feCloudFront, Fastly
Auth#e11d48#ffe4e6OAuth, IAM, Cognito
Storage#d97706#fef3c7S3, Blob Storage
Service#2563eb#dbeafeBackend services, APIs
Container#0284c7#bae6fdDocker, Kubernetes
Function#f59e0b#fef3c7Lambda, Serverless
Monitoring#84cc16#ecfccbPrometheus, Grafana
Web App#4f46e5#e0e7ffReact, Vue, Frontend
Mobile#6366f1#eef2ffiOS, Android

Theme Colors

Modern (Default)

  • Primary: #1971c2 / #e7f5ff
  • Neutral: #64748b / #f1f5f9

Sketchy (Hand-drawn)

  • Primary: #495057 / #f8f9fa
  • Use roughness: 2 and fontFamily: 3

Technical

  • Primary: #2f9e44 / #ebfbee
  • Use strokeStyle: "dashed" for connections

Icon Libraries (305 Components Total)

Professional icons are available in .excalidrawlib files in the assets/ directory:

LibraryComponentsContent
aws-architecture-icons.excalidrawlib248AWS service icons (Lambda, S3, EC2, RDS, DynamoDB, SQS, etc.)
system-design.excalidrawlib24System design elements (server, database, storage, etc.)
drwnio.excalidrawlib18Draw.io style icons
system-design-template.excalidrawlib8Pre-built templates (steps, flow, db, etc.)
software-architecture.excalidrawlib7Software architecture components

Using Library Components

Library files contain element arrays that can be embedded directly into your diagram. Each library item is an array of Excalidraw elements that form a component.

To use: Parse the library JSON, find the component you need, adjust positions, and include in your diagram's elements array.

Layout Guidelines

Architecture Diagram Layout

Organize components in layers (top to bottom):

Layer 1: Client/External (y: 100)
Layer 2: Edge/CDN (y: 280)
Layer 3: Gateway/Load Balancer (y: 460)
Layer 4: Services (y: 640)
Layer 5: Data/Cache (y: 820)
Layer 6: Infrastructure (y: 1000)

Spacing recommendations:

  • Horizontal spacing between components: 240px
  • Vertical spacing between layers: 180px
  • Component width: 180-220px
  • Component height: 80-100px

Flowchart Layout

Arrange left-to-right or top-to-bottom:

  • Step spacing: 200px
  • Use rectangles for processes
  • Use diamonds for decisions
  • Use ellipses for start/end

Mind Map Layout

Radial layout from center:

  • Center node at (400, 300)
  • Child nodes at radius 250px
  • Distribute evenly using angle calculations

Complete Example

{
  "type": "excalidraw",
  "version": 2,
  "source": "https://excalidraw.com",
  "elements": [
    {
      "id": "api-gateway",
      "type": "rectangle",
      "x": 100,
      "y": 100,
      "width": 180,
      "height": 80,
      "strokeColor": "#475569",
      "backgroundColor": "#f1f5f9",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "roughness": 0,
      "opacity": 100,
      "groupIds": ["group-1"],
      "seed": 1234
    },
    {
      "id": "api-gateway-label",
      "type": "text",
      "x": 140,
      "y": 130,
      "width": 100,
      "height": 25,
      "text": "API Gateway",
      "fontSize": 16,
      "fontFamily": 1,
      "textAlign": "center",
      "strokeColor": "#1e1e1e",
      "groupIds": ["group-1"]
    },
    {
      "id": "redis",
      "type": "diamond",
      "x": 100,
      "y": 280,
      "width": 180,
      "height": 100,
      "strokeColor": "#dc2626",
      "backgroundColor": "#fee2e2",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "roughness": 0,
      "seed": 5678
    },
    {
      "id": "redis-label",
      "type": "text",
      "x": 155,
      "y": 320,
      "width": 70,
      "height": 20,
      "text": "Redis",
      "fontSize": 16,
      "fontFamily": 1,
      "textAlign": "center",
      "strokeColor": "#1e1e1e"
    },
    {
      "id": "arrow-1",
      "type": "arrow",
      "x": 190,
      "y": 180,
      "width": 0,
      "height": 100,
      "strokeColor": "#64748b",
      "strokeWidth": 2,
      "points": [[0, 0], [0, 100]],
      "startBinding": {"elementId": "api-gateway", "focus": 0.5, "gap": 5},
      "endBinding": {"elementId": "redis", "focus": 0.5, "gap": 5},
      "endArrowhead": "arrow"
    }
  ],
  "appState": {
    "viewBackgroundColor": "#ffffff",
    "gridSize": null
  },
  "files": {}
}

Output

Save your diagram as a .excalidraw or .json file. It can be:

  • Imported directly into excalidraw.com
  • Embedded in documentation or web apps
  • Converted to PNG/SVG using Excalidraw's export features

Tips for AI Agents

  1. Generate unique IDs: Use UUIDs or sequential IDs for each element
  2. Group related elements: Use groupIds to keep shapes and their labels together
  3. Use bindings for arrows: Connect arrows to elements using startBinding and endBinding
  4. Apply consistent styling: Use the color palette for component types
  5. Calculate text positions: Center text within shapes by adjusting x/y based on text length
  6. Set seed values: Use seed: hash(id) for consistent hand-drawn rendering

python-backend

Production-ready Python backend patterns for FastAPI, SQLAlchemy, and Upstash.

When to Use This Skill

  • Building REST APIs with FastAPI
  • Implementing JWT/OAuth2 authentication
  • Setting up SQLAlchemy async databases
  • Integrating Redis/Upstash caching and rate limiting
  • Refactoring AI-generated Python code
  • Designing API patterns and project structure

Core Principles

  1. Async-first - Use async/await for I/O operations
  2. Type everything - Pydantic models for validation
  3. Dependency injection - Use FastAPI's Depends()
  4. Fail fast - Validate early, use HTTPException
  5. Security by default - Never trust user input

Quick Patterns

Project Structure

src/
├── auth/
│   ├── router.py      # endpoints
│   ├── schemas.py     # pydantic models
│   ├── models.py      # db models
│   ├── service.py     # business logic
│   └── dependencies.py
├── posts/
│   └── ...
├── config.py
├── database.py
└── main.py

Async Routes

# BAD - blocks event loop
@router.get("/")
async def bad():
    time.sleep(10)  # Blocking!

# GOOD - runs in threadpool
@router.get("/")
def good():
    time.sleep(10)  # OK in sync function

# BEST - non-blocking
@router.get("/")
async def best():
    await asyncio.sleep(10)  # Non-blocking

Pydantic Validation

from pydantic import BaseModel, EmailStr, Field

class UserCreate(BaseModel):
    email: EmailStr
    username: str = Field(min_length=3, max_length=50, pattern="^[a-zA-Z0-9_]+$")
    age: int = Field(ge=18)

Dependency Injection

async def get_current_user(token: str = Depends(oauth2_scheme)) -> User:
    payload = decode_token(token)
    user = await get_user(payload["sub"])
    if not user:
        raise HTTPException(401, "User not found")
    return user

@router.get("/me")
async def get_me(user: User = Depends(get_current_user)):
    return user

SQLAlchemy Async

from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine

engine = create_async_engine(DATABASE_URL, pool_pre_ping=True)
SessionLocal = async_sessionmaker(engine, expire_on_commit=False)

async def get_session() -> AsyncGenerator[AsyncSession, None]:
    async with SessionLocal() as session:
        yield session

Redis Caching

from upstash_redis import Redis

redis = Redis.from_env()

@app.get("/data/{id}")
def get_data(id: str):
    cached = redis.get(f"data:{id}")
    if cached:
        return cached
    data = fetch_from_db(id)
    redis.setex(f"data:{id}", 600, data)
    return data

Rate Limiting

from upstash_ratelimit import Ratelimit, SlidingWindow

ratelimit = Ratelimit(
    redis=Redis.from_env(),
    limiter=SlidingWindow(max_requests=10, window=60),
)

@app.get("/api/resource")
def protected(request: Request):
    result = ratelimit.limit(request.client.host)
    if not result.allowed:
        raise HTTPException(429, "Rate limit exceeded")
    return {"data": "..."}

Reference Documents

For detailed patterns, see:

DocumentContent
references/fastapi_patterns.mdProject structure, async, Pydantic, dependencies, testing
references/security_patterns.mdJWT, OAuth2, password hashing, CORS, API keys
references/database_patterns.mdSQLAlchemy async, transactions, eager loading, migrations
references/upstash_patterns.mdRedis, rate limiting, QStash background jobs

Resources

Logfire

Structured observability for Python using Pydantic Logfire - fast setup, powerful features, OpenTelemetry-compatible.

Quick Start

uv pip install logfire
import logfire

logfire.configure(service_name="my-api", service_version="1.0.0")
logfire.info("Application started")

Core Patterns

1. Service Configuration

Always set service metadata at startup:

import logfire

logfire.configure(
    service_name="backend",
    service_version="1.0.0",
    environment="production",
    console=False,           # Disable console output in production
    send_to_logfire=True,    # Send to Logfire platform
)

2. Framework Instrumentation

Instrument frameworks before creating clients/apps:

import logfire
from fastapi import FastAPI

# Configure FIRST
logfire.configure(service_name="backend")

# Then instrument
logfire.instrument_fastapi()
logfire.instrument_httpx()
logfire.instrument_sqlalchemy()

# Then create app
app = FastAPI()

3. Log Levels and Structured Logging

# All log levels (trace → fatal)
logfire.trace("Detailed trace", step=1)
logfire.debug("Debug context", variable=locals())
logfire.info("User action", action="login", success=True)
logfire.notice("Important event", event_type="milestone")
logfire.warn("Potential issue", threshold_exceeded=True)
logfire.error("Operation failed", error_code=500)
logfire.fatal("Critical failure", component="database")

# Python 3.11+ f-string magic (auto-extracts variables)
user_id = 123
status = "active"
logfire.info(f"User {user_id} status: {status}")
# Equivalent to: logfire.info("User {user_id}...", user_id=user_id, status=status)

# Exception logging with automatic traceback
try:
    risky_operation()
except Exception:
    logfire.exception("Operation failed", context="extra_info")

4. Manual Spans

# Spans for tracing operations
with logfire.span("Process order {order_id}", order_id="ORD-123"):
    logfire.info("Validating cart")
    # ... processing logic
    logfire.info("Order complete")

# Dynamic span attributes
with logfire.span("Database query") as span:
    results = execute_query()
    span.set_attribute("result_count", len(results))
    span.message = f"Query returned {len(results)} results"

5. Custom Metrics

# Counter - monotonically increasing
request_counter = logfire.metric_counter("http.requests", unit="1")
request_counter.add(1, {"endpoint": "/api/users", "method": "GET"})

# Gauge - current value
temperature = logfire.metric_gauge("temperature", unit="°C")
temperature.set(23.5)

# Histogram - distribution of values
latency = logfire.metric_histogram("request.duration", unit="ms")
latency.record(45.2, {"endpoint": "/api/data"})

6. LLM Observability

import logfire
from pydantic_ai import Agent

logfire.configure()
logfire.instrument_pydantic_ai()  # Traces all agent interactions

agent = Agent("openai:gpt-4o", system_prompt="You are helpful.")
result = agent.run_sync("Hello!")

7. Suppress Noisy Instrumentation

# Suppress entire scope (e.g., noisy library)
logfire.suppress_scopes("google.cloud.bigquery.opentelemetry_tracing")

# Suppress specific code block
with logfire.suppress_instrumentation():
    client.get("https://internal-healthcheck.local")  # Not traced

8. Sensitive Data Scrubbing

import logfire

# Add custom patterns to scrub
logfire.configure(
    scrubbing=logfire.ScrubbingOptions(
        extra_patterns=["api_key", "secret", "token"]
    )
)

# Custom callback for fine-grained control
def scrubbing_callback(match: logfire.ScrubMatch):
    if match.path == ("attributes", "safe_field"):
        return match.value  # Don't scrub this field
    return None  # Use default scrubbing

logfire.configure(
    scrubbing=logfire.ScrubbingOptions(callback=scrubbing_callback)
)

9. Sampling for High-Traffic Services

import logfire

# Sample 50% of traces
logfire.configure(sampling=logfire.SamplingOptions(head=0.5))

# Disable metrics to reduce volume
logfire.configure(metrics=False)

10. Testing

import logfire
from logfire.testing import CaptureLogfire

def test_user_creation(capfire: CaptureLogfire):
    create_user("Alice", "alice@example.com")
    
    spans = capfire.exporter.exported_spans
    assert len(spans) >= 1
    assert spans[0].attributes["user_name"] == "Alice"
    
    capfire.exporter.clear()  # Clean up for next test

Available Integrations

CategoryIntegrationMethod
WebFastAPIlogfire.instrument_fastapi(app)
Starlettelogfire.instrument_starlette(app)
Djangologfire.instrument_django()
Flasklogfire.instrument_flask(app)
AIOHTTP Serverlogfire.instrument_aiohttp_server()
ASGIlogfire.instrument_asgi(app)
WSGIlogfire.instrument_wsgi(app)
HTTPHTTPXlogfire.instrument_httpx()
Requestslogfire.instrument_requests()
AIOHTTP Clientlogfire.instrument_aiohttp_client()
DatabaseSQLAlchemylogfire.instrument_sqlalchemy(engine)
Asyncpglogfire.instrument_asyncpg()
Psycopglogfire.instrument_psycopg()
Redislogfire.instrument_redis()
PyMongologfire.instrument_pymongo()
LLMPydantic AIlogfire.instrument_pydantic_ai()
OpenAIlogfire.instrument_openai()
Anthropiclogfire.instrument_anthropic()
MCPlogfire.instrument_mcp()
TasksCelerylogfire.instrument_celery()
AWS Lambdalogfire.instrument_aws_lambda()
LoggingStandard logginglogfire.instrument_logging()
Structloglogfire.instrument_structlog()
Logurulogfire.instrument_loguru()
Printlogfire.instrument_print()
OtherPydanticlogfire.instrument_pydantic()
System Metricslogfire.instrument_system_metrics()

Common Pitfalls

IssueSymptomFix
Missing service nameSpans hard to find in UISet service_name in configure()
Late instrumentationNo spans capturedCall configure() before creating clients
High-cardinality attrsStorage explosionUse IDs, not full payloads as attributes
Console noiseLogs pollute stdoutSet console=False in production

References

pydantic

Type-driven validation and serialization using Pydantic models.

Overview

Pydantic validates data using Python type hints and provides rich serialization via model_dump() and JSON schema output.

When to Use

  • Validating request/response payloads
  • Normalizing untrusted input
  • Generating JSON schema for docs

Quick Start

uv pip install pydantic
from pydantic import BaseModel

class User(BaseModel):
    id: int
    email: str

user = User(id=1, email="a@example.com")

Core Patterns

  1. Typed fields: strict schema definitions.
  2. Field validators: custom validation logic.
  3. Model validators: cross-field checks.
  4. Serialization: model_dump() and model_dump_json().
  5. Settings: environment-driven config via BaseSettings.

Example: field_validator

from pydantic import BaseModel, field_validator

class Model(BaseModel):
    name: str

    @field_validator("name")
    @classmethod
    def ensure_not_empty(cls, v: str):
        if not v:
            raise ValueError("name required")
        return v

Example: model_validate + model_dump

from pydantic import BaseModel

class Model(BaseModel):
    foo: int

model = Model.model_validate({"foo": 1})
print(model.model_dump())

Troubleshooting

  • Coercion surprises: use strict types if needed
  • Slow validators: keep them minimal
  • Mutable defaults: use default_factory

References

Ruff Linting

Ruff is an extremely fast Python linter designed as a drop-in replacement for Flake8 (plus dozens of plugins), isort, pydocstyle, pyupgrade, autoflake, and more. Written in Rust, it offers 10-100x performance improvements over traditional Python linters.

Overview

Ruff provides a single CLI for linting with optional auto-fix. It supports an extensive rule set with 800+ built-in rules and integrates cleanly with pre-commit, CI systems, and modern editors.

Key Features

  • Extremely Fast: 10-100x faster than Flake8, Black, isort
  • Drop-in Replacement: Compatible with existing Flake8 plugins and configurations
  • Auto-fix Support: Automatically fix many common issues
  • Comprehensive Rules: 800+ built-in rules from popular linters
  • Single Tool: Replaces flake8, isort, pyupgrade, autoflake, pydocstyle, and more

When to Use

  • Standardizing code quality across a project or team
  • Enforcing consistent coding rules in CI/CD pipelines
  • Replacing multiple linting tools with a single fast solution
  • Auto-fixing common code style issues
  • Migrating from Flake8, isort, or other legacy linters

Quick Start

# Install Ruff
uv pip install ruff
# or
pip install ruff

# Run linting on current directory
ruff check .

# Run linting with auto-fix
ruff check . --fix

# Watch mode for development
ruff check --watch

Core Patterns

  1. Start minimal: Enable E and F rules first, then gradually expand
  2. Auto-fix safely: Use ruff check --fix for safe fixes only
  3. Per-file ignores: Use sparingly for generated code or special cases
  4. CI integration: Use ruff check --output-format github for GitHub Actions
  5. Single source of truth: Configure via pyproject.toml or ruff.toml

Rule Selection

Ruff uses a code system where each rule consists of a 1-3 letter prefix followed by digits (e.g., F401). Rules are controlled via lint.select, lint.extend-select, and lint.ignore.

Recommended Rule Sets

Minimal (Start Here):

[tool.ruff.lint]
select = ["E", "F"]  # pycodestyle errors + Pyflakes

Balanced (Recommended):

[tool.ruff.lint]
select = [
    "E",    # pycodestyle errors
    "F",    # Pyflakes
    "UP",   # pyupgrade
    "B",    # flake8-bugbear
    "SIM",  # flake8-simplify
    "I",    # isort
]

Comprehensive:

[tool.ruff.lint]
select = [
    "E",    # pycodestyle errors
    "W",    # pycodestyle warnings
    "F",    # Pyflakes
    "UP",   # pyupgrade
    "B",    # flake8-bugbear
    "SIM",  # flake8-simplify
    "I",    # isort
    "N",    # pep8-naming
    "S",    # flake8-bandit (security)
    "C4",   # flake8-comprehensions
    "DTZ",  # flake8-datetimez
    "T20",  # flake8-print
    "RUF",  # Ruff-specific rules
]
ignore = ["E501"]  # Line too long (handled by formatter)

Rule Priority

CLI options override pyproject.toml, which overrides inherited configs:

  1. CLI (--select, --ignore) - highest priority
  2. Current pyproject.toml
  3. Inherited pyproject.toml files

For detailed rule configuration, see references/rule_selection.md.

Configuration

pyproject.toml (Recommended)

[tool.ruff]
line-length = 88
target-version = "py311"
exclude = [".venv", "dist", "build", "*.pyi"]

[tool.ruff.lint]
select = ["E", "F", "UP", "B", "SIM", "I"]
ignore = ["E501"]
fixable = ["ALL"]
unfixable = []

[tool.ruff.lint.per-file-ignores]
"tests/**/*.py" = ["S101"]      # Allow assert in tests
"__init__.py" = ["F401"]         # Allow unused imports
"**/{tests,docs,tools}/*" = ["E402"]  # Allow late imports

[tool.ruff.lint.isort]
known-first-party = ["myproject"]

[tool.ruff.lint.pydocstyle]
convention = "google"

ruff.toml Alternative

line-length = 88
target-version = "py311"

[lint]
select = ["E", "F", "UP", "B", "SIM", "I"]
ignore = ["E501"]

[lint.per-file-ignores]
"tests/**/*.py" = ["S101"]

Fix Safety

Ruff categorizes fixes as safe or unsafe:

TypeBehaviorDefault
SafePreserves code semanticsEnabled
UnsafeMay change runtime behaviorDisabled
# Apply only safe fixes (default)
ruff check --fix

# Apply all fixes including unsafe
ruff check --fix --unsafe-fixes

# Show what unsafe fixes are available
ruff check --unsafe-fixes

Adjusting Fix Safety

[tool.ruff.lint]
# Promote unsafe fixes to safe
extend-safe-fixes = ["F601"]

# Demote safe fixes to unsafe
extend-unsafe-fixes = ["UP034"]

# Control which rules can be fixed
fixable = ["ALL"]
unfixable = ["F401"]  # Never auto-fix unused imports

For detailed fix safety documentation, see references/fix_safety.md.

Error Suppression

Line-Level (noqa)

x = 1  # noqa: F841           # Ignore specific rule
i = 1  # noqa: E741, F841     # Ignore multiple rules
x = 1  # noqa                  # Ignore all rules (avoid this)

File-Level

# ruff: noqa                   # Ignore all rules in file
# ruff: noqa: F841             # Ignore specific rule in file

Block-Level (Preview Mode)

# ruff: disable[E501]
VALUE_1 = "Very long string..."
VALUE_2 = "Another long string..."
# ruff: enable[E501]

For detailed suppression patterns, see references/error_suppression.md.

CLI Commands

# Basic linting
ruff check .                           # Lint current directory
ruff check path/to/file.py             # Lint specific file
ruff check . --fix                     # Lint and auto-fix
ruff check . --fix --unsafe-fixes      # Include unsafe fixes

# Output formats
ruff check . --output-format text      # Default human-readable
ruff check . --output-format github    # GitHub Actions annotations
ruff check . --output-format json      # JSON output
ruff check . --output-format sarif     # SARIF format

# Inspection
ruff check . --diff                    # Show what would change
ruff check . --show-fixes              # Show available fixes
ruff check . --statistics              # Show rule statistics
ruff rule F401                         # Explain a specific rule

# Development
ruff check --watch                     # Watch mode
ruff check . --add-noqa                # Add noqa comments
ruff check . --extend-select RUF100    # Find unused noqa comments

Exit Codes

CodeMeaning
0No violations found, or all fixed
1Violations found
2Configuration error or internal error

Modify exit behavior:

ruff check . --exit-zero               # Always exit 0
ruff check . --exit-non-zero-on-fix    # Exit 1 if any violations (even if fixed)

CI Integration

GitHub Actions

name: Lint
on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/ruff-action@v3
        with:
          args: "check --output-format github"

Pre-commit

repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.8.0
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format

Troubleshooting

IssueSolution
Rule conflicts with formatterIgnore formatting rules (E501) when using ruff format
Too many violationsStart with minimal rules (E, F), expand gradually
Too many per-file ignoresReview rule selection, consider disabling noisy rules
Slow on large codebaseEnsure .venv excluded, check for recursive symlinks
noqa not workingCheck syntax: # noqa: F401 (colon required)

Common Rule Prefixes

PrefixSourceDescription
E/WpycodestyleStyle errors/warnings
FPyflakesLogical errors
Bflake8-bugbearCommon bugs
IisortImport sorting
UPpyupgradePython version upgrades
SIMflake8-simplifyCode simplification
Npep8-namingNaming conventions
Sflake8-banditSecurity issues
C4flake8-comprehensionsComprehension style
RUFRuffRuff-specific rules

References

ty-skills

Master Python type checking with ty - the extremely fast type checker written in Rust by Astral (creators of uv and Ruff).

When to Use This Skill

  • Adding type annotations to Python code
  • Fixing type errors and diagnostics from ty
  • Configuring ty rules and severity levels
  • Migrating from mypy or pyright to ty
  • Understanding advanced type patterns (intersection types, protocols, generics)
  • Setting up ty language server in your editor

Quick Start

# Install
uv tool install ty
# or: pip install ty

# Check current directory
ty check

# Check specific files
ty check src/

# Full diagnostics
ty check --output-format full

Configuration

Configure via pyproject.toml:

[tool.ty.environment]
python-version = "3.12"
python = "./.venv"
python-platform = "linux"
root = ["./src"]
extra-paths = ["./typings"]

[tool.ty.rules]
# error: fail CI, warn: report, ignore: disable
possibly-unresolved-reference = "error"
invalid-argument-type = "error"
division-by-zero = "warn"
unused-ignore-comment = "warn"

[tool.ty.src]
include = ["src", "tests"]
exclude = ["src/migrations/"]

# Per-file overrides
[[tool.ty.overrides]]
include = ["tests/**"]

[tool.ty.overrides.rules]
possibly-unresolved-reference = "warn"

Rules Quick Reference

RuleDefaultDescription
possibly-unresolved-referenceerrorVariable might not be defined
invalid-argument-typeerrorArgument type mismatch
incompatible-assignmenterrorAssigned value incompatible
missing-argumenterrorRequired argument missing
unsupported-operatorerrorOperator not supported for types
invalid-return-typeerrorReturn type mismatch
division-by-zerowarnPotential division by zero
unused-ignore-commentwarnSuppression not needed
redundant-castwarnCast has no effect
possibly-unbound-attributewarnAttribute might not exist
index-out-of-boundswarnIndex might be out of range

Intersection Types (ty Exclusive)

ty has first-class intersection type support:

def output_as_json(obj: Serializable) -> str:
    if isinstance(obj, Versioned):
        reveal_type(obj)  # reveals: Serializable & Versioned
        return str({
            "data": obj.serialize_json(),  # From Serializable
            "version": obj.version          # From Versioned
        })
    return obj.serialize_json()

Suppression Comments

# Suppress single rule
x: int = "hello"  # type: ignore[incompatible-assignment]

# Suppress multiple
y = risky()  # type: ignore[possibly-unresolved-reference, invalid-argument-type]

Reference Documents

For detailed information, see:

DocumentContent
references/ty_rules_reference.mdAll rules with examples and fixes
references/typing_cheatsheet.mdPython typing module quick reference
references/advanced_patterns.mdProtocols, generics, type guards, variance
references/migration_guide.mdmypy/pyright → ty migration
references/common_errors.mdError solutions with examples
references/editor_setup/VS Code, Cursor, Neovim setup

Resources

Sponsors

Details

Marketplace
jiatastic/open-python-skills
Category
Community