Rate Limiting

Rate Limiting - Preventing Brute Force & Resource Abuse

Why Rate Limiting Matters

The Brute Force Problem

Without rate limiting, attackers can try thousands of passwords per second. A 6-character password has 308 million possible combinations.

Without rate limiting:

• At 1,000 attempts/second → Cracked in 5 minutes

With our rate limiting (5 requests/minute):

• At 5 attempts/minute → Would take 117 years

Real-World Brute Force Attacks

Zoom Credential Stuffing (2020): Attackers made over 500,000 login attempts using stolen credentials. Proper rate limiting would have detected and blocked this within the first few hundred attempts.

WordPress Distributed Attacks (2021): Multiple WordPress sites were targeted by distributed brute force attacks attempting millions of login combinations. Sites without rate limiting saw server costs spike as attackers consumed resources.

The Cost of Resource Abuse

Beyond security, rate limiting protects your infrastructure costs. Without it:

• Bots can spam your contact form thousands of times
• Attackers can abuse expensive operations (AI API calls, database queries)
• Your server bill skyrockets before you notice

Real Story: One startup built a "summarize any article" AI feature without rate limiting. A malicious user scripted 10,000 requests in minutes. At AI API costs, this generated $9,600 in charges in 10 minutes. The attack ran 4 hours unnoticed—total cost over $200,000.

Our Rate Limiting Architecture

Implementation Features

• 5 requests per minute per IP address - Balances usability and security
• In-memory tracking - Fast, no database overhead
• IP-based identification - Works behind proxies via x-forwarded-for
• HTTP 429 status - Standard "Too Many Requests" response
• Shared budget - All routes using withRateLimit() share same 5/min limit per IP

Why 5 Requests Per Minute?

Research on usability vs security shows that legitimate users rarely make more than 5 requests per minute to the same endpoint. This limit:

• ✅ Stops automated attacks
• ✅ Doesn't impact real users
• ✅ Allows reasonable form resubmissions
• ✅ Permits error recovery attempts

Why Per-IP Tracking?

• Individual users get individual limits
• An attack on one IP doesn't block others
• During distributed attack, each bot IP limited separately
• Makes attacks ineffective at scale

Implementation Files

• lib/withRateLimit.ts - Rate limiting middleware
• app/api/test-rate-limit/route.ts - Test endpoint
• scripts/test-rate-limit.js - Verification script

How to Use Rate Limiting

Basic Usage

For any endpoint that could be abused:

import { NextRequest, NextResponse } from 'next/server';
import { withRateLimit } from '@/lib/withRateLimit';

async function handler(request: NextRequest) {
  // Your business logic
  return NextResponse.json({ success: true });
}

// Apply rate limiting
export const POST = withRateLimit(handler);

export const config = {
  runtime: 'nodejs',
};

Combined with CSRF Protection

For maximum security on state-changing operations:

import { NextRequest, NextResponse } from 'next/server';
import { withRateLimit } from '@/lib/withRateLimit';
import { withCsrf } from '@/lib/withCsrf';

async function handler(request: NextRequest) {
  // Business logic
  return NextResponse.json({ success: true });
}

// Layer both protections (rate limit first, then CSRF)
export const POST = withRateLimit(withCsrf(handler));

export const config = {
  runtime: 'nodejs',
};

When to Apply Rate Limiting

✅ Always Apply To:

• Contact/support forms
• Newsletter signup
• Account creation
• Password reset requests
• File upload endpoints
• Search endpoints
• Data export endpoints
• Any expensive AI/API operations
• Webhook endpoints
• Comment/review submission
• Report generation
• Bulk operations

❌ Usually Not Needed For:

• Static asset requests (handled by CDN)
• Simple GET endpoints that only read public data
• Health check endpoints
• Endpoints already protected by authentication rate limits

Complete Examples

Example 1: Contact Form with Full Protection

// app/api/contact/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { withRateLimit } from '@/lib/withRateLimit';
import { withCsrf } from '@/lib/withCsrf';
import { validateRequest } from '@/lib/validateRequest';
import { contactFormSchema } from '@/lib/validation';
import { handleApiError } from '@/lib/errorHandler';

async function contactHandler(request: NextRequest) {
  try {
    const body = await request.json();

    const validation = validateRequest(contactFormSchema, body);
    if (!validation.success) {
      return validation.response;
    }

    const { name, email, subject, message } = validation.data;

    await sendEmail({
      to: 'support@yourapp.com',
      from: email,
      subject,
      message
    });

    return NextResponse.json({ success: true });

  } catch (error) {
    return handleApiError(error, 'contact-form');
  }
}

export const POST = withRateLimit(withCsrf(contactHandler));

export const config = {
  runtime: 'nodejs',
};

Example 2: AI API Endpoint (Cost Protection)

// app/api/summarize/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { withRateLimit } from '@/lib/withRateLimit';
import { auth } from '@clerk/nextjs/server';
import { handleApiError, handleUnauthorizedError } from '@/lib/errorHandler';
import OpenAI from 'openai';

const openai = new OpenAI();

async function summarizeHandler(request: NextRequest) {
  try {
    // Require authentication for expensive operations
    const { userId } = await auth();
    if (!userId) return handleUnauthorizedError();

    const { text } = await request.json();

    // Rate limiting prevents abuse of expensive AI API
    const response = await openai.chat.completions.create({
      model: 'gpt-4',
      messages: [
        { role: 'system', content: 'Summarize the following text concisely.' },
        { role: 'user', content: text }
      ],
      max_tokens: 150
    });

    return NextResponse.json({
      summary: response.choices[0].message.content
    });

  } catch (error) {
    return handleApiError(error, 'summarize');
  }
}

// Protect expensive AI operations with rate limiting
export const POST = withRateLimit(summarizeHandler);

export const config = {
  runtime: 'nodejs',
};

Example 3: File Upload Endpoint

// app/api/upload/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { withRateLimit } from '@/lib/withRateLimit';
import { auth } from '@clerk/nextjs/server';
import { handleApiError, handleUnauthorizedError } from '@/lib/errorHandler';

async function uploadHandler(request: NextRequest) {
  try {
    const { userId } = await auth();
    if (!userId) return handleUnauthorizedError();

    const formData = await request.formData();
    const file = formData.get('file') as File;

    if (!file) {
      return NextResponse.json(
        { error: 'No file provided' },
        { status: 400 }
      );
    }

    // Validate file size (10MB max)
    if (file.size > 10 * 1024 * 1024) {
      return NextResponse.json(
        { error: 'File too large (max 10MB)' },
        { status: 400 }
      );
    }

    // Process upload
    const uploadResult = await processFileUpload(file, userId);

    return NextResponse.json({ success: true, fileId: uploadResult.id });

  } catch (error) {
    return handleApiError(error, 'upload');
  }
}

// Prevent upload spam
export const POST = withRateLimit(uploadHandler);

export const config = {
  runtime: 'nodejs',
};

Technical Implementation Details

Rate Limiter Code (lib/withRateLimit.ts)

import { NextRequest, NextResponse } from 'next/server';

// In-memory storage for rate limiting
const rateLimitStore = new Map<string, { count: number; resetAt: number }>();

const RATE_LIMIT_WINDOW = 60 * 1000; // 1 minute in milliseconds
const MAX_REQUESTS = 5;

function getClientIp(request: NextRequest): string {
  // Check for forwarded IP (when behind proxy/load balancer)
  const forwarded = request.headers.get('x-forwarded-for');
  if (forwarded) {
    return forwarded.split(',')[0].trim();
  }

  const realIp = request.headers.get('x-real-ip');
  if (realIp) {
    return realIp;
  }

  // Fallback to direct IP
  return request.ip || 'unknown';
}

function cleanupExpiredEntries() {
  const now = Date.now();
  for (const [key, value] of rateLimitStore.entries()) {
    if (now > value.resetAt) {
      rateLimitStore.delete(key);
    }
  }
}

export function withRateLimit(
  handler: (request: NextRequest) => Promise<NextResponse>
) {
  return async (request: NextRequest) => {
    // Clean up expired entries periodically
    cleanupExpiredEntries();

    const clientIp = getClientIp(request);
    const now = Date.now();

    // Get or create rate limit entry
    let rateLimitEntry = rateLimitStore.get(clientIp);

    if (!rateLimitEntry || now > rateLimitEntry.resetAt) {
      // Create new entry or reset expired one
      rateLimitEntry = {
        count: 0,
        resetAt: now + RATE_LIMIT_WINDOW
      };
      rateLimitStore.set(clientIp, rateLimitEntry);
    }

    // Increment request count
    rateLimitEntry.count++;

    // Check if limit exceeded
    if (rateLimitEntry.count > MAX_REQUESTS) {
      const retryAfter = Math.ceil((rateLimitEntry.resetAt - now) / 1000);

      return NextResponse.json(
        {
          error: 'Too many requests',
          message: `Rate limit exceeded. Please try again in ${retryAfter} seconds.`,
          retryAfter
        },
        {
          status: 429,
          headers: {
            'Retry-After': retryAfter.toString(),
            'X-RateLimit-Limit': MAX_REQUESTS.toString(),
            'X-RateLimit-Remaining': '0',
            'X-RateLimit-Reset': rateLimitEntry.resetAt.toString()
          }
        }
      );
    }

    // Add rate limit headers to response
    const response = await handler(request);

    response.headers.set('X-RateLimit-Limit', MAX_REQUESTS.toString());
    response.headers.set(
      'X-RateLimit-Remaining',
      (MAX_REQUESTS - rateLimitEntry.count).toString()
    );
    response.headers.set('X-RateLimit-Reset', rateLimitEntry.resetAt.toString());

    return response;
  };
}

Client-Side Handling

Graceful Degradation

async function submitForm(data: FormData) {
  try {
    const response = await fetch('/api/contact', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(data)
    });

    if (response.status === 429) {
      // Rate limited
      const result = await response.json();
      alert(`Too many requests. Please wait ${result.retryAfter} seconds.`);
      return;
    }

    if (response.ok) {
      alert('Form submitted successfully!');
    } else {
      alert('Submission failed. Please try again.');
    }

  } catch (error) {
    console.error('Error:', error);
    alert('An error occurred. Please try again.');
  }
}

Automatic Retry with Exponential Backoff

async function submitWithRetry(
  data: FormData,
  maxRetries = 3
): Promise<Response> {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const response = await fetch('/api/contact', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(data)
    });

    if (response.status !== 429) {
      return response; // Success or non-rate-limit error
    }

    // Get retry-after header
    const retryAfter = parseInt(response.headers.get('Retry-After') || '60');

    if (attempt < maxRetries - 1) {
      // Wait with exponential backoff
      const delay = Math.min(retryAfter * 1000 * (2 ** attempt), 60000);
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }

  throw new Error('Max retries exceeded');
}

Show Rate Limit Status

'use client';

import { useState, useEffect } from 'react';

export function RateLimitStatus() {
  const [limit, setLimit] = useState({ remaining: 5, total: 5 });

  async function checkRateLimit() {
    const response = await fetch('/api/test-rate-limit');

    setLimit({
      remaining: parseInt(response.headers.get('X-RateLimit-Remaining') || '5'),
      total: parseInt(response.headers.get('X-RateLimit-Limit') || '5')
    });
  }

  return (
    <div>
      <p>Requests remaining: {limit.remaining}/{limit.total}</p>
      <button onClick={checkRateLimit}>Check Status</button>
    </div>
  );
}

Attack Scenarios & Protection

Attack 1: Brute Force Login

Attack:

# Attacker tries multiple passwords
for i in {1..1000}; do
  curl -X POST https://yourapp.com/api/login \
    -d "username=victim&password=attempt$i"
done

Protection:

• Requests 1-5: Processed normally
• Requests 6+: Blocked with HTTP 429
• Attack stopped after 5 attempts
• Attacker must wait 60 seconds between batches

Attack 2: Form Spam

Attack:

// Bot spams contact form
for (let i = 0; i < 1000; i++) {
  fetch('/api/contact', {
    method: 'POST',
    body: JSON.stringify({ message: 'spam' })
  });
}

Protection:

• First 5 requests: Accepted
• Subsequent requests: HTTP 429
• No email flooding
• No database spam

Attack 3: Resource Exhaustion (AI APIs)

Attack:

# Attacker abuses expensive AI endpoint
while true; do
  curl -X POST https://yourapp.com/api/summarize \
    -d '{"text": "very long text..."}'
done

Protection:

• Limited to 5 AI calls per minute per IP
• Prevents $1000s in unexpected API charges
• Legitimate users unaffected

Attack 4: Distributed Attack

Attack: Multiple IPs attacking simultaneously

Protection:

• Each IP tracked separately
• 5 requests/min limit per IP
• Attack must use 1000 IPs to make 5000 req/min
• More expensive/difficult for attacker

Testing Rate Limiting

Manual Testing

# Test rate limiting
for i in {1..10}; do
  echo "Request $i:"
  curl -s -o /dev/null -w "%{http_code}\n" \
    http://localhost:3000/api/test-rate-limit
  sleep 0.1
done

# Expected output:
# Request 1: 200
# Request 2: 200
# Request 3: 200
# Request 4: 200
# Request 5: 200
# Request 6: 429
# Request 7: 429
# Request 8: 429
# Request 9: 429
# Request 10: 429

Automated Test Script

# Run the provided test script
node scripts/test-rate-limit.js

# Expected output:
# Testing Rate Limiting (5 requests/minute per IP)
# Request  1: ✓ 200 - Success
# Request  2: ✓ 200 - Success
# Request  3: ✓ 200 - Success
# Request  4: ✓ 200 - Success
# Request  5: ✓ 200 - Success
# Request  6: ✗ 429 - Too many requests
# Request  7: ✗ 429 - Too many requests
# Request  8: ✗ 429 - Too many requests
# Request  9: ✗ 429 - Too many requests
# Request 10: ✗ 429 - Too many requests
# ✓ Rate limiting is working correctly!

Test Reset After Window

# Make 5 requests
for i in {1..5}; do
  curl http://localhost:3000/api/test-rate-limit
done

# Wait 61 seconds
sleep 61

# Try again - should succeed
curl http://localhost:3000/api/test-rate-limit

# Expected: 200 OK (limit reset)

Advanced: Custom Rate Limits

If you need different limits for different endpoints:

// lib/withCustomRateLimit.ts
import { NextRequest, NextResponse } from 'next/server';

const rateLimitStore = new Map<string, { count: number; resetAt: number }>();

export function withCustomRateLimit(
  maxRequests: number,
  windowMs: number
) {
  return (handler: (request: NextRequest) => Promise<NextResponse>) => {
    return async (request: NextRequest) => {
      const clientIp = getClientIp(request);
      const now = Date.now();
      const key = `${clientIp}:${request.url}`;

      let entry = rateLimitStore.get(key);

      if (!entry || now > entry.resetAt) {
        entry = { count: 0, resetAt: now + windowMs };
        rateLimitStore.set(key, entry);
      }

      entry.count++;

      if (entry.count > maxRequests) {
        return NextResponse.json(
          { error: 'Too many requests' },
          { status: 429 }
        );
      }

      return handler(request);
    };
  };
}

// Usage:
// export const POST = withCustomRateLimit(10, 60000)(handler); // 10 req/min
// export const POST = withCustomRateLimit(100, 3600000)(handler); // 100 req/hour

Production Considerations

Redis-Based Rate Limiting (Recommended for Production)

For production deployments with multiple servers, use Redis:

import Redis from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);

export function withRedisRateLimit(
  handler: (request: NextRequest) => Promise<NextResponse>
) {
  return async (request: NextRequest) => {
    const clientIp = getClientIp(request);
    const key = `rate-limit:${clientIp}`;

    const current = await redis.incr(key);

    if (current === 1) {
      await redis.expire(key, 60); // 60 second window
    }

    if (current > 5) {
      const ttl = await redis.ttl(key);
      return NextResponse.json(
        { error: 'Too many requests', retryAfter: ttl },
        { status: 429 }
      );
    }

    return handler(request);
  };
}

Logging Rate Limit Violations

import { logSecurityEvent } from '@/lib/security-logger';

// In withRateLimit function, when limit exceeded:
if (rateLimitEntry.count > MAX_REQUESTS) {
  // Log potential attack
  logSecurityEvent({
    type: 'RATE_LIMIT_EXCEEDED',
    ip: clientIp,
    path: request.nextUrl.pathname,
    count: rateLimitEntry.count,
    timestamp: new Date().toISOString()
  });

  return NextResponse.json(/* ... */);
}

What Rate Limiting Prevents

✅ Brute force password attacks - Main protection ✅ Credential stuffing - Automated login attempts ✅ API abuse and spam - Prevents bot spam ✅ Resource exhaustion (DoS) - Prevents overwhelming server ✅ Excessive costs from AI/paid APIs - Cost protection ✅ Scraping and data harvesting - Slows down bulk collection ✅ Account enumeration - Prevents discovering valid accounts

Common Mistakes to Avoid

❌ DON'T skip rate limiting on expensive operations ❌ DON'T use the same endpoint for different operations without separate limits ❌ DON'T rely solely on client-side rate limiting ❌ DON'T forget to handle HTTP 429 responses gracefully in frontend ❌ DON'T set limits too high (defeats purpose) or too low (hurts UX)

✅ DO apply to all public endpoints that could be abused ✅ DO use Redis in production for multi-server deployments ✅ DO log rate limit violations for security monitoring ✅ DO provide clear error messages with retry-after time ✅ DO combine with CSRF for maximum protection

References

• OWASP API Security Top 10 - Unrestricted Resource Consumption: https://owasp.org/API-Security/editions/2023/en/0xa4-unrestricted-resource-consumption/
• Rate Limiting Best Practices: https://cloud.google.com/architecture/rate-limiting-strategies-techniques
• HTTP 429 Status Code: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429

Next Steps

• For CSRF protection: Use csrf-protection skill
• For input validation: Use input-validation skill
• For testing rate limiting: Use security-testing skill
• For complete API security: Combine rate limiting + CSRF + validation