Implement hybrid rate limiting: built-in API + KV

[neuromechanist]

Summary

Hybrid rate limiting approach: Built-in Rate Limiting API for per-minute (bot protection) + Workers KV for per-hour (human abuse prevention).

Problem with Full Built-in API Approach

Initial attempt to use only the built-in API had critical issues:

1. Cannot do hourly limits - Only supports period: 10 or period: 60 seconds
2. Static configuration - Cannot dynamically vary limits for dev vs prod
3. Per-location enforcement - Not globally consistent across edge locations

Hybrid Solution

Use the best tool for each job:

Purpose	Tool	Why
Per-minute (bot protection)	Built-in API	Fast (<1ms), catches bots immediately
Per-hour (human abuse)	KV	Global consistency, arbitrary time windows

Benefits

• ✅ 50% reduction in KV writes - 1 write/request (hour only) vs 2 (minute + hour)
• ✅ Faster bot protection - <1ms (built-in API) vs ~10-50ms (KV) for critical first check
• ✅ Global hourly limits - KV provides consistency across all Cloudflare edge locations
• ✅ Pro Plan friendly - 1 write/request is unlimited on Pro Plan
• ✅ Best of both worlds - Speed where it matters (bots), global where it matters (humans)

Rate Limits

Per IP address:

Environment	Per Minute (Bot)	Per Hour (Human)
Production	10	20 (~1 question/3 min)
Development	60	100

Scope: Limits are per IP address, not per session. 20/hour prevents abuse while allowing reasonable research sessions.

Changes

wrangler.toml:

• Add [[ratelimits]] binding for per-minute (built-in API)
• Keep [[kv_namespaces]] for per-hour (KV)
• Separate namespace IDs for prod/dev rate limiters

index.js:

• Check built-in API first (fast bot check)
• Then check KV for hourly limit
• Only 1 KV write per request instead of 2

README.md:

• Document hybrid approach and rationale
• Explain rate limit scope (per IP)
• Update setup instructions

Implementation Details

Per-Minute (Built-in API):

# wrangler.toml
[[ratelimits]]
name = "RATE_LIMITER_MINUTE"
namespace_id = "1001"
simple = { limit = 10, period = 60 }

// Fast check, <1ms
const { success } = await env.RATE_LIMITER_MINUTE.limit({ key: ip });

Per-Hour (KV):

// Global consistency
const hourKey = `rl:hour:${ip}:${Math.floor(now / 3600)}`;
const hourCount = parseInt(await env.RATE_LIMITER_KV.get(hourKey) || '0');
if (hourCount >= CONFIG.RATE_LIMIT_PER_HOUR) {
  return { allowed: false };
}
await env.RATE_LIMITER_KV.put(hourKey, (hourCount + 1).toString(), { expirationTtl: 7200 });

Deployment

# Dev
cd workers/osa-worker
wrangler deploy --env dev

# Production
wrangler deploy

No additional setup needed - KV namespaces already exist, built-in API is automatic.

Testing

# Test per-minute limit (should hit after 10 requests)
for i in {1..15}; do 
  curl -s -X POST https://osa-worker-dev.yahyaqaraeen.workers.dev/hed/ask \
    -H "Content-Type: application/json" \
    -d '{"question":"test"}' \
    -w " - %{http_code}\n"
  sleep 0.5
done

Expected: First 10 fail Turnstile (403), then 429 (rate limit)

Performance Impact

Metric	Old (KV only)	New (Hybrid)
KV writes/request	2	1
Bot check latency	~10-50ms	<1ms
Hourly limit scope	Global ✓	Global ✓
Writes/hour @ 10 req/min	1,200	600

References

• Issue: Replace KV-based rate limiting with Cloudflare's built-in Rate Limiting API #129
• Cloudflare Rate Limiting API docs
• Review findings that identified built-in API limitations

Closes #129