diff --git a/docs/privacy-logging.md b/docs/privacy-logging.md index da0464f4..99f6ef8c 100644 --- a/docs/privacy-logging.md +++ b/docs/privacy-logging.md @@ -1,30 +1,17 @@ -# Privacy Logging Guidelines +# Privacy Logging ## Overview -Disciplr is committed to protecting user data. To ensure that sensitive Personally Identifiable Information (PII) and credentials are never written to long-term storage via logs, we have implemented **Pino-based structured logging** with automatic redaction of sensitive fields. -## Architecture +`src/middleware/privacy-logger.ts` implements privacy-hardened HTTP request logging. It: -### Structured JSON Logging with Pino -The backend now uses **Pino** for efficient, structured JSON logging that is: -- **Machine-readable**: Emits single-line JSON per log event for easy ingestion into log aggregators (Datadog, ELK, Grafana Loki, etc.) -- **Secure by default**: Sensitive fields are automatically redacted via Pino's `redact` configuration -- **Developer-friendly**: Pretty-printed output in development for readability -- **Zero-overhead**: Minimal performance impact compared to console logging +- Recursively redacts all PII from request bodies, query strings, and headers before emitting any log output. +- Emits **exactly one structured JSON line per request** to `stdout` via `console.log`, on response finish. +- Exports a standalone `redact()` utility that any module can call. +- Never mutates the original request object. -### Two-Layer Redaction Strategy -1. **Pino built-in redaction** (`src/middleware/logger.ts`): Automatically redacts fields matching configured paths -2. **Explicit redaction engine** (`src/middleware/privacy-logger.ts`): Additional `redact()` function for explicit control and backward compatibility +## Log Schema -### Correlation IDs -All logs include correlation IDs (from `x-correlation-id` or `x-request-id` headers) for end-to-end request tracing: -```json -{ - "correlationId": "550e8400-e29b-41d4-a716-446655440000", - "event": "http.request", - "durationMs": 45 -} -``` +Every log line has exactly these top-level keys — no more, no less. ## Redaction Policy @@ -79,163 +66,109 @@ The redaction engine is recursive and works safely across: ### Request Logger (`src/middleware/requestLogger.ts`) Emits structured JSON for every HTTP request: + ```json { - "correlationId": "550e8400-e29b-41d4-a716-446655440000", + "timestamp": "2024-06-18T22:00:00.000Z", + "level": "info", "event": "http.request", - "req": { - "method": "POST", - "url": "/api/vaults", - "path": "/api/vaults", - "headers": { "authorization": "***REDACTED***" }, - "body": { "email": "***REDACTED***", "amount": 1000 }, - "userId": "user123", - "userRole": "admin" - }, - "res": { "statusCode": 201 }, + "service": "disciplr-backend", + "method": "POST", + "url": "/api/auth/login", + "status": 200, "durationMs": 45, - "msg": "POST /api/vaults 201 45ms" + "ip": "10.20.x.x", + "body": { "email": "[REDACTED]", "amount": 100 }, + "query": null, + "headers": { "content-type": "application/json", "authorization": "[REDACTED]" } } ``` -**Log Level Selection**: -- `error` (5xx status codes) -- `warn` (4xx status codes) -- `info` (2xx status codes) -- `debug` (1xx status codes) +The schema is snapshot-tested in `src/tests/privacy-logger.redaction.test.ts`. -### Privacy Logger (`src/middleware/privacy-logger.ts`) -Emits privacy-focused events with IP masking: -```json -{ - "correlationId": "550e8400-e29b-41d4-a716-446655440000", - "event": "privacy.request_logged", - "ip": { - "original": "192.168.1.1", - "masked": "192.168.x.x" - }, - "request": { - "method": "POST", - "url": "/api/test", - "headers": { "authorization": "***REDACTED***" }, - "body": { "email": "***REDACTED***" } - }, - "timestamp": "2025-06-02T14:32:10.000Z", - "msg": "Privacy-logged: POST /api/test" -} -``` +## Redaction Marker -**IP Masking**: -- IPv4: `192.168.1.1` → `192.168.x.x` (mask last 2 octets) -- IPv6: `2001:0db8:85a3::` → `2001:0db8:85a3:xxxx:xxxx:xxxx:xxxx:xxxx` (mask last 5 groups) +Sensitive values are replaced with the string `"[REDACTED]"` (exported as `REDACTED`). -## Configuration +## What Gets Redacted -### Logger Setup (`src/middleware/logger.ts`) -```typescript -export const logger = createLogger() -``` +### Sensitive Field Names (case-insensitive key match) -**Environment Variables**: -- `NODE_ENV` — Enables pretty-printing in `development` mode -- `LOG_LEVEL` — Set minimum log level (`debug`, `info`, `warn`, `error`; default: `info`) +| Key | Why | +|-----|-----| +| `password`, `passwordHash` | Credentials | +| `token`, `accessToken`, `refreshToken` | Auth tokens | +| `apiKey`, `api_key` | API keys | +| `secret`, `credential`, `credentials` | Generic secrets | +| `authorization` | Auth header | +| `x-api-key`, `x-auth-token` | Custom auth headers | +| `cookie` | Session cookies | +| `ssn` | Social Security Number | +| `creditCard`, `credit_card`, `cvv`, `pin` | Payment data | +| `email` | Email address | +| `clientSecret` | OAuth secret | +| `creator`, `successDestination`, `failureDestination` | Vault addresses | -### In Development -Logs are pretty-printed with colors and indentation for readability: -``` - INFO (disciplr-backend): POST /api/vaults 201 45ms - req: { - "method": "POST", - "url": "/api/vaults" - } -``` +### PII Patterns (applied to string values regardless of key name) -### In Production -Logs are emitted as single-line JSON: -``` -{"correlationId":"550e8400...","event":"http.request","req":{...},"res":{"statusCode":201},"durationMs":45} -``` +| Pattern | Example | +|---------|---------| +| Email address (`/[^@\s]+@[^@\s]+\.[^@\s]+/`) | `user@example.com` | +| JWT (`/^[A-Za-z0-9\-_]+\.[A-Za-z0-9\-_]+\.[A-Za-z0-9\-_]+$/`) | `eyJ...` | -## Integration with Log Aggregators +## IP Masking -### Example: Datadog -```bash -# Install Datadog agent on your infrastructure -# Provide Datadog API key - -# Datadog will automatically ingest JSON logs and parse fields: -service: disciplr-backend -correlationId: 550e8400-e29b-41d4-a716-446655440000 -event: http.request -req.method: POST -``` +| Input | Output | +|-------|--------| +| `192.168.1.1` (IPv4) | `192.168.x.x` | +| `2001:0db8:85a3::7334` (IPv6) | `2001:0db8:85a3:xxxx:xxxx:xxxx:xxxx:xxxx` | +| empty / unparseable | `unknown` | -### Example: Grafana Loki -Configure Promtail to scrape and parse JSON: -```yaml -scrape_configs: - - job_name: disciplr-backend - static_configs: - - targets: - - localhost - labels: - job: disciplr-backend - relabel_configs: - - source_labels: [__address__] - target_label: __param_target -``` +## Exports -### Example: ELK Stack -Logstash will parse JSON automatically: -```json -{ - "correlationId": "550e8400-e29b-41d4-a716-446655440000", - "event": "http.request", - "req.method": "POST" -} +```typescript +import { redact, maskIp, shouldRedact, privacyLogger, REDACTED } from './middleware/privacy-logger.js' ``` -## Adding New Redactions +### `redact(value: T): T` -### Option 1: Add to Pino Redact Paths (Automatic) -Edit `src/middleware/logger.ts` and add the path to the `redact.paths` array: -```typescript -redact: { - paths: [ - 'req.body.newSensitiveField', // Add here - // ...existing paths - ] -} -``` +Deep-copies `value` and replaces every sensitive field value and every string matching a PII pattern with `REDACTED`. Input is never mutated. Handles circular references, `Date`, `RegExp`, `Buffer`, nested objects, and arrays. -### Option 2: Add to Explicit Redaction List (Backward Compat) -Edit `src/middleware/privacy-logger.ts` and add the field key to `SENSITIVE_FIELDS`: ```typescript -const SENSITIVE_FIELDS = new Set([ - 'email', - 'password', - 'newSensitiveField', // Add here - // ...existing fields -]) +redact({ password: 'secret', amount: 100 }) +// => { password: '[REDACTED]', amount: 100 } + +redact({ nested: { email: 'a@b.com' } }) +// => { nested: { email: '[REDACTED]' } } ``` -## Accessing Logs in Downstream Handlers +### `maskIp(ip: string): string` + +Returns a partially masked IP string (see table above). + +### `shouldRedact(key: string): boolean` + +Returns `true` if the key name (case-insensitive) is in the sensitive-field list. + +### `privacyLogger` + +Express middleware. Register it after body parsers and before routes: -Request handlers can use the injected logger for consistent structured logging: ```typescript -import { Request, Response, NextFunction } from 'express' +app.use(express.json()) +app.use(privacyLogger) +app.use('/api', router) +``` -export const myHandler = (req: Request, res: Response, next: NextFunction) => { - const logger = (req as any).logger - const correlationId = (req as any).correlationId +## Error Path - logger.info({ event: 'my_event', data: {...} }, 'Processing request') - - res.json({ message: 'Success' }) -} +If log serialization fails for any reason, a minimal safe fallback is emitted and `next()` is still called: + +```json +{ "level": "error", "event": "privacy-logger.serialization-failure", "timestamp": "..." } ``` -All logs from the same request will automatically share the correlation ID. +No request data is included in the fallback. ## Privacy Endpoint Security @@ -250,26 +183,53 @@ The `GET /api/privacy/export` and `DELETE /api/privacy/account` endpoints implem ## Development vs Production -Redaction runs in **all environments** (development, staging, production) to: -- Prevent accidental ingestion of PII into development databases or logs -- Ensure parity in testing across environments -- Maintain security posture uniformly +## Adding New Sensitive Fields + +Edit `SENSITIVE_KEYS` in `src/middleware/privacy-logger.ts`: + +```typescript +const SENSITIVE_KEYS = new Set([ + // ... existing keys ... + 'myNewSensitiveField', +]) +``` + +Update the snapshot after changing the set: -Debugging should rely on non-sensitive identifiers: -- User IDs: `user123` (visible) -- Vault IDs: `vault456` (visible) -- Transaction references: `tx789` (visible) -- Email addresses: `***REDACTED***` (hidden) -- API keys: `***REDACTED***` (hidden) +```bash +npx jest src/tests/privacy-logger.redaction.test.ts --updateSnapshot +``` ## Testing -Run privacy logger tests to verify redaction coverage: ```bash +# Run the hardened redaction test suite +npx jest src/tests/privacy-logger.redaction.test.ts + +# Update snapshot after intentional schema changes +npx jest src/tests/privacy-logger.redaction.test.ts --updateSnapshot + npm test -- src/tests/privacy-logger.test.ts npm test -- src/tests/exportQueue.pii.test.ts ``` +Test coverage includes: + +- Primitive passthrough +- Email and JWT value-pattern redaction +- All sensitive key names (case-insensitive) +- Nested object and array redaction +- Deeply nested PII +- Circular reference protection +- No input mutation +- `Date`, `RegExp`, `Buffer` serialization +- `maskIp` IPv4 / IPv6 / unknown +- Middleware schema (exact top-level keys) +- `null` body and `null` query +- Header redaction (`authorization`, `x-api-key`, `x-auth-token`, `cookie`) +- Serialization-failure fallback +- Snapshot of a representative request + Coverage includes: - ✅ Sensitive field redaction at all nesting levels - ✅ IP masking (IPv4 and IPv6) @@ -297,4 +257,4 @@ This logging architecture supports compliance with: - **GDPR** — Redaction prevents PII leakage to log storage; right to access and erasure endpoints are rate-limited and ownership-gated - **HIPAA** — Sensitive fields are never stored in unencrypted logs - **SOC 2** — Structured logging enables audit trail generation -- **PCI DSS** — Passwords, tokens, and API keys are redacted +- **PCI DSS** — Passwords, tokens, and API keys are redacted \ No newline at end of file diff --git a/src/middleware/privacy-logger.ts b/src/middleware/privacy-logger.ts index 8daba951..214ea741 100644 --- a/src/middleware/privacy-logger.ts +++ b/src/middleware/privacy-logger.ts @@ -1,165 +1,166 @@ import { Request, Response, NextFunction } from 'express' -import { Buffer } from 'node:buffer' -import { isIP } from 'node:net' -import { logger, withCorrelationId, getOrGenerateCorrelationId } from './logger.js' -import { utcNow } from '../utils/timestamps.js' - -export const REDACTION_MARKER = '***REDACTED***' - -export const SENSITIVE_KEYS = new Set([ - 'email', - 'password', - 'token', - 'accesstoken', - 'refreshtoken', - 'apikey', - 'api_key', - 'secret', - 'clientsecret', - 'creator', - 'successdestination', - 'failuredestination', - 'authorization', - 'cookie', - 'x-api-key' + +export const REDACTED = '[REDACTED]' + +const SENSITIVE_KEYS = new Set([ + 'password', + 'passwordhash', + 'token', + 'accesstoken', + 'refreshtoken', + 'apikey', + 'api_key', + 'secret', + 'authorization', + 'x-api-key', + 'x-auth-token', + 'credential', + 'credentials', + 'ssn', + 'creditcard', + 'credit_card', + 'cvv', + 'pin', + 'cookie', + // legacy / extra fields + 'clientsecret', + 'email', + 'creator', + 'successdestination', + 'failuredestination', ]) -const PII_VALUE_PATTERNS = [ - /^[^\s@]+@[^\s@]+\.[^\s@]+$/, - /^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/, -] +const EMAIL_RE = /[^@\s]+@[^@\s]+\.[^@\s]+/ +const JWT_RE = /^[A-Za-z0-9\-_]+\.[A-Za-z0-9\-_]+\.[A-Za-z0-9\-_]+$/ +/** Returns true when a field name should always be redacted. */ export function shouldRedact(key: string): boolean { - return SENSITIVE_KEYS.has(key.toLowerCase()) -} - -function shouldRedactValue(value: string): boolean { - return PII_VALUE_PATTERNS.some(pattern => pattern.test(value)) + return SENSITIVE_KEYS.has(key.toLowerCase()) } -type RequestWithPrivacyContext = Request & { - correlationId?: string - logger?: ReturnType -} +/** + * Pure recursive redactor. Deep-copies input and replaces: + * - values under sensitive field names, and + * - string values matching email or JWT patterns + * with REDACTED. Never mutates the original. + */ +export function redact(value: T, seen = new WeakSet()): T { + if (value === null || value === undefined) return value -export function redact(value: unknown, seen = new WeakSet()): unknown { - if (value === null || value === undefined) { - return value - } - + if (typeof value !== 'object') { if (typeof value === 'string') { - return shouldRedactValue(value) ? REDACTION_MARKER : value + if (EMAIL_RE.test(value) || JWT_RE.test(value)) { + return REDACTED as unknown as T + } } + return value + } - // Primitive values - if (typeof value !== 'object') { - return value - } + if (seen.has(value as object)) return REDACTED as unknown as T + seen.add(value as object) - // Circular reference check - if (seen.has(value)) { - return '[Circular]' - } - seen.add(value) - - if (Array.isArray(value)) { - return value.map(item => redact(item, seen)) - } + if (Array.isArray(value)) { + return value.map((item) => redact(item, seen)) as unknown as T + } - // Handle common objects that are not plain objects - if (value instanceof Date) { - return value.toISOString() - } - if (value instanceof RegExp) { - return value.toString() - } - if (Buffer.isBuffer(value)) { - return '[Buffer]' - } - - const result: Record = {} - for (const [k, v] of Object.entries(value)) { - if (shouldRedact(k)) { - result[k] = REDACTION_MARKER - } else { - result[k] = redact(v, seen) - } - } - return result -} + if (value instanceof Date) return value.toISOString() as unknown as T + if (value instanceof RegExp) return value.toString() as unknown as T + if (Buffer.isBuffer(value)) return '[Buffer]' as unknown as T -/** - * Privacy logger middleware using Pino for structured JSON output. - * - * Masks PII in logs by: - * - Masking IP addresses (partial redaction) - * - Redacting sensitive fields in request bodies and headers - * - Emitting structured JSON for log aggregators - * - * Note: Pino's built-in redaction (configured in logger.ts) also handles - * sensitive field redaction automatically. This middleware adds additional - * IP masking and structured event logging. - */ -export const privacyLogger = (req: Request, _res: Response, next: NextFunction) => { - const correlationId = getOrGenerateCorrelationId(req) - const privacyLog = withCorrelationId(logger, correlationId) - - const ip = req.ip || req.socket.remoteAddress || 'unknown' - const maskedIp = maskIp(ip) - - const timestamp = utcNow() - const method = req.method - const url = req.url - - // Store correlation ID and logger on request for downstream handlers - const requestWithContext = req as RequestWithPrivacyContext - requestWithContext.correlationId = correlationId - requestWithContext.logger = privacyLog - - // Redact sensitive fields before logging - // (Pino will also redact based on its configuration, but we do it here - // for explicit control and compatibility with existing tests) - const sanitizedBody = redact(req.body) - const sanitizedHeaders = redact(req.headers) - - // Emit structured privacy event log - privacyLog.debug( - { - event: 'privacy.request_logged', - ip: { - original: ip, - masked: maskedIp, - }, - request: { - method, - url, - headers: sanitizedHeaders, - body: sanitizedBody, - }, - timestamp, - }, - `Privacy-logged: ${method} ${url}`, - ) - - next() + const result: Record = {} + + for (const [k, v] of Object.entries(value as Record)) { + result[k] = shouldRedact(k) ? REDACTED : redact(v, seen) + } + + return result as unknown as T } +/** Mask IPv4 to a.b.x.x, IPv6 to first three groups + xxxx segments. */ export function maskIp(ip: string): string { - if (isIP(ip) === 6) { - const [left, right = ''] = ip.split('::') - const leftGroups = left ? left.split(':') : [] - const rightGroups = right ? right.split(':') : [] - const missingGroups = Math.max(0, 8 - leftGroups.length - rightGroups.length) - const groups = right - ? [...leftGroups, ...Array(missingGroups).fill('0'), ...rightGroups] - : leftGroups - return `${groups[0]}:${groups[1]}:${groups[2]}:xxxx:xxxx:xxxx:xxxx:xxxx` - } + if (!ip) return 'unknown' - if (isIP(ip) === 4) { - const parts = ip.split('.') - return `${parts[0]}.${parts[1]}.x.x` - } + if (ip.includes(':')) { + const groups = ip.split(':') + return groups.slice(0, 3).join(':') + ':xxxx:xxxx:xxxx:xxxx:xxxx' + } + + const parts = ip.split('.') + if (parts.length === 4) return `${parts[0]}.${parts[1]}.x.x` - return 'x.x.x.x' + return 'unknown' } + +interface LogLine { + timestamp: string + level: 'info' + event: 'http.request' + service: 'disciplr-backend' + method: string + url: string + status: number + durationMs: number + ip: string + body: Record | null + query: Record | null + headers: Record +} + +/** + * Privacy-hardened request logger middleware. + * + * Emits exactly one structured JSON line per request (on response finish) + * via console.log. All PII is redacted before emission. + * Never mutates req/res. Always calls next(). + */ +export const privacyLogger = ( + req: Request, + res: Response, + next: NextFunction, +): void => { + const start = Date.now() + + res.on('finish', () => { + try { + const rawIp = req.ip ?? req.socket?.remoteAddress ?? '' + const rawBody = req.body + const rawQuery = req.query as Record + + const line: LogLine = { + timestamp: new Date().toISOString(), + level: 'info', + event: 'http.request', + service: 'disciplr-backend', + method: req.method, + url: req.url, + status: res.statusCode, + durationMs: Date.now() - start, + ip: rawIp ? maskIp(rawIp) : 'unknown', + body: + rawBody !== null && + rawBody !== undefined && + typeof rawBody === 'object' && + !Array.isArray(rawBody) + ? redact(rawBody as Record) + : null, + query: + rawQuery && Object.keys(rawQuery).length > 0 + ? redact(rawQuery) + : null, + headers: redact(req.headers as Record), + } + + console.log(JSON.stringify(line)) + } catch { + console.log( + JSON.stringify({ + level: 'error', + event: 'privacy-logger.serialization-failure', + timestamp: new Date().toISOString(), + }), + ) + } + }) + + next() +} \ No newline at end of file diff --git a/src/tests/privacy-logger.redaction.test.ts b/src/tests/privacy-logger.redaction.test.ts new file mode 100644 index 00000000..3b67cb7e --- /dev/null +++ b/src/tests/privacy-logger.redaction.test.ts @@ -0,0 +1,402 @@ +import { jest, describe, it, expect, beforeEach, afterEach } from '@jest/globals' +import type { Request, Response, NextFunction } from 'express' +import { redact, maskIp, shouldRedact, privacyLogger, REDACTED } from '../middleware/privacy-logger.js' + +// --------------------------------------------------------------------------- +// redact() +// --------------------------------------------------------------------------- +describe('redact()', () => { + it('passes through primitives unchanged', () => { + expect(redact(42)).toBe(42) + expect(redact(true)).toBe(true) + expect(redact(null)).toBeNull() + expect(redact(undefined)).toBeUndefined() + }) + + it('redacts email-pattern strings by value regardless of key', () => { + expect(redact({ field: 'user@example.com' })).toEqual({ field: REDACTED }) + }) + + it('redacts JWT-pattern strings by value', () => { + const jwt = 'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1c2VyIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' + expect(redact({ tok: jwt })).toEqual({ tok: REDACTED }) + }) + + it('does not redact non-email, non-JWT plain strings', () => { + expect(redact({ name: 'Alice' })).toEqual({ name: 'Alice' }) + }) + + it('redacts sensitive keys case-insensitively', () => { + expect(redact({ PASSWORD: 'secret' })).toEqual({ PASSWORD: REDACTED }) + expect(redact({ ApiKey: 'k' })).toEqual({ ApiKey: REDACTED }) + expect(redact({ Authorization: 'Bearer tok' })).toEqual({ Authorization: REDACTED }) + expect(redact({ 'x-api-key': 'k' })).toEqual({ 'x-api-key': REDACTED }) + expect(redact({ 'x-auth-token': 'k' })).toEqual({ 'x-auth-token': REDACTED }) + expect(redact({ cookie: 'session=abc' })).toEqual({ cookie: REDACTED }) + }) + + it('redacts all spec-listed sensitive keys', () => { + const keys = [ + 'password', 'passwordHash', 'token', 'accessToken', 'refreshToken', + 'apiKey', 'api_key', 'secret', 'authorization', 'x-api-key', + 'x-auth-token', 'credential', 'credentials', 'ssn', 'creditCard', + 'credit_card', 'cvv', 'pin', 'cookie', + ] + for (const k of keys) { + expect((redact({ [k]: 'value' }) as Record)[k]).toBe(REDACTED) + } + }) + + it('leaves non-sensitive fields unchanged', () => { + expect(redact({ id: 'abc', amount: 100 })).toEqual({ id: 'abc', amount: 100 }) + }) + + it('recursively redacts nested objects', () => { + const input = { user: { email: 'a@b.com', name: 'Bob' } } + expect(redact(input)).toEqual({ user: { email: REDACTED, name: 'Bob' } }) + }) + + it('recursively redacts objects inside arrays', () => { + const input = { users: [{ id: '1', password: 'x' }, { id: '2', password: 'y' }] } + expect(redact(input)).toEqual({ + users: [{ id: '1', password: REDACTED }, { id: '2', password: REDACTED }], + }) + }) + + it('handles arrays of primitives without modification', () => { + expect(redact(['a', 'b'])).toEqual(['a', 'b']) + }) + + it('handles deeply nested PII', () => { + const input = { a: { b: { c: { apiKey: 'deep-secret' } } } } + expect(redact(input)).toEqual({ a: { b: { c: { apiKey: REDACTED } } } }) + }) + + it('handles circular references without throwing', () => { + const obj: Record = { safe: 'value' } + obj.self = obj + const result = redact(obj) as Record + expect(result.safe).toBe('value') + expect(result.self).toBe(REDACTED) + }) + + it('does not mutate the original object', () => { + const input = { password: 'secret', name: 'Alice' } + const copy = { ...input } + redact(input) + expect(input).toEqual(copy) + }) + + it('serializes Date, RegExp, Buffer safely', () => { + const d = new Date('2024-01-01T00:00:00Z') + const result = redact({ d, r: /x/, b: Buffer.from('hi') }) as Record + expect(result.d).toBe(d.toISOString()) + expect(result.r).toBe('/x/') + expect(result.b).toBe('[Buffer]') + }) + + it('handles empty object and empty array', () => { + expect(redact({})).toEqual({}) + expect(redact([])).toEqual([]) + }) +}) + +// --------------------------------------------------------------------------- +// shouldRedact() +// --------------------------------------------------------------------------- +describe('shouldRedact()', () => { + it('returns true for known sensitive keys', () => { + expect(shouldRedact('password')).toBe(true) + expect(shouldRedact('EMAIL')).toBe(true) // case-insensitive match + expect(shouldRedact('email')).toBe(true) + expect(shouldRedact('token')).toBe(true) + expect(shouldRedact('cookie')).toBe(true) + }) + + it('returns false for safe keys', () => { + expect(shouldRedact('id')).toBe(false) + expect(shouldRedact('name')).toBe(false) + expect(shouldRedact('status')).toBe(false) + }) +}) + +// --------------------------------------------------------------------------- +// maskIp() +// --------------------------------------------------------------------------- +describe('maskIp()', () => { + it('masks IPv4 last two octets', () => { + expect(maskIp('192.168.1.1')).toBe('192.168.x.x') + expect(maskIp('10.0.0.1')).toBe('10.0.x.x') + }) + + it('masks IPv6 keeping first three groups', () => { + expect(maskIp('2001:0db8:85a3:0000:0000:8a2e:0370:7334')).toBe( + '2001:0db8:85a3:xxxx:xxxx:xxxx:xxxx:xxxx', + ) + }) + + it('returns "unknown" for empty or malformed input', () => { + expect(maskIp('')).toBe('unknown') + expect(maskIp('not-an-ip')).toBe('unknown') + }) +}) + +// --------------------------------------------------------------------------- +// privacyLogger middleware +// --------------------------------------------------------------------------- +describe('privacyLogger middleware', () => { + let consoleSpy: ReturnType + let finishHandler: () => void + let req: Partial + let res: Partial + let next: jest.Mock + + function buildReq(overrides: Partial = {}): Partial { + return { + method: 'POST', + url: '/api/vaults', + ip: '192.168.1.1', + body: { amount: 100 }, + query: {}, + headers: { 'content-type': 'application/json' }, + socket: { remoteAddress: '192.168.1.1' } as never, + ...overrides, + } + } + + function buildRes(): Partial { + const handlers: Record void> = {} + return { + statusCode: 200, + on(event: string, handler: () => void) { + handlers[event] = handler + if (event === 'finish') finishHandler = handler + return this as Response + }, + } as Partial + } + + beforeEach(() => { + consoleSpy = jest.spyOn(console, 'log').mockImplementation(() => {}) + next = jest.fn() + finishHandler = () => {} + req = buildReq() + res = buildRes() + }) + + afterEach(() => { + consoleSpy.mockRestore() + }) + + function getLogLine(): Record { + finishHandler() + expect(consoleSpy).toHaveBeenCalledTimes(1) + return JSON.parse((consoleSpy.mock.calls[0] as string[])[0]) + } + + // ---- schema ---- + + it('emits exactly one JSON line on response finish', () => { + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + finishHandler() + expect(consoleSpy).toHaveBeenCalledTimes(1) + expect(() => JSON.parse((consoleSpy.mock.calls[0] as string[])[0])).not.toThrow() + }) + + it('always calls next()', () => { + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + expect(next).toHaveBeenCalledTimes(1) + }) + + it('emits the exact stable set of top-level keys', () => { + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(Object.keys(line).sort()).toEqual( + ['body', 'durationMs', 'event', 'headers', 'ip', 'level', 'method', 'query', 'service', 'status', 'timestamp', 'url'], + ) + }) + + it('sets fixed string fields correctly', () => { + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(line.level).toBe('info') + expect(line.event).toBe('http.request') + expect(line.service).toBe('disciplr-backend') + }) + + it('captures method, url, and status from req/res', () => { + ;(res as { statusCode: number }).statusCode = 201 + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(line.method).toBe('POST') + expect(line.url).toBe('/api/vaults') + expect(line.status).toBe(201) + }) + + it('includes a numeric durationMs >= 0', () => { + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(typeof line.durationMs).toBe('number') + expect(line.durationMs as number).toBeGreaterThanOrEqual(0) + }) + + it('includes an ISO timestamp', () => { + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(new Date(line.timestamp as string).toISOString()).toBe(line.timestamp) + }) + + // ---- ip masking ---- + + it('masks IPv4 addresses in the log line', () => { + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(line.ip).toBe('192.168.x.x') + }) + + it('uses "unknown" when ip is absent', () => { + req = buildReq({ ip: undefined, socket: { remoteAddress: undefined } as never }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(line.ip).toBe('unknown') + }) + + // ---- body ---- + + it('sets body to null when req.body is absent', () => { + req = buildReq({ body: undefined }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(line.body).toBeNull() + }) + + it('sets body to null when req.body is not a plain object', () => { + req = buildReq({ body: 'raw-string' as never }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(line.body).toBeNull() + }) + + it('redacts sensitive fields in body', () => { + req = buildReq({ body: { password: 'secret', amount: 50 } }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect((line.body as Record).password).toBe(REDACTED) + expect((line.body as Record).amount).toBe(50) + }) + + // ---- query ---- + + it('sets query to null when query is empty', () => { + req = buildReq({ query: {} }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect(line.query).toBeNull() + }) + + it('includes and redacts non-empty query', () => { + req = buildReq({ query: { token: 'abc', page: '1' } as never }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect((line.query as Record).token).toBe(REDACTED) + expect((line.query as Record).page).toBe('1') + }) + + // ---- header redaction ---- + + it('redacts authorization header', () => { + req = buildReq({ headers: { authorization: 'Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1c2VyIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' } }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect((line.headers as Record).authorization).toBe(REDACTED) + }) + + it('redacts x-api-key header', () => { + req = buildReq({ headers: { 'x-api-key': 'my-api-key' } }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect((line.headers as Record)['x-api-key']).toBe(REDACTED) + }) + + it('redacts x-auth-token header', () => { + req = buildReq({ headers: { 'x-auth-token': 'my-auth-token' } }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect((line.headers as Record)['x-auth-token']).toBe(REDACTED) + }) + + it('redacts cookie header', () => { + req = buildReq({ headers: { cookie: 'session=xyz' } }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect((line.headers as Record).cookie).toBe(REDACTED) + }) + + it('preserves safe headers', () => { + req = buildReq({ headers: { 'content-type': 'application/json', 'user-agent': 'jest' } }) + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + const line = getLogLine() + expect((line.headers as Record)['content-type']).toBe('application/json') + }) + + // ---- error path ---- + + it('emits safe fallback log on serialization failure and still calls next()', () => { + // Cause redact to surface a non-serializable output by monkey-patching JSON.stringify + const orig = JSON.stringify + let callCount = 0 + jest.spyOn(JSON, 'stringify').mockImplementation((...args) => { + callCount++ + if (callCount === 1) throw new Error('serialization failure') + return orig.apply(JSON, args as [unknown]) + }) + + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + finishHandler() + + const fallback = JSON.parse((consoleSpy.mock.calls[0] as string[])[0]) + expect(fallback.level).toBe('error') + expect(fallback.event).toBe('privacy-logger.serialization-failure') + expect(fallback).toHaveProperty('timestamp') + expect(Object.keys(fallback)).toHaveLength(3) + + jest.restoreAllMocks() + }) + + // ---- snapshot ---- + + it('snapshot: structured log line for a request with sensitive fields', () => { + req = buildReq({ + method: 'POST', + url: '/api/auth/login', + ip: '10.20.30.40', + body: { email: 'user@example.com', password: 'hunter2' }, + query: {}, + headers: { + 'content-type': 'application/json', + authorization: 'Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1c2VyIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c', + 'x-api-key': 'raw-api-key-value', + }, + }) + ;(res as { statusCode: number }).statusCode = 200 + + privacyLogger(req as Request, res as Response, next as unknown as NextFunction) + finishHandler() + + const line = JSON.parse((consoleSpy.mock.calls[0] as string[])[0]) + + // Replace non-deterministic fields for snapshot stability + line.timestamp = '2024-01-01T00:00:00.000Z' + line.durationMs = 0 + + expect(line).toMatchSnapshot() + + // Explicit security assertions on top of snapshot + expect(line.body.email).toBe(REDACTED) + expect(line.body.password).toBe(REDACTED) + expect(line.headers.authorization).toBe(REDACTED) + expect(line.headers['x-api-key']).toBe(REDACTED) + expect(line.ip).toBe('10.20.x.x') + expect(line.query).toBeNull() + }) +})