Skip to content

[Experimenting] Add Sentry AI monitoring for OpenAI backend - #1352

Draft
jboolean wants to merge 1 commit into
masterfrom
dd/feature/sentry-ai-monitor-openai
Draft

[Experimenting] Add Sentry AI monitoring for OpenAI backend#1352
jboolean wants to merge 1 commit into
masterfrom
dd/feature/sentry-ai-monitor-openai

Conversation

@jboolean

Copy link
Copy Markdown
Owner

Summary

Add Sentry AI monitoring instrumentation around backend OpenAI story-title generation so LLM activity is traceable in Sentry.

Changes

  • Updated backend/src/business/ai/AiService.ts to create a child Sentry span around the OpenAI chat completion call.
  • Added gen_ai.request span operation metadata with model information (gen_ai.request.model).
  • Captured token usage attributes from OpenAI responses (gen_ai.usage.input_tokens, gen_ai.usage.output_tokens) when available.
  • Marked the AI span as internal_error on failure and ensured span completion in a finally block.
  • Kept prompt/response content capture disabled (no prompt/output payload attributes recorded).

Testing

  • Ran formatter: prettier --cache --write backend/src/business/ai/AiService.ts (via Format tool).
  • Attempted lint via npm run lint -- src/business/ai/AiService.ts but project script fails due ESLint CLI/config mismatch (--ext invalid with current ESLint).
  • Attempted direct lint via npx eslint src/business/ai/AiService.ts but repository is missing flat config (eslint.config.*) for ESLint v9.
  • Attempted type check via npm run types; failed because tsc is not available in the current environment.

PR by Bits - View session in Datadog

Comment @DataDog to request changes

Co-authored-by: jboolean <406931+jboolean@users.noreply.github.com>
@datadog-official

datadog-official Bot commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

View session in Datadog

Bits Dev status: ✅ Done

CI Auto-fix: Disabled (last PR commit was more than 4 days ago) | Enable

Comment @DataDog to request changes

@datadog-official

Copy link
Copy Markdown
Contributor

I can only run on private repositories.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant