feat: openai-java v3.0+ instrumentation #9959

ygree · 2025-11-13T20:25:51Z

What Does This Do

Adds openai-java v3.0+ instrumentation for completions, chat completions, embeddings, and responses.

APM shared tests:

test_openai.py::TestOpenAiApm::test_chat_completion[java-test-ml-app-tcp-True] PASSED [ 12%]
test_openai.py::TestOpenAiApm::test_chat_completion[java-test-ml-app-tcp-False] PASSED [ 25%]
test_openai.py::TestOpenAiApm::test_completion[java-test-ml-app-tcp] PASSED [ 37%]
test_openai.py::TestOpenAiApm::test_embedding[java-test-ml-app-tcp] PASSED [ 50%]
test_openai.py::TestOpenAiApm::test_chat_completion_tool_call[java-test-ml-app-tcp-True] PASSED [ 62%]
test_openai.py::TestOpenAiApm::test_chat_completion_tool_call[java-test-ml-app-tcp-False] PASSED [ 75%]
test_openai.py::TestOpenAiApm::test_responses_create[java-test-ml-app-tcp-True] PASSED [ 87%]
test_openai.py::TestOpenAiApm::test_responses_create[java-test-ml-app-tcp-False] PASSED [100%]

LLMObs shared tests 14*/15:

test_openai.py::TestOpenAiLlmObs::test_chat_completion[java-test-ml-app-tcp-True] PASSED [  6%]
test_openai.py::TestOpenAiLlmObs::test_chat_completion[java-test-ml-app-tcp-False] PASSED [ 13%]
test_openai.py::TestOpenAiLlmObs::test_completion[java-test-ml-app-tcp] PASSED [ 20%]
test_openai.py::TestOpenAiLlmObs::test_embedding[java-test-ml-app-tcp] PASSED [ 26%]
test_openai.py::TestOpenAiLlmObs::test_chat_completion_tool_call[java-test-ml-app-tcp-True] PASSED [ 33%]
test_openai.py::TestOpenAiLlmObs::test_chat_completion_tool_call[java-test-ml-app-tcp-False] PASSED [ 40%]

test_openai.py::TestOpenAiLlmObs::test_chat_completion_telemetry[java-test-ml-app-tcp] FAILED* [ 46%]

test_openai.py::TestOpenAiLlmObs::test_responses_create[java-test-ml-app-tcp-True] PASSED [ 53%]
test_openai.py::TestOpenAiLlmObs::test_responses_create[java-test-ml-app-tcp-False] PASSED [ 60%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_tool_call[java-test-ml-app-tcp-True] PASSED [ 66%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_tool_call[java-test-ml-app-tcp-False] PASSED [ 73%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_reasoning[java-test-ml-app-tcp-True] PASSED [ 80%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_reasoning[java-test-ml-app-tcp-False] PASSED [ 86%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_tool_input[java-test-ml-app-tcp-True] PASSED [ 93%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_tool_input[java-test-ml-app-tcp-False] PASSED [100%]

- test_chat_completion_telemetry FAILED because the shared test expects a namespace for the entire batch, but it should allow the namespace to be part of the metric data. Since Java batches telemetry data across subsystems and cannot guarantee that all the data belongs to the same namespace.

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: AIDM-163

…stead of forcing it

…d "streamed async request completion test with withRawResponse"

Call decorateWithResponse from the wrappers

Rename span resources to be aligned with trace-py Add http.client resource assertion

Test case renaming

Reorder tests by synch, async

Fix Embeddings fixture for the latestDepTest when base64

…wrappers and instrumentation.

…not parsed

…s use ListWriter, and LLMObsSpanMapper is never reached.

github-actions · 2025-12-23T21:40:00Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

PerfectSlayer

First round of comments for the core part. I will let IDM review the instrumentation part.

PerfectSlayer · 2026-01-12T13:08:48Z

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/InstrumenterModule.java

+   *     <blockquote>
+   *     <p><b>NOTE:</b> The order matters. If the muzzle check fails with a NoClassDefFoundError
+   *     (as seen in build/reports/muzzle-*.txt), it may be because some helper classes depend on
+   *     each other. In this case, the order must be adjusted accordingly.
+   *     </blockquote>


❔ question: ‏Which order are you referring here? Helper orders but there is no helper. Or instrumentation order?

The returned helper class names are in order. For example, if you return an array containing "Foo" and "Bar," and "Foo" references "Bar," then it will fail with a NoClassDefFoundError, which might not be obvious since both are declared.

PerfectSlayer · 2026-01-12T13:11:24Z

dd-java-agent/testing/src/main/groovy/datadog/trace/agent/test/asserts/TagsAssert.groovy

      assert value =~ expected: "Tag \"$name\": \"${value.toString()}\" does not match pattern \"$expected\""
    } else if (expected instanceof Class) {
-      assert ((Class) expected).isInstance(value): "Tag \"$name\": instance check $expected failed for \"${value.toString()}\" of class \"${value.class}\""
+      assert ((Class) expected).isInstance(value): "Tag \"$name\": instance check $expected failed for \"${value.toString()}\" of class \"${value?.class}\""


🔨 issue: ‏Is that to prevent null value? If so, how the value.toString() did not crash earlier?

PerfectSlayer · 2026-01-12T13:13:25Z

dd-java-agent/instrumentation/openai-java/openai-java-3.0/build.gradle

@@ -0,0 +1,25 @@
+apply from: "$rootDir/gradle/java.gradle"
+apply plugin: 'idea'


❔ question: ‏What's the need for the idea plugin?

I don't know. Some modules have it. Is it no longer needed?

PerfectSlayer · 2026-01-12T13:16:26Z

dd-trace-core/src/main/java/datadog/trace/llmobs/writer/ddintake/LLMObsSpanMapper.java


+  private CharSequence llmObsSpanName(CoreSpan<?> span) {
+    CharSequence operationName = span.getOperationName();
+    CharSequence resourceName = span.getResourceName();


🔍 nitpick: ‏Resource name can only be fetched in case of OpenAI

Correct. I'll move this

PerfectSlayer · 2026-01-12T13:29:39Z

...ava-agent/testing/src/main/groovy/datadog/trace/agent/test/server/http/TestHttpServer.groovy

        this.contentLength = req.contentLength
        this.contentType = req.contentType?.split(";")
        this.body = req.inputStream.bytes
+        this.method = req.method


❔ question: ‏Is this useful? It's only used by OpenAI instrumentation tests and has only POST from their records.

It's useful for distinguishing request files with different HTTP methods but the same body. However, it can be removed since they are always POST.

ygree added 30 commits October 24, 2025 18:03

Initial APM-only openai-java instrumentation with a unit test.

e7bd8d9

Start llmobs system in tests and create llmobs span.

6feb61e

streamed request completion test

d541adb

README

c209bd0

Instrument sync streamed completion

a82856d

Mock Streamed Completion in Tests

3d3def9

Add failing test for Async Completion

3542b9e

Async Single Completion. Simplify tests. Remove println. Cleanup.

c12cd0a

Extract DDHttpResponseFor to intercept when the response is parsed in…

2af4e5f

…stead of forcing it

Instrument and test Async Stream Completion

cec33af

More tests "streamed request completion test with withRawResponse" an…

b2b4141

…d "streamed async request completion test with withRawResponse"

Fix muzzle check

b1b6db2

Add openai APM tags, assert tags, todo tags

baa687d

Wrap HttpResponseFor instead of forcing parsing. Add TODOs

b9ad963

Set response model tag

5bf16f4

Set "openai.organization.name"

be20fa1

Call decorateWithResponse from the wrappers

Set ratelimit metrics

dd230b7

api_base

c3f94b0

"openai.request.method" & "openai.request.endpoint"

ddf0e09

createChatCompletion instrumentation

fb9fd04

Reorder tests single then stream

a4358a8

Async Single Chat Completion

99dedfe

Async Streamed Chat Completion

d66a5ee

Rename assertChatCompletionTrace

f8af3ae

Instrument Embeddings

846e945

Rename span resources to be aligned with trace-py Add http.client resource assertion

ResponseService WIP

b14bc4b

Test case renaming

ResponseService synch

e8daf08

Reorder tests by synch, async

ResponseServiceAsyncInstrumentation

c8a9e6e

Fix Embeddings fixture for the latestDepTest when base64

Setup httpClient for tests WIP

9754cd8

Intercept Http req/resp with TestOpenAiHttpClient

aeaae2a

Fix unused import

854d7b0

ygree temporarily deployed to ci-build December 18, 2025 03:29 — with GitHub Actions Inactive

ygree added 2 commits December 18, 2025 16:40

Handle possible response parse errors

2d127f2

Clean up HttpResponse wrappers

8718f94

ygree temporarily deployed to ci-build December 19, 2025 03:11 — with GitHub Actions Inactive

ygree added 8 commits December 19, 2025 17:07

Properly close scope, finish span, and handle errors in HttpResponse …

a4d0421

…wrappers and instrumentation.

Refactor LLMObs getting current LLMObs parentSpanID

1ca2ae0

Do not decorate LLMObs specific tags when it's disabled

f8d8ce5

Finish span in HttpResponseWrapper when closing, if the response was …

1154c16

…not parsed

Clean up the unnecessary test subsystem for now because the unit test…

6e5cf86

…s use ListWriter, and LLMObsSpanMapper is never reached.

remove leftover

b19cbb9

Implement LLMObs "span.finished" metric

b341414

Merge branch 'master' into ygree/openai-java

9dedc4e

ygree temporarily deployed to ci-build December 23, 2025 20:26 — with GitHub Actions Inactive

update lockfile

ff22fa8

ygree temporarily deployed to ci-build December 23, 2025 20:30 — with GitHub Actions Inactive

ygree changed the title ~~[WIP] openai-java v3.0+ instrumentation~~ add openai-java v3.0+ instrumentation Dec 23, 2025

ygree mentioned this pull request Dec 23, 2025

Add support for instrumenting the OpenAI Client #9284

Closed

ygree changed the title ~~add openai-java v3.0+ instrumentation~~ feat: openai-java v3.0+ instrumentation Dec 23, 2025

ygree removed the tag: do not merge Do not merge changes label Dec 23, 2025

ygree marked this pull request as ready for review December 23, 2025 21:39

ygree requested review from a team as code owners December 23, 2025 21:39

ygree requested review from dougqh and removed request for a team December 23, 2025 21:39

ygree added the type: feature request label Dec 23, 2025

PerfectSlayer reviewed Jan 12, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: openai-java v3.0+ instrumentation #9959

feat: openai-java v3.0+ instrumentation #9959

Uh oh!

ygree commented Nov 13, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Dec 23, 2025

Uh oh!

PerfectSlayer left a comment

Uh oh!

PerfectSlayer Jan 12, 2026

Uh oh!

ygree Jan 13, 2026

Uh oh!

PerfectSlayer Jan 12, 2026

Uh oh!

PerfectSlayer Jan 12, 2026

Uh oh!

ygree Jan 13, 2026

Uh oh!

PerfectSlayer Jan 12, 2026

Uh oh!

ygree Jan 13, 2026

Uh oh!

PerfectSlayer Jan 12, 2026

Uh oh!

ygree Jan 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -0,0 +1,25 @@
		apply from: "$rootDir/gradle/java.gradle"
		apply plugin: 'idea'

feat: openai-java v3.0+ instrumentation #9959

Are you sure you want to change the base?

feat: openai-java v3.0+ instrumentation #9959

Uh oh!

Conversation

ygree commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Dec 23, 2025

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ygree commented Nov 13, 2025 •

edited

Loading