Skip to content

Commit ca9dafb

Browse files
ankrgylclaude
ankrgyl
and
claude
committed
fix(openai): align Decisions spans with Python SDK
Log OpenAI Decisions calls the way the Python SDK and the Braintrust gateway do, so the same call produces the same decision span: - use the decision span type instead of the legacy question type - give unnamed or repeated questions question_{index} IDs, and pair answers with questions by name before position - prefix boolean choice keys with boolean: only when they collide with a string choice, instead of quoting every key - log score criteria as {label, description} objects Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 2a4b1d1 commit ca9dafb

12 files changed

Lines changed: 261 additions & 93 deletions

‎.changeset/fix-openai-decision-spans.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
"braintrust": patch
3+
---
4+
5+
fix(openai): align Decisions spans with Python SDK
6+
7+
OpenAI Decisions spans now use the `decision` span type, `question_{index}` IDs for unnamed or repeated questions, `boolean:`-prefixed keys for boolean choices that collide with string choices, and `{label, description}` score criteria, matching the Python SDK and the Braintrust gateway.

‎e2e/scenarios/openai-instrumentation/__snapshots__/openai-v7-auto-hook.span-tree.json‎

Lines changed: 13 additions & 7 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎e2e/scenarios/openai-instrumentation/__snapshots__/openai-v7-auto-hook.span-tree.txt‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ span_tree:
2020
│ │ "caller_functionname": "runOperation",
2121
│ │ "caller_lineno": 0
2222
│ │ }
23-
│ │ └── openai.decisions.create [question]
23+
│ │ └── openai.decisions.create [decision]
2424
│ │ input: {
2525
│ │ "questions": [
2626
│ │ {
@@ -94,7 +94,7 @@ span_tree:
9494
│ │ "caller_functionname": "runOperation",
9595
│ │ "caller_lineno": 0
9696
│ │ }
97-
│ │ └── openai.decisions.create [question]
97+
│ │ └── openai.decisions.create [decision]
9898
│ │ input: {
9999
│ │ "questions": [
100100
│ │ {
@@ -154,7 +154,7 @@ span_tree:
154154
│ │ "caller_functionname": "runOperation",
155155
│ │ "caller_lineno": 0
156156
│ │ }
157-
│ │ └── openai.decisions.create [question]
157+
│ │ └── openai.decisions.create [decision]
158158
│ │ input: {
159159
│ │ "questions": [
160160
│ │ {
@@ -218,7 +218,7 @@ span_tree:
218218
│ │ "caller_functionname": "runOperation",
219219
│ │ "caller_lineno": 0
220220
│ │ }
221-
│ │ └── openai.decisions.create [question]
221+
│ │ └── openai.decisions.create [decision]
222222
│ │ input: {
223223
│ │ "questions": [
224224
│ │ {
@@ -237,8 +237,14 @@ span_tree:
237237
│ │ },
238238
│ │ {
239239
│ │ "criteria": [
240-
│ │ "routine: Can wait",
241-
│ │ "urgent: Needs attention today"
240+
│ │ {
241+
│ │ "description": "Can wait",
242+
│ │ "label": "routine"
243+
│ │ },
244+
│ │ {
245+
│ │ "description": "Needs attention today",
246+
│ │ "label": "urgent"
247+
│ │ }
242248
│ │ ],
243249
│ │ "id": "<span:3>",
244250
│ │ "instructions": "How urgent is this request?",
@@ -302,7 +308,7 @@ span_tree:
302308
│ │ "caller_functionname": "runOperation",
303309
│ │ "caller_lineno": 0
304310
│ │ }
305-
│ │ └── openai.decisions.create [question]
311+
│ │ └── openai.decisions.create [decision]
306312
│ │ input: {
307313
│ │ "questions": [
308314
│ │ {

‎e2e/scenarios/openai-instrumentation/__snapshots__/openai-v7-latest-auto-hook.span-tree.json‎

Lines changed: 13 additions & 7 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎e2e/scenarios/openai-instrumentation/__snapshots__/openai-v7-latest-auto-hook.span-tree.txt‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ span_tree:
2020
│ │ "caller_functionname": "runOperation",
2121
│ │ "caller_lineno": 0
2222
│ │ }
23-
│ │ └── openai.decisions.create [question]
23+
│ │ └── openai.decisions.create [decision]
2424
│ │ input: {
2525
│ │ "questions": [
2626
│ │ {
@@ -94,7 +94,7 @@ span_tree:
9494
│ │ "caller_functionname": "runOperation",
9595
│ │ "caller_lineno": 0
9696
│ │ }
97-
│ │ └── openai.decisions.create [question]
97+
│ │ └── openai.decisions.create [decision]
9898
│ │ input: {
9999
│ │ "questions": [
100100
│ │ {
@@ -154,7 +154,7 @@ span_tree:
154154
│ │ "caller_functionname": "runOperation",
155155
│ │ "caller_lineno": 0
156156
│ │ }
157-
│ │ └── openai.decisions.create [question]
157+
│ │ └── openai.decisions.create [decision]
158158
│ │ input: {
159159
│ │ "questions": [
160160
│ │ {
@@ -218,7 +218,7 @@ span_tree:
218218
│ │ "caller_functionname": "runOperation",
219219
│ │ "caller_lineno": 0
220220
│ │ }
221-
│ │ └── openai.decisions.create [question]
221+
│ │ └── openai.decisions.create [decision]
222222
│ │ input: {
223223
│ │ "questions": [
224224
│ │ {
@@ -237,8 +237,14 @@ span_tree:
237237
│ │ },
238238
│ │ {
239239
│ │ "criteria": [
240-
│ │ "routine: Can wait",
241-
│ │ "urgent: Needs attention today"
240+
│ │ {
241+
│ │ "description": "Can wait",
242+
│ │ "label": "routine"
243+
│ │ },
244+
│ │ {
245+
│ │ "description": "Needs attention today",
246+
│ │ "label": "urgent"
247+
│ │ }
242248
│ │ ],
243249
│ │ "id": "<span:3>",
244250
│ │ "instructions": "How urgent is this request?",
@@ -302,7 +308,7 @@ span_tree:
302308
│ │ "caller_functionname": "runOperation",
303309
│ │ "caller_lineno": 0
304310
│ │ }
305-
│ │ └── openai.decisions.create [question]
311+
│ │ └── openai.decisions.create [decision]
306312
│ │ input: {
307313
│ │ "questions": [
308314
│ │ {

‎e2e/scenarios/openai-instrumentation/__snapshots__/openai-v7-latest-wrapped.span-tree.json‎

Lines changed: 13 additions & 7 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎e2e/scenarios/openai-instrumentation/__snapshots__/openai-v7-latest-wrapped.span-tree.txt‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ span_tree:
2020
│ │ "caller_functionname": "runOperation",
2121
│ │ "caller_lineno": 0
2222
│ │ }
23-
│ │ └── openai.decisions.create [question]
23+
│ │ └── openai.decisions.create [decision]
2424
│ │ input: {
2525
│ │ "questions": [
2626
│ │ {
@@ -94,7 +94,7 @@ span_tree:
9494
│ │ "caller_functionname": "runOperation",
9595
│ │ "caller_lineno": 0
9696
│ │ }
97-
│ │ └── openai.decisions.create [question]
97+
│ │ └── openai.decisions.create [decision]
9898
│ │ input: {
9999
│ │ "questions": [
100100
│ │ {
@@ -154,7 +154,7 @@ span_tree:
154154
│ │ "caller_functionname": "runOperation",
155155
│ │ "caller_lineno": 0
156156
│ │ }
157-
│ │ └── openai.decisions.create [question]
157+
│ │ └── openai.decisions.create [decision]
158158
│ │ input: {
159159
│ │ "questions": [
160160
│ │ {
@@ -218,7 +218,7 @@ span_tree:
218218
│ │ "caller_functionname": "runOperation",
219219
│ │ "caller_lineno": 0
220220
│ │ }
221-
│ │ └── openai.decisions.create [question]
221+
│ │ └── openai.decisions.create [decision]
222222
│ │ input: {
223223
│ │ "questions": [
224224
│ │ {
@@ -237,8 +237,14 @@ span_tree:
237237
│ │ },
238238
│ │ {
239239
│ │ "criteria": [
240-
│ │ "routine: Can wait",
241-
│ │ "urgent: Needs attention today"
240+
│ │ {
241+
│ │ "description": "Can wait",
242+
│ │ "label": "routine"
243+
│ │ },
244+
│ │ {
245+
│ │ "description": "Needs attention today",
246+
│ │ "label": "urgent"
247+
│ │ }
242248
│ │ ],
243249
│ │ "id": "<span:3>",
244250
│ │ "instructions": "How urgent is this request?",
@@ -302,7 +308,7 @@ span_tree:
302308
│ │ "caller_functionname": "runOperation",
303309
│ │ "caller_lineno": 0
304310
│ │ }
305-
│ │ └── openai.decisions.create [question]
311+
│ │ └── openai.decisions.create [decision]
306312
│ │ input: {
307313
│ │ "questions": [
308314
│ │ {

0 commit comments

Comments
 (0)