Skip to content

[php]: add endpoints to support OpenTelemetry logs#6646

Open
rachelyangdog wants to merge 11 commits into
mainfrom
rachel.yang/php-otel-logs
Open

[php]: add endpoints to support OpenTelemetry logs#6646
rachelyangdog wants to merge 11 commits into
mainfrom
rachel.yang/php-otel-logs

Conversation

@rachelyangdog
Copy link
Copy Markdown
Contributor

Motivation

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@rachelyangdog rachelyangdog requested review from a team as code owners March 30, 2026 14:19
@rachelyangdog rachelyangdog marked this pull request as draft March 30, 2026 14:19
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 30, 2026

CODEOWNERS have been resolved as:

manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
run.sh                                                                  @DataDog/system-tests-core
utils/build/docker/php/parametric/composer.json                         @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/php/parametric/composer.lock                         @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/php/parametric/server.php                            @DataDog/apm-php @DataDog/system-tests-core

rachelyangdog and others added 2 commits March 30, 2026 10:31
…ridge

- server.php: replace manual LogsExporterFactory + BatchLogRecordProcessor +
  SDKLoggerProvider wiring with Globals::loggerProvider(), aligning with the
  Node.js and Python parametric servers. Drop the hardcoded putenv that
  forced OTEL_EXPORTER_OTLP_ENDPOINT to port 4318 and prevented the tracer's
  DatadogResolver from picking the correct port per protocol.
- /log/otel/flush: wrap forceFlush in try/catch and return get_class on the
  provider, matching the Python/Node.js shape.
- Dockerfile: set DD_AUTOLOAD_NO_COMPILE=true so ddtrace loads the per-file
  bridge (src/bridge/_files_*.php) rather than the stale bundled
  src/bridge/_generated_*.php from the released package, allowing local PR
  source overrides in /binaries/src to take effect.
- Remove GrpcTransport.php: orphaned shim, never registered with the OTel
  Registry and required PECL ext-grpc which isn't installed in the image.
  gRPC support belongs as an opt-in dep on the user side (parallels
  dd-trace-py's opentelemetry-exporter-otlp[grpc]).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@datadog-prod-us1-5
Copy link
Copy Markdown

datadog-prod-us1-5 Bot commented May 7, 2026

Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 23 Tests failed

tests.parametric.test_otel_logs.Test_FR01_Enable_OTLP_Log_Collection.test_otlp_logs_enabled[library_env0, parametric-php] from system_tests_suite   View in Datadog   (Fix with Cursor)
ValueError: Number 1 of logs not available from test agent, got 0

self = <tests.parametric.test_otel_logs.Test_FR01_Enable_OTLP_Log_Collection object at 0x7fd5ce6a6270>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fd599c3e5d0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fd59a360530>

    @pytest.mark.parametrize(
        "library_env",
        [
        ],
...
tests.parametric.test_otel_logs.Test_FR03_Resource_Attributes.test_dd_env_vars_override_otel[library_env0, parametric-php] from system_tests_suite   View in Datadog   (Fix with Cursor)
ValueError: Number 1 of logs not available from test agent, got 0

self = <tests.parametric.test_otel_logs.Test_FR03_Resource_Attributes object at 0x7fd5ce6a5a30>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fd59a363c20>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fd599c3e810>

    @pytest.mark.parametrize(
        "library_env",
        [
            {
...
tests.parametric.test_otel_logs.Test_FR03_Resource_Attributes.test_otel_resource_attributes[library_env0, parametric-php] from system_tests_suite   View in Datadog   (Fix with Cursor)
ValueError: Number 1 of logs not available from test agent, got 0

self = <tests.parametric.test_otel_logs.Test_FR03_Resource_Attributes object at 0x7fd5ce6a6c00>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fd59a1b3ef0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fd59a1b0620>

    @pytest.mark.parametrize(
        "library_env",
        [
            {
...
View all

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0a73b44 | Docs | Datadog PR Page | Give us feedback!

rachelyangdog and others added 4 commits May 7, 2026 15:52
Mirrors the metrics-style pattern: dd-trace-php's DatadogResolver supplies
agent-aware defaults (OTEL_EXPORTER_OTLP_LOGS_ENDPOINT, etc.) and the
parametric server, standing in for user code, builds the LoggerProvider
itself with LogsExporterFactory + BatchLogRecordProcessor +
ResourceInfoFactory::defaultResource. Avoids relying on an
OTEL_PHP_AUTOLOAD_ENABLED override in the tracer, which would also stand up
unrequested Tracer/Meter providers and OTel auto-instrumentations.

Wraps construction in try/catch so the server keeps responding when the
configured protocol's transport isn't available (e.g. grpc without
ext-grpc + open-telemetry/transport-grpc), letting that test fail cleanly
on missing payloads instead of erroring at server startup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Test_FR06_OTLP_Protocols: gRPC requires user-installed
  open-telemetry/transport-grpc + ext-grpc; dd-trace-php does not bundle
  either (parallels dd-trace-py's opt-in posture for grpc). HTTP/protobuf
  is exercised via FR01/FR05/FR08, so no real coverage loss.
- Test_FR10_Timeout_Configuration / Test_FR11_Telemetry: OTel config
  telemetry tracking (otel.log_records count metric, OTEL_EXPORTER_OTLP_*
  configuration reporting) is out of scope for the initial logs PR; will
  follow up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Manifest validator requires sorted keys. Test_FR06 now precedes Test_FR07.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rachelyangdog rachelyangdog changed the title WIP: php endpoints to support opentelemtry logs [php]: add endpoints to support OpenTelemetry logs May 11, 2026
Comment thread utils/build/docker/php/parametric/Dockerfile
// DatadogResolver fills in OTEL_EXPORTER_OTLP_LOGS_ENDPOINT from the agent host
// and the resource detector hooks (Service / Environment / Host) populate the
// ResourceInfo with DD_SERVICE / DD_ENV / DD_VERSION / DD_HOSTNAME.
$sdkLoggerProvider = SDKLoggerProvider::builder()->build();
Copy link
Copy Markdown
Contributor

@bwoebi bwoebi May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would actually add open-telemetry/opentelemetry-logger-monolog as a composer dependency (and use monolog then) - so that the test actually tests what it's intended to test: that the log injection doesn't mess with this.

Comment on lines +565 to +572
$severityMap = [
'TRACE' => ['number' => 1, 'text' => 'TRACE'],
'DEBUG' => ['number' => 5, 'text' => 'DEBUG'],
'INFO' => ['number' => 9, 'text' => 'INFO'],
'WARN' => ['number' => 13, 'text' => 'WARN'],
'ERROR' => ['number' => 17, 'text' => 'ERROR'],
'FATAL' => ['number' => 21, 'text' => 'FATAL'],
];
Copy link
Copy Markdown
Contributor

@bwoebi bwoebi May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why map to itself, is ['TRACE' => 1, ...] not good enough?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That map was in the pre-Monolog version of /otel/logger/write and got removed in the commit e3ab113. The current severity map lives inside the custom handler at server.php:580 and uses Monolog level_name keys, where the text differs from the key on 5 of 8 rows (NOTICE→INFO, WARNING→WARN, CRITICAL→FATAL, ALERT→FATAL, EMERGENCY→FATAL). The 3 self-mapping rows (DEBUG/INFO/ERROR) are kept inline for symmetry.

Comment thread run.sh
Comment on lines +434 to +436
if [ "${library}" = "php" ] && [ "${pytest_numprocesses}" = "auto" ]; then
pytest_numprocesses=4
fi
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added because PHP parametric workers each launch a dd-apm-test-agent + php-test-client container pair so i capped the concurrency to keep the docker daemon responsive.

@rachelyangdog rachelyangdog marked this pull request as ready for review May 11, 2026 19:27
rachelyangdog and others added 4 commits May 11, 2026 15:27
Per Bob's review feedback: the FR09 log-injection test was passing
trivially before because the parametric server emitted OTel LogRecords
directly via the SDK Logger — so dd-trace-php's LogsIntegration PSR-3
hook (the real log-injection path) never fired, and the
shouldSkipForOtelLogs guard was never actually validated.

This change adds open-telemetry/opentelemetry-logger-monolog to the
parametric server's composer and routes /otel/logger/{create,write}
through a Monolog\Logger with an OTel Handler attached, so the PSR-3
hook in LogsIntegration runs and the integration's OTel-routed-Monolog
detection is genuinely exercised.

Two small wrinkles in the OTel Monolog handler that needed handling:

- The handler calls $provider->getLogger($monologChannel) with no scope
  params, so FR13 (version / schema_url / attributes) would lose what
  create_logger sent. Wrap the SDK provider in an anonymous class that
  stores the create_logger scope params and re-injects them when the
  handler asks for a Logger.

- The handler writes $record->level_name through verbatim as
  severity_text, so Monolog's "WARNING"/"CRITICAL" surface instead of
  OTel-canonical "WARN"/"FATAL" (FR12 [warning] failure).
  Monolog\LogRecord::with() doesn't allow overriding level_name (it's
  derived from the Level enum, not a stored property), so subclass the
  handler and re-implement write() with our own severity map. The
  subclass keeps `instanceof OpenTelemetry\Contrib\Logs\Monolog\Handler`
  true so LogsIntegration's detection still matches.

Full parametric suite still: 24 passed, 5 xfailed (out of scope: grpc +
telemetry), 1 xpassed (FR06 http_protobuf — collateral of the
class-level missing_feature gate), 1 skipped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes two changes that only helped local PR-iteration and shouldn't
ship in the parametric image:

- ENV DD_AUTOLOAD_NO_COMPILE=true (made ddtrace load src/bridge/_files_*.php
  instead of the bundled _generated_*.php, so /binaries/src overrides
  could take effect without regenerating the bundle).
- The cp /binaries/src/. -> dd-trace-sources/src/ hunk (the override
  mechanism itself).

Local developers iterating on dd-trace-php PHP-level sources should now
either regenerate _generated_*.php locally (tooling/generation, composer
generate) and rsync that with the rest of src/, or ship a full tarball.
The parametric image is back to production-faithful: it runs the released
package's bundle, not a developer's loose source files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread run.sh
Comment on lines +434 to +436
if [ "${library}" = "php" ] && [ "${pytest_numprocesses}" = "auto" ]; then
pytest_numprocesses=4
fi
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised to see this, what is the link with the PR ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants