Repository navigation
[SDTEST-4061] [SDTEST-4062] Add Swift test automation exclusions - #329
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@autotest Review |
![datadog-datadog-prod-us1[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
The new ATR exclusion guard removes Swift’s implicit return, causing a missing-return compilation error that prevents SDK source builds.
🤖 Bits Code Review · Commit d55ae74
There was a problem hiding this comment.
The ATR configuration hook no longer compiles because the new exclusion guard requires an explicit return on the retry path. This blocks SDK builds even when no test uses the new annotations.
🤖 Bits Code Review · Commit d55ae74
|
@autotest Review |
There was a problem hiding this comment.
More details
The exclusion paths preserve framework-native behavior and backend assignment tags while overriding Datadog retry and failure-suppression policies. Apple-platform compilation and runtime behavior remain unverified.
🤖 Bits Code Review · Commit c208768
…DTEST-4061) DDTag.retriableType and DDTag.retriableInstanceMethod returned the TIA skippable tags, so Objective-C tests couldn't declare retriable tags. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion tags (SDTEST-4061, SDTEST-4062) - Drop the separate noAutoRetry tag: non-retriable tests are the `no_auto_retry` annotation. Report `test.test_management.no_auto_retry` for them and exclude them from Attempt to Fix before resolving disabled/quarantined properties. - Add type-level `preventFailureSuppression` tag for XCTest (Swift and ObjC) with method-level override, same as `retriable`. - Add `.dd.allowFailureSuppression` Swift Testing tag to override `.dd.preventFailureSuppression` of a parent suite. - Resolve Swift Testing tags in declaration order so the most specific declaration wins. Test-level `.dd.nonretriable` and `.dd.tia.unskippable` now override suite-level positive tags too. - Apply test annotations to Test Management properties in one place. - Fix README examples (internal Selector-based tagger API) and document suite-level declarations and overrides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…61, SDTEST-4062) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@autotest review |
There was a problem hiding this comment.
More details
The retry and failure-suppression exclusions remain independent, and backend assignment tags are preserved. Runtime behavior remains unverified because macOS/Xcode tests could not run in the Linux sandbox.
🤖 Bits Code Review · Commit 2ea6118
What and why?
Allow XCTest and Swift Testing tests to opt out of Datadog automatic retries and failure suppression, per the Shepherd exclusion contract.
retriabletag set tofalse/.dd.nonretriable) are excluded from Auto Test Retries, Early Flake Detection and Attempt to Fix. Quarantine and disabling still apply to them.Both can be declared on a suite (type) and overridden on a single test.
Implements SDTEST-4061 and SDTEST-4062.
How?
RetriableTaginstead of a separate retry opt-out tag. Non-retriable tests reporttest.test_management.no_auto_retry: true.PreventFailureSuppressionTag:TypeTag.preventFailureSuppressionandInstanceMethodTag.preventFailureSuppression(a method tag overrides the type tag), with ObjC bridgesDDTag.preventFailureSuppressionType/DDTag.preventFailureSuppressionInstanceMethod..dd.preventFailureSuppressionand.dd.allowFailureSuppression, so a test or nested suite can re-allow suppression.test.test_management.prevent_failure_suppression: true..dd.nonretriable/.dd.tia.unskippablecould not override a suite-level.dd.retriable/.dd.tia.skippable. That now works too.Test.applying(tags:)): Attempt to Fix is dropped for non-retriable tests before disabled/quarantined are resolved, and protected tests are excluded from disabling and quarantine. Backend-assignedis_quarantined/is_test_disabled/is_attempt_to_fixtags are still reported.DDTag.retriableType/DDTag.retriableInstanceMethod, which returned the TIA skippable tags.RuntimeInfo.Unit tests (
DatadogSDKTestingTests, macOS) pass locally.Review checklist
🤖 Generated with Claude Code