feat(jira): add JIRA_SKIP_UNPARSEABLE_ISSUES to continue collectIssues on parse failure (#8949) - #9048
Closed
kotwal-itpro wants to merge 1 commit into
Closed
Conversation
…s on parse failure (apache#8949) On large Jira instances, jira:collectIssues occasionally receives an otherwise-successful (2xx) response whose body cannot be decoded as JSON — typically a truncated body from a dropped connection or, in rare misconfigured-proxy cases, an HTML body served with a 200. The default ResponseParser returns an error, which aborts the entire subtask and kills the whole Jira collection even though thousands of other pages succeeded. This is distinct from HTTP error statuses, which are already handled by the collector's retry logic; the request is specifically about parse failures on successful responses. Introduce a JIRA_SKIP_UNPARSEABLE_ISSUES env var: - default (unset or anything other than "true"): existing behaviour — surface the parse error and fail the subtask. - "true" (case-insensitive): log a warning and skip the unparseable page so the rest of the collection completes. The warning includes body length and a 256-byte prefix so operators can still investigate. Share the parser between the V2 (Server) and V3 (Cloud) collectors via a new parseIssuesResponse(logger) helper. Tests cover the happy path, the default error path, the skip-flag success path, case-insensitivity, and an explicit "false" value.
Author
|
Superseded by #9026 (which uses |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #8949.
Problem
On large Jira instances,
jira:collectIssuesoccasionally receives an otherwise-successful (2xx) response whose body cannot be decoded as JSON — typically a truncated body from a dropped connection, or (in rare misconfigured-proxy cases) an HTML body served with a 200. The currentResponseParserreturns an error, which aborts the entire subtask and kills the whole Jira collection even though thousands of other pages succeeded.This is distinct from HTTP error statuses (
502/503/504), which are already handled by the collector's retry logic — the request is specifically about parse failures on successful responses.Fix
Introduce a
JIRA_SKIP_UNPARSEABLE_ISSUESenv var:"true"): existing behaviour — surface the parse error and fail the subtask."true"(case-insensitive): log a warning and skip the unparseable page so the rest of the collection completes. The warning includes body length and a 256-byte body prefix so operators can still investigate without hiding the problem.The V2 (Server) and V3 (Cloud) collectors previously duplicated the response-parsing closure. Factored the shared logic into a
parseIssuesResponse(logger)helper so the new behaviour applies to both.Test
Adds
Test_parseIssuesResponseinissue_collector_test.gocovering:JIRA_SKIP_UNPARSEABLE_ISSUES=truereturns an empty non-nil slice and logs exactly one warning.TRUE)."false"behaves like default.Uses
t.Setenvfor isolation. Fullplugins/jira/taskssuite still passes.