fix(autoreport): treat non-finite decimal values as not_provided DEV-1797#340
Merged
noliveleger merged 1 commit intomainfrom Mar 4, 2026
Merged
Conversation
rgraber
requested changes
Mar 4, 2026
Contributor
rgraber
left a comment
rgraber
left a comment
There was a problem hiding this comment.
Request for 1 additional test
| for i, stat in enumerate(stats): | ||
| assert stat == expected[i] | ||
|
|
||
| def test_stats_with_non_finite_float_value(self): |
Contributor
There was a problem hiding this comment.
can we add a test for when every response is infinite?
rgraber
approved these changes
Mar 4, 2026
Contributor
rgraber
left a comment
rgraber
left a comment
There was a problem hiding this comment.
Approving for time's sake. Hopefully we can add tests when we're not in a rush for release
noliveleger
added a commit
to kobotoolbox/kpi
that referenced
this pull request
Mar 4, 2026
…onses DEV-1797 (#6779) ### 📣 Summary Fixed an issue where the reports page would fail to load when a decimal field contained a non-finite value (e.g. `Infinity` or `NaN`) in a submission. Also fixes the statistics computation so that submissions containing non-finite decimal values are treated as **not provided**, rather than corrupting counts and aggregates. ### 📖 Description When a submission contains `Infinity` or `NaN` for a decimal field, two problems occur: 1. **Invalid JSON response (this PR):** The backend statistics computation (mean, median, stdev, mode) can produce non-finite float values. These are not valid JSON, causing `JSON.parse` in the browser to fail and the reports page to display an error. Fixed by introducing a `SanitizedJSONRenderer` that recursively replaces any non-finite float (`NaN`, `Infinity`, `-Infinity`) with `null` before serializing the HTTP response. 2. **Incorrect statistics ([formpack#340](kobotoolbox/formpack#340 `NumField.parse_values()` was silently passing non-finite floats through, causing them to be counted as valid provided responses and corrupting `provided`, `total_count`, and aggregate statistics (mean, median, etc.). Fixed upstream by raising a `ValueError` for non-finite floats — the existing `_calculate_stats` error handler already treats these as `not_provided`. This PR bumps the formpack dependency to include that fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was done
NumField.parse_valuesnow raises aValueErrorwhen adecimalfield yields a non-finite float (Inf,-Inf,nan)ValueErrorhandling in_calculate_statsalready treats these as blank responses, incrementingnot_providedwithout affectingprovidedortotal_countWhy
Non-finite float values (
Inf,-Inf,nan) can appear in submissions (e.g. from buggy clients or division-by-zero in calculated fields). Previously these were passed through silently, corrupting statistics (mean,median, etc.) and inflatingprovided/total_count.Testing
test_stats_with_non_finite_float_valueintests/test_autoreport.pycoveringInf,-Inf, andnanvalues for adecimalfieldnot_providedincrements correctly and thatprovided/total_countremain accurate