Skip to content

Fix precision/rounding corruption when cos/sin/tan throw - #265

Open
afonsojanu wants to merge 1 commit into
MikeMcl:masterfrom
afonsojanu:fix-trig-precision-exception-safety
Open

Fix precision/rounding corruption when cos/sin/tan throw#265
afonsojanu wants to merge 1 commit into
MikeMcl:masterfrom
afonsojanu:fix-trig-precision-exception-safety

Conversation

@afonsojanu

Copy link
Copy Markdown

cos, sin and tan all follow the same pattern: save Ctor.precision/Ctor.rounding, raise Ctor.precision (and pin rounding to 1) for the internal computation, do the computation, then put the two settings back before returning.

The problem is that "put the two settings back" only runs if the computation in between doesn't throw. It can throw: toLessThanHalfPi, which both cos and sin call to reduce the argument, does a lookup of the internal Pi constant at whatever the currently-elevated precision is, and that lookup throws Precision limit exceeded once the requested precision goes past the 1025 digits Pi is hard-coded to. Once that happens, the constructor is left at the raised precision (and rounding mode 1) permanently, well after the cos()/sin()/tan() call has returned control to the caller via a caught exception.

const Decimal = require('decimal.js')

Decimal.precision = 1030
Decimal.rounding = 4

try {
  new Decimal(1).cos()
} catch (e) {
  console.log(e.message) // [DecimalError] Precision limit exceeded
}

console.log(Decimal.precision) // 1038, not 1030
console.log(Decimal.rounding)  // 1, not 4

tan has the same problem twice over, since it delegates to sin internally and also does its own precision/rounding save-restore around that call.

getLn10 already has to deal with this in the ln/log code path and resets Ctor.precision right before it throws for the same reason, so there's precedent for treating this as worth fixing rather than living with it. I used a plain try/finally instead of following getLn10's exact approach, mostly because it also puts Ctor.rounding back, which getLn10's fix does not need to since only precision gets elevated in that code path.

Applied the same change to decimal.mjs, which has its own copy of these three functions.

Testing

Added one case to each of test/modules/cos.js, sin.js and tan.js that pushes the precision past the point where the Pi lookup throws and checks Decimal.precision/Decimal.rounding afterwards. Confirmed each one fails on master (precision comes back changed) and passes with this branch, then ran the full suite (npm test) with everything green.

This addresses the cos/sin/tan part of #97. A few other methods share the underlying "save/raise/restore" shape without a try/finally (atan2 in particular, in its x.s < 0 branch), which I left alone here since #97 is really asking for a broader look at the pattern across the library rather than a single fix; happy to take a look at those too if that would help.

cos, sin and tan all raise Ctor.precision (and set Ctor.rounding to 1)
before doing their trig computation, then lower it back to the
original value once that computation returns. If anything in between
throws, for example toLessThanHalfPi's internal pi lookup once the
raised precision exceeds the hard-coded 1025-digit limit on the PI
constant, the lowering step never runs and the constructor is left
with the raised precision and rounding mode permanently, even though
the caller only asked for a single trig call.

getLn10 already resets Ctor.precision before throwing for the same
reason in the log/ln code path, so this brings cos/sin/tan in line
with that existing convention using a plain try/finally, which also
covers the rounding mode that getLn10's approach does not.

Added a regression case to each of the cos, sin and tan test modules
that raises the precision high enough to trip the pi lookup's limit,
and checks that Decimal.precision and Decimal.rounding come back
unchanged afterwards. Confirmed they fail without the fix (precision
comes back changed) and pass with it, and ran the full suite locally
with no other regressions.

Fixes MikeMcl#97 for these three methods; the same save/restore pattern
shows up in a few other trig-adjacent methods (atan2 in particular)
that this doesn't touch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant