Do not round exact values up under :up rounding#239
Open
b-erdem wants to merge 1 commit into
Open
Conversation
Round-up (`:up`, away from zero) incremented the coefficient whenever the
discarded part was non-empty, even when every discarded digit was zero.
An exact value with trailing zeros beyond the precision was therefore
rounded up: at precision 1, `Decimal.add(0, Decimal.new("9.0"))` returned
`10` and `Decimal.round(Decimal.new("9.0"), 0, :up)` returned `10`, where
the value is exactly 9.
Per the General Decimal Arithmetic spec, round-up leaves the value
unchanged when all discarded digits are zero. `:ceiling` and `:floor`
already check `any_nonzero?/2`; `:up` now does too, so for a positive
value `:up` and `:ceiling` agree (as they must). Python's decimal returns
9 for these cases.
The clause is shared by every context operation and by round/3.
Verified against Python's decimal: 0 value mismatches across ~3,800
:up cases spanning add/sub/mult/div and round/3.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows up on the
:upbug I noted in #236 — independent of #237 and #238.Problem
:up(round away from zero) increments the coefficient whenever the discarded part is non-empty, even when all discarded digits are zero. An exact value with trailing zeros beyond the precision is rounded up:Per the General Decimal Arithmetic spec, round-up leaves the value unchanged when every discarded digit is zero. Python's
decimalreturns9for both.:ceilingand:flooralready guard onany_nonzero?/2— so today, for a positive value,:upand:ceilingdisagree even though both round away from zero (Decimal.add(0, ~d"9.0")at precision 1 gives10under:upbut9under:ceiling). They must agree.Fix
increment?(:up, …)now checksany_nonzero?(remain, sticky?)instead of returningtrueunconditionally — the same check:ceiling/:flooruse. One clause.Verification
Differential fuzz against Python's
decimal(the GDA reference), comparing numeric values: 0 mismatches across ~3,800:upcases spanningadd/sub/mult/divandround/3, at precisions 1–5. Genuine round-ups still round up (add(0, 3.1)at precision 1 →4;round(9.01, 0, :up)→10).Tests
Extends the existing
with_context/2: uptest with the exact-value cases (and a nonzero-discarded case that still rounds up). Full suite passes (101 doctests, 27 properties, 101 tests).Notes
## Unreleasedsections added by Carry the division remainder into rounding as a sticky bit (#236) #237 and Re-round coefficient when a rounding carry exceeds precision (#236) #238 — happy to add one (or fold all three into a single entry) once you've decided merge order.