Chromium Code Reviews
DescriptionAvoid uint64_t overflow in Decimal::operator/()
This patch changes Decimal::operator/() not to cause uint64_t overflow in
multiplication and addition during calculating quotient. Original code
wrongly assumed each division loop generate at most two digits, by
|MaxCoefficient < 100|, however this assumption is wrong such as
50,000 / 99,9999.
This patch also fixes assertion failure |n> Precision|, where |Precision| == 18, in |scaleUp(x, n)| via |Decimal::ceil()|. Before this patch, we don't have 18 digits quotient with 10^-18 exponent from result of division operator.
BUG=463510
TEST=blink_platform_unittests --gtest_filter=DecimalTest.Division
Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=191294
Patch Set 1 #Patch Set 2 : 2015-03-04T15:05:41 #
Messages
Total messages: 6 (3 generated)
|
||||||||||||||||||||||||||||