桁落ち
け
た
お
ち
Meanings
1
cancellation of significant digits
Technical term in numerical analysis: loss of precision when subtracting two nearly equal numbers.
In floating-point arithmetic, one must be careful about cancellation of significant digits.
cancellation of significant digits
Technical term in numerical analysis: loss of precision when subtracting two nearly equal numbers.
In floating-point arithmetic, one must be careful about cancellation of significant digits.