-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Shell arithmetic is broken for integer values beyond the system's maximum float precision #771
Comments
Just some links to remember when debugging this later: |
To help clarify the extent of impact of ksh's use of double (64bit) floats for integer operations, here are some observations: From Double-precision floating-point format, the following is stated:
Let's test that on various hardware. Intel x86_64 having 80bit long doubles having extended presicion:
Aarch64 with 128bit long doubles ARMv8:
ARMv7l with 64bit doubles:
Confirmed. Integer operations are safe up to the 53bits (253) on machines that do not possess a greater precision for long double than 64bits as pointed out being Apple M1, M2, and M3 and ARMv7 chips. Not sure if Apple M4 still only has 64bit max precision as M4 maybe ARMv9 compatiable chipset. Almost all ARMv8 chips have supported 128bit floats for many years. For Intel, since forever. I hope Apple closes this gap and includes a floating point processor that supports a better precision than Intel such as 128bit or better. Since Intel's significand is 63bits in size, ksh should be able to support at least signed 64bit integers with a high probability of unsigned 64 bit integers as sign bit is available in addition to the significand. On the above output for Intel using ksh, the following is displayed, Please also note the difference in output when output goes beyond 64 bits, 32bit system was "-1, 0, 1" but for 64bit systems was "0, 1, 2". |
Recently discovered this independently - been a long time since I found such a... um... fun(?)... bug! Not sure it adds much, but for the record (on x86_64): $ i=9223372036854775807
$ echo $i
9223372036854775807
$ j=$((i+1))
$ echo $j
9.22337203685477581e+18
$ printf '%d\n' $j
ksh93: printf: warning: 9.22337203685477581e+18: overflow exception
9223372036854775807
$ echo $((j-1))
9.22337203685477581e+18
$ echo $((j-i))
3
$ echo $((j-3))
9223372036854775807
$ while test j -gt i; do j=$((j - 1)); done # <- Infinite loop FWIW also verified this affects 93u+ 2012-08-01 (the earliest version I had available) - which is as you likely expected anyway. |
Is always true. It compares two strings. |
Not on ksh93; Since that behaviour is not POSIX compliant, I've disabled that in the POSIX mode on ksh 93u+m (full rationale in c898e3e):
Besides (as you can see from the error message above), |
The C standard says (1, 2):
Thing is, ksh internally typecasts all values for shell arithmetic to
Sfdouble_t
, a.k.a._ast_fltmax_t
as derived by thefeatures/float
test, i.e., the system's maximum-size float type.So, all arithmetic is internally done with
long double
ordouble
values (depending on the system). See streval.c and arith.c. This breaks long integers, as their values may be too large to be represented exactly by theSfdouble_t
type, in which case we hit the implementation-defined behaviour case and the number will be somehow approximated. This is disastrous, because integer calculations that remain within type range must always be exact.The problem is particularly terrible on ARM systems, whose hardware does not support
long double
, soSfdouble_t
isdouble
. E.g., on my Mac with an M1 processor:Whereas, on x86_64, we get inexact representations when we go beyond
LLONG_MAX
:What this shows is that the whole arithmetic subsystem is broken by design. Internally converting integers to floats and back is bogus, because even the largest float type cannot store all the possible integer values. Integers must be stored and calculated as integers, and nothing else.
We need to find a way, somehow, of making that happen, while still supporting floating point arithmetic as well. But the streval.c and arith.c code is so inscrutable, I'm not very much closer to understanding it now than I was when I forked ksh four years ago.
The text was updated successfully, but these errors were encountered: