Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ryu incorrectly formats this f32 number #235

Open
e00E opened this issue Oct 2, 2024 · 0 comments
Open

ryu incorrectly formats this f32 number #235

e00E opened this issue Oct 2, 2024 · 0 comments

Comments

@e00E
Copy link

e00E commented Oct 2, 2024

An f32 (32 bit floating point value) can represent the number 2147483904 exactly. Ryu formats this number incorrectly. Ryu formats the number as the string 2.147484E9. Ryu should format the number as the string 2147483904.

This is a bug in Ryu because it violates correctness criteria 3 from the paper, "correct rounding". Both strings satisfy criteria 1 and 2 but the string 2147483904 is closer to the original number. It has a distance of 0 to the number. Ryu's output has a distance of 96.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant