Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I tried 0.1 + 0.2 in both google docs and excel. I got 0.3000000000000000000, not 0.30000000000000004, like you normally would using floating point.


I think there's internal rounding that hides it. Try A1=1e50, B1=A1+1, and C1=B1-A1.

Excel definitely used floating point in the past (because you used to be able to use it to test for the Pentium FDIV bug) and according to the documentation, it still does: https://support.microsoft.com/en-us/kb/78113


Rounding happens when the value is displayed. To see the raw value, try: =(0.1+0.2)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: