Discussion:
Improper rounding of float value by using round function in sybase
(too old to reply)
Subind
2009-06-04 12:32:29 UTC
Permalink
Hi,
I'm facing issue with rounding of any float value. These are the two
ways I tried, both gave different results.
The method 1 doesn't do the rounding. why is that so?
Can anybody please throw some light on this issue?

Method 1
------------------------
declare @b float
select @b=2.135
select round(@b,2)
Result:2.13

Method 2
------------------------
select round(2.135,2)
Result:2.14
------------------------
David Kerber
2009-06-04 13:16:10 UTC
Permalink
Post by Subind
Hi,
I'm facing issue with rounding of any float value. These are the two
ways I tried, both gave different results.
The method 1 doesn't do the rounding. why is that so?
Can anybody please throw some light on this issue?
Method 1
------------------------
Result:2.13
Method 2
------------------------
select round(2.135,2)
Result:2.14
------------------------
Most likely, when 2.135 is stored into variable b, it can't be exactly
represented in binary floating point, so its internal representation is
equal to something like 2.134999999999999, which of course rounds to
2.13. If you declare b as a DECIMAL or equivalent type, it will
probably work.

In the second case, 2.135 is probably representated as a Decimal (or
other exact) data type, and therefore rounds correctly.
--
/~\ The ASCII
\ / Ribbon Campaign
X Against HTML
/ \ Email!

Remove the ns_ from if replying by e-mail (but keep posts in the
newsgroups if possible).
Loading...