Opened 7 years ago
Closed 7 years ago
#1708 closed defect (fixed)
Rasql_Round float number should have a consistent length for fractional part
Reported by: | Bang Pham Huu | Owned by: | bbell |
---|---|---|---|
Priority: | major | Milestone: | 9.6 |
Component: | rasql | Version: | development |
Keywords: | Cc: | Dimitar Misev | |
Complexity: | Medium |
Description
This query:
select stddev_samp(<[0:5] 28,29,30,31,32,33>) Result element 1: 1.87083 (5 digits after .) number before round: 1.87082869339
and this query:
rasql -q 'select arctan(-10294012870517d)' --out string Result element 1: -1.5708 (4 digits after .) number before round: -1.57079632679
it should have either 4 or 5 digits length for fractional part, not both.
Change History (4)
comment:1 by , 7 years ago
comment:2 by , 7 years ago
First we should find out why it even bothers to round in the first place. Last I checked, we removed the base precision enforcement in csv/json output; perhaps it is still present in our string output?
comment:3 by , 7 years ago
This is not in csv, it's in rasql.cc, which prints scalar results.
See browser:applications/rasql/rasql.cc#L541, for float/double it just does cout << scalar
. We should make it consistent with the CSV/JSON output?
comment:4 by , 7 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
Note:
See TracTickets
for help on using tickets.
We could add an option to rasql to control the output formatting.