Opened 3 years ago

Closed 3 years ago

#748 closed defect (fixed)

Wrong ranges for float/double min/max in ps_intervals

Reported by: mase Owned by: pcampalani
Priority: minor Milestone: 9.0.x
Component: petascope Version: 9.0.0
Keywords: float min max Cc: jpass
Complexity: Easy

Description

I noticed that the minimum and maximum values for float and double in the ps_interval table were:

float: min = max = 162142775926304050882.5588505358010521
double: min = max = 2833951766629323244804321902155471172912937794865349090046421346168433254453061.0659271326051107

Obviously we shouldn't have min = max but also the usual correct values for IEEE 754 float and double are +/- 3.402...E38 for float and +/-1.797...E308 for double.

I found /usr/share/rasdaman/petascope/update8/global_const.sql has the lines:

    SELECT cset('FLOAT_MIN',              -3.4028234^38::numeric);
    SELECT cset('FLOAT_MAX',               3.4028234^38::numeric);
    SELECT cset('DOUBLE_MIN',   -1.7976931348623157^308::numeric);
    SELECT cset('DOUBLE_MAX',    1.7976931348623157^308::numeric);

It seems to be the use of ^ rather than E for the exponent that causes the problem. If I use -3.4028234E38 instead the values come out correctly. I would just change the above lines to use E rather than ^ but I did try setting the values myself manually in my database and, although they appear correct when querying the table ps_interval, when I do a DescribeCoverage on a coverage which is assigned that interval I now get a range of 0 0 appearing in the output so there seems to be some other problem setting the values correctly.

I'm using postgresql v8.4.

Change History (9)

comment:1 Changed 3 years ago by pcampalani

Hi Marcus,

the error there is that the minus operator has priority wrt the exponentiation, so braces are missing, like:

SELECT cset('FLOAT_MIN',             -(3.4028234^38)::numeric);
SELECT cset('FLOAT_MAX',               3.4028234^38::numeric);
SELECT cset('DOUBLE_MIN',  -(1.7976931348623157^308)::numeric);
SELECT cset('DOUBLE_MAX',    1.7976931348623157^308::numeric);

While we prepare an updateX.sql file to fix the issue, you can manually update the table, I suggest.

petascopedb=# update ps_interval set min=-(3.4028234^38) where min=3.4028234^38;
petascopedb=# update ps_interval set min=-(1.7976931348623157^308) where min=1.7976931348623157^308;

With this fixes, I am getting proper values in the XML responses.
let me know!

Unluckily our test covs are either chars or short typed: we should add variety there too (I created #754).

comment:2 Changed 3 years ago by mase

This doesn't work for me and isn't consistent with the errors I had. Both maximum and minimum values were incorrect as noted above.

If I do

update ps_interval set min=-(3.4028234^38) where id=9;

(id 9 is the relevant row for floats in our table) and then do

select min from ps_interval where id=9;

I get the result

 -162142775926304050882.5588505358010521

and this is output in the range intervals of relevant coverages. As described above using E notation gets the correct values put into the petascope tables but then the DescribeCoverage? output sets them to 0.

I haven't checked the postgres documentation but I wonder whether there is some change from v8.4 (which we are using) and later versions?

comment:3 Changed 3 years ago by pcampalani

hi Marcus,
sorry I don't understand a couple of things:

  • why you say both max and min values are wrong: isn't 3.4028234^38 ok as max float value, for instance?
  • why getting -162142775926304050882.5588505358010521 in you coverage description is not ok?

comment:4 follow-up: Changed 3 years ago by mase

Sorry I wasn't paying close enough attention.

The point is that 3.4028234^38 is 3.438 but the maximum value of a float data type is 3.4028234E38 or 3.4028234x1038. Similarly for min and the max and min values of double. I suppose realistically our data lies in a much smaller range (except that in some cases nil values are indicated by setting the value to -3.4028234x1038) so I could put a smaller range in but the implication of the default values in the Petascope tables is that they correspond to the range of float and double data types so I think these should correspond (near enough) to the actual minimum and maximum values of the respective data types.

comment:5 in reply to: ↑ 4 Changed 3 years ago by pcampalani

Ah yes I see, Gosh.

Regarding the zeros you get in the description, there is a bug in the stripDecimalZeros utility for BigDecimals. I'm patching it.

comment:6 Changed 3 years ago by pcampalani

  • Status changed from new to accepted

comment:7 Changed 3 years ago by pcampalani

Fixes in changeset:3f664c2 and changeset:3ae1c74 (run update_petascopedb.sh to make the latter effective).

Marcus, feel free to close the ticket when you can evaluate the fix.

comment:8 Changed 3 years ago by mase

It is unlikely that I will get to test this until the fix appears in a minor RPM release so, as it is a fairly straightforward issue and it isn't getting in the way of any work, I'm happy for you to close it if you think you've fixed it. I'll re-open in the unlikely event that it turns out not to be.

comment:9 Changed 3 years ago by pcampalani

  • Complexity changed from Medium to Easy
  • Keywords float min max added
  • Milestone set to 9.0.x
  • Resolution set to fixed
  • Status changed from accepted to closed

yes I guessed.. so yes, let's close it.
thanks again for the catch: now we added a 4D floating-point based systemtest coverage, changeset:1b57f46).

Note: See TracTickets for help on using tickets.