Opened 9 years ago

Closed 8 years ago

#1231 closed defect (invalid)

Rasql_Wrong negative value in calculation

Reported by: Bang Pham Huu Owned by:
Priority: major Milestone: 10.0
Component: undecided Version: development
Keywords: WCPS, wrong value, Rasql, negative number Cc: Dimitar Misev
Complexity: Medium

Description

WCPS query

for c in (rgb) return encode( (unsigned char)(c.red - c.blue)[i(0:0), j(0:0)] , "csv")

with c.red[0:0,0:0] = 119 and c.blue[0:0,0:0] = 248

then (c.red - c.blue)[i(0:0), j(0:0)] = 119 - 248 = 127 ?

Change History (3)

comment:1 by Dimitar Misev, 9 years ago

But you do a cast to (unsigned char), what should be the proper result?

comment:2 by Bang Pham Huu, 9 years ago

Sorry, actually the query does not need encode in "unsigned char", example below

for c in (rgb) return encode(  (c.red  - 248)[i(0:0), j(0:0)] , "csv")

will return {-129}

but

for c in (rgb) return encode(  (c.red  - c.blue)[i(0:0), j(0:0)] , "csv")

will return {127}

and I expected it should be negative number.

comment:3 by Dimitar Misev, 8 years ago

Resolution: invalid
Status: newclosed

The original types are unsigned char of c.red and c.blue, the minus operation doesn't automatically expand the types. You should cast both operands to short for example for this exapmle to work.

for c in (rgb) return encode(  ((short)c.red  - (short)c.blue)[i(0:0), j(0:0)] , "csv")
Note: See TracTickets for help on using tickets.