Opened 4 years ago

Closed 4 years ago

#376 closed defect (fixed)

Aggregation operations in petascope don't work on multiband covs

Reported by: pcampalani Owned by: abeccati
Priority: blocker Milestone: 9.0
Component: petascope Version: 8.4
Keywords: wcps multiband operation Cc: j.oosthoek@…, uagha
Complexity: Hard

Description (last modified by pcampalani)

Petascope seems to provide wrong (and different at every request) results to operations applied on multiband coverages, whether the operation is applied on a single band or not.

Simple examples on the rgb test coverage:

  1. AVG on 1 RGB cell (should return the cell itself?):
    for m in (rgb) return encode(
        trim(m,
            {x:"CRS:1"(200:200),y:"CRS:1"(200:200)}),
        "csv") 
    
    ==> {"242 27 103"}
    
    for m in (rgb) return encode(
        avg(
            trim(m,
                {x:"CRS:1"(200:200),y:"CRS:1"(200:200)})
        ), "csv") 
    
    ==> {"0 8.70018e-313 7.19645e-262"}
    
  1. AVG on 1 RGB cell, only the RED band (should return the single value itself):
    for m in (rgb) return encode(
        trim(m,
            {x:"CRS:1"(200:200),y:"CRS:1"(200:200)}),
        "csv") 
    
    ==> {242}
    
    for m in (rgb) return encode(
        avg(
            trim(m,
                {x:"CRS:1"(200:200),y:"CRS:1"(200:200)})
        ), "csv") 
    
    ==> {0}
    
  1. ADD over a multiple bands:
    for m in (rgb) return encode(
        trim(m,
            {x:"CRS:1"(200:201),y:"CRS:1"(200:201)}),
        "csv") 
    
    ==> {"242 27 103","242 27 103"}
    
    for m in (rgb) return encode(
        add(
            (float)trim(m,
                {x:"CRS:1"(200:200),y:"CRS:1"(200:201)})
        ), "csv") 
    
    ==> {"7.26573e-262 3.61544e-313 8.49162e-314"}
    ==> {"484 7563062 16"} without (float) cast
    ==> {"484 7563062 0"}  without (float) cast
    
  1. ADD over a single band:
    for m in (rgb) return encode(
        trim(m.red,
            {x:"CRS:1"(200:201),y:"CRS:1"(200:201)}),
        "csv") 
    
    ==> {242,242}
    
    for m in (rgb) return encode(
        add(
            (float)trim(m.red,
                {x:"CRS:1"(200:200),y:"CRS:1"(200:201)})
        ), "csv") 
    
    ==> {8.10648e+16}
    

. Multiplication of multi-band results (but maybe this is not implemented at all):

for m in (rgb) return encode(
    (float)trim(m,{x:"CRS:1"(200:200),y:"CRS:1"(200:200)}) *
    (float)trim(m,{x:"CRS:1"(200:200),y:"CRS:1"(200:200)}), "csv") 

==> Error: Could not evaluate rasdaman query: 
'select csv((float)(((m[200:200,200:200])*((float)(m[200:200,200:200]))))) from rgb AS m'
Cause: Error evaluating rasdaman query: 'select csv((float)(((m[200:200,200:200])*((float)(m[200:200,200:200]))))) from rgb AS m

Change History (15)

comment:1 Changed 4 years ago by pcampalani

  • Description modified (diff)

comment:2 Changed 4 years ago by dmisev

  1. I get "Error: Exception: Base type typeName: typeID: 7 typeSize: 16 struct { typeName: RAS_DOUBLE typeID: 15 typeSize: 8 red, typeName: RAS_DOUBLE typeID: 15 typeSize: 8 green } as ElementType? is not supported yet." for
    for m in (rgb) return
        avg(m)
    
  2. I get 242.0
    for m in (rgb) return
        avg(
            trim(m.red,
                {x:"CRS:1"(200:200),y:"CRS:1"(200:200)})
        )
    
  3. Similarly gives "Error: Exception: Base type typeName: typeID: 7 typeSize: 16 struct { typeName: RAS_DOUBLE typeID: 15 typeSize: 8 red, typeName: RAS_DOUBLE typeID: 15 typeSize: 8 green } as ElementType? is not supported yet."
    for m in (rgb) return
        add(
            (float)trim(m,
                {x:"CRS:1"(200:200),y:"CRS:1"(200:201)})
        )
    
  4. I'm sure if you don't use the encode it will be fine as well like 2.
  5. If you remove the casts it will work
    for m in (rgb) return encode(
        trim(m,{x:"CRS:1"(200:200),y:"CRS:1"(200:200)}) *
        trim(m,{x:"CRS:1"(200:200),y:"CRS:1"(200:200)}), "csv") 
    

comment:3 Changed 4 years ago by dmisev

  • Summary changed from Bad (and time-varying) WCPS operations results with multi-band coverages to Aggregation operations in petascope don't work on multiband covs

So in conclusion the real bug seems to be that aggregation operations over composite types do not work in petascope, but they work just fine in rasql.

comment:4 Changed 4 years ago by dmisev

  • Priority changed from blocker to major

Wait, why is this blocker? You can certainly work around it.

comment:5 Changed 4 years ago by pcampalani

Yes, the problem seems to reside indeed in the "csv" encoding, so no RasQL problem.

This works:

for m in (rgb) return
    avg(trim(m.1,
        {x:"CRS:1"(200:200),y:"CRS:1"(200:200)})
)

However condensing in parallel on a tuple still is not available:

Error: Exception: Base type typeName: typeID: 7 typeSize: 16 struct { typeName: RAS_DOUBLE typeID: 15 typeSize: 8 red, typeName: RAS_DOUBLE typeID: 15 typeSize: 8 green } as ElementType is not supported yet.

comment:6 Changed 4 years ago by dmisev

Yes, csv should is meant to be used for array results, it doesn't make sense for scalars.

comment:7 Changed 4 years ago by dmisev

  • Milestone changed from 8.5 to 9.0

comment:8 Changed 4 years ago by dmisev

  • Resolution set to fixed
  • Status changed from new to closed

comment:9 Changed 4 years ago by pcampalani

  • Cc uagha added
  • Resolution fixed deleted
  • Status changed from closed to reopened

Re-opening since the associated WCPS test (n.85) fails again now:

select avg_cells(c[100:102,241:243]) from rgb AS c

I get this error:

SEVERE:   rasj[0] RasRNPImplementation.getResponse: type not supported: typeName: 
 typeID:  7
 typeSize: 16
struct 
{
  typeName: RAS_DOUBLE
 typeID:  15
 typeSize: 8
 red, 
typeName: RAS_DOUBLE
 typeID:  15
 typeSize: 8
 green
}

comment:10 Changed 4 years ago by pcampalani

  • Priority changed from major to blocker

comment:11 Changed 4 years ago by dmisev

I think it is unnecessary to reopen this, as the issue is in rasql and you have already opened #638

comment:12 Changed 4 years ago by dmisev

Ah sorry, I didn't read your comment fully.

That should not happen, do you have the latest master?

comment:13 Changed 4 years ago by pcampalani

Yes, maybe I did not include the very latest patches: I am recompiling now to make sure (do you get that too btw?)

comment:14 Changed 4 years ago by dmisev

I'm not sure about this one, but #638 is confirmed.

comment:15 Changed 4 years ago by pcampalani

  • Resolution set to fixed
  • Status changed from reopened to closed

Ok, re-compiling the query runs fine. WCPS test n.85 valid again.

Note: See TracTickets for help on using tickets.