#359 closed defect (invalid)
Error translating to RasQL for complex WCPS query
| Reported by: | Piero Campalani | Owned by: | Dimitar Misev |
|---|---|---|---|
| Priority: | critical | Milestone: | 8.4.2 |
| Component: | qlparser | Version: | 8.4 |
| Keywords: | Cc: | Dirk Daems | |
| Complexity: | Hard |
Description
(issued discovered by Dirk Daems)
PREFACE: currently the specific cause of the hereby discussed error is not known: as soon as it will be is identified, the ticket title will change accordingly.
Briefly: starting from a valid (and huge) add() clause, it cannot resolve to a valid RasQL query if wrapped by e.g. a sqrt() operation.
Working query:
for c in (NDVI_SPOT_VGT)
return
add(
coverage averagesOverTime
over $T t(0:14)
values(
(
(
((1/(count((c[x(15:20),y(0:5),t(2+$T*36)]) <= 250)) * add(((c[x(15:20),y(0:5),t(2+$T*36)]) * ((c[x(15:20),y(0:5),t(2+$T*36)]) <= 250))) )) * 0.004 - 0.08)
-
avg(coverage averagesOverTime2
over $T2 t(0:14)
values(((1/(count((c[x(15:20),y(0:5),t(2+$T2*36)]) <= 250)) * add(((c[x(15:20),y(0:5),t(2+$T2*36)]) * ((c[x(15:20),y(0:5),t(2+$T2*36)]) <= 250))) )) * 0.004 - 0.08)
)
)
*
(
(((1/(count((c[x(15:20),y(0:5),t(2+$T*36)]) <= 250)) * add(((c[x(15:20),y(0:5),t(2+$T*36)]) * ((c[x(15:20),y(0:5),t(2+$T*36)]) <= 250))) )) * 0.004 - 0.08)
-
avg(coverage averagesOverTime3
over $T3 t(0:14)
values(((1/(count((c[x(15:20),y(0:5),t(2+$T3*36)]) <= 250)) * add(((c[x(15:20),y(0:5),t(2+$T3*36)]) * ((c[x(15:20),y(0:5),t(2+$T3*36)]) <= 250))) )) * 0.004 - 0.08)
)
)
)
)/15)
..which is translated into the following RasQL query:
select add_cells(marray i_q in [0:14] values ((((((1)/(count_cells((((c) [(2)+((i_q[0])*(36)),3696:4256,3696:4256])<=(250)))))*(add_cells((((c) [(2)+((i_q[0])*(36)),3696:4256,3696:4256])*((((c) [(2)+((i_q[0])*(36)),3696:4256,3696:4256])<=(250)))))))*(0.004))-(0.08))-(avg_cells(marray i_s in [0:14] values ((((1)/(count_cells((((c) [(2)+((i_s[0])*(36)),3696:4256,3696:4256])<=(250)))))*(add_cells((((c) [(2)+((i_s[0])*(36)),3696:4256,3696:4256])*((((c) [(2)+((i_s[0])*(36)),3696:4256,3696:4256])<=(250)))))))*(0.004))-(0.08))))*((((((1)/(count_cells((((c) [(2)+((i_q[0])*(36)),3696:4256,3696:4256])<=(250)))))*(add_cells((((c) [(2)+((i_q[0])*(36)),3696:4256,3696:4256])*((((c) [(2)+((i_q[0])*(36)),3696:4256,3696:4256])<=(250)))))))*(0.004))-(0.08))-(avg_cells(marray i_t in [0:14] values ((((1)/(count_cells((((c) [(2)+((i_t[0])*(36)),3696:4256,3696:4256])<=(250)))))*(add_cells((((c) [(2)+((i_t[0])*(36)),3696:4256,3696:4256])*((((c) [(2)+((i_t[0])*(36)),3696:4256,3696:4256])<=(250)))))))*(0.004))-(0.08))))) from NDVI_SPOT_VGT AS c
Previous WCPS query with an additional external sqrt() call:
for c in (NDVI_SPOT_VGT)
return
sqrt(
add(
coverage averagesOverTime
over $T t(0:14)
values(
(
(((1/(count((c[x(15:20),y(0:5),t(2+$T*36)]) <= 250)) * add(((c[x(15:20),y(0:5),t(2+$T*36)]) * ((c[x(15:20),y(0:5),t(2+$T*36)]) <= 250))) )) * 0.004 - 0.08)
-
avg(coverage averagesOverTime2
over $T2 t(0:14)
values(((1/(count((c[x(15:20),y(0:5),t(2+$T2*36)]) <= 250)) * add(((c[x(15:20),y(0:5),t(2+$T2*36)]) * ((c[x(15:20),y(0:5),t(2+$T2*36)]) <= 250))) )) * 0.004 - 0.08)
)
)
*
(
(((1/(count((c[x(15:20),y(0:5),t(2+$T*36)]) <= 250)) * add(((c[x(15:20),y(0:5),t(2+$T*36)]) * ((c[x(15:20),y(0:5),t(2+$T*36)]) <= 250))) )) * 0.004 - 0.08)
-
avg(coverage averagesOverTime3
over $T3 t(0:14)
values(((1/(count((c[x(15:20),y(0:5),t(2+$T3*36)]) <= 250)) * add(((c[x(15:20),y(0:5),t(2+$T3*36)]) * ((c[x(15:20),y(0:5),t(2+$T3*36)]) <= 250))) )) * 0.004 - 0.08)
)
)
)
)/15))
…which is translated into the following RasQL query:
select sqrt((add_cells(marray i_y in [0:14] values ((((((1)/(count_cells((((c) [(2)+((i_y[0])*(36)),3696:4256,3696:4256])<=(250)))))*(add_cells((((c) [(2)+((i_y[0])*(36)),3696:4256,3696:4256])*((((c) [(2)+((i_y[0])*(36)),3696:4256,3696:4256])<=(250)))))))*(0.004))-(0.08))-(avg_cells(marray i_{ in [0:14] values ((((1)/(count_cells((((c) [(2)+((i_{[0])*(36)),3696:4256,3696:4256])<=(250)))))*(add_cells((((c) [(2)+((i_{[0])*(36)),3696:4256,3696:4256])*((((c) [(2)+((i_{[0])*(36)),3696:4256,3696:4256])<=(250)))))))*(0.004))-(0.08))))*((((((1)/(count_cells((((c) [(2)+((i_y[0])*(36)),3696:4256,3696:4256])<=(250)))))*(add_cells((((c) [(2)+((i_y[0])*(36)),3696:4256,3696:4256])*((((c) [(2)+((i_y[0])*(36)),3696:4256,3696:4256])<=(250)))))))*(0.004))-(0.08))-(avg_cells(marray i_| in [0:14] values ((((1)/(count_cells((((c) [(2)+((i_|[0])*(36)),3696:4256,3696:4256])<=(250)))))*(add_cells((((c) [(2)+((i_|[0])*(36)),3696:4256,3696:4256])*((((c) [(2)+((i_|[0])*(36)),3696:4256,3696:4256])<=(250)))))))*(0.004))-(0.08))))))/(15)) from NDVI_SPOT_VGT AS c
…and which produces the following error:
Caused by: org.odmg.QueryException: Parsing error 300 in line 1, column 294: Unexpected name {.
at rasj.odmg.RasOQLQuery.execute(RasOQLQuery.java:249)
at petascope.util.ras.RasUtil.executeRasqlQuery(RasUtil.java:111)
... 18 more
It looks like the name used in the RASQL array constructor (… marray i_{ in …) is causing the error.
Change History (11)
comment:1 by , 11 years ago
comment:2 by , 11 years ago
| Component: | catalogmgr → qlparser |
|---|
comment:3 by , 11 years ago
Dimitar,
I believe this is a ticket for Petascope: the problem is that it creates a variable called '"i_{"', which is not accepted by the qlparser:
$ rasql -q "SELECT add_cells(marray i_{ in [0:1] values 1c) FROM mean_summer_airtemp AS c" --out string
rasql: rasdaman query tool v1.0, rasdaman v8.4.0rc.2-10-g96e9534 -- generated on 08.04.2013 16:58:10.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...rasdaman error 300: Parsing error 300 in line 1, column 27: Unexpected name {.
aborting transaction...ok
rasql done.
comment:4 by , 11 years ago
Domain-info on the dataset used in the complex query above:
name | name | axistype | i | numlo | numhi | name ---------------+------+----------+---+-------------+------------+-------------------------------------------- NDVI_SPOT_VGT | t | t | 0 | 0 | 540 | CRS:1 NDVI_SPOT_VGT | x | x | 1 | -18.0044643 | 52.0044643 | http://www.opengis.net/def/crs/EPSG/0/4326 NDVI_SPOT_VGT | y | y | 2 | -36.0044643 | 38.0044643 | http://www.opengis.net/def/crs/EPSG/0/4326 (3 rows)
follow-up: 7 comment:5 by , 11 years ago
| Status: | new → accepted |
|---|
I can't reproduce it, can you try with the latest petascope Dirk?
I tested with this query:
for c in (eobstest)
return sqrt(
(
(
add(
coverage averagesOverTime
over $T t(0:2)
values
(
(((1/(count((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) <= 250)) * add(((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) * ((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) <= 250))) )) * 0.004 - 0.08)
-
avg(coverage averagesOverTime2
over $T2 t(0:2)
values(((1/(count((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T2 + 1)]) <= 250)) * add(((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T2 + 1)]) * ((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T2 + 1)]) <= 250))) )) * 0.004 - 0.08))
)
*
(
(((1/(count((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) <= 250)) * add(((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) * ((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) <= 250))) )) * 0.004 - 0.08)
-
avg(coverage averagesOverTime3
over $T3 t(0:2)
values(((1/(count((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T3 + 1)]) <= 250)) * add(((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T3 + 1)]) * ((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T3 + 1)]) <= 250))) )) * 0.004 - 0.08)
)
)
)
)) / 15)
result rasql is
select sqrt((add_cells(marray i_i in [0:2] values ((((((1)/(count_cells((((c) [((1)*(i_i[0]))+(1),15:20,0:5])<=(250)))))*(add_cells((((c) [((1)*(i_i[0]))+(1),15:20,0:5])*((((c) [((1)*(i_i[0]))+(1),15:20,0:5])<=(250)))))))*(0.004))-(0.08))-(avg_cells(marray i_j in [0:2] values ((((1)/(count_cells((((c) [((1)*(i_j[0]))+(1),15:20,0:5])<=(250)))))*(add_cells((((c) [((1)*(i_j[0]))+(1),15:20,0:5])*((((c) [((1)*(i_j[0]))+(1),15:20,0:5])<=(250)))))))*(0.004))-(0.08))))*((((((1)/(count_cells((((c) [((1)*(i_i[0]))+(1),15:20,0:5])<=(250)))))*(add_cells((((c) [((1)*(i_i[0]))+(1),15:20,0:5])*((((c) [((1)*(i_i[0]))+(1),15:20,0:5])<=(250)))))))*(0.004))-(0.08))-(avg_cells(marray i_k in [0:2] values ((((1)/(count_cells((((c) [((1)*(i_k[0]))+(1),15:20,0:5])<=(250)))))*(add_cells((((c) [((1)*(i_k[0]))+(1),15:20,0:5])*((((c) [((1)*(i_k[0]))+(1),15:20,0:5])<=(250)))))))*(0.004))-(0.08))))))/(15)) from eobstest AS c
comment:6 by , 11 years ago
I added a test to the systemtest in changeset:ae213f5 which demonstrates this works fine for the eobstest coverage in the tests.
comment:7 by , 11 years ago
Dimitar,
Can you let me know how I can upload the eobstest coverage? I can't find it in the repository.
Thanks,
Dirk
Replying to dmisev:
I can't reproduce it, can you try with the latest petascope Dirk?
I tested with this query:
for c in (eobstest) return sqrt( ( ( add( coverage averagesOverTime over $T t(0:2) values ( (((1/(count((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) <= 250)) * add(((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) * ((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) <= 250))) )) * 0.004 - 0.08) - avg(coverage averagesOverTime2 over $T2 t(0:2) values(((1/(count((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T2 + 1)]) <= 250)) * add(((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T2 + 1)]) * ((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T2 + 1)]) <= 250))) )) * 0.004 - 0.08)) ) * ( (((1/(count((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) <= 250)) * add(((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) * ((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T + 1)]) <= 250))) )) * 0.004 - 0.08) - avg(coverage averagesOverTime3 over $T3 t(0:2) values(((1/(count((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T3 + 1)]) <= 250)) * add(((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T3 + 1)]) * ((c[x:"CRS:1"(15:20),y:"CRS:1"(0:5),t(1 * $T3 + 1)]) <= 250))) )) * 0.004 - 0.08) ) ) ) )) / 15)result rasql is
select sqrt((add_cells(marray i_i in [0:2] values ((((((1)/(count_cells((((c) [((1)*(i_i[0]))+(1),15:20,0:5])<=(250)))))*(add_cells((((c) [((1)*(i_i[0]))+(1),15:20,0:5])*((((c) [((1)*(i_i[0]))+(1),15:20,0:5])<=(250)))))))*(0.004))-(0.08))-(avg_cells(marray i_j in [0:2] values ((((1)/(count_cells((((c) [((1)*(i_j[0]))+(1),15:20,0:5])<=(250)))))*(add_cells((((c) [((1)*(i_j[0]))+(1),15:20,0:5])*((((c) [((1)*(i_j[0]))+(1),15:20,0:5])<=(250)))))))*(0.004))-(0.08))))*((((((1)/(count_cells((((c) [((1)*(i_i[0]))+(1),15:20,0:5])<=(250)))))*(add_cells((((c) [((1)*(i_i[0]))+(1),15:20,0:5])*((((c) [((1)*(i_i[0]))+(1),15:20,0:5])<=(250)))))))*(0.004))-(0.08))-(avg_cells(marray i_k in [0:2] values ((((1)/(count_cells((((c) [((1)*(i_k[0]))+(1),15:20,0:5])<=(250)))))*(add_cells((((c) [((1)*(i_k[0]))+(1),15:20,0:5])*((((c) [((1)*(i_k[0]))+(1),15:20,0:5])<=(250)))))))*(0.004))-(0.08))))))/(15)) from eobstest AS c
comment:8 by , 11 years ago
Hi Dirk,
the dataset is in "systemtest/testcases_petascope/test_wcps/testdata/eobs.nc", but you can get is automatically inserted if you run the WCS or WCPS tests in "systemtest/testcases_petascope/".
See: wiki:RasdamanTestSuites#Petascopetests
Let me know if that works.
comment:9 by , 11 years ago
The problem seems to be fixed. Works for both the eobs test coverage and our NDVI_SPOT_VGT coverage.
comment:10 by , 11 years ago
| Resolution: | → invalid |
|---|---|
| Status: | accepted → closed |
Yes, I think you just had a bit older petascope. Closing the ticket then.
comment:11 by , 11 years ago
| Milestone: | 9.0 → 8.4.2 |
|---|
Note:
See TracTickets
for help on using tickets.
I tried to reproduce the error with some similar but much shorter WCPS query.
But they all worked:
for c in (mean_summer_airtemp) return sqrt( add( coverage averagesOverTime over $T t(0:4) values $T ) )for c in (mean_summer_airtemp) return sqrt( add( coverage outerCov over $T t(0:4) values max( coverage innerCov over $T t(0:5) values $T ) ) )for c in (mean_summer_airtemp) return sqrt( max( coverage innerCov over $T t(0:5) values c[x:"CRS:1"($T),y:"CRS:1"(300)] ) )for c in (mean_summer_airtemp) return sqrt( add( coverage outerCov over $T t(0:4) values 1 + max( coverage innerCov over $T t(0:5) values c[x:"CRS:1"($T),y:"CRS:1"(300)] ) ) )