Opened 12 years ago
Closed 12 years ago
#172 closed defect (fixed)
WCPS ConstructCoverageExpr
| Reported by: | Piero Campalani | Owned by: | Dimitar Misev |
|---|---|---|---|
| Priority: | major | Milestone: | 8.4 |
| Component: | petascope | Version: | 8.3 |
| Keywords: | constructcoverageexpr over | Cc: | |
| Complexity: | Medium |
Description
There seems to be a bug when executing a WCPS ConstructCoverageExpr. E.g.:
for cover in (mean_summer_airtemp)
return encode(
coverage temp over $px x (0:0)
values 1,
"csv")
which throws error in ConstructCoverageExpr.java::buildMetadata() method, at the line:
xq.getMetadataSource().read(covName);
Stack trace:
SEVERE: InvalidRequest: Coverage 'temp' has invalid metadata
at petascope.core.DbMetadataSource.read(DbMetadataSource.java:632)
at petascope.wcps.server.core.ConstructCoverageExpr.buildMetadata(ConstructCoverageExpr.java:158)
at petascope.wcps.server.core.ConstructCoverageExpr.<init>(ConstructCoverageExpr.java:68)
at petascope.wcps.server.core.CoverageExpr.<init>(CoverageExpr.java:87)
at petascope.wcps.server.core.EncodeDataExpr.<init>(EncodeDataExpr.java:60)
at petascope.wcps.server.core.XmlQuery.startParsing(XmlQuery.java:105)
at petascope.wcps.server.core.ProcessCoveragesRequest.<init>(ProcessCoveragesRequest.java:98)
at petascope.wcps.server.core.Wcps.pcPrepare(Wcps.java:167)
at petascope.wcps.server.core.Wcps.pcPrepare(Wcps.java:161)
at petascope.wcps.server.servlet.WcpsServlet.doPost(WcpsServlet.java:173)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:688)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1542)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)
at com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:849)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:746)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1045)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:228)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:679)
Caused by: InvalidRequest: Coverage 'temp' is not served by this server
at petascope.core.DbMetadataSource.read(DbMetadataSource.java:449)
... 35 more
As a temporal workaround, commenting out that line seem to make things work, but this may not be and probably is not an optimal solution.
Change History (11)
comment:1 by , 12 years ago
comment:2 by , 12 years ago
Whereas this query works (after proposed workaround!):
for c in (mean_summer_airtemp) return encode(
coverage temp over $px x (0:2)
values avg(c[
x:"CRS:1"(0:1000),
y:"CRS:1"(0:1000)]),
"csv")
OUTPUT: {48.6546,48.6546,48.6546}
..adding the axis label of the constructed coverage in the subsets, gives error:
1) for c in (mean_summer_airtemp) return encode(
coverage temp over $px x (0:2)
values avg(c[
x:"CRS:1"($px),
y:"CRS:1"(1000)]), "csv")
ERROR:
$ rasql -q "select csv(marray i_i in [0:2] values avg_cells((c) [i_i[0],1000])) from mean_summer_airtemp AS c" --out string
rasql: rasdaman query tool v1.0, rasdaman v8 -- generated on 10.04.2012 15:44:22.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...rasdaman error 353: Execution error 353 in line 1, column 39, near token avg_cells : Operand of quantifier must be multidimensional.
aborting transaction...ok
rasql done.
2) for c in (mean_summer_airtemp) return encode(
coverage temp over $px x (0:2)
values avg(c[
x:"CRS:1"($px),
y:"CRS:1"(0:1000)]), "csv")
ERROR: line 2:33 no viable alternative at input 'over'
line 5:10 mismatched input 'y' expecting STRING
WCPS: error
WCPS: setting response mimetype to text/html; charset=utf-8
WCPS: returning the following error message
java.lang.NullPointerException
at petascope.wcps.grammar.EncodedCoverageExpr.<init>(EncodedCoverageExpr.java:40)
at petascope.wcps.grammar.wcpsParser.encodedCoverageExpr(wcpsParser.java:811)
at petascope.wcps.grammar.wcpsParser.processingExpr(wcpsParser.java:676)
at petascope.wcps.grammar.wcpsParser.returnClause(wcpsParser.java:510)
at petascope.wcps.grammar.wcpsParser.wcpsRequest(wcpsParser.java:219)
at petascope.wcps.server.core.ProcessCoveragesRequest.abstractQueryToXmlQuery(ProcessCoveragesRequest.java:132)
at petascope.wcps.server.servlet.WCPSServlet.doPost(WCPSServlet.java:155)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:679)
WCPS: end of error message
WCPS: done with error
comment:3 by , 12 years ago
Just to bump this, we are also suffereing with the same bug. Primarily the use of geospatial subsetting in the coverage constructor query.
comment:4 by , 12 years ago
Hi olcl,
WCPS requests with geographic subsets should work:
for c in (mean_summer_airtemp) return encode(
coverage temp over $px x (0:2)
values avg(c[
x:"http://www.opengis.net/def/crs/EPSG/0/4326"(115:120),
y:"http://www.opengis.net/def/crs/EPSG/0/4326"(-40:-35)]),
"csv")
Response: {0.201255,0.201255,0.201255}.
Assumptions: the native CRS of mean_summer_airtemp is http://www.opengis.net/def/crs/EPSG/0/4326*, and the above-mentioned workaround is applied.
- Check with:
petascopedb=# select ps_coverage.name, ps_domain.name, ps_crs.name from ps_crsset, ps_crs, ps_domain, ps_coverage where ps_crsset.crs=ps_crs.id and ps_crsset.axis=ps_domain.id and ps_domain.coverage=ps_coverage.id and ps_coverage.id≤coverageName>;
comment:5 by , 12 years ago
Hi Piero,
the query i want to use is :
for c in ( myocean_test )
return
encode( (float)
coverage histogram over $pt t( 0 : 10 )
values avg((c[c[x:"http://www.opengis.net/def/crs/EPSG/0/4326"(-4.552002:-4.552002), y:"http://www.opengis.net/def/crs/EPSG/0/4326"(53.904338:53.904338),t($pt)] > 0.0) * (c[c[x:"http://www.opengis.net/def/crs/EPSG/0/4326"(-4.552002:-4.552002), y:"http://www.opengis.net/def/crs/EPSG/0/4326"(53.904338:53.904338),t($pt)])),
"csv")
where i uses geographic susetting within a coverage constructor. Does the proposed work around fix this?
The current error i get is :
line 5:23 no viable alternative at input 'over' line 6:90 mismatched input 'y' expecting STRING WCPS: error WCPS: setting response mimetype to text/html; charset=utf-8 WCPS: returning the following error message java.lang.NullPointerException at petascope.wcps.grammar.EncodedCoverageExpr.<init>(EncodedCoverageExpr.java:40) at petascope.wcps.grammar.wcpsParser.encodedCoverageExpr(wcpsParser.java:811) at petascope.wcps.grammar.wcpsParser.processingExpr(wcpsParser.java:676) at petascope.wcps.grammar.wcpsParser.returnClause(wcpsParser.java:510) at petascope.wcps.grammar.wcpsParser.wcpsRequest(wcpsParser.java:219) at petascope.wcps.server.core.ProcessCoveragesRequest.abstractQueryToXmlQuery(ProcessCoveragesRequest.java:132) at petascope.wcps.server.servlet.WCPSServlet.doPost(WCPSServlet.java:155) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:679) WCPS: end of error message WCPS: done with error
comment:6 by , 12 years ago
the above has a typo in, the query should have read :
for c in ( myocean_test )
return
encode( (float)
coverage histogram over $pt t( 0 : 10 )
values avg((c[x:"http://www.opengis.net/def/crs/EPSG/0/4326"(-4.552002:-4.552002), y:"http://www.opengis.net/def/crs/EPSG/0/4326"(53.904338:53.904338)] > 0.0) * (c[c[x:"http://www.opengis.net/def/crs/EPSG/0/4326"(-4.552002:-4.552002), y:"http://www.opengis.net/def/crs/EPSG/0/4326"(53.904338:53.904338)])),
"csv")
sorry
comment:7 by , 12 years ago
Hi,
firstly, the hereby proposed workaround seems to create leaks in other WCPS requests which do not involve a ConstructCoverage expression (Jelmer's WCPS client started having problems). I couldn't verify the actual effects of that workaround for general WCPS queries.
Anyway, your syntax seems fine, a part maybe from an other typo you did (… 0.0) * (c[c[x: …).
You are also requesting slices, not trims, so you may remove that duplicated second extent in each subset.
I will make some test locally, hope to get back with some news.
If you want to try out the workaround, just contact me, I will provide a .war for Petascope to try out.
comment:8 by , 12 years ago
| Priority: | minor → major |
|---|
comment:9 by , 12 years ago
First of all we should mind the ticket #64, so that in the ConstructCoverage we should force subsetting of time as well: t($pt:$pt) instead of simply t($pt), not to mix slices with trims inside the same square brackets.
You can appreciate results on a single timestamp with the following query (I simplified it a little for readability):
for c in ( myocean_test ) return encode( (float)
avg(c[
x:"http://www.opengis.net/def/crs/EPSG/0/4326"(-4.552002:-4.552002),
y:"http://www.opengis.net/def/crs/EPSG/0/4326"(53.904338:53.904338),
t:"CRS:1"(5:5)]),
"csv")
Instead, when wanting to construct an histogram of values, there is a bug in the parser, before looking at any metadata:
for c in ( myocean_test ) return encode( (float)
coverage histogram over $pt t( 0 : 10 ) values
avg(c[
x:"http://www.opengis.net/def/crs/EPSG/0/4326"(-4.552002:-4.552002),
y:"http://www.opengis.net/def/crs/EPSG/0/4326"(53.904338:53.904338),
t:"CRS:1"($pt:$pt)]),
"csv")
gives (similarly to ticket #64) :
SEVERE: line 2:24 no viable alternative at input 'over' SEVERE: line 5:10 mismatched input 'y' expecting STRING INFO: WCPS: error
comment:10 by , 12 years ago
I might have found a workaround.
Firstly:
Lemma: references to a variable $var defined in the ConstructCoverageExpr can be put only inside of slices operations (no trims).
Then, since mixing of trims and slices within a bracket subsetting is currently not allowed (ticket #64), an average over different time slides and over a restricted spatial domain could be achieved this way:
for c in (MODIS_33N_2010170_WGS84) return encode((float)
coverage temp over $t t (0:3)
values avg(slice(
trim(c, {
x(10:11),
y(46:47)}), {
t($t)})),
"csv")
This works when commenting out the reported line in ConstructCoverageExpr.java::buildMetadata(), and this however seems to create problems with other WCPS expressions; going to search for a better solution.
comment:11 by , 12 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
I believe we fixed this in the latest sources
Involved WCPS grammar, as currently defined in Petascope:
coverageConstructorExpr returns[CoverageConstructorExpr value] : COVERAGE coverage=coverageName OVER ail=axisIteratorList VALUES se=scalarExpr { $value = new CoverageConstructorExpr($coverage.value, $ail.value, $se.value); } ; axisIteratorList returns[AxisIteratorList value] : vn=variableName an=axisName LPAREN ie=intervalExpr RPAREN { $value = new AxisIteratorList(new AxisIterator($vn.value, $an.value, $ie.value)); } (COMMA vn2=variableName an2=axisName LPAREN ie2=intervalExpr RPAREN { $value = new AxisIteratorList(new AxisIterator($vn2.value, $an2.value, $ie2.value), $value); })* ; scalarExpr returns[ScalarExpr value] : e1=metaDataExpr { $value = new ScalarExpr($e1.value); } | e2=condenseExpr { $value = new ScalarExpr($e2.value); } | e3=booleanScalarExpr { $value = new ScalarExpr($e3.value); } | e4=numericScalarExpr { $value = new ScalarExpr($e4.value); } | e5=stringScalarExpr { $value = new ScalarExpr($e5.value); } | LPAREN e6=scalarExpr RPAREN { $value = $e6.value; } intervalExpr returns[IntervalExpr value] : lo=indexExpr COLON hi=indexExpr { $value = new IntervalExpr($lo.value, $hi.value); } | IMAGECRSDOMAIN LPAREN e1=coverageName COMMA e2=axisName RPAREN { $value = new IntervalExpr($e1.value, $e2.value); } ; variableName returns[String value] : var=VARIABLE_DOLLAR { $value = new String($var.text); } ; VARIABLE_DOLLAR: '$'(('a'..'z'|'A'..'Z'|'0'..'9'|'_')*) {setText(getText().substring(1, getText().length())); } ;