Opened 4 years ago

Closed 8 months ago

#406 closed defect (fixed)

WCPS1.5_Omitting whitespace in expressions sometimes results in errors

Reported by: dirk.daems@… Owned by: mdumitru
Priority: major Milestone: 9.3
Component: petascope Version: development
Keywords: parser exception operands whitespace Cc: dmisev, mdumitru
Complexity: Medium

Description

Omitting whitespace in expressions sometimes results in errors.

The following WCPS query is evaluated fine:

  for c in (NDVI_SPOT_VGT)
  return encode
     (coverage NDVI_HIST_STDEV
               over $pt t(0:1)
      values (
               ((c[t(28 + 36*$pt),x:"CRS:1"(4001),y:"CRS:1"(4001)] * 0.004) - 0.08)
             )
     ,"csv")

But when I omit the space character before the number 0.08 ( ((c[t(28 + 36*$pt),x:"CRS:1"(4001),y:"CRS:1"(4001)] * 0.004) -0.08) ), I get a Petascope error:

[08:24:58] ERROR PetascopeInterface@436: Error stack trace:
RuntimeError: Runtime error while processing request: null
	at petascope.PetascopeInterface.doGet(PetascopeInterface.java:391)
	at petascope.PetascopeInterface.doPost(PetascopeInterface.java:214)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.NullPointerException
	at petascope.wcps.grammar.EncodedCoverageExpr.<init>(EncodedCoverageExpr.java:42)
	at petascope.wcps.grammar.wcpsParser.encodedCoverageExpr(wcpsParser.java:811)
	at petascope.wcps.grammar.wcpsParser.processingExpr(wcpsParser.java:676)
	at petascope.wcps.grammar.wcpsParser.returnClause(wcpsParser.java:510)
	at petascope.wcps.grammar.wcpsParser.wcpsRequest(wcpsParser.java:219)
	at petascope.util.ras.RasUtil.abstractWCPStoXML(RasUtil.java:225)
	at petascope.PetascopeInterface.doGet(PetascopeInterface.java:320)
	... 15 more

Nevertheless, omitting the space character before the number 0.004 is not causing problems.

Change History (23)

comment:1 Changed 4 years ago by abeccati

  • Cc swingit pcampalani added
  • Keywords parser exception operands whitespace added
  • Priority changed from major to minor

Thanks for reporting

I guess this depends from the fact that -NN can be correctly interpreted as a negative number, thus missing an operator, while *NN cannot be interpreted as such therefore there is ambiguity in the minus sign case, while there is not with the *

This is also a good example of a case for exception improvement, adding swingit in copy and lowering prio since it has a straightforward workaround of keeping spaces to separate operators from operands.

comment:2 Changed 4 years ago by dirk.daems@…

I understand your point but in other query languages (e.g. SQL) the behavior is different. I would suggest to check whether this is clear from the spec (OGC 08-062r2), as for me it wasn't.

comment:3 Changed 4 years ago by dmisev

  • Owner changed from abeccati to swingit
  • Status changed from new to assigned

comment:4 Changed 3 years ago by dmisev

  • Resolution set to fixed
  • Status changed from assigned to closed

comment:5 Changed 3 years ago by dirk.daems@…

  • Resolution fixed deleted
  • Status changed from closed to reopened

In which version was this issue fixed?
In our current version (9.0 pre-release 2) I still get the same NPE error.

comment:6 Changed 3 years ago by dmisev

  • Owner changed from swingit to dmisev
  • Status changed from reopened to assigned

comment:7 Changed 3 years ago by dmisev

  • Milestone set to 9.0

comment:8 Changed 2 years ago by dmisev

  • Cc swingit pcampalani removed
  • Owner changed from dmisev to mdumitru

Alex can you check this in WCPS 1.5?

comment:9 Changed 14 months ago by dmisev

  • Owner changed from mdumitru to bphamhuu

Bang, please verify if this is still the case with the latest petascope (and the new parser).

comment:10 Changed 14 months ago by bphamhuu

  • Cc dmisev added

Dimitar, I just try to mimic the behavior as WCPS is too fragile in syntax error and I don't know what is right or wrong query, so the best way is use WCPS test and modify it. I cannot prove this ticket has been fixed as it does not show runtime error but from the problem of omitting space, I see some logic errors here.

  1. The original query
for c in (irr_cube_2) return encode (
   coverage max_delta_pixel_history2
   over     $t t ( 0 : 3 )
   values   max( (c[E(100000:110000), N(5100000:5101000), ansi:"CRS:1"($t)]).b1 ), "csv")

result {8.64451,3.11651,7.79278,2.5487}

+ Test case 1 (not correct) -3

for c in (irr_cube_2) return encode (
   coverage max_delta_pixel_history2
   over     $t t ( 0 : 3 )
   values   max( (c[E(100000:110000), N(5100000:5101000), ansi:"CRS:1"($t)]).b1 ) -3, "csv")

result is the same

+ Test case 1 (correct) - 3

for c in (irr_cube_2) return encode (
   coverage max_delta_pixel_history2
   over     $t t ( 0 : 3 )
   values   max( (c[E(100000:110000), N(5100000:5101000), ansi:"CRS:1"($t)]).b1 ) - 3, "csv")

result {5.64451,0.116507,4.79278,-0.451298}

+ Test case 2 (not correct) with +5 or + 5
result is the same

+ Test case 3 correct with /5 or / 5

+ Test case 4 correct with * 5 or * 5

So the error with +5 + 5 and -5 need to be fixed.

  1. Error case with another query

Query

for c in (eobstest) return encode(  (coverage newcov
  over $px x(0:2), 
       $py y(0:2), 
       $pt t(0:5)
  values c[t($pt), Lat($px), Long($py)]
  )
, "csv")

+ Test case 2 (Runtime error with processing null) with +3, + 3

for c in (eobstest) return encode(  (coverage newcov
  over $px x(0:2), 
       $py y(0:2), 
       $pt t(0:5)
  values c[t($pt), Lat($px), Long($py)] +3
  )
, "csv")

+ Test case 2 (not correct) with -3 but correct with - 3
+ Test case 2 (correct) with *3, * 3
+ Test case 2 (correct) with /3, / 3

So the same error with +3, + 3, -3.

Conclusion, + operand, +operand and -operand will trigger an error (logic or runtime error).

comment:11 Changed 14 months ago by dmisev

In petascope there are two wcps parsers currently; please make sure you are testing with the new one (v1.5 or v2, not sure).

comment:12 Changed 14 months ago by bphamhuu

so I've to debug to see which wcps is parsing, then report later.

comment:13 Changed 14 months ago by dmisev

You can ask Alex to quickly give you a hint how to make sure you're using the new parser.

comment:14 Changed 14 months ago by dmisev

  • Cc mdumitru added

comment:15 Changed 14 months ago by bphamhuu

In petascope it go to handleWCPS1Request with the query

http://localhost:8088/rasdaman/ows?service=WCS&version=2.0.1&request=ProcessCoverages&query=for c in (irr_cube_2) return encode ( coverage max_delta_pixel_history2 over $t t ( 0 : 3 ) values max( (c[E(100000:110000), N(5100000:5101000), ansi:"CRS:1"($t)]).b1 ) - 3, "csv")

comment:16 Changed 14 months ago by bphamhuu

change to WCPS2 parser I get error with querry

http://localhost:8088/rasdaman/ows?service=WCS&version=2.0.1&request=ProcessCoverages&query=for c in (irr_cube_2) return encode ( coverage max_delta_pixel_history2 over $t t ( 0 : 3 ) values max( (c[E(100000:110000), N(5100000:5101000), ansi:"CRS:1"($t)]).b1 ) -3, "csv")
petascope.wcps2.error.managed.processing.WCPSProcessingError : A parsing error occurred at position=1:169. Offending symbol is=[@50,169:170='-3',<77>,1:169]. Parser message=extraneous input '-3' expecting {AND, ',', '/', '.', '=', '>', '>=', '[', '<', '<=', '-', '*', '!=', OR, OVERLAY, '+', XOR}.

comment:17 Changed 14 months ago by dmisev

So that works at least more correctly, giving an error instead of erroneous output.

comment:18 Changed 14 months ago by bphamhuu

So you'll assign this ticket back to Alex to fix the error in WCPS parser? as I could only test for problem.

comment:19 Changed 14 months ago by dmisev

  • Owner changed from bphamhuu to mdumitru

comment:20 Changed 14 months ago by dmisev

  • Milestone changed from 9.0.x to Future

comment:21 Changed 14 months ago by mdumitru

  • Milestone changed from Future to 9.0.x

Seems there is some error in the parser. I'm not sure antlr can handle two rules targeting same token order, I will check it.

comment:22 Changed 14 months ago by bphamhuu

  • Milestone changed from 9.0.x to 10.0
  • Priority changed from minor to major
  • Summary changed from Omitting whitespace in expressions sometimes results in errors to WCPS1.5_Omitting whitespace in expressions sometimes results in errors
  • Version changed from 8.4 to development

This ticket relates to WCPS new version, then I will set it more important.

comment:23 Changed 8 months ago by bphamhuu

  • Milestone changed from 10.0 to 9.3
  • Resolution set to fixed
  • Status changed from assigned to closed

the patch was accepted, close ticket, thanks.

Note: See TracTickets for help on using tickets.