#1064 closed defect (fixed)

wcst_import extract recipe fails

Reported by: vmerticariu Owned by: bphamhuu
Priority: major Milestone:
Component: petascope Version: development
Keywords: Cc:
Complexity: Medium

Description

When executing an import of a coverage from another server, the initial coverage that is inserted contains 1 point, but it contains several coefficients (which indicates several slices), causing the request to fail.

Attached is the initial coverage that is inserted, in GML format.

Attachments (1)

initial_coverage.json (5.5 KB) - added by vmerticariu 17 months ago.

Download all attachments as: .zip

Change History (12)

Changed 17 months ago by vmerticariu

comment:1 Changed 17 months ago by mdumitru

  • Owner changed from mdumitru to bphamhuu
  • Status changed from new to assigned

comment:2 Changed 17 months ago by mdumitru

Try to ingest this coverage http://flanche.com:9090/rasdaman/ows?SERVICE=WCS&VERSION=2.0.1&REQUEST=DescribeCoverage&coverageId=AvgLandTemp using the wcs extract recipe and see where the error comes from.

comment:3 Changed 17 months ago by bphamhuu

ok, thanks for assign to me (a short message to let you know that I will do it now or soon).

comment:4 Changed 17 months ago by bphamhuu

Alex or Vlad, now I need to know what am I doing is correct or not, please check
Recipe wcs_extract http://pastebin.com/qCwvLnFs

and here is what I got error from your coverage in your host.

Exception thrown when trying to insert slice: 
{Axis Subset: Lat(-90.0,90.0) Long(-180.0,180.0) ansi(145763.0) 
Data Provider: http://flanche.com:9090/rasdaman/ows?service=WCS&version=2.0.1&request=GetCoverage&coverageId=AvgLandTemp&format=image/tiff&subset=Lat(-90.0,90.0)&subset=Long(-180.0,180.0)&subset=ansi(145763.0)}
Retrying, you can safely ignore the warning for now. Tried 1 times.

Progress: [------------------------------] 0/185 0.00% 
Failed to insert slice. Attempted 1 times.
Service Call: http://localhost:8080/rasdaman/ows?service=WCS&version=2.0.1&request=UpdateCoverage&coverageId=alex&subset1=Lat(-90.0,90.0)&inputCoverageRef=file:///tmp/30aaaa1e_5b3c_48fc_8844_3ddf18203ee0.gml&subset3=ansi(145763.0)&subset2=Long(-180.0,180.0)
Error Code: OperationNotSupported
Error Text: <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ows:ExceptionReport version="2.0.0"
    xsd:schemaLocation="http://www.opengis.net/ows/2.0 http://schemas.opengis.net/ows/2.0/owsExceptionReport.xsd"
    xmlns:ows="http://www.opengis.net/ows/2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink">
    <ows:Exception exceptionCode="OperationNotSupported">
        <ows:ExceptionText>Can not add new slice in between existing slices on irregular axis. Only adding slices on top is currently supported.</ows:ExceptionText>
    </ows:Exception>

</ows:ExceptionReport>

If you see it is correct then I can debug WCST-Import to see more detail. Thanks.

comment:5 Changed 17 months ago by vmerticariu

All seems fine, except for the tiling parameter in the ingredient, it should probably be something 3D, i.e.:

ALIGNED [0:500, 0:500, 0]

The fix consists in making sure that in the json file generated for the first slice (the one that I attached), <gmlrgrid:coefficients> contains only 1 element (0), as the coverage contains only 1 point at that moment.

comment:6 Changed 17 months ago by mdumitru

Vlad meant GML file. You should check in util/coverage_reader.py why the slices have more than one coefficient generated. Normally each slice should only have one coefficient.

comment:7 Changed 17 months ago by bphamhuu

Ok, thanks for both of you, and Alex's translation from Vlad's meaning (I have no idea with "point"). I have not debug yet, but go with some information from wcs_extract and DescribeCoverage? in GML. Please also check, I will keep it short:

+ wcs_extract recipe to get separately each layer of coverage (for example: 3D like AvgLandTemp?) by downloading it to /tmp then make a GML for it and import it to local database?

+ coefficient of this coverage is "ansi time"? As I see in "DescribeCoverage?" and gmlrgrid:coefficients (this coverage has 185 slices) and coefficients from 0:5599 (irregular time series).

I would like to confirm as the error " new slice in between existing slices on irregular axis " not looks like what I think now (extract slice from lowest layer and import respectively).

comment:8 Changed 17 months ago by bphamhuu

@Alex, it is not related to this ticket but sometime I see error from your hosting, so I can not debug now (before it works and it can download tiff file), I will try to investigate more detail soon.

<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows/2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" version="2.0.0" xsd:schemaLocation="http://www.opengis.net/ows/2.0 http://schemas.opengis.net/ows/2.0/owsExceptionReport.xsd">
<ows:Exception exceptionCode="RasdamanRequestFailed">
<ows:ExceptionText>
Error executing rasql query: Error evaluating rasdaman query: 'select csv(c[0:3600,0:1800,*:*]) from AvgLandTemp AS c where oid(c)=10753
</ows:ExceptionText>
</ows:Exception>
<ows:Exception exceptionCode="RasdamanRequestFailed">
<ows:ExceptionText>
Error evaluating rasdaman query: 'select csv(c[0:3600,0:1800,*:*]) from AvgLandTemp AS c where oid(c)=10753
</ows:ExceptionText>
</ows:Exception>
<ows:Exception exceptionCode="RuntimeError">
<ows:ExceptionText>
org.odmg.QueryException : Unexpected internal server error.
</ows:ExceptionText>
</ows:Exception>
</ows:ExceptionReport>

comment:9 Changed 17 months ago by mdumitru

@Bang: It's back now, please don't try to retrieve all the data for that coverage in one step. It's around 10GB so it's more than the server can return in one request.

The problem has nothing to do with inserting a slice in the middle. What happens is that the initial InsertCoverage? request that wcst_import makes contains all the coefficients (thus, when an UpdateCoverage? request comes, it has nowhere to put the new slice in). It should contain only the first one.

comment:10 Changed 17 months ago by bphamhuu

@Alex: ah I see idea about the coefficients, that I have to log in by phone to say thanks you.

comment:11 Changed 16 months ago by bphamhuu

  • Resolution set to fixed
  • Status changed from assigned to closed

thanks Alex for the accepted code review, I will submit a patch and close this ticket.

Note: See TracTickets for help on using tickets.