wcst_import extract recipe fails

[Vlad Merticariu]

When executing an import of a coverage from another server, the initial coverage that is inserted contains 1 point, but it contains several coefficients (which indicates several slices), causing the request to fail.

Attached is the initial coverage that is inserted, in GML format.

[Alex Dumitru]

Try to ingest this coverage ​http://flanche.com:9090/rasdaman/ows?SERVICE=WCS&VERSION=2.0.1&REQUEST=DescribeCoverage&coverageId=AvgLandTemp using the wcs extract recipe and see where the error comes from.

[Bang Pham Huu]

ok, thanks for assign to me (a short message to let you know that I will do it now or soon).

[Bang Pham Huu]

Alex or Vlad, now I need to know what am I doing is correct or not, please check
Recipe wcs_extract ​http://pastebin.com/qCwvLnFs

and here is what I got error from your coverage in your host.

Exception thrown when trying to insert slice: 
{Axis Subset: Lat(-90.0,90.0) Long(-180.0,180.0) ansi(145763.0) 
Data Provider: http://flanche.com:9090/rasdaman/ows?service=WCS&version=2.0.1&request=GetCoverage&coverageId=AvgLandTemp&format=image/tiff&subset=Lat(-90.0,90.0)&subset=Long(-180.0,180.0)&subset=ansi(145763.0)}
Retrying, you can safely ignore the warning for now. Tried 1 times.

Progress: [------------------------------] 0/185 0.00% 
Failed to insert slice. Attempted 1 times.
Service Call: http://localhost:8080/rasdaman/ows?service=WCS&version=2.0.1&request=UpdateCoverage&coverageId=alex&subset1=Lat(-90.0,90.0)&inputCoverageRef=file:///tmp/30aaaa1e_5b3c_48fc_8844_3ddf18203ee0.gml&subset3=ansi(145763.0)&subset2=Long(-180.0,180.0)
Error Code: OperationNotSupported
Error Text: <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ows:ExceptionReport version="2.0.0"
    xsd:schemaLocation="http://www.opengis.net/ows/2.0 http://schemas.opengis.net/ows/2.0/owsExceptionReport.xsd"
    xmlns:ows="http://www.opengis.net/ows/2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink">
    <ows:Exception exceptionCode="OperationNotSupported">
        <ows:ExceptionText>Can not add new slice in between existing slices on irregular axis. Only adding slices on top is currently supported.</ows:ExceptionText>
    </ows:Exception>

</ows:ExceptionReport>

If you see it is correct then I can debug WCST-Import to see more detail. Thanks.

[Vlad Merticariu]

All seems fine, except for the tiling parameter in the ingredient, it should probably be something 3D, i.e.:

ALIGNED [0:500, 0:500, 0]

The fix consists in making sure that in the json file generated for the first slice (the one that I attached), <gmlrgrid:coefficients> contains only 1 element (0), as the coverage contains only 1 point at that moment.

[Alex Dumitru]

Vlad meant GML file. You should check in util/coverage_reader.py why the slices have more than one coefficient generated. Normally each slice should only have one coefficient.

[Bang Pham Huu]

Ok, thanks for both of you, and Alex's translation from Vlad's meaning (I have no idea with "point"). I have not debug yet, but go with some information from wcs_extract and DescribeCoverage in GML. Please also check, I will keep it short:

+ wcs_extract recipe to get separately each layer of coverage (for example: 3D like AvgLandTemp) by downloading it to /tmp then make a GML for it and import it to local database?

+ coefficient of this coverage is "ansi time"? As I see in "DescribeCoverage" and gmlrgrid:coefficients (this coverage has 185 slices) and coefficients from 0:5599 (irregular time series).

I would like to confirm as the error " new slice in between existing slices on irregular axis " not looks like what I think now (extract slice from lowest layer and import respectively).

[Bang Pham Huu]

@Alex, it is not related to this ticket but sometime I see error from your hosting, so I can not debug now (before it works and it can download tiff file), I will try to investigate more detail soon.

<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows/2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" version="2.0.0" xsd:schemaLocation="http://www.opengis.net/ows/2.0 http://schemas.opengis.net/ows/2.0/owsExceptionReport.xsd">
<ows:Exception exceptionCode="RasdamanRequestFailed">
<ows:ExceptionText>
Error executing rasql query: Error evaluating rasdaman query: 'select csv(c[0:3600,0:1800,*:*]) from AvgLandTemp AS c where oid(c)=10753
</ows:ExceptionText>
</ows:Exception>
<ows:Exception exceptionCode="RasdamanRequestFailed">
<ows:ExceptionText>
Error evaluating rasdaman query: 'select csv(c[0:3600,0:1800,*:*]) from AvgLandTemp AS c where oid(c)=10753
</ows:ExceptionText>
</ows:Exception>
<ows:Exception exceptionCode="RuntimeError">
<ows:ExceptionText>
org.odmg.QueryException : Unexpected internal server error.
</ows:ExceptionText>
</ows:Exception>
</ows:ExceptionReport>

[Alex Dumitru]

@Bang: It's back now, please don't try to retrieve all the data for that coverage in one step. It's around 10GB so it's more than the server can return in one request.

The problem has nothing to do with inserting a slice in the middle. What happens is that the initial InsertCoverage request that wcst_import makes contains all the coefficients (thus, when an UpdateCoverage request comes, it has nowhere to put the new slice in). It should contain only the first one.

[Bang Pham Huu]

@Alex: ah I see idea about the coefficients, that I have to log in by phone to say thanks you.

[Bang Pham Huu]

thanks Alex for the accepted code review, I will submit a patch and close this ticket.