FIX - match multiple input files to defined areasOfValidity in a single ingredients file

[Bang Pham Huu]

in areasOfValidity it can contain a list of elements, and wcst_import can match each input file to the corresponding areasOfValidity element, by each slice's min and max see:

"min": "datetime(regex_extract('${file:name}', 'DLT_(....)_.*', 1), 'YYYY')"

and areasOfValidity's start and end.

Example:

{
  "config": {
    "service_url": "http://localhost:8080/rasdaman/ows",
    "tmp_directory": "/tmp/",
    "automated": true,
    "track_files": false,
    "insitu": true,
    "default_null_values": [
      250
    ]
  },
  "input": {
    "coverage_id": "dominant_leaf_type_20m",
    "paths": [
      "/tmp/2012/*.tif",
      "/tmp/2015/*.tif"
    ]
  },
  "recipe": {
    "name": "general_coverage",
    "options": {
      "wms_import": true,
      "coverage": {
        "crs": "OGC/0/AnsiDate?axis-label=\"time\"@EPSG/0/3035",
        "metadata": {
          "type": "xml",
          "global": {
            "title": "Dominant Leaf Type"
          }
        },
        "slicer": {
          "type": "gdal",
          "bands": [
            {
              "identifier": "0",
              "name": "dlt",
              "label": "dominant leaf type map of Europe"
            }
          ],
          "axes": {
            "time": {
              "min": "datetime(regex_extract('${file:name}', 'DLT_(....)_.*', 1), 'YYYY')",
              "irregular": true,
              "dataBound": false,
              "areasOfValidity": [
                {
                  "start": "2011",
                  "end": "2013"
                },
                {
                  "start": "2014",
                  "end": "2016"
                }
              ]
            },
            "X": {
              "min": "${gdal:minX}",
              "max": "${gdal:maxX}",
              "resolution": "${gdal:resolutionX}"
            },
            "Y": {
              "min": "${gdal:minY}",
              "max": "${gdal:maxY}",
              "resolution": "${gdal:resolutionY}"
            }
          }
        }
      },
      "tiling": "ALIGNED [0:0, 0:2047, 0:2047] TILE SIZE 4194304"
    }
  }

[Bang Pham Huu]

Also, wcst_import needs to validate that the datetime of min should be between the start and end of areasOfValidity.

For example, min points to "2018" which is resolved as 2018-01-01T00:00:00.000Z and it does not exist between "2018-03-01T00:00:00.000Z":"2008-12-31T23:59:59.999Z":

          "axes": {
            "time": {
              "min": "datetime(regex_extract('${file:path}', '.*/base/(....)/.*', 1), 'YYYY')",
              "irregular": true,
              "dataBound": false,
              "areasOfValidity": [
                {
                  "start": "2018-03",
                  "end": "2018-10"
                }
              ]
            },

[Bang Pham Huu]

and if min value of X/Y axes are wrong like below

 "X": {
              "min": "2018-03",
              "max": "${gdal:maxX}",
              "resolution": "${gdal:resolutionX}"
            },
            "Y": {
              "min": "2018-03",
              "max": "${gdal:maxY}",
              "resolution": "${gdal:resolutionY}"
            }

then wcst_import should throw proper exception instead

Failed to import recipe at '/home/rasdev/Ingredients/grassland/2018_10m/ingest_base.json'. Reason: [<class 'decimal.ConversionSyntax'>]
Stack trace: Traceback (most recent call last):
  File "/opt/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/abstract_to_coverage_converter.py", line 610, in _create_coverage_slices
    coverage_slice = self._create_coverage_slice(file, crs_axes, self.evaluator_slice, axis_resolutions, user_axes)
  File "/opt/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/abstract_to_coverage_converter.py", line 686, in _create_coverage_slice
    axis_subset = self._axis_subset(crs_axes[i], evaluator_slice, resolution)
  File "/opt/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/gdal_to_coverage_converter.py", line 199, in _axis_subset
    - decimal.Decimal(str(user_axis.interval.low))
decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]

[Peter Baumann]

Ticket retargeted after milestone closed