Pre/post-processing hooks in wcst_import

[Dimitar Misev]

Sometimes it is useful to run a script or program before or after ingesting each file. This would allow for preprocessing, e.g. doing gdalwarp automatically, and cleanup afterwards, helping to more fully automate the ingestion process.

Before executing the hook:

• wcst_import can provide the path to the file being processed. In the hook call this needs to be specified with ${file:path}
• in general any expressions as listed in the docs can be used: ​http://doc.rasdaman.org/05_geo-services-guide.html#possible-expressions

After executing the hook:

• stdout, stderr, and exit code are captured internally by wcst_import and logged
• Options only valid for pre-hooks:
  • abort_on_error: abort the ingestion if exit code != 0
  • replace_path: wcst_import will consider the specified path(s) as the actual file(s) to be ingested after running the hook, rather than the original file. This is an array of paths where globbing is allowed (same as the "input":"paths" option).

Exemplified syntax in ingredients files:

"hooks": [
  {
    "description": "reproject input files.",
    "when": "before_ingestion",
    "cmd": "gdalwarp ... ${file:path} ${file:path}.projected",
    "abort_on_error": true,
    "replace_path": ["${file:path}.projected"]
  },
  {
    "description": "remove generated files.",
    "when": "after_ingestion",
    "cmd": "rm -f ${file:path}.projected"
  },
]

Note: this ticket depends on #1897, the current model would not allow for general pre/post-processing of each file