wiki:RasgeoUserGuide

rasgeo

rasgeo is Alexander Herzig's contribution to the rasdaman FOSS project to ease creation, update and deletion of 2D/3D (geo) raster datasets.

Besides this page, it is recommended to read the rich README file that is available, with synopsis and examples. It is important to underline that rasgeo configuration is independent, and relies on its rasconnect configuration file.

As a separate internal module, rasgeo includes import tools for the OGC Web Map Service (WMS) that Petascope provides.

Note: rasgeo has been deprecated; the recommended way to import geo-referenced data into rasdaman is with wcst_import.

Terms

It is important here to underline the exact meanings that we assign to words, in order to have a clear distinction between the indexed semantics of rasdaman's collections and the geo-semantics of coverages.

marray
a multidimensional array addressed by integer internal coordinates (CRS:1) and composed of cells.
oid
unique identifier (integer number) for an marray.
collection
a bag of marrays with a readable label.
coverage
digital representation of a spatio-temporally varying phenomenon.
grid coverage
a coverage showing an nD topology which retains a grid.

Grid coverages' topology can be easily stored inside the rasdaman Array DBMS as an marray, given that the external metadata is properly handled.

Although a collection can technically contain multiple marrays, a coverage is linked to a single marray via oid identifier; this constraint inhibits illegal aggregation of marrays which would break the geometry of a gridded coverage. Collection names and coverage names need not be the same: a collection can act like a folder of marrays, each one being published as a separate coverage.

Examples

In this section we explain how to use rasgeo by means of examples, for different kinds of datasets. Further use cases can be found in its README file.

rasimport

rasimport is the import/update utility of rasgeo. It handles ingestion/update of both marrays in rasdaman and (geo)metadata in petascopedb: when a coverage is not present yet in the rasdaman/Petascope stack, than a new collection/coverage is created, otherwise subsequent executions to the same coverage will perform update operations.

Such updates can mean a replacement of existing cell values, or the insertion of a new sub-array of data in the nD container (which in this context usually translates to the insertion of a new 2D image in a 3D cube).

Test image: mean_summer_airtemp

Size is 886, 711
Pixel Size = (0.050000000000000,-0.050000000000000)
Upper Left  ( 111.9750000,  -8.9750000) 
Lower Left  ( 111.9750000, -44.5250000) 
Upper Right ( 156.2750000,  -8.9750000) 
Lower Right ( 156.2750000, -44.5250000)

Following subsections will always use this image, optionally stacked over itself to simulate 3D spatial and spatio-temporal cubes. We assume WGS84 latitude/longitude coordinates (EPSG:4326 in the 2D space, and EPSG:4327 in 3D). In some example we also show how to create a coverage without geo-reference, as a normal TIFF: this can be interesting for other applications which have to treat Big Data outside the remote sensing realm.

2D geoimage

Single 2D WGS84 image mean_summer_airtemp.

> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp \
    --coverage-name msat_cov \
    -t GreyImage:GreySet \
    --crs-uri 'http://www.opengis.net/def/crs/EPSG/0/4326' \
    --crs-order 1:0 # Lat Long

GML Coverage description (if Petascope is deployed):

http://<host>[:port]/rasdaman/ows/wcs2?
    service=WCS
   &version=2.0.1
   &request=DescribeCoverage
   &coverageid=msat_cov

3D geocube

2D WGS84 image mean_summer_airtemp at regular layers of altitude (each 200m), starting from 800m: Layer1 [800,1000), Layer2 [1000,1200), Layer3 [1200,1400).

Note that we have geo-voxels with 3D sample size, and the origin of a coverage has to be set in the centre of the sample space as by ISO/OGC recommendation.

'--3D top' argument is necessary to build a 3D cube of data and you must set the correct collection and marray types from the very first rasimport call, so that radsaman can host the cube.

When adding new slices, you can either:

  • insert immediately on top/bottom (--3D option) while still confirming the sample space of your data along the thirs z dimension (--csz)
  • insert it to an absolute z position (--shift): in this case the --3D option, though still necessary, will ignore its values since top or bottom insertion (contiguous or not) will depend on the z coordinate of the shift (empty layers are automatically added to the marray in case the slice is not contiguous with current bounds).

Min that in the latter case, also --csz is still necessary since geo-z coordinates need to be translated to rasdaman indexes. See example below:

> # first layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_3D \
    --coverage-name msat_cov_3D \
    -t GreyCube:GreySet3 \
    --crs-uri 'http://www.opengis.net/def/crs/EPSG/0/4327' \
    --crs-order 1:0:2 \
    --3D top \
    --csz 200 \
    --shift 0:0:800     # bottom border --> origin_z = 800+(200/2) = 900 [m]
> # second layer on top
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_3D \
    --coverage-name msat_cov_3D \
    -t GreyCube:GreySet3 \
    --crs-uri 'http://www.opengis.net/def/crs/EPSG/0/4327' \
    --crs-order 1:0:2 \
    --3D top \
    --csz 200 \
    --shift 0:0:1000 # not necessary: this is a contiguous slice
> # third layer on top
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_3D \
    --coverage-name msat_cov_3D \
    -t GreyCube:GreySet3 \
    --crs-uri 'http://www.opengis.net/def/crs/EPSG/0/4327' \
    --crs-order 1:0:2 \
    --3D top \
    --csz 200 \
    --shift 0:0:1200     # not necessary: this is a contiguous slice

Try to add a layer Layer0 before the origin (bottom), for heights [600,800):

> # fourth layer on bottom
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_3D \
    --coverage-name msat_cov_3D \
    -t GreyCube:GreySet3 \
    --crs-uri 'http://www.opengis.net/def/crs/EPSG/0/4327' \
    --crs-order 1:0:2 \
    --3D bottom \
    --csz 200 \
    --shift 0:0:600  # [[ ORIGIN + (-1^{bottom})*(shift) ]]_z = 800 + (-200) = 600
                     # (not necessary: this is a contiguous slice)

GML Coverage description (if Petascope is deployed):

http://<host>[:port]/rasdaman/ows/wcs2?
    service=WCS
   &version=2.0.1
   &request=DescribeCoverage
   &coverageid=msat_cov_3D

3D indexed cube

3D cube of images referenced by an Index CRS. This means that the coordinates are integers, and the sample size of each voxel is a 0D point. Origin is set to {0,0,0} and each grid point has unity distance to the contiguous grid points.

> # first layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_Index3D \
    --coverage-name msat_cov_Index3D \
    -t GreyCube:GreySet3 \
    --3d top
> # second layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_Index3D \
    --coverage-name msat_cov_Index3D \
    -t GreyCube:GreySet3 \
    --3D top
> # third layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_Index3D \
    --coverage-name msat_cov_Index3D \
    -t GreyCube:GreySet3 \
    --3D top

GML Coverage description (if Petascope is deployed):

http://<host>[:port]/rasdaman/ows/wcs2?
    service=WCS
   &version=2.0.1
   &request=DescribeCoverage
   &coverageid=msat_cov_Index3D

Regular time-series

Time series of weekly aggregate regular images starting from the 1st of February 2014 (time is the third z dimension):

Here we use they '%SECORE_URL%' parameter to let Petascope replace it with the configure SECORE resolver.

> # first layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_rts \
    --coverage-name msat_cov_rts \
    -t GreyCube:GreySet3 \
    --crs-uri '%SECORE_URL%/crs/EPSG/0/4326':'%SECORE_URL%/crs/OGC/0/AnsiDate' \
    --crs-order 1:0:2 \
    --3D top \
    --csz 7 \
    --shift 0:0:150877
> # second layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_rts \
    --coverage-name msat_cov_rts \
    -t GreyCube:GreySet3 \
    --crs-uri '%SECORE_URL%/crs/EPSG/0/4326':'%SECORE_URL%/crs/OGC/0/AnsiDate' \
    --crs-order 1:0:2 \
    --3D top \
    --csz 7
> # third layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_rts \
    --coverage-name msat_cov_rts \
    -t GreyCube:GreySet3 \
    --crs-uri '%SECORE_URL%/crs/EPSG/0/4326':'%SECORE_URL%/crs/OGC/0/AnsiDate' \
    --crs-order 1:0:2 \
    --3D top \
    --csz 7

GML Coverage description (if Petascope is deployed):

http://<host>[:port]/rasdaman/ows/wcs2?
    service=WCS
   &version=2.0.1
   &request=DescribeCoverage
   &coverageid=msat_cov_rts

To understand how Petascope handles time, see this page.

Irregular time-series

Time series of three regular images on the 1st, 3rd and 7th of February 2014. In this example we also decide to add some GMLCOV annotation to our coverage.

> # first layer + add GMLCOV metadata
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_irts \
    --coverage-name msat_cov_irts \
    -t GreyCube:GreySet3 \
    --crs-uri '%SECORE_URL%/crs/EPSG/0/4326':'%SECORE_URL%/crs/OGC/0/AnsiDate' \
    --crs-order 1:0:2 \
    --csz 1 \
    --z-coords 150877 \
    --metadata gmlcov='<myGmlcovXmlFragmentHere>'
> # second layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_irts \
    --coverage-name msat_cov_irts \
    -t GreyCube:GreySet3 \
    --crs-uri '%SECORE_URL%/crs/EPSG/0/4326':'%SECORE_URL%/crs/OGC/0/AnsiDate' \
    --crs-order 1:0:2 \
    --csz 1 \
    --z-coords 150879
> # third layer
> rasimport \
    -f $HOME/rasdaman/systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif \
    --coll mean_summer_airtemp_irts \
    --coverage-name msat_cov_irts \
    -t GreyCube:GreySet3 \
    --crs-uri '%SECORE_URL%/crs/EPSG/0/4326':'%SECORE_URL%/crs/OGC/0/AnsiDate' \
    --crs-order 1:0:2 \
    --csz 1 \
    --z-coords 150883

GML Coverage description (if Petascope is deployed):

http://<host>[:port]/rasdaman/ows/wcs2?
    service=WCS
   &version=2.0.1
   &request=DescribeCoverage
   &coverageid=msat_cov_irts

To understand how Petascope handles time, see this page.

raserase

raserase is the utility for dropping collections and coverages. There are multiple options for this tool too:

Usage: raserase {--coll <collection> [--oid <OID>] | --coverage <coverage_name>}
                [--conn <connection file>]
                [--ps-metadata]

 --coll        name of rasdaman collection to delete
 --conn        connection file specifying rasdaman and postgres DB connection parameters
 --coverage    delete rasdaman image which is exposed as 'coverage_name' by petascope web services
 --oid         local object identifier (OID) of image to delete
 --ps-metadata delete only petascope metadata, which is associated with 
               the specified coverage (no image(s) are deleted!)

This means that a coverage can be dropped by referring to its coverage label, collection label or OID. Moreover, in case only the geosemantics are to be deleted, --ps-metadata will keep the collection in rasdaman while dropping the coverage semantics (Petascope will lose the coverage, but not rasdaman)

For instance, the 2D example we proposed:

$ raserase --coverage msat_cov                     [--ps-metadata]
$ raserase --coll mean_summer_airtemp              [--ps-metadata]
$ raserase --oid <OID(mean_summer_airtemp/marray)> [--ps-metadata]
Last modified 15 months ago Last modified on Dec 22, 2015 3:03:49 PM