wiki:PetascopeUserGuide

Version 109 (modified by bphamhuu, 5 weeks ago) (diff)

Remove the workaround for SECORE HTTPS

Petascope User guide

The petascope component of rasdaman implements the OGC interface standards WCS 2.0, WCS-T 1.4, WCPS 1.0, WPS 1.0, and WMS 1.1. For this purpose, petascope maintains its additional metadata (such as georeferencing) which is kept in separate relational tables. Note that not all rasdaman raster objects and collections are available through petascope by default; rather, they need to be registered through the petascope administration interface.

Petascope is implemented as a war file of servlets which give access to coverages (in the OGC sense) stored in rasdaman. Internally, incoming requests requiring coverage evaluation are translated into rasql queries by petascope. These queries are passed on to rasdaman, which constitutes the central workhorse. Results returned from rasdaman are forwarded to the client, finally.

Petascope servlet endpoints

Once the petascope servlet is deployed (see installation guide]), the following service endpoints are available:

  • /rasdaman : context path
    • ./ows : serving OGC Web Services (OWS) like WCS, WCPS and WCS-T:
      • ./wcs' * ./wcps (deprecated; use rasdaman/ows?SERVICE=WCS&REQUEST=ProcessCoverages&... instead)
      • ./wms
    • ./rasql : direct RasQL.

For example, assuming that the service's IP address is 123.456.789.1 and the service port is 8080, the following request URLs would deliver the Capabilities documents for OGC WMS and WCS, respectively:

http://123.456.789.1:8080/rasdaman/ows?SERVICE=WMS&REQUEST=GetCapabilities
http://123.456.789.1:8080/rasdaman/ows?SERVICE=WCS&REQUEST=GetCapabilities

Coordinate Reference System (CRS) handling

Petascope relies on a SECORE Coordinate Reference System (CRS) resolver that can provide proper metadata on, indeed, coverage's native CRSs. One could either deploy a local SECORE instance, or use the official OGC SECORE resolver (http://www.opengis.net/def/crs/). CRS resources are identified then by HTTP URIs, following the related OGC policy document of 2011, based on the White Paper OGC Identifiers – the case for http URIs. These HTTP URIs must resolve to GML resources that describe the CRS, such as http://rasdaman.org:8080/def/crs/EPSG/0/27700 that themselves contain only resolvable HTTP URIs pointing to additional definitions within the CRS; so for example http://www.epsg-registry.org/export.htm?gml=urn:ogc:def:crs:EPSG::27700 is not allowed because, though it is a resolvable HTTP URI pointing at a GML resource that describes the CRS, internally it uses URNs which SECORE is unable to resolve.

Related Pages

  • For the installation, a .war file needs to be deployed and configured. See the installation section for details.
  • For a developer introduction to petascope and its metadata database, see the presentation (v8.x specific, #715) and the related developer guide.
  • For the interpretation and usage of subsets in petascope, see this page.
  • For the handling of time series see this page.
  • For insights on dimensionless index coordinate reference systems see this page.

Caches

Petascope currently keeps a few internal caches, especially for SECORE CRS resources and responses: the gain is both on performance and on robustness against resolver's or network problems. Caching information about CRSs is safe: CRSs can be considered static resources, they might not change for years, or never.

It should be noted that even in case of a locally deployed resolver, still coverages can be configured to a CRS whose definition is provided by an external resolver.

Thanks to caches the network traffic for elaborate a request goes down to zero, so the performance gain is relevant: indeed you will observe slow responses on a newly deployed Petascope, but when caches start to get filled then response times get much shorter. It is suggested to run a WCS GetCapabilities after a fresh new deployment, so that the CRS definitions of all the offered coverages are cached: after that single request, mainly almost all the CRS-related information has already been cached.

Static tables

A subset of the tables in the metadata database (petascopedb) is read just once at deployment, stored in memory and never read again. When changing these tables, Petascope will be out of sync and has to be redeployed so that these static tables are read again.

The static tables either do not have coverage scope or are catalog-tables. Complete list:

  • ps_service_identification
  • ps_service_provider
  • ps_extra_metadata_type
  • ps_gml_subtype
  • ps_format
  • ps_mime_type
  • ps_gdal_format
  • ps_range_data_type

Refer to the dev guide for more insight on petascopedb tables.

As done with the WMS implementation, a ReloadCapabilities request (a hack, not a standard request) could be implemented to refresh Petascope WCS and WCPS metadata without restarting the web server or redeploying Petascope.

The world of coverages: WCS and WCPS services

Offset vectors and coefficients

In ISO and OGC specifications, a coverage is defined as a function from a spatial and/or temporal domain to an attribute range. Such domain can represent both vector or raster datasets, including a variety of different topologies like images, point clouds, polygons, triangulated irregular networks, etc. Coverages in the Earth Observation (EO) domain usually tend to model coverages as a geometric grid of points, a.k.a. grid coverages.

Grid coverages are a network of points connected by lines, retaining a gridded structure. Their geometric domain can either be expressed in an analytic form ((geo)rectified grids), or they need non-affine transforms to translate the internal indexed grid to the external (geo) Coordinate Reference System (CRS). Grid coverages inherently have a dimensionality, which is determined by the number of its axis. These axes are not to be confused with the axes of the CRS which defines dimensionality of the tuples of coordinates of each grid point. Indeed the dimensionality of a grid is not necessarily equal to the dimensionality of its CRS (its however surely not greater for geometric constraints): an (oblique?) 2D grid can always be put inside a 3D CRS, for instance.

Petascope currently supports grid topologies whose axes are aligned with the axes of the CRS. Such kind of grids are a subset of GML rectified grids, with the constraint that the offset vectors — the vectors which determine the (fixed) relative geometric distance between grid points along a grid axis — need to be parallel to an axis of the (external) CRS. In such cases, an offset vector can be regarded as resolution of the grid along an axis.

Rectified grids with non-aligned grid axis / offset vectors are not (yet) supported.

          ======= ALIGNED GRID =======             ===== NON-ALIGNED GRID =====

          N ^    ^ 1                               N ^           / 1
            |    |                                   |          .
            |    .-----.-----.                       |         / \.
            |    |     |     |                       |        /  /\
            |    .-----.-----.                       |       .  /  .
            |    |     |     |                       |      / \.  /
            |    ^-----.-----.                       |     /  /\ /
            | v1 |     |     |                 GO(N) x- - @  /  .
      GO(N) x- - @---->.-----.---> 0                 |    |\.  /  
            |  GO| v0                                |    | \./
         CO O____x________________> E             CO O____x__\__________> E
               GO(E)                                    GO(E) \
                                                               0

     @ = Grid Origin (GO)          O = CRS Origin (CO)
     . = grid point              E,N = CRS axis labels
   0,1 = grid axis labels        --> = offset vector (v0,v1)

In addition (starting from version 9.0.0) Petascope supports aligned grids with arbitrary (irregular) spacing between points along one or more (or all) grid axes. This kind of geometry is a subtype of (geo)referenceable grids and can be defined by attaching a set of coefficients (or weights) to an offset vector. The series of coefficients determines how many offset vectors is a grid point geometrically distant from the grid origin, and their cardinality must coincide with the cardinality of the grid points along that axis. Rectified grids (conceivable as a subset of referenceable grids in an Euler diagram) have an inherent series of incremental integer coefficients attached to each offset vector, so that e.g. the third point along axis 0 is computed as [GO + 2*v0] (indexes start from 0).

A graphical example:

          ======= IRREGULAR ALIGNED GRID =======              =========== WARPED GRID ===========

          N ^    ^ 1                                          N ^           1
            |    |                                              |         /
            |    .------.--.------------.                       |        .-----.----.
            |    |      |  |            |                       |       /      |     \
            |    .------.--.------------. P                     |      .------.-------.
            |    |      |  |            |                       |     /      /       /
            |    ^------.--.------------.                       |    .------.-------.
            | v1 |      |  |            |                       |    |      |      /
      GO(N) x- - @----->.--.------------.---> 0           GO(N) x- - @------.-----.-----> 0
            |  GO|  v0                                          |  GO|
         CO O____x___________________________> E             CO O____x___________________________> E
               GO(E)

     @ = Grid Origin (GO)          O = CRS Origin (CO)
     . = grid point (e.g. P)     E,N = CRS axis labels
   0,1 = grid axis labels        --> = offset vector (v0,v1)

In this example, the grid is still aligned with CRS axes E/N, but the spacing is irregular along grid axis 0. We then need to explicitly define a series of 4 coefficients (one for each grid point along 0) that weight their distance to the grid origin (in terms of v0): in our case the weights are c0={0, 1, 1.5, 3.5}. Indeed the point P in the graphical example above — which has internal (rasdaman) grid coordinates {3,2} (origin is {0,0}) — can hence be geometrically expressed as : (GO + c0[3]*v0 + 2*v1) = (GO + 3.5*v0 + 2*v1).

It is underlined that the irregular spacing must be fixed for each grid line along a certain grid axis. If not so, the referenceable grid becomes warped and the domain needs to be addressed with explicit CRS coordinates for each single grid point (look-up tables).

Grid axis labels and CRS axis labels

Now that the difference between a grid axis and a CRS axis has been cleared, we address the issue of determining (and customizing) the axis labels a coverage in Petascope.

When importing a coverage, a spatio-temporal CRS needs to be assigned to it, in order to give a meaning to its domain. Composition of CRSs is possible via the OGC SECORE CRS resolver. For instance a time-series of WGS84 images can have the following native CRS: http://<secore-resolverX-domain>/def/crs-compound?1=http://<secore-resolverY-domain>/def/crs/EPSG/0/4326&2=http://<secore-resolverZ-domain>/def/crs/<AUTH>/<VERSION>/<CODE-OF-A-TIME-CRS>

Note: currently gml:CompoundCRS is not supported (#679) so, for example, http://www.opengis.net/def/crs/EPSG/0/7415 would have to be represented by composing its components using the same format as above i.e. http://<secore-resolverX-domain>/def/crs-compound?1=http://www.opengis.net/def/crs/EPSG/0/28992&2=http://www.opengis.net/def/crs/EPSG/0/5709

In order to verify the CRS assigned to a coverage offered by Petascope, there are several ways: i) check the wcs:CoverageSummary/ows:BoundingBox@crs attribute in a WCS GetCapabilities response; ii) check the @srsName attribute in the @{gml:SRSReferenceGroup} attributes group in WCS DescribeCoverage response (gml:domainSet); iii) use the WCPS function crsSet(<coverage_name>); iv) directly check the metadata database: petascopedb=# SELECT * FROM getCrs('<coverage_name>').

It is important to understand that the assigned CRS automatically determines the CRS axis labels (and all other axis semantics like direction and unit of measure), and these are the same labels targeted in the subsets of the WCS and WCPS requests. Such labels correspond to the gml:axisAbbrev elements in the CRS definition (mind that ellipsoidal Coordinate Systems (CS) do not count in case of projected CRSs, which build a further CS on top of it).

This excerpt from the CRS definition of the WGS84 / UTM zone 33N projection shows how the first axis defined by this CRS is the easting, with label E and metres [m] as Unit of Measure (UoM, see gml:CoordinateSystemAxis@uom link):

<gml:CartesianCS>
[...]
  <gml:axis>
    <gml:CoordinateSystemAxis gml:id="epsg-axis-1" uom="http://www.opengis.net/def/uom/EPSG/0/9001">
      <gml:descriptionReference xlink:href="http://www.opengis.net/def/axis-name/EPSG/0/9906"/>
      <gml:identifier codeSpace="OGP">http://www.opengis.net/def/axis/EPSG/0/1</gml:identifier>
      <gml:axisAbbrev>E</gml:axisAbbrev>
      <gml:axisDirection codeSpace="EPSG">east</gml:axisDirection>
    </gml:CoordinateSystemAxis>
  </gml:axis>
[...]
</gml:CartesianCS>

Since only aligned grids are supported, we decided to assign the same CRS axes labels to the grid axes. Such labels are listed in the gml:domainSet/gml:axisLabels element of a WCS coverage description, and are not to be confused with the labels of the CRS axes, which are instead listed in the @{gml:SRSReferenceGroup} attributes group, as said.

Indeed, despite the labels of grid and CRS axes will be the same, their order can actually differ. Many geographic CRSs (like the well-known WGS84 / EPSG:4326) define latitudes first, whereas it is GIS practice to always place longitudes in the first place, just like rasdaman does when storing the multidimensional-arrays (marrays).

With regards to this long-standing issue, Petascope strictly keeps the CRS axis order which is defined in its definition when it comes to GML, whereas GIS order (longitude first) is kept for other binary encodings like GeoTiff or NetCDF, so to keep metadata consistency with common GIS libraries (e.g. GDAL). On the other hand, the order of grid axis labels need to follow the internal grid topology of marrays inside rasdaman.

To make things clearer, an excerpt of the GML domain of our 3D systemtest coverage eobstest (regular time series of EO imagery) is proposed:

Axes types in a grid domainSet.

The CRS of the coverage is an (ordered) composition of a temporal CRS (linear count of days [d] from the epoch 1950-01-01T00:00:00) and a geospatial CRS where latitude is defined first (the well-known EPSG:4326). This means that every tuple of spatio-temporal coordinates in the coverage's domain will be a 3D tuple listing the count of days from 1st of January 1950, then latitude degrees then longitude degrees, like shown in the gml:origin/gml:pos element: the origin of the 3D grid is set on 1st of January 1950, 75.5 degrees north and 25 degrees east (with respect to the origin of the cartesian CS defined in EPSG:4326).

Grid coordinates follow instead the internal grid space, which is not aware of any spatio-temporal attribute, and follows the order of axis as they are stored in rasdaman: in the example, it is expressed that the collection is composed of a 6x101x232 marray, having t (time) as first axis, then Long then Lat. The spatio-temporal coordinates are instead expressed following the order of the CRS definition, hence with latitude degrees before longitudes.

A final remark goes to the customization of CRS (and consequently grid) axes labels, which can be particularly needed for temporal CRSs, especially in case of multiple time axis in the same CRS. Concrete CRS definitions are a static XML tree of GML elements defining axis, geographic coordinate systems, datums, and so on. The candidate standard OGC CRS Name-Type Specification offers a new kind of CRS, a parametrized CRS, which can be bound to a concrete definition, a CRS template, and which offers customization of one or more GML elements directly via key-value pairs in the query component of HTTP URL identifying the CRS.

As a practical example, we propose the complete XML definition of the parametrized CRS defining ANSI dates, identified by the URI http://rasdaman.org:8080/def/crs/OGC/0/AnsiDate:

<ParameterizedCRS xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.opengis.net/CRS-NTS/1.0" xmlns:epsg="urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset" xmlns:rim="urn:oasis:names:tc:ebxml-regrep:xsd:rim:3.0" gml:id="param-ansi-date-crs">
  <description>Parametrized temporal CRS of days elapsed from 1-Jan-1601 (00h00 UTC).</description>
  <gml:identifier codeSpace="http://www.ietf.org/rfc/rfc3986">http://rasdaman.org:8080/def/crs/OGC/0/AnsiDate</gml:identifier>
  <parameters>
    <parameter name="axis-label">
      <value>"ansi"</value>
      <target>//gml:CoordinateSystemAxis/gml:axisAbbrev</target>
    </parameter>
  </parameters>
  <targetCRS xlink:href="http://rasdaman.org:8080/def/crs/OGC/0/.AnsiDate-template"/>
</ParameterizedCRS>

This single-parameter definition allow the customization of the concrete CRS template OGC:.AnsiDate-template (identified by http://rasdaman.org:8080/def/crs/OGC/0/.AnsiDate-template) on its unique axis label (crsnts:parameter/crsnts:target), via a parameter labeled axis-label, and default value ansi.

This way, when we assign this parametrized CRS to a coverage, we can either leave the default ansi label to the time axis, or change it to some other value by setting the parameter in the URL query:

0-dimensional grids

What happens when all the CRS dimensions get sliced in a WCS request? How to encode the GMLCOV description?

The dimension of the CRS (and in our case also of the aligned grids) gets decreased by 1 at every slicing, but at the same time the domain set of a coverage cannot have 0 dimensionality: the //gml:domainSet/@dimension attribute shall be greater or equal than 1.

The two solutions that came to our minds are:

  1. leave empty all the elements and attributes inside of the domain set, while just keeping a dimension attribute equal to 0 (see #281);
  2. leave a completely blank domain set : <gml:domainSet/>.

While there is no clear recommendation from OGC, we have decided to opt for the latter, as both being more elegant and legal by XSD.

As an example, you can see the oracle of our WCS test for 0-dimensional output rectified grids: browser:systemtest/testcases_services/test_wcs/oracle/18-get_coverage_0D.oracle .

OGC Web Services

WCS

"The OpenGIS Web Coverage Service Interface Standard (WCS) defines a standard interface and operations that enables interoperable access to geospatial coverages." (WCS standards)

Petascope supports WCS 2.0; see this demo.

You can use either http://your.server/rasdaman/ows/wcs or http://your.server/rasdaman/ows/wcs2 as service endpoints to which to send WCS requests (although the latter is going to be deprecated, see #307).

NOTE: An user oriented "howto" guide on publishing a coverage is not yet available for the upcoming 9.0, you might want to refer to the test dataset ingestion script to get an idea on how to load georeferenced data. Once rasimport gets updated (#169) an howto based on it can be easily produced.

Metadata

GMLCOV and OWS extra annotations

Additional "extra" metadata for a coverage "myCoverage" can be manually inserted in the metadata database. This can (currently) either be:

  • GMLCOV metadata to appear in the coverage description (DescribeCoverage response)
  • OWS metadata to appear in the coverage summary (GetCapabilities response)

To do so, you need to properly fill the petascopedb:ps_extra_metadata table (in petascopedb:ps_extra_metadata_type you can find the extensible dictionary of supported "extra" metadata types):

INSERT INTO ps_extra_metadata (coverage_id, metadata_type_id, value) VALUES (
               (SELECT id FROM ps_coverage WHERE name = 'myCoverage'), 
               (SELECT id FROM ps_extra_metadata_type WHERE type='gmlcov'),
               'myGmlcovMetadata'); 
INSERT INTO ps_extra_metadata (coverage_id, metadata_type_id, value) VALUES (
               (SELECT id FROM ps_coverage WHERE name = 'myCoverage'), 
               (SELECT id FROM ps_extra_metadata_type WHERE type='ows'),
               'myOwsMetadata'); 

If you are using the rasgeo, you can also use it to add extra GMLCOV/OWS annotations:

$ rasimport [...] --metadata gmlcov='<put_gmlcov_fragment_here>':ows='<put_ows_fragment_here>' 

After this operation, GMLCOV metadata will show up in GetCoverage/DescribeCoverage responses, while OWS metadata in the coverage summaries of GetCapabilities documents:

<gmlcov:metadata>myGmlcovMetadata</gmlcov:metadata>
<ows:Metadata>myOwsMetadata</ows:Metadata>

Note that, as by definition (http://schemas.opengis.net/gmlcov/1.0/coverage.xsd), the metadata content should make use of the XML extension mechanism, although this does not represent a restriction:

This is a hook for adding any further information to a coverage, such as domain-specific metadata. Recommended use is to use the XML extension mechanism, such as in a WCS extension or Application Profile, to define the desired metadata structure.

In this case, remember to escape those XML predefined entities inside elements' values and attributes' values that could break the whole XML document. Additionally the prefixes should be bound (embed XML namespaces 'xmlsn:<prefix>=<uri>' in your root elements).

As a practical example, if you need to show the following OWS or GMLCOV metadata in your WCS documents then you firstly need to turn it to a valid XML fragment by escaping reserved XML values which are found in //myns:OnlineResource/@xlink:href and //myns:OnlineResource itself:

<myns:myType  xmlns:myns="http://highway.to.hell/myns">
    <myns:OnlineResource 
     xmlns:xlink="http://www.w3.org/1999/xlink" 
     xlink:type="simple" 
---  xlink:href="http://somewhere.org/random/path?key=value&otherKey=otherValue">
+++  xlink:href="http://somewhere.org/random/path?key=value&amp;otherKey=otherValue">
---     "Audentis Fortuna iuvat"
+++     &quot;Audentis Fortuna iuvat&quot;
    </myns:OnlineResource>
</myns:myType>

...and then you can proceed with the INSERTion in the database.

To escape XML, you could use Python (see below) or some online tool.

$ python
>>> from xml.sax.saxutils import escape
>>> escape("\" < > & Hello dudes \'", {'"':'&quot;', "'":'&apos;'})
'&quot; &lt; &gt; &amp; Hello dudes &apos;'

If you want to actively use the attributes of an ows:Metadata element (eg see #807), you may also insert the full element in ps_extra_metadata::value (>= v9.0.4).

OWS metadata can be also disabled by setting the 'enable_ows_metadata' parameter to false in the configuration file of Petascope.

Service related metadata

Many service metadata that are visualized on a GetCapabilities response can be customised in petascopedb. Some of them are handled dynamically by Petascope so to keep integrity with the implemented available features. For what concerns personal information instead, the relevant tables (as described in more detail in the dev guide) are:

  • ps_service_provider : to modify the information in ows:ServiceProvider
  • ps_service_identification : to modify the information in ows:ServiceIdentification
    • ps_description : holding 1+ title/abstract/keywords for the service identification
      • ps_keyword_group/ps_keyword : holding keywords for the service identification (attributes include keyword group type plus codespace, and keyword language)

In some cases, an extra customization of the capabilities document is needed which the database cannot handle, for example to include a server-specific wcs:extension to the service metadata section. It is then possible to manually edit the templates in petascope.wcs2.templates before deploying Petascope (or directly in the web server folders):

  • ServiceIdentification.templ
    Encoding of the common ServiceIdentification section of the GetCapabilities operation response (.xsd).
  • ServiceProvider.templ
    Encoding of the the common ServiceProvider section of the GetCapabilities operation response (.xsd).
  • ServiceMetadata.templ
    Encoding of the information describing the WCS service on hand. Extension elements allow WCS extension standards to define their individual extra service metadata (.xsd).

If you need to add a wcs:extension to your server metadata, then you can manually insert the XML fragment into inside the wcs:ServiceMetadata element of the template ServiceMetadata.templ.

See #366 for a some practical use case.

Feature space description

Metadata regarding the range (feature space) of a coverage "myCoverage" is a fundamental part of a GMLCOV coverage model. Responses to WCS DescribeCoverage and GetCoverage will show such information in the gmlcov:rangeType element, encoded as fields of the OGC SWE data model.

Currently, not all of the vast set of SWE metadata is supported by Petascope (#582). Firstly, only swe:Quantity types are supported: the values in the range of a coverage will hence be decimals representing a continuous quantity. Support for other types like swe:Category for categorical data or swe:Count for discrete counts is left for future development.

As explained in more detail in the dev guide, range type information metadata is stored across several tables in petascopedb: every band (range element) of a coverage is stored in ps_range_type_component table, which encapsulates a SWE quantity and additionally sets the band label, order and data type. Quantities are defined by an independent set of tables (ps_quantity, ps_quantity_interval, ps_interval and ps_uom) and allows for further metadata including descriptive text, units of measure, allowed intervals and significant figures.

The metadata database is filled with so-called primitive quantities (see this file), which will never be dropped from the database (if not manually deleted) and serves as default quantities in case no particular semantic is needed. These primitives define dimensionless quantities (uom = 10^0) associated with many common integer or floating point data types.

For instance, the range type of our systemtest coverage mr, associated with the primitive quantity with unsigned char values is the following:

<gmlcov:rangeType>
  <swe:DataRecord>
    <swe:field name="value">
      <swe:Quantity definition="http://www.opengis.net/def/dataType/OGC/0/unsignedByte">
        <swe:label>unsigned char</swe:label>
        <swe:description>primitive</swe:description>
        <swe:uom code="10^0"/>
        <swe:constraint>
          <swe:AllowedValues>
            <swe:interval>0 255</swe:interval>
          </swe:AllowedValues>
        </swe:constraint>
      </swe:Quantity>
    </swe:field>
  </swe:DataRecord>
</gmlcov:rangeType>

Again, you can exploit the getRangeType() SQL macro to verify the range metadata of your coverage:

petascopedb=# SELECT * FROM getRangeType('mr');
 component order | name  | SWE type |   data type   | UoM  | allowed interval(s) 
-----------------+-------+----------+---------------+------+---------------------
               0 | value | Quantity | unsigned char | 10^0 | (0,255)
(1 row)

There is currently no utility to customize range metadata out of the box, manual SQL is required. With the help of the dev guide and the examples provided by the primitive quantities, customizing range metadata is straightforward. Primitives should not be updated: new quantities should be instead defined when specific metadata is needed. For instance, to assign a new floating-point quantity to the first range component of "myCoverage" with Kelvin (K) as unit of measure, and allowed values in the range {0,400}, run:

petascopedb=# INSERT INTO ps_uom (code) VALUES ('K');
petascopedb=# INSERT INTO ps_quantity (uom_id, label) VALUES ((SELECT id FROM ps_uom WHERE code='K'), 'Kelvin temperatures');
petascopedb=# INSERT INTO ps_interval (min, max) VALUES (0, 400);
petascopedb=# INSERT INTO ps_quantity_interval VALUES 
petascopedb-#  ((SELECT id FROM ps_quantity WHERE label='Kelvin temperatures'),
petascopedb-#   (SELECT id FROM ps_interval WHERE min=0 AND max=400));

Note that a quantity can be associated with multiple allowed intervals, as by SWE specifications.

Declarations of NIL values are also possible (see #476): one or more values representing not available data or which have special meanings can be declared along with related reasons, which are expressed via URIs (see http://www.opengis.net/def/nil/OGC/0/ for official NIL resources provided by OGC).

Current Deviations

  • 0D coverages can be retrieved through a d-dimensional slicing on a d-dimensional coverage. The resulting coverage is disallowed by GML, however petascope delivers it for user convenience. Should that not be desired then a trimming with lower bound = upper bound in all d dimensions delivers a valid coverage with the same value.

WCPS

"The OpenGIS Web Coverage Service Interface Standard (WCS) defines a protocol-independent language for the extraction, processing, and analysis of multi-dimensional gridded coverages representing sensor, image, or statistics data. Services implementing this language provide access to original or derived sets of geospatial coverage information, in forms that are useful for client-side rendering, input into scientific models, and other client applications. Further information about WPCS can be found at the WCPS Service page of the OGC Network. (http://www.opengeospatial.org/standards/wcps)

The WCPS language is independent from any particular request and response encoding, allowing embedding of WCPS into different target service frameworks like WCS and WPS. The following documents are relevant for WCPS; they can be downloaded from www.opengeospatial.org/standards/wcps:

  • OGC 08-068r2: The protocol-independent ("abstract") syntax definition; this is the core document. Document type: IS (Interface Standard.
  • OGC 08-059r3: This document defines the embedding of WCPS into WCS by specifying a concrete protocol which adds an optional ProcessCoverages request type to WCS. Document type: IS (Interface Standard.
  • OGC 09-045: This draft document defines the embedding of WCPS into WPS as an application profile by specifying a concrete subtype of the Execute request type.

There is a demo sandbox and demo video; see also the WCPS manual and tutorial.

The Petascope implementation supports both XML and Abstract Syntax. You can use http://your.server/rasdaman/ows/wcps as service endpoints to which to send WCPS requests.

  • To send a abstract syntax request: An HTTP GET or POST request can be sent. The query string must be sent in a parameter named query which should be URL encoded in the case of a GET request (query=<%-encoded-abstract-wcps-query>).
  • To send a request as a XML file, a multi-part HTTP POST request must be sent. Only one file must be sent. The MIME type has to be text/xml.
  • To send a request as XML: an HTTP POST request must be sent. A post parameter named xml must be attached.

If you intend to implement forms-based access: To obtain a proper query from an XML page a form must be used. The method should be post and action should be an URL pointing to the servlet access point. The form should contain either a file field or a text/hidden field that has the name xml or query. An error in the access method will generate an HTML query input form.

For guidelines on how to safely build and troubleshoot WCPS query with Petascope, see this topic in the mailing-list.

Current Deviations

  • WCPS coverage variables in petascope must not have a $ prefix, in contrast to the WCPS specification.
  • imageCrsDomain can be used in the OVER clause of a coverage constructor (cf. #475) and the scale operation (cf. #387)
  • SWITCH statement. It can be used as in the following example:
    for c in mean_summer_airtemp 
    return 
      encode( 
        switch 
          case c>0 return log(c)
          case c=0 return (char)0
          default return (char)1,
        "csv")
    

WPS

"The OpenGIS Web Processing Service (WPS) Interface Standard provides rules for standardizing how inputs and outputs (requests and responses) for geospatial processing services, such as polygon overlay. The standard also defines how a client can request the execution of a process, and how the output from the process is handled. It defines an interface that facilitates the publishing of geospatial processes and clients’ discovery of and binding to those processes. The data required by the WPS can be delivered across a network or they can be available at the server." (OGC Web Processing Service)

Petascope integrates with the 52°north WPS allowing to submit a WCPS request via WPS and deliver sets of coverages or scalars as results. The WCPS query can be in either XML or abstract syntax.

Deployment

First, the original war file from 52 north needs to be adapted for this to work. This can be done with make wps from the petascope directory:

  • make wps wpswar=/path/to/52n-wps-webapp-2.0.war

The 52n admin WPS page can be accessed then at http://localhost:8080/wps/webAdmin/index.jsp with default user/pass: wps/wps

To integrate a newly deployed custom n52 war with petascope WCPS

  1. go to Upload Process and upload petascope/src/main/resources/n52/ProcessCoverages.java and petascope/src/main/resources/n52/ProcessCoverages.xml
  2. Save configuration
  3. In Algorithm Repositories a property "Algorithm=petascope.wps.n52.ProcessCoverages" for the UploadedAlgorithmRepository needs to be added.
  4. Deactivate the LocalAlgorithmRepository

Internals

Incoming WPS Execute requests invoking the ProcessCoverages process are forwarded by the 52°north to petascope, stripped off the WPS specifics. The resulting WCPS request is processed, and the result is returned to WPS.

Technically, the coupling consists of an adapter class (a Process). The process identifier is petascope.wps.n52.ProcessCoverages.

Demo

See this small earthlook we have set a demo too, available EarthLook WPS demo. There is a simple test client done by n52.

WMS

"The OpenGIS Web Map Service Interface Standard (WMS) provides a simple HTTP interface for requesting geo-registered map images from one or more distributed geospatial databases. A WMS request defines the geographic layer(s) and area of interest to be processed. The response to the request is one or more geo-registered map images (returned as JPEG, PNG, etc) that can be displayed in a browser application. The interface also supports the ability to specify whether the returned images should be transparent so that layers from multiple servers can be combined or not."

Petascope supports WMS 1.0.0/1.1.0. How to publish a WMS layer.

Output formats

WCS formats are requested via the format KVP key (<gml:format> elements for XML POST requests), and take a valid MIME type as value.

Output encoding is passed on to the the GDAL library, so the limitations on output formats are devised accordingly by the supported raster formats of GDAL.

In case of encoded processing expressions, WCPS instead (and RasQL) can accept either GDAL Code format identifiers or other commonly-used format abbreviations like "CSV" for Comma-Separated-Values for instance.

Here we propose a mapping of GDAL codes, MIME types and well-known abbreviations (you may also manually check then in our metadata database):

petascopedb=# SELECT name AS "Format abbrv", mime_type AS "MIME", g.gdal_id AS "GDAL code"
petascopedb-#   FROM ps_format AS f
petascopedb-#   LEFT OUTER JOIN ps_mime_type AS m ON (f.mime_type_id=m.id)
petascopedb-#   LEFT OUTER JOIN ps_gdal_format AS g ON (f.gdal_id=g.id)
petascopedb-# ORDER BY f.name;
  Format abbrv   |               MIME                |    GDAL code    
-----------------+-----------------------------------+-----------------
 aaigrid         | image/x-aaigrid                   | AAIGrid
 ace2            | application/x-ogc-ace2            | ACE2
 adrg            | application/x-ogc-adrg            | ADRG
 aig             | application/x-ogc-aig             | AIG
 airsar          | application/x-ogc-airsar          | AIRSAR
 bag             | application/x-ogc-bag             | BAG
 blx             | application/x-ogc-blx             | BLX
 bmp             | image/bmp                         | BMP
 bsb             | application/x-ogc-bsb             | BSB
 bt              | application/x-ogc-bt              | BT
 ceos            | application/x-ogc-ceos            | CEOS
 coasp           | application/x-ogc-coasp           | COASP
 cosar           | application/x-ogc-cosar           | COSAR
 cpg             | application/x-ogc-cpg             | CPG
 csv             | text/plain                        | 
 ctg             | application/x-ogc-ctg             | CTG
 dimap           | application/x-ogc-dimap           | DIMAP
 dipex           | application/x-ogc-dipex           | DIPEx
 dods            | application/x-ogc-dods            | DODS
 doq1            | application/x-ogc-doq1            | DOQ1
 doq2            | application/x-ogc-doq2            | DOQ2
 dted            | application/x-ogc-dted            | DTED
 e00grid         | application/x-ogc-e00grid         | E00GRID
 ecrgtoc         | application/x-ogc-ecrgtoc         | ECRGTOC
 ecw             | image/x-imagewebserver-ecw        | ECW
 ehdr            | application/x-ogc-ehdr            | EHdr
 eir             | application/x-ogc-eir             | EIR
 elas            | application/x-ogc-elas            | ELAS
 envi            | application/x-ogc-envi            | ENVI
 epsilon         | application/x-ogc-epsilon         | EPSILON
 ers             | application/x-ogc-ers             | ERS
 esat            | application/x-ogc-esat            | ESAT
 fast            | application/x-ogc-fast            | FAST
 fit             | application/x-ogc-fit             | FIT
 fits            | application/x-ogc-fits            | FITS
 fujibas         | application/x-ogc-fujibas         | FujiBAS
 genbin          | application/x-ogc-genbin          | GENBIN
 georaster       | application/x-ogc-georaster       | GEORASTER
 gff             | application/x-ogc-gff             | GFF
 gif             | image/gif                         | GIF
 gmt             | application/x-netcdf-gmt          | GMT
 grass           | application/x-ogc-grass           | GRASS
 grassasciigrid  | application/x-ogc-grass_asciigrid | GRASSASCIIGrid
 grib            | application/x-ogc-grib            | GRIB
 gs7bg           | application/x-ogc-gs7bg           | GS7BG
 gsag            | application/x-ogc-gsag            | GSAG
 gsbg            | application/x-ogc-gsbg            | GSBG
 gsc             | application/x-ogc-gsc             | GSC
 gta             | application/x-ogc-gta             | GTA
 gtiff           | image/tiff                        | GTiff
 gtx             | image/x-gtx                       | GTX
 gxf             | application/x-ogc-gxf             | GXF
 hdf4            | application/x-hdf4                | HDF4
 hdf5            | application/x-hdf5                | HDF5
 hf2             | application/x-ogc-hf2             | HF2
 hfa             | application/x-erdas-hfa           | HFA
 ida             | application/x-ogc-ida             | IDA
 ilwis           | application/x-ogc-ilwis           | ILWIS
 ingr            | application/x-ogc-ingr            | INGR
 isis2           | application/x-ogc-isis2           | ISIS2
 isis3           | application/x-ogc-isis3           | ISIS3
 jaxapalsar      | application/x-ogc-jaxapalsar      | JAXAPALSAR
 jdem            | application/x-ogc-jdem            | JDEM
 jp2ecw          | image/jp2                         | JP2ECW
 jp2kak          | image/jp2                         | JP2KAK
 jp2mrsid        | image/jp2                         | JP2MrSID
 jp2openjpeg     | image/jp2                         | JP2OpenJPEG
 jpeg            | image/jpeg                        | JPEG
 jpeg2000        | image/jp2                         | JPEG2000
 jpegls          | image/jpeg                        | JPEGLS
 jpg             | image/jpeg                        | JPEG
 jpipkak         | image/jpip-stream                 | JPIPKAK
 kmlsuperoverlay | application/x-ogc-kmlsuperoverlay | KMLSUPEROVERLAY
 l1b             | application/x-ogc-l1b             | L1B
 lan             | application/x-erdas-lan           | LAN
 lcp             | application/x-ogc-lcp             | LCP
 leveller        | application/x-ogc-leveller        | Leveller
 loslas          | application/x-ogc-loslas          | LOSLAS
 mbtiles         | application/x-ogc-mbtiles         | MBTiles
 mem             | application/x-ogc-mem             | MEM
 mff             | application/x-ogc-mff             | MFF
 mff2            | application/x-ogc-mff2            | MFF2 (HKV)
 mg4lidar        | application/x-ogc-mg4lidar        | MG4Lidar
 mrsid           | image/x-mrsid                     | MrSID
 msg             | application/x-ogc-msg             | MSG
 msgn            | application/x-ogc-msgn            | MSGN
 ndf             | application/x-ogc-ndf             | NDF
 netcdf          | application/netcdf                | netCDF
 ngsgeoid        | application/x-ogc-ngsgeoid        | NGSGEOID
 nitf            | application/x-ogc-nitf            | NITF
 ntv2            | application/x-ogc-ntv2            | NTv2
 nwt_grc         | application/x-ogc-nwt_grc         | NWT_GRC
 nwt_grd         | application/x-ogc-nwt_grd         | NWT_GRD
 ogdi            | application/x-ogc-ogdi            | OGDI
 ozi             | application/x-ogc-ozi             | OZI
 paux            | application/x-ogc-paux            | PAux
 pcidsk          | application/x-ogc-pcidsk          | PCIDSK
 pcraster        | application/x-ogc-pcraster        | PCRaster
 pdf             | application/x-ogc-pdf             | PDF
 pds             | application/x-ogc-pds             | PDS
 png             | image/png                         | PNG
 pnm             | application/x-ogc-pnm             | PNM
 postgisraster   | application/x-ogc-postgisraster   | PostGISRaster
 r               | text/x-r                          | R
 rasdaman        | application/x-ogc-rasdaman        | RASDAMAN
 rasterlite      | application/x-ogc-rasterlite      | Rasterlite
 raw             | application/x-octet-stream        | 
 rik             | application/x-ogc-rik             | RIK
 rmf             | application/x-ogc-rmf             | RMF
 rpftoc          | application/x-ogc-rpftoc          | RPFTOC
 rs2             | application/x-ogc-rs2             | RS2
 rst             | application/x-ogc-rst             | RST
 saga            | application/x-ogc-saga            | SAGA
 sar_ceos        | application/x-ogc-sar_ceos        | SAR_CEOS
 sde             | application/x-ogc-sde             | SDE
 sdts            | application/x-ogc-sdts            | SDTS
 sgi             | image/x-sgi                       | SGI
 snodas          | application/x-ogc-snodas          | SNODAS
 srp             | application/x-ogc-srp             | SRP
 srtmhgt         | application/x-ogc-srtmhgt         | SRTMHGT
 terragen        | application/x-ogc-terragen        | TERRAGEN
 tif             | image/tiff                        | GTiff
 tiff            | image/tiff                        | GTiff
 til             | application/x-ogc-til             | TIL
 tsx             | application/x-ogc-tsx             | TSX
 usgsdem         | application/x-ogc-usgsdem         | USGSDEM
 vrt             | application/x-ogc-vrt             | VRT
 wcs             | application/x-ogc-wcs             | WCS
 webp            | application/x-ogc-webp            | WEBP
 wms             | application/x-ogc-wms             | WMS
 xpm             | image/xpm                         | XPM
 xyz             | application/x-ogc-xyz             | XYZ
 zmap            | application/x-ogc-zmap            | ZMap
(133 rows)

Limitations

  • building WCPS queries can be tricky due to some unforeseen behaviour of the syntax parser: check out this thread for some guidance.
  • WCS extensions status:
    • Scaling : fully implemented.
    • Range Subsetting : fully implemented.
    • Processing : 90% fully implemented (see #614).
    • Interpolation : implemented (nearest-neighbor).
    • Earth-Observation : not implemented (refer to EOxServer).
    • CRS : not implemented (refer to rasdaman GmbH).
  • Regarding grid coverages, only grids whose lines are rectilinear and aligned with a Cartesian CRS are supported. This means: no rotated nor warped (curvilinear) grids.
  • ...

Tickets reporting deviations from standards

Tickets reporting deviations should be tagged with the keyword "deviation" so they can get listed here and assessed for documentation.

No results

Attachments (1)

Download all attachments as: .zip