wiki:WorldWindChallenge/2014

Version 1 (modified by pcampalani, 3 years ago) (diff)

first draft

NASA WORLD WIND Europa Challenge 2014 : Cross-coverage atmosphere data analytics with the WCPS query language

In short

This demo will experiment with support and performance of pixel data fusion done at server levels, leveraging the OGC Web Coverage Service (WCS) processing extension and its relevant operators --- a.k.a. the Web Coverage Processing Service (WCPS) grammar.

We investigate by implementing examples of well-known functions and parameters that are extracted from datasets for environmental analysis (aerosol optical properties). The datasets used in the experiments are gridded rectified products, which are best suited for the open source implementation of WCPS used.

All information shown during the demo is entirely retrieved from a WCPS server. Our chosen implementation is provided by the open-source rasdaman array DBMS (http://rasdaman.org). WCPS service endpoint is provided by MEEO Srl (http://www.meeo.it/).

In depth

The new paradigm introduced by this deluge of data that is hitting us every day from satellite and air-borne sensors is changing the workflow of environmental data analysts and modellers. Web geo-services play now a fundamental role, and no longer they are needed to preliminary offline data download and storage but rather they interact in real-time with GIS applications.

Due to the very Big amount of data that is curated and made available by web services, it is crucial to deploy smart solutions for i) optimizing network bandwidth by enabling download of high-entropy data to the user ii) reduce duplication of data that can be otherwise efficiently computed on-the-fly by the server, moving the processing to the data.

Our WW demonstrations acts as a showcase for the OGC Web Coverage Service potential: what can a client get from a WCS server today? The most exciting aspect of WCS is represented by its processing extension, a.k.a. the OGC Web Coverage Processing Service (WCPS) standard. A WCPS-compliant service allows a client to attach a processing query: by exploiting a full grammar for multidimensional-array algebra, several different kinds of information can be retrieved from one or more datasets together: scalar condensers, cross-sectional profiles, comparison maps and plots, etc.

The purpose of our demo is to evaluate and show the application of WCPS queries for analysis and cross-comparison of atmospheric datasets with different spatial and temporal resolutions. Our data provider at MEEO Srl offers maps of columnar Aerosol Optical Thickness (AOT -- 550 nm channel) from different sources, namely the MODIS radiometer, ECMWF forecasts and ESA CCI products. The characteristics of each time-series of AOT data are presented in the next section.

Datasets

MOD08
The MOD08 (http://modis.gsfc.nasa.gov/data/dataprod/dataproducts.php?MOD_NUMBER=08) are MODIS Level 3 Atmosphere Products, each covering a different temporal scale: Daily, 8-Day, and Monthly. Each of these Level 3 products contains statistics derived from over 100 science parameters from the Level 2 Atmosphere products: Aerosol, Precipitable Water, Cloud, and Atmospheric Profiles. Statistics are aggregated to a 1° × 1° equal-angle global grid. See also: http://modis.gsfc.nasa.gov/data/dataprod/pdf/MOD_08.pdf MOD08
  • spatial resolution: 1x1 degrees2
  • temporal resolution: 1 day (24h sample space too)
  • # of bands: 1 (MEAN/MIN/MAX bands are extracted to separate 1-band coverages)
  • data type: float (NODATA: -9999=>inversion_fail; 0=>missing)
  • spacetime bbox: year 2008; world-wide
MACC
(http://apps.ecmwf.int/datasets/data/macc_reanalysis/) The European Centre for Medium-Range Weather Forecasts (ECMWF) develops and operates global models and data-assimilation systems for the dynamics, thermodynamics and composition of the Earth's fluid envelope and interacting parts of the Earth-system. Aerosols from MACC Reanalysis (http://apps.ecmwf.int/datasets/data/macc_reanalysis/) consists of global maps with 1.125° x 1.125° spatial resolution and 3-hourly temporal frequency. See also http://www.gmes-atmosphere.eu/services/aqac/
  • spatial resolution: 1.125x1.125 degrees2
  • temporal resolution: 3 hours (3h sample space too)
  • # of bands: 1
  • data type: float (NODATA: 0=>missing)
  • spacetime bbox: 2007-Dec-12 to 2009-Jan-05; world-wide
ESA CCI
The primary products concerned in the aerosol_cci project are level 2 (daily 10km and 50km pixel products) and level 3 (aggregated monthly gridded datasets) multi-spectral AOD and associated probabilities of predefined aerosol types for a number of European satellite instruments (ATSR-2, AATSR, MERIS, POLDER, GOME, SCIAMACHY, OMI, GOME-2, AVHRR/3); stratospheric aerosols are observed with GOMOS (and tested for SCIAMACHY). Freely available ESA CCI Aerosol products (http://www.esa-aerosol-cci.org/?q=node/148) are available.
  • spatial resolution: 1x1 degrees2
  • temporal resolution: 1 day (24h sample space too)
  • # of bands: 1 (MEAN/STDEV bands are extracted to separate 1-band coverages)
  • data type: float (NODATA: -9999=>inversion_fail; 0=>missing)
  • spacetime bbox: year 2008; world-wide

Demo Features

In this section we describe the intended flow of analysis of our demo.

Area and Time of interest: data access and analysis

As the WW globe appears, the user has to choose its Region of Interest (RoI):

  • click on the 'Select interest area' button
  • draw the RoI (rectangular bounding box) on the globe

It is recommended to select a relatively big RoIs?: the scale of available datasets are meant for global analysis. By means of the Clear selection button, the initial state of the demo can be set again and RoI can be re-drawn.

Once the RoI is correctly defined, the user can click on the 'Start analysis of selected area' button to trigger the retrieval of the three AOT products (MODIS, CCI, and MACC) over those areas: the frame of analysis of the demo is now spatially defined, but the date of analysis still needs to be selected.

By default, the demo Day of Interest (DoI) is set to 1st of January 2008 (first available date), and the correspondent maps are visualized in the left-side panel 'Time slices preview'. The user can then move the analysis window along the time by visualizing so-called Hovmoeller time-latitude and time-longitude profiles along selected orthogonal cross-sections which the used can define by single mouse clicks on the WW globe, within the RoI. As this is done, Hovmoeller maps will appear in the bottom panel of the same name.

When an interesting AOT situation is recognized on the Hovmoeller maps, the calendar in the bottom-left corner can be used to move the time DoI to a different date: the preview of the time slices are then consequently refreshed in the left-side panel.

To aid the user on spotting an interesting DoI, by clicking on a product preview, the image is overlaid on the WW globe and an additional panel called 'Statistics' shows sample mean and standard deviation of the data, plus a the full plot of temporal evolution of AOT mean values on the RoI.

After RoI/DoI spatio-temporal window is set, the use can start using the tools for cross-comparing the three products with each other.

AOT cross-comparison:

The demo offers 3 different features for aid the numerical and visual comparison of two separate AOT products: A In all the cases

WCPS handles all NODATA and different spatial temporal resolutions.

Credits

http://www.meeo.it/wp/wp-content/uploads/2014/01/meeo_logo_trans.png


  • PC : Piero Campalani
  • AD : Alex Dumitru
  • VM : Vlad Merticariu
  • AB : Alan Beccati
  • SM : Simone Mantovani (MEEO Srl)