NEW - wcst_import replaces the existing values in current grid domains with null values before updating

[Bang Pham Huu]

Usecase

The situation below is used typically for weather forecast data.

• At time t, values t+1 are forecast and imported to rasdaman at some grid domains (e.g. [1,0:300,0:500])
• At time t+1, values for t+1 are actual recorded and updated to these grid domains (e.g. [1,0:300,0:500])

Requirement:

• At time t, grid domains for t+1 contain only forecast values
• At time t+1, grid domains for t+1 must contain only the actual recorded values (no forecast values are left)

Solution:

• At time t+1, grid domains for t+1 must be swept and filled with null values before the actual pixel values for t+1 imported.

Example

White color are null values.

• File A imported with some red values as below:

• File B (same grid extents as file B) has some blue values, (NOTE: blue pixels coordinates are not matched with red pixels coordinates above):

• Then, file B is imported. It will end up like below (blue pixel values from file B cannot replaces completely the red ones from file A):

Solution

In wcst_import, there is an optional boolean setting, called replace_imported_pixels.

• If it is set to True, then, wcst_import sends UpdateCoverage request to petascope with the extra parameter replaceImportedPixels=True.
• In petascope, it will run two rasql queries:
  • First (NOTE: only run when coverage has null values), update the whole grid domains for the subsets (Lat, Lon, time, height,…) from the UpdateCoverage request with the null values of the updating coverage.
    • NOTE: it uses this rasql trick, e.g. below null value is -9e+33f.

      rasql -q 'update test_weather_netcdf 
      set test_weather_netcdf[0,29,0:656,0:1376]
      assign ((marray x in [0:656,0:1376] values -9e+33f) null values [0f])' --user rasadmin --passwd rasadmin
      

  • After this update command, the grid domains [0,29,0:656,0:1376] only contains null value -9e+33f. Hence, when file B is imported, only the blue pixels will be set.

• Then, update the coverage normally from the input file