Opened 15 months ago

Last modified 9 months ago

#2775 closed enhancement

NEW - wcst_import should generate a reasonable default tiling — at Version 4

Reported by: Bang Pham Huu Owned by: Bang Pham Huu
Priority: major Milestone: 10.3
Component: wcst_import Version: 10.2
Keywords: Cc: Dimitar Misev, Vlad Merticariu
Complexity: Medium

Description (last modified by Dimitar Misev)

wcst_import should generate a reasonable default tiling for rasdaman collection in case this setting is not specified by user in the ingredients file.

Currently, for a 3D coverage, if tiling is not specified, petascope creates this tile

TILING ALIGNED [0:*,0:*,0:*] tile size 4194304

which is not good as it has these tiles in rasdaman:

 "[5:5,35136:35199,0:64999]",
        "[5:5,40320:40383,0:64999]",
        "[5:5,40384:40447,0:64999]",
        "[5:5,40448:40511,0:64999]",
        "[5:5,40512:40575,0:64999]",
        "[5:5,40576:40639,0:64999]",
        "[5:5,40640:40703,0:64999]",
...

It has to consider multiple details:

  • datacube dimension
  • which axes are spatial
  • Number of bands and types
  • What is primary access pattern: spatial slices, or across non-spatial axes, or something inbetween?
    • gdal slicer imports 2D data organized in spatial slices, so it's very likely the access will be on spatial slices and the tiling should focus on the spatial axes
    • netcdf/grib slicers probably import 3D+ data, so there can be some extension of the tiles to the non-spatial axes
  • Insitu: if original files are tiled/chunked (tiff and netcdf especially), the tiling in rasdaman should be a multiple of the chunk sizes

wcst_import should show a WARNING that no tiling was specified, and print the generated tiling.

Change History (4)

comment:1 by Bang Pham Huu, 15 months ago

Owner: set to Bang Pham Huu

comment:2 by Dimitar Misev, 15 months ago

Milestone: 10.210.3

comment:3 by Dimitar Misev, 15 months ago

Description: modified (diff)
Summary: FIX - wcst_import creates a good default tiling in case it is not specified in the ingredients fileNEW - wcst_import should generate a reasonable default tiling

comment:4 by Dimitar Misev, 15 months ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.