Getting information about arrays

[Dimitar Misev]

A function which returns information about an array in the database would be very useful. The information we can extract right now is limited to:

• array dimensionality and extents
• array type by specifying --type to rasql

Further info that would be highly useful:

• tiling scheme and configuration, number of tiles
• the index used
• storage format
• size in bytes of an array; often arrays are only partially materialized, so this would be useful information.

This could be implemented as an info function, which similarly to RAS_COLLECTIONNAMES returns a char array with the above (and possibly further) information.

[Dimitar Misev]

Proposed solution:

$ rasql -q 'select info(c) from mr as c' --out string
rasql: rasdaman query tool v1.0, rasdaman v8.3.1-g1ecc2e1 -- generated on 07.12.2012 16:06:36.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...ok
Query result collection has 3 element(s):
  Result object 1: Oid: 342529; Tiles: 1; Object size: 54016B; Tiling scheme: no tiling; Tile size: 2097152; Tile configuration: [0:511,0:511]; Index type: rpt_index; Index size: 0; PCT max: 4096B; PCT min: 2048B; Base type: marray <char, 2>
  Result object 2: Oid: 343041; Tiles: 1; Object size: 54016B; Tiling scheme: no tiling; Tile size: 2097152; Tile configuration: [0:511,0:511]; Index type: rpt_index; Index size: 0; PCT max: 4096B; PCT min: 2048B; Base type: marray <char, 2>
  Result object 3: Oid: 343553; Tiles: 1; Object size: 54016B; Tiling scheme: no tiling; Tile size: 2097152; Tile configuration: [0:511,0:511]; Index type: rpt_index; Index size: 0; PCT max: 4096B; PCT min: 2048B; Base type: marray <char, 2>
rasql done.

[Dimitar Misev]

Alternative JSON output encoding:

$ rasql -q 'select info(c) from mr as c' --out string
rasql: rasdaman query tool v1.0, rasdaman v8.3.1-g8897c51 -- generated on 07.12.2012 16:06:36.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...ok
Query result collection has 4 element(s):
  Result object 1: {
 "oid": "150529",
 "baseType": "marray <char>",
 "tileNo": "1",
 "totalSize": "54016B",
 "tiling": {
	"tilingScheme": "no_tiling",
	"tileSize": "2097152",
	"tileConfiguration": "[0:511,0:511]"
 },
 "index": {
	"type": "rpt_index",
	"indexSize": "0",
	"PCTmax": "4096B",
	"PCTmin": "2048B"
 }
}

[Dimitar Misev]

Fixed in changeset:ce08b30e549711685a044384f867831352c51276

Example use:

   $ rasql -q 'select dbinfo(eobs) from eobs' --out string
   rasql: rasdaman query tool v1.0, rasdaman osgeolive.5-24-g3ee4345 -- generated on 12.02.2013 09:39:11.
   opening database RASBASE at localhost:7001...ok
   Executing retrieval query...ok
   Query result collection has 1 element(s):
      Result object 1: {
     "oid": "111617",
     "baseType": "marray <short, 5>",
     "tileNo": "6344",
     "totalSize": "1063437888",
     "tiling": {
        "tilingScheme": "aligned",
        "tileSize": "4194304",
        "tileConfiguration": "[0:1023,0:1023]"
     },
     "index": {
        "type": "rpt_index",
        "PCTmax": "4096",
        "PCTmin": "2048"
     }
   }
   rasql done. 

[Peter Baumann]

Let us reconsider syntax to make it more database style. Generalizing the idea, we might think of cases like:

DESCRIBE COLLECTION collName
DESCRIBE ARRAY oid
DESCRIBE TYPE typeName

[Dirk Daems]

The output of the tile configuration doesn't seem to be correct in version 8.4 rc2:

[rasdaman@epod8 Biopar]$ rasql -q "select dbinfo(c) from Biopar_Africa_LAI as c" --out string
rasql: rasdaman query tool v1.0, rasdaman v8.4.0rc.2-3-g27a914b -- generated on 20.03.2013 11:53:11.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...ok
Query result collection has 1 element(s):
  Result object 1: {
 "oid": "3585",
 "baseType": "marray <char, 3>",
 "tileNo": "2592",
 "totalSize": "2890782756",
 "tiling": {
	"tilingScheme": "aligned",
	"tileSize": "4194304",
	"tileConfiguration": "[0:1023,0:1023]"
 },
 "index": {
	"type": "rpt_index",
	"PCTmax": "4096",
	"PCTmin": "2048"
 }
}
rasql done.

In this case, it should be:

...
"tileConfiguration": "[0:3,0:1023,0:1023]"
...

Which would also match the tile size: 4*1021*1024 = 4194304.

[Dimitar Misev]

Did you specify the tiling yourself when initializing the object?

The above tiling conf matches better the totalSize / tileNo

[Dirk Daems]

We used initmdd to initialize the collection and to specify the tiling strategy, but the tile configuration shown by dbinfo() doesn't match the one used with initmdd.

[Dimitar Misev]

initmdd may not be completely compatible with rasql. You should switch to using rasql for the initialization too (tiling has been fixed in the meantime), see wiki:Tiling