Opened 7 months ago

Closed 2 months ago

Last modified 2 months ago

#1403 closed defect (fixed)

Rasql_Wrong transposition between X and Y axis when encode in netcdf

Reported by: bphamhuu Owned by: bbell
Priority: major Milestone: 9.4
Component: rasql Version: development
Keywords: Cc: dmisev, vmerticariu
Complexity: Medium

Description (last modified by dmisev)

The problem is when encode a coverage in netcdf, the output is transposed between X and Y (in case of 2D).

for example: encode rgb 2D in netcdf (the i1 and k3 as dimensions can be random names so try to swap these variables does not help (e.g: i,j to j,i).

Also the transpose option from http://rasdaman.org/wiki/CommonFormatsInterface does not work (i.e: set values to [1,0] or [111,0] does not have any affect).

rasql -q 'SELECT encode(c[0:10,0:3].0, "netcdf" , "{\"dimensions\":[\"i1\", \"k3\"],\"variables\":{

	\"i1\":{\"type\":\"double\",\"data\":[0,2]},

	\"k3\":{\"type\":\"double\",\"data\":[1,2]},
	\"red\":{\"type\":\"unsigned char\",
\"metadata\":{\"units\":\"10^0\"},\"name\":\"red\"}},
\"metadata\":{}}")

 FROM test_rgb AS c' --out file

when use gdalinfo to show bounding box (wrong: size X(4), size Y (11))

Size is 4, 11
Coordinate System is `'
Metadata:
  red#units=10^0
  red#valid_max=255
  red#valid_min=0
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,   11.0)
Upper Right (    4.0,    0.0)
Lower Right (    4.0,   11.0)
Center      (    2.0,    5.5)

if encode in png

rasql -q 'select encode(c[0:10,0:3].0, "png") from test_rgb as c' --out file

the bounding box is expected (size X: 11, size Y: 4)

Size is 11, 4
Coordinate System is `'
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,    4.0)
Upper Right (   11.0,    0.0)
Lower Right (   11.0,    4.0)
Center      (    5.5,    2.0

Change History (14)

comment:1 Changed 7 months ago by dmisev

  • Description modified (diff)

Yes transpose is not implemented at the moment.

comment:2 Changed 6 months ago by dmisev

We need to implement the transpose format parameter, documented here: http://rasdaman.org/wiki/CommonFormatsInterface

It can be restricted to the last two dimensions for now which makes it very simple to support. The function to do this is already implemented:
http://rasdaman.org/browser/conversion/convertor.hh#L173

And it's used in conversion/grib.cc at the moment. We just need to invoke it in conversion/netcdf.cc in case transpose is specified in the format parameters.

comment:3 Changed 6 months ago by dmisev

  • Owner changed from dmisev to drusu
  • Status changed from new to assigned

comment:4 Changed 6 months ago by bphamhuu

NOTE: not every netCDF file is in variable order (Lat, Long) which will be flipped when displaying. If input coverage is in order (Long, Lat) respectively, it will be displayed correctly.

http://www.unidata.ucar.edu/software/netcdf/examples/tos_O1_2001-2002.cdl

so WCPS can know when should use the transposition from the axis order.

comment:5 Changed 6 months ago by vmerticariu

CF compliant ones will always be flipped.
But indeed, we should be able to differentiate, maybe at import. We need to discuss this in a f2f meeting next week.

comment:6 Changed 3 months ago by dmisev

  • Owner changed from drusu to bbell

comment:7 Changed 3 months ago by dmisev

Danut, can you attach a patch of the modifications (including the test) that you did so far?

comment:8 Changed 3 months ago by dmisev

  • Milestone changed from 9.3 to 9.4

comment:9 Changed 2 months ago by dmisev

  • Resolution set to fixed
  • Status changed from assigned to closed

comment:10 follow-up: Changed 2 months ago by dmisev

Bang, do we have transpose option in wcst_import recipes?

comment:11 in reply to: ↑ 10 Changed 2 months ago by bphamhuu

Replying to dmisev:

Bang, do we have transpose option in wcst_import recipes?

don't have this option yet.

comment:12 Changed 2 months ago by vmerticariu

Let's discuss how we want to achieve this in the group meeting today. We have 2 options when importing netcdf:

  • transpose at ingest time, deliver as is for any format except netcdf and apply traspose again when delivering netcdf
  • store the data as is, deliver as is for netcdf, deliver transposed for all others.

We will need to think about how to store this information in the coverage.

comment:13 Changed 2 months ago by bphamhuu

I thik it should be consistent, when ingesting with 1 option tranpose_enable: true, it will transpose when ingesting coverage and getCoverage without regarding to any format types.

comment:14 Changed 2 months ago by dmisev

It would be good in any case to be able to specify during ingestion which option is preferred, as both are effective in different situations:

  1. Preferable if the data is primarily exported for visualization in common 2D formats
  2. Preferable if the data is primarily exported in netcdf
Note: See TracTickets for help on using tickets.