Opened 8 months ago

Closed 7 months ago

#1409 closed task (fixed)

Test nectdf import + export

Reported by: vmerticariu Owned by: bphamhuu
Priority: major Milestone: 9.3
Component: rasserver Version: development
Keywords: Cc: abadoi, dmisev, bphamhuu
Complexity: Medium

Description

Task is:

  • download the attached netcdf file and inspect them in a netcdf client (e.g. panoply)
  • import the netcdfs into rasdaman (either 1 band at a time, or several bands)
  • export the data from rasdaman in png and netcdf
  • inspect the results using the same client, check that the files resulting from rasdaman are the same as the original

Examples on how to import netcdf data in rasaman can be found in systemtests/testcases_mandatory/test_netcdf

Attachments (3)

upsize_down.png (103.5 KB) - added by bphamhuu 7 months ago.
netcdf compare to png
netcdf_mirror.png (144.0 KB) - added by bphamhuu 7 months ago.
diffrent_netcdfs.png (67.5 KB) - added by bphamhuu 7 months ago.

Download all attachments as: .zip

Change History (16)

comment:2 Changed 7 months ago by dmisev

Here's some more sample data: https://rsg.pml.ac.uk/shared_files/olcl/data_for_vlad/

Incl. wcst_import ingredients :

comment:3 Changed 7 months ago by bphamhuu

  • Cc bphamhuu added
  • Milestone set to 9.3

Changed 7 months ago by bphamhuu

netcdf compare to png

comment:4 Changed 7 months ago by bphamhuu

  • Owner changed from drusu to bphamhuu
  • Status changed from new to assigned

I see there is a patch for this ticket was uploaded in patch manager and it was failed. The change in this patch with calculation of the Lat (negative direction) leads to different subsets (netCDF is top and PNG is bottom).

http://rasdaman.org/attachment/ticket/1409/upsize_down.png

So I'll take this ticket and check it a bit more careful, thanks Drusu for doing it first.

comment:5 Changed 7 months ago by bphamhuu

So the problem with the reversed image is invalid, sorry, (because the netcdf was flipped then I saw wrong direction, checked by Vlad).

There is a problem now is the netCDF when do subsets (e.g: Lat(-45:45) is missing value 0, then it is shifted by 1 pixel).
e.g: subsets

45, 35, 25, 15, 5, -5, -15, -25 ;

e.g: the unsubsetted result:

 90, 80, 70, 60, 50, 40, 30, 20, 10, 0, -10, -20, -30, -40, -50, -60, -70, -80 ;

Changed 7 months ago by bphamhuu

comment:6 Changed 7 months ago by bphamhuu

@Vlad: if we write the Lat axis top-down (90, 88, ... -90) it will be mirrored in Panoply. If write bottom up (-90, -88, ... 90) then it display correctly.

http://rasdaman.org/attachment/ticket/1409/netcdf_mirror.png

comment:7 Changed 7 months ago by vmerticariu

It doesn't matter. As you see in the left-hand side image, panoply believes that the north pole is at -90 degrees, which is false.

Bottom up is correct for linear negative axes.

comment:8 Changed 7 months ago by bphamhuu

Yes, but in the patch for this ticket it writes top-down (which is in the right-hand side image), then I fixed it to write bottom-up (left-hand side image), you see in both case Panoly displays bottom-up (North is -90 degree in axis number).

maybe I'm confusing with the axis, sorry.

Last edited 7 months ago by bphamhuu (previous) (diff)

comment:9 follow-up: Changed 7 months ago by vmerticariu

The way the image is written by the patch is correct.

Right-hand side = correct

Left-hand side = incorrect

comment:10 in reply to: ↑ 9 Changed 7 months ago by bphamhuu

Replying to vmerticariu:

The way the image is written by the patch is correct.

Right-hand side = correct

Left-hand side = incorrect

yeah, I just realize what you meant after comment, you are rightthanks.

Changed 7 months ago by bphamhuu

comment:11 Changed 7 months ago by bphamhuu

I imported the file http://flanche.net/geodata/nc-201003.nc with this recipe http://pastebin.com/C9bhSiB7

then compare in Panoply (left is encoded netCDF output, right is the input file). There are different

http://rasdaman.org/attachment/ticket/1409/diffrent_netcdfs.png

+ the left image is shifted by 0.21 degree.
+ the left image null value should be -999999 and in recipe set nilValue.
+ also it has an extended bound > -90 (Lat) and > 180 (Long), (not sure it was fixed).

    Lat (measured in degree ) with domain extent from -123.33333333347065 to 90.00000000003335.

    Long (measured in degree ) with domain extent from -180.00000000033336 to 203.99999999997385.

comment:12 Changed 7 months ago by bphamhuu

@Vlad: in http://rasdaman.org/ticket/1409#comment:5 I meant it missed the 0 value, actually, due to the formula current in NetCdfDecodeFactory?:

for (i = 0l i < points; i++) {
 if (axisDirection > 0)
     coord = geoDomMax.subtract(resolution * i)
 else 
     coord = geoDomMin.add(resolution * i) 
}

So if I have a coverage with offset vector is (Long: 10,Lat:-10) and bounding box: (Lat: -90: 90, Long: -180:180), then I subset: Lat(-20:15), it returns a coverage with data: Lat = 15, 5, -5 ;

this is wrong coordinates and what need to do is shifted the input subset to correct geo coordinate (http://codereview.rasdaman.org/D249, wait for your review), so Lat(-20:15) will be shifted to (-20:20) and the netCDF formula above will return data: Lat = 20, 10, 0, -10 ; which is correct points.

comment:13 Changed 7 months ago by dmisev

  • Resolution set to fixed
  • Status changed from assigned to closed

I guess can be closed.

Note: See TracTickets for help on using tickets.