Opened 7 years ago
Closed 7 years ago
#1734 closed defect (fixed)
WCST_Import error from Numpy with metadata "auto"
Reported by: | Bang Pham Huu | Owned by: | Bang Pham Huu |
---|---|---|---|
Priority: | major | Milestone: | 9.6 |
Component: | wcst_import | Version: | development |
Keywords: | Cc: | Dimitar Misev, Vlad Merticariu | |
Complexity: | Medium |
Description
From PML's netCDF ingredient with setting metadata: "auto"
"coverage": { "crs": "EPSG/0/4326@OGC/0/AnsiDate", "metadata": { "type": "xml", "global": "auto" },
which throws error from WCST_Import.sh
File "/home/rasdaman/rasdaman_community/rasdaman/applications/wcst_import/wcst_import.py", line 130, in main reg.run_recipe(session) File "/home/rasdaman/rasdaman_community/rasdaman/applications/wcst_import/master/recipe/recipe_registry.py", line 72, in run_recipe recipe.describe() File "/home/rasdaman/rasdaman_community/rasdaman/applications/wcst_import/recipes/general_coverage/recipe.py", line 140, in describe importer = self._get_importer() File "/home/rasdaman/rasdaman_community/rasdaman/applications/wcst_import/recipes/general_coverage/recipe.py", line 461, in _get_importer self.importer = Importer(self._get_coverage(), self.options['wms_import'], File "/home/rasdaman/rasdaman_community/rasdaman/applications/wcst_import/recipes/general_coverage/recipe.py", line 381, in _get_coverage coverage = self._get_netcdf_coverage(recipe_type) File "/home/rasdaman/rasdaman_community/rasdaman/applications/wcst_import/recipes/general_coverage/recipe.py", line 423, in _get_netcdf_coverage self.options['tiling'], self._netcdf_global_metadata_fields(), File "/home/rasdaman/rasdaman_community/rasdaman/applications/wcst_import/recipes/general_coverage/recipe.py", line 282, in _netcdf_global_metadata_fields metadata_dict = escape_metadata_dict(self.__parse_netcdf_global_metadata()) File "/home/rasdaman/rasdaman_community/rasdaman/applications/wcst_import/util/string_util.py", line 56, in escape_metadata_dict metadata_dict[key] = escape(value) File "/usr/lib64/python2.7/xml/sax/saxutils.py", line 32, in escape data = data.replace("&", "&") AttributeError: 'numpy.float32' object has no attribute 'replace'
and global attributes of this netCDF file is:
// global attributes: :Conventions = "CF-1.6" ; :Metadata_Conventions = "Unidata Dataset Discovery v1.0" ; :cdm_data_type = "Grid" ; :comment = "See summary attribute" ; :creator_email = "rsghelp@pml.ac.uk" ; :creator_name = "Plymouth Marine Laboratory" ; :creator_url = "http://rsg.pml.ac.uk" ; :geospatial_lat_max = 90.f ; :geospatial_lat_min = -90.f ; :geospatial_lat_resolution = ".04166666666666666666" ; :geospatial_lat_units = "decimal degrees north" ; :geospatial_lon_max = 180.f ; :geospatial_lon_min = -180.f ; :geospatial_lon_resolution = ".04166666666666666666" ; :geospatial_lon_units = "decimal degrees east" ; :geospatial_vertical_max = 0.f ; :geospatial_vertical_min = 0.f ; :institution = "Plymouth Marine Laboratory" ; :keywords = "satellite,observation,ocean,ocean colour" ; :keywords_vocabulary = "none" ; :license = "ESA CCI Data Policy: free and open access" ; :naming_authority = "uk.ac.pml" ; :number_of_optical_water_types = "14" ; :platform = "Orbview-2/SeaStar,Aqua,Envisat" ; :processing_level = "Level-3" ; :product_version = "2.0" ; :project = "Climate Change Initiative - European Space Agency" ; :references = "http://www.esa-oceancolour-cci.org/" ; :sensor = "SeaWiFS,MODIS,MERIS" ; :source = "SeaWiFS R2010.0, MODISA R2013.1, MERIS L1B 3rd reprocessing + OCL corrections" ; :spatial_resolution = "4km nominal at equator" ; :standard_name_vocabulary = "NetCDF Climate and Forecast (CF) Metadata Conventions Version 1.6" ; :title = "ESA CCI Ocean Colour Product" ; :number_of_files_composited = 31 ; :summary = "Data products generated by the Ocean Colour component of the European Space Agency\'s Climate Change Initiative project. These files are monthly composites of merged sensor (MERIS, MODIS, SeaWiFS) products, where MODIS and MERIS wereband-shifted and bias-corrected to SeaWiFS bands and values. MODIS and SeaWiFS RRS products were derived from 4km resolution level-3 binned standard NASA products; MERIS was derived from a 4km level-3 binning of the output of the HYGEOS POLYMER algorithm. Derived products, such as Kd and chlorophyll were computed with the standard SeaDAS (SeaDAS v6.4-OCCCI) algorithms. QAA IOPs were derived using the standard SeaDAS (SeaDAS v6.4-OCCCI) algorithm, with a modified backscatter table to match that used in the bandshifting. Uncertainty estimates were added using Tim Moore\'s fuzzy water classification and uncertainty estimation algorithm." ; :tracking_id = "4d11baa9-933d-4484-822e-c2d92c826653" ; :id = "ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1M_MONTHLY_4km_GEO_PML_OC4v6_QAA-200007-fv2.0.nc" ; :creation_date = "Thu Apr 23 15:38:25 2015" ; :date_created = "Thu Apr 23 15:38:25 2015" ; :time_coverage_resolution = "P1M" ; :time_coverage_duration = "P1M" ; :start_date = "01-JUL-2000 00:00:00.000000" ; :stop_date = "31-JUL-2000 23:59:00.000000" ; :time_coverage_start = "200007010000Z" ; :time_coverage_end = "200007312359Z" ; :history = "Fri Apr 24 18:46:45 2015: ncks --variable lat,lon --exclude -C --overwrite --chunk_dimension lat,256 --chunk_dimension lon,256 /data/caspian2/scratch/cci/v2.0-production/stage11-composites/geographic/monthly/chlor_a/2000/ESACCI-OC-L3S-CHLOR_A-MERGED-1M_MONTHLY_4km_GEO_PML_OC4v6-200007-fv2.0.nc ESACCI-OC-L3S-CHLOR_A-MERGED-1M_MONTHLY_4km_GEO_PML_OC4v6-200007-fv2.0.nc\n", "Source data were: ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000701-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000702-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000703-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000704-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000705-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000706-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000707-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000708-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000709-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000710-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000711-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000712-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000713-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000714-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000715-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000716-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000717-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000718-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000719-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000720-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000721-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000722-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000723-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000724-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000725-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000726-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000727-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000728-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000729-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000730-fv2.0.nc, ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1D_DAILY_4km_GEO_PML_OC4v6_QAA-20000731-fv2.0.nc; netcdf_compositor_cci composites Rrs_412, Rrs_443, Rrs_490, Rrs_510, Rrs_555, Rrs_670, atot_412, atot_443, atot_490, atot_510, atot_555, atot_670, chlor_a, aph_412, aph_443, aph_490, aph_510, aph_555, aph_670, adg_412, adg_443, adg_490, adg_510, adg_555, adg_670, bbp_412, bbp_443, bbp_490, bbp_510, bbp_555, bbp_670, kd_490, water_class1, water_class2, water_class3, water_class4, water_class5, water_class6, water_class7, water_class8, water_class9, water_class10, water_class11, water_class12, water_class13, water_class14, Rrs_412_bias, Rrs_443_bias, Rrs_490_bias, Rrs_510_bias, Rrs_555_bias, Rrs_670_bias, aph_412_bias, aph_443_bias, aph_490_bias, aph_510_bias, aph_555_bias, aph_670_bias, adg_412_bias, adg_443_bias, adg_490_bias, adg_510_bias, adg_555_bias, adg_670_bias, kd_490_bias, chlor_a_log10_bias with --mean, chlor_a_log10_rmsd, Rrs_412_rmsd, Rrs_443_rmsd, Rrs_490_rmsd, Rrs_510_rmsd, Rrs_555_rmsd, Rrs_670_rmsd, aph_412_rmsd, aph_443_rmsd, aph_490_rmsd, aph_510_rmsd, aph_555_rmsd, aph_670_rmsd, adg_412_rmsd, adg_443_rmsd, adg_490_rmsd, adg_510_rmsd, adg_555_rmsd, adg_670_rmsd, kd_490_rmsd with --root-mean-square, and total_nobs, MODISA_nobs, MERIS_nobs, SeaWiFS_nobs - with --total\n", "1429804840 Subsetted from standardised_geo/ESACCI-OC-L3S-OC_PRODUCTS-MERGED-1M_MONTHLY_4km_GEO_PML_OC4v6_QAA-200007-fv2.0.nc to only include variables MERIS_nobs_sum,MODISA_nobs_sum,SeaWiFS_nobs_sum,chlor_a,chlor_a_log10_bias,chlor_a_log10_rmsd,crs,lat,lon,time,total_nobs_sum" ; :netcdf_file_type = "NETCDF4_CLASSIC" ; :NCO = "4.3.0" ;
Reference: http://earthserver.eu/trac2/ticket/259
Note:
See TracTickets
for help on using tickets.