Opened 2 months ago

Closed 7 weeks ago

#1528 closed defect (fixed)

wcst_import should preserve all metadata from netcdf

Reported by: dmisev Owned by: bphamhuu
Priority: major Milestone: 9.4
Component: wcst_import Version: development
Keywords: Cc: vmerticariu
Complexity: Medium

Description (last modified by bphamhuu)

Currently it preserves only the metadata explicitly specified by the user. This should be changed to the following:

  1. in case metadata: global, in the ingredient is an object (even empty), the current behavior is kept (the metadata indicated in the ingredient is kept)
  2. in case metadata: global has value "auto", all fields from the file are kept, specified as:
     "metadata": {
        "type": "xml",
        "global": "auto"
    }
    
    or
    
    "metadata": {
        "type": "json",
        "global": "auto"
    }
    
  3. in case metadata: global does not exist, the "auto" behavior is assumed.

Case 2: If user wants to add some metadata and keep the existing ones, then he has to add all manually.

Change History (13)

comment:1 Changed 2 months ago by bphamhuu

  • Component changed from petascope to wcst_import

It should be added new if the metadata field does not exist and overwrite if exists.

comment:2 Changed 2 months ago by vmerticariu

Consider 4 use-cases:

  1. user wants to keep all metadata unchanged
  2. user wants to keep all metadata, and add some
  3. user wants to keep only some of the metadata from the file
  4. user wants to keep all metadata, with different keys

All these are currently supported. If we go for automatic preserving, only 1 and 2 are supported.

comment:3 Changed 2 months ago by dmisev

We can support 3 and 4 by allowing to disable auto-preservation of metadata (in which case it would equate to the current behavior)

comment:4 Changed 2 months ago by vmerticariu

So after discussion we have decided that:

  • in case metadata: global, in the ingredient is an object (even empty), the current behavior is kept (the metadata indicated in the ingredient is kept)
  • in case metadata: global has value "auto", all fields from the file are kept
  • in case metadata: global does not exist, the "auto" behavior happens.

Please correct if there's anything wrong.

comment:5 Changed 2 months ago by dmisev

Can you give a specific example of how the second point would look like in the ingredients? So we have it documented here.

comment:6 Changed 2 months ago by vmerticariu

Case 2:

"metadata": {

"type": "xml",
"global": "auto"

}

or

"metadata": {

"type": "json",
"global": "auto"

}

Last edited 2 months ago by vmerticariu (previous) (diff)

comment:7 Changed 2 months ago by dmisev

  • Description modified (diff)

comment:8 Changed 2 months ago by vmerticariu

This should produce empty metadata:

"metadata": {

"type": "json",
"global": {}

}

While this should produce "auto" metadata:

"metadata": {

"type": "json"

}

Same in case "metadata" is not specified at all.

comment:9 Changed 2 months ago by dmisev

  • Description modified (diff)

comment:10 Changed 2 months ago by dmisev

what about metadata of variables?

comment:11 Changed 2 months ago by bphamhuu

  • Description modified (diff)

comment:12 Changed 2 months ago by vmerticariu

We should open a new ticket for that. Right now, metadata of variables is saved into the RageField? representing the variable, and its contents are limited to the schema of swe commons. At some point we should treat is as we treat global metadata, and simply add it below "global", in a field with the name of the variable. There is some work on this started by FZ Juelich, but I don't know how advanced it is.

Last edited 2 months ago by vmerticariu (previous) (diff)

comment:13 Changed 7 weeks ago by bphamhuu

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.