TIFF to GeoTIFF in petascope

[Dimitar Misev]

I have an old version of petascope from Michael, which adds geo-spatial information to TIFF files, converting them to GeoTIFF.

This should be integrated into the current petascope code.

[Dimitar Misev]

There are two ways to fix this converstion:

1. Allow setting of tags in the tiff convertor itself, so that you can do e.g.

   select tiff(c,"TIFF_GEOTAG1=value1,TIFF_GEOTAG2=value2,...") from COLL as c
   

   This would be done in rasdaman (source:conversion/tiff.cc), and then accordingly use the tiff function with the right GeoTIFF tags from petascope. In this way you can get a GeoTIFF straight from rasdaman.

2. The other way is as Michael has done it: get a regular TIFF file from rasdaman (so just tiff(c) without any parameters), and then convert it to GeoTIFF in petascope. In this case you'd just need to incorporate the attachment:relevant_sources.tgz​ in the current petascope.

[Dimitar Misev]

Replying to dmisev:

and then convert it to GeoTIFF in petascope

Note that convert to GeoTIFF means just updating the TIFF file metadata (in-memory), so it's not some expensive operation to do.

[Dimitar Misev]

How can you test your fix: run a WCPS query

for c in (coverage)
return
encode(c, "tiff")

And then verify with gdalinfo if the result file has the expected geo-information.

[p.baumann@…]

note that "in-memory operation" assumes we can hold the result in main memory, which may not be the case - in other words, we limit ourselves in file size. Further, re-reading a file might cause additional overhead, while rasdaman can do it on the fly while generating the file, hence more efficiently. In the extreme case, this might mean a factor of 2 in performance.

[p.baumann@…]

discussion:
pro 1: general mechanism - this will be useful, eg, once we want to set further storage parameters, such as tiling, compression, or JPEG quality
pro 1: scalable, as it does not touch the file a second time (after rasdaman generation)
pro 3: scalable beyond server main memory limits
pro 2: done already, and GeoTIFF is geo and, hence, petascope domain.
My favourite is 1, but I'm ready to listen.

[Dimitar Misev]

The mechanism for specifying conversion parameters is already there (QL guide, section 9.8), e.g. for TIFF you have these possible parameters:

comptype=[none|ccittrle|ccittfax3|ccittfa
x4|lzw|ojpeg|jpeg|next|ccittrlew|packbits
|thunderscan|pixarfilm|pixarlog|deflate|d
cs|jbig]
quality=%i (default:80)

So we just need to add a parameter to set specified tags, e.g. something like

tifftags=TAG_NAME1:TAG_VALUE1;TAG_NAME2:TAG_VALUE2;..

Not sure if it's always possible to fit a tag value easily in a string though. Also there are things like TIFF directories, besides the tags.

GeoTIFF documentation btw: ​http://www.remotesensing.org/geotiff/spec/contents.html

[Dimitar Misev]

Alireza what's the status, have you looked at this?

[Dimitar Misev]

Obsolete, will be done using the new encode function.

[Dimitar Misev]

Ok, encode function has been implemented, what is needed now is to just pass the georeferencing information to it.

This should be done when the Geotiff/JPEG2000 format extensions in petascope are invoked by a request. When a WCS request is translated to WCPS, these additional parameters need to be passed to the WCPS encode function (which is now same as the rasql one): xmin,ymin,xmax,ymax,crs

On the WCPS level, the encode function should be modified to directly invoke rasql's encode, instead of png(), tiff(), or similar.

I will fix it in WCPS, and I think it's best to assign the WCS level to Piero.

[Piero Campalani]

Ok, agreed.

[Dimitar Misev]

Replying to dmisev:

On the WCPS level, the encode function should be modified to directly invoke rasql's encode, instead of png(), tiff(), or similar.

This is done and already in the repo.

[Peter Baumann]

what's the status of this? Several people are waiting for this…

[Piero Campalani]

I've been working on other tasks, but I can prioritize it now.

[Piero Campalani]

Dimitar,
on the WCPS level, should the user explicitly set the format parameters like:

for c in (mean_summer_airtemp)
  return encode(c, "GTiff", "xmin=XX;ymin=-YY;...")

so that only on the WCS level this would be automatically done?
I guess that at a second step we should also let the WCPS call rasql encode function with automatic filling of some parameters like bbox and CRS.

[Piero Campalani]

OGC 12-100 (pending):
—-
Requirement 5 /req/geotiff-coverage/mime-type-identifier:
GeoTIFF encoding of a coverage shall be indicated by the following MIME type
identifier: image/tiff
—-

[Dimitar Misev]

For now let's make it at WCS because that's easy, at WCPS level it would be quite hard I think because we'll have to investigate the WCPS query tree in order to determine the geo-bounds of the result.

[Piero Campalani]

Patch submitted.
Tested queries:

• No subset ​http://localhost:8080/petascope/wcs2?service=WCS&version=2.0.0&request=GetCoverage&coverageid=mean_summer_airtemp&format=image/tiff
• One subset ​http://localhost:8080/petascope/wcs2?service=WCS&version=2.0.0&request=GetCoverage&coverageid=mean_summer_airtemp&subset=x(120,130)&format=image/tiff
• Two subsets ​http://localhost:8080/petascope/wcs2?service=WCS&version=2.0.0&request=GetCoverage&coverageid=mean_summer_airtemp&subset=x(120,130)&subset=y(-30,-20)&format=image/tiff
• Slice away t on eobstest ​http://localhost:8080/petascope/wcs2?service=WCS&version=2.0.0&request=GetCoverage&coverageid=eobstest&subset=t(1601-01-02)&subset=x(30,40)&subset=y(60,70)&format=image/tiff
• 3 bands (RGB) ​http://localhost:8080/petascope/wcs2?service=WCS&version=2.0.0&request=GetCoverage&coverageid=NIR&subset=x(0,100)&subset=y(0,200)&format=image/tiff
• ERROR: output dimension 1 ​http://localhost:8080/petascope/wcs2?service=WCS&version=2.0.0&request=GetCoverage&coverageid=mean_summer_airtemp&subset=x(120,130)&subset=y(-20)&format=image/tiff
• ERROR: output dimension 3 ​http://localhost:8080/petascope/wcs2?service=WCS&version=2.0.0&request=GetCoverage&coverageid=eobstest&subset=t(1601-01-02,1601-01-03)&subset=x(30,40)&subset=y(60,70)&format=image/tiff

Next step: extend behavior to JPEG2000 format.

[Piero Campalani]

Patches accepted, we can fix this.

1. WCS to set proper geo-paramters for GTiff encoding (ticket #164). 
2. WCS to set proper geo-paramters as well for JP2 encoding (ticket #164). 

[Dimitar Misev]

Reopening so we can fix this for WCPS as well. A preliminary, rudimentary patch: attachment:wcps.patch​