Opened 8 years ago

Closed 8 years ago

#1537 closed defect (fixed)

Profile petascope, rasj for excessive RAM usage

Reported by: Dimitar Misev Owned by: Dimitar Misev
Priority: critical Milestone: 9.4
Component: petascope Version: development
Keywords: Cc: Vlad Merticariu
Complexity: Medium

Description

It seems that petascope uses more memory than it theoretically should. The following query selects a 1GB PNG from rasdaman:

for b4 in(s2a_l2a_t32tps_B04_10m), b8 in (s2a_l2a_t32tps_B08_10m)
return encode(
 (b8[ansi("2016-11-08T15:47:44.000Z")] - b4[ansi("2016-11-08T15:47:44.000Z")]) / 
 (b8[ansi("2016-11-08T15:47:44.000Z")] + b4[ansi("2016-11-08T15:47:44.000Z")])
,"png")

however, Tomcat consumes at least 4GB, even though petascope needs to transfer only 1GB in total from rasdaman to the client.

Change History (8)

comment:1 by Bang Pham Huu, 8 years ago

Summary: Profile petascopeRasserver takes 4.5 GB for a request with result in 40 MB.

The problem is not from Petascope as Rasserver takes 4.5 GB Ram (monitored by htop and System Monitor) and the result returns only 40 MB.

the coverages were imported with 1 band for 1 coverage (band04: 113.6 MB and band08: 135.3 MB) JPG2 and the WCPS query is

for c in (sentinel2_10m_B04), d in (sentinel2_10m_B08)
return encode(
 (c - d) / 
 (c + d)
,"png")

and Rasql query is

SELECT encode( ( c - d )  /  ( c + d ) , "PNG" , "{\"geoReference\":{\"crs\":\"EPSG:32633\",\"bbox\":{\"xmin\":300000.0,\"ymin\":5290200.0,\"xmax\":409800.0,\"ymax\":5400000.0}}}") FROM sentinel2_10m_B04 AS c,sentinel2_10m_B08 AS d

comment:2 by Bang Pham Huu, 8 years ago

Owner: changed from Bang Pham Huu to Dimitar Misev
Status: newassigned

comment:3 by Bang Pham Huu, 8 years ago

Summary: Rasserver takes 4.5 GB for a request with result in 40 MB.Petascope profile

comment:4 by Dimitar Misev, 8 years ago

Owner: changed from Dimitar Misev to Bang Pham Huu
Summary: Petascope profileProfile petascope for excessive RAM usage

comment:5 by Dimitar Misev, 8 years ago

Milestone: 10.09.4
Priority: majorcritical

comment:6 by Bang Pham Huu, 8 years ago

There is a problem with memory usage in rasj library for Tomcat which occupies the byte array multiple times from the rasql result (e.g: a rasql returns 600 MB Tiff file, Tomcat will need to set heap maximum memory at least 2.4 GB). It will need to remove the initialization unnecessarily in rasj for storing result.

comment:7 by Bang Pham Huu, 8 years ago

Owner: changed from Bang Pham Huu to Dimitar Misev

comment:8 by Bang Pham Huu, 8 years ago

Resolution: fixed
Status: assignedclosed
Summary: Profile petascope for excessive RAM usageProfile petascope, rasj for excessive RAM usage
Note: See TracTickets for help on using tickets.