Opened 10 months ago

Closed 4 weeks ago

#1388 closed defect (fixed)

Rasql_Memory leak with swith case expression

Reported by: bphamhuu Owned by: bbell
Priority: blocker Milestone: 9.4
Component: rasserver Version: development
Keywords: Cc: dmisev
Complexity: Medium

Description

Dimitar, this script will run a simple WCPS query with switch and case expression, however, it will take around 100 MB Ram for 2 queries and when running attached script, the memory is not released (i.e: it will quickly fill the RAM).

WCPS query

for data in (mr )return encode(switch case data = 0 return {red:  0; green:  0; blue:  0} 
case data = 1 return {red:  200; green:  200; blue:  200} case data = 7 return  {red:  255; green:  0; blue:  0} 
case data = 9 return  {red:  255; green:  255; blue:  255} 
case data = 14 return  {red:  149; green:  0; blue:  148} 
case data = 16 return  {red:  254; green:  168; blue:  0} 
case data = 20 return  {red:  23; green:  250; blue:  255} 
case data = 26 return  {red:  255; green:  255; blue:  0} 
case data = 33 return  {red:  255; green:  235; blue:  0} 
case data = 34 return  {red:  147; green:  112; blue:  219} 
case data = 39 return  {red:  176; green:  48; blue:  96} 
case data = 47 return  {red:  255; green:  127; blue:  80}
 case data = 48 return  {red:  231; green:  142; blue:  79} 
case data = 49 return  {red:  178; green:  178; blue:  178} 
case data = 59 return  {red:  255; green:  165; blue:  0} 
case data = 62 return  {red:  107; green:  142; blue:  35} 
default return  {red:  0; green:  0; blue:  255} , "png")

Attachments (2)

test.sh (1.2 KB) - added by bphamhuu 10 months ago.
log.tar.gz (14.5 KB) - added by bphamhuu 9 months ago.
log error

Download all attachments as: .zip

Change History (11)

Changed 10 months ago by bphamhuu

comment:1 follow-up: Changed 9 months ago by dmisev

Can you put the rasql query here from the petascope log?

Changed 9 months ago by bphamhuu

log error

comment:2 in reply to: ↑ 1 Changed 9 months ago by bphamhuu

Replying to dmisev:

Can you put the rasql query here from the petascope log?

http://rasdaman.org/attachment/ticket/1388/log.tar.gz please check here.

comment:3 Changed 9 months ago by bphamhuu

sorry for misread your request, I put the rasql query

SELECT encode(CASE WHEN ( data = 0 ) THEN ( (0) * { 1c,0c,0c } + (0) * { 0c,1c,0c } + (0) * { 0c,0c,1c } ) WHEN ( data = 1 ) THEN ( (200) * { 1c,0c,0c } + (200) * { 0c,1c,0c } + (200) * { 0c,0c,1c } ) WHEN ( data = 7 ) THEN ( (255) * { 1c,0c,0c } + (0) * { 0c,1c,0c } + (0) * { 0c,0c,1c } ) WHEN ( data = 9 ) THEN ( (255) * { 1c,0c,0c } + (255) * { 0c,1c,0c } + (255) * { 0c,0c,1c } ) WHEN ( data = 14 ) THEN ( (149) * { 1c,0c,0c } + (0) * { 0c,1c,0c } + (148) * { 0c,0c,1c } ) WHEN ( data = 16 ) THEN ( (254) * { 1c,0c,0c } + (168) * { 0c,1c,0c } + (0) * { 0c,0c,1c } ) WHEN ( data = 20 ) THEN ( (23) * { 1c,0c,0c } + (250) * { 0c,1c,0c } + (255) * { 0c,0c,1c } ) WHEN ( data = 26 ) THEN ( (255) * { 1c,0c,0c } + (255) * { 0c,1c,0c } + (0) * { 0c,0c,1c } ) WHEN ( data = 33 ) THEN ( (255) * { 1c,0c,0c } + (235) * { 0c,1c,0c } + (0) * { 0c,0c,1c } ) WHEN ( data = 34 ) THEN ( (147) * { 1c,0c,0c } + (112) * { 0c,1c,0c } + (219) * { 0c,0c,1c } ) WHEN ( data = 39 ) THEN ( (176) * { 1c,0c,0c } + (48) * { 0c,1c,0c } + (96) * { 0c,0c,1c } ) WHEN ( data = 47 ) THEN ( (255) * { 1c,0c,0c } + (127) * { 0c,1c,0c } + (80) * { 0c,0c,1c } ) WHEN ( data = 48 ) THEN ( (231) * { 1c,0c,0c } + (142) * { 0c,1c,0c } + (79) * { 0c,0c,1c } ) WHEN ( data = 49 ) THEN ( (178) * { 1c,0c,0c } + (178) * { 0c,1c,0c } + (178) * { 0c,0c,1c } ) WHEN ( data = 59 ) THEN ( (255) * { 1c,0c,0c } + (165) * { 0c,1c,0c } + (0) * { 0c,0c,1c } ) WHEN ( data = 62 ) THEN ( (107) * { 1c,0c,0c } + (142) * { 0c,1c,0c } + (35) * { 0c,0c,1c } ) ELSE ( (0) * { 1c,0c,0c } + (0) * { 0c,1c,0c } + (255) * { 0c,0c,1c } ) END, "png\" , "xmin=0.0;ymin=0.0;xmax=256.0;ymax=211.0;crs=OGC:Index2D") FROM test_mr AS data

comment:4 follow-up: Changed 7 months ago by dmisev

Patch submitted for review, here's the most important valgrind output:

==20489== 20,737,128 bytes in 864,047 blocks are definitely lost in loss record 356 of 357
==20489==    at 0x4C2E0EF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20489==    by 0xA5E41C: QtCaseOp::evaluateInducedOp(std::vector<QtData*, std::allocator<QtData*> >*) (qtcaseop.cc:257)
==20489==    by 0xA5F3FF: QtCaseOp::evaluate(std::vector<QtData*, std::allocator<QtData*> >*) (qtcaseop.cc:399)
==20489==    by 0xA2AF29: QtConversion::evaluate(std::vector<QtData*, std::allocator<QtData*> >*) (qtconversion.cc:155)
==20489==    by 0xA01215: QtOperationIterator::next() (qtoperationiterator.cc:231)
==20489==    by 0xA51B13: QueryTree::evaluateRetrieval() (querytree.cc:172)
==20489==    by 0x99CAF0: ServerComm::executeQuery(unsigned long, char const*, ExecuteQueryRes&) (servercomm2.cc:1600)
==20489==    by 0x7E0504: doStuff() (directql.cc:1062)
==20489==    by 0x7E0C4A: main (directql.cc:1128)
==20489== 
==20489== 20,737,344 bytes in 864,056 blocks are definitely lost in loss record 357 of 357
==20489==    at 0x4C2E0EF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20489==    by 0xA5E438: QtCaseOp::evaluateInducedOp(std::vector<QtData*, std::allocator<QtData*> >*) (qtcaseop.cc:258)
==20489==    by 0xA5F3FF: QtCaseOp::evaluate(std::vector<QtData*, std::allocator<QtData*> >*) (qtcaseop.cc:399)
==20489==    by 0xA2AF29: QtConversion::evaluate(std::vector<QtData*, std::allocator<QtData*> >*) (qtconversion.cc:155)
==20489==    by 0xA01215: QtOperationIterator::next() (qtoperationiterator.cc:231)
==20489==    by 0xA51B13: QueryTree::evaluateRetrieval() (querytree.cc:172)
==20489==    by 0x99CAF0: ServerComm::executeQuery(unsigned long, char const*, ExecuteQueryRes&) (servercomm2.cc:1600)
==20489==    by 0x7E0504: doStuff() (directql.cc:1062)
==20489==    by 0x7E0C4A: main (directql.cc:1128)
==20489== 
==20489== LEAK SUMMARY:
==20489==    definitely lost: 41,478,816 bytes in 1,728,284 blocks
==20489==    indirectly lost: 5 bytes in 1 blocks
==20489==      possibly lost: 4,200 bytes in 175 blocks
==20489==    still reachable: 135,805 bytes in 662 blocks
==20489==         suppressed: 0 bytes in 0 blocks

comment:5 Changed 5 months ago by bphamhuu

  • Milestone changed from 9.3 to 9.4

comment:6 in reply to: ↑ 4 Changed 7 weeks ago by dmisev

  • Owner set to vmerticariu
  • Status changed from new to assigned

Replying to dmisev:

Patch submitted for review, here's the most important valgrind output:

http://codereview.rasdaman.org/D266

So it seems like Vlad was supposed to take over, but I can't see anything committed yet.
@Vlad, what's the status about this issue?

comment:7 Changed 6 weeks ago by dmisev

  • Priority changed from major to blocker

Bumping up priority, memory leaks should really be eliminated as much as possible.

comment:8 Changed 5 weeks ago by dmisev

  • Owner changed from vmerticariu to bbell

comment:9 Changed 4 weeks ago by bbell

  • Resolution set to fixed
  • Status changed from assigned to closed
Note: See TracTickets for help on using tickets.