Opened 4 weeks ago

Closed 4 weeks ago

#1500 closed defect (wontfix)

petascope should set up rasql connection once

Reported by: dmisev Owned by: bphamhuu
Priority: critical Milestone: 9.4
Component: petascope Version: development
Keywords: Cc: vmerticariu, pbaumann
Complexity: Medium

Description

As ticket #1499 demonstrates (especially notice my comment with the measurements, setting up the connection to rasql is pretty expensive (up to 2 seconds in total!). Starting and committing transactions is fast otherwise.

Petascope seems to repeat the setup for every query (RasUtil.executeRasqlQuery), this needs be fixed as soon as possible so that it's done only once when petascope is started.

Attachments (2)

test_rasql_implementation.sh (291 bytes) - added by bphamhuu 4 weeks ago.
test scripts with 10000 requests
0001-ticket-1500-Petascope-using-one-RasqlImplementation.patch (21.0 KB) - added by bphamhuu 4 weeks ago.
patch which tried to test

Download all attachments as: .zip

Change History (8)

comment:1 Changed 4 weeks ago by dmisev

Most time is spent on

  • Setting user identification: 347 ms
  • Opening database: 80 ms

comment:2 follow-up: Changed 4 weeks ago by dmisev

Note: when fixing this, please test if petascope still works as expected after restarting rasdaman.

comment:3 in reply to: ↑ 2 Changed 4 weeks ago by bphamhuu

Replying to dmisev:

Note: when fixing this, please test if petascope still works as expected after restarting rasdaman.

Update: Actually, this method rasqlImplementation.setUserIdentification(ConfigManager?.RASDAMAN_USER, ConfigManager?.RASDAMAN_PASS);

also initialize a serverID to rasqlImplementation object each time (it seems like it check if a Rasserver with UUID is free then it open connection to this server and it throw error if using only 1 server UUID from Petascope initialization).


When I set static RasImplemenetation? from Petascope Initialization (singleton) and restart rasdaman, it will have this error (so seem like to support this optimization, must restart Petascope as well).

Caused by: java.lang.RuntimeException?: The client with client ID c8d616f3-3f8b-4fdb-9c76-0b1661ff5bb7 does not exist.

at org.rasdaman.rasnet.util.GrpcUtils?.convertStatusToRuntimeException(GrpcUtils?.java:57)

Last edited 4 weeks ago by bphamhuu (previous) (diff)

comment:4 Changed 4 weeks ago by dmisev

You should not need to restart petascope in this case, but catch the exception and reinitialize the connection?

comment:5 Changed 4 weeks ago by dmisev

Update: Actually, this method rasqlImplementation.setUserIdentification(ConfigManager??.RASDAMAN_USER, ConfigManager??.RASDAMAN_PASS);

also initialize a serverID to rasqlImplementation object each time (it seems like it check if a Rasserver with UUID is free then it open connection to this server and it throw error if using only 1 server UUID from Petascope initialization).

Ah I see, it's unexpected that a method setUserIdentification would do that! Hmm so this seems a bit more involved than I thought then.

Changed 4 weeks ago by bphamhuu

test scripts with 10000 requests

Changed 4 weeks ago by bphamhuu

patch which tried to test

comment:6 Changed 4 weeks ago by bphamhuu

  • Resolution set to wontfix
  • Status changed from new to closed

So I had a few tests and dicussed with Dimitar, the results for 10000 requests before the patch and after the patch (restart rasdaman, rebuild petascope each time) is:

with my patch
real 11m24.681s

and without my patch
real 11m16.243s

as the bottleneck is the need to make a new connection to rasserver if client-id is not the same and catch exception could lead to delay the time in reality so we consider to close this ticket now.

Note: See TracTickets for help on using tickets.