Opened 3 years ago

Closed 3 years ago

#621 closed defect (fixed)

Tile locking performance issue

Reported by: dmisev Owned by: klipskoch
Priority: major Milestone: 9.0.x
Component: lockmgr Version: development
Keywords: Cc: pbaumann
Complexity: Medium

Description

Locking seems to have a large overhead, to reproduce

#!/bin/bash

rasql -q 'create collection test GreySet'
# this will insert an object of 2601 tiles
rasql -q 'insert into test values marray i in [0:100,0:100] values 1c tiling aligned [0:1,0:1] tile size 4'
time rasql -q 'select sdom(c) from test as c' --out string 

Attachments (1)

lockmgr_measurements.txt (1.5 KB) - added by klipskoch 3 years ago.

Download all attachments as: .zip

Change History (14)

comment:1 Changed 3 years ago by dmisev

I submitted a patch that drastically reduces the performance overhead, however there is still significant overhead, I assume related to ecpg connections to postgres.

I haven't looked at it, are they established and terminated for every tile or something similar?

comment:2 Changed 3 years ago by klipskoch

The connection is established once at the creation of the lockmanager instance and closed once at the end of it's work.

comment:3 Changed 3 years ago by klipskoch

I will take a look and give some feedback as soon as I can.

comment:4 Changed 3 years ago by dmisev

I submitted another patch that moves the prepared statements into the connect() method. This further improved the performance, so the above query sdom() query takes 1.6s now.

comment:5 Changed 3 years ago by dmisev

  • Priority changed from blocker to major

Lowering priority as performance has been substantially improved.

comment:6 Changed 3 years ago by klipskoch

A patch simplifying the queries and the table structure was submitted and applied.
I will do further measurements to evaluate the current performance.

Changed 3 years ago by klipskoch

comment:7 Changed 3 years ago by klipskoch

Above are the measurements I made before and after the performance improvements.

Now, on my university computer, the select needs below 1 second to execute.

comment:8 Changed 3 years ago by pbaumann

thanks for providing the results. 2 things I notice:

  • select gets slower by a factor of 40 in the optimized lockmgr; we definitely need to work on this.
  • insert: the "optimized lockmgr" version is the slowest, slower than unoptimized?

comment:9 Changed 3 years ago by dmisev

Will you submit the patch soon Kinga?

comment:10 Changed 3 years ago by klipskoch

Yes, I will submit the patch soon. I wanted to finish my repeated measurements and
add some code documentation and will be able to submit the corresponding patch.

comment:11 Changed 3 years ago by klipskoch

Patch submitted.

comment:12 Changed 3 years ago by klipskoch

The patch was applied, so after a point you can close the ticket.

comment:13 Changed 3 years ago by dmisev

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.