Opened 9 years ago

Closed 9 years ago

#938 closed defect (fixed)

WCS-T performance improvement

Reported by: Dimitar Misev Owned by: Dimitar Misev
Priority: major Milestone: 9.2
Component: petascope Version: development
Keywords: Cc: Alex Dumitru, Vlad Merticariu, mantovani, damiano, Peter Baumann
Complexity: Medium

Description (last modified by Dimitar Misev)

The WCS-T ingestion workflow (from wcst_import, via WCS-T petascope to rasdaman) needs to be instrumented and optimized where possible.

Some initial ideas

  • Parallelize WCS-T in petascope
    • Adding an ORM for the relational database would make this simple
    • Parallelize wcst_import
  • Refactoring the metadata access patterns, there are many redundant queries
  • Pooling the rasdaman requests, we can create a queue of them so that we do not wait on each response before processing the next.
  • Communication with SECORE looks like it could be a bottleneck, it's quite present in the petascope log

I'll try to make initial profiling so we have a better idea of how time is spent.

Change History (3)

comment:1 by Dimitar Misev, 9 years ago

Description: modified (diff)

comment:2 by Dimitar Misev, 9 years ago

Milestone: 9.1 → 9.2

comment:3 by Dimitar Misev, 9 years ago

Resolution: → fixed
Status: new → closed

The issue has been addressed as far as I know.

Note: See TracTickets for help on using tickets.