Opened 9 years ago
Last modified 9 years ago
#938 closed defect
WCS-T performance improvement — at Version 1
| Reported by: | Dimitar Misev | Owned by: | Dimitar Misev |
|---|---|---|---|
| Priority: | major | Milestone: | 9.2 |
| Component: | petascope | Version: | development |
| Keywords: | Cc: | Alex Dumitru, Vlad Merticariu, mantovani, damiano, Peter Baumann | |
| Complexity: | Medium |
Description (last modified by )
The WCS-T ingestion workflow (from wcst_import, via WCS-T petascope to rasdaman) needs to be instrumented and optimized where possible.
Some initial ideas
- Parallelize WCS-T in petascope
- Adding an ORM for the relational database would make this simple
- Parallelize wcst_import
- Refactoring the metadata access patterns, there are many redundant queries
- Pooling the rasdaman requests, we can create a queue of them so that we do not wait on each response before processing the next.
- Communication with SECORE looks like it could be a bottleneck, it's quite present in the petascope log
I'll try to make initial profiling so we have a better idea of how time is spent.
Note:
See TracTickets
for help on using tickets.
