Opened 10 years ago
Closed 9 years ago
#938 closed defect (fixed)
WCS-T performance improvement
| Reported by: | Dimitar Misev | Owned by: | Dimitar Misev |
|---|---|---|---|
| Priority: | major | Milestone: | 9.2 |
| Component: | petascope | Version: | development |
| Keywords: | Cc: | Alex Dumitru, Vlad Merticariu, mantovani, damiano, Peter Baumann | |
| Complexity: | Medium |
Description (last modified by )
The WCS-T ingestion workflow (from wcst_import, via WCS-T petascope to rasdaman) needs to be instrumented and optimized where possible.
Some initial ideas
- Parallelize WCS-T in petascope
- Adding an ORM for the relational database would make this simple
- Parallelize wcst_import
- Refactoring the metadata access patterns, there are many redundant queries
- Pooling the rasdaman requests, we can create a queue of them so that we do not wait on each response before processing the next.
- Communication with SECORE looks like it could be a bottleneck, it's quite present in the petascope log
I'll try to make initial profiling so we have a better idea of how time is spent.
Change History (3)
comment:1 by , 10 years ago
| Description: | modified (diff) |
|---|
comment:2 by , 9 years ago
| Milestone: | 9.1 → 9.2 |
|---|
comment:3 by , 9 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Note:
See TracTickets
for help on using tickets.
The issue has been addressed as far as I know.