Opened 7 years ago
Last modified 7 years ago
#1615 closed defect
RASQL_GRIB updates existing collection is really slow — at Initial Version
Reported by: | Bang Pham Huu | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | 9.5 |
Component: | rasql | Version: | development |
Keywords: | Cc: | Dimitar Misev, Vlad Merticariu, bbell | |
Complexity: | Medium |
Description
I have a test case like this, a GRIB file (5 messages) in system test / test all wcst_import with 0.5 MB could be ingested as a new collection in 2 seconds, Run the same rasql query to update this collection will take 6 minutes.
[INFO] - 09/10/2017 11:03:42.302743: evaluating... [INFO] - 09/10/2017 11:03:42.302787: ok, result type 'set <marray <char, [0:93]>>', 1 element(s), total size 94 bytes. [INFO] - 09/10/2017 11:03:54.166550: Connected successfully to '/home/rasdaman/install/data/RASBASE' [INFO] - 09/10/2017 11:03:54.167140: Request: 'CREATE COLLECTION test_grib_regular_5_messages_pixel_is_point_new test_grib_regular_5_messages_Set'... [INFO] - 09/10/2017 11:03:54.167154: parsing... [INFO] - 09/10/2017 11:03:54.167232: checking semantics... [INFO] - 09/10/2017 11:03:54.167239: evaluating... [INFO] - 09/10/2017 11:03:54.167373: ok [INFO] - 09/10/2017 11:03:54.167381: ok [INFO] - 09/10/2017 11:03:54.223283: Connected successfully to '/home/rasdaman/install/data/RASBASE' [INFO] - 09/10/2017 11:03:54.224243: Request: 'INSERT INTO test_grib_regular_5_messages_pixel_is_point_new VALUES <[0:0,0:0,0:0,0:0] 0d> TILING ALIGNED [0:13, 0:10, 0:0, 0:10] TILE SIZE 4000000'... [INFO] - 09/10/2017 11:03:54.224262: parsing... [INFO] - 09/10/2017 11:03:54.224386: checking semantics... [INFO] - 09/10/2017 11:03:54.224398: evaluating... [INFO] - 09/10/2017 11:03:54.225199: ok, result type 'set<long>', 1 element(s). [INFO] - 09/10/2017 11:03:54.225231: ok [INFO] - 09/10/2017 11:03:54.284370: Connected successfully to '/home/rasdaman/install/data/RASBASE' [INFO] - 09/10/2017 11:03:54.285169: Request: 'SELECT oid(test_grib_regular_5_messages_pixel_is_point_new) FROM test_grib_regular_5_messages_pixel_is_point_new'... [INFO] - 09/10/2017 11:03:54.285188: parsing... [INFO] - 09/10/2017 11:03:54.285269: checking semantics... [INFO] - 09/10/2017 11:03:54.285449: evaluating... [INFO] - 09/10/2017 11:03:54.285650: ok, result type 'set<double>', 1 element(s). [INFO] - 09/10/2017 11:03:54.285670: ok [INFO] - 09/10/2017 11:03:55.95120: Connected successfully to '/home/rasdaman/install/data/RASBASE' [INFO] - 09/10/2017 11:03:55.95743: Request: 'UPDATE test_grib_regular_5_messages_pixel_is_point_new SET test_grib_regular_5_messages_pixel_is_point_new[0:4,0:0,0:719,0:360] ASSIGN shift(decode(<[0:0] 1c>, "GRIB", "{\"internalStructure\":{\"messageDomains\":[{\"msgId\":1,\"domain\":\"[4:4,0:0,0:719,0:360]\"},{\"msgId\":2,\"domain\":\"[3:3,0:0,0:719,0:360]\"},{\"msgId\":3,\"domain\":\"[2:2,0:0,0:719,0:360]\"},{\"msgId\":4,\"domain\":\"[1:1,0:0,0:719,0:360]\"},{\"msgId\":5,\"domain\":\"[0:0,0:0,0:719,0:360]\"}]},\"filePaths\":[\"/home/rasdaman/test_server/rasdaman/systemtest/testcases_services/test_all_wcst_import/test_data/grib_regular_5_messages_pixel_is_point/test5messages.grib2\"]}"), [0,0,0,0]) WHERE oid(test_grib_regular_5_messages_pixel_is_point_new) = 62465'... [INFO] - 09/10/2017 11:03:55.95755: parsing... [INFO] - 09/10/2017 11:03:55.95879: checking semantics... [INFO] - 09/10/2017 11:03:55.96007: evaluating... [INFO] - 09/10/2017 11:03:56.155001: ok [INFO] - 09/10/2017 11:03:56.155026: ok [INFO] - 09/10/2017 11:04:10.312154: Connected successfully to '/home/rasdaman/install/data/RASBASE' [INFO] - 09/10/2017 11:04:10.312849: Request: 'UPDATE test_grib_regular_5_messages_pixel_is_point_new SET test_grib_regular_5_messages_pixel_is_point_new[0:4,0:0,0:719,0:360] ASSIGN shift(decode(<[0:0] 1c>, "GRIB", "{\"internalStructure\":{\"messageDomains\":[{\"msgId\":1,\"domain\":\"[4:4,0:0,0:719,0:360]\"},{\"msgId\":2,\"domain\":\"[3:3,0:0,0:719,0:360]\"},{\"msgId\":3,\"domain\":\"[2:2,0:0,0:719,0:360]\"},{\"msgId\":4,\"domain\":\"[1:1,0:0,0:719,0:360]\"},{\"msgId\":5,\"domain\":\"[0:0,0:0,0:719,0:360]\"}]},\"filePaths\":[\"/home/rasdaman/test_server/rasdaman/systemtest/testcases_services/test_all_wcst_import/test_data/grib_regular_5_messages_pixel_is_point/test5messages.grib2\"]}"), [0,0,0,0]) WHERE oid(test_grib_regular_5_messages_pixel_is_point_new) = 62465'... [INFO] - 09/10/2017 11:04:10.312867: parsing... [INFO] - 09/10/2017 11:04:10.312995: checking semantics... [INFO] - 09/10/2017 11:04:10.313120: evaluating... [INFO] - 09/10/2017 11:10:10.110952: ok [INFO] - 09/10/2017 11:10:10.110979: ok
Note:
See TracTickets
for help on using tickets.