Parallel ingestion issue

[Dimitar Misev]

"Transfer failed" error when importing in parallel with two rasimport commands like

rasimport --coll TEST -t GreyImage:GreySet -f /home/rasdaman/data/single/GLC2000/glc2000.tiff --crs-uri %SECORE_URL%/crs/EPSG/0/4326

rasimport --coll TEST2 -t GreyImage:GreySet -f /home/rasdaman/data/single/GLC2000/glc2000.tiff --crs-uri %SECORE_URL%/crs/EPSG/0/4326 

[Dimitar Misev]

RasdamanHelper2::updateImage():image update failed: RasManager Error: Write transaction in progress, please retry again later.

The first rasimport command finishes fine, the second fails with a "write transaction in progress" error though, even though I have 40 servers started and --enable-tilelocking in rasmgr.conf

So the tile locking doesn't seem to work properly as far as I can tell.

Kinga do you maybe have any idea about this?

[Kinga Lipskoch]

According to the error message the query/queries try to execute a write on the same tile.
I would first check if that is the case or not.

[Dimitar Misev]

That should not be the case because

1. it is an insert
2. into different collections TEST and TEST2

[Kinga Lipskoch]

But, insert does not use the lockmanager as far as I remember, only select and update do.

[Dimitar Misev]

Hmm so what happens on insert, the whole database is locked? Why?

Same thing happens with rasql as well btw.

I think my rasmgr.conf should be ok, right? Each server is defined like this:

define srv N40 -host hifi -type n -port 90040 -dbh rasdaman_host
change srv N40 -countdown 200 -autorestart on -xp --timeout 300 --enable-tilelocking

[Kinga Lipskoch]

From the lockmanager's perspective I implemented nothing should be locked with an insert.
Maybe some other code does that.
I think the rasmgr.conf is fine, at least from what I remember.

[Peter Baumann]

at least from the classical perspective, the whole database is locked, so the engine may fall back to that. Anyway, insert is a situation any lockmgr implementation should take into account - see std SQL.

[Dimitar Misev]

Ok I removed the whole database write lock in rasmgr, and now for the second request I get this in the rasserver log:

Warning/error in DBMDDObj::insertInDb():
SQLSTATE: 23505 SQLCODE: -403

Throwing Exception (SQLCODE=-403): Warning/error in DBMDDObj::insertInDb():
SQLSTATE: 23505 SQLCODE: -403


Warning/error in DBStorageLayout::insertInDb() INSERT INTO RAS_STORAGE:
SQLSTATE: 25P02 SQLCODE: -400

Throwing Exception (SQLCODE=-400): Warning/error in DBStorageLayout::insertInDb() INSERT INTO RAS_STORAGE:
SQLSTATE: 25P02 SQLCODE: -400


Warning/error in DBMinterval::insertInDb() INSERT INTO RAS_DOMAINS:
SQLSTATE: 25P02 SQLCODE: -400

Throwing Exception (SQLCODE=-400): Warning/error in DBMinterval::insertInDb() INSERT INTO RAS_DOMAINS:
SQLSTATE: 25P02 SQLCODE: -400

From the ECPG docs:

• -403 (ECPG_DUPLICATE_KEY)

Duplicate key error, violation of unique constraint. (SQLSTATE 23505)

• -400 (ECPG_PGSQL)

Some error caused by the PostgreSQL server. The message contains the error message from the PostgreSQL server.

[Peter Baumann]

this is likely more involved than just removing the whole lock (BTW, why is this still present in the code?). Kinga, although officially you have other obligations now, can I ask you to finalize the lockmgr job? thanks!

[Dimitar Misev]

@Kinga: the write transaction check is in source:rasmgr/rasmgr_master_nb.cc, line 691

[Dimitar Misev]

I have to wait for the new protocol code to be submitted in the repo before I can test the lockmgr (due to some limitations in RNP), so a few more days.

[Dimitar Misev]

rasimport is outdated and petascope/wcst_import.sh work fine now with parallel ingestion.