wiki:Petascope_9.5

Version 15 (modified by bphamhuu, 3 months ago) (diff)

add the description for how migration script will work

Petacope 9.5

There are two major changes:

  • internally it uses the Spring Boot Framework with Hibernate as object relational mapping data model
  • implements support for the Coverage Schema Implementation (CIS version 1.1) for GridCoverage, RectifiedGridCoverage and ReferenceableGridCoverage.

This brings the following advantages:

  • Petascope can be deployed on more backend DBMS beside PostgreSQL like MySQL, Oracle, DB2, HSQLDB, H2, etc. Postgresql is still the most stable database for deploying Petascope, but the user can easily switch to other databases by changing the configuration in petascope.properties (see list of tested databases).
    • With the data migration tool Liquibase, existing petascopedb can be migrated to the new schema in database petascopedb_cis
  • The Spring Boot Framework provides many utilities that aid in quicker development of petascope.
  • Petascope can now start as an embedded web application with an internal embedded Tomcat (i.e: no need to deploy to external Tomcat).

Changes in petascope.properties

The only change is triggered by the support for new JDBC database connections (see list of tested databases). The options following the spring.* convention are used directly by the Spring framework. For example, to use postgresql as a backend database for petascope, the following (default) configuration is specified:

spring.datasource.url=jdbc:postgresql://localhost:5432/petascopedb_cis
spring.datasource.username=petauser
spring.datasource.password=petapasswd
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect

The existing database configuration is kept without any change (note that it is only used during migration, as the source of existing petascopedb or, later on, petascopedb_cis):

metadata_url=jdbc:postgresql://localhost:5432/petascopedb
metadata_user=petauser
metadata_pass=petapasswd

If you have been using the embedded Jetty setup of petascope: the jetty_port option is now renamed to server.port (it is another option directly used by the Spring framework).

old: jetty_port=8080
new: server.port=8080

How to migrate

The database schema in petascope 9.5 has changed, so existing rasdaman installation would need to migrate by executing a migration script update_petascopedb.sh.

This will create a new database petascopedb_cis, and will not modify the existing petascopedb.

Developer's guide on update_petascopedb.sh

Before version 9.5, update_petascopedb.sh is a script to update old petascopedb in Postgresql with some changes, such as: add some columns, tables, triggers, functions,...Every change in petascopedb was marked as a update version (current version is .17). This script cannot rollback to previous version of petascopedb (e.g: version .17 to version .8). This script has a big disadvantage is: it can only run and update to Postgresql as the SQL syntax are different from every Relational Database.

Therefore, in new Petascope with using Hibernate as ORM data model, we cannot add the SQL queries explicitly anymore. To support different kinds of Relation Database, we use the Liquibase which could create the changes for each version to XML and later on generated to target SQL syntax of the current database (e.g: Postgresql, MySql?, HSQL,...). There is a migration table is added in new Petascope database to lock the application when the process is running (i.e: no multiple migrations are doing at the same time).

We will make a small Java Application to migrate from old petascopedb to petascopedb_cis. This application also uses Spring and Hibernate and can be invoked from update_petascopedb.sh script (i.e: no expose the service to migrate in new Petascope).

Below are the description of the migration from old database to new database:

NOTE: if you are using an embedded Database like HSQLDB, H2,... which does not support multiple connections from different application, you will need to stop the current running Petascope (either Tomcat external or embedded) first.

  • User execute the update_petascopedb.sh script by:
    ./update_petascopedb.sh
    
  • All the imported coverages in old petascopedb will be read by old CoverageMetadata? model which is imported to new Petascope as a legacy package.
  • The migration process will check if coverage id exists in new petascopedb_cis. If it does not, a process to translate from old CoverageMetadata? model to CIS Coverage data model is done and then persist this new object to new database.
  • While running the migration, all the services to the new Petascope web application, such as: WCS, WCPS, WMS, WCST will not available to make sure data is migrated safely.

NOTE: The old database is kept after the migration as if new Petascope might not work correctly, then user can switch back to old Petascope and old petascopedb. This could be done because, in version 9.5, old petascope deployed in old Tomcat will be renamed from rasdaman.war to rasdaman.war.bak, the petascope.properties is already backuped to petascope.properties.DATE.bak by update_properties.sh script. Then, user can rollback to use the old Petacope without any problem, just by renaming the new Petascope (rasdaman.war, petascope.properties to rasdaman.war.new and petascope.properties.new), then unrename the old ones (rasdaman.war.bak and petascope.properties.DATE.bak).

List of tested databases

Note that only a single datasource can be specified in petascope.properties; more than one is not supported.

# Postgresql (default)
spring.datasource.url=jdbc:postgresql://localhost:5432/petascopedb_cis
spring.datasource.username=petauser
spring.datasource.password=petapasswd
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect

# Mysql
spring.datasource.url=jdbc:mysql://localhost:3306/petascopedb_cis?createDatabaseIfNotExist=true
spring.datasource.username=root
spring.datasource.password=abc!123!ABC!123
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQLDialect

# HSQLDB
spring.datasource.url=jdbc:hsqldb:file://home/rasdaman/petascopedb_cis.db
spring.datasource.username=sa
spring.datasource.password=

# H2
spring.datasource.url=jdbc:h2:file://home/rasdaman/petascopedb_cis.db;DB_CLOSE_ON_EXIT=FALSE;DB_CLOSE_DELAY=-1
spring.datasource.username=sa
spring.datasource.password=
spring.h2.console.enabled=true

3rd party libraries

Petascope 9.5 uses various 3rd party libraries, documented on the overall rasdaman code provenance page.

TODO

  • Support dynamic loading of JDBC drivers by specifying the path to the jar file in petascope.properties