wiki:Petascope_9.5

Version 24 (modified by bphamhuu, 2 weeks ago) (diff)

No Mysql as does not support sequence

Petacope 9.5

There are two major changes:

  • internally it uses the Spring Boot Framework with Hibernate as object relational mapping data model
  • implements support for the Coverage Schema Implementation (CIS version 1.1) for GridCoverage, RectifiedGridCoverage and ReferenceableGridCoverage.

This brings the following advantages:

  • Petascope can be deployed on more backend DBMS beside PostgreSQL like MySQL, Oracle, DB2, HSQLDB, H2, etc. Postgresql is still the most stable database for deploying Petascope, but the user can easily switch to other databases by changing the configuration in petascope.properties (see list of tested databases).
  • The Spring Boot Framework provides many utilities that aid in quicker development of petascope.
  • Petascope can now start as an embedded web application with an internal embedded Tomcat (i.e: no need to deploy to external Tomcat).

Changes in petascope.properties

The only change is triggered by the support for new JDBC database connections (see list of tested databases). The options following the spring.* convention are used directly by the Spring framework. For example, to use postgresql as a backend database for petascope, the following (default) configuration is specified:

spring.datasource.url=jdbc:postgresql://localhost:5432/petascopedb
spring.datasource.username=petauser
spring.datasource.password=petapasswd
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect

The existing database configuration is kept without any change (note that it is only used during migration, as the source of existing petascopedb or, later on, petascopedb):

metadata_url=jdbc:postgresql://localhost:5432/petascopedb
metadata_user=petauser
metadata_pass=petapasswd

If you have been using the embedded Jetty setup of petascope: the jetty_port option is now renamed to server.port (it is another option directly used by the Spring framework).

old: jetty_port=8080
new: server.port=8080

How to migrate

The database schema in petascope 9.5 has changed, so existing rasdaman installation would need to migrate by executing a migration script update_petascopedb.sh.

This will create a new database petascopedb, and will not modify the existing petascopedb (except rename it to: petascopedb_94_backup).

Developer's guide on update_petascopedb.sh

update_petascopedb.sh is a script that allows to manually apply changes to the schema of petascopedb, like add columns, tables, etc. Every change in petascopedb is marked as an update version (current version is 17). The bigest disadvantage is that it only supports postgresql.

Petascope 9.5 uses Hibernate as ORM data model, and we cannot use update_petascopedb.sh with explicit SQL queries. To support different kinds of databases, we use the Liquibase, which creates the changes for each update in XML and uses that to generate the SQL statements for the target database (e.g: Postgresql, MySql?, HSQLDB, etc). A migration table is added in new Petascope database to lock the application when the process is running (to make sure multiple migrations are not happening at the same time).

To perform migrations it is necessary to implement a small command-line java tool. This tool also uses Spring and Hibernate and can be invoked from the update_petascopedb.sh script. Below we outline the steps for migrating petascopedb.

  1. If using an embedded database like HSQLDB or H2, which does not support multiple connections from different applications, make sure that the (new) petascope 9.5 is stopped.
  2. Execute the migration script: ./update_petascopedb.sh
  3. All coverages in pre 9.5 petascopedb will be read by the old CoverageMetadata model which is imported in the new petascope as a legacy package.
  4. If coverage id doesn't exist in the new petascopedb, a process to translate from old CoverageMetadata model to CIS coverage data model is done and then persisted in petascopedb.
  5. While running the migration, all services of the new petascope web application, such as: WCS, WCPS, WMS, and WCS-T, will not be available to make sure the data is migrated safely.

Rollback

The following parts of petascope are saved to allow for rollback in the unlikely case of an unsuccessful migration.

  • The old petascopedb is preserved (except it is renamed to: petascopedb_94_backup) to allow the user to test the new petascope version first. It is their decision, ultimately, whether to drop the previous database or not.
  • The old petascope web application will be moved from rasdaman.war to rasdaman.war.old in the Tomcat webapps directory
  • The petascope.properties is backed up to petascope.properties.DATE.bak by the update_properties.sh script.

So to rollback:

  1. rename the new petascope (rasdaman.war and petascope.properties to rasdaman.war.new and petascope.properties.new)
  2. rename the old petascope back to the original (rasdaman.war.bak and petascope.properties.DATE.bak to rasdaman.war and petascope.properties).
  3. rename the backuped database from petascopedb_94_backup to petascopedb.
  4. restart Tomcat

List of tested databases

Below is a list of settings for different databases that have been tested successfully with rasdaman for holding the array metadata (not the array data!); replace *db-username* and db-password* with your individual values (it is a major security risk to use the default values coming during installation). Note that only a single datasource can be specified in petascope.properties at any given time, more than one is not supported for simultaneous use. Note also that changing to another database system requires more than just changing these entries, some migration process is involved instead.

# Postgresql (default)
spring.datasource.url=jdbc:postgresql://localhost:5432/petascopedb
spring.datasource.username=*db-username*
spring.datasource.password=*db-password*
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect

# HSQLDB
spring.datasource.url=jdbc:hsqldb:file://home/rasdaman/petascopedb.db
spring.datasource.username=*db-username*
spring.datasource.password=*db-password*

# H2
spring.datasource.url=jdbc:h2:file://home/rasdaman/petascopedb.db;DB_CLOSE_ON_EXIT=FALSE;DB_CLOSE_DELAY=-1
spring.datasource.username=*db-username*
spring.datasource.password=*db-password*
spring.h2.console.enabled=true

3rd party libraries

Petascope 9.5 uses various 3rd party libraries, documented on the overall rasdaman code provenance page.

TODO

  • Support dynamic loading of JDBC drivers by specifying the path to the jar file in petascope.properties