Remove rasdl

[Dimitar Misev]

rasdl is obsolete with the introduction of type management in rasql. The only important functionality now is RASBASE initialization (option --createdatabase).

I'm inclined to remove rasdl completely in v10, and move RASBASE initialization to rasserver, which could be done in two ways:

1. have an explicit option, rasserver --initdb for example
2. automatically initialize if RASBASE is not present

I'd be more in favor of the first option. Any opinions?

[Peter Baumann]

con 1: DB definition "hardwired" in rasmgr.conf; likely several more options to be added (eg, data dir)
pro 2: convenient
con 2: no way to influence (eg, directory)

Alt 3: add "create database" to rasql, following SQL syntax. Also, add a "drop database"
con 3: likely more work
pro 3: QL can handle all situations in life, parameters easy to add

[Dimitar Misev]

Here's the way some typical DBMS do what we do with rasdl --createdatabase:

• ​PostgreSQL: initdb [option...] [--pgdata | -D] directory
• ​MySQL: mysql_install_db handles initialization tasks that must be performed before the MySQL server, mysqld, is ready to use; mysql_install_db is deprecated as of MySQL 5.7.6 because its functionality has been integrated into mysqld, the MySQL server. To initialize a MySQL installation, invoke mysqld with the —initialize or —initialize-insecure option.
• ​MonetDB: monetdbd create /path/to/mydbfarm

[Peter Baumann]

As per Wikipedia: "The CREATE command is used to establish a new database, table, index, or stored procedure."

This is what SQL Server does (​https://msdn.microsoft.com/en-us/library/ms176061.aspx):

CREATE DATABASE Sales
ON 
( NAME = Sales_dat,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\DATA\saledat.mdf',
    SIZE = 10,
    MAXSIZE = 50,
    FILEGROWTH = 5 )
LOG ON
( NAME = Sales_log,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\DATA\salelog.ldf',
    SIZE = 5MB,
    MAXSIZE = 25MB,
    FILEGROWTH = 5MB ) ;

Oracle, as usual, is more complicated (​http://docs.oracle.com/cd/B28359_01/server.111/b28310/create003.htm); at some time you will say:

CREATE DATABASE mynewdb
   USER SYS IDENTIFIED BY sys_password
   USER SYSTEM IDENTIFIED BY system_password
   LOGFILE GROUP 1 ('/u01/app/oracle/oradata/mynewdb/redo01.log') SIZE 100M,
           GROUP 2 ('/u01/app/oracle/oradata/mynewdb/redo02.log') SIZE 100M,
           GROUP 3 ('/u01/app/oracle/oradata/mynewdb/redo03.log') SIZE 100M
   MAXLOGFILES 5
   MAXLOGMEMBERS 5
   MAXLOGHISTORY 1
   MAXDATAFILES 100
   CHARACTER SET US7ASCII
   NATIONAL CHARACTER SET AL16UTF16
   EXTENT MANAGEMENT LOCAL
   DATAFILE '/u01/app/oracle/oradata/mynewdb/system01.dbf' SIZE 325M REUSE
   SYSAUX DATAFILE '/u01/app/oracle/oradata/mynewdb/sysaux01.dbf' SIZE 325M REUSE
   DEFAULT TABLESPACE users
      DATAFILE '/u01/app/oracle/oradata/mynewdb/users01.dbf'
      SIZE 500M REUSE AUTOEXTEND ON MAXSIZE UNLIMITED
   DEFAULT TEMPORARY TABLESPACE tempts1
      TEMPFILE '/u01/app/oracle/oradata/mynewdb/temp01.dbf'
      SIZE 20M REUSE
   UNDO TABLESPACE undotbs
      DATAFILE '/u01/app/oracle/oradata/mynewdb/undotbs01.dbf'
      SIZE 200M REUSE AUTOEXTEND ON MAXSIZE UNLIMITED;

IBM DB2 (​http://www-01.ibm.com/support/knowledgecenter/SSEPEK_11.0.0/com.ibm.db2z11.doc.sqlref/src/tpc/db2z_sql_createdatabase.dita):

   CREATE DATABASE DSN8D11P
     STOGROUP DSN8G110
     BUFFERPOOL BP8K1
     INDEXBP BP2;

[Peter Baumann]

BTW, PostgreSQL (​http://www.postgresql.org/docs/9.0/static/sql-createdatabase.html):

CREATE DATABASE sales OWNER salesapp TABLESPACE salesspace;

"The program createdb is a wrapper program around this command, provided for convenience."

[Dimitar Misev]

The "create database" does not exist in standard SQL actually if I'm not mistaken, that's why there's all this variety.

Currently in rasmgr.conf we can have something like this:

define dbh rasdaman_host -connect /path/to/RASBASE

define db RASBASE -dbh rasdaman_host

So we can allow connecting to different logical databases, however they must refer to separate physical RASBASE rasdaman_host (whether in postgres or sqlite).

Ideally we should support multiple define db per single dbh, this is how it works in the typical DBMS. What would the effect of "create database" be in this case though? It cannot be creating the physical RASBASE containing the RAS_* tables, these concepts are separated in this case.

[Peter Baumann]

right, just checked: "create database" is not in ISO SQL.

[Dimitar Misev]

To summarize: currently we use rasdl --createdatabase to initialize RASBASE (simply create the RAS_* tables). It's not worth it to have a separate executable just for this purpose, so I suggest to migrate this functionality to another place and remove rasdl altogether.
Note that initializing RASBASE is different from the CREATE DATABASE concept in SQL databases btw.

In my opinion there is no reason this functionality should be compiled into an executable. All that it does is creating tables and indexes. It could simply be an SQL script in update_db.sh.

Any thoughts?

[Vlad Merticariu]

Maybe we can put the functionality in create_db.sh?

[Dimitar Misev]

Yes exactly.

[Peter Baumann]

sounds good to me, as it does not affect user visible behavior in the core task of database creation.

[Dimitar Misev]

Currently all functionality of rasdl is available via rasserver (i.e. directql). rasdl is not used anymore in create_db.sh or anywhere else in rasdaman. So it's a good time to remove the code as well the relevant documentation.

[Peter Baumann]

increasing prio as obviously libeccodes and other difficulties bring along problems as long as we have rasdl.

[Dimitar Misev]

libeccodes is a dependency of conversion = decode($1, "GRIB"), so not directly of rasdl.