wiki:InstallFromSource

Version 42 (modified by dmisev, 3 months ago) (diff)

--

Compile rasdaman from Source

This is a compilation guide for rasdaman; it has been successfully tested on various Linux systems; see the platforms supported for details.

Best use our new installer. This will guide you through the process of installing from source. If, however, you are curious about the details and/or want to have full control over all parameters, follow guidance below.

Settings

A couple of variables are used in this page:

  • $USER refers to the Linux user id that will be used to install and manage the rasdaman server. It is recommended that this user is a separate account dedicated for this purpose, such as rasdaman.
    • To create a new user rasdaman for example: adduser rasdaman; switch to it from your regular user: sudo -u rasdaman -i
  • $RMANHOME is the target directory where rasdaman will be installed; for details, see the description of the rasdaman installation directory structure.

Packages required

The following packages are required for rasdaman and need to be installed:

  • tools:
    • git -- needed to clone the rasdaman git repository
    • autoconf, automake, m4 -- for generating the configure script and makefiles needed to compile rasdaman
    • make, libtool, pkg-config -- general tools needed to run the configure script and compile rasdaman
    • flex, bison, g++, libstdc++ -- required for compilation of the C++ codebase
    • unzip, curl -- for compiling 3rd party dependencies of rasnet (grpc and protobuf)
    • maven2, OpenJDK 6+ -- required for compilation of the Java code (Java client API, petascope OGC frontend)
  • general libraries:
    • libssl-dev, libncurses5-dev, libedit-dev, libboost-dev (v1.48+), libffi-dev -- required for various system tasks
    • libgdal-dev, libtiff-dev, libjpeg8-dev, libpng12-dev, libnetpbm10-dev -- required fro data format support
    • grpc, protobuf -- required for the rasnet client/server protocol
  • database stuff: Pick one option below for rasdaman storage:
    • libecpg-dev -- required for PostgreSQL (version 9.x is recommended; 8.3.0 to 8.3.6 known to not work) to hold rasdaman arrays and/or petascope geo metadata
    • libsqlite, libsqlite-dev, sqlite3 packages -- required for storing arrays in a file system directory and the rasdaman technical metadata in SQLite; see details; note that petascope currently requires PostgreSQL independently from the PostgreSQL / file system array decision - in other words: even if for the array engine you chose to not use PostgreSQL you currently still need to install it for storing the geo metadata making an array an OGC coverage)
  • optional packages (use --with-XXX in the configure step to activate use by rasdaman):
    • libhdf4g-dev -- required for HDF4 support, configure with --with-hdf4
    • netcdf4 -- required for NetCDF support, configure with --with-netcdf
    • doxygen -- required for HTML system documentationm configure with --with-docs
    • Tomcat (or another suitable servlet container) -- required for running the petascope and SECORE servlets; default, changed to embedded jetty support with --enable-jetty
    • r-base, r-base-dev -- required for RRasdaman, a R package providing database interface for rasdaman, configure with --enable-r
    • performance boosters and additional service components offered by rasdaman GmbH
  • geo data support (optional):
    • libecpg-dev -- required for PostgreSQL (version 9.x is recommended; 8.3.0 to 8.3.6 known to not work) to hold petascope geo metadata (this dependency to be removed in future)
    • python-dateutil python-lxml python-pip python-gdal python-glob2 python-magic (required by wcst_import, a tool for importing geo-referenced data into rasdaman/petascope)

Installation commands for the packages is depending on the platform used, here a guidance for some of the most frequently asked for:

  • CentOS:
    sudo yum install \
      git make automake autoconf libtool pkgconfig m4 unzip curl \
      bison gcc gcc-c++ libedit-devel zlib-devel openssl-devel flex flex-devel boost-devel libstdc++-static \
      gdal-devel libtiff-devel hdf-devel libjpeg-devel libpng12-devel netcdf-devel netcdf-cxx-devel libpng-devel netpbm-devel \
      postgresql-devel sqlite-devel \
      gdal-python python-setuptools \
      java-1.7.0-openjdk-devel tomcat maven2
    sudo easy_install glob2
    
  • Debian 7 & 8:
    sudo apt-get install \
      git make automake autotools-dev libtool pkg-config m4 unzip curl \
      bison flex g++ libncurses5-dev libedit-dev libboost-all-dev \
      libtiff-dev libgdal1-dev gdal-bin libnetpbm10-dev libjpeg-dev libpng-dev libnetcdf-c++4-dev \
      postgresql postgresql-contrib libecpg-dev libsqlite-dev sqlite3 \
      python-dateutil python-lxml python-gdal python-setuptools \
      openjdk-7-jdk tomcat7 maven2
    sudo easy_install glob2
    
  • Ubuntu 16.04

{{ #!sh sudo apt-get install \

make libtool gawk autoconf cmake ccache \ bison flex git g++ unzip libboost-all-dev \ libtiff-dev libgdal-dev zlib1g-dev libffi-dev \ libnetcdf-dev libedit-dev libreadline-dev libecpg-dev \ libsqlite3-dev libgrib-api-dev libgrib2c-dev curl \ postgresql postgresql-contrib sqlite3 zlib1g \ gdal-bin python-dev debianutils \ python-dateutil python-lxml python-grib python-pip \ python-gdal netcdf-bin libecpg6 \ libnetcdf-cxx-legacy-dev libnetcdf-c++4 python-netcdf4 \ libboost-all-dev libedit-dev libreadline-dev \ openjdk-8-jdk maven ant libgdal-java \ bc vim-common

}}}

Prerequisites

  1. Make sure that java and javac are installed and the same version:
    java -version
    javac -version
    
  2. Allow your user to deploy to the tomcat webapps directory (only if you (i) want to run the petascope or SECORE servlets and (ii) want to use tomcat, not the embedded jetty servlet container):
    sudo adduser $USER tomcat7
    
  3. If PostgreSQL support is needed (see options): Add user in postgres:
    sudo -u postgres createuser -s $USER
    
  4. If Tomcat is used: Increase the maximum heap space for Tomcat (>=1GB). Usually this can be done by setting -Xmx1024m in JAVA_OPTS in /etc/default/tomcat7 and then restarting tomcat with sudo service tomcat7 restart.

Getting the Source

The rasdaman source tree can be fetched from the repository. This includes a version control system, effectively making different revisions of the source code available. The following alternatives describe how to clone from the repository and use it:

  • Option 1: Get the most recent development version:
    git clone git://rasdaman.org/rasdaman.git # this creates subdirectory rasdaman/
    cd rasdaman/
    
  • Option 2: Select a tagged stable release. To activate a particular tagged version use its name prefixed with a "v" (lower case V !), see the example in line 3:
    git clone git://rasdaman.org/rasdaman.git # this creates subdirectory rasdaman/
    cd rasdaman/
    git checkout v9.2.0-beta1   # to obtain a particular tagged version
    

Compile and Install rasdaman

  1. Make sure to be logged in as the rasdaman user.
  2. Choose an installation directory (here: /opt/rasdaman) and further settings. It is best to put these in your ~/.bashrc for automatic loading upon login; don't forget to reload it first time through source ~/.bashrc)
    • Debian 8:
      export RMANHOME=/opt/rasdaman
      export RMANSRC=/home/rasdaman/rasdaman # rasdaman sources
      export RASDATA="$RMANHOME/data"
      export CATALINA_HOME=/var/lib/tomcat7
      export PATH=$PATH:$RMANHOME/bin:/usr/lib/postgresql/9.4/bin
      
    • CentOS 7:
      export RMANHOME=/opt/rasdaman
      export RMANSRC=/home/rasdaman/rasdaman # rasdaman sources
      export RASDATA="$RMANHOME/data"
      export CATALINA_HOME=/var/lib/tomcat
      export PATH=$PATH:$RMANHOME/bin
      
  3. Make sure to load the above variables: source ~/.bashrc
  4. Configure and build rasdaman:
    1. Using CMake (v3+)
    2. Using autotools (deprecated)

Rasdaman and petascope are now installed; in the following sections we will initialize and configure them.

Initialize rasdaman

  1. Initialize the rasdaman schema:
    create_db.sh
    update_db.sh
    
  2. Start the rasdaman server:
    start_rasdaman.sh
    
  3. Insert demo data:
    rasdaman_insertdemo.sh localhost 7001 $RMANHOME/share/rasdaman/examples/images/ rasadmin rasadmin
    
  4. Check if demo data have been inserted properly (you should see a list of collection names, such as mr, mr2, anthur):
    rasql -q 'select c from RAS_COLLECTIONNAMES as c' --out string
    
  5. Optionally, run the system tests via the build directory to perform a deep checking of the installation:
    make check
    

Congratulations! With this, rasdaman is running and ready for use. In $RMANHOME/etc there are a couple of configuration files that can be modified to adjust the rasdaman server, like logging, host name, etc. Each change requires restarting the rasdaman server:

stop_rasdaman.sh
start_rasdaman.sh

Initialize geo service support

Petascope is the geo Web service frontend of rasdaman. It adds geo semantics on top of arrays, thereby enabling regular and irregular grids based on the OGC coverage standards. To activate this functionality, perform the steps of the petascope installation guide.

Following successful deployment, petascope accepts OGC W*S requests at URL http://localhost:8080/rasdaman/ows.

Next steps

After successful compilation you may want to proceed to check the documentation, FAQ, and known issues list.

For more in depth details on rasdaman's installation and configuration, see rasdaman installation guide and PostgreSQL configuration hints.

For updating an existing installation see the system update guide.