wiki:InstallFromSource/v9.1

Version 3 (modified by dmisev, 21 months ago) (diff)

--

(Note: this page is outdated; see wiki:InstallFromSource for the most up-to-date version)

rasdaman installation guide

This is an installation guide for rasdaman, which works generally across various Linux systems, but includes directions for several flavors (Debian and CentOS) to simplify the installation.

Administrative configuration

A couple of variables are used in this guide:

  • $USER refers to the Linux user that will be used to install and manage the rasdaman server. It is recommended that this user is a separate account dedicated for this purpose.
    • To create a new user rasdaman for example: adduser rasdaman
    • Switch to from your regular user: sudo -u rasdaman -i
  • $RMANHOME is the target directory where rasdaman will be installed.
    • $RMANHOME/bin contains the rasdaman programs and scripts
    • $RMANHOME/etc contains the configuration
    • $RMANHOME/log the server logs.
  • $RASDATA is the directory used in rasdaman for storing data.

To prepare for an installation of rasdaman:

  1. Install required packages:
    • Debian 8
      sudo apt-get install \
        git make automake autotools-dev libtool pkg-config m4 gawk \
        bison flex g++ libncurses5-dev libedit-dev libboost-all-dev \
        libtiff-dev libgdal1-dev gdal-bin libnetpbm10-dev libjpeg-dev libpng-dev \
        postgresql postgresql-contrib libecpg-dev libsqlite-dev sqlite3 \
        python-dateutil python-lxml python-gdal python-setuptools \
        openjdk-7-jdk tomcat7
      sudo easy_install install glob2
      
    • CentOS 7
      sudo yum install \
        git make automake autoconf libtool pkgconfig m4 gawk \
        bison gcc gcc-c++ libedit-devel zlib-devel openssl-devel flex flex-devel boost-devel \
        gdal-devel libtiff-devel hdf-devel libjpeg-devel libpng12-devel netcdf-devel netcdf-cxx-devel libpng-devel netpbm-devel \
        postgresql-devel libsqlite-devel \
        gdal-python python-setuptools \
        java-1.7.0-openjdk-devel tomcat7
      sudo easy_install glob2
      
    • If you have a different distribution, please see this page for a general list of required packages.
  2. Make sure that java and javac are installed and the same version:
    java -version
    javac -version
    
  3. Allow your user to deploy to the tomcat webapps directory:
    sudo adduser $USER tomcat7
    
  4. Add user in postgres:
    sudo -u postgres createuser -s $USER
    
  5. Increase the maximum heap space for tomcat (>=1GB). Usually this can be done by setting -Xmx1024m in JAVA_OPTS in /etc/default/tomcat7 and then restarting tomcat with sudo service tomcat7 restart.

Compile and install rasdaman

  1. Make sure to be logged in as the rasdaman user.
  2. Choose an installation directory (here: /home/rasdaman/install) and further settings. It is best to put these in your ~/.bashrc for automatic loading upon login; don't forget to reload it first time through source ~/.bashrc)
    • Debian 8:
      export RMANHOME=/home/rasdaman/install
      export RASDATA="$RMANHOME/data"
      export CATALINA_HOME=/var/lib/tomcat7
      export PATH=$PATH:$RMANHOME/bin:/usr/lib/postgresql/9.4/bin
      
    • CentOS 7:
      export RMANHOME=/home/rasdaman/install
      export RASDATA="$RMANHOME/data"
      export CATALINA_HOME=/var/lib/tomcat
      export PATH=$PATH:$RMANHOME/bin
      
  3. Make sure to load the above variables: source ~/.bashrc
  4. Get and configure rasdaman:
    git clone git://rasdaman.org/rasdaman.git
    cd rasdaman
    
    # generate the configure script and Makefile templates
    autoreconf -fi
    
    # run the configure script to generate the Makefiles and check if the required 
    # packages have been installed;
    # run "./configure --help" to see a full list of the available options
    ./configure --prefix="$RMANHOME" --with-wardir=$CATALINA_HOME/webapps/ --with-netcdf \
                --with-default-basedb=sqlite --with-filedatadir="$RASDATA"
    
  1. Compile
    make -j2
    
  2. Install
    mkdir -p "$RMANHOME"
    make install
    

Rasdaman and petascope are now installed, in the next section we will initialize and configure them, before they can be started and used.

Initialize rasdaman

  1. Initialize the rasdaman schema (RASBASE):
    create_db.sh
    update_db.sh
    
  2. Start the rasdaman server:
    start_rasdaman.sh
    
  3. Insert demo data:
    rasdaman_insertdemo.sh localhost 7001 $RMANHOME/share/rasdaman/examples/images/ rasadmin rasadmin
    
  4. Check if demo data has been inserted
    rasql -q 'select c from RAS_COLLECTIONNAMES as c' --out string
    
  5. If you want a deep checking of the installation, you can run the systemtests:
    cd systemtests
    make check
    

With this rasdaman is running and ready for use. In $RMANHOME/etc there are a couple of configuration files that can be modified to adjust the rasdaman server, like logging, host name, etc. Each change requires restarting the rasdaman server:

stop_rasdaman.sh
start_rasdaman.sh

Initialize petascope

petascope is the OGC web-frontend for rasdaman, implemented as a Java servlet. See the user guide for more details and a list of the supported OGC standards.

  1. Prepare postgres:
    # if postgres has not been initialized yet:
    sudo service postgresql initdb
    
    # you may need to allow trust-based access in PostgreSQL
    # NOTE: On Debian 8 this file is in /etc/postgresql/9.4/main/pg_hba.conf
    #       On CentOS 7 this file can be found in /var/lib/pgsql/data/pg_hba.conf
    editor /etc/postgresql/9.4/main/pg_hba.conf
    
    # add the below to the pg_hba.conf (before the ident lines):
    
    #------------------------------------------------------------
    # "local" is for Unix domain socket connections only
    local   all         all                               trust
    # IPv4 local connections:
    host    all         all         127.0.0.1/32          trust
    # IPv6 local connections:
    host    all         all         ::1/128               trust
    #------------------------------------------------------------
    
    # reload PostgreSQL so that the new configuration will take effect
    sudo service postgresql reload
    
  2. Add a user to PostgreSQL for petascope:
    sudo -u postgres createuser -s petauser -P
    
  3. Set the metadata_user and metadata_pass in $RMANHOME/etc/petascope.properties to the above user and password.
  4. Initialize the petascope database schema (PostgreSQL needs to be running):
    update_petascopedb.sh
    
  5. Restart Tomcat:
    sudo service tomcat7 restart
    

Petascope should be running now, and can be accessed at http://localhost:8080/rasdaman/ows.

In case of problems it is best to look at the Tomcat log file, in /var/log/tomcat7/catalina.out on Debian 8, or /usr/share/tomcat/petascope.log on CentOS 7.

rasnet communication protocol

Up to and including rasdaman v9.2, RNP is the default client/server communication protocol used; in the near-future version rasnet will become the default communication protocol, RNP will be obsoleted (as RPC and HTTP are already).

  1. Dependencies that need to be additionally installed for rasnet:
  2. It is best to install GRPC, GRPC-Java and Google Protocol Buffers from source. We recommend that you read each software's installation instructions. The following instructions have been tested on Debian 8 and Ubuntu 14.04, but should work for similar distros.
    1. Install GRPC:
      git clone https://github.com/grpc/grpc.git
      cd grpc
      git checkout release-0_12 
      git submodule update --init
      
      # You must install Google Protobuf manually because it is not installed 
      # automatically when doing make && make install in $GRPC_SRC
      cd third_party/protobuf
      ./autogen.sh
      ./configure
      make
      sudo make install
      
      # You can now resume the installation of GRPC and its other dependencies.
      cd -
      make
      sudo make install
      
    2. To use a Java client or petascope you require the GRPC-Java compiler plugin. This plugin is used to generate Java code from the Protocol Buffers specification. Detailed instructions for installing the plugin can be found here. For version GRPC-Java version 0.9.0 the following commands are sufficient. After compiling, the plugin can be found in compiler/build/binaries/java_pluginExecutable/.
      git clone https://github.com/grpc/grpc-java
      cd grpc-java/compiler
      ../gradlew java_pluginExecutable
      
  3. rasdaman can now be configured to use rasnet, if the specified dependencies are installed in their default location (including PATH settings):
    ./configure ... --enable-rasnet
    
  4. If the dependencies are installed in a custom location their paths must be communicated additionally to the configure script:
    • Boost:
      ./configure ... --enable-rasnet --with-boost=<path to boost installation>
      
    • Google Protobuf:
      ./configure ... --enable-rasnet PKG_CONFIG_PATH=<path to google protobuf installation>/lib/pkgconfig
      
    • GRPC C++ Plugin:
      --with-grpc-cpp-plugin=<path to grpc_cpp_plugin>
      
    • GRPC-Java Plugin:
      1. Use the ./configure flag (recommended).
        --with-grpc-java-plugin=<path to protoc-gen-grpc-java>
        
      2. Create a symbolic link in /usr/local/bin
         grpc-java/compiler/build/binaries/java_pluginExecutable/protoc-gen-grpc-java in /usr/local/bin
        
  5. rasdaman can now be reinstalled in the usual way (make, make install), but before doing that it is necessary to run:
    stop_rasdaman.sh
    make clean
    

Note: This protocol is in Beta release.

rasgeo (deprecated)

rasgeo is a client component of petascope that is able to easily import data into rasdaman along with the geo-metadata into petascope, provided that the data can be read by GDAL. This has been now superceded by the wcst_import tool.

To setup rasgeo its configuration file (~/.rasdaman/rasconnect) needs to be updated as needed:

# Example file:
-------------------------------------------------
host=localhost
rasport=7001
pgport=5432
rasdbname=RASBASE
petadbname=petascopedb

# RASBASE login
rasuser=rasdaman
raspassword=rasdaman

# petascopedb login
petauser=petauser
petapassword=petapasswd

# for rasdaman 9.0
# rasdaman login
rasloginuser=rasadmin
rasloginpassword=rasadmin
------------------------------------------------

rview (deprecated)

rview is a desktop visualization client component of rasdaman, that communicates with rasdaman via RPC, so rpcbind/portmapper should be installed and running in order to use rview.

The rpcbind service has to be started in insecure mode, or rasdaman has to be started as root, in order for rview to be able to connect to rasdaman.

To start rpcbind in insecure mode:

# note that the service script location may vary among different Linux distributions
editor /etc/init.d/rpcbind

# or in recent Ubuntu's
editor /etc/init/portmap.conf

# then add '-i' to the OPTIONS variable

After the rpcbind is restarted, restart rasdaman and then rview can be opened with

cd $RMANHOME/bin
./rview

Important note: Rview is deprecated and unmaintained. For building clients use one of the Web client or the raswct library instead. In fact, there are strong reasons not to use rview: As highlighted using the RPC communication protocol is highly insecure and not recommended. Further, RPC communication is deprecated and will soon be replaced (together with HTTP and RNP) by rasnet.