Changes between Version 1 and Version 2 of InstallFromSource

Dec 26, 2015 4:16:59 PM (21 months ago)



  • InstallFromSource

    v1 v2  
    11See [wiki:"InstallFromSourceORIG"].
     6= Compile rasdaman from Source =
     8This is a compilation guide for rasdaman; it has been successfully tested on various Linux systems; for Mac OS X please consider [wiki:OSXInstall this page] in addition.
     10== Definitions ==
     12A couple of variables are used in this guide:
     13* `$USER` refers to the Linux user that will be used to install and manage the rasdaman server. It is recommended that this user is a separate account dedicated for this purpose.
     14 * To create a new user rasdaman for example: `adduser rasdaman`; switch to it from your regular user: `sudo -u rasdaman -i`
     15* `$RMANHOME` is the target directory where rasdaman will be installed, by default: `/usr/local/rasdaman`.
     16 * `$RMANHOME/bin` contains the rasdaman programs and scripts
     17 * `$RMANHOME/etc` contains the configuration
     18 * `$RMANHOME/log` the server logs.
     19* `$RASDATA` is the directory used in rasdaman for storing array data, when not stored in a PostgreSQL database.
     21== Packages required ==
     23You may want to first check the [wiki:Platforms platforms supported].
     25The following packages are required for rasdaman and need to be installed:
     26* '''tools:'''
     27  * git -- needed to clone the rasdaman git repository
     28  * autoconf, automake, m4 -- for generating the configure script and makefiles needed to compile rasdaman
     29  * make, libtool, pkg-config, gawk -- general tools needed to run the configure script and compile rasdaman
     30  * flex, bison, g++, libstdc++ -- required for compilation of the C++ codebase
     31  * [ OpenJDK 6+] -- required for compilation of the Java code (Java client API, petascope OGC frontend)
     32* '''general libraries:'''
     33  * libssl-dev, libncurses5-dev, libedit-dev, libboost-dev (v1.48+), libffi-dev -- required for various system tasks
     34  * libgdal-dev, libtiff-dev, libjpeg8-dev, [ libpng12-dev], [ libnetpbm10-dev] -- required fro data format support
     35* '''database stuff:''' Pick one option below for rasdaman storage:
     36  * [ libecpg-dev] -- required for [ PostgreSQL] (version 9.x is recommended; 8.3.0 to 8.3.6 known to '''not''' work) to hold rasdaman arrays and/or petascope geo metadata
     37  * libsqlite, libsqlite-dev, sqlite3 packages -- required for storing arrays in a file system directory and the rasdaman technical metadata in SQLite; see [wiki:InstallFromSource#SQLiteFilesystembackend details]; note that petascope currently requires PostgreSQL independently from the PostgreSQL / file system array decision - in other words: even if for the array engine you chose to not use PostgreSQL you currently still need to install it for storing the geo metadata making an array an OGC coverage)
     38* '''optional packages''' (use --with-XXX in the configure step to activate use by rasdaman):
     39  * [ libhdf4g-dev] -- required for HDF4 support, configure with `--enable-hdf`
     40  * netcdf4 -- required for NetCDF support, configure with `--enable-netcdf`
     41  * doxygen -- required for HTML system documentationm configure with `--with-docs`
     42  * grpc, protobuf -- required for the rasnet client/server protocol, configure with `--enable-rasnet`
     43  * [ Tomcat] (or another suitable servlet container) -- required for running the petascope and SECORE servlets; default, changed to embedded jetty support with `--enable-jetty`
     44  * performance boosters and additional service components offered by [ rasdaman GmbH]
     45* '''geo data support''' (optional):
     46  * [ libecpg-dev] -- required for [ PostgreSQL] (version 9.x is recommended; 8.3.0 to 8.3.6 known to '''not''' work) to hold petascope geo metadata (this dependency to be removed in future)
     47  * python-dateutil python-lxml python-pip python-gdal python-glob2 python-magic (required by [wiki:WCSTImportGuide wcst_import], a tool for importing geo-referenced data into rasdaman/petascope)
     49== OS specific guidelines ==
     51=== CentOS 7 ===
     54sudo yum install \
     55  git make automake autoconf libtool pkgconfig m4 gawk \
     56  bison gcc gcc-c++ libedit-devel zlib-devel openssl-devel flex flex-devel boost-devel \
     57  gdal-devel libtiff-devel hdf-devel libjpeg-devel libpng12-devel netcdf-devel netcdf-cxx-devel libpng-devel netpbm-devel \
     58  postgresql-devel libsqlite-devel \
     59  gdal-python python-setuptools \
     60  java-1.7.0-openjdk-devel tomcat7
     61sudo easy_install glob2
     64=== Debian 7 & 8 ===
     67sudo apt-get install \
     68  git make automake autotools-dev libtool pkg-config m4 gawk \
     69  bison flex g++ libncurses5-dev libedit-dev libboost-all-dev \
     70  libtiff-dev libgdal1-dev gdal-bin libnetpbm10-dev libjpeg-dev libpng-dev \
     71  postgresql postgresql-contrib libecpg-dev libsqlite-dev sqlite3 \
     72  python-dateutil python-lxml python-gdal python-setuptools \
     73  openjdk-7-jdk tomcat7
     74sudo easy_install install glob2
     77=== Debian 6 ===
     80sudo apt-get install \
     81  git make automake autotools-dev libtool pkg-config m4 gawk \
     82  bison flex g++ comerr-dev libncurses5-dev libedit-dev \
     83  libtiff4-dev gdal-bin libgdal1-dev libnetpbm10-dev \
     84  postgresql libecpg-dev libsqlite-dev sqlite3 \
     85  python-dateutil python-lxml python-gdal python-setuptools
     86  openjdk-6-jdk tomcat6
     87sudo easy_install install glob2
     90Having considered this overview, best continue with [wiki:InstallFromSource rasdaman installation]
     931. Make sure that java and javac are installed and the same version:
     96java -version
     97javac -version
     991. Allow your user to deploy to the tomcat webapps directory:
     102sudo adduser $USER tomcat7
     1041. Add user in postgres:
     107sudo -u postgres createuser -s $USER
     1091. Increase the maximum heap space for Tomcat (>=1GB). Usually this can be done by setting `-Xmx1024m` in JAVA_OPTS in `/etc/default/tomcat7` and then restarting tomcat with `sudo service tomcat7 restart`.
     111== Getting the Source ==
     113The rasdaman source tree can be fetched from the repository. This includes a version control system, effectively making different revisions of the source code available. The following alternatives describe how to clone from the repository and use it:
     114* '''`Option 1`''': Get the most recent development version:
     117git clone git:// # this creates subdirectory rasdaman/
     118cd rasdaman/
     120* '''`Option 2`''': Select a tagged stable release. To activate a [wiki:Versions particular tagged version] use its name prefixed with a "v" (lower case V !), see the example in line 3:
     123git clone git:// # this creates subdirectory rasdaman/
     124cd rasdaman/
     125git checkout v9.1.0-beta2   # to obtain a particular tagged version
     130== Compile and Install rasdaman ==
     1321. Make sure to be logged in as the rasdaman user.
     1331. Choose an installation directory (here: `/home/rasdaman/install`) and further settings. It is best to put these in your `~/.bashrc` for automatic loading upon login; don't forget to reload it first time through `source ~/.bashrc`)
     134 * Debian 8:
     137export RMANHOME=/home/rasdaman/install
     138export RMANSRC=/home/rasdaman/rasdaman # rasdaman sources
     139export RASDATA="$RMANHOME/data"
     140export CATALINA_HOME=/var/lib/tomcat7
     141export PATH=$PATH:$RMANHOME/bin:/usr/lib/postgresql/9.4/bin
     143 * CentOS 7:
     146export RMANHOME=/home/rasdaman/install
     147export RMANSRC=/home/rasdaman/rasdaman # rasdaman sources
     148export RASDATA="$RMANHOME/data"
     149export CATALINA_HOME=/var/lib/tomcat
     150export PATH=$PATH:$RMANHOME/bin
     1521. Make sure to load the above variables: `source ~/.bashrc`
     1531. Configure rasdaman:
     156# generate the configure script and Makefile templates
     157cd "$RMANSRC"
     158autoreconf -fi
     160# run the configure script to generate the Makefiles and check if the required
     161# packages have been installed;
     162# run "./configure --help" to see a full list of the available options
     163./configure --prefix="$RMANHOME" --with-wardir=$CATALINA_HOME/webapps/ --with-netcdf \
     164            --with-default-basedb=sqlite --with-filedatadir="$RASDATA"
     166 * `--prefix` specifies the absolute path where rasdaman will be installed;
     167 * `--with-wardir` specifies that petascope (`rasdaman.war`) and SECORE (`def.war`) will be installed in `$CATALINA_HOME/webapps/`;
     168 * `--with-netcdf` says that we want the netcdf support in rasdaman to be enabled;
     169 * For more details on the rasdaman storage backends (`--with-default-basedb=sqlite --with-filedatadir="$RASDATA"`) see [wiki:RasdamanStorageBackend this page].
     1701. To compile the rasdaman system (for speedup, you may want to add a parallelization option such as `-j2`):
     175 * The following `make` targets are available:
     176  * `all`: compile and link all (default)
     177  * `install`: install system to target location (and compile before, if not up-to date)
     178  * `clean`: remove all object and library files generated
     1791. Install
     182mkdir -p "$RMANHOME"
     183make install
     186Rasdaman and petascope are now installed; in the following sections we will initialize and configure them.
     188== Initialize rasdaman ==
     1901. Initialize the rasdaman schema:
     1961. Start the rasdaman server:
     2011. Insert demo data:
     203#!sh localhost 7001 $RMANHOME/share/rasdaman/examples/images/ rasadmin rasadmin
     2061. Check if demo data have been inserted properly (you should see a list of collection names, such as `mr`, `mr2`, `anthur`):
     209rasql -q 'select c from RAS_COLLECTIONNAMES as c' --out string
     2111. Optionally, run the systemtests to perform a deep checking of the installation:
     214cd systemtests
     215make check
     218Congratulations! With this, rasdaman is running and ready for use. In `$RMANHOME/etc` there are a couple of configuration files that can be modified to adjust the rasdaman server, like logging, host name, etc. Each change requires restarting the rasdaman server:
     225== Initialize petascope ==
     227Petascope is the geo Web service frontend of rasdaman. It adds geo semantics on top of arrays, thereby enabling regular and irregular grids based on the [ OGC coverage standards]. Following successful deployment, petascope accepts OGC W*S requests at URL `http://localhost:8080/rasdaman/ows`.
     229For more details, see the [wiki:PetascopeUserGuide user guide] and [wiki:PetascopeDevGuide developer guide].
     231=== PostgreSQL ===
     233To implement the geo semantics, petascope uses a relational database for the geo-related metadata. Currently, only PostgreSQL is supported; for the future, support of SQLite is planned (in parallel to the rasdaman array engine). This is how to set up PostgreSQL for use by petascope:
     2351. Prepare postgres:
     238# if postgres has not been initialized yet:
     239sudo service postgresql initdb
     241# you may need to allow trust-based access in PostgreSQL
     242# NOTE: On Debian 8 this file is in /etc/postgresql/9.4/main/pg_hba.conf
     243#       On CentOS 7 this file can be found in /var/lib/pgsql/data/pg_hba.conf
     244editor /etc/postgresql/9.4/main/pg_hba.conf
     246# add the below to the pg_hba.conf (before the ident lines):
     249# "local" is for Unix domain socket connections only
     250local   all         all                               trust
     251# IPv4 local connections:
     252host    all         all          trust
     253# IPv6 local connections:
     254host    all         all         ::1/128               trust
     257# reload PostgreSQL so that the new configuration will take effect
     258sudo service postgresql reload
     2601. Add a user to PostgreSQL for petascope:
     262sudo -u postgres createuser -s petauser -P
     2641. Set the metadata_user and metadata_pass in `$RMANHOME/etc/` to the above user and password.
     2651. Initialize the petascope database schema (PostgreSQL needs to be running):
     271=== Running the Servlet ===
     273As petascope is implemented as a Java servlet it needs a servlet container for its operations. Two alternatives are available:
     274* separately install some servlet container, like Tomcat, and deploy petascope there; In case of problems, consult the servlet container log files (such as `/var/log/tomcat7/catalina.out` on Debian 8 and `/usr/share/tomcat/petascope.log` on CentOS 7).
     275* run the petascope service directly, by way of the embedded servlet container, jetty; More information on how to use this option can be found on [wiki:PetascopeEmbedded this page].
     277== Initialize SECORE ==
     279SECORE (Semantic Coordinate Reference System Resolver) is a service that maps CRS URLs to CRS definitions. This component, which is part of the standard rasdaman distribution, is used by the [ Open Geospatial Consortium] (OGC) for operating their official CRS resolver.
     281SECORE, being a servlet like petascope, gets installed automatically unless `--disable-java` is specified at `configure`. The deployment directory of all war files can be set during the `configure` step with the `--with-wardir=DIR` option.
     283For further details on SECORE management, security and troubleshooting see the [wiki:SecoreAdministration administration] and [wiki:SecoreDevGuide developer guide] pages.
     286== Updating the installation ==
     288For staying up-to-date with the latest fixes, you can update the repository and recompile at any time.
     289Assuming you have cloned the rasdaman repository from git in directory `~/rasdaman`, you need to pull the latest commits, recompile and reinstall rasdaman:
     292cd ~/rasdaman
     294git pull           # get latest commits
     295make clean         # clean old compilation artefacts
     296make               # compile the updated code
     298# it is recommended to stop rasdaman/Tomcat before installing the new executables
     299sudo service tomcat7 stop
     302# install rasdaman/petascope, and update the database schemas
     303make install
     307# start services again
     309sudo service tomcat7 start
     312== Next steps ==
     314After successful compilation you may want to proceed to check the [wiki:Documentation documentation], [wiki:FAQ FAQ], and [wiki:KnownIssues known issues list].
     316For more in depth details on rasdaman's installation and configuration, see [browser:manuals_and_examples/manuals/doc-guides/inst-guide.pdf rasdaman installation guide] and [browser:manuals_and_examples/manuals/doc-guides/external-guide-POSTGRESQL.pdf PostgreSQL configuration hints].