Changes between Version 1 and Version 2 of InstallFromSource


Ignore:
Timestamp:
Dec 26, 2015 4:16:59 PM (21 months ago)
Author:
pbaumann
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • InstallFromSource

    v1 v2  
    11See [wiki:"InstallFromSourceORIG"].
     2
     3[[PageOutline]]
     4[[TracNav(TOC)]]
     5
     6= Compile rasdaman from Source =
     7
     8This is a compilation guide for rasdaman; it has been successfully tested on various Linux systems; for Mac OS X please consider [wiki:OSXInstall this page] in addition.
     9
     10== Definitions ==
     11
     12A couple of variables are used in this guide:
     13* `$USER` refers to the Linux user that will be used to install and manage the rasdaman server. It is recommended that this user is a separate account dedicated for this purpose.
     14 * To create a new user rasdaman for example: `adduser rasdaman`; switch to it from your regular user: `sudo -u rasdaman -i`
     15* `$RMANHOME` is the target directory where rasdaman will be installed, by default: `/usr/local/rasdaman`.
     16 * `$RMANHOME/bin` contains the rasdaman programs and scripts
     17 * `$RMANHOME/etc` contains the configuration
     18 * `$RMANHOME/log` the server logs.
     19* `$RASDATA` is the directory used in rasdaman for storing array data, when not stored in a PostgreSQL database.
     20
     21== Packages required ==
     22
     23You may want to first check the [wiki:Platforms platforms supported].
     24
     25The following packages are required for rasdaman and need to be installed:
     26* '''tools:'''
     27  * git -- needed to clone the rasdaman git repository
     28  * autoconf, automake, m4 -- for generating the configure script and makefiles needed to compile rasdaman
     29  * make, libtool, pkg-config, gawk -- general tools needed to run the configure script and compile rasdaman
     30  * flex, bison, g++, libstdc++ -- required for compilation of the C++ codebase
     31  * [http://openjdk.java.net/ OpenJDK 6+] -- required for compilation of the Java code (Java client API, petascope OGC frontend)
     32* '''general libraries:'''
     33  * libssl-dev, libncurses5-dev, libedit-dev, libboost-dev (v1.48+), libffi-dev -- required for various system tasks
     34  * libgdal-dev, libtiff-dev, libjpeg8-dev, [http://packages.debian.org/en/lenny/libpng12-dev libpng12-dev], [http://packages.debian.org/en/lenny/libnetpbm10-dev libnetpbm10-dev] -- required fro data format support
     35* '''database stuff:''' Pick one option below for rasdaman storage:
     36  * [http://packages.debian.org/en/lenny/libecpg-dev libecpg-dev] -- required for [http://www.postgresql.org PostgreSQL] (version 9.x is recommended; 8.3.0 to 8.3.6 known to '''not''' work) to hold rasdaman arrays and/or petascope geo metadata
     37  * libsqlite, libsqlite-dev, sqlite3 packages -- required for storing arrays in a file system directory and the rasdaman technical metadata in SQLite; see [wiki:InstallFromSource#SQLiteFilesystembackend details]; note that petascope currently requires PostgreSQL independently from the PostgreSQL / file system array decision - in other words: even if for the array engine you chose to not use PostgreSQL you currently still need to install it for storing the geo metadata making an array an OGC coverage)
     38* '''optional packages''' (use --with-XXX in the configure step to activate use by rasdaman):
     39  * [http://packages.debian.org/en/lenny/libhdf4g-dev libhdf4g-dev] -- required for HDF4 support, configure with `--enable-hdf`
     40  * netcdf4 -- required for NetCDF support, configure with `--enable-netcdf`
     41  * doxygen -- required for HTML system documentationm configure with `--with-docs`
     42  * grpc, protobuf -- required for the rasnet client/server protocol, configure with `--enable-rasnet`
     43  * [http://tomcat.apache.org/ Tomcat] (or another suitable servlet container) -- required for running the petascope and SECORE servlets; default, changed to embedded jetty support with `--enable-jetty`
     44  * performance boosters and additional service components offered by [http://www.rasdaman.com rasdaman GmbH]
     45* '''geo data support''' (optional):
     46  * [http://packages.debian.org/en/lenny/libecpg-dev libecpg-dev] -- required for [http://www.postgresql.org PostgreSQL] (version 9.x is recommended; 8.3.0 to 8.3.6 known to '''not''' work) to hold petascope geo metadata (this dependency to be removed in future)
     47  * python-dateutil python-lxml python-pip python-gdal python-glob2 python-magic (required by [wiki:WCSTImportGuide wcst_import], a tool for importing geo-referenced data into rasdaman/petascope)
     48
     49== OS specific guidelines ==
     50
     51=== CentOS 7 ===
     52{{{
     53#!sh
     54sudo yum install \
     55  git make automake autoconf libtool pkgconfig m4 gawk \
     56  bison gcc gcc-c++ libedit-devel zlib-devel openssl-devel flex flex-devel boost-devel \
     57  gdal-devel libtiff-devel hdf-devel libjpeg-devel libpng12-devel netcdf-devel netcdf-cxx-devel libpng-devel netpbm-devel \
     58  postgresql-devel libsqlite-devel \
     59  gdal-python python-setuptools \
     60  java-1.7.0-openjdk-devel tomcat7
     61sudo easy_install glob2
     62}}}
     63
     64=== Debian 7 & 8 ===
     65{{{
     66#!sh
     67sudo apt-get install \
     68  git make automake autotools-dev libtool pkg-config m4 gawk \
     69  bison flex g++ libncurses5-dev libedit-dev libboost-all-dev \
     70  libtiff-dev libgdal1-dev gdal-bin libnetpbm10-dev libjpeg-dev libpng-dev \
     71  postgresql postgresql-contrib libecpg-dev libsqlite-dev sqlite3 \
     72  python-dateutil python-lxml python-gdal python-setuptools \
     73  openjdk-7-jdk tomcat7
     74sudo easy_install install glob2
     75}}}
     76
     77=== Debian 6 ===
     78{{{
     79#!sh
     80sudo apt-get install \
     81  git make automake autotools-dev libtool pkg-config m4 gawk \
     82  bison flex g++ comerr-dev libncurses5-dev libedit-dev \
     83  libtiff4-dev gdal-bin libgdal1-dev libnetpbm10-dev \
     84  postgresql libecpg-dev libsqlite-dev sqlite3 \
     85  python-dateutil python-lxml python-gdal python-setuptools
     86  openjdk-6-jdk tomcat6
     87sudo easy_install install glob2
     88}}}
     89
     90Having considered this overview, best continue with [wiki:InstallFromSource rasdaman installation]
     91.
     92
     931. Make sure that java and javac are installed and the same version:
     94{{{
     95#!sh
     96java -version
     97javac -version
     98}}}
     991. Allow your user to deploy to the tomcat webapps directory:
     100{{{
     101#!sh
     102sudo adduser $USER tomcat7
     103}}}
     1041. Add user in postgres:
     105{{{
     106#!sh
     107sudo -u postgres createuser -s $USER
     108}}}
     1091. Increase the maximum heap space for Tomcat (>=1GB). Usually this can be done by setting `-Xmx1024m` in JAVA_OPTS in `/etc/default/tomcat7` and then restarting tomcat with `sudo service tomcat7 restart`.
     110
     111== Getting the Source ==
     112
     113The rasdaman source tree can be fetched from the repository. This includes a version control system, effectively making different revisions of the source code available. The following alternatives describe how to clone from the repository and use it:
     114* '''`Option 1`''': Get the most recent development version:
     115{{{
     116#!sh
     117git clone git://rasdaman.org/rasdaman.git # this creates subdirectory rasdaman/
     118cd rasdaman/
     119}}}
     120* '''`Option 2`''': Select a tagged stable release. To activate a [wiki:Versions particular tagged version] use its name prefixed with a "v" (lower case V !), see the example in line 3:
     121{{{
     122#!sh
     123git clone git://rasdaman.org/rasdaman.git # this creates subdirectory rasdaman/
     124cd rasdaman/
     125git checkout v9.1.0-beta2   # to obtain a particular tagged version
     126}}}
     127
     128
     129
     130== Compile and Install rasdaman ==
     131
     1321. Make sure to be logged in as the rasdaman user.
     1331. Choose an installation directory (here: `/home/rasdaman/install`) and further settings. It is best to put these in your `~/.bashrc` for automatic loading upon login; don't forget to reload it first time through `source ~/.bashrc`)
     134 * Debian 8:
     135{{{
     136#!sh
     137export RMANHOME=/home/rasdaman/install
     138export RMANSRC=/home/rasdaman/rasdaman # rasdaman sources
     139export RASDATA="$RMANHOME/data"
     140export CATALINA_HOME=/var/lib/tomcat7
     141export PATH=$PATH:$RMANHOME/bin:/usr/lib/postgresql/9.4/bin
     142}}}
     143 * CentOS 7:
     144{{{
     145#!sh
     146export RMANHOME=/home/rasdaman/install
     147export RMANSRC=/home/rasdaman/rasdaman # rasdaman sources
     148export RASDATA="$RMANHOME/data"
     149export CATALINA_HOME=/var/lib/tomcat
     150export PATH=$PATH:$RMANHOME/bin
     151}}}
     1521. Make sure to load the above variables: `source ~/.bashrc`
     1531. Configure rasdaman:
     154{{{
     155#!sh
     156# generate the configure script and Makefile templates
     157cd "$RMANSRC"
     158autoreconf -fi
     159
     160# run the configure script to generate the Makefiles and check if the required
     161# packages have been installed;
     162# run "./configure --help" to see a full list of the available options
     163./configure --prefix="$RMANHOME" --with-wardir=$CATALINA_HOME/webapps/ --with-netcdf \
     164            --with-default-basedb=sqlite --with-filedatadir="$RASDATA"
     165}}}
     166 * `--prefix` specifies the absolute path where rasdaman will be installed;
     167 * `--with-wardir` specifies that petascope (`rasdaman.war`) and SECORE (`def.war`) will be installed in `$CATALINA_HOME/webapps/`;
     168 * `--with-netcdf` says that we want the netcdf support in rasdaman to be enabled;
     169 * For more details on the rasdaman storage backends (`--with-default-basedb=sqlite --with-filedatadir="$RASDATA"`) see [wiki:RasdamanStorageBackend this page].
     1701. To compile the rasdaman system (for speedup, you may want to add a parallelization option such as `-j2`):
     171{{{
     172#!sh
     173make
     174}}}
     175 * The following `make` targets are available:
     176  * `all`: compile and link all (default)
     177  * `install`: install system to target location (and compile before, if not up-to date)
     178  * `clean`: remove all object and library files generated
     1791. Install
     180{{{
     181#!sh
     182mkdir -p "$RMANHOME"
     183make install
     184}}}
     185
     186Rasdaman and petascope are now installed; in the following sections we will initialize and configure them.
     187
     188== Initialize rasdaman ==
     189
     1901. Initialize the rasdaman schema:
     191{{{
     192#!sh
     193create_db.sh
     194update_db.sh
     195}}}
     1961. Start the rasdaman server:
     197{{{
     198#!sh
     199start_rasdaman.sh
     200}}}
     2011. Insert demo data:
     202{{{
     203#!sh
     204rasdaman_insertdemo.sh localhost 7001 $RMANHOME/share/rasdaman/examples/images/ rasadmin rasadmin
     205}}}
     2061. Check if demo data have been inserted properly (you should see a list of collection names, such as `mr`, `mr2`, `anthur`):
     207{{{
     208#!sh
     209rasql -q 'select c from RAS_COLLECTIONNAMES as c' --out string
     210}}}
     2111. Optionally, run the systemtests to perform a deep checking of the installation:
     212{{{
     213#!sh
     214cd systemtests
     215make check
     216}}}
     217
     218Congratulations! With this, rasdaman is running and ready for use. In `$RMANHOME/etc` there are a couple of configuration files that can be modified to adjust the rasdaman server, like logging, host name, etc. Each change requires restarting the rasdaman server:
     219{{{
     220#!sh
     221stop_rasdaman.sh
     222start_rasdaman.sh
     223}}}
     224
     225== Initialize petascope ==
     226
     227Petascope is the geo Web service frontend of rasdaman. It adds geo semantics on top of arrays, thereby enabling regular and irregular grids based on the [http://external.opengeospatial.org/twiki_public/CoveragesDWG/WebHome OGC coverage standards]. Following successful deployment, petascope accepts OGC W*S requests at URL `http://localhost:8080/rasdaman/ows`.
     228
     229For more details, see the [wiki:PetascopeUserGuide user guide] and [wiki:PetascopeDevGuide developer guide].
     230
     231=== PostgreSQL ===
     232
     233To implement the geo semantics, petascope uses a relational database for the geo-related metadata. Currently, only PostgreSQL is supported; for the future, support of SQLite is planned (in parallel to the rasdaman array engine). This is how to set up PostgreSQL for use by petascope:
     234
     2351. Prepare postgres:
     236{{{
     237#!sh
     238# if postgres has not been initialized yet:
     239sudo service postgresql initdb
     240
     241# you may need to allow trust-based access in PostgreSQL
     242# NOTE: On Debian 8 this file is in /etc/postgresql/9.4/main/pg_hba.conf
     243#       On CentOS 7 this file can be found in /var/lib/pgsql/data/pg_hba.conf
     244editor /etc/postgresql/9.4/main/pg_hba.conf
     245
     246# add the below to the pg_hba.conf (before the ident lines):
     247
     248#------------------------------------------------------------
     249# "local" is for Unix domain socket connections only
     250local   all         all                               trust
     251# IPv4 local connections:
     252host    all         all         127.0.0.1/32          trust
     253# IPv6 local connections:
     254host    all         all         ::1/128               trust
     255#------------------------------------------------------------
     256
     257# reload PostgreSQL so that the new configuration will take effect
     258sudo service postgresql reload
     259}}}
     2601. Add a user to PostgreSQL for petascope:
     261{{{
     262sudo -u postgres createuser -s petauser -P
     263}}}
     2641. Set the metadata_user and metadata_pass in `$RMANHOME/etc/petascope.properties` to the above user and password.
     2651. Initialize the petascope database schema (PostgreSQL needs to be running):
     266{{{
     267#!sh
     268update_petascopedb.sh
     269}}}
     270
     271=== Running the Servlet ===
     272
     273As petascope is implemented as a Java servlet it needs a servlet container for its operations. Two alternatives are available:
     274* separately install some servlet container, like Tomcat, and deploy petascope there; In case of problems, consult the servlet container log files (such as `/var/log/tomcat7/catalina.out` on Debian 8 and `/usr/share/tomcat/petascope.log` on CentOS 7).
     275* run the petascope service directly, by way of the embedded servlet container, jetty; More information on how to use this option can be found on [wiki:PetascopeEmbedded this page].
     276
     277== Initialize SECORE ==
     278
     279SECORE (Semantic Coordinate Reference System Resolver) is a service that maps CRS URLs to CRS definitions. This component, which is part of the standard rasdaman distribution, is used by the [http://www.opengeospatial.com Open Geospatial Consortium] (OGC) for operating their official CRS resolver.
     280
     281SECORE, being a servlet like petascope, gets installed automatically unless `--disable-java` is specified at `configure`. The deployment directory of all war files can be set during the `configure` step with the `--with-wardir=DIR` option.
     282
     283For further details on SECORE management, security and troubleshooting see the [wiki:SecoreAdministration administration] and [wiki:SecoreDevGuide developer guide] pages.
     284
     285
     286== Updating the installation ==
     287
     288For staying up-to-date with the latest fixes, you can update the repository and recompile at any time.
     289Assuming you have cloned the rasdaman repository from git in directory `~/rasdaman`, you need to pull the latest commits, recompile and reinstall rasdaman:
     290{{{
     291#!sh
     292cd ~/rasdaman
     293
     294git pull           # get latest commits
     295make clean         # clean old compilation artefacts
     296make               # compile the updated code
     297
     298# it is recommended to stop rasdaman/Tomcat before installing the new executables
     299sudo service tomcat7 stop
     300stop_rasdaman.sh
     301
     302# install rasdaman/petascope, and update the database schemas
     303make install
     304update_db.sh
     305update_petascopedb.sh
     306
     307# start services again
     308start_rasdaman.sh
     309sudo service tomcat7 start
     310}}}
     311
     312== Next steps ==
     313
     314After successful compilation you may want to proceed to check the [wiki:Documentation documentation], [wiki:FAQ FAQ], and [wiki:KnownIssues known issues list].
     315
     316For more in depth details on rasdaman's installation and configuration, see [browser:manuals_and_examples/manuals/doc-guides/inst-guide.pdf rasdaman installation guide] and [browser:manuals_and_examples/manuals/doc-guides/external-guide-POSTGRESQL.pdf PostgreSQL configuration hints].