embed jetty with petascope

[Dimitar Misev]

Using a small embedded servlet container to deploy petascope, as an alternative to Tomcat, would simplify rasdaman's installation.

The idea is that start/stop_rasdaman.sh would handle starting and stopping of petascope as well, which would live in $RMANHOME/bin or so.

Combined with #959 this would result in an alternative rasdaman, which is self-contained and independent of external servers.

[Bang Pham Huu]

Hi Dimitar,

I've a short report about integrating Rasdaman with Jetty. My current OS system is Centos 7, Rasdaman version 9.0. I aslo test petascope at least with the GetCapabilities with Jetty and it returns result to let me know it is working.

If you have a comment or a suggestion, please let me know.

Thanks.

1. Install Java 8 for Jetty 9 [1]
For 32 bit use the file name: jdk-8u45-linux-i586.tar.gz
For 64 bit use the file name: jdk-8u45-linux-x64.tar.gz

Go to the folder /opt/
# cd /opt/

Download the zipped file (note: this is 64 bit file, 32 bit use jdk-8u45-linux-i586.tar.gz)
# wget —no-cookies —no-check-certificate —header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "​http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.tar.gz"

Unzip the file to folder Java JDK
# tar xzf jdk-8u45-linux-x64.tar.gz

NOTE: use the "alternatives" command to install it. alternatives command is available in chkconfig package.

# cd /opt/jdk1.8.0_45/
# alternatives —install /usr/bin/java java /opt/jdk1.8.0_45/bin/java 2
# alternatives —config java

There are 3 programs which provide 'java'. Choose '3' to use Java jdk8 to start applications like Jetty.

Selection Command

---

• 1 /opt/jdk1.7.0_71/bin/java

2 /opt/jdk1.8.0_25/bin/java
3 /opt/jdk1.8.0_45/bin/java

Enter to keep the current selection[+], or type selection number: 3

At this point JAVA 8 has been successfully installed on your system. Also setup javac and jar commands path using alternatives

# alternatives —install /usr/bin/jar jar /opt/jdk1.8.0_45/bin/jar 2
# alternatives —install /usr/bin/javac javac /opt/jdk1.8.0_45/bin/javac 2
# alternatives —set jar /opt/jdk1.8.0_45/bin/jar
# alternatives —set javac /opt/jdk1.8.0_45/bin/javac

Check the installed version of Java to make sure it is working with Java 8
# java -version

java version "1.8.0_45"
Java™ SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot™ 64-Bit Server VM (build 25.45-b02, mixed mode)

2. Install Jetty: [2]
First, fill the most update of Jetty (now is Jetty stable 9.3.2)
​http://download.eclipse.org/jetty/stable-9/dist/

Download the file jetty-distribution-9.3.2.v20150730.tar.gz to some folder, for example /tmp
# cd /tmp
# sudo wget ​http://eclipse.mirror.triple-it.nl/jetty/stable-9/dist/jetty-distribution-9.3.2.v20150730.tar.gz

Extract the zipped file in the /tmp folder:
# sudo tar xzvf jetty-distribution-9.3.2.v20150730.tar.gz

Now move Jetty to folder /opt/jetty
# sudo mv jetty-distribution-9.3.2.v20150730 /opt/jetty

Remove the temp folder
# sudo rm jetty-distribution-9.3.2.v20150730.tar.gz

Now create a user called jetty to run jetty web server on system start-up
# usderadd -m jetty

Changed ownership of jetty directory (/opt/jetty) to group and user jetty
# chown -R jetty:jetty /opt/jetty

Copy Symlink jetty.sh to /etc/init.d directory to create a start up script file for jetty web server.
# ln -s /opt/jetty/bin/jetty.sh /etc/init.d/jetty

Add script jetty.sh
# chkconfig —add jetty

Now create a file to store jetty properties to start.
# nano /etc/default/jetty (or vi, gedit,…)
JETTY_HOME=/opt/jetty
JETTY_USER=jetty
JETTY_PORT=8088 #(default port Tomcart is 8080, it just a test)
JETTY_HOST=localhost # could change to IP address (example: 192.168.12.10)
JETTY_LOGS=/opt/jetty/logs/ # for error could view in this folder

Start the jetty service
# service jetty start

---

Copy the folder of rasdaman.war (petascope) to /opt/jetty/webapps
Open Web Browser, run: ​http://localhost:8088/rasdaman/ows. And click on 'Get capabilities', if see the description in 'Service identification', 'Service provider' and 'GML server capabilities' then petascope has start working good with Jetty 9.x

View Jetty Status to see the current status of running Jetty.
# service jetty status

References:

[1] Install Java 8 on Centos 7 ​http://tecadmin.net/install-java-8-on-centos-rhel-and-fedora/
[2] Install Jetty on Centos 7 ​http://www.itzgeek.com/how-tos/linux/centos-how-tos/install-jetty-web-server-on-centos-7-rhel-7.html#axzz3i0oEwFuv

[Peter Baumann]

good work, Bang!
Now to the next step: can you integrate this into the "configure && make && make install" cycle?
You need to check prerequisites (like other dependencies we have, you can learn from these examples), generate the rasdaman-jetty, and likely provide a configuration file.

[Dimitar Misev]

Bang,

the idea is to embed Jetty within petascope's code (source:applications/petascope)
Have a look at these links to get an idea what I mean:

• ​http://zetcode.com/java/jetty/embedded/
• ​http://www.hascode.com/2013/07/embedding-jetty-or-tomcat-in-your-java-application/
• ​http://www.eclipse.org/jetty/documentation/9.3.0.v20150612/embedding-jetty.html

[Bang Pham Huu]

Thanks Prof. Peter and Dimitar, I've problem that I think as a manual way by downloading, extract and setup.

But this is better when could embedded jetty to petascope (rasdaman.war) in folder (now is /usr/share/rasdaman/)

I will try to config Jetty in the way Dimitar has guided.

[Bang Pham Huu]

Thanks Dimitar for the link, I've attached successful jetty.jar with rasdaman.war (petascope) and make a file .java to start both of these service.

However, when I tried to build Rasdaman from source code (I'm using Centos 7), the problem is

(my folder is /home/rasdaman/)

git clone ​git://rasdaman/rasdaman.git
cd rasdaman
autoreconf -fi
./configure

and is has an error here "
configure: error: in `/home/rasdaman/rasdaman':
configure: error: libcrypto.a not found! Please install.
See `config.log' for more details
"

So I'm stuck at this and could not try to make with make install and test with Jetty embedded as Prof. Peter required. I hope Dimitar could help and I could make it quickly because I've been late for 1 week without notice anything.

Thanks you in advance.

[Dimitar Misev]

Please consult the documentation on how to install rasdaman from source: ​http://rasdaman.org/wiki/Install

Installing on Debian might be easier. For installing the required packages on CentOS with yum for example see source:packaging/rpm/rasdaman-sqlite.spec (the BuildRequires lists)

[Dimitar Misev]

You could actually create a quick CentOS 7 installation guide as you compile and install rasdaman, like the one we have for Debian: wiki:RasdamanQuickInstallDebian7

[Bang Pham Huu]

Yes, Dimitar, I also want to fill the gap of guide If I could follow your path successful. I'll do it now and If I have error I'll need more help.

Thanks for your hard working !

[Bang Pham Huu]

Thanks again, Dimitar,

As you suggested link ​http://www.rasdaman.org/wiki/RasdamanQuickInstallDebian7 and after tracking file configure.ac to install library step by step.
I could installed Rasdaman on Centos 7 (test ok with rasdaman_insertdemo.sh), but actually it is a lot of tricky (especially with libpng15 (Rasdaman code using deprecated pointer (I don't know it is the same with ​http://stackoverflow.com/questions/10507610/libpng-1-5-10-error-dereferencing-pointer-to-incomplete-type) or not but I've to downgrade to libpng12 and no complain when "make" - compile source code).

Also a lot of library are changed in Centos 7 at least by their name and some time need to find the library in some package website. I'll try to write the guide right now, however, I don't think this guide could cover 100% from downloading code to make install, may be in some step, I've installed library before and don't need install again but the other could follow my steps and have an idea.

I'll combine petascope with embedded jetty to start_rasdaman.sh, "make" after that.

[Dimitar Misev]

Great Bang. You should also make sure that you can successfully install petascope, see ​http://rasdaman.org/wiki/Install#Installingpetascope

[Bang Pham Huu]

Hi Dimitar,

Please help when I've tested Petascope with Jetty after compiled Rasdaman in Centos 7. Yesterday, I could run normally without any error (for GetCoverages, DescribeCoverages).

+ It looks like when I compile rasdaman from source code in Centos 7 (this morning), it is not good as your compiled RPM, because:

• No petascope.properties and log4j.properties has been created in /etc/rasdaman/

So when I tried to run Petascope (Rasdaman.war) with Jetty embedded it has error like this:

log4j:ERROR Ignoring configuration file [/etc/rasdaman/log4j.properties].
log4j:WARN No appenders could be found for logger (petascope.ConfigManager).
log4j:WARN Please initialize the log4j system properly.
2015-08-20 15:17:31.631:WARN:/rasdaman:main: unavailablejava.lang.RuntimeException: Failed loading the settings file /etc/rasdaman/petascope.properties

+ I also want to insert test data to petascope database by rasimport but also error
petascope_insertdemo.sh

petascope_insertdemo.sh: inserting `mean_summer_airtemp' 2D demo dataset for WC*S and WMS services... rasimport::main(): could not access connection file '/root/.rasdaman/rasconnect'!

I have no idea about the "dot" in /.rasdaman/ (what is this mean?). It looks like the make install could not create the "file rasconnect" too.

Thanks!

[Dimitar Misev]

Please check wiki:Install#Installingpetascope, petascope.properties etc. are created in $RMANHOME/etc/ (RMANHOME is the directory where you install rasdaman, you give this directory when you ./configure —prefix=$RMANHOME).

Read this on how to setup rasimport: wiki:Install#rasgeo

[Bang Pham Huu]

Thanks Dimitar,

I've reinstalled with RMANHOME (/home/rasdaman/install) and could see the file petascope.properties and log4j.proeperties. However, they don't exist in folder /etc/rasdaman then I've to copy file to these folder.

When I've shown you problem with connect to database postgres from petascope.properties, it's not need to follow the guide by creating user postgresql (I've created it before), it's error because I just thought your parser could pass common "# character, then when this guide ​http://rasdaman.org/wiki/Install#PreparePostgreSQL said should modify

metadata_user=rasdaman   # petauser (Here is what I've done) <--- then when run jetty jave, it complained about read metadata error

Now I could run jetty as yesterday, but because Petascope need file 'def.war' too, so I need to figure out how to embedded petascope.war and def.war with jetty.jar tomorrow.

About 'rasgeo' and 'rascontrol', I could create this connection file by using this command. So the "dot" in .rasdaman is just to make it special ("hidden from user"), that is all I want to know. After that, I've run petascope_import and it look like it could insert coverage to database (tested by view on localhost:8080/rasdaman/ows), but still have warning

inserting `mean_summer_airtemp' 2D demo dataset for WC*S and WMS services... petascope_insertdemo.sh: Warning: WMS initialization for mean_summer_airtemp failed.
petascope_insertdemo.sh: Reloading the server capabilities might have failed (assuming localhost:8080): please try to restart the server and check WMS GetCapabilities.
petascope_insertdemo.sh: Warning: WMS pyramid creation for mean_summer_airtemp failed.

Anyway, thanks for your day.

[Bang Pham Huu]

Hi Dimitar,

Please help with some error when I've tried to combine def.war and rasdaman.war with embedded jetty. I don't know how to test it correctly (I just tried to test on web page localhost:8080/rasdaman/ows and stuck at DescribeCoverage function).

Have a look on these image below, thanks!

1. I run the java file to start jetty server embedded, it looks ok (although it complained about WMS Server not found but still run the Web Server) when I could access the address of Petascope.

2. After that, some query to GetCapabilities of server has done, it looks ok.

3. The problem when I click on the Coverage named 'mean_summer_airtemp' and it tried to DescribeCoverage of this coverage and has some error with Secore (I guess).

4. However, if I click "DescribeCoverage" again, it appears the basic coverage of this coverage and notice that GML of Secore CRS has been cached.

5. Last, although I've installed Rasdaman 9.1.0 but I still could not insert coverage with wcst_import.sh and JSON ingredient with the file of Rasdaman's example (mean_summerair_temp.tif) and it is error.

Have a lunch first and hope could see your help.

Thanks.

[Dimitar Misev]

Maybe you can change the secore endpoint in petascope.properties, instead of localhost to kahlua.eecs.jacobs-university.de to eliminate that error.

I'm not sure about the nil value error, in the latest petascope it should be fixed. Can you try adding in your ingredients file below the "mock" option and see if it helps:

"default_null_values": ["0"]

[Bang Pham Huu]

As this "over complicated" ticket has been accepted. I will happily close it and if it has a new bug please open new ticket.

ticket:960 - embed jetty with petascope BangPH <b.phamhuu@…> master 2015-10-28 16:23:54 Download patch APPLIED SUCCESS