Embedded rasdaman.war process leak when stopping rasdaman

[Dimitar Misev]

Situation: rasdaman is hanging with returning query result, e.g. from 'SELECT a FROM RAS_STRUCT_TYPES a' which petascope sends on startup; petascope keeps waiting for the response from rasdaman and cannot start.

Problem: user stops rasdaman with stop_rasdaman.sh or so. This script will attempt to kill the embedded tomcat petascope process, however, the normal kill fails. So rasdaman.war keeps running. On the next start_rasdaman.sh another embedded tomcat is started.

Solution: rasdaman.war should be stopped with kill -9 in this case.

Solution 2: I'm not sure if in petascope we need to do something to handle this case where rasql query is hanging on startup?

[Dimitar Misev]

I think the best handling of this is as follows:

• start_rasdaman.sh waits for 10s or so for the rasdaman.war process to be gone
• if it isn't done yet in 10s, it will print an error:

  Failed stopping geo services, please rerun with --force
  

• i.e. we should support such an option which in stop_rasdaman.sh will do kill -9 when invoking one of
  • stop_rasdaman.sh --force
  • service rasdaman kill
  • systemctl kill rasdaman.service

[Dimitar Misev]

If the petascope process is still running after 10 seconds timeout, stop_rasdaman.sh will print a warning about it, and a hint to rerun stop_rasdaman.sh --force to kill -9 the process:

$ stop_rasdaman.sh
stop_rasdaman.sh: stopping rasdaman v10.0.0-beta2 build g9e1594838...
stop_rasdaman.sh:   stopping petascope... failed, please check the rasdaman.war 
                      process manually, or stop rasdaman with the --force option.
stop_rasdaman.sh:   stopping rasserver processes... ok.
stop_rasdaman.sh:   terminating rasmanager. Good Bye!
stop_rasdaman.sh: done.

$ stop_rasdaman.sh --force
stop_rasdaman.sh: stopping rasdaman v10.0.0-beta2 build g9e1594838...
stop_rasdaman.sh:   stopping petascope... ok.
stop_rasdaman.sh: rasdaman is already down, nothing to do.

It will do the same sort of handling for rasmgr/rasserver as well.