Enhancement of petascope regression test suite

[abeccati]

As part of the proposed projects we have regression test refactoring

Wrt regression testing we need improvement especially with petascope tests (documented here)

Most compelling is the inclusion of an oracle or similar structure for evaluating query output and enable also queries that purposefully generate errors to be part of successful test cases.

Investigation of current status and proposal of solutions is required

[Dimitar Misev]

Note that I have submitted some systemtest refactoring patch, you should do your work after applying that on.

[Dimitar Misev]

Note I've updated a bit the test page at wiki:RasdamanTestSuites#Petascopetests

The configuration file for petascope and the whole systemtest now is source:systemtest/conf/test.cfg

[Piero Campalani]

As soon as the oracle-based testing system is set up, as a next step we should add (many) error-prone queries and verify they actually return an error/exception (this will help us reviewing error messages and identifying cumbersome ones, e.g. see #394).

[Dimitar Misev]

That should go in another ticket Piero, I don't think Nikolce has the background to add such tests.

[Piero Campalani]

Yes, it was general next-step, not Nikolce's next step
It was a note, then we can either create a new ticket or go on with this one, whatever.

[abeccati]

I strongly second Piero's proposal and created #396 for that purpose (we should keep tasks/ticket as "atomic" tasks as much as we can to ease management)
Let's finalize infrastructure first with this ticket.

@nkolev: Please "accept" the ticket (accept option at bottom of modify ticket) so we can track active assignment.

[Dimitar Misev]

patch submitted

[Piero Campalani]

Missing ref from comment:8 : changeset:a6625cd.

Now WCS tests are left.

[abeccati]

Also WMS tests are missing, basic ones should be included. Meanwhile patch submitted to master (2013-07-23 14:02:17) for WMS inclusion: links directly to the test.sh engine script.