wiki:RasdamanTestSuites

Version 5 (modified by abeccati, 4 years ago) (diff)

work in progress notice

NOTE: this page is work in an ongoing effort to document the test system used to test rasdaman

Overview

"make check" will automatically find all the tests inside the four test case directories, specifically, testcases_mandatory, testcases_petascope, testcases_fixed and testcases_open.

  1. Whenever a bug is found, there should be a corresponding test be created in the testcases_open directory;
  2. when the bug is fixed, the test should be moved to the testcases_fixed directory;
  3. testcases_petascope holds the test cases for petascope;
  4. testcases_mandatory  holds the mandatory test cases.
  1. Each test should have a folder which is inside one of the above mentioned directories;
  2. the test should be invoked by a shell script inside the folder;
  3. the test result should be told by its EXIT code;
  4. the test can be a pure shell script or a shell script with corresponding test queries and/or test oracles;
  5. test detail should be log in the same folder.

Adding a test to the suite

Howto with a sample test

System test

Let us assume that we want to add a test case for our Rasdaman building, you can do as below:

  • how to add a test case?
      1. copy the test package to rasdaman/systemtest/test_open
      2. rename the test packet to test_<testcaseName> 
      3. rename the main shell script in rasdaman/systemtest/test_open/test_<testcaseName> to test.sh 
    
  • how to add a test query?
      1. save the test query as <queryName>.rasql 
      2. copy the query to rasdaman/systemtest/test_mandatory/test_select/test_rasql
      3. save the related query result as a test oracle with the name <queryName>.rasql (the same name with test query file)
      4. copy the oracle to rasdaman/systemtest/test_mandatory/test_select/oracle 
    
  • how to generate a test oracle?
      1. check the result correctness 
      2. if the result is a scalar, run
    rasql -q  "$QUERY" --out string|grep Result > oracleName
      3. if the result is an array, run
     rasql -q  "$QUERY" --out file --outfile oracleName
    
  • how to run the test?
      1. run 'make check' under rasdaman or rasdaman/systemtest directory