Opened 11 years ago
Last modified 11 years ago
#458 closed enhancement
Test engine for tickets — at Version 1
Reported by: | Dimitar Misev | Owned by: | Dimitar Misev |
---|---|---|---|
Priority: | major | Milestone: | 9.0 |
Component: | systemtest | Version: | development |
Keywords: | Cc: | Jinsongdi Yu, Piero Campalani, abeccati | |
Complexity: | Medium |
Description (last modified by )
Add test engine for testing open bugs reported in tickets, in source:systemtest/testcases_open/test_tickets
The source:systemtest/testcases_open/test_tickets/README summarizes how the test engine works:
====
This test suite includes test cases for bugs reported in tickets. The following directories are important for adding tests here: - queries - files containing queries to be tested. Each file should contain one query. The files should be specifically named, as the test execution and reporting depends on this: NNN-TTT_test-case-name.EXT where NNN is a test case sequence number, TTT is ticket number. EXT indicates the type of the query, and right now can be one of * rasql * wcps (abstract syntax) * wcs (KVP URLs) * secore (KVP URLs) Additional bash script can accompany each test case, for any extra steps that need to be done before/after running the test case, like importing/removing testdata, or for custom check of results. The bash scripts should be named: * NNN-TTT_test-case-name.EXT.pre.sh (run before the test) * NNN-TTT_test-case-name.EXT.post.sh (run after the test) * NNN-TTT_test-case-name.EXT.check.sh (custom output check) - oracle - expected data for testcases, name should be same as the name of the corresponding test case, with .oracle appended: NNN-TTT_test-case-name.EXT.oracle - testdata - any data used by the tests. Testdata from rasql tests (test_grey, test_grey2, test_rgb2) and petascope tests (mr, rgb, eobstest) is automatically inserted and available in the tests. To run the tests execute test.sh. The test script always returns 0 (successfull execution), while the summary says that a test is either PASSED if the result is as expected, or SKIPPED if it failed (ticket still not fixed). Once a ticket has been fixed and a tests passes, the query in queries should be renamed to NNN-TTT_test-case-name.EXT.fixed When a query is marked as fixed in this way, the test suite behaves as regular test suite, i.e. when the test fails the test script exits with an error.
Note:
See TracTickets
for help on using tickets.
Patch submitted