Opened 11 years ago
Closed 11 years ago
#458 closed enhancement (fixed)
Test engine for tickets
Reported by: | Dimitar Misev | Owned by: | Dimitar Misev |
---|---|---|---|
Priority: | major | Milestone: | 9.0 |
Component: | systemtest | Version: | development |
Keywords: | Cc: | Jinsongdi Yu, Piero Campalani, abeccati | |
Complexity: | Medium |
Description (last modified by )
Add test engine for testing open bugs reported in tickets, in source:systemtest/testcases_open/test_tickets
The source:systemtest/testcases_open/test_tickets/README summarizes how the test engine works:
====
This test suite includes test cases for bugs reported in tickets. The following directories are important for adding tests here: - queries - files containing queries to be tested. Each file should contain one query. The files should be specifically named, as the test execution and reporting depends on this: NNN-TTT_test-case-name.EXT where NNN is a test case sequence number, TTT is ticket number. EXT indicates the type of the query, and right now can be one of * rasql * wcps (abstract syntax) * wcs (KVP URLs) * secore (KVP URLs) Additional bash script can accompany each test case, for any extra steps that need to be done before/after running the test case, like importing/removing testdata, or for custom check of results. The bash scripts should be named: * NNN-TTT_test-case-name.EXT.pre.sh (run before the test) * NNN-TTT_test-case-name.EXT.post.sh (run after the test) * NNN-TTT_test-case-name.EXT.check.sh (custom output check) - oracle - expected data for testcases, name should be same as the name of the corresponding test case, with .oracle appended: NNN-TTT_test-case-name.EXT.oracle - testdata - any data used by the tests. Testdata from rasql tests (test_grey, test_grey2, test_rgb2) and petascope tests (mr, rgb, eobstest) is automatically inserted and available in the tests. To run the tests execute test.sh. The test script always returns 0 (successfull execution), while the summary says that a test is either PASSED if the result is as expected, or SKIPPED if it failed (ticket still not fixed). Once a ticket has been fixed and a tests passes, the query in queries should be renamed to NNN-TTT_test-case-name.EXT.fixed When a query is marked as fixed in this way, the test suite behaves as regular test suite, i.e. when the test fails the test script exits with an error.
Change History (6)
comment:1 by , 11 years ago
Description: | modified (diff) |
---|---|
Status: | new → accepted |
comment:2 by , 11 years ago
Resolution: | → fixed |
---|---|
Status: | accepted → closed |
comment:3 by , 11 years ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
comment:4 by , 11 years ago
Status: | reopened → assigned |
---|
Changeset c75cf3eb487edaefdc1946054796e35d61cc52ae removed loading of the mean_summer_airtemp dataset for testing WMS. WMS tests now fail with empty result.
Coverage insertion has to be re-introduced for WMS tests to work.
comment:6 by , 11 years ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
Note:
See TracTickets
for help on using tickets.
Patch submitted