Opened 5 months ago

Closed 4 months ago

#1495 closed defect (fixed)

Fix the mischecking xml output in system tests and refactor the checking process

Reported by: bbell Owned by: bphamhuu
Priority: major Milestone: 9.4
Component: systemtest Version: development
Keywords: test Cc: dmisev
Complexity: Medium

Description (last modified by bphamhuu)

Update: There was a problem with system test, when output especially in xml always return true from validation. The systemtest need to be fixed.

run test.sh in systemtest/testcases_mandatory/test_nullvalues/test.sh
first time --> everything passes
second time --> encode.rasql fails
third time --> encode.rasql fails
fourth time --> everything passes

After some investigation, the behaviour in lines 839-892 of systemtest/util/common.sh creates redundantly-named output files which conflict with what is being tested on subsequent tests.

Compare lines 854-869 with lines 875-890. The former, which is for *.tiff files, uses $output_tmp, while the latter uses 2>&1 > /dev/null. Could this discrepancy be what is causing the errors in the tests? Perhaps removing the files after comparison could resolve this issue.

Change History (3)

comment:1 Changed 5 months ago by bphamhuu

  • Component changed from undecided to systemtest
  • Milestone set to 9.4

comment:2 Changed 5 months ago by bphamhuu

  • Complexity changed from Easy to Medium
  • Description modified (diff)
  • Summary changed from test.sh in test_nullvalues passes ONLY on its (1+3k)th runs, where k >= 0. to Fix the mischecking xml output in system tests and refactor the checking process

comment:3 Changed 4 months ago by bphamhuu

  • Resolution set to fixed
  • Status changed from new to closed

The patch was applied, also the problem Brennan found also not appear again.

Note: See TracTickets for help on using tickets.