whytofly
/
qt6-bb10
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

TAP test logger: skip XFail results as well as pass after an XFail 

The rationale for skipping pass after XFail is "to emit a single test
point for" the test; emitting several XFails violates that aim.

Task-number: QTBUG-96844
Change-Id: Ia8626dfc2dded234b3aa530fc2dc2324f1e28400
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
bb10
Edward Welbourne 2021-09-27 16:05:46 +02:00
parent 24b4f5b16c
commit 8d0b5270fa
3 changed files with 2 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
src/testlib/qtaptestlogger.cpp
View File

@ -115,7 +115,8 @@ void QTapTestLogger::outputTestLine(bool ok, int testNumber, QTestCharBuffer &di
void QTapTestLogger::addIncident(IncidentTypes type, const char *description,
const char *file, int line)
{
if (m_wasExpectedFail && (type == Pass || type == BlacklistedPass)) {
if (m_wasExpectedFail && (type == Pass || type == BlacklistedPass
|| type == XFail || type == BlacklistedXFail)) {
// XFail comes with a corresponding Pass incident, but we only want
// to emit a single test point for it, so skip the this pass.
return;

12
tests/auto/testlib/selftests/expected_expectfail.tap
View File

@ -70,12 +70,6 @@ not ok 11 - xfailWithQString() # TODO A string
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 0
...
not ok 11 - xfailWithQString() # TODO Bug 5 (The message)
---
at: tst_ExpectFail::xfailWithQString() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:0)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 0
...
ok 12 - xfailDataDrivenWithQString(Pass Abort) # SKIP Each Continue or Pass reports this and increments skip-count
ok 13 - xfailDataDrivenWithQString(Pass Continue) # SKIP Each Continue or Pass reports this and increments skip-count
not ok 14 - xfailDataDrivenWithQString(Fail Abort) # TODO A string
@ -90,12 +84,6 @@ not ok 15 - xfailDataDrivenWithQString(Fail Continue) # TODO A string
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 0
...
not ok 15 - xfailDataDrivenWithQString(Fail Continue) # TODO Bug 5 (The message)
---
at: tst_ExpectFail::xfailDataDrivenWithQString() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:0)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 0
...
ok 15 - xfailDataDrivenWithQString(Fail Continue) # SKIP Each Continue or Pass reports this and increments skip-count
ok 16 - xfailDataDrivenWithQVerify(Pass Abort)
ok 17 - xfailDataDrivenWithQVerify(Pass Continue)

12
tests/auto/testlib/selftests/expected_strcmp.tap
View File

@ -2,18 +2,6 @@ TAP version 13
# tst_StrCmp
ok 1 - initTestCase()
ok 2 - compareCharStars()
not ok 3 - compareByteArray() # TODO Next test should fail
---
at: tst_StrCmp::compareByteArray() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:0)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 0
...
not ok 3 - compareByteArray() # TODO Next test should fail
---
at: tst_StrCmp::compareByteArray() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:0)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 0
...
not ok 3 - compareByteArray() # TODO Next test should fail
---
at: tst_StrCmp::compareByteArray() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:0)