C++26 adds std::saturate_cast, so follow the established pattern of
other similar "backported" APIs. The old name is left around while
we port other submodules.
While at it, move qt_saturate's tests to the qnumeric test.
Change-Id: I653a2e3d936081378298a9c8e51e7c1a2d438d83
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit 7447ad503330ed176cf369792ffb33b7e00a58d3)
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Drive-by remove an explicit NUL-termination (std::snprintf() does
that) and port a repeated use of printf argument checking to the
protect() idiom.
Change-Id: Ida15940fe9aef0622e9836a229a398c909503a9a
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
(cherry picked from commit b5115d1c2fc73feb149a8ee97de011b3c75694fb)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
Unit test failure should not happen, so mark failure formatters as
cold (like qFail() itself).
As a drive-by, mark formatFailMessage() overloads file-static, too.
Change-Id: I92f5b674a7282f4ebaa9250894a9971a524f4108
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit 69c510d6bcc52ac133a82d7a9493ffba037c4521)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
Once we ported the formatting to use formatFailMessage() in
aa8a4be9e9eae82b6c6b8875371114dbfb3ec06a, it became apparent that the
output contained an extra leading newline compared QCOMPARE use of
formatFailMessage(). We kept the newline in the patch that was picked
to older branches. Since this is a stylistic discrepancy, though,
standardize on no leading newline going forward.
Change-Id: I4badf700900a106d835f3f72e92441d8a8912ff7
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
(cherry picked from commit 4bc81fe5d1d1e4587075a82a45be9b34825b5233)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
The patch that originally added the function used std::clamp() because
qt_saturate() wasn't available in all the branches to which the patch
was backported. This patch modernizes the code for newer branches.
Pick-to: 6.7 6.5
Change-Id: I1b764d303e00ec04858643efed3dcc71f2c7ce4c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit d042c14c9940c6aabaa5b39d3061c0f9e50fff06)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
The mbstowcs() function is not re-entrant, and while we can probably
rule out that our test macros are being executed concurrently, we
can't rule out that other parts of the test concurrently execute
std::mbstowcs().
To fix, de-inline the code so we can re-use the existing
formatFailMessage() in qtestresult.cpp, which has already been fixed
before.
Amends 930e59b798.
Pick-to: 6.7 6.5 6.2
Change-Id: I61144af13a41ea0b4fba17bd232e660ef33dbd20
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
(cherry picked from commit aa8a4be9e9eae82b6c6b8875371114dbfb3ec06a)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
This function is not re-entrant, and while we can probably rule out
that our test macros are being executed concurrently, we can't rule
out that other parts of the test concurrently execute std::mbstowcs().
Use the re-entrant version and also fix the format string (%* requires
int, not size_t).
Saturation is ok here, because, while std::mbsrtowcs() will happily
return a value > maxMsgLen when dst == nullptr and the string is
longer, we will never actually print more than maxMsgLen of it.
As a drive-by, port to std and adjust the QNX comments to describe the
quirky QNX behavior, if, indeed, still present in the <cwchar>
version, better. This author had to go back in the git history to
figure out what it wanted to tell him.
Amends 2d8028d696 (mbstowcs) and
d040681b6f (%*).
Pick-to: 6.7 6.5 6.2 5.15
Change-Id: I6215713c643647727f73f4e2f2a7ac34204af40d
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
(cherry picked from commit 1579b18dfa49252318b4a2b3cfa83a4ce325ba0c)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
This is causing huge code bloat because everything is a local lambda.
Instead, pass direct type-erased function and data pointers to the
replacement function. Testing with tst_qcborvalue, this reduces the
compilation time and the output binary size significantly:
Before After
Compiler Time Size Time Size
GCC 13.2 136.99 s 202.3 MB 13.88 s 14.3 MB
GCC 14.0 131.49 s 202.7 MB 14.69 s 14.4 MB
Clang 17 77.2 s 146.7 MB 13.62 s 12.2 MB
Clang 18 141.9 s 187.1 MB 13.62 s 12.4 MB
This causes a difference in how toString() overloads are
found. Previously it would match far more overloads because the
toString() calls were expanded by the macro. Now, we depend on
Argument-Dependent Lookup and associated namespaces, so toString()
overloads should not be in the QTest namespace any more.
With this patch applied, the testlib testcase of tst_selftest
started failing, because nullptr is now handled differently.
However, I consider it as a bugfix, because previously it was
falling back to a default implementation, and now it is using
the QTest::toString(std::nullptr_t) overload, which is a
desired behavior. Update the reference files for tst_selftest
with the new expected output.
Task-number: QTBUG-124272
Change-Id: Ie28eadac333c4bcd8c08fffd17c5484186accdf6
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
We need to be able to handle warnings that may occur when temporary
objects, that were created in a test function, are destroyed.
For example, now we miss all warnings that might be triggered from
the object destructor, if the object's deletion was delayed (e.g. via
deleteLater()). Also we miss all the warnings that were triggered on
the test's cleanup() call.
To fix this we need simply move QTestLog::clearFailOnWarnings()
from QTestResult::finishedCurrentTestData() to the later stage,
i.e. into QTestLog::clearCurrentTestState() which is actually called
in appropriate time from QTestResult::finishedCurrentTestDataCleanup().
Same for QTestLog::clearIgnoreMessages(), since they are interrelated,
so we need to clear them at the same time.
We need this change for QML tests in particularly, to be able
fail on warnings that might be triggered from Component.onDestruction()
of some temporary test object.
Pick-to: 6.6 6.5
Change-Id: I58a57691f20761619f56bd1bea3a862f2c26f569
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
QPalette specifically has quite a large amount of output (1363
characters) when toString
is called on it. We should make sure that we can fit that in our
failure messages. This patch does that by increasing the limit from
1024 characters to 4096.
Fixes: QTBUG-5903
Fixes: QTBUG-87039
Pick-to: 6.5 6.6
Change-Id: I1dc5078ad05858bb6542c3a06c6b84711af79e4f
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
This amends commit 115f828ae4
"QTestEventLoop: stop when the test fails" from Qt 6.3 to not attempt to
stop the QTestEventLoop instance if the failure happened outside the
main thread. This is to prevent the QTEL from being created outside the
main thread and thus cause warnings about QObject in addition to and
logged before the real test failure.
[ChangeLog][QtTest][QTestEventLoop] The QTestEventLoop no longer
attempts to exit its event loop if the failure was detected outside the
main thread.
QWARN : tst_QMutex::tryLock_non_recursive() QObject: Cannot create children for a parent that is in a different thread.
(Parent is QCoreApplication(0x7fffffffe710), parent's thread is QThread(0x802e29000), current thread is QThread(0x7fffffffd5a8)
FAIL! : tst_QMutex::tryLock_non_recursive() 'timer.elapsed() >= waitTime - systemTimersResolution' returned FALSE. ()
Change-Id: Ieab617d69f3b4b54ab30fffd175b139322a1f02c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
The parameter names were previously "lhs" and "rhs", which is incredibly
abstract for a testing framework. One of the parameters will tend to be
a baseline value to compare against while the other is the result of
some action we want to test. Thus I suggest they be renamed "computed"
and "baseline".
This way we can, hopefully, retain the semantic that the 'left'/first
argument is the computed ('actual' in QCOMPARE) value while the
'right'/second argument is the baseline ('expected' in QCOMPARE.)
Change-Id: I3e0fdce2a3f1faca06fdf7184ef6e0eb9724d990
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
This is a combination of Q_UNREACHABLE() with a return statement.
ATM, the return statement is unconditionally included. If we notice
that some compilers warn about return after __builtin_unreachable(),
then we can map Q_UNREACHABLE_RETURN(...) to Q_UNREACHABLE() without
having to touch all the code that uses explicit Q_UNREACHABLE() +
return.
The fact that Boost has BOOST_UNREACHABLE_RETURN() indicates that
there are compilers that complain about a lack of return after
Q_UNREACHABLE (we know that MSVC, ICC, and GHS are among them), as
well as compilers that complained about a return being present
(Coverity). Take this opportunity to properly adapt to Coverity, by
leaving out the return statement on this compiler.
Apply the macro around the code base, using a clang-tidy transformer
rule:
const std::string unr = "unr", val = "val", ret = "ret";
auto makeUnreachableReturn = cat("Q_UNREACHABLE_RETURN(",
ifBound(val, cat(node(val)), cat("")),
")");
auto ignoringSwitchCases = [](auto stmt) {
return anyOf(stmt, switchCase(subStmt(stmt)));
};
makeRule(
stmt(ignoringSwitchCases(stmt(isExpandedFromMacro("Q_UNREACHABLE")).bind(unr)),
nextStmt(returnStmt(optionally(hasReturnValue(expr().bind(val)))).bind(ret))),
{changeTo(node(unr), cat(makeUnreachableReturn,
";")), // TODO: why is the ; lost w/o this?
changeTo(node(ret), cat(""))},
cat("use ", makeUnreachableReturn))
);
where nextStmt() is copied from some upstream clang-tidy check's
private implementation and subStmt() is a private matcher that gives
access to SwitchCase's SubStmt.
A.k.a. qt-use-unreachable-return.
There were some false positives, suppressed them with NOLINTNEXTLINE.
They're not really false positiives, it's just that Clang sees the
world in one way and if conditonal compilation (#if) differs for other
compilers, Clang doesn't know better. This is an artifact of matching
two consecutive statements.
I haven't figured out how to remove the empty line left by the
deletion of the return statement, if it, indeed, was on a separate
line, so post-processed the patch to remove all the lines matching
^\+ *$ from the diff:
git commit -am meep
git reset --hard HEAD^
git diff HEAD..HEAD@{1} | sed '/^\+ *$/d' | recountdiff - | patch -p1
[ChangeLog][QtCore][QtAssert] Added Q_UNREACHABLE_RETURN() macro.
Change-Id: I9782939f16091c964f25b7826e1c0dbd13a71305
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Before 0681a2dd5a, QCOMPARE'ing types
for which no QTest::toString specialization exists did not output
Actual and Expected lines on failure, as that would only print <null>
for both values (which then look like the same value, confusingly).
Commit 0681a2dd5a changed that behavior,
and started printing the confusing <null> values.
Take care of the logic in the formatFailMessage function: if both values
are nullptr, then print only the variable names, but not the confusing
<null> text representation of the values.
Remove dead and duplicated code related to the formatting logic, add a
self-test function, and update the expected_cmptest files.
Fixes: QTBUG-104867
Pick-to: 6.4
Change-Id: I4be98e79f91196b14690a2cc0a68ffd50b431a45
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The old code allocated a stack buffer, but asked the runtime to
zero-initialize it. That's 1KiB of writes to the stack on every
QCOMPARE and QVERIFY before any actual work is done. Fixing this
little laissez-faire to just initialize the first character in the
buffer results in nice little speedups of ~40%.
This fixes the issue in reportResult(), a Cut'n'paste from compare()
and verify(), which have since been fixed by a previous commit.
Change-Id: I5cad57299490925b88e768dc751304699274db2e
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
[ChangeLog][QTestLib] QCOMPARE now evaluates toString() on its
arguments lazily, speeding up the general case where the comparison
doesn't fail. This is true for the QCOMPARE functionality provided
by Qt. If you specialized qCompare() for your own types, then you
need to change its implementation in line with Qt's own qCompare()
specializations in order to enable this feature.
[ChangeLog][QTestLib] QCOMPARE calls with nullptr argument(s) will
now print the actual and expected values upon failure.
Previously it was not like that because of the compareHelper()
overload in qtestresult.cpp that treated the presence of
nullptr-arguments as a reason to ignore formatFailMessage() call.
New implementation does not have this check, and correctly
executes formatFailMessage() for all arguments.
Note that the qCompare() overloads that call QTestResult::compare()
internally were not affected by this patch, because they already
defer toString() invocation until the comparison fails.
Some numbers, collected against shared release developer build.
I checked how this change affects the test execution. The idea was
to pick some tests for types that do not have a specific
QTestResult::compare overload, so I picked a couple of QByteArray
tests.
The comparison is done by running a test 10 times and taking the
average execution duration, as reported in the log.
tst_qbytearrayapisymmetry:
Before: 15.6 ms
After: 14.2 ms
tst_qbytearray:
Before: 41 ms
After: 36 ms
The benefit is around 9% and 12% respectively.
Fixes: QTBUG-98874
Change-Id: I7d59ddc760168b15974e7720930f629fb34efa13
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
[ChangeLog][QTestLib] Add QCOMPARE_{EQ,NE,LT,LE,GT,GE}()
macros. These new macros behave similarly to QVERIFY(a op b),
where 'op' is ==, !=, <, <=, >, >= respectively, but print
a formatted error message with argument values in case of failure.
The formatting is done lazily, which means that the strings will
be generated only when the comparison fails.
Also add a new test for tst_selftest and generate expected output
for it.
Fixes: QTBUG-98873
Task-number: QTBUG-98874
Change-Id: Ic8074798901d7a469b1f58d5cd28bbf49a3da1db
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The old code allocated a stack buffer, but asked the runtime to
zero-initialize it. That's 1KiB of writes to the stack on every
QCOMPARE and QVERIFY before any actual work is done. Fixing this
little laissez-faire to just initialize the first character in the
buffer results in nice little speedups of up to or exceeding 2x.
Amends d946507727.
[ChangeLog][QtTestLib] Optimized successful QCOMPARE and QVERIFY for
an up to 2x speedup.
This has the potential to meaningfully reduce the load on the CI, so
picking all the way to 5.15.
Pick-to: 6.3 6.2 5.15
Change-Id: Ib93d69282ec87cbd26a60e4ac14413e8cef8ff78
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Remove unneeded \fn qdoc lines as a drive-by.
Task-number: QTBUG-98434
Change-Id: Id93ddbb38b97a8f5a6734bfbc82686ccb3a87aa6
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Added tests for repeated skips and failures (from within void lambdas,
to simulate skips and failures from within event handlers). These
exhibit yet more ways to count more than one outcome for a test. The
new QTest::failOnWarning() can also provoke more than one failure from
a single test, and several existing selftests exhibited various ways
for the Totals line's counts to add up to more than the number of
actual tests run.
Fixed counting so that only the first decisive incident is counted.
Tests can still report later failure or skipping, but only the first
is counted.
Added a currentTestState in qtestlog.cpp, by which it keeps track of
whether the test has resolved to a result, and clearCurrentTestState()
by which other code can reset that at the end of each test. This
brought to light various places where test-end clean-up was not being
handled - due to failure or skipping in a *_data() method or init, or
a skip in cleanup.
Fixes: QTBUG-95661
Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
This solves the long-standing problem of not being able to easily
fail a test when a certain warning is output.
[ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test
function, any warning that matches the given pattern will cause a test
failure. The test will continue execution when a failure is added.
All patterns are cleared at the end of each test function.
Fixes: QTBUG-70029
Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
None of the loggers rely on this at the moment, but one could imagine
one that does. Plus, doing any sort of debug logging in leaveTestFunction
will at the moment be attributed to UnknownTestFunc().
Pick-to: 6.2
Change-Id: I284b2785a276e028b9f57c26357679fd9e045ca7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
It makes no sense for the event loop of a test to keep running after a
test has failed. This lets test code simply use the usual testlib
macros to compare and verify values in asynchronous tests that would
otherwise need to hand-test values and send a signal on failure (that
the main test can connect to an event-loops quit() or equivalent).
For example, QLocalSocket's benchmark simply uses the usual macros,
without doing anything to stop its event loop if they fail, with the
sad result that, when a test fails, it does so repeatedly and then
times out, causing the test program to be killed without running later
tests. With this change, that test code (once converted to use
QTestEventLoop) is able to exit gracefully on the first failure.
[ChangeLog][QtTest][QTestEventLoop] The QTestEventLoop new exits its
event loop as soon as the test is known to be failing.
Task-number: QTBUG-91713
Change-Id: If0d455741668722034906763025dda496d2afbb4
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
We try our best to pass on the file location of a failure, including for
fatal errors, but the reporting or logging machinery should not assume
there is one.
By passing on nullptr for the file location we allow the logging backends
to decide how to handle the situation, e.g. by not emitting extra fields
for failure location.
This effectively reverts c25687fa0b,
in favor of relying on the backends to cope with null filename,
which they already did.
As qFatal uses QMessageLogger, which by default disables file/line
information in release builds, we need to explicitly enable this in
our self-tests, to get uniform test results. Similarly, we disable
file/line info from testlib itself, as reporting Qt internal file
and line information for user diagnostics is less useful. The odd
one out there is qtestdata.cpp, which still ends up in test output
due to using QTEST_ASSERT instead of qFatal for its diagnostics.
Cleaning up that, and unifying how we report testlib issues to the
user, is left for another day.
Pick-to: 6.2
Change-Id: Ib9451b8eed86fe3ade4a4dcaf0037e1a3450321c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Previously, it went direct to QTestResults::addFailure() without going
via the checking for expected failure. Add QTestResults::fail() to
take care of this checking, as for verify() and compare().
Tidied up the code implementing expected failure and QFAIL(), while I
was about it. Adjusted an existing test to verify that expecting a
QFAIL() works, by using QFAIL() instead of QVERIFY(false).
Remove the QVERIFY(false) whose comment brought this to my attention.
[ChangeLog][QtTestLib][QFAIL] QEXPECT_FAIL() now correctly anticipates
a subsequent QFAIL(). Previously QFAIL() counted as a fail regardless.
Change-Id: Icc28cf70e5ff3006363791ea03aa01f2f591eb71
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Add a qCompare() overload for qsizetype, preventing it
falling back to the slow code path pre-formatting expected/actual.
toString() should use the correct format from the already present
int-types (see macro TO_STRING_IMPL).
Complements 94aa350621.
Task-number: QTBUG-38890
Change-Id: Ieb8cea7de086141a2c80f93b4c1be01572be96df
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
When debugging a spurious failure it's extremely useful to
run the test repeadtly into a debugger until a failure appears.
When that happens, one wants to immediately start debugging.
In so far, this has only been possible by placing breakpoints
inside Qt itself (when a failure is logged). Add another way:
an env variable, similar to QT_FATAL_WARNINGS, that makes
any failure fatal (terminate() gets called. Bonus: you can
control the termination handler!)
[ChangeLog][QtTestLib][QtTest] When the QTEST_FATAL_FAIL
environment variable is set to a non-zero value, a test
immediately aborts its execution. This is useful to debug
intermittent failures.
Change-Id: If2395f964ea482c30b8c8feab98db7fdee701cd3
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
addBFail() asserts on the file being non-null. The convention seems to
be "Unknown File" for cases where we cannot determine the file.
Change-Id: I3a4d0130352d77d75f264fad6f3bd47c6700ef4c
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Add overloads for qCompare() for QStringView making use of the fast
formatting helper introduced by 94aa350621
for int.
Speeds up the bug report example by a factor of 3..4.
Task-number: QTBUG-38890
Change-Id: Icc706618b2f1d23b37d354a04d4e1d1cc4b5aee3
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Apply Fixits by Qt Creator with some amendments.
Task-number: QTBUG-69413
Change-Id: I620e40a277dc2b20b0ec26fc32577e66b0456bb3
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Factor out a helper template formatting the QCOMPARE failure message
delaying the formatting of the parameters with toString() and use that
for float, double, int and unsigned. This removes the need to always
format and allocate strings for the operands even in the success case,
speeding up the QColor test from 3.3s to 700ms (Windows/release).
Task-number: QTBUG-38890
Change-Id: I999484765bdaed921d3fc35f35a9fbbcd82a9704
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Prioritize blacklisting over QEXPECT_FAIL so that a test that is
blacklisted no longer fails if QEXPECT_FAIL returns true unexpectedly. To
reflect this state properly, the two values of BXPASS and BXFAIL were
added to testlib's output.
[ChangeLog][Important Behavior Changes][QtTestLib] Blacklisting of tests
will be taken into account for XPASS and XFAIL. A blacklisted test that
causes an XPASS will no longer be a fail.
Task-number: QTBUG-72928
Change-Id: Ia2232fdc714d405fa3fd9aea6c89eb2836bc5950
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The Test Anything Protocol (TAP), was originally Perl's simple text-based
interface between testing modules and test harnesses, but has since been
adopted by a large number of producers and consumers in many different
languages, which allows colorizing and summarizing test results.
The format is very simple:
TAP version 13
ok 1 - test description
not ok 2 - test description
---
message: 'Failure message'
severity: fail
expected: 123
actual: 456
...
ok 3 - test description # SKIP
1..3
The specification [1] is very brief, so the implementation has been
based on how typical consumers behave, especially when it comes to
the undefined diagnostics block.
[1] http://testanything.org/tap-version-13-specification.html
Change-Id: I616e802ea380165c678510e940ddc6607d39c92d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
From Qt 5.7 -> LGPL v2.1 isn't an option anymore, see
http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/
Updated license headers to use new LGPL header instead of LGPL21 one
(in those files which will be under LGPL v3)
Change-Id: I046ec3e47b1876cd7b4b0353a576b352e3a946d9
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
Brings down gui/painting/qcolor from 7s to 3s, reducing the calls
to printf() helpers from 16E6 to 10E6.
Task-number: QTBUG-38890
Change-Id: I34065e6f09fc9a14920b06aa6936908229f3f9c4
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
Qt copyrights are now in The Qt Company, so we could update the source
code headers accordingly. In the same go we should also fix the links to
point to qt.io.
Outdated header.LGPL removed (use header.LGPL21 instead)
Old header.LGPL3 renamed to header.LGPL3-COMM to match actual licensing
combination. New header.LGPL-COMM taken in the use file which were
using old header.LGPL3 (src/plugins/platforms/android/extract.cpp)
Added new header.LGPL3 containing Commercial + LGPLv3 + GPLv2 license
combination
Change-Id: I6f49b819a8a20cc4f88b794a8f6726d975e8ffbe
Reviewed-by: Matti Paaso <matti.paaso@theqtcompany.com>
We need to have a finer grained control over the tests
we skip in our CI system. This adds a blacklisting
mechanism that allows blacklisting individual test
functions (or even test data) using a set of predefined
matching keys for the operating system and some other
relevant variables.
QTestlib will search for a file called BLACKLIST in the test
directory and parse it if found. The file contains a simple
ini style list of functions to blacklist. For details see
qtestblacklist.cpp.
Change-Id: Id3fae4b264ca99970cbf9f45bfb85fa75c1fd823
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
On QNX mbstowcs does not ignore last parameter (max) when first (dest)
is NULL. Set it to sufficiently large value to yield proper results on
QNX. Other platforms (standard libraries) will ignore this value anyway.
Change-Id: Ie4695254d45082e151a052bf16de684af3b1ba1e
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
It's declared in qtestcase.h as a function, so let's not declare the one
in qtestresult.cpp as a static variable. None of the variables in
qtestresult.cpp need to be in the QTest namespace, but we don't need to
change them now.
Change-Id: If6cc34642fdfe3ccda3b8cea7d053ead0db9ccbd
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
d040681b6f added support for aligning the
test results for easier side-by-side comparison of the actual and
expected values. However, it didn't take into account multibyte strings.
That is, we would see:
FAIL! tst_testcase::testcase: Compared values are not the same
Actual (QString("é")): F0O
Expected (expected) : FOO
We use mbstowcs (multibyte string to wide char string) that calculates
the length in wide chars of the output string. That's roughly equivalent
to QString::fromLocal8Bit(string).toUcs4().size().
Change-Id: Ic2649951c50e05143da32a7fbef00a01e385c542
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Except where we're actually testing QCoreApplication::applicationName()
and friends.
Change-Id: I25514884c11f43a4f82b1f818f822dc3d79f69a3
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@digia.com>
Before:
FAIL! tst_testcase::testcase: Compared values are not the same
Actual (actual): F0O
Expected (expected): FOO
Now:
FAIL! tst_testcase::testcase: Compared values are not the same
Actual (actual) : F0O
Expected (expected): FOO
Change-Id: I6f0768e4ef53e065b85a56879cecbad06fa34aef
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Change copyrights and license headers from Nokia to Digia
Change-Id: If1cc974286d29fd01ec6c19dd4719a67f4c3f00e
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com>
Make the various versions of the failure message align consistently so
that it's a little easier to compare the actual and expected values. Of
course, the value won't align nicely unless the "actual" and "expected"
strings are the same length, but at least this commit makes that
consistent across all versions of the message.
Change-Id: If9ce231df3b5d279a06f6458fdb5da0aa4586068
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The statementStr parameter should always be non-null, so assert if it is
null. The description parameter can be null in some cases (particularly
when the verify is successful, and thus no error description is going to
be displayed), so tolerate this.
Change-Id: I87b416d5f3b793bc608cd4aca14a4f7fe7527488
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Giuseppe D'Angelo
Marc Mutz
Thiago Macieira
Vladimir Belyavsky
Mitch Curtis
Mårten Nordheim
Volker Hilsheimer
Ivan Solovev
Lucie Gérard
Sona Kurazyan
Edward Welbourne
Tor Arne Vestbø
Friedemann Kleint
Ulf Hermann
Oliver Wolff
Jani Heikkinen
Friedemann Kleint
Matti Paaso
Lars Knoll
Bernd Weimer
Tor Arne Vestbø
Sergio Ahumada
Iikka Eklund
Jason McDonald