Commit Graph

1634 Commits (dcacb0be973cfc17f495fef6d2da73471c3360ea)

Author SHA1 Message Date
Edward Welbourne 970841235b Split multi-file Files entries in qt_attribution.json as lists
This is now the official format for Files, when there's more than one,
rather than using space-joined lists.

Pick-to: 6.5
Change-Id: I4a6247fff0ece8ece2944178af38894fd5a2e1e2
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Kai Köhne <kai.koehne@qt.io>
2023-04-20 14:17:26 +01:00
Edward Welbourne ce8839e056 Deploy Comment fields in qt_attribution.json files
Replace the old abuse of other fields as comments, to be overwritten
by a later setting to a proper value, with actual Comment fields, now
that we have them.

Added a new comment to the valgrind files to say where they come from
in the upstream.

Pick-to: 6.5
Change-Id: I2edcfa2949fa9e59f3f67d3e578d8e5009854cf6
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Kai Köhne <kai.koehne@qt.io>
2023-04-20 14:17:26 +01:00
Friedemann Kleint 69d4ecd6ef Remove unused documentation code snippets
Modules:
- Core
- Gui
- Widgets
- Open(Widgets)
- PrintSupport
- Sql
- Network
- Concurrent
- Testlib

Pick-to: 6.5
Change-Id: I63e58c01bec4bd162486020f0085227fdaa83b18
Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
Reviewed-by: Christian Ehrlicher <ch.ehrlicher@gmx.de>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2023-04-17 21:02:32 +02:00
Ahmad Samir 7191b8fe38 Fix some narrowing conversion warnings
ioctl call for FIONREAD takes an int arg, it won't work correctly
otherwise. Cast the return of the read() call to int, because it won't
read more than buffSize (which is an int).

Change-Id: I130202a732684257bbb0e79c9358b60a61010c46
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2023-04-08 13:24:04 +02:00
Edward Welbourne 9896ebf93d Testlib: fix the last few s/Q_OS_MAC/Q_OS_DARWIN/w
Change-Id: I6fe6a865aea37c6a2d153bd2c3aace5242362c88
Reviewed-by: Christian Ehrlicher <ch.ehrlicher@gmx.de>
2023-03-20 19:18:52 +01:00
Ahmad Samir f6b780b583 QtMiscUtils: add isAsciiPrintable() helper
- isAsciiPrintable: borrowed the logic from isPrintable(uchar) in
  qdebug.cpp, now isPrintable(uchar) calls isAsciiPrintable

- Replace ::isspace() with ascii_isspace() from qlocale_p.h

Functions from ctype.h have a couple of issues:
- they're locale-dependent, which isn't what's required here, the code
  is only interested in ASCII
- their args should be representable as uchar or EOF otherwise it's
  undefined behavior

An Early-Warning could use something like:
grep -rP "\b(isalnum|isalpha|isblank|iscntrl|isdigit|islower|isgraph|\
         "isprint|ispunct|isspace|isupper|isxdigit|tolower|toupper)\b"

Pick-to: 6.5
Task-number: QTBUG-111262
Change-Id: I1f47f41bd56735297078ff0d0ddc2ebf999abf8b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2023-03-18 12:00:56 +02:00
Mårten Nordheim 1352e294d2 QTest: Rename QCOMPARE_XX parameters to add meaning
The parameter names were previously "lhs" and "rhs", which is incredibly
abstract for a testing framework. One of the parameters will tend to be
a baseline value to compare against while the other is the result of
some action we want to test. Thus I suggest they be renamed "computed"
and "baseline".

This way we can, hopefully, retain the semantic that the 'left'/first
argument is the computed ('actual' in QCOMPARE) value while the
'right'/second argument is the baseline ('expected' in QCOMPARE.)

Change-Id: I3e0fdce2a3f1faca06fdf7184ef6e0eb9724d990
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
2023-03-08 15:10:21 +01:00
Volker Hilsheimer 9e31631557 Doc: fix link to QTest::failOnWarning
Also fix incorrect failOnWarning*s* elsewhere in comments.

Pick-to: 6.5 6.5.0
Change-Id: I0bbc5e71060348153876f3d9a7c77c67f3c2e00d
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2023-03-05 06:00:39 +01:00
Ahmad Samir 6f9ace5685 QSignalSpy: add wait(std::chrono::milliseconds) overload
Make the wait(int) overload call the new one.

Task-number: QTBUG-110059
Fixes: QTBUG-100041
Change-Id: Ia085453c05e09e219ba56010b2504113bbc1dd34
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2023-03-03 21:36:55 +02:00
Ahmad Samir 2a495c2596 QTestEventLoop: add enterLoop(std::chrono::milliseconds) overload
Task-number: QTBUG-110059
Change-Id: Ibf1d76afd313e390103be4a22e44af7fb41ace1b
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2023-03-03 21:36:48 +02:00
Volker Hilsheimer c1cbb12e57 Doc: fix links to qFuzzyIsNull
The function is in the QtNumeric documentation context.

Pick-to: 6.5 6.5.0
Change-Id: Ibc6a8961bc04efd4ab1eb50bb371e6a169d513dc
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2023-03-03 20:05:27 +01:00
Andreas Eliasson 8ecd81ae86 Doc: Remove duplicate words
Change-Id: Ia7a38a1035bd34d00f20351a0adc3927e473b2e7
Pick-to: 6.5 6.4 6.2
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
2023-02-28 16:53:10 +00:00
Ahmad Samir 7941d3ab3e QTestTable: quote tag name in duplicate tag warning message
Easier to see what the duplicate tag name is.

Change-Id: Iee156aa7d6766628ec60e712811a83a2ff66dbb1
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2023-02-22 14:47:14 +02:00
Edward Welbourne daceee6a90 Update Valgrind to its 3.20.0
It now generates its valgrind.h from a valgrind.h.in, but there were
only two @...@ tokens to substitute, so that was easy enough.

From reading the diff, important changes:
* Added x86 support on FreeBSD (32- and 64-bit)
* Added nanoMIPS support on Linux
* Inline assembly fixes for S390
* Added VALGRIND_CLO_CHANGE request

Pick-to: 6.5
Task-number: QTBUG-110999
Change-Id: I66990c958cf40c34e7d58dd60b1e11984303fb64
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2023-02-20 18:30:44 +01:00
Friedemann Kleint 08c2010708 src: Remove remains of qmake conversion from CMakeLists.txt files
Pick-to: 6.5
Change-Id: Id644d322a602038403bb7f46c532744575fbf6d3
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2023-02-10 23:36:51 +01:00
Edward Welbourne 24f3e0f21c Improve documentation of data-driven tests
Mention addRow() as well as newRow(), even though the example only
uses the latter. Link the best-practice section to the fuller story.

Fixed a minor grammar glitch in the manual while I was about it.

Pick-to: 6.5
Change-Id: Ib1c52cd8d2b6a04ea944d24d9d26c901b6cdf4e7
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
2023-01-31 17:35:13 +01:00
Edward Welbourne 2d73cc2b1d Rewrite the QtTest best practice guide on testing for warnings
It previously only covered ignoring them; we can now turn them into
errors. Expand the section to include some motivation for why it is
best practice to purge warnings from test output. That prepares the
scene for how to suppress what's expected and make errors of the rest.

Pick-to: 6.5
Change-Id: Ieb4b14b2b048d1db2fead004ee7471b82507722f
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
2023-01-31 17:35:13 +01:00
Edward Welbourne 2d1fd400fc Object to creating duplicate entries in a test-data table
Tero Heikkinen caught tst_QQuaternion() using a duplicated test data
tag and was surprised that testlib let it get away with that. That
seems like a reasonable thing to discourage. While I'm at it,
duplicate columns should be discouraged.

[ChangeLog][QtTest] Duplicate data tags are now warned about. Every
call to QTest::newRow() or QTest::addRow() should result in a distinct
data tag. If they do not, a warning is produced. Likewise, duplicate
column names are forbidden; each call to QTest::addColumn() should use
a distinct name.

Fixes: QTBUG-107185
Pick-to: 6.5
Change-Id: Idfdb7cdfdd71e1fe3db5cadb243eeecc83032922
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2023-01-31 17:35:12 +01:00
Edward Welbourne 9a756e3e01 Cross-reference newRow() and addRow() in QTest docs and improve wording
Mitch helpfully pointed out, in review of my change to the "Data
Driven Testing" doc that the two functions didn't \sa each other, so I
added that. Which naturally prompted a review of their language.
Also, split a long line in the code.

Pick-to: 6.5
Change-Id: I5f3e973052e0107bd5bdaeecaebe218e043380ff
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2023-01-31 04:46:58 +01:00
Mikolaj Boc 9d5035be9f Don't implicitly cast BATCHED_TEST_NAME to qstring
Some tests disable implicit casts from char * to qstring. Use
QStringLiteral in qtest.h to avoid compilation problems with such
tests.

Change-Id: I977b6d90e5a7a8f23298b13789302501606f63d3
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2023-01-31 02:59:17 +01:00
Friedemann Kleint 69e478480a QTestlib: Disambiguate static functions
They cause clashes in CMake Unity (Jumbo) builds.

Task-number: QTBUG-109394
Pick-to: 6.5
Change-Id: I2920cc3f36ad60be1ee618c16653df5bc259019a
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2023-01-23 14:57:55 +01:00
Tasuku Suzuki e0a5915f88 Fix build with -no-feature-shortcut
qplatformtheme.h complains about Q_GADGET without <QtCore/QKeySequence>

Pick-to: 6.5
Change-Id: I7561bd3e2484c44842cd25ab12aacb9c00ebedfd
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2023-01-17 03:59:09 +09:00
Kai Köhne 9ae91ac318 Doc: Mark qRegisterTestCase as new in Qt 6.5
Also remove reference to QTestCaseEvent loop class, which
is not documented.

Pick-to: 6.5
Change-Id: Ic09b3b102db535dfb090b9a4072ad29515a911ae
Reviewed-by: Mikołaj Boc <Mikolaj.Boc@qt.io>
2023-01-16 08:38:52 +01:00
Jaishree Vyas 46d2b45d2c Doc: Increase Test Function Timeout
Documentation on QTEST_FUNCTION_TIMEOUT environment variable

Pick-to: 5.15 6.2 6.4 6.5
Fixes: QTBUG-88652
Change-Id: Ib851eb2312088cf6b9ab277faa571757f4076ad4
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-12-21 16:48:50 +00:00
Marc Mutz 26c190f57e QTest::WatchDog: fix missing timeout resets on test function change
Since e0cad1aab5, the code suffered
from an ABA problem where the TestFunctionStart expectation is set
in testFinished(), but by the time WatchDog::run() gets around to
examining the state when returning from condition_variable::wait()
in waitFor(), the next beginTest() has already set the expectation
back to TestFunctionEnd.

There are several known solutions for ABA problems. Embedding a
generation count into the expectation state seemed to be the most
straight-forward fix, and can be done without DWCAS support, because
the state is just 2 bits, leaving the other 30 or 62 bits for the
generation counter, so do that.

[ChangeLog][QTestLib] Fixed a bug which caused
QTEST_FUNCTION_TIMEOUT to be applied to the whole test execution,
as opposed to each test function.

Fixes: QTBUG-109466
Pick-to: 6.5 6.4 6.2 5.15
Change-Id: If71ade932330407b85d204d45c74350c651325fe
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-12-16 22:58:10 +01:00
Marc Mutz 17a7e5cfdd QTest::WatchDog: Extract Method setExpectation()
Keeps the code DRY and enables a follow-up commit to fix QTBUG-109466.

Task-number: QTBUG-109466
Pick-to: 6.5 6.4 6.2 5.15
Change-Id: I2b904ea7b38286b07049524ba63c2c5028e680bb
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-12-16 22:58:10 +01:00
Kai Köhne 486c8aae62 Doc: Change links from froglogic to qt domain
Pick-to: 5.15 6.2 6.4
Change-Id: Ic249f8dfa168948eddc83f35314708ffb40e75b1
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
2022-12-09 11:40:49 +01:00
Andreas Eliasson ef1c1132e6 Doc: Remove qtestlib-tools section
qtestlib-tools is dead.

Fixes: QTBUG-107806
Pick-to: 6.4 6.2
Change-Id: Ic479b9e133c1889a046d1b7483af9d8ad8ac4968
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
2022-11-30 14:48:50 +01:00
Tor Arne Vestbø cbe7cead44 Add note to blacklist documentation about how keywords relate
Fix up deprecated macOS keyword while we're at it.

Change-Id: I6f6ba21bde161d3dd3ac6c243f1b5f7c16ef3228
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2022-11-18 19:25:20 +01:00
Thiago Macieira 224d795ec9 QPlainTestLogger: include extra information in each result
RESULT : tst_MyClass::QString_toInt()
     367 nsecs per iteration (total: 3,672,848, iterations: 10000)
     1,019 CPU cycles per iteration, 2.775 GHz (total: 10,193,637, iterations: 10000)
     3,123 instructions per iteration, 3.055 instr/cycle (total: 3,123,000,194, iterations: 1000000)
     536 branch instructions per iteration, 1.459 G/sec (total: 5,360,022, iterations: 10000)

Change-Id: I3c79b7e08fa346988dfefffd17203fff3ff5b774
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 21:34:13 -07:00
Thiago Macieira f341e99aab QPlainTestLogger: don't print "RESULT" before each additional result
We do that by passing the full list of results to the logger, to a
virtual that is present in the base class to call the existing
function. For all but the plain logger, we'll just print multiple
results. The plain logger now prints:

RESULT : tst_MyClass::QString_toInt()
     383 nsecs per iteration (total: 3,837,324, iterations: 10000)
     1,069 CPU cycles per iteration (total: 10,692,457, iterations: 10000)
     3,123 instructions per iteration (total: 31,230,101, iterations: 10000)
     536 branch instructions per iteration (total: 5,360,022, iterations: 10000)

Change-Id: I3c79b7e08fa346988dfefffd17203cb5802693dd
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 20:34:12 -08:00
Thiago Macieira 0e8eb20af4 QPlainTestLogger: simplify printBenchmarkResult()
Avoids having lots of buffers that occupy the stack.

Change-Id: I3c79b7e08fa346988dfefffd17203c5045a7fc86
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 21:34:11 -07:00
Thiago Macieira 393d5efda3 QVariant: make a major simplification in the numeric comparison
The code implementing the C++ rules of type promotion and conversion
was too pedantic. There's no need to follow the letter of the standard,
not when we can now assume that everything is two's complement (this was
true for all architectures we supported when I wrote this code in 2014,
but wasn't required by the standard).

So we can reduce this to fewer comparisons and fewer rules, using the
size of the type, not just the type ID.

Change-Id: I3d74c753055744deb8acfffd172446b02444c0c0
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 04:05:50 -07:00
Thiago Macieira a90f041a0a QPlainTestLogger::formatResult: use QByteArray instead of QString
Avoids operating in UTF-16 and having to convert.

Drive-by fix int to qsizetype.

Change-Id: I3c79b7e08fa346988dfefffd17203b044914d68d
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 02:18:45 -07:00
Thiago Macieira dbc9467ca3 QBenchlib/Perf: use one system call to enable/disable measurements
The man page says the prctl() should apply to any performance counters
measuring the current process, so it should work with a perf stat
started with --delay=-1... but I couldn't make that work.

Change-Id: I3c79b7e08fa346988dfefffd172032f06cc10fa0
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 02:18:45 -07:00
Thiago Macieira 1fef390ad7 QBenchlib/Perf: use C++11 range-fors for the event list
This code predated C++11 and thus had a sentinel element.

Change-Id: I3c79b7e08fa346988dfefffd17202ef3f9735683
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 02:18:45 -07:00
Thiago Macieira 3cafd26c1d QBenchlib/Perf: parse multiple counters from -perfcounter
Change-Id: I3c79b7e08fa346988dfefffd17202ec98947308e
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 02:18:45 -07:00
Thiago Macieira 4731baf6d3 QBenchlib: add support for a measurer reporting multiple results
Implemented for the Linux Perf measurer, with four measurements by
default.

RESULT : tst_MyClass::QString_toInt():
     149.574444 CPU cycles per iteration (total: 149,574,445, iterations: 1000000)
RESULT : tst_MyClass::QString_toInt():
     620.000181 instructions per iteration (total: 620,000,182, iterations: 1000000)
RESULT : tst_MyClass::QString_toInt():
     131.000046 branch instructions per iteration (total: 131,000,047, iterations: 1000000)
RESULT : tst_MyClass::QString_toInt():
     32.118771 nsecs per iteration (total: 32,118,771, iterations: 1000000)

Change-Id: I3c79b7e08fa346988dfefffd17202cda3df8431b
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 02:18:45 -07:00
Thiago Macieira b5b00e7790 QBenchlib: use QBenchmarkMeasurerBase::Measurement in QBenchmarkResult
Change-Id: I3c79b7e08fa346988dfefffd17202a818cde1d84
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 02:18:44 -07:00
Thiago Macieira 985b942152 QBenchlib: pass the metric type alongside the measurement value
And pass the value in a qreal, which is what QBenchlib stores
anyway. This increases the code size a little because the conversion
from integer to qreal is in multiple places, but doesn't meaningfully
increase the overhead: in the SysV ABI, we still return Measurement in
registers and even using the floating point registers where applicable.

This is the first step in allowing the Perf benchmarker to benchmark more
than one event.

Change-Id: I3c79b7e08fa346988dfefffd172027a8677f17c0
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-11-09 02:18:44 -07:00
David Skoland ffb44ad8ef wasm: Add missing batchtestrunner files to copy logic
qtestoutputreporter.js and qtestoutputreporter.css were added to
the batchtestrunner, but not to the list of files copied from
sources, so make sure to add those and add a note for future reference.

Change-Id: If7f323b3051ee9b6fb4bedc3ee314e4941b41f0b
Reviewed-by: Mikołaj Boc <Mikolaj.Boc@qt.io>
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2022-11-07 16:56:46 +02:00
Marc Mutz 1c6bf3e09e Port from container::count() and length() to size() - V5
This is a semantic patch using ClangTidyTransformator as in
qtbase/df9d882d41b741fef7c5beeddb0abe9d904443d8, but extended to
handle typedefs and accesses through pointers, too:

    const std::string o = "object";

    auto hasTypeIgnoringPointer = [](auto type) { return anyOf(hasType(type), hasType(pointsTo(type))); };

    auto derivedFromAnyOfClasses = [&](ArrayRef<StringRef> classes) {
        auto exprOfDeclaredType = [&](auto decl) {
            return expr(hasTypeIgnoringPointer(hasUnqualifiedDesugaredType(recordType(hasDeclaration(decl))))).bind(o);
        };
        return exprOfDeclaredType(cxxRecordDecl(isSameOrDerivedFrom(hasAnyName(classes))));
    };

    auto renameMethod = [&] (ArrayRef<StringRef> classes,
                            StringRef from, StringRef to) {
        return makeRule(cxxMemberCallExpr(on(derivedFromAnyOfClasses(classes)),
                            callee(cxxMethodDecl(hasName(from), parameterCountIs(0)))),
                        changeTo(cat(access(o, cat(to)), "()")),
                        cat("use '", to, "' instead of '", from, "'"));
    };

    renameMethod(<classes>, "count", "size");
    renameMethod(<classes>, "length", "size");

except that the on() matcher has been replaced by one that doesn't
ignoreParens().

a.k.a qt-port-to-std-compatible-api V5 with config Scope: 'Container'.

Added two NOLINTNEXTLINEs in tst_qbitarray and tst_qcontiguouscache,
to avoid porting calls that explicitly test count().

Change-Id: Icfb8808c2ff4a30187e9935a51cad26987451c22
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-11-03 14:59:24 +01:00
Mitch Curtis c30c76c844 Doc: link to the testlib tutorial section on skipping
Short of expanding Qt Test Best Practices to have a section about
QEXPECT_FAIL, we can link to some existing documentation that briefly
covers it.

Pick-to: 6.2 6.4
Change-Id: I37bf2672e4b5b6e7315e009f86d03eeb7937d50e
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
2022-10-31 15:38:47 +08:00
Edward Welbourne df1935df0e Correct (and reflow) an out-of-date comment on benchmarking
What's now called TestMethods::invokeTest() was apparently once called
qInvokeTestMethod(). Also, its current instance is (now) cleared by
its destructor rather than (overtly) at the end of that function.

Change-Id: I04de7ca2247dd640a398b3e4e7bf410401f3cbbf
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2022-10-28 23:42:39 +02:00
Thiago Macieira 209adad078 QBenchlib/Perf: remove the handling of attributes in -perfcounter
I don't think I understood what this does when I coded it. It wasn't
needed with the Linux kernels of the era.

[ChangeLog][QtTest] Specifying attributes with the -perfcounter command-
line option for Linux performance counters is deprecated. QtTest will
ignore the colon and any attributes listed there, though future versions
of QtTest may reintroduce attributes if needed.

Change-Id: I3c79b7e08fa346988dfefffd17202684fc2cf650
Reviewed-by: David Faure <david.faure@kdab.com>
2022-10-25 15:13:57 -07:00
Thiago Macieira e1089e0520 QBenchlib/Perf: don't try to benchmark the kernel
The kernel has become more paranoid since 2013, when I originally wrote
this code. The kernel.perf_event_paranoid sysctl controls the paranoia
level[1]:

===  ==================================================================
 -1  Allow use of (almost) all events by all users.

     Ignore mlock limit after perf_event_mlock_kb without
     ``CAP_IPC_LOCK``.

>=0  Disallow ftrace function tracepoint by users without
     ``CAP_PERFMON``.

     Disallow raw tracepoint access by users without ``CAP_PERFMON``.

>=1  Disallow CPU event access by users without ``CAP_PERFMON``.

>=2  Disallow kernel profiling by users without ``CAP_PERFMON``.
===  ==================================================================

Since the default is 2, we QBenchlib has been failing with EACCESS:

PASS   : tst_MyClass::initTestCase()
QBenchmarkPerfEventsMeasurer::start: perf_event_open: Permission denied

[ChangeLog][QtTest] Fixed support of Linux performance counters for
QBENCHMARK, which used to fail with "Permission denied" errors in
default configurations. Now, QtTest will automatically fall back to
profiling only userspace, like the perf(1) tool does.

[1] https://github.com/torvalds/linux/blob/master/Documentation/admin-guide/sysctl/kernel.rst#perf-event-paranoid

Pick-to: 5.15 6.2 6.4
Change-Id: I3c79b7e08fa346988dfefffd171f897be7794935
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
Reviewed-by: David Faure <david.faure@kdab.com>
2022-10-25 15:13:57 -07:00
Thiago Macieira feb6463772 QBenchlib: force the warmup to run a single iteration
The purpose of warming up is to get all code paths executed, so any lazy
function resolving is processed, statics are allocated, etc. There's no
reason to run it more than once -- if you're trying to train the Branch
Predictor Unit, you'd want to do it another way anyway. This is useful
when benchmarking with -iterations N, because QBenchlib currently runs
2*N iterations because of the warm up. That just wastes time.

Change-Id: I3c79b7e08fa346988dfefffd172030c889b31a1f
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-10-25 11:14:35 -07:00
Thiago Macieira 739fb98233 QBenchlib: stop exporting {begin,end}BenchmarkMeasurement
I don't see anything using them in any unit test or other Qt module, and
they are marked internal.

Change-Id: I3c79b7e08fa346988dfefffd172028dae6d9e1ba
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-10-24 21:00:11 -07:00
Thiago Macieira 1a769c36e4 QBenchlib: remove unused checkpoint() virtual
It was probably part of the original design to have an intermediate
measurement to determine whether the results were acceptable or not, but
it was never used. So just remove.

Change-Id: I3c79b7e08fa346988dfefffd1720281b2512b8c1
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2022-10-24 21:00:11 -07:00
Thiago Macieira 148c3eaa29 QBenchlib: remove unused repeatCount() virtual
It's not called.

Change-Id: I3c79b7e08fa346988dfefffd172026d41b70ad6f
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-10-24 21:00:11 -07:00
Thiago Macieira ef6b5fe63f QBenchlib/Perf: remove ioctl(FD_CLOEXEC)
Commit 8995045c62 forgot to remove this
portion.

Pick-to: 6.4
Change-Id: I3c79b7e08fa346988dfefffd17202b19665ddc8a
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
2022-10-24 21:00:11 -07:00
Thiago Macieira e693a5ccaf QBenchlib/Perf: use WalltimeNanoseconds for the task time
Avoids the special case in the code and removes the need to run a large
number of iterations to get any useful result, plus makes it's more
readable:

From:
    0.00002125 msecs per iteration (total: 2,125, iterations: 100000000)

To:
    22.082 nsecs per iteration (total: 22,082, iterations: 1000)

Pick-to: 6.4
Change-Id: I3c79b7e08fa346988dfefffd171facc98af9e9fc
Reviewed-by: David Faure <david.faure@kdab.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-10-24 21:00:11 -07:00
Luca Di Sera e2fef733be Replace usages of Q_CLANG_QDOC with Q_QDOC
To allow the user to customize the C++ code that QDoc sees, so as to be
able to work-around some limitations on QDoc itself, QDoc defines two
symbols: Q_QDOC and Q_CLANG_QDOC, both of which are "true" during an
entire execution of QDoc.

At a certain point in time, QDoc allowed the user the choice between a
custom C++ parser and a Clang based one.

The Q_QDOC symbol would always be defined while the Q_CLANG_QDOC symbol
would be defined only when the Clang based parser was chosen.

In more recent times, QDoc always uses a Clang based parser, such that
both Q_CLANG_QDOC and Q_QDOC are always defined, making them equivalent.

To avoid using different symbols, and the possible confusion and
fragmentation that derives from it, all usages of Q_CLANG_QDOC are now
replaced by the equivalent usages of Q_QDOC.

Change-Id: I5810abb9ad1016a4c5bbea99acd03381b8514b3f
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2022-10-21 09:48:36 +02:00
Thiago Macieira 8995045c62 QBenchlib/Perf: open the perffd with PERF_FLAG_FD_CLOEXEC
This flag was introduced in the Linux kernel version 3.14. Trying to use
perf_event_open() now with a kernel older than that will cause EINVAL
errors, but 3.14 is from 2014, so most likely old enough for all of
Qt 6.

For that, I updated the copied header from v6.0.

Pick-to: 6.4
Change-Id: I3c79b7e08fa346988dfefffd171f895201ceb4f4
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-10-20 08:39:16 -07:00
Edward Welbourne d39f1da6e3 Simplify a list-removal operation in QTestLog's handleIgnoredMessage()
When list->next is nullptr, setting ignoreResultList to list->next is
the same as setting it to nullptr.

Change-Id: If328ce31db4344cf0136b7a827be871eb89e35b5
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2022-10-18 14:13:28 +02:00
Shawn Rutledge 0f94430a0f testlib: make it possible to test double-clicks with discrete events
The timestamp will no longer be incremented by 500ms after a mouse
release if the delay has been explicitly specified.

The default delay is 1 ms since f5010c49a3
but the running timestamp was unconditionally post-incremented by 500ms
after every mouse release, to prevent double-clicks, which were always
deemed as unintended (because we have a mouseDClick function for that).
Now, we do that 500ms increment only if the user has not provided a
delay value in the function argument at all. We have often found it
useful in our own tests to generate double-clicks "the hard way", by
sending indivdual events, so as to be able to check state in some target
object at each step, as shown in the new snippet.

[ChangeLog][QtTest] QTest::mouseRelease() and mouseClick() can now be
used to test double-clicks, by specifying a realistic timestamp delay.

Fixes: QTBUG-102441
Change-Id: I8e8d242061f79efb4c6e02638645e03661a9cd92
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
2022-10-16 08:36:03 +02:00
Marc Mutz fc76767692 Long live Q_UNREACHABLE_RETURN()!
This is a combination of Q_UNREACHABLE() with a return statement.

ATM, the return statement is unconditionally included. If we notice
that some compilers warn about return after __builtin_unreachable(),
then we can map Q_UNREACHABLE_RETURN(...) to Q_UNREACHABLE() without
having to touch all the code that uses explicit Q_UNREACHABLE() +
return.

The fact that Boost has BOOST_UNREACHABLE_RETURN() indicates that
there are compilers that complain about a lack of return after
Q_UNREACHABLE (we know that MSVC, ICC, and GHS are among them), as
well as compilers that complained about a return being present
(Coverity). Take this opportunity to properly adapt to Coverity, by
leaving out the return statement on this compiler.

Apply the macro around the code base, using a clang-tidy transformer
rule:

    const std::string unr = "unr", val = "val", ret = "ret";
    auto makeUnreachableReturn = cat("Q_UNREACHABLE_RETURN(",
                                    ifBound(val, cat(node(val)), cat("")),
                                    ")");
    auto ignoringSwitchCases = [](auto stmt) {
        return anyOf(stmt, switchCase(subStmt(stmt)));
    };

    makeRule(
       stmt(ignoringSwitchCases(stmt(isExpandedFromMacro("Q_UNREACHABLE")).bind(unr)),
            nextStmt(returnStmt(optionally(hasReturnValue(expr().bind(val)))).bind(ret))),
       {changeTo(node(unr), cat(makeUnreachableReturn,
                                ";")),  // TODO: why is the ; lost w/o this?
        changeTo(node(ret), cat(""))},
       cat("use ", makeUnreachableReturn))
    );

where nextStmt() is copied from some upstream clang-tidy check's
private implementation and subStmt() is a private matcher that gives
access to SwitchCase's SubStmt.

A.k.a. qt-use-unreachable-return.

There were some false positives, suppressed them with NOLINTNEXTLINE.

They're not really false positiives, it's just that Clang sees the
world in one way and if conditonal compilation (#if) differs for other
compilers, Clang doesn't know better. This is an artifact of matching
two consecutive statements.

I haven't figured out how to remove the empty line left by the
deletion of the return statement, if it, indeed, was on a separate
line, so post-processed the patch to remove all the lines matching
^\+ *$ from the diff:

  git commit -am meep
  git reset --hard HEAD^
  git diff HEAD..HEAD@{1} | sed '/^\+ *$/d' | recountdiff - | patch -p1

[ChangeLog][QtCore][QtAssert] Added Q_UNREACHABLE_RETURN() macro.

Change-Id: I9782939f16091c964f25b7826e1c0dbd13a71305
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-10-15 22:11:47 +02:00
David Skoland 306f1af145 wasm: change logic for copying batchedtestrunner files to build dir
The previous config did not work for prefix builds (files were not
copied over). To fix this, we simply copy over the files in case
of a prefix build.

Additionally removed batch_test_feature from the condition,
as this is now the de facto wasm test runner, which supports
batched and unbatched tests.

Change-Id: Ib232c7898de0a0d750e4ca5ebf1da8cbcc7da3e0
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-10-15 19:50:41 +02:00
Marc Mutz aa37e67ef7 Port from qAsConst() to std::as_const()
We've been requiring C++17 since Qt 6.0, and our qAsConst use finally
starts to bother us (QTBUG-99313), so time to port away from it
now.

Since qAsConst has exactly the same semantics as std::as_const (down
to rvalue treatment, constexpr'ness and noexcept'ness), there's really
nothing more to it than a global search-and-replace, with manual
unstaging of the actual definition and documentation in dist/,
src/corelib/doc/ and src/corelib/global/.

Task-number: QTBUG-99313
Change-Id: I4c7114444a325ad4e62d0fcbfd347d2bbfb21541
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
2022-10-11 23:17:18 +02:00
Dmitry Shachnev 638c4a3014 Fix file which had SPDX-License-Identifier on two lines
This breaks our automation in Debian.

Pick-to: 6.4
Change-Id: Iee035dee8e0d89e22540a2691689da46c4c301a3
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2022-10-11 14:10:58 +00:00
Kai Köhne 7c43feb456 Add "We mean it" header
Also use canonical Qt header include.

Change-Id: I5c1133fc088ef3e29d3336591bc1f9a98868db8c
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-10-05 09:05:54 +02:00
Mikolaj Boc b9887d51c3 Provide visual output in page in WASM test runner
There will now be a visual output in page if the qvisualoutput query
parameter is supplied. This simplifies debugging.

The main html resource has been renamed test_batch.html to reflect
the name of the actual test unit, not functionality.

Change-Id: Ib6cd4712de9c47cfcc5f670e7b34f998858f99b7
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2022-10-05 00:36:41 +02:00
Marc Mutz df9d882d41 Port from container.count()/length() to size()
This is semantic patch using ClangTidyTransformator:

  auto QtContainerClass = expr(hasType(namedDecl(hasAnyName(<classes>)))).bind(o)
  makeRule(cxxMemberCallExpr(on(QtContainerClass),
                             callee(cxxMethodDecl(hasAnyName({"count", "length"),
                                                  parameterCountIs(0))))),
           changeTo(cat(access(o, cat("size"), "()"))),
           cat("use 'size()' instead of 'count()/length()'"))

a.k.a qt-port-to-std-compatible-api with config Scope: 'Container'.

<classes> are:

    // sequential:
    "QByteArray",
    "QList",
    "QQueue",
    "QStack",
    "QString",
    "QVarLengthArray",
    "QVector",
    // associative:
    "QHash",
    "QMultiHash",
    "QMap",
    "QMultiMap",
    "QSet",
    // Qt has no QMultiSet

Change-Id: Ibe8837be96e8d30d1846881ecd65180c1bc459af
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-10-04 07:40:08 +02:00
Edward Welbourne 75d59c9502 Add QTest::currentTestResolved() and use in testlib
The QTRY_* macros and QTestEventLoop have (since 6.3) been exiting
their loops early if the test has failed. Where that was appropriate,
they should also have been exiting early on skip.

[ChangeLog][QtTest] Added QTest::currentTestResolved(), which is true
if the test has failed or skipped. The QTRY_*() macros and
QTestEventLoop now use this, rather than QTest::currentTestFailed(),
to test whether they should stop looping, so that they also do so on a
skip.

Task-number: QTBUG-104441
Change-Id: Ibf3d5a095b35e6670bc3daf756f05b66f7f3ef9b
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2022-09-22 17:34:51 +02:00
Edward Welbourne b3262c7e54 QDoc-ify QTest::runningTest()'s comment
While it's being picked into 6.3, it's not present in 6.3.[01], so
I've left it out of QDoc's sight (even though it's still \internal)
for the version picked to 6.3, but let's include it in internal docs
as "from 6.4" since that's the first minor release to contain it.

Pick-to: 6.4
Change-Id: I1704a1ca4ba1231d0213e9ca236ef8401a59ddd0
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2022-09-22 17:34:51 +02:00
Edward Welbourne 0462dba766 Skip early return from test loops during cleanup()
The QTRY_* macros and QTestEventLoop exit early if the test has
resolved; however, in the cleanup phase of a test, even if the test
has failed, these loops should continue as normal.

[ChangeLog][QtTest] During the cleanup() phase of a test, the QTRY_*
macros and QTestEventLoop now ignore the test resolution, in contrast
to when they are used from the test itself, which (since 6.3.0) exits
the loops early if the test has failed.

Pick-to: 6.4 6.3
Fixes: QTBUG-104441
Change-Id: I2673161967cbbc57815155af698a9338ab98a686
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2022-09-22 17:34:51 +02:00
Edward Welbourne abab3c5dad Fix error in TAP test logger's regex to recognize QVERIFY() output
The regex used a greedy .+ to match an actual dot and a space,
preceding an open parenthesis, with the result that if the message
contained any parentheses that .+ swallowed everything up to the last
of them. Correct the regex so that recently-added tests' error
messages show up correctly.

Change-Id: I6e52c9b2a6e7959335fcddbb4266f65b589eba68
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2022-09-22 17:34:51 +02:00
Thiago Macieira 986fef5ca1 QTeamCityLogger: fix build (INT_MAX undefined)
Change-Id: If4c23ea3719947d790d4fffd171525faa9f9ae09
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-09-16 03:01:58 -07:00
Edward Welbourne 5832087859 Include global data tags in teamcity's test function names
Use the standard mechanism instead of a hand-rolled way of identifying
function/data-tag combinations. Adds missing data relevant to
identifying test cases, some of which previously had apparently the
same name, with no hint at why the test-case was seemingly run
repeatedly.

Change-Id: I6225c6d1990069c94a1f1c8dbb179993b96076e7
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-09-13 14:51:00 +02:00
Edward Welbourne ceba44a3bf Convert teamcity logger to use QTestCharBuffer
All our other loggers use it and teamcity's use of QString implied
lots of conversions from UTF-8 and back, or in some cases to native
encoding, that ended up failing to correctly propagate Unicode tokens
that the other test loggers do deal with correctly. This does not
change the generated output on Linux.

Change-Id: I036d8043d7b3b19e7a0e1f296a23628f98223563
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2022-09-13 14:50:59 +02:00
Mikolaj Boc 7dbbe0a222 Adapt the js batched test runner to emrun interface
This makes the js batched test runner cooperate with emrun.
The runner sends back the output and exit messages to emrun to inform it
about the test execution state.
The code is based on emrun_postjs.js from the emsdk.

Change-Id: I758f2c185797f4000810eb4314423eebc1c5d457
Reviewed-by: David Skoland <david.skoland@qt.io>
2022-09-12 22:28:10 +02:00
Mårten Nordheim 95e315a66e JUnit: Don't turn seconds into kilo-seconds
Overlooked in the review where this changed, this value was already in
seconds and needs to be converted to millis before converting back.

Now printing output such as:

<testcase name="initTestCase" classname="tst_QHash" time="0.001"/>

Amends bb74e72aa9

Pick-to: 6.2 6.3 6.4 6.4.0
Fixes: QTBUG-106222
Change-Id: I1f8b774eea3dcbe2b4e822e2b0b2efb1ccc01abb
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-09-08 21:01:54 +00:00
Edward Welbourne f94f165336 Document shell-friendly data tags as best practice
Also follow this best practice in testlib's own documentation and
examples.

Pick-to: 6.4
Change-Id: I8b57dfa8f88835adae8fceeb122a16635708e338
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2022-09-07 15:29:38 +02:00
Alexey Edelev 05cc5b3541 Introduce the qt_deprecates pragma
The qt_deprecates pragma indicates that the file passed as argument
is deprecated and supposed to be replaced by the file where the pragma
is defined. Syncqt procedure generates the file passed as argument
automatically with the deprecation warning. After the deprecation
period the pragma should be removed and the deprecated file will not
be included to the Qt installation too.
The pragma is only handled by the cpp version of syncqt cpp and
supposed to replace the 'deprecatedheaders' record in sync.profiles.

Change-Id: Ibe69423a5de67f58907a3edbc5961f5ab63944de
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-09-06 20:15:00 +02:00
Volker Hilsheimer b05a2768c2 Fix warning from testing assignment
Tests that do

QTRY_VERIFY(a = b);

generate a compiler warning:

using the result of an assignment as a condition without parentheses
[-Wparentheses]: place parentheses around the assignment to silence
this warning

Do that centrally in the QTRY_TIMEOUT_DEBUG_IMPL macro definition.

Pick-to: 6.4
Change-Id: I4a4b0161c4e16c0e1e27a68f33a41efdaa2c962c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-09-03 19:10:42 +02:00
Edward Welbourne 6dd6bf3467 Fix two trivial defects in testlib docs
A fragment of inline code wasn't wrapped in \c{}.
Doing so made the quotes it was wrapped in redundant.

Documentation of of QSKIP's parameter claimed it does something it
should, but it's up to the client code using it to write a description
that lives up to that.

Pick-to: 6.4
Change-Id: Ib355b9b85c9986f244ae01479134245f182c912c
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2022-09-01 19:44:19 +02:00
Edward Welbourne 8814fb5f4f Provide CMake guidance for skipping whole tests
The testlib docs on how to skip whole tests was qmake-specific, so add
the missing details for CMake.

Pick-to: 6.4
Change-Id: I247d08262302dcedb98ffbf301932d6dd84d7ff7
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Venugopal Shivashankar <Venugopal.Shivashankar@qt.io>
2022-09-01 19:44:19 +02:00
Sona Kurazyan 711105058a Apply Q_CONSTINIT where beneficial
Applied Q_CONSTINIT to variables with static storage duration, but
skipped the POD types with core constant initializers.

Task-number: QTBUG-100486
Change-Id: Iaabf824e9cb0f29a405a149912200d4e4b3573c1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-09-01 19:44:19 +02:00
Mikolaj Boc 619cb795b2 Plug the WASM batched test runner into the build system
The new batched test runner is now used for running the tests instead
of the wasm shell, which runs for single test cases.

Change-Id: I7b7e6dd7993ba7937124c5843356b6891301b893
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-08-31 16:53:52 +02:00
Mikolaj Boc 8d728a0ed9 Implement the batch_tests feature
An approach of test batching (joining multiple tests into a single
binary) has been taken, due to long linking times/binary size on certain
platforms, including WASM. This change adds a new feature
'batch_test_support' in Qt testlib. Based on the value of the feature,
test batching may become enabled with the -batch-tests switch.

Batching works for every target added via qt_internal_add_test. When
first such target is being processed, a new combined target for all of
the future test sources is created under the name of 'test_batch'.
CMake attempts to merge the parameters of each of the tests, and some
basic checks are run for parameter differences that are impossible to
reconcile.

On the C++ level, convenience macros instantiating the tests are
redefined when batch_tests is on. The new, changed behavior triggered
by the changes in the macros registers the tests in a central test
registry, where they are available for execution based solely on their
test name. The test name is interoperable with the names CMake is aware
of, so CTest is able to run the tests one by one in the combined binary.

Task-number: QTBUG-105273
Change-Id: I2b6071d58be16979bd967eab2d405249f5a4e658
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
2022-08-24 02:46:37 +02:00
Lucie Gérard 32df595275 Change the license of all CMakeLists.txt and *.cmake files to BSD
Task-number: QTBUG-105718
Change-Id: I5d3ef70a31235868b9be6cb479b7621bf2a8ba39
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-23 23:58:42 +02:00
Mikolaj Boc 1b4a792071 Make it clear that a hyphen may be used to log to stdout in qtest
Clarify that the hyphen is the actual character that one can use.
The way the documentation was written might have been misleading.

Pick-to: 6.4
Change-Id: I55a9ede9903a8ac5bd33d8e138b3a0a21a820406
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2022-08-18 07:57:15 +02:00
Edward Welbourne f9b58b5c14 Include current test name in crash reports
Previously a crashing test's output would end with the last completed
test's output followed by a crash report, leading readers
(understandably enough) to conclude that the last-named test is the
one that crashed. In fact the crashing test is typically the next one
in the class definition.

Include the current test function's name (when non-null) in the output
accompanying crash logs. This always goes to stderr so does not show
up in the expected output.

Pick-to: 6.4
Change-Id: Icab0ccd1fe434827ee92459ab0c97f9dc034754e
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2022-08-12 20:32:30 +02:00
Oliver Wolff 9a5ac4bcb3 Add keyword msvc-2022 to testlib blacklisting
Pick-to: 6.2 6.3 6.4
Change-Id: I04942dc9474fd2abfc341a1dd6434f7c78b6b5bd
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-08-04 10:18:41 +00:00
Lucie Gérard fb1b20eab3 Add license headers to cmake files
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.

Existing copyright statements remain intact

Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-03 17:14:55 +02:00
André Klitzing d1dd944239 Fix race condition with QTest::ignoreMessage
If another spawned thread will log at the same time
a crash was possible because IgnoreResultList
was not thread-safe.

Task-number: QTBUG-104840
Pick-to: 6.4 6.3 6.2 5.15
Change-Id: I251c83bb28cb75e55d477706bf2524c061a6eab7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-07-28 13:11:54 +00:00
Thiago Macieira 37aa9a9cef WindowsFaultHandler: print the backtrace to stderr, not stdout
Otherwise the contents that vary per architecture and build will make it
impossible to self-check (tst_selftests)

Change-Id: Ibcde9b9795ad42ac9978fffd16f2cbc352c3503c
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-07-27 08:43:00 +02:00
Edward Welbourne 2648b4ad2c Skip some spurious parentheses on macro argument uses
When an argument to one macro appears as a single argument to another
macro, the usual "put it in parentheses" rule doesn't apply since it's
necessarily something the right shape to be a macro parameter. In some
cases, furthermore, the added parentheses could show up in the output
produced by a test when reporting an expression that was compared or
verified. Also removed the parentheses around one comparison.

Pick-to: 6.4
Change-Id: I226f4356fb057351187dac2134cc12c06026c40c
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
2022-07-25 19:29:07 +02:00
Edward Welbourne a670a00aed Skip semicolon after uses of QTRY_IMPL() and QTRY_LOOP_IMPL()
The macros' expansions end with an if-block and a for-block,
respectively, so the following semicolon is superfluous. Since we
control these macros' use (they're internals), just skip the
semicolons, rather than wrapping their bodies in do-while(0).

Pick-to: 6.4
Change-Id: I53f7786a66a0dc7709ac9f96d49985edafceec39
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
2022-07-25 19:29:07 +02:00
Edward Welbourne 4048efc80c Move the early-failure checks in QTRY_IMPL()
These checks should not be included in the #expr reported when the
test fails. They're an internal implementation detail of the loop.
Split a long line while I was about it.

Pick-to: 6.4
Change-Id: Iaea0478967d01cd72ef5a5e9a6501d4be2324b18
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
2022-07-25 19:29:07 +02:00
Tor Arne Vestbø 030923c0de Add -Wshorten-64-to-32 to headersclean
Fix existing warnings by casting to the appropriate type.

Change-Id: Ic44d2a71e1a2e508199dbb46bea7a19e183ec42c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-07-21 17:14:08 +02:00
Volker Hilsheimer 92e696b4ba Use debug stream in QTest::toString's default fallback if possible
For built-in types, this is a compile-time assert - we should not have
any types in Qt for which we have neither debug streaming nor a
QTest::toString specialization implemented. A build of most of Qt
submodules passes with this change, after minor modifications to some
tests. We cannot declare QSizeHint::Policy as a metatype after the
QMetaType has already been instantiated for it, and the QDebug stream
operator for QElaspedTimer needs to be correctly declared within the
namespace.

Add a self-test function for a custom type, and update reference files
of the self-test.

Task-number: QTBUG-104867
Pick-to: 6.4
Change-Id: I2936db5933f4589fce45f47cf2f3224ed614d8c9
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2022-07-13 15:14:35 +02:00
Edward Welbourne 281cada9c1 Replace 0 with \nullptr in QTest::currentDataTag()'s doc
The function's return type is a pointer, not an integer.

Change-Id: Iab6686ac7e4e24b9b1bd0127346c5854cf593a57
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-07-11 21:41:51 +02:00
Edward Welbourne 4d88c80fbd Improve formatting of QTest message on missing function
If there were no matches to the name given on the command line, the
message reported included a "Possible matches:" preamble for a list of
functions containing the requested function name. This looked
incongruous when no actual functions matched.

Turn that preamble into an optional parameter to qPrintTestSlots(),
that it'll output before the first match if it finds one, rework
qPrintTestSlots() to package its matching condition in a lambda and
return true if it found any matches.  Change this caller to output a
newline in place of the preamble (which ended in a newline), if no
match was found.

Change-Id: I9716ffa29c3c46e3c7e7fcf25a676c0356dab91c
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-07-11 14:42:27 +02:00
Volker Hilsheimer 54b276be0b testlib: Don't print QCOMPARE values if they lack string representation
Before 0681a2dd5a, QCOMPARE'ing types
for which no QTest::toString specialization exists did not output
Actual and Expected lines on failure, as that would only print <null>
for both values (which then look like the same value, confusingly).

Commit 0681a2dd5a changed that behavior,
and started printing the confusing <null> values.

Take care of the logic in the formatFailMessage function: if both values
are nullptr, then print only the variable names, but not the confusing
<null> text representation of the values.

Remove dead and duplicated code related to the formatting logic, add a
self-test function, and update the expected_cmptest files.

Fixes: QTBUG-104867
Pick-to: 6.4
Change-Id: I4be98e79f91196b14690a2cc0a68ffd50b431a45
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-07-10 03:08:32 +00:00
Laszlo Papp 07ada0b971 QTest: Support QKeySequence compare
Add a QTest::toString() override for QKeySequence. This is just calling
QKeySequence::toString(). The default format is PortableText, which
should be ascii. Deliberately avoided NativeText as it can return
unicode, e.g. on Mac.

This is necessary to get helpful information for test failures when
using QCOMPARE for QKeySequence instances.

Currently, the returned output would not only be not helpful, but even
misleading:

   Actual   (edit.keySequence()): <null>
   Expected (QKeySequence())    : <null>

After adding the override, the output is neither misleading nor
unhelpful:

   Actual   (edit.keySequence()): ""
   Expected (keySequence)       : "Return"

Some special characters would be escaped in the output, like the literal
double quote:

   Actual   (edit.keySequence()): "Ctrl+N"
   Expected (QKeySequence("\"")): "\""

Change-Id: Ib4b28fca30f6f2ad86c62530767f94a151332e0a
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-07-09 21:08:10 +01:00
Shawn Rutledge e1f25b1c8d Make it possible to check the accepted state of touch events in tests
QTouchEventSequence simulates a QPA touch event, potentially containing
multiple points. (Despite the name, it only calls qt_handleTouchEvent()
once, so it cannot really send a sequence of events; however, one event
can contain multiple touchpoints.) Delivery is synchronous, and we keep
return values through the QWindowSystemInterface::handleTouchEvent()
template functions; so the remaining step is to return a bool from
qt_handleTouchEvent(), so that we can return a bool from commit().
This allows tests to see the same perspective as a platform plugin can:
check whether the event was accepted or not, after delivery is complete.
Some tests in Qt Quick need to start doing that, to enforce correct
behavior in QQuickDeliveryAgent.

[ChangeLog][QtTestLib] QTouchEventSequence::commit() now returns a bool
so that tests can check whether the event was accepted during delivery.

Pick-to: 6.4
Task-number: QTBUG-104656
Change-Id: I9cf87909a3f847dedbdeca257013e309ac19cf0d
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-07-08 06:12:03 +02:00
Volker Hilsheimer 76b07b05f0 Doc: add missing "see also"
Pick-to: 6.4
Change-Id: Ibeb7d48a8c4c3e0e2000280bd85db943fc73c1c7
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
2022-07-04 12:34:40 +02:00
Laszlo Papp ca11725659 QCOMPARE: Fix a typo in the documentation
Change-Id: If21359397ef7873592b94838647aee121b84b33a
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2022-07-01 13:29:59 +01:00
Dimitrios Apostolou d23dcc0542 qtestlib: exit with error if a test is invoked with unknown data tag
Previously trying to execute a test function with an unknown data tag
would print an error message but exit with 0.

This patch stores a test failure, and continues trying to execute the
rest of the command line arguments, if any. In the end the process exits
with the usual exit code (number of failed tests) which is now !=0.

Pick-to: 6.4 6.3 6.2
Fixes: QTBUG-24240
Change-Id: Id4d422035f173e01e77ca88028dfd94dc0f9085c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-28 18:06:33 +02:00
Thiago Macieira 5e67e7efaa FatalSignalHandler: print some more information from siginfo_t
The siginfo_t parameter allows us to show what process sent a signal or
the crashing address. Additionally, it allows us to determine if the
crashing signal was indeed sent due to a crash.

The selftest tst_crashes produces now:

$ QTEST_DISABLE_STACK_DUMP=1 ./crashes
********* Start testing of tst_Crashes *********
Config: Using QtTest library 6.4.0, Qt 6.4.0 (x86_64-little_endian-lp64 shared (dynamic) debug build; by GCC 11.2.1 20220420 [revision 691af15031e00227ba6d5935c1d737026cda4129]), opensuse-tumbleweed 20220428
PASS   : tst_Crashes::initTestCase()
Received signal 11 (SIGSEGV), code 1, for address 0x0000000000000004
         Function time: 0ms, total time: 0ms
[1]    201995 segmentation fault (core dumped)  QTEST_DISABLE_STACK_DUMP=1 ./crashes

The last line comes from the shell. The code isn't decoded, but on Linux
it's a SEGV_MAPERR. macOS prints exactly the same thing.

I've updated one of the expected_crashes_*.txt output that doesn't seem
possible (the "Received a fatal error" message does not appear in Qt
anywhere).

Pick-to: 6.4
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16ebc8391234f0e2
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-06-28 00:39:35 +00:00
Mårten Nordheim 44927b801a QTest: switch some text-conversion functions to use qsizetype
To avoid potential narrowing.

Pick-to: 6.4
Task-number: QTBUG-104125
Change-Id: I37bfc5c49e7c919f5204a76a905758a92527d864
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-06-28 00:39:35 +00:00
Andreas Eliasson a6df765b03 Doc: Revise Qt Test Module landing page
Most of the module landing pages now follow a certain content structure.
In this case, the reference section should be placed before the
licenses and attributions section. Also, to make the wording consistent
across all module landing pages, use the globally available word
snippets.

Pick-to: 6.4
Change-Id: I2b2164eb1b8a0eb91db0f1efea8390956d54b29e
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2022-06-27 11:05:23 +00:00
Alexey Edelev 36ea34c714 Add the qt_class pragma to file listed in sync.profile classnames map
This makes header files self-contained and reduces the number of
'sources of truth' for syncqt procedure.

Change-Id: I7f5865abc69934603139d23e1b5452da46ccb110
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-06-22 03:03:31 +02:00
Andreas Eliasson ac171469a0 Doc: Revise Qt Test tutorial
- Make it explicit that each chapter can be run as a stand-alone test
application
- Add a CMake section on how to build the executable

Task-number: QTBUG-103730
Pick-to: 6.4 6.3
Change-Id: Id4c87c454521f698dcf16cfdc176318dd3e16786
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2022-06-21 11:08:49 +00:00
Kai Köhne 9d2cc4dd76 Fix typos in docs and comments
Found by codespell

Pick-to: 6.4
Change-Id: Ie3e301a23830c773a2e9aff487c702a223d246eb
Reviewed-by: Nicholas Bennett <nicholas.bennett@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-15 21:31:02 +02:00
Edward Welbourne 37bad1f43b Don't exit(1) on unrecognised test function name, just report a failure
This way, if you name several test functions on the command-line,
you'll at least get the ones that do exist run (and you'll be told all
of the ones that don't exist, rather than only the first).

Pick-to: 6.4 6.3 6.2
Change-Id: I14a515fcfacb6ca49e0470b236c05475b25db4f2
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
2022-06-15 20:23:06 +02:00
Thiago Macieira 17454bf9c6 FatalSignalHandler: print the signal name on crash
It's easier to remember what "SIGSEGV" means instead of "11".

GNU libc has offered sigabbrev_np() (non-portable) since 2.32; for older
libcs, we'll be happy with a hardcoded list.

Selftest updated to match... though it didn't seem to be necessary.

Pick-to: 6.4
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16ebc66ecf6e9465
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-08 20:39:18 +00:00
Mårten Nordheim 4d2ef82089 QTest: Remove pre-Qt6 code
It's disabled now, so can be deleted

Pick-to: 6.4 6.3 6.2
Change-Id: I0d548327e7ef42bbca9ed88556bf9f8456038cc7
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-08 19:08:49 +02:00
Mikolaj Boc 3df23b1fe4 Define out unused functions on WASM build
The struct iovec conversion functions that are needed on Unix are
unused on WASM build. This makes the build fail with -Werror on Mac
since the WASM build is treated as a variant of Unix. Cross-compilation
with clang:
Apple clang version 13.0.0 (clang-1300.0.27.3)
Target: arm64-apple-darwin21.3.0

Fixes: QTBUG-103974
Pick-to: 6.3 6.4
Change-Id: I34c65a18832ceedb9064a98f5729e45667749461
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-06-08 08:30:45 +00:00
Ivan Solovev 78c677376a QTestLib: improve documentation
Add missing '\since' version numbers.

This commit amends cc6d984390
and 0681a2dd5a

Pick-to: 6.4
Change-Id: Ia10b991c996fb58f08a17e485c4dfcbfbe8eba0a
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-08 10:30:20 +02:00
Thiago Macieira 8b26233573 qtestcase.cpp: create a common function to print the test runtime
The time was getting printed twice for the stack trace, but zero for the
watch dog if the stack trace was disabled.

Selftest appears to pass (though I thought it shouldn't).

Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16ebc4ec99e7fc5a
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-06-03 16:52:41 -07:00
Marc Mutz 9fea0c613e QCOMPARE/QVERIFY: fix huge pessimisation in QTestResult::reportResult
The old code allocated a stack buffer, but asked the runtime to
zero-initialize it. That's 1KiB of writes to the stack on every
QCOMPARE and QVERIFY before any actual work is done. Fixing this
little laissez-faire to just initialize the first character in the
buffer results in nice little speedups of ~40%.

This fixes the issue in reportResult(), a Cut'n'paste from compare()
and verify(), which have since been fixed by a previous commit.

Change-Id: I5cad57299490925b88e768dc751304699274db2e
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-06-03 20:37:51 +00:00
Ivan Solovev 0681a2dd5a QTestLib: rework QTest::compare_helper()
[ChangeLog][QTestLib] QCOMPARE now evaluates toString() on its
arguments lazily, speeding up the general case where the comparison
doesn't fail. This is true for the QCOMPARE functionality provided
by Qt. If you specialized qCompare() for your own types, then you
need to change its implementation in line with Qt's own qCompare()
specializations in order to enable this feature.

[ChangeLog][QTestLib] QCOMPARE calls with nullptr argument(s) will
now print the actual and expected values upon failure.

Previously it was not like that because of the compareHelper()
overload in qtestresult.cpp that treated the presence of
nullptr-arguments as a reason to ignore formatFailMessage() call.
New implementation does not have this check, and correctly
executes formatFailMessage() for all arguments.

Note that the qCompare() overloads that call QTestResult::compare()
internally were not affected by this patch, because they already
defer toString() invocation until the comparison fails.

Some numbers, collected against shared release developer build.
I checked how this change affects the test execution. The idea was
to pick some tests for types that do not have a specific
QTestResult::compare overload, so I picked a couple of QByteArray
tests.
The comparison is done by running a test 10 times and taking the
average execution duration, as reported in the log.

tst_qbytearrayapisymmetry:
 Before: 15.6 ms
 After:  14.2 ms

tst_qbytearray:
 Before: 41 ms
 After:  36 ms

The benefit is around 9% and 12% respectively.

Fixes: QTBUG-98874
Change-Id: I7d59ddc760168b15974e7720930f629fb34efa13
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-06-03 20:37:51 +00:00
Ivan Solovev cc6d984390 Add QTRY_COMPARE_{EQ,NE,LT,LE,GT,GE}_WITH_TIMEOUT()
[ChangeLog][QTestLib] Add QTRY_COMPARE_{EQ,NE,LT,LE,GT,GE}_WITH_TIMEOUT
macros that repeatedly execute QCOMPARE_{EQ,NE,LT,LE,GT,GE} until either
the comparison returns true or the timeout expires. Also add
QTRY_COMPARE_{EQ,NE,LT,LE,GT,GE} macros that simply invoke the
*_WITH_TIMEOUT versions with the usual timeout of five seconds.

Task-number: QTBUG-98873
Change-Id: Ib0d7d1c8c997f442b46acd85da738a8f512cc875
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-06-03 22:37:50 +02:00
Ivan Solovev 343e0ff485 Add QCOMPARE_{EQ,NE,LT,LE,GT,GE}()
[ChangeLog][QTestLib] Add QCOMPARE_{EQ,NE,LT,LE,GT,GE}()
macros. These new macros behave similarly to QVERIFY(a op b),
where 'op' is ==, !=, <, <=, >, >= respectively, but print
a formatted error message with argument values in case of failure.
The formatting is done lazily, which means that the strings will
be generated only when the comparison fails.

Also add a new test for tst_selftest and generate expected output
for it.

Fixes: QTBUG-98873
Task-number: QTBUG-98874
Change-Id: Ic8074798901d7a469b1f58d5cd28bbf49a3da1db
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-03 22:37:50 +02:00
Ivan Solovev 9327d1aaf7 Testlib: teach TAP test logger to support new QCOMPARE_* operators
The TAP test logger will now correctly print comparison types such
as QCOMPARE_NE or QCOMPARE_LT, and also provide a proper expected/wanted
value (by adding proper arithmetical operators in front of the value).
Sample output:
  type: QCOMPARE_GE
  message: Left value is expected to be greater than or equal to right value, but is not
  wanted: >= 1 (rhs)
  found: 0 (lhs)
  expected: >= 1 (rhs)
  actual: 0 (lhs)
  at: tst_ExtendedCompare::compareUnregistereEnum() (tst_extendedcompare.cpp:232)
  file: tst_extendedcompare.cpp
  line: 232

As a drive-by: make some variables const.

Task-number: QTBUG-98873
Change-Id: Idb54eaabcb937b42d3fc844f30041aab82d73f69
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-03 22:37:50 +02:00
Marc Mutz 474792a751 QCOMPARE/QVERIFY: fix huge pessimisation in QTestResult
The old code allocated a stack buffer, but asked the runtime to
zero-initialize it. That's 1KiB of writes to the stack on every
QCOMPARE and QVERIFY before any actual work is done. Fixing this
little laissez-faire to just initialize the first character in the
buffer results in nice little speedups of up to or exceeding 2x.

Amends d946507727.

[ChangeLog][QtTestLib] Optimized successful QCOMPARE and QVERIFY for
an up to 2x speedup.

This has the potential to meaningfully reduce the load on the CI, so
picking all the way to 5.15.

Pick-to: 6.3 6.2 5.15
Change-Id: Ib93d69282ec87cbd26a60e4ac14413e8cef8ff78
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-03 09:01:46 +00:00
Thiago Macieira dface58142 QtTest: move the disabling of core dumps from load-time to qInit()
There's no reason it has to be done THAT early. It was added in commit
aec85a53df to have the selftests (which do
crash) not leave lots of core files around. I could replace it with a
QProcess::setChildProcessModifier() function, but the variable is now
documented in the QtTest manual, so it would be no gain.

Function renamed to reflect its implementation.

Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16ebc25df97a98e3
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-01 22:12:17 -07:00
Volker Hilsheimer fb8a88b05c Use a precise timer in QTestEventLoop if timeout is sub-second
QTestEventLoop is used by QSignalSpy::wait. If tests want to verify
that a signal gets emitted within a second or less, we should use
a precise timer to reduce the chance of timer imprecision breaking
the test.

Change-Id: I2a2ca3bbdcaf032288f1fdc2aecc6afd05a6c76c
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-05-31 21:20:41 +02:00
Thiago Macieira 92046dde55 FatalSignalHandler: handle SIGABRT too
Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb77e9c77d10ba
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-30 21:27:03 -07:00
Thiago Macieira 7ad8a2e8fe FatalSignalHandler: expand to more Unix OSes than just Linux and macOS
At least for my FreeBSD this makes sense. The default debugger search is
gdb first then lldb on Linux-not-Android and for QNX (qcc is GCC after
all), but lldb first everywhere else.

With LLVM14 from Ports, I get:

$ tests/auto/corelib/tools/qhashseed/tst_qhashseed | head -1
********* Start testing of tst_QHashSeed *********
Config: Using QtTest library 6.4.0, Qt 6.4.0 (x86_64-little_endian-lp64 shared (dynamic) debug build; by Clang 14.0.0), freebsd 13.0
=== Received signal at function time: 1ms, total time: 3ms, dumping stack ===
(lldb) process attach --pid 1782
Process 1782 stopped

Executable module set to "/usr/home/tjmaciei/obj/qt/qt6/qtbase/tests/auto/corelib/tools/qhashseed/tst_qhashseed".
Architecture set to: x86_64--freebsd13.0.
(lldb) bt all
* thread #1, name = 'tst_qhashseed'
  * frame #0: 0x0000000800f227c8 libc.so.7`_wait4 at _wait4.S:4
    frame #1: 0x00000008003243bc libthr.so.3`__thr_wait4(pid=<unavailable>, status=<unavailable>, options=<unavailable>, rusage=<unavailable>) at thr_syscalls.c:581:8
    frame #2: 0x00000008002b9c73 libQt6Test.t.so.6`(anonymous namespace)::StackTraceHandler::generate() at qtestcase.cpp:393:9
[...]
=== End of stack trace ===
Received signal 13
         Function time: 1ms Total time: 3ms

Support for Windows left as an exercise for later. The
WindowsFaultHandler code doesn't even call generateStackTrace().

Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eba471f58ff3cb
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-05-30 15:56:38 -07:00
Thiago Macieira d5f4f91c3c FatalSignalHandler: use mmap() to create the alternate stack
So we can mark the bottom page as inaccessible, thus be able to catch a
stack overflow in the handler itself. Our code shouldn't cause
overflows, but it's possible that a chained handler does more work than
expected.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb83a294ab7958
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-05-30 15:56:26 -07:00
Thiago Macieira 9c0c1be122 StackTraceHandler: call the debugger directly, instead of via /bin/sh
This removes one middle-man and a 512-byte variable in favor of a simple
32-bit enum. This was done in a way so we can extend to use either gdb
or lldb in any OS.

I've renamed the debuggerPresent() function to make its meaning clearer.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eba561628ff89b
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-05-23 14:53:18 -07:00
Thiago Macieira 4ea5e39b27 StackTraceHandler: simplify the gdb and lldb commands
Instead of piping stuff via the shell into them, just use batch mode.
And also take the opportunity to tell them not to read their user-
provided configuration files.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eba21e71afaefa
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-23 14:53:18 -07:00
Thiago Macieira 3d9e56aa6c FatalSignalHandler: chain back to the original crash handler
If a previous handler was already installed, ensure it is called,
because there may be a reason why it was there. For example, the Android
ART adds a signal action to every fatal signal for logging purposes. We
do that by restoring the signal handler we had and re-raising the
signal.

If our handler was overridden by something else, then that handler was
already called, but will get uninstalled after our code runs. It won't
be a problem, because the application is exiting anyway.

[ChangeLog][QtTest][Behavior Change] On Unix, the QtTest code to
handle Unix/POSIX fatal signals will now call back to the original
handler that was installed, if there was one. This allows logging
frameworks (such as Android ART's), for example, to log the crash too.
Additionally, if there was no handler, the application should exit with
the correct signal instead of SIGABRT.

Fixes: QTBUG-97652
Pick-to: 6.3
Change-Id: Ifc4fca159b490d8d0b614d736e46caefcb903a4c
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-20 12:01:38 -07:00
Thiago Macieira 29dd43d5ab FatalSignalHandler: remember the previous signal's handler
And only restore those signals, instead of iterating over all possible
signals, instead of attempting to restore to SIG_DFL. Also, as a
consequence, we will install our handler even if there was already a
handler installed for the signal.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb756685f4e8b8
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-05-20 12:01:37 -07:00
Thiago Macieira c4600b2e25 FatalSignalHandler: simplify SA_RESETHAND code
And remove the unreachable code after std::abort() that was meant to
mimic that in INTEGRITY. The next commit will fix this properly.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb772018add694
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-20 12:01:36 -07:00
Thiago Macieira 3e596da488 FatalSignalHandler: simplify the SA_SIGINFO handling
The #if around the function declaration was ugly.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb7540f5da080f
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-20 12:01:35 -07:00
Thiago Macieira 65ff731b62 FatalSignalHandler: use std::optional instead of QScopedPointer
We get stack space reserved instead of using the heap.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb73fff0174150
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-20 12:01:34 -07:00
Thiago Macieira 3bd1aa09b6 FatalSignalHandler: split the Windows and Unix contents
They're very different, so there's no point in having them even in the
same class body. I've renamed the Windows one because Windows does not
report crashes via signals anyway. If you have an IDE that can scan both
branches of the #if, it will help you find the Windows-specific code
too.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb73ba196cfb74
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-05-20 12:01:33 -07:00
Thiago Macieira 4e8da8f1ec Put DebugSymbolResolver and FatalSignalHandler in an unnamed namespace
Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb735315b337b6
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-20 12:01:32 -07:00
Thiago Macieira 55a4d35dc3 FatalSignalHandler: remove call to qEnvironmentXxx from handler code
The Qt environment handling functions lock a mutex. That's a big no-no
in signal handlers.

[ChangeLog][QtTest][Behavior Change] QtTest will now check the value of
the environment variable QTEST_PAUSE_ON_CRASH in QTest::qRun(), so if a
test wants to modify this variable, it must do so from the main() or
initMain() functions, not in the test itself (including initTestCase()).

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb78867cd8f54e
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-05-20 12:01:31 -07:00
Thiago Macieira c8d9b7291a qtestcase: make the stack trace generation more signal-safe
This should work so long as there's no async-unsafe pthread_atfork()-
registered callback.

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb72e4e313bc19
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-20 12:01:30 -07:00
Thiago Macieira d75c595eac qtestcase.cpp: fix warning about ignoring the result of writev()
I don't get it on openSUSE Tumbleweed, but this is not the only place I
don't get a warn_unused_result warning and it happens elsewhere.

Change-Id: I7e305799c8594ebab255fffd16ee12ff30df6d4d
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-05-19 14:00:23 -07:00
Andreas Buhr 2f35653a30 Use QAbstractItemModelTester or QFileSystemModel
This patch enables usage of QAbstractItemModelTester on
QFileSystemModel. QAbstractItemModelTester called fetchMore()
on all items. QFileSystemModel represents the whole file system.
This led to very long test runs. To avoid this, this patch
introduces a new feature in QAbstractItemModelTester, namely
to disable calling of fetchMore().

Change-Id: Ie5d2e22fa4c143be7c080d9f79632cd2cbe07aac
Reviewed-by: David Faure <david.faure@kdab.com>
2022-05-18 12:58:06 +02:00
Lucie Gérard 05fc3aef53 Use SPDX license identifiers
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.

Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-16 16:37:38 +02:00
Alexey Edelev b98706f122 Add missing header files to the module sources
All module header files should be listed in the corresponding sections
of modules SOURCEs to be accessible in CMake routines.

Task-number: QTBUG-103196
Change-Id: Ieb77ae70557e35e546a5b00387e1e0aa40338239
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-05-12 16:34:25 +02:00
Thiago Macieira 046bc971c3 qtestcase/Unix: remove fprintf from signal handler
It's not async-signal-safe, so don't use it. Instead, let's use writev()
(which should write atomically and we won't need to have a buffer) and
std::to_chars to format numbers where available (where not, we roll out
our own implementation with divisions).

Pick-to: 6.3
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb70a158f44864
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-05 20:35:42 -07:00
Marc Mutz 664b84c137 De-pessimize QBenchmarkValgrindUtils::extractResult()
As if QIODevice::readLine() and QIODevice::Text open-mode aren't slow
enough, the old code took the line QByteArray, converted it to a
QString so it could apply a trivial regex to it: '^summary: (\d+)'.

We can, of course, use QByteArray::startsWith("summary: ") followed by
std::from_chars(), these days, to get the same effect, without having
to JIT-compile an RX and convert every line into UTF-16 first.

Change-Id: I20d80ffb469329d3c3efd08c71fcf5abf9f486d3
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-05 10:39:37 +02:00
Thiago Macieira 30067d102c qtestcase.cpp: use #ifdef RLIMIT_CORE to guard RLIMIT_CORE usage
Instead of trying to guess which OSes have this functionality.

Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16ebc20fe7eca346
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-04 20:15:36 -07:00
Marc Mutz c9f4cb2c92 QTeamCityLogger: fix .arg() placeholder injection
Since each .arg() call starts from scratch, a file name containing a
suitable %n would mess up the formatting of the following .arg() call.

Fix by using multiArg(), which requires to pre-format the line into a
QString, but which performs only a single-pass, so doesn't suffer from
the placeholder injection problem that plagues .arg()-chaining.

Pick-to: 6.3 6.2 5.15
Change-Id: I549527643da657fca0bea63d5e3becadac529d4b
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
2022-05-04 07:09:25 +02:00
Marc Mutz ff4f2e9397 QBenchmarkValgrindUtils::extractResult(): use std::optional
... instead of a pair of val and valSeen variables.

More elegant.

Pick-to: 6.3
Change-Id: I0fcf9a3b5611e30fb81d92619993a7828496cd1b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-05-03 11:44:56 +02:00
Marc Mutz 8d6eae7ffd QAbstractTestLogger: disable copying
Fixes clazy-copyable-polymorphic.

Pick-to: 6.3 6.2 5.15
Change-Id: I31a68da6e402ada91099b09540b29a1d56f9cc56
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-03 00:10:05 +02:00
Sona Kurazyan 2b3782132f QTestLog: simplify construction of a log message
Change-Id: I055efa45ec468cae43a38a9746a47af0382ca8c1
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-05-02 20:06:20 +00:00
Sona Kurazyan 89225c951e QtTestLib: replace remaining uses of QL1String with QL1StringView
Remove unneeded \fn qdoc lines as a drive-by.

Task-number: QTBUG-98434
Change-Id: Id93ddbb38b97a8f5a6734bfbc82686ccb3a87aa6
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-02 20:06:20 +00:00
Sona Kurazyan 7baa06fe0c QtTestLib: replace QLatin1String uses with _L1/_s/QStringLiteral
Task-number: QTBUG-98434
Change-Id: Ie327fd4af1880002e5a1e09b43384f2b709625e7
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-02 20:06:20 +00:00
Florian Bruhin b6759ff81c QAbstractItemModelTester: Fix typos in debug output
Introduced in 2f6faca901

Change-Id: Iab01bbd572ee1fdc7f590b608d7bf504dbf18769
Reviewed-by: David Faure <david.faure@kdab.com>
2022-04-29 23:16:54 +02:00
Sona Kurazyan 79cf895c70 QtTestLib: stop using QLatin1Char constructor for creating char literals
Required for porting away from QLatin1Char/QLatin1String in scope of
QTBUG-98434.

Change-Id: I3debae5f481037958bfb65caabca97a0d4681eb9
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-04-28 11:15:11 +02:00
Marc Mutz 82cf06fc21 QtTest: includemocs
Including moc files directly into their classes' TU tends to improve
codegen and enables extended compiler warnings, e.g. about unused
private functions or fields.

Pick-to: 6.3 6.2 5.15
Task-number: QTBUG-102886
Change-Id: I9378ddcabbfa0ec5ca895f6ce55ab5f1ec2cade3
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-04-28 11:15:03 +02:00
Edward Welbourne 383368a321 Support test-case selection based on global data tag
In the same ways as for blacklisting. In the process, move reporting
of an unrecognised tag outside the global-data loop, as the tag might
be, or start with, a global data tag; and also report the global data
alongside the test-specific data. Indent the lines reporting tags, as
this makes the structure of the lists more evident. Converted the tag
argument to QLatin1String() to facilitate searching and matching.

Updated documentation, including expanding on the option of specifying
the data tags on the command-line. Drive-by: fixed a link in the
documentation, removed a snippet that was nowhere used.

[ChangeLog][QtTest] When specifying what tests to run on a QtTest
command-line, it is now possible to include a global data tag in the
same way as a function-specific data tag or in combination with it.
Thus if the test reports itself as tst_Class::function(global:local),
command-line option function:global:local will select that specific
test and function:global will run all data-rows for the function on
the given global data tag.

Fixes: QTBUG-102269
Change-Id: I7c86d6026933b05f7994bfc3663a2ea6a47f5f38
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-04-25 17:47:21 +02:00
Edward Welbourne ec59ae6189 Spacing-only change: fix indents and spacing in qtestcase.cpp
Separated from other clean-ups to avoid inanity 'bot complaints.

Change-Id: Ia33f87da224dba7c50bbf14beb642825dc46fe90
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-04-14 16:58:17 +02:00
Edward Welbourne f3184770e2 qtestcase.cpp: s/QString::fromLatin1/QLatin1String/ where viable
Now that QL1S has its own arg(), we can use that directly instead of
going via QString before calling arg(). Only works in most cases, as
the QL1S::arg() overload set is more limited.

Change-Id: Ie4c566c5998538b1b8a953b1ba4481e0f1663bf7
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-04-14 16:58:17 +02:00
Thiago Macieira fbe6f19853 QBENCHMARK: Replace the double underscore with something not reserved
Clang with -Wreserved-identifier complains.

Pick-to: 6.3
Fixes: QTBUG-102431
Change-Id: If2e0f4b2190341ebaa31fffd16e441ee35b0e3b9
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
2022-04-14 03:17:42 +00:00
Sona Kurazyan 80363889fe Replace uses of _qba with _ba
Task-number: QTBUG-101408
Change-Id: I5175428c2be934b09f45bd06b0b47643003e25c7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-04-07 19:30:17 +02:00
Allan Sandfeld Jensen ca106e261e Make keyClick safer for QWindows going away on presses or during wait
We already have similar logic in the QWidget variant.

Pick-to: 6.3 6.2
Task-number: QTBUG-102253
Change-Id: Idf1e3474f3189909f1cba7d5dedc2cde12f1338e
Reviewed-by: Michal Klocek <michal.klocek@qt.io>
2022-04-06 20:39:33 +02:00
Edward Welbourne 0bdc7322d9 Tidy up QTapTestLogger::addIncident()'s regex parsing
It had some repetition that could be refactored out, some long lines
and a bool it worked out the hard way, when it had previously
determined the answer in passing without recording it.

Change-Id: I9e53ed087dfbe8067686b27b6cf9ac32040fbf19
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-03-29 22:21:58 +02:00
Edward Welbourne 784f5d76b9 TAP test logger: report B?XFAIL (mostly) as a message
Previously, only the first B?XFAIL would be reported, all others would
be discarded. Furthermore, if a B?XFAIL had happened, B?PASS was also
not reported, since the B?XFAIL served as test line. However, if the
B?XFAIL was followed by a SKIP, B?XPASS or B?FAIL, these were reported
as normal, producing exactly the kind of duplicated test line that the
skipping of B?PASS was meant to supply.

So change B?XFAIL to be reported among the messages, but retain the
TODO annotation of the first on the test line of a subsequent B?PASS,
if nothing more drastic happens in the mean time. So now more than one
B?XFAIL can be reported, the test is still marked as a TODO and we
don't get duplicate test lines for a subsequent non-passing result.

This replaces the bool m_wasExpectedFail member with a QTestCharBuffer
m_firstExpectedFail that records the first XFAIL's TODO line (so its
isEmpty() fully replaces m_wasExpectedFail).

Previously, the at/file/line information for a B?XFail would be
supplied as top-level keys in the YAML block for a "Pass" reported as
not ok due to the XFail, as this location information is now part of
the B?XFail's message in the extensions/messages block. Duplicating
the first B?XFail's location at top level would add complexity and is
arguably misleading, as the test result is really a pass (after
ignoring known issues), and the location of the pass is indeterminate
(nominally the end of the test function, but actually also after the
cleanup() call for this test, when relevant), which is why a Pass has
no location information.

Task-number: QTBUG-96844
Change-Id: Ib3f24f56266ff49bf3bc9759ac8263ac69a62130
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 22:21:58 +02:00
Edward Welbourne ac38401d31 Move TAP's messages block to after the primary report details
In the process, split it from its comments block and don't bother with
a YAML block if it would only have contained comments.

Task-number: QTBUG-96844
Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 22:21:58 +02:00
Ivan Solovev 3c5b3a36af Introduce QTestPrivate::androidCompatibleShow() helper function
This function should be used in QWidget-related test cases instead of
QWidget::show() when you need to move or resize the widget and then
compare its position or geometry with the expected value.
The reason for introducing it is that on Android QWidget::show() is
showing the widget maximized by default, so its position and size
can't be changed.

Task-number: QTBUG-87668
Pick-to: 6.3 6.2
Change-Id: I8ec5c07b73968b98d924a41e5d41ddb4d7be1ced
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-03-29 13:56:14 +01:00
Edward Welbourne 7d60fde9a7 Include file and line information in TAP diagnostics blocks
When producing a diagnostics block, include the file and line
information, if we have it, to describe it. This presently only adds
this information for skip, but could in principle do the same for a
B?XPass.

Task-number: QTBUG-96844
Change-Id: I6cc375d98e2369eba262010f9c2dfbcba931a6f1
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 00:05:12 +01:00
Edward Welbourne ae37fa0464 TAP test logger: move messages into the diagnostics block
Our TAP output was delivering messages as comments before the test
line, where TAP clearly expects the details of a test to follow its
test line. Version 13 provides a YAML block to deliver diagnostics and
encourages use of this, so accumulate our messages in a
QTestCharBuffer instead of emitting them one by one.

However, messages produced after a test has produced its test line
belong to that test, but are too late to be included in its
diagnostics block, so should be emitted immediately as before, albeit
now with a type prefix. This at least separates such messages, from
the end of one test, from messages produced early in the next.

In the process, add a type-prefix to each, to make clear what type of
message it was. Since the Yamlish supported by TAP consumers doesn't
support a way to have many messages, use the extensions: top-level
hash tag with a messages: sub-tag to gather our messages as a list.
(This expands at least one expected output file significantly and
substantially rewrites some others.)

Add methods to QTestCharBuffer, and a helper function, to support this.

Task-number: QTBUG-96844
Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 00:05:12 +01:00
Topi Reinio 62a26df2ff Doc: Add \relates command for documented global functions/macros
Global functions need an associated header file/class/namespace to make
the documentation visible.

Remove QDoc comment tag (/*!) for an internal function.

Pick-to: 6.3
Task-number: QTBUG-99578
Change-Id: Id390d433d7e081fc90fa57a02097b6c37fa13bee
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2022-03-25 16:19:20 +01:00
Giuseppe D'Angelo 56f2407dc0 QAbstractItemModelTester: use the right types for some roles
Models are expected to return Qt::Alignment and Qt::CheckState
when asked for the text alignment / check state roles, respectively.
However, for legacy reasons, models may return `int` for either
possibility. Extend the tester to cover this.

Task-number: QTBUG-75172
Change-Id: I64bfc898466e8bb4d4a8a35726986ae610a8ffbe
Pick-to: 6.2 6.3
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
2022-03-24 07:19:54 +01:00
Edward Welbourne 5a4106748c Document some details of TAP's use of YAMLish
... along with how QtTest soon shall be using it.

Change-Id: I9c723ebdb8ee5be1c4152aa911ae7c846ea61b5c
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-22 17:43:13 +01:00
Edward Welbourne 90406af31a Use a cheaper is-empty test for a C-string
We only need to look at the first byte to determine whether the length
is > 0; no need to count how many more bytes after it aren't '\0'.

Change-Id: Id2db2a9a993086c0aee02c99396fcb346b1a348e
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-03-22 17:43:13 +01:00
Edward Welbourne 02d130db7e Avoid duplicate descriptions in TAP's incident YAML
When a B?Fail's description doesn't match the QVERIFY/QCOMPARE
regexes, it got output as a comment in the YAML block after the test
line had already reported it as a TODO comment. An empty description
would also have lead to an empty comment in the YAML block. Condition
this fallback output case on there being a description that hasn't yet
been reported in the test line.

Task-number: QTBUG-96844
Change-Id: Id7fe81d26ddb01da3d8003ada8fa590a5e1a166f
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-22 17:43:13 +01:00
Edward Welbourne d52fbfe924 Fix indentation of some continuation lines in qtaptestlogger{.cpp,_p.h}
Change-Id: I33dc05b61b07f077edc7c4816d43d0b53f5f4b43
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-22 17:43:13 +01:00
Edward Welbourne 58b0d72f8d TAP test logger: treat blacklisted XFail the same as XFail
In all places but one they were treated the same; fix that last one to
match the rest. This removes one line from the YAML block for each
blacklisted XFail test; in each case, this message duplicates the one
on its "not ok ... # TODO..." test line.

Change-Id: Iff2028afccd979db7f2c84e85d1b78541900008e
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-03-22 17:43:13 +01:00
Edward Welbourne 44d0cbc7b1 Match QtTest command-line output options to the help output
Reorder the different formats to match the order in the help output
and include missing entries from the list of formats supported by the
-o filename,format variant.

Change-Id: I884f9facc7cfde7cec0bee75944efdd72d66a83e
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-22 17:43:13 +01:00
Edward Welbourne 0e5c39ee13 Document what QtTest's various loggers do
Provide basic (internal) documentation of each logging class and link
the command-line documentation to pages relevant to the formats not
defined by Qt.

Change-Id: I3251dd1304203c6ab87dfe1f2dec0e9787ab69f8
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-22 17:43:13 +01:00
Mårten Nordheim 6eda249402 Fix deprecated use of QBA/Q*String::count
'Use size() or length() instead'

Change-Id: I284fce29727c4c1ec9ea38a4e8ea13a9e0af5390
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
2022-03-15 20:18:56 +01:00
Edward Welbourne b15e01ff4e Use LogElementType rather than int as test-element type
All callers do in fact pass members of the enum, so just making the
defualt value be LET_Undefined rather than -1 eliminates any need for
it to be an int.

Change-Id: Ic1c9ae1522363666d1208aba60d4b4df5eff6ce0
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 20:16:35 +01:00
Edward Welbourne 479c95729a Support global data tags in blacklisting identification of test-cases
Previously the blacklisting file format only worked with slot:data or
plain slot names for the items to blacklist. However, tests with
global data report themselves with the global data-row tag in the same
way as function-specific ones do; and tests which have both join the
two as slot(global:data) in the test output name, so the reader is apt
to mistake global:data for a data tag. In any case, it is potentially
desirable to be able to blacklist a function with either or both of
global and local data-row tags specified. Add support for that and
remove a blacklisting that was only needed due to the lack of this
support.

For now, make the new parameter to checkBlackLists() optional, so
that qtdeclarative's qmltest framework can adapt to this change.

Fixes: QTBUG-100870
Change-Id: I9125811ebdab75d3fb462ba8b60561f003426502
Reviewed-by: Pasi Petäjäjärvi <pasi.petajajarvi@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 20:16:35 +01:00
Sona Kurazyan 28d2514428 Use qstringfwd.h when forward declaring string classes
Task-number: QTBUG-98434
Change-Id: I37b70a1b7bc164fa7dc9cb702827297ca66a6fdc
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-03-12 01:05:45 +01:00
Mårten Nordheim 034d8898f8 Fix deprecated uses of QScopedPointer
By changing it to unique_ptr.

Pick-to: 6.2 6.3
Change-Id: I91abb69445b537d4c95983ae735341882352b29d
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-03-08 15:44:17 +01:00
Marc Mutz 53890b5d3c qtestmouse.h: compile with QT_TYPESAFE_FLAGS
Also include the comprehensive tests for bool cast compilation which I
originally wrote to confirm that the QTEST_ASSERT() change should be™
correct.

Pick-to: 6.3
Task-number: QTBUG-101406
Change-Id: I9a2871bfd4be9999b7a720bec775bba7aeffbe24
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
2022-03-07 18:25:35 +01:00
Giuseppe D'Angelo 5e91f142aa Miscellanea fixes for QT_TYPESAFE_FLAGS in our headers
In preparation for adding it to headersclean.

Some remarks:

* QStandardItemModel builds just fine (QFlags has comparison operators
against literal zero); the warning we however get is about 0 converted
to a null pointer constant. There's nothing we can do about that one
(even <compare> gives such a warning).

* Several code was depending on flags->int conversions. Add toInt(),
but also cast again to the expected type to avoid warnings in case
toInt() returns unsigned int.

* Ported to explicit casts to bool rather than test(Any)Flag to minimize
confusion for people unfamiliar with the test*Flag methods.

Change-Id: I5be280ac33a0b38e2680096f0e79129fd55ba241
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-03-07 02:13:14 +01:00
Marc Mutz e93bf391fa Fix QTest::toString() over QT_TYPESAFE_FLAGS QFlags
QT_TYPESAFE_FLAGS allows explicit casts only to QFlags<T>::Int, which
is either int or unsigned int. The cast to the resp. other type fails.

To fix, first convert to QFlags<T>::Int with toInt(), and only then
cast to int or unsigned int.

Fixes: QTBUG-101399
Pick-to: 6.3
Change-Id: Ie74d53adc601cdf19708265b040092780676058f
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
2022-03-04 01:26:53 +01:00
Marc Mutz ea3a7aaf2b QTest: fix QCOMPARE with QT_TYPESAFE_FLAGS
Amends 631a0cc45c.

Pick-to: 6.3
Fixes: QTBUG-101299
Change-Id: Ib17e405de4db9fb592aba4ea842d3768a95cad74
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-03-02 15:38:27 +00:00
Samuel Mira 38a86afcc4 Tidy up QTest::FatalSignalHandler
Rename some variables: avoid abbreviated names.
(Don't bother with renaming variables an imminent commit removes, though.)

Pick-to: 6.2 6.3
Change-Id: I3bd11f37ca687047555d43dff83c3c2f935135d8
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-03-02 14:58:10 +01:00
Edward Welbourne fcecaf53f5 Adapt QTest::FatalSignalHandler() to support sa_sigaction when available
We don't currently have a use for the extra parameters, but Samuel
Mira's fix to ensure we don't miss fatal signals on Android needs to
be able to call the prior handler, which may need these parameters.

In the process, check returns from sigaction() and use its nullptr
semantics to query first when detecting whether to restore the default
when done, rather than setting and then restoring if it turns out
we've been replaced in the meantime.

Task-number: QTBUG-97652
Pick-to: 6.2 6.3
Change-Id: If30a0db35946c3dceb409ae3f41aa437149472e6
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-03-02 14:58:10 +01:00
Thiago Macieira 75e0d1b5c1 Make sure all qtbase private headers include at least one other
See script in qtbase/util/includeprivate for the rules.

Since these files are being touched anyway, I also ran the
updatecopyright.pl script too.

Change-Id: Ib056b47dde3341ef9a52ffff13ef677e471674b6
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-02-24 12:07:00 -08:00
Andreas Buhr 2f6faca901 Add tests to QAbstractItemModelTester checking only one change in flight
This patch adds checks to QAbstractItemModelTester checking
that only one change is in flight at a time.

Change-Id: I15fe1ad1274649aa652ed6489a889df72c8a8292
Reviewed-by: David Faure <david.faure@kdab.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-02-23 14:42:27 +01:00
Noah Davis bfcf47ab9d testlib/qasciikey: Add keypad navigation keys
Needed to prevent qasciikey.cpp from failing an assertion when
Qt::Key_Select is used in tests.

Change-Id: I56aa517e8f6f332636b3df106d7265677d1b33ee
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
2022-02-17 16:05:23 -05:00
Thiago Macieira 64d568b550 QCborArray: use QCborValueConstRef
Change-Id: I5e52dc5b093c43a3b678fffd16b6077e5c4abd63
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
2022-02-15 17:00:03 -08:00
Kai Köhne 8275611766 Core: Remove 'properties' feature
Even QtCore alone cannot be built without the properties feature since
Qt 5.5. While fixing this is easy, other modules like dbus,
networking are also using QObject::property() and friends liberally.

All in all I doubt that anybody will miss the feature (otherwise it
would have been fixed in the last decade).

Change-Id: Iaf3cc20bda54ee2ff3b809fac8fa82b94ecc88c0
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-02-14 12:50:59 +01:00
Tor Arne Vestbø 985f0e9ca5 macOS: Don't print stack trace via lldb on test failure if SIP prevents it
If the CSR_ALLOW_UNRESTRICTED_FS bit of System Integrity Protection (SIP)
is enabled lldb will fail to print a valid stack trace when launched from
the crashed process, and might also resulting in hanging or crashing the
parent process (Terminal e.g.):

  https://github.com/llvm/llvm-project/issues/53254

We detect this situation and avoid printing a stack trace if so.

Pick-to: 6.3 6.2
Change-Id: Iad8cab5fcdc545d810ca4d4e985aefc0988d0234
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2022-02-09 18:19:57 +01:00
Giuseppe D'Angelo c7a52d9d54 QTestLib: code tidies
Rename the `failureLocation` variable to `messageLocation`, as
it does not necessarily contain information about a failure.

Change-Id: I9e78dc76c8111c78e34c36f6623e9a8cc9e4764e
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-02-03 04:34:08 +01:00
Giuseppe D'Angelo a605b9297d QPlainTestLogger: do not print "failure location" if there's no failure
When running a test under increased verbosity levels, QTestLib would
print "failure location" for every QTest function (QVERIFY, QCOMPARE,
...), even if there's no failure at all.

Keep the code centralized, but split the formatting of failures and
non-failures (other messages).

Task-number: QTBUG-96844
Change-Id: I3c508653176b68579dc0eb0cffcc153a52da2e2c
Pick-to: 5.15 6.2 6.3
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-02-02 23:08:37 +01:00
Giuseppe D'Angelo 64e33727bb QPlainTestLogger: code tidies
Remove a couple of unnecessary string literals that only coped with an
impossible situation (default return after a switch that cover all the
enumerators).
Add a Q_UNREACHABLE() for good measure.

Change-Id: I8065218554edf98408d9c0823b8c301932681c59
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-02-02 20:08:25 +01:00
Tor Arne Vestbø 7d19efa7bb testlib: Disable window restoration on macOS in a non-persistent way
The recommended way to register defaults on macOS is via registerDefaults:
which puts the key/value into the volatile NSRegistrationDomain.

This stops testlib from dumping preference files into ~/Library/Preferences:

  ❯ plutil -p tst_qwidget.plist
  {
    "ApplePersistenceIgnoreState" => 1
  }

Pick-to: 6.2 6.3 5.15
Change-Id: I4ac812014f9ff97bc446806eaf2108cd1c8b89c3
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2022-02-01 16:56:02 +01:00
Edward Welbourne 13d2e13290 Make counts of various types of test result add up correctly
Added tests for repeated skips and failures (from within void lambdas,
to simulate skips and failures from within event handlers). These
exhibit yet more ways to count more than one outcome for a test.  The
new QTest::failOnWarning() can also provoke more than one failure from
a single test, and several existing selftests exhibited various ways
for the Totals line's counts to add up to more than the number of
actual tests run.

Fixed counting so that only the first decisive incident is counted.
Tests can still report later failure or skipping, but only the first
is counted.

Added a currentTestState in qtestlog.cpp, by which it keeps track of
whether the test has resolved to a result, and clearCurrentTestState()
by which other code can reset that at the end of each test. This
brought to light various places where test-end clean-up was not being
handled - due to failure or skipping in a *_data() method or init, or
a skip in cleanup.

Fixes: QTBUG-95661
Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-01-13 19:47:22 +01:00
Marc Mutz 7c8126d909 QTestData: fix streaming of u8 string literals in C++20 mode
...fixing tst_QLocale:

  QDEBUG : tst_QLocale::formatDate() expected data of type 'QString', got 'char8_t [64]' for element 1 of data with tag 'unicode in format string'
  QFATAL : tst_QLocale::formatDate() ASSERT: false in file /home/marc/Qt/qtbase/src/testlib/qtestdata.cpp, line 105

In C++20, the type of u8 string literals changed from const char[] to
const char8_t[].

[ChangeLog][QtTest][QTestData] Fixed streaming of u8 string literals
in C++20 mode.

Pick-to: 6.3 6.2 5.15
Task-number: QTBUG-99480
Task-number: QTBUG-99481
Change-Id: I53bd73522a4ebce307f360e225f25590f981d44a
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2022-01-03 12:22:53 +01:00
Janne Juntunen 02f14d7afe qtestblacklist: add webOS support
Added support for blacklisting autotests for webOS platform
since it looks like we might have to blacklist some of those.

Task-number: QTQAINFRA-4706
Change-Id: I212be15161bd042f1511000ced2c8c022c3206e1
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2021-12-21 12:13:57 +02:00
Dimitrios Apostolou 725fc4da44 Properly indent qtestlib's XML output
Also regenerate the expected output for tst_selftests, to match new
output. Changed one line source code in tst_seftests for the
same purpose.

Change-Id: I930ba4bb290568d6f67a8910a781725f01f08bf1
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-12-17 07:03:26 +01:00
Kai Köhne b34be3868b Doc: Use find_package(Qt6 REQUIRED COMPONENTS ...) idiom
Using REQUIRED as a prefix instead of suffix works better with
OPTIONAL_COMPONENTS, and is also the order in the CMake manual.

Task-number: QTBUG-98867
Pick-to: 6.2
Change-Id: I1ab68408b95d8edf06272a3b9fceccd8d8e597fc
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2021-12-09 22:20:04 +01:00
Edward Welbourne 7586e1980f Use QLatin1String::arg() directly
No need to wrap the QLatin1String in a QString() to get at arg now.

Change-Id: Iac90e052b88077b9c2bd3e7eca6bc31935baa4b8
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-12-09 20:54:26 +01:00
Edward Welbourne 91e8d8027d Tidy up in QTapTestLogger::addIncident()
After we've checked m_wasExpectedFail, we might as well set it
promptly, since one of our earlier checks is on the same
condition. Explain what that check is about, fix a typo in another
comment and make the local ok variable const, since it can be.

Change-Id: Ie04c0ad4248e47b116d3e22046116d39cf2ac6df
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-12-09 20:54:26 +01:00
Edward Welbourne 6729a4b29c TAP test logger: avoid dangling space on TODO and SKIP lines
If an incident has an empty description we produced output with a
dangling space. Avoid triggering the inanity 'bot with those.
Reorganise the code to save some duplication.

Change-Id: I1dc29fa8ad4449a4584f11dddcf002e405cd9238
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-12-09 20:54:26 +01:00