Since e0cad1aab5, the code suffered
from an ABA problem where the TestFunctionStart expectation is set
in testFinished(), but by the time WatchDog::run() gets around to
examining the state when returning from condition_variable::wait()
in waitFor(), the next beginTest() has already set the expectation
back to TestFunctionEnd.
There are several known solutions for ABA problems. Embedding a
generation count into the expectation state seemed to be the most
straight-forward fix, and can be done without DWCAS support, because
the state is just 2 bits, leaving the other 30 or 62 bits for the
generation counter, so do that.
[ChangeLog][QTestLib] Fixed a bug which caused
QTEST_FUNCTION_TIMEOUT to be applied to the whole test execution,
as opposed to each test function.
Fixes: QTBUG-109466
Pick-to: 6.5 6.4 6.2 5.15
Change-Id: If71ade932330407b85d204d45c74350c651325fe
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Fix up deprecated macOS keyword while we're at it.
Change-Id: I6f6ba21bde161d3dd3ac6c243f1b5f7c16ef3228
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
We do that by passing the full list of results to the logger, to a
virtual that is present in the base class to call the existing
function. For all but the plain logger, we'll just print multiple
results. The plain logger now prints:
RESULT : tst_MyClass::QString_toInt()
383 nsecs per iteration (total: 3,837,324, iterations: 10000)
1,069 CPU cycles per iteration (total: 10,692,457, iterations: 10000)
3,123 instructions per iteration (total: 31,230,101, iterations: 10000)
536 branch instructions per iteration (total: 5,360,022, iterations: 10000)
Change-Id: I3c79b7e08fa346988dfefffd17203cb5802693dd
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Avoids having lots of buffers that occupy the stack.
Change-Id: I3c79b7e08fa346988dfefffd17203c5045a7fc86
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
The code implementing the C++ rules of type promotion and conversion
was too pedantic. There's no need to follow the letter of the standard,
not when we can now assume that everything is two's complement (this was
true for all architectures we supported when I wrote this code in 2014,
but wasn't required by the standard).
So we can reduce this to fewer comparisons and fewer rules, using the
size of the type, not just the type ID.
Change-Id: I3d74c753055744deb8acfffd172446b02444c0c0
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Avoids operating in UTF-16 and having to convert.
Drive-by fix int to qsizetype.
Change-Id: I3c79b7e08fa346988dfefffd17203b044914d68d
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
The man page says the prctl() should apply to any performance counters
measuring the current process, so it should work with a perf stat
started with --delay=-1... but I couldn't make that work.
Change-Id: I3c79b7e08fa346988dfefffd172032f06cc10fa0
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
This code predated C++11 and thus had a sentinel element.
Change-Id: I3c79b7e08fa346988dfefffd17202ef3f9735683
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Implemented for the Linux Perf measurer, with four measurements by
default.
RESULT : tst_MyClass::QString_toInt():
149.574444 CPU cycles per iteration (total: 149,574,445, iterations: 1000000)
RESULT : tst_MyClass::QString_toInt():
620.000181 instructions per iteration (total: 620,000,182, iterations: 1000000)
RESULT : tst_MyClass::QString_toInt():
131.000046 branch instructions per iteration (total: 131,000,047, iterations: 1000000)
RESULT : tst_MyClass::QString_toInt():
32.118771 nsecs per iteration (total: 32,118,771, iterations: 1000000)
Change-Id: I3c79b7e08fa346988dfefffd17202cda3df8431b
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
And pass the value in a qreal, which is what QBenchlib stores
anyway. This increases the code size a little because the conversion
from integer to qreal is in multiple places, but doesn't meaningfully
increase the overhead: in the SysV ABI, we still return Measurement in
registers and even using the floating point registers where applicable.
This is the first step in allowing the Perf benchmarker to benchmark more
than one event.
Change-Id: I3c79b7e08fa346988dfefffd172027a8677f17c0
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
qtestoutputreporter.js and qtestoutputreporter.css were added to
the batchtestrunner, but not to the list of files copied from
sources, so make sure to add those and add a note for future reference.
Change-Id: If7f323b3051ee9b6fb4bedc3ee314e4941b41f0b
Reviewed-by: Mikołaj Boc <Mikolaj.Boc@qt.io>
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
This is a semantic patch using ClangTidyTransformator as in
qtbase/df9d882d41b741fef7c5beeddb0abe9d904443d8, but extended to
handle typedefs and accesses through pointers, too:
const std::string o = "object";
auto hasTypeIgnoringPointer = [](auto type) { return anyOf(hasType(type), hasType(pointsTo(type))); };
auto derivedFromAnyOfClasses = [&](ArrayRef<StringRef> classes) {
auto exprOfDeclaredType = [&](auto decl) {
return expr(hasTypeIgnoringPointer(hasUnqualifiedDesugaredType(recordType(hasDeclaration(decl))))).bind(o);
};
return exprOfDeclaredType(cxxRecordDecl(isSameOrDerivedFrom(hasAnyName(classes))));
};
auto renameMethod = [&] (ArrayRef<StringRef> classes,
StringRef from, StringRef to) {
return makeRule(cxxMemberCallExpr(on(derivedFromAnyOfClasses(classes)),
callee(cxxMethodDecl(hasName(from), parameterCountIs(0)))),
changeTo(cat(access(o, cat(to)), "()")),
cat("use '", to, "' instead of '", from, "'"));
};
renameMethod(<classes>, "count", "size");
renameMethod(<classes>, "length", "size");
except that the on() matcher has been replaced by one that doesn't
ignoreParens().
a.k.a qt-port-to-std-compatible-api V5 with config Scope: 'Container'.
Added two NOLINTNEXTLINEs in tst_qbitarray and tst_qcontiguouscache,
to avoid porting calls that explicitly test count().
Change-Id: Icfb8808c2ff4a30187e9935a51cad26987451c22
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Short of expanding Qt Test Best Practices to have a section about
QEXPECT_FAIL, we can link to some existing documentation that briefly
covers it.
Pick-to: 6.2 6.4
Change-Id: I37bf2672e4b5b6e7315e009f86d03eeb7937d50e
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
What's now called TestMethods::invokeTest() was apparently once called
qInvokeTestMethod(). Also, its current instance is (now) cleared by
its destructor rather than (overtly) at the end of that function.
Change-Id: I04de7ca2247dd640a398b3e4e7bf410401f3cbbf
Reviewed-by: Jason McDonald <macadder1@gmail.com>
I don't think I understood what this does when I coded it. It wasn't
needed with the Linux kernels of the era.
[ChangeLog][QtTest] Specifying attributes with the -perfcounter command-
line option for Linux performance counters is deprecated. QtTest will
ignore the colon and any attributes listed there, though future versions
of QtTest may reintroduce attributes if needed.
Change-Id: I3c79b7e08fa346988dfefffd17202684fc2cf650
Reviewed-by: David Faure <david.faure@kdab.com>
The kernel has become more paranoid since 2013, when I originally wrote
this code. The kernel.perf_event_paranoid sysctl controls the paranoia
level[1]:
=== ==================================================================
-1 Allow use of (almost) all events by all users.
Ignore mlock limit after perf_event_mlock_kb without
``CAP_IPC_LOCK``.
>=0 Disallow ftrace function tracepoint by users without
``CAP_PERFMON``.
Disallow raw tracepoint access by users without ``CAP_PERFMON``.
>=1 Disallow CPU event access by users without ``CAP_PERFMON``.
>=2 Disallow kernel profiling by users without ``CAP_PERFMON``.
=== ==================================================================
Since the default is 2, we QBenchlib has been failing with EACCESS:
PASS : tst_MyClass::initTestCase()
QBenchmarkPerfEventsMeasurer::start: perf_event_open: Permission denied
[ChangeLog][QtTest] Fixed support of Linux performance counters for
QBENCHMARK, which used to fail with "Permission denied" errors in
default configurations. Now, QtTest will automatically fall back to
profiling only userspace, like the perf(1) tool does.
[1] https://github.com/torvalds/linux/blob/master/Documentation/admin-guide/sysctl/kernel.rst#perf-event-paranoid
Pick-to: 5.15 6.2 6.4
Change-Id: I3c79b7e08fa346988dfefffd171f897be7794935
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
Reviewed-by: David Faure <david.faure@kdab.com>
The purpose of warming up is to get all code paths executed, so any lazy
function resolving is processed, statics are allocated, etc. There's no
reason to run it more than once -- if you're trying to train the Branch
Predictor Unit, you'd want to do it another way anyway. This is useful
when benchmarking with -iterations N, because QBenchlib currently runs
2*N iterations because of the warm up. That just wastes time.
Change-Id: I3c79b7e08fa346988dfefffd172030c889b31a1f
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
I don't see anything using them in any unit test or other Qt module, and
they are marked internal.
Change-Id: I3c79b7e08fa346988dfefffd172028dae6d9e1ba
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
It was probably part of the original design to have an intermediate
measurement to determine whether the results were acceptable or not, but
it was never used. So just remove.
Change-Id: I3c79b7e08fa346988dfefffd1720281b2512b8c1
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Commit 8995045c62 forgot to remove this
portion.
Pick-to: 6.4
Change-Id: I3c79b7e08fa346988dfefffd17202b19665ddc8a
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
Avoids the special case in the code and removes the need to run a large
number of iterations to get any useful result, plus makes it's more
readable:
From:
0.00002125 msecs per iteration (total: 2,125, iterations: 100000000)
To:
22.082 nsecs per iteration (total: 22,082, iterations: 1000)
Pick-to: 6.4
Change-Id: I3c79b7e08fa346988dfefffd171facc98af9e9fc
Reviewed-by: David Faure <david.faure@kdab.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
To allow the user to customize the C++ code that QDoc sees, so as to be
able to work-around some limitations on QDoc itself, QDoc defines two
symbols: Q_QDOC and Q_CLANG_QDOC, both of which are "true" during an
entire execution of QDoc.
At a certain point in time, QDoc allowed the user the choice between a
custom C++ parser and a Clang based one.
The Q_QDOC symbol would always be defined while the Q_CLANG_QDOC symbol
would be defined only when the Clang based parser was chosen.
In more recent times, QDoc always uses a Clang based parser, such that
both Q_CLANG_QDOC and Q_QDOC are always defined, making them equivalent.
To avoid using different symbols, and the possible confusion and
fragmentation that derives from it, all usages of Q_CLANG_QDOC are now
replaced by the equivalent usages of Q_QDOC.
Change-Id: I5810abb9ad1016a4c5bbea99acd03381b8514b3f
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
This flag was introduced in the Linux kernel version 3.14. Trying to use
perf_event_open() now with a kernel older than that will cause EINVAL
errors, but 3.14 is from 2014, so most likely old enough for all of
Qt 6.
For that, I updated the copied header from v6.0.
Pick-to: 6.4
Change-Id: I3c79b7e08fa346988dfefffd171f895201ceb4f4
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
When list->next is nullptr, setting ignoreResultList to list->next is
the same as setting it to nullptr.
Change-Id: If328ce31db4344cf0136b7a827be871eb89e35b5
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
The timestamp will no longer be incremented by 500ms after a mouse
release if the delay has been explicitly specified.
The default delay is 1 ms since f5010c49a3
but the running timestamp was unconditionally post-incremented by 500ms
after every mouse release, to prevent double-clicks, which were always
deemed as unintended (because we have a mouseDClick function for that).
Now, we do that 500ms increment only if the user has not provided a
delay value in the function argument at all. We have often found it
useful in our own tests to generate double-clicks "the hard way", by
sending indivdual events, so as to be able to check state in some target
object at each step, as shown in the new snippet.
[ChangeLog][QtTest] QTest::mouseRelease() and mouseClick() can now be
used to test double-clicks, by specifying a realistic timestamp delay.
Fixes: QTBUG-102441
Change-Id: I8e8d242061f79efb4c6e02638645e03661a9cd92
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
This is a combination of Q_UNREACHABLE() with a return statement.
ATM, the return statement is unconditionally included. If we notice
that some compilers warn about return after __builtin_unreachable(),
then we can map Q_UNREACHABLE_RETURN(...) to Q_UNREACHABLE() without
having to touch all the code that uses explicit Q_UNREACHABLE() +
return.
The fact that Boost has BOOST_UNREACHABLE_RETURN() indicates that
there are compilers that complain about a lack of return after
Q_UNREACHABLE (we know that MSVC, ICC, and GHS are among them), as
well as compilers that complained about a return being present
(Coverity). Take this opportunity to properly adapt to Coverity, by
leaving out the return statement on this compiler.
Apply the macro around the code base, using a clang-tidy transformer
rule:
const std::string unr = "unr", val = "val", ret = "ret";
auto makeUnreachableReturn = cat("Q_UNREACHABLE_RETURN(",
ifBound(val, cat(node(val)), cat("")),
")");
auto ignoringSwitchCases = [](auto stmt) {
return anyOf(stmt, switchCase(subStmt(stmt)));
};
makeRule(
stmt(ignoringSwitchCases(stmt(isExpandedFromMacro("Q_UNREACHABLE")).bind(unr)),
nextStmt(returnStmt(optionally(hasReturnValue(expr().bind(val)))).bind(ret))),
{changeTo(node(unr), cat(makeUnreachableReturn,
";")), // TODO: why is the ; lost w/o this?
changeTo(node(ret), cat(""))},
cat("use ", makeUnreachableReturn))
);
where nextStmt() is copied from some upstream clang-tidy check's
private implementation and subStmt() is a private matcher that gives
access to SwitchCase's SubStmt.
A.k.a. qt-use-unreachable-return.
There were some false positives, suppressed them with NOLINTNEXTLINE.
They're not really false positiives, it's just that Clang sees the
world in one way and if conditonal compilation (#if) differs for other
compilers, Clang doesn't know better. This is an artifact of matching
two consecutive statements.
I haven't figured out how to remove the empty line left by the
deletion of the return statement, if it, indeed, was on a separate
line, so post-processed the patch to remove all the lines matching
^\+ *$ from the diff:
git commit -am meep
git reset --hard HEAD^
git diff HEAD..HEAD@{1} | sed '/^\+ *$/d' | recountdiff - | patch -p1
[ChangeLog][QtCore][QtAssert] Added Q_UNREACHABLE_RETURN() macro.
Change-Id: I9782939f16091c964f25b7826e1c0dbd13a71305
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The previous config did not work for prefix builds (files were not
copied over). To fix this, we simply copy over the files in case
of a prefix build.
Additionally removed batch_test_feature from the condition,
as this is now the de facto wasm test runner, which supports
batched and unbatched tests.
Change-Id: Ib232c7898de0a0d750e4ca5ebf1da8cbcc7da3e0
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
We've been requiring C++17 since Qt 6.0, and our qAsConst use finally
starts to bother us (QTBUG-99313), so time to port away from it
now.
Since qAsConst has exactly the same semantics as std::as_const (down
to rvalue treatment, constexpr'ness and noexcept'ness), there's really
nothing more to it than a global search-and-replace, with manual
unstaging of the actual definition and documentation in dist/,
src/corelib/doc/ and src/corelib/global/.
Task-number: QTBUG-99313
Change-Id: I4c7114444a325ad4e62d0fcbfd347d2bbfb21541
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
There will now be a visual output in page if the qvisualoutput query
parameter is supplied. This simplifies debugging.
The main html resource has been renamed test_batch.html to reflect
the name of the actual test unit, not functionality.
Change-Id: Ib6cd4712de9c47cfcc5f670e7b34f998858f99b7
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
The QTRY_* macros and QTestEventLoop have (since 6.3) been exiting
their loops early if the test has failed. Where that was appropriate,
they should also have been exiting early on skip.
[ChangeLog][QtTest] Added QTest::currentTestResolved(), which is true
if the test has failed or skipped. The QTRY_*() macros and
QTestEventLoop now use this, rather than QTest::currentTestFailed(),
to test whether they should stop looping, so that they also do so on a
skip.
Task-number: QTBUG-104441
Change-Id: Ibf3d5a095b35e6670bc3daf756f05b66f7f3ef9b
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
While it's being picked into 6.3, it's not present in 6.3.[01], so
I've left it out of QDoc's sight (even though it's still \internal)
for the version picked to 6.3, but let's include it in internal docs
as "from 6.4" since that's the first minor release to contain it.
Pick-to: 6.4
Change-Id: I1704a1ca4ba1231d0213e9ca236ef8401a59ddd0
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
The QTRY_* macros and QTestEventLoop exit early if the test has
resolved; however, in the cleanup phase of a test, even if the test
has failed, these loops should continue as normal.
[ChangeLog][QtTest] During the cleanup() phase of a test, the QTRY_*
macros and QTestEventLoop now ignore the test resolution, in contrast
to when they are used from the test itself, which (since 6.3.0) exits
the loops early if the test has failed.
Pick-to: 6.4 6.3
Fixes: QTBUG-104441
Change-Id: I2673161967cbbc57815155af698a9338ab98a686
Reviewed-by: Jason McDonald <macadder1@gmail.com>
The regex used a greedy .+ to match an actual dot and a space,
preceding an open parenthesis, with the result that if the message
contained any parentheses that .+ swallowed everything up to the last
of them. Correct the regex so that recently-added tests' error
messages show up correctly.
Change-Id: I6e52c9b2a6e7959335fcddbb4266f65b589eba68
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Use the standard mechanism instead of a hand-rolled way of identifying
function/data-tag combinations. Adds missing data relevant to
identifying test cases, some of which previously had apparently the
same name, with no hint at why the test-case was seemingly run
repeatedly.
Change-Id: I6225c6d1990069c94a1f1c8dbb179993b96076e7
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
All our other loggers use it and teamcity's use of QString implied
lots of conversions from UTF-8 and back, or in some cases to native
encoding, that ended up failing to correctly propagate Unicode tokens
that the other test loggers do deal with correctly. This does not
change the generated output on Linux.
Change-Id: I036d8043d7b3b19e7a0e1f296a23628f98223563
Reviewed-by: Jason McDonald <macadder1@gmail.com>
This makes the js batched test runner cooperate with emrun.
The runner sends back the output and exit messages to emrun to inform it
about the test execution state.
The code is based on emrun_postjs.js from the emsdk.
Change-Id: I758f2c185797f4000810eb4314423eebc1c5d457
Reviewed-by: David Skoland <david.skoland@qt.io>
Overlooked in the review where this changed, this value was already in
seconds and needs to be converted to millis before converting back.
Now printing output such as:
<testcase name="initTestCase" classname="tst_QHash" time="0.001"/>
Amends bb74e72aa9
Pick-to: 6.2 6.3 6.4 6.4.0
Fixes: QTBUG-106222
Change-Id: I1f8b774eea3dcbe2b4e822e2b0b2efb1ccc01abb
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Also follow this best practice in testlib's own documentation and
examples.
Pick-to: 6.4
Change-Id: I8b57dfa8f88835adae8fceeb122a16635708e338
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Jaishree Vyas
Marc Mutz
Kai Köhne
Andreas Eliasson
Tor Arne Vestbø
Thiago Macieira
David Skoland
Mitch Curtis
Edward Welbourne
Luca Di Sera
Shawn Rutledge
Dmitry Shachnev
Mikolaj Boc
Mårten Nordheim