Use lldb to generate a stack trace.
Change-Id: I30b0ab656dda66b6aaa05841df641bee004702f0
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@theqtcompany.com>
Reduce the side-effects of QThread start-up happening concurrently while test
functions begin to execute by terminating the WatchDog constructor only when
the thread itself is up and running.
Change-Id: I21ee44206d789a7ecb5a220ee4729572548c0277
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Dump a stack trace on Linux if the process crashes. This will give us
much better log output in the new CI system. In addition, create a
watch dog thread, that kills the test if the test function times
out (ie. hangs)
Implementations of the stack trace dumping for Mac and Windows
are still pending.
Change-Id: I65426c5afe290a0a2019b881436a0c278f1cafaf
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
Will be active when running test apps through Xcode's 'test' action,
and reports QtTestLib test objects and functions to Xcode as XCTest
cases.
This allows running tests on both iOS Simulator and iOS devices from
the command line, through xcodebuild, without relying on any 3rd party
tools. It also integrates Qt test failures and passes into the Xcode
IDE, which may be useful for closer investigation of test failures.
The feature is limited to Xcode 6.x.
Change-Id: I33d39edbabdbaebef48d2d0eb7e08a1ffb72c397
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@theqtcompany.com>
qtbase/src/corelib/itemmodels/qitemselectionmodel.cpp:1199: warning: Undocumented parameter 'model' in QItemSelectionModel::modelChanged()
qtbase/src/corelib/plugin/qpluginloader.cpp:420: warning: Can't link to 'staticPlugin()'
qtbase/src/gui/text/qtextdocument.cpp:1452: warning: No such parameter 'from' in QTextDocument::find()
qtbase/src/testlib/qtestcase.cpp:2770: warning: No documentation for 'QTest::qExtractTestData()'
Change-Id: I76758dbdf3ad7b0ae1f14de0407a6fde523827d0
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
Similar to what we've done to QString, only we print each byte that is
not ASCII as \OOO (octal representation).
[ChangeLog][QtTest] QtTest now prints an escaped version of
QByteArrays that failed to compare with QCOMPARE, instead of the hex
dump.
Change-Id: I6a8c43f138c66c998280998a242b43cd579666a0
Reviewed-by: Richard J. Moore <rich@kde.org>
Qt copyrights are now in The Qt Company, so we could update the source
code headers accordingly. In the same go we should also fix the links to
point to qt.io.
Outdated header.LGPL removed (use header.LGPL21 instead)
Old header.LGPL3 renamed to header.LGPL3-COMM to match actual licensing
combination. New header.LGPL-COMM taken in the use file which were
using old header.LGPL3 (src/plugins/platforms/android/extract.cpp)
Added new header.LGPL3 containing Commercial + LGPLv3 + GPLv2 license
combination
Change-Id: I6f49b819a8a20cc4f88b794a8f6726d975e8ffbe
Reviewed-by: Matti Paaso <matti.paaso@theqtcompany.com>
... in QCOMPAREs.
The implementation is hidden in a nested Internal namespace that retrieves
the strings without strdup()ing. That makes it easier to compose these
functions as there is no need to delete character arrays when using them.
The public interface (which qstrdup()s) is implemented on top of these.
[ChangeLog][QtTest] QCOMPARE now pretty-prints QSizePolicy{,::Policy,::ControlType{,s}}.
Change-Id: Ib03d969847e5a12474c71a7921366b400025f680
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
[ChangeLog][QtCore][QDebug] Printing of QStrings and QStringRefs
whenever "noquote" is not active now prints the strings in a format that
can be copied back to C++ code. All characters that aren't printable in
US-ASCII are escaped (this includes printable Unicode characters outside
of US-ASCII). Pretty-printing will not respect QTextFormat padding or
field widths.
Change-Id: I169a8a0508e24693f5652f0129defe7f709e5d08
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@theqtcompany.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Add an 'info' message type that can be used for messages that are neither
warnings (QtWarningMsg), nor for debugging only (QtDebugMsg). This is
useful mainly for applications that do not have to adhere to the
'do not print anything by default' paradigm that we have for
the Qt libraries itself.
[ChangeLog][QtCore][Logging] QtInfoMsg got added as a new QtMsgType.
Use the new qInfo(), qCInfo() macros to log to it.
Change-Id: I810995d63de46c41a9a99a34d37c0d417fa87a05
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Following c61f8df404, we now prefer overloading
toString() in the type's namespace over specializing the primary template.
Let the docs reflect that and add an example. Also suggest to delegate the messy
raw char pointer handling to the existing toString(QString)/toString(QBA)
overloads.
Change-Id: Id76181faba86aea52588611ea64ea9b95371a733
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Change-Id: Ia624fcefe77a456424bcaa00b106ef7f69abb6b9
Reviewed-by: Simon Hausmann <simon.hausmann@digia.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
On Android, none of the file system paths used for QFINDTESTDATA
currently are suitable. We do have the possibility of putting
test data in qrc, but in cases where the test is specifically
testing access to the regular file system, we need a way to
find files there as well. We add a fifth step when all other
fail, which searches the current active directory for the data.
Change-Id: I4f02f8530b5843eb282bd112ea03ed6a476593d6
Reviewed-by: BogDan Vatra <bogdan@kde.org>
It's a simple enough function, but we don't need to duplicate those 17
bytes all over the place. Now they'll be duplicated at most once per
library.
Change-Id: Ic995e2a934b005e7e996e70f2ee644bfa948eb38
Reviewed-by: Jason McDonald <macadder1@gmail.com>
qdevel is an option used from winrtrunner to indicate a specific
environment to the app. QTestLib needs to ignore that parameter similar
to the Visual Studio generated arguments.
Change-Id: I00a3abe19f1e5b4159e65d26050f04e28f40316f
Reviewed-by: Oliver Wolff <oliver.wolff@theqtcompany.com>
Reviewed-by: Andrew Knight <andrew.knight@theqtcompany.com>
We're not ready.
[ChangeLog][EDITORIAL] Remove all mentions of QVersionNumber.
Change-Id: I03ad95992982eb3177f982c1eeddb6a6bc29336c
Reviewed-by: Keith Gardner <kreios4004@gmail.com>
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Set QT_QTESTLIB_RUNNING=1 on QTestLib startup.
This allows Qt to adapt its behavior to the testing
environment, WHICH IS NORMALLY NOT SOMETHING YOU WANT
TO DO. Use with caution.
Change-Id: I31de80e7c2cc91ff1ed8137926e6b729ef6efdc6
Reviewed-by: Laszlo Agocs <laszlo.agocs@digia.com>
For QLatin1String, operator== is overloaded, so comparing to a latin-1
(C) string literal is efficient, since strlen() is comparatively fast.
OTOH, QStringLiteral, when not using RVO, litters the code with
QString dtor calls, which are not inline. Worse, absent lambdas,
it even allocates memory.
So, just compare using QLatin1String instead.
Change-Id: I7af3bf3a67c55dae33ffaf9922d004fa168a3f9c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QtTest has its own logging environment, such as XML output, so we don't
want to have messages formatted differently than expected or sent to an
unexpected place.
Change-Id: If665c9d7121267269e5b2063f49468eb2b9b9d08
Reviewed-by: Jason McDonald <macadder1@gmail.com>
We need to have a finer grained control over the tests
we skip in our CI system. This adds a blacklisting
mechanism that allows blacklisting individual test
functions (or even test data) using a set of predefined
matching keys for the operating system and some other
relevant variables.
QTestlib will search for a file called BLACKLIST in the test
directory and parse it if found. The file contains a simple
ini style list of functions to blacklist. For details see
qtestblacklist.cpp.
Change-Id: Id3fae4b264ca99970cbf9f45bfb85fa75c1fd823
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
The class provides compare operators, stream operators, and hashing
functions. This class aims to be compatible with (but not restricted to)
the Semantic Versioning 2.0 standard (semver.org).
[ChangeLog][QtCore] Added QVersionNumber class
Done-with: Marc Mutz <marc.mutz@kdab.com>
Change-Id: I244c8ccc002909af03987a2df052734d1a8621a9
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
If none of the other locations work, we can fall back to bundling
the test data in qrc to make it possible to include test data on
platforms that do not have any good way of deploying files to the
traditional file system, like Android.
Change-Id: I7779ce6be2380d551697cecfb9f76652d1fd0a38
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@digia.com>
The collected data doesn't reveal any obvious correlation between sleep
times and test failures but the many warnings are a hassle for anyone
reading the test results.
Change-Id: I71bd0c90c20c730573693f23f4435e538b635d44
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
Like this we can find out more easily if autotests fail because of
flaky timing.
Change-Id: I57b10f5fc4908bed7d00990641b2ddfd4ea84a11
Reviewed-by: Jan Arve Sæther <jan-arve.saether@digia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
If there's no qApp, then QCoreApplication::arguments() returns an empty
list (after printing a warning). That means the callgrind runner can't
get the arguments it needs in order to rerun the benchmark.
The crash happens because it always uses .at(0) to try and get the
executable's path. Even if we get the path from somewhere else, we still
need the arguments.
Change-Id: I5c74af4d96fc5824b2b7fd7a89648d78393016e2
Reviewed-by: Sergio Ahumada <sahumada@blackberry.com>
This also has the advantage of not requiring the use of the locale
codec. Quite an advantage if you're debugging the locale codec. But it's
mostly so that we don't get question marks that hide the difference we
were trying to locate.
[ChangeLog][QtTest] QtTest now prints an escaped version of QStrings
that failed to compare with QCOMPARE. That is, instead of converting
non-printable characters to question marks, QtTest will print the
Unicode representation of the character in question.
Change-Id: I44c1ef3246b188c913dacd3ca4df02581356ea41
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
This is only useful for logging benchmarks, since it won't print test
passes, failures, etc. It's useful for importing to spreadsheets to do
number-crunching.
[ChangeLog][QtTest]Added a CSV logging mode that is suitable for
importing benchmark results into spreadsheets. This can be enabled by
the -csv option on the command-line. The CSV logging mode will not print
test failures, debug messages, warnings, etc.
Change-Id: I245d6f86bb380645c9bc0d748cf474b3ed42cab8
Reviewed-by: Sergio Ahumada <sahumada@blackberry.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
For the conflicts in msvc_nmake.cpp the ifdefs are extended since we
need to support windows phone in the target branch while it is not there
in the current stable branch (as of Qt 5.2).
Conflicts:
configure
qmake/generators/win32/msvc_nmake.cpp
src/3rdparty/angle/src/libEGL/Surface.cpp
src/angle/src/common/common.pri
src/corelib/global/qglobal.h
src/corelib/io/qstandardpaths.cpp
src/plugins/platforms/qnx/qqnxintegration.cpp
src/plugins/platforms/qnx/qqnxscreeneventhandler.h
src/plugins/platforms/xcb/qglxintegration.h
src/widgets/kernel/win.pri
tests/auto/corelib/thread/qreadwritelock/tst_qreadwritelock.cpp
tests/auto/corelib/tools/qdatetime/tst_qdatetime.cpp
tests/auto/gui/text/qtextdocument/tst_qtextdocument.cpp
tools/configure/configureapp.cpp
Change-Id: I00b579eefebaf61d26ab9b00046d2b5bd5958812
New macro QVERIFY_EXCEPTION_THROWN may be used to check that some code
really throws an exception of specified type.
Change-Id: I7cf499c7c37a35407862bc604c6eb862c5f329d3
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Make it possible to match messages by a pattern.
Change-Id: I713312e86db5471755459f1ecc43e8f1ac7a95fb
Reviewed-by: Jason McDonald <macadder1@gmail.com>
While exception handling methods are not available on WinRT, the debug
mode ones are. This way we can suppress error dialogs from popping up
like on desktop windows.
Change-Id: Id6e9666af9274d373e7d46e3f9e44892a2bd0dbc
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
This reverts commit 48586b2bac.
This causes problems in QtCreator without anyone being able to say why.
Task-number: QTBUG-34397
Change-Id: I9ea5457724d5af8d87e0bc40e6615748daf9c04c
Reviewed-by: Simon Hausmann <simon.hausmann@digia.com>
Visual Studio debuggers send an argument to specify the debugger port.
This argument needs to be skipped while parsing and not to be interpreted
as an option or test function.
Change-Id: I24efb52fbd668a7bc3388c876f5ea0d950de1d5b
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
Default values should have mark-up to denote that they are code.
This commit changes:
-"property is true" to "property is \c true".
-"Returns true" to "Returns \c true".
-"property is false" to "property is \c false".
-"returns true" to "returns \c true".
-"returns false" to "returns \c false".
src/3rdparty and non-documentation instances were ignored.
Task-number: QTBUG-33360
Change-Id: Ie87eaa57af947caa1230602b61c5c46292a4cf4e
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
Reviewed-by: Jerome Pasion <jerome.pasion@digia.com>
Except where we're actually testing QCoreApplication::applicationName()
and friends.
Change-Id: I25514884c11f43a4f82b1f818f822dc3d79f69a3
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@digia.com>
Various global changes, primarily preprocessor flow, to support the
WinRT platform.
Change-Id: I3fa9cf91d5fb24019362e88fcf205e31b4f810b5
Reviewed-by: Andrew Knight <andrew.knight@digia.com>
Sorting is O(NlogN) complexity, while nth_element is linear.
Change-Id: Ic6596affe183494e87abe7bdaa7c9985f5b7cd58
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
The error dialog is sometimes desired when trying to attach a
debugger. Use the -nocrashhandler option for it, which already
exists on UNIX.
Change-Id: I42c9311242cdeaa81a148af9bf61875e1bc6f32c
Reviewed-by: Simon Hausmann <simon.hausmann@digia.com>
All occurrences of `#if defined(Q_OS_MAC) && !defined(Q_OS_IOS)` have
been replaced with `#if defined(Q_OS_MACX)`.
Change-Id: I5055d9bd1845136beb8ed1c79a8f0f2c0897751a
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@digia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Short-lived benchmarks (benchmarks that complete in a very short period
of measured time) are often more affected by jitter and warm-up effects
than longer running benchmarks. Since QBENCHMARK stores the median
result of all accepted benchmark runs, a larger number of aggregation
runs is preferable for short-lived tests, but not necessarily for longer
running ones. The minimumtotal option, specified in units of the
selected measurement, will make the benchmark repeat a benchmark until
the total measured cost exceeds the specified threshold. The displayed
median result will then tend to be more accurate. This is especially
useful for data-driven benchmarks in case the data tags scale the
benchmarked operation from little to large cost.
Change-Id: Ib857de64aaffc77715a0000d36f0245f31d86b9a
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Added support for most hardware and software counters available on
Linux 3.3. The cache-based counters are missing because they need
special handling.
Also added an option that lists available counters. Note that the list
is of counters the library knows about, not the counters that the
hardware can measure.
Change-Id: I9f6fb09b5460bf4ac6082081611c1d6ff806a3fa
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Currently, we only support one event type: counting CPU cycles with
hardware counters. There are no fallbacks if this hardware counter is
not available, and there is currently no way to specify other
counters.
Benchlib only supports reporting one event per benchmark, even though
the event counter interface allows specifying more than one. Still,
the hardware usually has limitations on how many events it can monitor
at a time, and we'd prefer to have the counter running at 100% of the
time, so this will not change.
Change-Id: I79858a3ad1e696dc4b7b72c420e5a04b67cd55de
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Currently, it only prints "perf available" if you use the -perf option
and perf is available. The implementation comes in the next commits.
Change-Id: Ic6cdee70e21df25780799a4bc31ca2c2d923b9f8
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Change-Id: I491edcae84c149775cf3640626ccff8cbf0d69fe
Reviewed-by: Jon Severinsson <jon@severinsson.net>
Reviewed-by: Andreas Aardal Hanssen <andreas@hanssen.name>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@digia.com>
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Reviewed-by: Stephen Kelly <stephen.kelly@kdab.com>
QtTestLib and QTestLib don't exist. The proper name is "QtTest" (code)
or "Qt Test" (English)
http://qt-project.org/wiki/Spelling_Module_Names_in_Qt_Documentationhttp://lists.qt-project.org/pipermail/interest/2012-December/005221.html
Files paths in qttestlib.qdocconf can't be changed easily however, as it
breaks things. So, they're left as they are.
Change-Id: Ifbc44ea858c453bedad8cd7723f847e67fc7a85a
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Jerome Pasion <jerome.pasion@digia.com>
Only QStringList was handled before, now any QList is handled.
A specialization for QStringList is still needed though, due to the way
template matching works.
Example with QList<int>. Before:
FAIL! : tst_QTextCodec::threadSafety() Compared values are not the same
Loc: [../tst_qtextcodec.cpp(2057)]
After:
FAIL! : tst_QTextCodec::threadSafety() Compared lists differ at index 0.
Actual (res2.toList()): '0'
Expected (mibList): '3'
Loc: [../tst_qtextcodec.cpp(2057)]
Change-Id: If0fdec3236ddb78a679ee549aba569ef5571c395
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
Added prepare_docs to qt_build_config.prf (it was added
directly in configure in the source branch)
Conflicts:
configure
tools/configure/configureapp.cpp
Change-Id: I1337c69fc62b1c934e3e39b4409e4857440c9db8
1. Remove the code for activate application, due to Qt 5 already
has this feature
2. Update the code for power setting
Task-number: QTBUG-23267
Change-Id: Id257c2e10e8e750a4b68cd2995c2ac4b70f8910c
Reviewed-by: Jiang Jiang <gzjjgod@gmail.com>
Reviewed-by: Liang Qi <liang.qi@digia.com>
Replace "QTestLib" with "Qt Test". Some style and language edits.
Remove some empty headings.
Change-Id: I568b4bbde981c3ef1899c830931892c16b66b4e2
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
List functions matching the command line parameter instead
of dumping all functions.
Change-Id: Ic504587b1036f09702f47579f90406333c4efbeb
Reviewed-by: Caroline Chao <caroline.chao@digia.com>
MinGW-64 defines _CrtSetReportMode, but only as a noop, resulting
in a warning "statement has no effect [-Wunused-value]"
Change-Id: I6e509246134dd5929b23b07abdb6d705db603793
Reviewed-by: Jonathan Liu <net147@gmail.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
qCompare() was both overloaded and specialised, but always as a template.
This lead to the QIcon specialisation actually invoking
qCompare(QFlags<void*>, ...)
when specifically asking for qCompare<void*>() (detected by adding
underlying-type detection to QFlags).
Fix by preferring overloading and not specialising anything.
Change-Id: Ie001ebb9dfb0847c6c33a3f45177a61579fd61ee
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
Change copyrights and license headers from Nokia to Digia
Change-Id: If1cc974286d29fd01ec6c19dd4719a67f4c3f00e
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com>
The documentation for QTest::qExec states that command line arguments
are optional and gives the following example:
MyTestObject test1;
QTest::qExec(&test1);
However, running this example leads to crash as argv[0] is accessed
without testing argc. This change fixes this bug.
Change-Id: I2ec016e02869d21b24bc11f1851a760036640191
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
The QWindow* overload was introduced for Qt 5.0, but is unused already.
Change-Id: Ic07f19eac87bc93a589017407668760fceb2f632
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Use \b instead of \bold
* widnow -> window typo
* QTest::touchEvent's second parameter is QTouchDevice *, not
QTouchEvent::DeviceType, and autoCommit defaults to true
* QSignalSpy::wait() isn't const
Change-Id: I75a91c1cd98b1c582cd654b2b72ad61e5c0e31d2
Reviewed-by: David Faure <faure@kde.org>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Except where using the contents of the variable, in which case
collapse two calls to qgetenv() for the same variable into one
that stores the result in a temporary QByteArray and continues
working with that one instead.
Change-Id: I6c09a20ae946327ccb85e4833a60a373a8a07355
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
- Implement waitForWindowExposed() for toplevel windows.
- Implement waitForWindowShown(QWidget *) and mark as
deprecated in line with waitForWindowShown(QWindow*).
- Use in tests.
- Simplify tests (collapse waitForExposed, setActive
into setActiveWindow, waitForActive), remove most
hard-coded timeouts.
- Stabilize graphicsview tests by using waitForWindowActive.
Change-Id: Ic7c061e2745b36f71a715ee4e47c0346b11a91e8
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
This change moves the snippets to the modularized directories.
Change-Id: I917dd1dae5ee5d4b6bd5a0390783a8b9a99edc06
Reviewed-by: Marius Storm-Olsen <marius.storm-olsen@nokia.com>
Use QMetaMethod::name() instead of parsing the signature.
Use QMetaMethod::returnType() instead of checking the length of
typeName().
Use QMetaMethod::parameterCount() instead of checking the
size of parameterTypes().
Change-Id: I424370b19b5b150865377666dca0fba5f29ad30f
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
qWaitForWindowShown() should check window->isActive() instead of
window->isExposed() and return false if timeout.
Add two new qWaitForWindowActive() and qWaitForWindowExposed()
functions.
Change-Id: Idd9601805c2e84b0d36ddd5471031b627d289953
Reviewed-by: Samuel Rødal <samuel.rodal@nokia.com>
Previously, verbose (-v2) and XPASS test output showed all QCOMPAREs as
"COMPARE()", making it impossible to see what was compared and difficult
to match the output to the source of a test containing many calls to
QCOMPARE.
This commit changes testlib's internal compare_helper API so that string
representations of the compared expressions are always passed to
QTestResult::compare() when available, and can thus be shown in the
verbose and XPASS output. The XPASS output has also been changed to
state explicitly that the comparison succeeded unexpectedly, bringing it
in line with the XPASS output resulting from a call to QVERIFY.
This commit also changes all calls to compare_helper() to call the
eight-argument version of the function, which simplifies much of the
calling code. The now obsolete four-argument version of
compare_helper() has been changed to output a warning that it is
obsolete. It will be removed once other modules have had some time to
catch up.
The improved XPASS and verbose output is demonstrated by the expectfail
and verbose2 selftests.
Change-Id: I8baa46d5dd30e6c43b26f366c34dc5b64aab5f7c
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
QDoc now has support for Doxygen style commands for italics, bold
and list items. This change applies that change in QDoc to the
actual documentation.
Task-number: QTBUG-24578
Change-Id: I519bf9c29b14092e3ab6067612f42bf749eeedf5
Reviewed-by: Shane Kearns <shane.kearns@accenture.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Change the help output to show which options only work for plain text
logging.
Change-Id: I39eb7cd0793cdbe553c230334c6cd532b4929f61
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
In Qt5 the meta-data format will be changed to not store the
method signature string explicitly; the signature will be
reconstructed on demand from the method name and parameter type
information.
The QMetaMethod::signature() method returns a const char pointer.
Changing the return type to QByteArray can lead to silent bugs due to
the implicit conversion to char *. Even though it's a source-
incompatible change, it's therefore better to introduce a new
function, methodSignature(), and remove the old signature().
Task-number: QTBUG-24154
Change-Id: Ib3579dedd27a3c7c8914d5f1b231947be2cf4027
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: João Abecasis <joao.abecasis@nokia.com>
Previously the silent logging mode suppressed passes, skips and internal
testlib info messages, but did not suppress debugging output, making it
hard to see the fails in a noisy test. This commit changes silent mode
so that it suppresses all output except test failures and fatal errors,
making silent mode truly useful for seeing just the important test
output.
This commit also adds a selftest to verify the behaviour of silent mode.
Change-Id: I75420aead03682306210746a87e2a3b608b58fc6
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Call it from qtest_qParseArgs() instead of qExec(). This will allow to
remove the code coverage calls from quicktest.
Both the installation and the saving of code coverage data are now
gathered in testlib.
Change-Id: If3baedb0cff9d894e6f232b947af7e3158703d9d
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
as QChar is actually an ushort and there is no point in taking its
address.
Merge-request: 69
Change-Id: Idcc9d621e5627514ade006aa12a789a88929d48b
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
Prior to this commit, a benchmark test could report 0..n passes and 0..m
fails or skips, where n is the number of accumulation iterations used to
collect benchmark data and m is the number of times the test function
was invoked. Depending on the type of benchmark measurer being used,
this could result in a very large volume of test output and inconsistent
pass, fail and skip counts between test runs.
This commit changes the behaviour so that each benchmark test reports
one pass, fail or skip, regardless of the number of iterations used to
collect benchmark data.
This commit also prevents benchmark data being reported in the test
output if the benchmark test failed or skipped, as any benchmark data is
of dubious value in such cases.
The latter change in behaviour requires a minor modification to the
badxml selftest, which now tests quoting of literal strings in xml test
output for both passing and failing benchmarks.
Finally, this commit also adds a new selftest specifically for verifying
correct behaviour for benchmarks that fail or skip.
Task-number: QTBUG-24313
Change-Id: I3426dc659a7511b62fd183a031c7235bc753f497
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Prior to this commit, the following statement in the qtestlib
documentation was untrue: "If init() fails, the following testfunction
will not be executed, the test will proceed to the next testfunction."
If init() called QSKIP, the test function would be skipped, but if
init() reported a failure, the test function would still be executed
(even though doing so could be unsafe).
This commit makes testlib skip a test function if init() reports a
failure and enhances the selftests to cover skips and fails in both
init() and cleanup().
Task-number: QTBUG-20371
Change-Id: Id1cc8464ae0b8c257ae1b74dbe9189a501f5366b
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Testlib no longer does anything with the test location, and neither do
any of Qt's tests, so the code is no longer needed.
Change-Id: I1a464116179f3e26efadad6901f00f3c33e0ce84
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Some headers and constants are available in MinGW-w64 that are not
available in the official MinGW. STATE_SYSTEM_HASPOPUP and
STATE_SYSTEM_PROTECTED constants are defined if they are not already
defined by including oleacc.h. _CrtSetReportMode is not used and
crtdbg.h is not included when using official MinGW as crtdbg.h is
missing from official MinGW.
Change-Id: Ie7f3f3726a1663d0fdeb6ee17b86873ae3f61860
Reviewed-by: Jonathan Liu <net147@gmail.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
For data-driven tests, testlib previously counted one fail or skip for
each data row that failed or skipped, while it counted only one pass
for a test function where all rows passed and counted no passes for a
test function where some rows passed and some rows failed. A similar
problem also existed for benchmark tests, which could run multiple
iterations of the same test, with each fail and skip being counted but
only a single pass being counted for the entire series of iterations.
This commit makes testlib count one pass, fail or skip for each data
row. Test functions that are not data-driven count one result for the
test function, as before. Benchmark tests count one pass, fail or skip
per iteration.
A side-effect of this change is that the test output in plain text, xml
and light xml formats now shows a result for every data row and
benchmark iteration executed, allowing post-processors to correctly
calculate the total number of tests executed. Previously, individual
rows were not shown in the test output if they passed, making such
calculations impossible.
The only change to the xunitxml output format is to correct a bug where
no test result was recorded for a test function if the last data row
was skipped and all other rows passed -- in which case the overall
result should be a pass. Note that there is also a pre-existing bug
in the xunit logger, where no result is reported if all rows are
skipped; that bug is unaffected by this commit.
Task-number: QTBUG-21848
Task-number: QTBUG-22124
Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b
Reviewed-by: Ed Baak <ed.baak@nokia.com>
Set QT_TESTCOCOON_ACTIVE environment variable when the coverage is installed
for a test and unset it when the coverage data is saved. Tests that run when
QT_TESTCOCOON_ACTIVE is set are subtests and will not be considered as
stand-alone tests for the coverage.
When a test is run as a subtest its coverage data will not be saved for
itself but for the main test it is merged with. Also its status will not be
reported since only the status of the main test is expected in the test report,
e.g. the test tests/auto/testlib/selftests.
Change-Id: Icfdf99300aae18040e1a3441a8af21f68df4c0db
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
A messagebox will be shown for an assertion in
debug mode. This introduces a need for user
interaction to proceed the execution of the
unit test. Setting the Report mode to debug,
will only print the assertion to stderr.
Change-Id: If8ae80ea96d6608cba77b9c6ca176f97d1680932
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Reviewed-by: Joerg Bornemann <joerg.bornemann@nokia.com>
Previously, expected failures were only cleared at the end of each test
function, i.e. after all data rows were finished. This meant that if a
data-driven test function called QEXPECT_FAIL and then didn't perform
any further verification steps to trigger the expected failure, the
expected failure would be carried over to the next data row, probably
causing the first verification step in the test function to XPASS (with
a seemingly irrelevant error message) for the next data row.
This commit adds the new function QTestResult::finishedCurrentTestData()
to cleanup after each data row is executed. This function treats calls
to QEXPECT_FAIL without subsequent verification steps as a test failure.
This commit also adds a regression test to demonstrate that expected
failures can no longer be carried over from one data row to another. If
run against the previous version of testlib, the new test would report a
pass instead of an error.
Change-Id: Ida5c7f080815b0dca9531131fed582b0918334cb
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Currently when tests are crashing, aborting or hanging, their status are
reported with a status "Unknown" because SaveCoverageTool is never called
for them. The status of the test given by the coverage tool should be the
same as the one reported in the log output when the test is run. This change
will allow more accuracy in the status reported.
Add QTestResult::setCurrentAppname and QTestResult::currentAppname.
To retrieve the name of the current application running.
Task-number: QTQAINFRA-460
Change-Id: Icc476dc2d6cb28185e5447f1e79da6a8a31cad54
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
The problem in qmltest has now been fixed by adding a dummy column, so this
should be good to re-add. See qtdeclarative change id
I1bfed663fa7b58d71e0cf898db6bd8ae61dae46b.
This reverts commit 963958b4f62513cbc76fb06920de80f7f906f523.
Task-number: QTBUG-23845
Change-Id: I4b6755d3ea97a4624480cc008d280fc8e33743a6
Reviewed-by: Jonas Gastal <jgastal@profusion.mobi>
Reviewed-by: Yunqiao Yin <charles.yin@nokia.com>
As in the past, to avoid rewriting various autotests that contain
line-number information, an extra blank line has been inserted at the
end of the license text to ensure that this commit does not change the
total number of lines in the license header.
Change-Id: I311e001373776812699d6efc045b5f742890c689
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The removed function was just a wrapper around QTestLog::ignoreMessage()
and was only called twice (once in qtestcase.cpp and once in
qtdeclarative). Better to just call the desired function directly and
avoid the indirection.
Change-Id: Ib900f35f5d584e8f323cef9b0e7b4f8a507c199e
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This seems to break a number of tests using the qmltest library.
Functionality will be restored in a future commit, once the breakage has been
identified.
This reverts commit fdedb49b76.
Task-number: QTBUG-23845
Change-Id: I0ca4dbd54132e4250702f06738edbad5e8fc4f36
Reviewed-by: Alan Alpert <alan.alpert@nokia.com>
Previously, the assertation triggered was rather unhelpful:
QFATAL : tst_QHash::qhash_qt4() ASSERT: "d->dataCount <
d->parent->elementCount()" in file qtestdata.cpp, line 88"
We now try a bit harder to be user-friendly.
Change-Id: I2e3a5ae27914d44fc1dc89af2a084e3d798fe221
Reviewed-by: Jonas Gastal <jgastal@profusion.mobi>
Reviewed-by: Giuseppe D'Angelo <dangelog@gmail.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
This change will enable further simplification of QTestResult and
QTestLog in the future, including removing a circular dependency between
the two classes.
The "getter" functions in QTestResult are retained for now, but will be
removed in a future commit, once QtQuickTest has been changed to call
the getters that this commit adds to QTestLog.
This commit is adapted from an unfinished change by Henrik Hartz.
Change-Id: Ife7f80ac6a4310449a4712e96e0bea6c02139a5a
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Introduce QTRY_VERIFY_WITH_TIMEOUT and QTRY_COMPARE_WITH_TIMEOUT
to be able to specify a timeout value.
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Change-Id: Iaeaa4938eb14f2c431537055f626510cba183ce3
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
- Scan the parent directory of the executable if it is located
in a 'Debug' or 'Release' directory
- Report with native separators in the log.
- Use the QString::arg() overloads with several string parameters.
Change-Id: I5ea84411e12978f8f958a0bce3ae10da44cc4e3f
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Use QTestLog::info() rather than qDebug() to output informational
messages from testlib. Source file and line are deliberately omitted as
they would come from testlib rather than from a test program.
Change-Id: I7b479bba4d3d553c6fa846d8d5ea2c29a8ef42b8
Reviewed-by: Robin Burchell <robin+qt@viroteck.net>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Previously, if the signal dumper was switched on with the -vs
command-line switch, it would never be switched off again.
Change-Id: I192e188010471525723fad0844ff33e9482128ea
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This was an internal function that used to act like qsnprintf() but also
filtered unprintable characters out of the test output. The filtering
has been moved somewhere more appropriate and this function is no longer
used by testlib.
Unfortunately, the function was exposed in the public API due to its
former use in the implementation of a public macro.
In the unlikely event that any code outside testlib calls this function,
the call should be replaced by calling qsnprintf(), which comes from the
QtCore/QByteArray header.
Change-Id: Iddc17b4361d16ebddd19346ae7d1064951dd7738
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Make it clear that the delay is applied before each key-click is
simulated.
Change-Id: Id100f1f2db1a5b1651c3046905719d7eb06ec1a0
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Update the docs to describe the behaviour that resulted from moving the
traditional Qt widgets into a separate library from the rest of GUI
classes.
Change-Id: Ibd0ef05cc871b8f5a6700e421aa41bdf64c1210b
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The documentation stated that QStringList objects would be compared
starting from the end of the lists. The implementation in qtest.h
actually starts at the beginning of the list, and always has done since
qtestlib became part of Qt.
Change-Id: I056f584564d46402ba23fc6a89c801cb5c3c6262
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The overload is used in Qt Creator
(see src/libs/extensionsystem/pluginmanager.cpp).
The use case here is an application whose internal
QObjects can be tested by passing a command line parameter.
For this use case, it is inconvenient to have to allocate
memory and create a char argv[]- array.
This reverts commit ad80d42f8e.
Change-Id: I2a2f91e2840100fd62743f6d03b33005d67b18f8
Reviewed-by: Daniel Teske <daniel.teske@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
This option does nothing -- the only function that checks if the option
has been set is never called. This appears to have been the case ever
since testlib was imported into the Qt repository in October 2005.
Change-Id: I837aa957e2d8bd47c3d1c551f1b94d4374daa25e
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
After the previous commit, QTest::qt_snprintf() is equivalent to
qsnprintf(), so just use that instead.
Change-Id: I89ad6e3749ba5efb1926b0b618a904e8baca9f52
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Previously, unprintable characters were filtered out of test output
while the output strings were being formatted by either qt_snprintf() or
qt_asprintf(). Any strings not formatted by one of those functions
weren't filtered at all, and any strings passed more than once would be
filtered more than once.
This commit separates the filtering of output strings from their
formatting, leaving the filtering until just before the strings are
written to the output stream. For now, the filtering is done by a
protected method of QAbstractTestLogger, but this could easily be
changed to a virtual method in future to allow different filtering
for loggers with different output character sets.
Change-Id: Ia4bb49cd10d37c84af75d2cf58325d27f0e16d99
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The overload of QTest::qExec() that takes a QStringList is not used
anywhere in Qt's autotests, despite having been in the qtestlib API
since Qt 4.4.
This lack of use most likely derives from the fact that none of the
QTEST_MAIN macros use the overload, and more than 99% of Qt's tests
use those macros to avoid explicitly calling QTest::qExec().
Change-Id: I264b21d7fe1a9f2d565f748cf8bbe32414a73bb0
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Previously, autotests silently ignored the -graphicssystem and -qws
parameters so that test-runners did not need to include logic to decide
whether to pass those parameters depending on whether a particular test
was a GUI test or a non-GUI test.
In Qt 5, both of those parameters have been removed from QApplication,
so tests don't need to ignore them anymore as they should no longer be
supplied to any test.
Change-Id: Ic59e38ddfa3303e760645ed1782a211340ce70f2
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Update the documentation to make it clear that regular test applications
should not call QTest::qExec() more than once. Also minor rewording of
description of return value.
Change-Id: I45bdf520ed10fd3c9232847a0ec0bc2b32d4caf3
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Automated tests often need to load some data from external files.
Currently, a wide variety of approaches for this have been used in Qt
autotests, including:
- embed the source directory into the test binary at compile time, and
find the testdata relative to that; this fails when the source tree
is no longer available (e.g. when the tests are deployed to a device).
- use a path relative to the current working directory, and trust that
the caller always sets the current working directory such that the
testdata can be found; this fails when the caller uses a different
working directory than expected.
- use a path relative to QCoreApplication::applicationDirPath();
this fails when source tree != build tree (since testdata is not
automatically copied into the build tree).
- compile the files into the binary using the Qt resource system; this
should work, but does not allow for testing of code which genuinely
needs external files.
It seems that there is not a simple method for determining the testdata
path which can be reliably used in all circumstances, so various tests
have reinvented the testdata location method in different ways.
Therefore, this is a good candidate for an addition to the testlib API.
The current implementation of QFINDTESTDATA is able to find testdata
in all three of (build tree, install tree, source tree), in that order.
Change-Id: Ib2fed860723ccf437240da3b00db22dfe1a6b56c
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Currently QTest::newRow function expects to have correct tag string for
each test data. This was enforced by a simple assert. By adding a
an assert with message we might simplify debugging of a test that
generates test cases.
Change-Id: I84ebb397264c99e7bb46d281fd38583dd2d6964c
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
To instrument a Qt application or library with the TestCocoon coverage
tool, do `CONFIG+=testcocoon' in the application .pro file.
To instrument Qt itself with testcocoon, use the `-testcocoon' configure
option.
Change-Id: Ie77109a078d11ea51f7a073621e0df9c752c44ae
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@nokia.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
(Note: This feature is ported from Qt 4.8.
See the following commits:
01575deafb7d26ca2431374e92c6d71de96547c7
4866d1ba8afbab61e102942d1ea93b81fea053d6
)
Passing the -datatags option to a QTestLib program prints the
available data tags to standard output.
For completeness, the test case name is also printed
at the start of each output line. (Although the file name
is supposed to match the lower-case version of the test case
name, this is currently not true in all cases (particularly not
under tests/benchmarks/). Even if there was a script to enforce this
convention, the -datatags option provides this information in a
reliable way.)
Data tags for each test function (f() in this case) are printed in
four different ways depending on the presence of local and global
data tags:
Case 1: No tags:
tst_MyTestCasetst_MyTestCase f
Case 2: Local tags only:
tst_MyTestCase f local tag 1
tst_MyTestCase f local tag 2
...
Case 3: Global tags only:
tst_MyTestCase f __global__ global tag 1
tst_MyTestCase f __global__ global tag 2
...
Case 4: Local and global tags:
tst_MyTestCase f local tag 1 __global__ global tag 1
tst_MyTestCase f local tag 2 __global__ global tag 1
...
tst_MyTestCase f local tag 1 __global__ global tag 2
tst_MyTestCase f local tag 2 __global__ global tag 2
...
...
Note that the string __global__ is assumed to be highly unlikely to occur
in a data tag (if it does, an ambiguity results).
Change-Id: Ib51aa0c3c32ad52e52ce519729292cf8f0ec5d50
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Reviewed-by: Sergio Ahumada <sergio.ahumada@nokia.com>
The standard C getenv() returns NULL if the requested environment
variable is not found.
In Qt4 and later, qgetenv() does not return a null pointer if the
requested environment string is not defined. Instead it returns a
QByteArray containing an empty string. If using qgetenv(), there is no
way to tell the difference between an undefined environment variable
and one which is defined to be the empty string.
In testlib, all calls to qgetenv() were checking whether the returned
QByteArray's constData() returned a null pointer, but that would never
happen. These calls must instead check whether the QByteArray contains
a non-empty string.
Change-Id: I342f0e8b196896c26cccce3ff169fa1b9669b5ff
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
When executing a data-driven test, testlib executes the _data function
once, then repeatedly executes init(), then the test function, then
cleanup() for each row of test data.
Change-Id: Icfa1dd19a52fb1debbc92b7cbe13d85bfb0418c7
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
When the SkipAll mode is used, tests only report a SKIP for the first
line of test data and subsequent lines are not reported at all. This
behaviour makes it impossible for anything post-processing test results
to accurately report test pass- and run- rates because they cannot see
how many lines of test data were skipped.
This commit removes SkipMode. QSKIPs in regular test functions and data
functions are treated the same as SkipSingle, so that every skipped line
of local or global test data is reported in the test log. QSKIPs
elsewhere are treated the same as SkipAll -- skipping in init() causes
the next test function to be skipped entirely, and skipping in
initTestCase() or initTestCase_data() causes all test functions to be
skipped.
This commit only changes qtestlib and the selftests. A further commit
will change the autotests to remove the SkipMode parameter from QSKIP
calls.
Note that the change in expected output for the globaldata selftest is
deliberate, as the QSKIP in the skipLocal test function has effectively
changed from SkipAll to SkipSingle.
Task-number: QTBUG-21851, QTBUG-21652
Change-Id: I7b1c53fe7ca9dde032810b789d967e2a402bbe5d
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Alex <alex.blasche@nokia.com>
These functions have lived in tests/shared/util.h for a long time, but
they really belong in qtestlib.
Change-Id: I60d569d002dea220b51563931d8b7aa77a20b98b
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This macro is no longer used in Qt's tests and encourages writing tests
in a way that makes test reporting less accurate -- remove it to prevent
further misuse. If a test can be determined at compile-time to be
inapplicable, it should be omitted from the build via .pro file logic.
If that is not possible (e.g. there is no suitable qmake variable), the
test's initTestCase() function should call QSKIP to skip the entire test
with a meaningful explanation.
Task-number: QTBUG-21851
Change-Id: Icacc8c5567a700191b6ef3fa94ee52ede94c5b34
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Remove the usage of Q_MAC_USE_COCOA and Carbon code paths.
Change-Id: Ib569ad8c6d9ffe258f454b3c3b06e95294a10112
Reviewed-on: http://codereview.qt-project.org/5100
Reviewed-by: Morten Johan Sørvig <morten.sorvig@nokia.com>
Sanity-Review: Morten Johan Sørvig <morten.sorvig@nokia.com>
This change makes QTest::qWarn() consistent with similar testlib
functions, which take __FILE__ and __LINE__.
Change-Id: I12977cb30672899ab38877b4a656f0cc0c7ea02c
Reviewed-on: http://codereview.qt-project.org/3559
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Add support for test cases which require an event loop, but do not need
GUI up and running. Such cases are best accommodated by code similar to
QTEST_MAIN, but using QCoreApplication instead of QApplication.
Change-Id: I04ce82d26b80a4edeba6bf7604a7f8d974232a11
Merge-request: 919
Reviewed-on: http://codereview.qt-project.org/5427
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Each destination and the format of output to write there is specified by
adding "-o filename,format" to the command-line. The special filename
"-" indicates that the log output is written to the standard output
stream, though standard output can be used as a destination at most
once.
The old-style testlib output options are still supported, but can only
be used to specify one logging destination, as before.
If no logging options are given on the command-line, a plain text log
will go to the console, as before.
To log to the console in plain text and to the file "test_output" in
xunit format, one would invoke a test in the following way:
tst_foo -o test_output,xunitxml -o -,txt
This commit also enhances the selftests to test with multiple loggers,
but negative tests (e.g. bad combinations of command-line options) are
left for future task QTBUG-21567.
Task-number: QTBUG-20615
Change-Id: If91e752bc7001657e15e427aba9d25ab0a29a0b0
Reviewed-on: http://codereview.qt-project.org/4125
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Pass the filename when creating the logger rather than when commencing
output. This will simplify forthcoming changes for running multiple
loggers.
Change-Id: I563aa97661caf279a5dcccb1321ff3ce4725b332
Reviewed-on: http://codereview.qt.nokia.com/4116
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Both the old and new XML loggers produce the Complete and Light XML log
formats, while only the new logger produces Xunit XML logs. The
disadvantage of the new logger is that it is more complex and doesn't
produce a partial log if the test fails to terminate gracefully. This
behaviour arises because Xunit format output cannot be written correctly
until all tests have been executed.
This commit removes the Complete and Light XML formats from the new
logger, using the old logger to produce those formats and the new logger
to produce only Xunit XML. Prior to this commit, the qtestlib selftests
demonstrate that the old and new loggers produce identical output for
Complete and Light XML.
This commit also removes the undocumented -flush command-line option,
which was used rather obscurely to select between the old and new
loggers.
The newer logger will be renamed to QXunitTestLogger in a subsequent
commit.
Change-Id: Id304f5b411bdd520409ee233f6bc34e8917942ab
Reviewed-on: http://codereview.qt.nokia.com/3923
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Remove the undocumented feature that allows test functions to be
executed in random order. The feature was designed to expose unintended
dependencies between test functions -- test functions are only supposed
to depend on the initTestCase() and init() functions.
Aside from the lack of documentation, there are a number of problems
with this feature. Most importantly, running the tests in random order
has only a 50% chance of exposing dependencies between test functions.
A better strategy would be to run the test functions in reverse order
and complain if that produces different results to running the tests in
the normal order.
Additionally, the random order is not deterministic, so even if a
dependency is exposed during a test run, there's no guarantee that it
will be exposed again. The feature allows the user to optionally
supply a random seed to make the "random" order deterministic, but as
rand() implementations are not identical across platforms, even that
does not guarantee that dependencies between test functions will be
exposed deterministically.
Change-Id: I39eac34c532ccb988116778bbc5ab05d835874c5
Reviewed-on: http://codereview.qt.nokia.com/3720
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Tests that are expected to return a non-zero exitcode should be marked
with "CONFIG+=insignificant_test" in their .pro file.
Change-Id: Iebb9c7129c08833ed517115f569086d6fcfe827b
Reviewed-on: http://codereview.qt.nokia.com/3689
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The charting functionality was removed in Qt 4.6, though the
command-line option was retained to print a warning that the feature had
been removed. Sufficient time has passed for this to be removed
completely.
Change-Id: I2adf2818c6a6e57e765104de97b28dbf6914e3fa
Reviewed-on: http://codereview.qt.nokia.com/3638
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The commit changes printf's that output error and warning messages to
send their text to the stderr stream. Non-error output, such as that
produced by passing the -help option to a test, still goes to stdout.
Change-Id: Iea4d62451e3e7e84c654859cb09ea7e717511d13
Reviewed-on: http://codereview.qt.nokia.com/3636
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The mode parameter is never used by the logging code, so there's little
value in passing it there.
Change-Id: Ibe2cbe5eaf457a7e3ffd3aea3a4be7c8278c91b6
Reviewed-on: http://codereview.qt.nokia.com/3547
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This is the beginning of revision history for this module. If you
want to look at revision history older than this, please refer to the
Qt Git wiki for how to use Git history grafting. At the time of
writing, this wiki is located here:
http://qt.gitorious.org/qt/pages/GitIntroductionWithQt
If you have already performed the grafting and you don't see any
history beyond this commit, try running "git log" with the "--follow"
argument.
Branched from the monolithic repo, Qt master branch, at commit
896db169ea224deb96c59ce8af800d019de63f12
Simon Hausmann
Stephan Binner
Lars Knoll
Tor Arne Vestbø
Rainer Keller
Thiago Macieira
Friedemann Kleint
Jani Heikkinen
Marc Mutz
Kai Koehne
ala
Frederik Gladhorn
Eskil Abrahamsen Blomfeldt
Maurice Kalinowski
Morten Johan Sørvig
Matti Paaso
Frederik Gladhorn
Lars Knoll
Sérgio Martins
Keith Gardner
Eskil Abrahamsen Blomfeldt
Giuseppe D'Angelo
Ulf Hermann
Friedemann Kleint
Sergio Ahumada
Jason McDonald
Dmitry Ashkadov
Maurice Kalinowski
Jerome Pasion
Andrew Knight
Jake Petroules
Mirko Boehm (AWS)
Jon Severinsson
Sze Howe Koh
David Faure
Sergio Ahumada
Christian Stenger
Christian Strømme
Eskil Abrahamsen Blomfeldt
Liang Qi
Leena Miettinen
Kai Koehne
Iikka Eklund
Jeremy Lainé
Stephen Kelly
Jeremy Katz
David Faure
Friedemann Kleint
Girish Ramakrishnan
Konstantin Ritt
Casper van Donderen
Laszlo Agocs
Kent Hansen
Jason McDonald
Charles Yin
Lars Knoll
João Abecasis
Caroline Chao
Jonathan Liu
Andreas Holzammer
Robin Burchell
Rohan McGovern
Jędrzej Nowacki
Jo Asplin
Morten Sorvig
Jan Kundrát
Jyri Tahtela
Qt by Nokia