The newer test logger did not produce the same light XML output as the
logger that it replaced. In particular, it did not output the <DataTag>
tag and it incorrectly nested a <Message> tag inside the <Incident> tag
when a fatal error occured in a test.
Unfortunately, it appears that the expected lightxml output for the
selftests was produced by running tests using the newer logger, while
the selftests did not use the older lightxml logger. Thus the errors
were not detected by the selftests.
This commit adds the older lightxml logger to the selftests, updates the
expected test data accordingly, and modifies the newer lightxml logger
to behave correctly. This last item is achieved by making the lightxml
streamer copy most of the code from the xml streamer -- lightxml output
is supposed to be same as xml, except for the omission of the root and
<TestCase> tags.
Change-Id: Ie6e1f69dd6000df2b9d0c5c8e2109fe7bbff3956
Reviewed-on: http://codereview.qt.nokia.com/3902
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
For XML logs, an Incident tag with no sub-tags has a space before the
closing slash. For Light XML logs, this space was missing unless the
-flush command line option was supplied.
Change-Id: I6de2e140f9309f333b5b97ed7f8116e4e727a149
Reviewed-on: http://codereview.qt.nokia.com/3871
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Remove the undocumented feature that allows test functions to be
executed in random order. The feature was designed to expose unintended
dependencies between test functions -- test functions are only supposed
to depend on the initTestCase() and init() functions.
Aside from the lack of documentation, there are a number of problems
with this feature. Most importantly, running the tests in random order
has only a 50% chance of exposing dependencies between test functions.
A better strategy would be to run the test functions in reverse order
and complain if that produces different results to running the tests in
the normal order.
Additionally, the random order is not deterministic, so even if a
dependency is exposed during a test run, there's no guarantee that it
will be exposed again. The feature allows the user to optionally
supply a random seed to make the "random" order deterministic, but as
rand() implementations are not identical across platforms, even that
does not guarantee that dependencies between test functions will be
exposed deterministically.
Change-Id: I39eac34c532ccb988116778bbc5ab05d835874c5
Reviewed-on: http://codereview.qt.nokia.com/3720
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Tests that are expected to return a non-zero exitcode should be marked
with "CONFIG+=insignificant_test" in their .pro file.
Change-Id: Iebb9c7129c08833ed517115f569086d6fcfe827b
Reviewed-on: http://codereview.qt.nokia.com/3689
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The constructor asserts if either parameter is null, so having defaults
seems to be pointless.
Change-Id: I8cec52e17e5f94458e8d8323855eaed6433686e7
Reviewed-on: http://codereview.qt.nokia.com/3644
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
QTEST_EMBED is not defined in Qt itself, nor is it defined by any of the
CI builds for Qt's supported platforms.
Change-Id: I73a3979630130fc8f1ef99dcbc17b4d1875ba246
Reviewed-on: http://codereview.qt.nokia.com/3641
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The charting functionality was removed in Qt 4.6, though the
command-line option was retained to print a warning that the feature had
been removed. Sufficient time has passed for this to be removed
completely.
Change-Id: I2adf2818c6a6e57e765104de97b28dbf6914e3fa
Reviewed-on: http://codereview.qt.nokia.com/3638
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The commit changes printf's that output error and warning messages to
send their text to the stderr stream. Non-error output, such as that
produced by passing the -help option to a test, still goes to stdout.
Change-Id: Iea4d62451e3e7e84c654859cb09ea7e717511d13
Reviewed-on: http://codereview.qt.nokia.com/3636
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
QTestLogger was counting various test outputs, but was only using three
of the counters internally and the rest were not used at all. This
commit removes the unused counters and all of the getter methods.
Change-Id: I447183dcaf3e6cc335bbf58656e25b1d32ba810a
Reviewed-on: http://codereview.qt.nokia.com/3437
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The setLogFormat() and logFormat() methods are never called -- the log
format is set in the call to the constructor and it would not make sense
to change it during a test run.
Change-Id: I59256f17f28bbc72d86cabfb2a961d2faf0e2d52
Reviewed-on: http://codereview.qt.nokia.com/3435
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
QTestLog is an entirely static class and its constructor and destructor
are declared private to prevent accidental construction of an instance
of the class. Therefore, the constructor and destructor do not need to
be defined.
Change-Id: I860f1344c5032091f5c641a20e1656bb52a6f07e
Reviewed-on: http://codereview.qt.nokia.com/3466
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
For both QTestLog::addFail() and QTestLog::addIgnoreMessage(), passing
a null message does not make sense and is therefore an error that
should be brought to the developer's attention.
Change-Id: Ib09ad90b70d74f7432c08708db8a70dee008cce4
Reviewed-on: http://codereview.qt.nokia.com/3470
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The mode parameter is never used by the logging code, so there's little
value in passing it there.
Change-Id: Ibe2cbe5eaf457a7e3ffd3aea3a4be7c8278c91b6
Reviewed-on: http://codereview.qt.nokia.com/3547
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This class is not very useful -- it just creates a text file that
records each occasion that an autotest enters a test function as a
stop-gap solution for the fact that the newer XML logger produces no
output if the autotest fails to terminate gracefully. Addressing
QTBUG-20615 will provide a better solution by allowing the user to get a
partial plain text test log in those circumstances.
Change-Id: I179bb98dbd696d0734cd3f12046e5c567def30cc
Reviewed-on: http://codereview.qt.nokia.com/3390
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
It doesn't make sense to change the logger associated with a streamer in
the middle of a test run, so only allow the logger to be set when
constructing a streamer.
Change-Id: I02661de2b6071c74d10bc854cbe436581978d2d9
Reviewed-on: http://codereview.qt.nokia.com/3622
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Previously QAbstractTestLogger used a global variable for the file
pointer to which it was writing test output. This effectively meant
that only one instance of this or its derived classes could exist at any
time.
This commit moves the file pointer inside the class, so that multiple
loggers can exist at the same time. This means that the outputString()
method can no longer be static, which in turn means that several
functions used by QPlainTestLogger need to move from the QTest namespace
into the class, and also that QTestBasicStreamer must hold a non-const
pointer to its associated logger instead of a const pointer.
Task-number: QTBUG-20615
Change-Id: If941f1f9399cf20fb93e3e87f3390bceeca1cbfc
Reviewed-on: http://codereview.qt.nokia.com/3576
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This feature was undocumented and only worked if a secret environment
variable was set, the test output was going to a real console, and plain
text test results were being generated. Including code in testlib for
fancy presentation of test results conflicts with testlib's stated goal
of being a lightweight framework where every feature contributes to
finding bugs, so this feature is being removed. If fancy presentation
of test output is required, it should be achieved by post-processing the
test output outside of the test framework.
Change-Id: I872165c4d2c3d2498c3aa039070ecf319e237ca1
Reviewed-on: http://codereview.qt.nokia.com/3432
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Pass the output file name from the QTestLog to the test logger when
commencing logging rather than having the logger call back into the
QTestLog.
Change-Id: Id484635f9fcfca08a66c92f3442887e9473b6f9b
Reviewed-on: http://codereview.qt.nokia.com/3454
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Instead of using the file output functions inherited from
QAbstractTestLogger, QTestLogger relied on QTestBasicStreamer having a
copy of these functions. This commit removes the copied functions from
QTestBasicStreamer and makes it and QTestLogger use the original
functions from QAbstractTestLogger.
Change-Id: Icac1ae9d85cd39efd4c67c79104404dd56766b17
Reviewed-on: http://codereview.qt.nokia.com/3565
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
X11 is currently the only platform with a "real" implementation of
QTest::qWaitForWindowShown. All others merely process events for some
fixed interval, and assume that this is sufficient for the window to be
placed. In practice, the current interval is too short for slower
machines.
Change-Id: I956cd747d0b30577f5072a9a08da88dabc76906c
Reviewed-on: http://codereview.qt.nokia.com/3060
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Sergio Ahumada <sergio.ahumada@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
The actual file defined in `sync.profile' and generated
by 'syncqt' is `qttestversion.h' instead of `qttestlibversion.h'
Change-Id: I90ed2d5b7cd9f98855a878c97afceced01e7e8b0
Reviewed-on: http://codereview.qt.nokia.com/2050
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
cycle_p.h is only needed when HAVE_TICK_COUNTER is defined, and should
not be included otherwise. It is also a 3rd party header which is not
exported as a private header so far, which makes compilation of
dependenty modules fail.
Change-Id: I5c2546eae7d65fc68b8411c20634aca541c327d6
Reviewed-on: http://codereview.qt.nokia.com/465
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Marius Storm-Olsen <marius.storm-olsen@nokia.com>
This enables external modules to also make use of them without having
access to the complete QtBase source code.
Change-Id: I056e45cba6c6798b76670b8d238dadb2d9f9c092
Task: QTBUG-19585
Reviewed-on: http://codereview.qt.nokia.com/234
Reviewed-by: Marius Storm-Olsen <marius.storm-olsen@nokia.com>
QTEST_ACCESSIBILITY was always defined and only used in one autotest.
Code that uses accessibility features should be excluded if Qt was built
without accessibility rather than based on a define in the test
framework.
Change-Id: I3a517a579a51f536a0983b43bd99e86292026552
Reviewed-by: pending
Reviewed-on: http://codereview.qt.nokia.com/129
Reviewed-by: Morten Johan Sørvig <morten.sorvig@nokia.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
QSignalSpy's constructor failed gracefully for some problems with the
parameters, but not for null parameters, for which there was only a
Q_ASSERT. This commit makes the handling of null parameters consistent
with the handling of other errors -- output a meaningful error message
with qWarning() and return, so that isValid() will subsequently return
false.
Change-Id: I7f5677a4c10185e30403ce3e12a022de8c13bc1c
Task-number: QTBUG-14283
Reviewed-by: Rohan McGovern
This is the beginning of revision history for this module. If you
want to look at revision history older than this, please refer to the
Qt Git wiki for how to use Git history grafting. At the time of
writing, this wiki is located here:
http://qt.gitorious.org/qt/pages/GitIntroductionWithQt
If you have already performed the grafting and you don't see any
history beyond this commit, try running "git log" with the "--follow"
argument.
Branched from the monolithic repo, Qt master branch, at commit
896db169ea224deb96c59ce8af800d019de63f12
Jason McDonald
Samuel Rødal
Friedemann Kleint
Gunnar Sletta
Rohan McGovern
Charles Yin
Sergio Ahumada
Jørgen Lind
Eckhart Koppen
Marius Storm-Olsen
axis
Paul Olav Tvete
Jyri Tahtela
Liang Qi