We already have:
a, a timeout as part of QtTest. By default it lets each test function
run for 5 minutes. This timeout can be configured using
QTEST_FUNCTION_TIMEOUT.
b, maxTimeBetweenOutput in the CI. The CI will kill the process if too
much time passes between individual output lines of a test.
c, maxTimeInSeconds in the CI. This does exactly the same as the
androidtestrunner timeout.
The CI timeouts can be centrally tuned per platform and Qt module. This
is preferable over a special timeout just for android.
As other people may be using androidtestrunner for unrelated projects,
don't delete the timeout, but simply disable it from CMake by setting
it to -1.
Task-number: QTBUG-106479
Task-number: QTBUG-101596
Task-number: QTBUG-100242
Change-Id: If4ce00948e204182bb12ac4859d3b0dd193de7ad
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Currently, arguments passed to the app through applicationArguments
extra bundle treat every space as an argument separator. This then
doesn't handle the case where an argument is a space separated quoted
multi-word. This is more apparent when androidtestrunner is passing
test arguments to the app where an argument can be a test case with
a data tag that contains a space, which then is treated as two separate
tag names.
This change makes sure that androidtestrunner quotes each argument,
and the app doesn't split the arguments list by spaces, but rather
passed the argument string directly to c++ where
QProcess::splitCommand() is used to get the correct set of arguments
that will be passed to main().
Pick-to: 6.4 6.3 6.2
Task-number: QTBUG-104730
Change-Id: I45d8ca979d90f2a383c84623f0eb2eec29bba727
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.
Existing copyright statements remain intact
Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
It doesn't use any Gui classes and thus shouldn't depend on Gui.
Also change PUBLIC_LIBRARIES to LIBRARIES, executables don't need to
propagate library dependencies.
Pick-to: 6.2 6.3 6.4
Change-Id: I9edae7e555e1d74d63b00afbf9e3931963b502c2
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Move shellquote helper functions into a common place instead of having a
copy in each tool's code.
Change-Id: I9723c11f894a211864788a7635773610c0fde739
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Required for porting away from QLatin1Char/QLatin1String in scope of
QTBUG-98434.
As a drive-by, fix qsizetype -> int narrowing conversion warnings for
the touched lines.
Change-Id: Iebcbdbd7cecac09d0a7039e3ef6a4509d33039ba
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Make sure androidtestrunner reports the following cases as failures:
* XPASS test case
* The test output doesn't have closing block.
* The test result read from output.txt is empty
Pick-to: 6.2 6.3
Fixes: QTBUG-100312
Task-number: QTBUG-100470
Change-Id: Ifbb3c31fa22159df0c39b264e95e2f1ccd16e5ff
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
qmake had a feature to set androidtestrunner's timeout. This
feature is not yet ported to CMake. So with CMake, we always
have androidtestrunner's default timeout. Which was
five minutes. We have several tests which take just shy of five
minutes and are thus flaky.
This patch extends the timeout to eight minutes. This should be
a temporary workaround until the timeout feature is implemented
in CMake.
Task-number: QTBUG-101596
Pick-to: 6.2 6.3
Change-Id: Ia67ed6ab40a40f1b82861bd15bd04528ee9334ee
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Previously, when running unit tests on Android, the log presented
to the user was the log as known to the Qt logging system. This
does not include log messages generated by Java code using e.g.
Log.d("message"). Neither does it include log messages by system
libs. This patch changes androidtestrunner to capture the full
logcat output for the unit test. This improves the developer
experience when working with unit tests.
Fixes: QTBUG-93438
Change-Id: I580f728349041eb8a84a32d187754b7b5448f512
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
It happens on Android Automotive emulator, that output file is in other
directory.
According android spec [1]:
"If a default user isn't specified, each adb subcommand has a different
user. The best practice is to retrieve the user ID with am
get-current-user and then explicitly use --user <userId> for any
command that supports it."
That is the reason why output file can be found in
/data/user/USER_ID/PACKAGE_NAME directory.
Checking path related to current user was added as backup solution.
[1]https://source.android.com/devices/tech/admin/multi-user-testing
Pick-to: 6.2
Change-Id: Id7e6ddef74f4f20b7469a07bba6a71d3622c4e20
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Introduce a new macro qt_internal_return_unless_building_tools which
simply calls return() if tools are not built. This macro is supposed to
be called after qt_internal_add_tool().
Using this macro avoids having to special-case code for when
qt_internal_add_tool() creates imported targets in cross-builds.
Adjust pro2cmake accordingly.
Task-number: QTBUG-85084
Change-Id: I9e1c455c29535dd8c318efa890ebd739c42effc1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
When doing a cross build for Android we need only the host tools which
are available under cmake's host_tools target, but androiddeployqt and
androidtestrunner are not part of that. This fixes that.
Pick-to: 6.2
Change-Id: Icdbc4a78ca050b66ec8df656d9ec766ef6c9f4b9
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Some Qt modules/tests go through features that require run-time
Android permissions which are granted by user input, that of
course is not ideally possible while running unit tests. So when
installing the app adb can do that for us.
Pick-to: 6.2
Change-Id: I92d33d5213c3779d7ca246ec0fb359d2ead4fa6f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The format was renamed in 27db9e458c.
Pick-to: 6.2
Change-Id: I53975c7467d8768dc9dc9ac2d89c42eefa12e22f
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This reverts commit 1365a1c7a7.
Reason for revert: Causes the issue in QTBUG-94445.
Pick-to: 6.2
Task-number: QTBUG-88508
Fixes: QTBUG-94445
Change-Id: I131234c9b1daeecd5f5f4921a9150d65a26db972
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Add new line between each argument type in the help print to make it
easier to read and not a big wall of text.
Change-Id: I7c7b2d2b59ef3a0889d1bb8c7af3d00b82bf820d
Reviewed-by: Andy Shaw <andy.shaw@qt.io>
Before, building ${target}_make_apk always re-built the apk, instead of
rebuilding the apk only when inputs changed. This patch fixes that by
moving the creation code from a custom target to a custom command with
proper dependencies.
The androidtestrunner tool now does not check for the existence of an
apk anymore and always runs the make command that is supposed to build
the apk.
The ${target}_prepare_apk_dir target is not needed anymore by the Qt
build but is still used by Qt Creator's Android support. Add a
clarifying comment.
Fixes: QTBUG-93431
Change-Id: I00d65d616fef9511b03b65f879c4bc6cb92dfc30
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Remove the qmake project files for most of Qt.
Leave the qmake project files for examples, because we still test those
in the CI to ensure qmake does not regress.
Also leave the qmake project files for utils and other minor parts that
lack CMake project files.
Task-number: QTBUG-88742
Change-Id: I6cdf059e6204816f617f9624f3ea9822703f73cc
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
Modify special case locations to use the new API as well.
Clean up some stale .prev files that are not needed anymore.
Clean up some project files that are not used anymore.
Task-number: QTBUG-86815
Change-Id: I9947da921f98686023c6bb053dfcc101851276b5
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
make apk takes a LOT of time, showing that it's not hanged it's useful.
Change-Id: I4fb11e0f535cf2311d71dff3710135b465ecc06b
Reviewed-by: Andy Shaw <andy.shaw@qt.io>
First and foremost the condition was wrong ...
In order to help running qml test cases on Android without any android
specific hack on
user's .pro files we copy the entire project folder to assets. I copy the
entire folder
and not only the tst_*.qml files because it might contain data which is
needed by the tests to run.
Change-Id: I06323d9d52904317410dd2f440de65a0766a48b5
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Add the above option to android test runner since its not required
when building with Cmake.
Change-Id: Ie5c9b0af29cfa76802723960071b3bd2a37a56a5
Reviewed-by: Qt CMake Build Bot
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
This is really just an optimization, but I suspect the author of the code
assumed that the bitwise &= operator short-cicuits. (It doesn't). So this
patch should then fix the code to have the intended behavior.
We don't have to check for failures in the remaining output files once
we've found one. pullFiles() returns just a bool indicating if one of the
output files has a recorded fail, so checking the remaining output files
after the first found failure is just a waste (and ideally they should
contain the same failure (however, flaky tests might break this ideal)).
Drive-by fix.
Change-Id: I951e500a69198e7ed124be32199ac81bbddb9bf7
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
g_options.buildPath did not have native separators. This caused
INSTALL_ROOT to not have native path separators, which again made
'mingw32-make check' fail to deploy the .apk. (The actual failure was that
mkdir failed because it did not accept paths with forward-slashes, so the
install_target rule failed. For the record, the command of that rule had
the following call to mkdir:
mkdir $(INSTALL_ROOT:@msyshack@%=%)\libs\x86
Change-Id: Id792c36986b52a527546d48aa9f7d9587e7a18d9
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
First, we cannot quote the whole command in double quotes, since the
command itself very often have double quotes inside it.
Secondly, we should use the same that the ssh(1) command does:
Instead of adb shell setprop foo 'a b' we should use
adb shell setprop foo "'a b'" as explained here:
[https://developer.android.com/studio/command-line/adb#shellcommands]
Last hunk in isRunning():
The pipe character got eaten by the shell if we used a single quote (so
the stuff after the pipe (grep) was actually run locally).
Switching to ssh-style quoting fixed that.
Change-Id: I3075cdf8595ac2549cec8019f2cba79f77815e0b
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
androidtestrunner now checks is the apk is build and if it is, it will skip the build phase.
Now we can build the apks in parallel (which takes most of the time) and run them sequentially.
This way running tests on Android is much faster.
Change-Id: I82f34723ac08f7728cc0daab3366e03821335eed
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
Ant is history for so many years, yet we forgot to remove it from
androiddeployqt.
[ChangeLog][Android] Remove ant support from androiddeployqt
Change-Id: I093295e18e8710c565e9d101e21c49b3c62f6322
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
androidtestrunner is a tool needed to run qt tests on Android.
Now you can run tests as simple as you run them on Linux, macOS,
Windows.
"$ make check" it's all you need to run tests on the default android
device.
ANDROID_DEVICE_SERIAL env variable can be used to use a specific android
serial.
Use cases:
$ make -j1 check
-j1 is needed to make sure we don't run multiple tests in parallel.
$ ANDROID_DEVICE_SERIAL="emulator-5554" make check
Run the test on "emulator-5554"
$ make TESTARGS="-- -xml" check
Switch to xml output. All params after -- are passed to test
application.
$ make TESTARGS="-- -o out.xml,xml -o out.txt,txt -o -,tap -vs" check
Create two files out.xml and out.txt in the current folder and print
"tap" format to stdout and enable logging of every signal emission.
[ChangeLog][Android] Make it easy to run Qt tests on Android.
"$ make check" is all it's needed to run a test on an Android device.
Change-Id: I1a7f64b62608f7367b5a6aabf5d6c6e7e50242e6
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
Ulf Hermann
Lucie Gérard
Assam Boudjelthia
Alexandru Croitor
Sona Kurazyan
Andreas Buhr
Bartlomiej Moskal
Joerg Bornemann
Tor Arne Vestbø
BogDan Vatra
Lars Knoll
Leander Beernaert
Friedemann Kleint
Jan Arve Sæther
Daniel Smith