Currently, if the cmake program path passed to androidtestrunner
contains space, QProcess will get an invalid program path. To fix
that, quote the cmake command passed to androidtestrunner and use
QProcess::splitCommand() to get the program and args list.
Task-number: QTBUG-105524
Pick-to: 6.7
Change-Id: Icb665033884127972bff205157ff22aa469c135c
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* Don't add \n when using qDebug()
* Trim testAppArgs of extra space at the end.
* Don't print get-current-user command by default, that value
is not that useful to be always printed.
* Update outdated bit of the help printout to reflect what the
test runner does.
Pick-to: 6.7
Change-Id: I998bafc3aef524b368654f7b35a2a8e084d17542
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
This contains test specific variables like pid, sdkversion,
isPackageInstalled and isTestRunnerInterrupted which don't strictly
fall under 'Options'.
Pick-to: 6.7
Change-Id: I7c4a3422813f464b407d9b2be98c5b4a807c4f1e
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Leave a log message to know when the test runner timed out or the test
finished.
Pick-to: 6.7 6.6 6.5
Change-Id: If56ecaa5b0e3af22b3e26480a584e2d52ac97553
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If the test runner was interrupted, stop waiting for the test
to start or to finish, go fetch the available test results
and logcat and uninstall the test app.
Also, set CMake TIMEOUT_SIGNAL_NAME to SIGINT and
TIMEOUT_SIGNAL_GRACE_PERIOD to 10 seconds to allow enough
time to fetch the logs and uninstall the test app.
Task-number: QTBUG-106479
Pick-to: 6.7 6.6 6.5
Change-Id: I4820cfe58f05d15179b4af819caa92e475881634
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
First, replace double quote characters with 3 escaped double quotes
to allow QProcess::splitCommand() to treat it as an actual quote
character that's part of the tag.
Then, escape single quote characters so they don't interfere with
the shell command and also to be treated as part of the argument.
Lastly, surround the args with escaped double quote so that args
with spaces are also treated as one.
Amends b044323c16.
Example of this:
tst_qkeyevent::modifiers("M","e","t","a") for double quotes
tst_qunicodetools::wordBreakClass(two words) for spaces
tst_QSpinBox::stepSelectAll("don't select all") for single quotes
Task-number: QTQAINFRA-5703
Change-Id: Ie4317e4350bbac619bac41e41f42613f50cf1ad4
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Cleanup the includes, and remove unused include dir in CMakeLists.txt.
Task-number: QTQAINFRA-5928
Change-Id: I477024291042042765b666bda882bab4a5f8313f
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Using QProcess would make the test runner more robust when dealing
with quoted arguments since it won't be using the system shell and
handles quoting under the hood.
Fixes: QTBUG-105524
Fixes: QTQAINFRA-5703
Change-Id: Ib666ffea33302f1dfc7e8972bd7750f14065c4fc
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
This has two parts, under testrunner, make sure to pass test args
that has quotes as \\\" so that the Android app gets a \" after parsing
the intent extras. The app args are then is passed to C++ where
QProcess::splitCommand() is used to parse the args string into a list,
and to preserve the quotes in this case, the former call expects triple
quotes to to get the quote character preserved in the split list.
Fixes: QTQAINFRA-5703
Change-Id: Iad81cadf24a6def21ef1536c882ecd510c4426cf
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Pass a CMake test TIMEOUT argument to androidtestrunner, using
a value of 95% of that timeout to allow time for the test runner
to do any cleanup before being killed.
If no test argument is provided, use the value from CMake property
DART_TESTING_TIMEOUT or CTEST_TEST_TIMEOUT. If that's not provided
default to 25 minutes which is the default for DART_TESTING_TIMEOUT.
Along the way set the default androidtestrunner timeout to 10 minutes
and fix the wrong timeout in the help menu.
Fixes: QTBUG-106479
Pick-to: 6.6 6.5
Change-Id: I12cd531583dd94954caf8044c37c22382d53d43c
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Check if the common testlib environment variables are set on the
host and pass them to the test app. This include any env variable
starting with "QTEST_".
Fixes: QTBUG-106478
Task-number: QTBUG-106479
Pick-to: 6.6 6.5
Change-Id: I99e1b314b106cda20a66e3cac9a92b463b94f5c9
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Currently, if the test runner is interrupted by SIGINT and SIGTERM,
the semaphore is never released and thus consecutive test runner
executions will wait forever. To fix that, a signal handler for the
former signals is added to release the QSystemSemaphore. Also,
remove RunnerLocker and simply use QSystemSemaphore directly,
and explicitly release the semaphore at the end of the test runner
execution.
Fixes: QTBUG-115298
Pick-to: 6.6 6.5
Change-Id: I4f6cc0e4d837460de9a66248e09d3cbaf24ce959
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Reviewed-by: Zoltan Gera <zoltan.gera@qt.io>
Use ndk-stack [*] tool provided by the Android NDK to pretty
print crash stacktraces to contain <source-file>:<line-number>
lines for calls, making debugging crashes much easier.
The ndk-stack path can be provided with the new param --ndk-stack,
otherwise, it would be deduced using ANDROID_NDK_ROOT env var.
If the tool is not found or the unstripped libs path cannot be
obtained, the default crash report is printed instead.
[*] https://developer.android.com/ndk/guides/ndk-stack
Task-number: QTQAINFRA-5928
Pick-to: 6.6 6.5
Change-Id: I22c3ba131a44050c8fcbfd176d5ced096761d229
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Currently, under waitToFinish(), we wait for the app to start and if
it doesn't start or starts and exits too quickly for the test runner
to catch it, it fails. The test runner tries to get the pid of the test
after it detects the app has started. However, we don't really need to
fail, the test runner could simply continue the execution and assume
the test was run and finished, and proceed to fetching the test results.
Since, the results fetching don't anyway rely on the pid, the test
runner can let that operation decide whether to fail or not (if not
output is found).
Also, along the way, instead of issuing a different command to get the
pid (i.e. adb shell pidof), we can use the same "adb shell ps" command
that is used to check if the app is running, to obtain the pid.
Fixes: QTQAINFRA-5928
Fixes: QTBUG-88508
Pick-to: 6.6 6.5
Change-Id: Ice945fcb686c4ef21b5f1c143aa22922ae928333
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
For better debugging for test failures, print both logcat and crash
stacktrace (since the start of the current test only), to have an
immediate idea of the reason of the failure or crash if any.
The crash report and the logcat for Android before level 23, is fetched
from the device's time of the app start since those two cases don't
have an option for filtering by pid.
Task-number: QTQAINFRA-5928
Fixes: QTBUG-114898
Pick-to: 6.6 6.5
Change-Id: I7760985032f342da4165cdb7573d4cfe5369ae03
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
To make debugging easier, print a log for each step in the test
output retrieval process instead of one error message that cover
multiple potential error cases.
Task-number: QTQAINFRA-5928
Pick-to: 6.6 6.5
Change-Id: Ia68c15e34cf4b0f8e2b0d351af785bb753807721
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Some of our tools don't have the `-h`, or `-v` flag, or it could be
that the `-v` flag also prints the entire `--help` as well, e.g.,
`androiddeployqt`. When running in Jenkins, this may lead to a message
box being shown and consequently stopping the build. By customizing the
flag per tool, and limiting the TRY_RUN to tools that support `-v` or
`--version`, we can avoid this.
Also removed TRY_RUN from `macdeployqt` which doesn't need it anyway.
Amend 41b32cd2c4
Pick-to: 6.5.2 6.5 6.6
Fixes: QTBUG-114530
Change-Id: I78e3344d2553c0050c285ae86f2310bd373c6c57
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
On Windows, we will try to run our host tools after a successful build.
If the build fails because of a missing DLL, we will be able to throw an
error with some direction on what might be the cause, and how to resolve
it.
Pick-to: 6.5 6.6
Fixes: QTBUG-113273
Change-Id: Iba548829bc41fbee95cef288faaf7edca118ee33
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
And deprecate the non-native key support API. Qt 7 may not even store
the old, non-native Qt.
Documentation in a new commit, when the dust settles.
Change-Id: I12a088d1ae424825abd3fffd171d2b549eeed040
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
We already have:
a, a timeout as part of QtTest. By default it lets each test function
run for 5 minutes. This timeout can be configured using
QTEST_FUNCTION_TIMEOUT.
b, maxTimeBetweenOutput in the CI. The CI will kill the process if too
much time passes between individual output lines of a test.
c, maxTimeInSeconds in the CI. This does exactly the same as the
androidtestrunner timeout.
The CI timeouts can be centrally tuned per platform and Qt module. This
is preferable over a special timeout just for android.
As other people may be using androidtestrunner for unrelated projects,
don't delete the timeout, but simply disable it from CMake by setting
it to -1.
Task-number: QTBUG-106479
Task-number: QTBUG-101596
Task-number: QTBUG-100242
Change-Id: If4ce00948e204182bb12ac4859d3b0dd193de7ad
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Currently, arguments passed to the app through applicationArguments
extra bundle treat every space as an argument separator. This then
doesn't handle the case where an argument is a space separated quoted
multi-word. This is more apparent when androidtestrunner is passing
test arguments to the app where an argument can be a test case with
a data tag that contains a space, which then is treated as two separate
tag names.
This change makes sure that androidtestrunner quotes each argument,
and the app doesn't split the arguments list by spaces, but rather
passed the argument string directly to c++ where
QProcess::splitCommand() is used to get the correct set of arguments
that will be passed to main().
Pick-to: 6.4 6.3 6.2
Task-number: QTBUG-104730
Change-Id: I45d8ca979d90f2a383c84623f0eb2eec29bba727
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.
Existing copyright statements remain intact
Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
It doesn't use any Gui classes and thus shouldn't depend on Gui.
Also change PUBLIC_LIBRARIES to LIBRARIES, executables don't need to
propagate library dependencies.
Pick-to: 6.2 6.3 6.4
Change-Id: I9edae7e555e1d74d63b00afbf9e3931963b502c2
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Move shellquote helper functions into a common place instead of having a
copy in each tool's code.
Change-Id: I9723c11f894a211864788a7635773610c0fde739
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Required for porting away from QLatin1Char/QLatin1String in scope of
QTBUG-98434.
As a drive-by, fix qsizetype -> int narrowing conversion warnings for
the touched lines.
Change-Id: Iebcbdbd7cecac09d0a7039e3ef6a4509d33039ba
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Make sure androidtestrunner reports the following cases as failures:
* XPASS test case
* The test output doesn't have closing block.
* The test result read from output.txt is empty
Pick-to: 6.2 6.3
Fixes: QTBUG-100312
Task-number: QTBUG-100470
Change-Id: Ifbb3c31fa22159df0c39b264e95e2f1ccd16e5ff
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
qmake had a feature to set androidtestrunner's timeout. This
feature is not yet ported to CMake. So with CMake, we always
have androidtestrunner's default timeout. Which was
five minutes. We have several tests which take just shy of five
minutes and are thus flaky.
This patch extends the timeout to eight minutes. This should be
a temporary workaround until the timeout feature is implemented
in CMake.
Task-number: QTBUG-101596
Pick-to: 6.2 6.3
Change-Id: Ia67ed6ab40a40f1b82861bd15bd04528ee9334ee
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Previously, when running unit tests on Android, the log presented
to the user was the log as known to the Qt logging system. This
does not include log messages generated by Java code using e.g.
Log.d("message"). Neither does it include log messages by system
libs. This patch changes androidtestrunner to capture the full
logcat output for the unit test. This improves the developer
experience when working with unit tests.
Fixes: QTBUG-93438
Change-Id: I580f728349041eb8a84a32d187754b7b5448f512
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
It happens on Android Automotive emulator, that output file is in other
directory.
According android spec [1]:
"If a default user isn't specified, each adb subcommand has a different
user. The best practice is to retrieve the user ID with am
get-current-user and then explicitly use --user <userId> for any
command that supports it."
That is the reason why output file can be found in
/data/user/USER_ID/PACKAGE_NAME directory.
Checking path related to current user was added as backup solution.
[1]https://source.android.com/devices/tech/admin/multi-user-testing
Pick-to: 6.2
Change-Id: Id7e6ddef74f4f20b7469a07bba6a71d3622c4e20
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Introduce a new macro qt_internal_return_unless_building_tools which
simply calls return() if tools are not built. This macro is supposed to
be called after qt_internal_add_tool().
Using this macro avoids having to special-case code for when
qt_internal_add_tool() creates imported targets in cross-builds.
Adjust pro2cmake accordingly.
Task-number: QTBUG-85084
Change-Id: I9e1c455c29535dd8c318efa890ebd739c42effc1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
When doing a cross build for Android we need only the host tools which
are available under cmake's host_tools target, but androiddeployqt and
androidtestrunner are not part of that. This fixes that.
Pick-to: 6.2
Change-Id: Icdbc4a78ca050b66ec8df656d9ec766ef6c9f4b9
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Some Qt modules/tests go through features that require run-time
Android permissions which are granted by user input, that of
course is not ideally possible while running unit tests. So when
installing the app adb can do that for us.
Pick-to: 6.2
Change-Id: I92d33d5213c3779d7ca246ec0fb359d2ead4fa6f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The format was renamed in 27db9e458c.
Pick-to: 6.2
Change-Id: I53975c7467d8768dc9dc9ac2d89c42eefa12e22f
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This reverts commit 1365a1c7a7.
Reason for revert: Causes the issue in QTBUG-94445.
Pick-to: 6.2
Task-number: QTBUG-88508
Fixes: QTBUG-94445
Change-Id: I131234c9b1daeecd5f5f4921a9150d65a26db972
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Add new line between each argument type in the help print to make it
easier to read and not a big wall of text.
Change-Id: I7c7b2d2b59ef3a0889d1bb8c7af3d00b82bf820d
Reviewed-by: Andy Shaw <andy.shaw@qt.io>
Before, building ${target}_make_apk always re-built the apk, instead of
rebuilding the apk only when inputs changed. This patch fixes that by
moving the creation code from a custom target to a custom command with
proper dependencies.
The androidtestrunner tool now does not check for the existence of an
apk anymore and always runs the make command that is supposed to build
the apk.
The ${target}_prepare_apk_dir target is not needed anymore by the Qt
build but is still used by Qt Creator's Android support. Add a
clarifying comment.
Fixes: QTBUG-93431
Change-Id: I00d65d616fef9511b03b65f879c4bc6cb92dfc30
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Remove the qmake project files for most of Qt.
Leave the qmake project files for examples, because we still test those
in the CI to ensure qmake does not regress.
Also leave the qmake project files for utils and other minor parts that
lack CMake project files.
Task-number: QTBUG-88742
Change-Id: I6cdf059e6204816f617f9624f3ea9822703f73cc
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
Modify special case locations to use the new API as well.
Clean up some stale .prev files that are not needed anymore.
Clean up some project files that are not used anymore.
Task-number: QTBUG-86815
Change-Id: I9947da921f98686023c6bb053dfcc101851276b5
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
make apk takes a LOT of time, showing that it's not hanged it's useful.
Change-Id: I4fb11e0f535cf2311d71dff3710135b465ecc06b
Reviewed-by: Andy Shaw <andy.shaw@qt.io>
First and foremost the condition was wrong ...
In order to help running qml test cases on Android without any android
specific hack on
user's .pro files we copy the entire project folder to assets. I copy the
entire folder
and not only the tst_*.qml files because it might contain data which is
needed by the tests to run.
Change-Id: I06323d9d52904317410dd2f440de65a0766a48b5
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Assam Boudjelthia
Alexandru Croitor
Amir Masoud Abdol
Friedemann Kleint
Thiago Macieira
Ulf Hermann
Lucie Gérard
Sona Kurazyan
Andreas Buhr
Bartlomiej Moskal
Joerg Bornemann
Tor Arne Vestbø
BogDan Vatra