If the VerifySBOM feature is enabled, pass additional options to
CMake configuration:
- runs syntactic verification of the SBOM
- so it converts the generated SBOM files into JSON files (and
installs them). This does additional useful validations.
- runs the NTIA SBOM verifier
- shows some of the content in the SBOM in a more user-friendly table
format via the sbom2doc python app
- runs sbomaudit to show things like packages without a license
expression, doesn't exit with errors if issues are found.
To ensure the tables are wide enough in the log on the CI, we set
an explicit value for COLUMNS env var, which is used by sbom2doc
to determine the table size.
To ensure the sbom2doc and sbomaudit python applications are found,
we supply additional locations where they can be found, via the
env vars that coin python provisioning sets.
We also make sure to pass the found application paths when executing
the python apps, because they might not be in PATH by default.
Task-number: QTBUG-122899
Change-Id: I0baef8b9c949209b15ab304e1e840b4dcdf5a61c
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
(cherry picked from commit 83ff34d1c626759224a95f39bca8337a8390ed52)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
The use case is where blocking test building is wanted but they cannot
be uploaded due to being static or they would simply be unused.
Pick-to: 6.7
Change-Id: I32ad20fea2ec032fb1f1a9b86025b34f192e0ed2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
(cherry picked from commit 6a0c94b62d948d3da0da241cf1e6c15188069169)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
The instructions will only run if a coin platform config has the
'GenerateSBOM' feature.
The -sbom configure option is not sticky across repos,
similar to -headersclean, so that if a user wants to build another
qt repo against a prebuilt qt, it won't try to generate the sbom
automatically.
That's why we append -sbom to all the repos configure args, and not
just the qtbase one.
Task-number: QTBUG-122899
Change-Id: Ice91cbc430826838119ce3085cbd52c0909f2746
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
(cherry picked from commit d940399b3e6e9403f94190ffc71a5f5db68f51f3)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
The last 30 days shows that these configs are stable, so we can
make them significant:
http://testresults.qt.io/grafana/goto/jBYi1JsSR?orgId=1
The tst_qheif::writeImage test has been blacklisted separately.
Change-Id: Ibe7981bce2db1ec90781d3ea33fe7d4541990aea
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
(cherry picked from commit 70ace7ae78d6637c21a646e726caffb36a1968a6)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
By selectively setting the COIN_CTEST_IGNORE_EXIT_CODE we can allow
modules to gradually become significant, without touching qt5.git
Change-Id: Ie5b6e683cd755b1cf46886bb30803a56885a11a6
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This deduplicates some of the coin instruction files, and opens up the
possibility for other instructions to determine whether a test run
is significant or not.
The only module still using the coin_module_test_template_v2 template
is the QtMQTT module, so the behavior of ignore the test run has been
kept.
The static library opt out was a remnant from the cmake porting, and
can be removed.
Change-Id: I0f9fcbbd3a5a18bd9c8e0f545b0f0656e03bf388
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Coin is currently missing support for arm64 signing. Drop the signing
on arm64 for time being to allow packaging flag to be added to export
arm64 builds.
Change-Id: If684fa4d787fccb99ac9a0ebad56ec723faba354
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
When building examples as part of Qt in the CI, it's best to skip
running deployment for all examples to save resources
(space and time).
Add a QT_DEPLOY_MINIMAL_EXAMPLES option similar to our
QT_BUILD_MINIMAL_STATIC_TESTS option, which will set
QT_INTERNAL_SKIP_DEPLOYMENT at the root examples directory scope, to
skip deployment for all examples.
Each example can then opt into the minimal subset by unsetting the
QT_INTERNAL_SKIP_DEPLOYMENT variable before its
qt_internal_add_example call.
Add Coin instructions to enable this option when building our examples
in the CI.
Pick-to: 6.7
Task-number: QTBUG-90820
Task-number: QTBUG-96232
Task-number: QTBUG-102057
Change-Id: I2efcda455b400c27fe1efd1bcf81b133137fa2d1
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
If a qt5.git configuration is marked with the StandaloneTests feature,
tell CMake not to build the examples in-tree, and instead use
qt-internal-configure-examples to build the examples out-of-tree.
This brings a couple of improvements:
- higher chance of building examples as ExternalProjects without
issues
- ability to use deployment api in examples, without installing the
examples into a main install prefix
The new coin instructions files use a copy of the standalone tests
instructions as a base.
Task-number: QTBUG-90820
Task-number: QTBUG-96232
Change-Id: If89e6da0b327a38b9c2738f58aa1b5b5fb9fda37
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
We have certain platforms where we build on one OS version (the
latest) and then test on an older version.
macOS is an example of such a platform.
Set the COIN_CROSS_PLATFORM_VERSION_TEST environment variable to true
for such platforms. They usually have a dependency on the main OS
version and are marked with the TestOnly feature.
This is useful information that can be used by CMake build tests in
order to skip those that do not account for the difference in OS
versions.
Change-Id: I6eeeac98abe6beb00ce431e17589f8e093ed23e4
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Mount on Virtual machine uses NFS version 4 as default,
and it was causing disk I/O errors when trying to access disk.
Forcing the mount use NFS version 3.
Change-Id: Ibc5958d8a7e71c463ffa9c3282344c08b2c5019a
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
... from CMake so that Coin doesn't end up killing the whole VM if aRows
test gets stuck and instead allow androidtestrunner to cleanup and fetch
the logs so we know what happened, and even potentially ending up re-run
the test and succeeding if it was flaky.
Also, since CMake sets the timeout during configuration time, coin needs
to set COIN_COMMAND_OUTPUT_TIMEOUT under the build target where tests
are configured, so this moves the setting of that env var from the test
target to the build target.
Pick-to: 6.7 6.6 6.5
Change-Id: I9883ea1e98c93f79a088067518d09ca8acd5fdfd
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
During tqtc repo submodule updates, the repo names contain the tqtc-
prefix.
But during the final tqtc-qt5.git integration, the submodule names
don't contain the tqtc- prefix.
This causes errors like:
CMake Error: Error processing file: /qt-cmake-private-install.cmake
Use contains_value in the condition instead of equals_value, to ensure
it works for both scenarios. This assumes there will be no repos that
contain 'qtbase' in their name which are not the original qtbase
repo.
Amends 7e334bc7fc4c6c9de379e19076197af5b7fef9ee
picked from commit 6556a943ce8625ce1043a2da64deed187b7ce209 in the
tqtc repo, so that we don't need to do it for every new lts branch.
Change-Id: Ib38c855525c91032d35c235bf33d860204474fe0
Reviewed-by: Toni Saario <toni.saario@qt.io>
Adds VxWorks conditions to match target instructions
and sets env prefix to wrenv.
Change-Id: If9a8ffcdf4b60cc4b45eb2a336737d77d2f8804a
Reviewed-by: Jukka Jokiniva <jukka.jokiniva@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
This feature is to be used when testing Qt in Coin (our continuous
integration platform). The command line option --output-junit is
supported in CMake 3.21 or greater. However we have one particular
configuration that tests with CMake 3.16, so we exclude that one.
We have already configured the individual tests to print detailed
results in their own XML files, using QTest's functionality. But then,
when a qt test process crashes, it leaves behind a half-baked XML
file, or no file at all.
CTest on the contrary knows nothing about the internals of a Qt test,
and only cares about the process exit code. Even if a test crashes, it
catches that and reports it as a failure.
We enable this functionality here to use in addition to what QTest's
XML files provide, in order to collect more complete information on
test results.
Pick-to: 6.6 6.5
Task-number: QTQAINFRA-5146
Change-Id: I226203f93413689f5767a4d01ea1852b453ebf31
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
It is outdated and it was discussed that it would be removed in favor
of other options.
Change-Id: If2468c84d8ca8197587577ae3d72aeb96c822c5f
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Jukka Jokiniva <jukka.jokiniva@qt.io>
For now this affects only the ASAN-enabled testruns that are run
without "-V" in ctest's command line.
Pick-to: 6.6 6.5
Change-Id: Iaa889f950e5de0657523dc9bcb457968c1cbf29e
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reason is that after changing the sanitized testruns to quiet, we
have been hitting the 15min output-inactivity timeout.
Set it to the same value as the total workitem timeout, effectively
disabling it.
Pick-to: 6.6 6.5
Change-Id: I13c2d97b25cf338873bb89ef1ed45d58e3cdd170
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
This is purely cosmetic, to avoid confusion when reading the logs and
find many "FAIL!" strings all over the place. The ASAN testrun only
cares for "ERROR: AddressSanitizer" type errors, and tests pass even
if they print "FAIL!" (that is taken care by sanitizer-testrunner.py).
Fixes: QTQAINFRA-5896
Pick-to: 6.6 6.5
Change-Id: Ib7a5fb2c3ee56581db20efb4dd7cf24a053d5c13
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
It moved to libexec at some point in 6.5.
Pick-to: 6.5 6.6
Change-Id: I1e80c0dd1dc253c08bf509aeb605b863a5d67e47
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
The android emulator launcher script is mainly used in CI and thus
makes sense to be moved there. This also makes it easier to make changes
without having to wait for submodule updates.
ANDROID_EMULATOR_RUNNER is defined in qt5's Android provisioning script.
Also, change the order of instructions execution to set COIN_CTEST_RESULTSDIR
env variable before running the emulator script. That var points to the path
to where the emulator logs are saved.
Task-number: QTQAINFRA-5596
Change-Id: If6ac285fecdc49611c398e8185e6e3e6b9c353fb
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The /coin directory is not installed, so trying to include a .qdocconf file using a relative path from an (installed) global configuration will fail.
Task-number: QTBUG-113326
Change-Id: I761b1a7eb19ca9d6a768f1367e30d5eaf80cb816
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
The most common limit for the maximum number of allowed documentation
warnings is zero. Use a global value for 'warninglimit', adopted by
all Qt module documentation projects that include the configuration
from qtbase/doc/global.
This allows for a temporary increase of the limit across all modules as
needed - for example, when updating the QDoc binary that the CI
provisions to a version that introduces new types of documentation
warnings.
Increase this base limit temporarily to 10 to help re-enable
documentation testing in CI as it's currently disabled.
Task-number: QTBUG-113326
Change-Id: I8b66951ca9324bcfaec3b5a7ec2cff544c62feb0
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
IFW requires static build of Qt which does not build tests on all
configurations. This allows building modules without them building tests. Previously IFW used 5.15 repos, those build tests in test workitem, not running into the issue.
This is only relevant if the tested repo i.e IFW implements it's own
test instructions.
Pick-to: 6.6 6.5 6.2
Change-Id: I1e03f3b577d7be314f92ed3989e5f6daf2a46da1
Reviewed-by: Toni Saario <toni.saario@qt.io>
This will cause coin to test the newly integrated HEAD for the
given modules. In case a failure occurs in the check a message
in gerrit is sent to the commits which were integrated in the
HEAD.
Change-Id: I656316c7cceb126ccf570fd3db3f7fd2b22cc4fa
Reviewed-by: Simo Fält <simo.falt@qt.io>
More cores allow faster building and reduces variance when cache
is cold.
Task-number: COIN-1001
Change-Id: Idc08f19b788dcf2483342b8ae2c76b663eba537b
Reviewed-by: Simo Fält <simo.falt@qt.io>
We wasting time and resources by signing binaries that
are for internal use only.
Change-Id: Ic3271af7f11dac05979ff14a7ffa3f9075fe0ecb
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
We agreed that qt-configure-module needs to come back to bin/ as it's
user-facing.
Amends d77ce33082
[ChangeLog][CMake] Upon further consideration, qt-configure-module
was deemed user-facing, and was thus moved back to ./bin on
all platforms.
Task-number: QTBUG-107621
Change-Id: I1f7874436e5b3988242091fc0303ea828f29d5c6
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This reverts commit 3cb3218797.
Reason for revert: Breaks in qtdeclarative
Change-Id: Iefe2954826bca80ebb2aecb8e8d8367290c218d0
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
COIN_CONFIGURE_ARGS would only contain -make examples in qtbase
builds. In other repos it would use values like
NON_QTBASE_CONFIGURE_ARGS, which would not contain -make examples.
Check CONFIGURE_ARGS and TARGET_CONFIGURE_ARGS which are guaranteed to
contain -make examples when building any repo.
Amends 440438092b
Change-Id: I9bf086f8104da85ed8ece335a45c6628ea591f2a
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
After this change, private CMake scripts are mostly live in
`libexec/`, except the `qt-cmake` which will stay in `bin/`.
This doesn't affect the Windows configuration.
- `qt-cmake` stays in `bin/`
- `qt-configure-module` moves into `libexec/`
- `qt-cmake-private` moves into `libexec/`
- `qt-cmake-private-install.cmake` moves into `libexec/`
- `qt-cmake-standalone-test` moves into `libexec/`
- `qt-internal-configure-test` moves into `libexec/`
In cases where `QT_GENERATE_WRAPPER_SCRIPTS_FOR_ALL_HOSTS` is set to
ON, e.g., ANDROID, WASM, both Batch and Bash files will be generated
and placed in `bin/` and `libexec/` accordingly; in both cases,
qt-cmake and qt-cmake.bat will be in `bin/` anyway.
[ChangeLog][CMake] The private Qt CMake scripts, i.e.,
qt-configure-module, qt-cmake-private, qt-cmake-private-install.cmake,
qt-cmake-standalone-test and qt-internal-configure-test were moved
into $prefix/libexec on Unix platforms.
Fixes: QTBUG-107621
Change-Id: Ic4f4ec85f64d2ede0e208bca928959e30be906a6
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Our tests have the unfortunate tendency to not clean up their resources;
as a consequence the leaksan check creates ton of (non-actionable)
warnings.
Our main focus for now are catching non-leak related issues via ASAN
(e.g. use-after-free).
The special casing for the cmake API tests stays in place in case we
want to re-enable leaksan checks for most tests again.
Task-number: QTQAINFRA-5315
Change-Id: I3e50ebf06ebd373f0ce0a4bb790ec8d38c344515
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
This patchset adds the rules to utilize build artifacts from multiple
android abi builds and to use them for building and running tests.
If target.arch has the value 'Multi', coin will now skip building Qt
itself, since this means that all Qt versions were build previously.
It only builds test using the pre-built Qt for Android archives and
then runs the regular Android tests using multi-ABI apks. The rules
are implemented for qtbase only.
Change-Id: I265c70c45caa2155c75cd15c7e3a4f6a863252b8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
We need to make a test for wasm run in CI, and have to start small
with qtbase. This lets the tests compile and run.
Additionally, reordered the platforms for full consistency.
Apply the same change to the qtbase config.
Change-Id: If3cedffdd7f1b21215c05b9b9302df8234a47a0c
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
Alexandru Croitor
Toni Saario
Tor Arne Vestbø
Dimitrios Apostolou
Simo Fält
Matti Paaso
Tero Heikkinen
Assam Boudjelthia
Casimir Saastamoinen
Topi Reinio
Heikki Halmet
Jörg Bornemann
Amir Masoud Abdol
Fabian Kosmale
Alexey Edelev
David Skoland