ff0285ce6b
Backstory. The current approach for building documentation in the CI is that we build a static qdoc in a Jenkins job, manually provision it in qt5.git, and then build qt documentation using that somewhat old qdoc version. Updating the provisioned qdoc is a very slow process, because changes first need to be merged into qttools, then a Jenkins job needs to be manually run, then provisioning needs to be updated, and sometimes it also depends on a full qt5.git submodule update or has to wait for other unrelated changes to qt5.git. The new approach. This change adds CI instructions to build a static qdoc (and other relevant tools necessary for documentation generation) using the latest qttools/dev/HEAD sha1 in a specific host platform's qtbase build instructions, and then uses those tools for documentation generation and doc warning checks in each repo's TestDocs CI test phase. To enable these new instructions, the host platform and the doc checking platform need to be tagged with the 'DocsGenerationV2' feature. The built tools are cached in a separate archive using a new Coin feature, which is then extracted during each repo's TestDocs phase. Pros and cons. The benefit of this new approach is that we will always use a mostly up-to-date qdoc from qttools/dev/HEAD when generating documentation, making the documentation team's life easier. Specifically, once the doc tools are built in qtbase with the latest qttools/dev/HEAD sha1 that was available during that integration, the resulting built tools are then used for doc checks in dependent repos. A newer qttools/dev/HEAD sha1 will only be used for a particular repo once the qtbase dependencies.yaml sha1 is updated, or when doing qt5.git changes that cause a rebuild of qtbase. Note that while new tools are built due to dependency updates, the used qttools sha1 is not recorded in any way in any of the dependencies.yaml files. qtbase doc checks will always use the latest qttools/dev/HEAD sha1 at the integration time, because the tools will always be built from scratch. The downside of the new approach is that, by default, the doc generation will not use a pinned version of qdoc anymore, but rather follow the flow described above. This has a chance of introducing failures which are unrelated to integrating changes. This should happen rarely because the doc team usually tries to fix doc warnings before changing qdoc's code. If temporary pinning of qttools or qt5.git is required when testing the docs of a specific repo, it can be achieved as follows: - set the QT_CI_BUILD_REPO_DOC_TOOLS env var to "1" in the repo coin/module_config.yaml file build instructions, to ensure the doc tools are built in that repo, rather than reuse the ones built in qtbase - set the QT_CI_DOC_TOOLS_PIN_GIT_REF env var to the desired qttools sha1 or other git ref, in the same build instructions phase - optionally set the QT_CI_DOC_TOOLS_TOP_LEVEL_PIN_GIT_REF env var to the desired qt5.git sha1 or other git ref, in the same build instructions phase, if a different version of qt5.git is required - optionally set the QT_CI_DOC_TOOLS_USE_CI_TOP_LEVEL_BRANCH env var to ON, in case if the module branch that the CI job computes should be used as the qt5.git branch - set the QT_CI_FETCH_REPO_DOC_TOOLS env var to "1" in the repo test instructions, to ensure the just built doc tools are fetched instead of the qtbase-built ones Test running this new approach will allow us to collect some feedback on how often breakages happen, and how much easier it makes the documentation team's work, or how much harder it makes the life of regular integrations. The abundant amount of optional pinning options should be enough to avoid any permanent integration dead locks. Task-number: QTBUG-128730 Change-Id: I8606cb3076036a4a0ec652d0fa74d270e8f5dfdf Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Paul Wicking <paul.wicking@qt.io> (cherry picked from commit 1f646bb978ed94c25f6224e801779c929096c0ae) Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org> |
||
|---|---|---|
| .. | ||
| debian | ||
| qmake | ||
| qmake_examples | ||
| README.md | ||
| call_cmake_for_standalone_examples.yaml | ||
| call_cmake_for_standalone_tests.yaml | ||
| call_configure_module.yaml | ||
| call_configure_qtbase.yaml | ||
| call_host_install.yaml | ||
| call_target_install.yaml | ||
| cmake_build_and_upload_test_artifacts.yaml | ||
| cmake_build_and_upload_test_artifacts_host.yaml | ||
| cmake_build_and_upload_test_artifacts_target.yaml | ||
| cmake_build_standalone_examples.yaml | ||
| cmake_build_standalone_examples_host.yaml | ||
| cmake_build_standalone_examples_target.yaml | ||
| cmake_cross_compilation_module_build_instructions.yaml | ||
| cmake_cross_compilation_qtbase_build_instructions.yaml | ||
| cmake_documentation_build.yaml | ||
| cmake_module_build_instructions.yaml | ||
| cmake_qtbase_build_instructions.yaml | ||
| cmake_regular_test_instructions.yaml | ||
| cmake_run_ctest.yaml | ||
| cmake_setup_running_qnxqemu_tests_env_vars.yaml | ||
| cmake_setup_running_tests_env_vars.yaml | ||
| coin_build_doc_tools.yaml | ||
| coin_build_doc_tools_checked.yaml | ||
| coin_fetch_doc_tools.yaml | ||
| coin_module_axivion_template_v2.yaml | ||
| coin_module_build_template_v2.yaml | ||
| coin_module_test_android_start_emulator.yaml | ||
| coin_module_test_docs.yaml | ||
| coin_module_test_qemu_env_vars.yaml | ||
| coin_module_test_qnx_start_emulator.yaml | ||
| coin_module_test_template_common.yaml | ||
| coin_module_test_template_v2.yaml | ||
| coin_module_test_template_v3.yaml | ||
| coin_qtbase_build_template_v2.yaml | ||
| coin_qtbase_test_docs.yaml | ||
| coin_test_docs_common.yaml | ||
| prepare_android_multi_abi_env.yaml | ||
| prepare_building_env.yaml | ||
| prepare_configure_executable.yaml | ||
| prepare_configure_module_executable.yaml | ||
| prepare_install_dir_suffix_host.yaml | ||
| prepare_install_dir_suffix_target.yaml | ||
| run_license_check.yaml | ||
README.md
Information about Coin instruction templates
Build templates
coin_qtbase_build_template_v2.yamldid not exist. The build instructions were directly embedded intomodule_config.yamland did not support repos outside of qtbase, also no cross-compilation.coin_qtbase_build_template_v2introduced support for building other repos, and also enabled build cross-compiling targets likeAndroidandiOS. A bit later the template gained the ability to buildqemucross-compiling configurations. The counterpart to qtbase to build other repositories iscoin_module_build_template_v2
Test templates
coin_module_test_template_v1did not exist. The test instructions were directly embedded intomodule_config.yamland did not support repos outside of qtbase, also no cross-compilation.coin_module_test_template_v2introduced support for building tests for other repos, and made sure not to build and run tests on cross-compiling configuration. A bit later the template gained the ability to build and run tests forqemucross-compiling configurations.coin_module_test_template_v3changed the run test instructions to not ignore the exit code and thus enforce that tests pass in the CI.
Environment variable description and usage
The following environment variables are used in Coin instructions when building Qt, tests, etc:
CONFIGURE_ARGS - contains platform-specific configure-style arguments
(e.g. -shared), that will be passed to a qtbase configure call
CMAKE_ARGS - contains platform-specific CMake-style arguments
(e.g. -DOPENSSL_ROOT_DIR=Foo) that will be passed to a qtbase
configure call
NON_QTBASE_CONFIGURE_ARGS - contains platform-specific configure-style arguments
that will be passed to a non-qtbase qt-configure-module call
NON_QTBASE_CMAKE_ARGS - contains platform-specific CMake-style arguments
that will be passed to a non-qtbase qt-configure-module call
COMMON_CMAKE_ARGS - platform-independent CMake-style args set in
prepare_building_env.yaml that apply to qtbase configurations
only.
COMMON_NON_QTBASE_CMAKE_ARGS - platform-independent CMake-style args set in
prepare_building_env.yaml that apply to
configuration of repos other than qtbase
COMMON_TEST_CMAKE_ARGS - platform-independent CMake-style args set in
prepare_building_env.yaml that apply to configuration of
all standalone tests
All of the above apply to host builds only.
There is a a set of environment variables that apply to target builds when cross-compiling which mirror the ones above. They are:
TARGET_CONFIGURE_ARGS
TARGET_CMAKE_ARGS
NON_QTBASE_TARGET_CONFIGURE_ARGS
NON_QTBASE_TARGET_CMAKE_ARGS
COMMON_TARGET_CMAKE_ARGS
COMMON_NON_QTBASE_TARGET_CMAKE_ARGS
COMMON_TARGET_TEST_CMAKE_ARGS
Currently, there are no common configure-style variables for configuring
repos or tests, only ``CMake-style` ones.
COIN_CMAKE_ARGS contains the final set of cmake args that is passed to
configure / qt-configure-module, it is built up from the variables above + any additional values added
by custom instructions, like specification of CMAKE_INSTALL_PREFIX etc.
INSTALL_DIR_SUFFIX is used to append either /host or /target suffixes to install paths in
instructions when cross-building.
CONFIGURE_EXECUTABLE contains a platform-specific path to configure / qt-configure-module
or cmake/ qt-cmake depending on whether UseConfigure feature is enabled.
CONFIGURE_ENV_PREFIX contains the value of either ENV_PREFIX or TARGET_ENV_PREFIX depending on
whether it's a cross-build configure call. The values are used when configuring and building, to ensure
that things like compilers are found correctly.
We use unixPathSeparators to pass an install prefix with forward slashes even on Windows,
to avoid escaping issues when using configure.