We don't know which versions these blacklistings actually apply on
unless we actually get macOS 10.14 and 10.15 into the CI and running
tests, so let's start with that, and then granularize the blacklists
after that.
Task-number: QTBUG-75786
Change-Id: Id79642afa50cb20efa2cd209286b6933918d3a4a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
This patch was generated with tooling from patchset 31 of
https://codereview.qt-project.org/c/qt/qtqa/+/267034 in interactive
mode. General platform names were chosen if greater than 60% of the
currently active platforms of a given type in COIN recently failed.
Change-Id: Ia4bde7f0ec422bbb727dc9d7151295159094f146
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
These tests have not failed on the removed platforms for at least 60 days
Task-number: QTBUG-76608
Change-Id: If7a9f4db907124e3cd54e3f4b0ad3e20717d1912
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Using the information from grafana we can unblacklist all the things
which are consistently passing.
Change-Id: I79917ca9c40e1df2dab46bb54cc0a2bd4a1a4621
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
For some reason behavior of SecureTransport has changed from 10.12 to 10.13
and then to 10.14. On 10.13 SecureTransport fails upon receiving the server's
certificate with 'Unrecoverable error', before we can do a manual verification
and accept the certificate as trusted. Analysis of available source code
shows that they, apparently, do not like MD5 hash which our server is using.
Until certificate is updated on the server or we switch completely to
the Docker-based solution we have to BLACKLIST tests that connect to our
current network test-server. Oddly enough, on 10.14 SecureTransport is
less mean.
Task-number: QTBUG-69873
Change-Id: I7da1883e0970a2f6ddd8385f193b76116d6983e0
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
... and unblacklist it.
It was blacklisted some years ago because it was failing too often.
It was failing because the ssl socket had already received and decrypted
all the data it was going to get, meaning the waitForReadyRead call was
just going to block forever.
Change-Id: Ia540735177d4e1be8696f2d752f1d7813faecfe5
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This reverts commit 96c27f0dfa.
We now use a custom keychain that should fix the original
problem with the test.
Change-Id: I52e4105f34a46ad7080750d9a62480ebe3a56e68
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
protocolServerSide test fails with 'ssl3-any' and 'tls1.0-any' combo
on RHEL 7.2 with OpenSSL v1.0.1e-fips, it would work as OPENSSL_NO_SSL2
code-path, but OPENSSL_NO_SSL2 is not defined, so socket is in state
connected while it's expected to be unconnected.
Task-number: QTBUG-53608
Change-Id: Ib321b95129a6ab5b0691b40cb4acc612125633b2
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
They didn't show up in the "old" CI runs because they usually pass the second
time they are executed - which the testrunner does. The new CI doesn't do that
anymore, instead we now mark those tests explicitly and will track their record
of passing and failing in the new metrics database.
Change-Id: Id34dd6f792f38995b07b6fec88f833df64de2f8b
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Tor Arne Vestbø
Daniel Smith
Mårten Nordheim
Timur Pocheptsov
Christian Ehrlicher
Tony Sarajärvi
Timur Pocheptsov
Simon Hausmann
Tony Sarajärvi