Commit Graph

427 Commits (2937cf91c74b6562bf56e8872dfd2bfaafebb3cc)

Author SHA1 Message Date
Marc Mutz 2fd990b386 Port qCompress() to zstream/deflate()
The zlib convenience API we've been using so far has two problems:

- On Windows-64, where sizeof(long) == 4, the use of ulong for sizes
  meant that we could not compress data compressable on other 64-bit
  platforms (Unix). While zstream also uses ulong, being a stream API,
  it allows feeding data in chunks. The total_in and total_out members
  are only required for gzip compression and are otherwise just
  informational. They're unsigned, so their overflow does not cause
  UB. In summary, using zstream + deflate() allows us to compress more
  than 4GiB of data even on Windows-64.

- On all platforms, we always allocated the output buffer in such a
  way as to accommodate the pathological case of random, incompressible
  data, so the output buffer was larger than the input. Using zstream
  + deflate(), we can start with a smaller buffer, then let zlib pick
  up where it left off when it ran out of output buffer space, saving
  memory in the common case that compression meaningfully reduces the
  size. To avoid the first few rounds of reallocations, we continue to
  use zlib's compressBound() for input less than 256KiB.

This completely fixes the compression side of QTBUG-106542 and
QTBUG-104972.

Pick-to: 6.4 6.3 6.2
Fixes: QTBUG-104972
Fixes: QTBUG-106542
Change-Id: Ia7e6c38403906b35462480fd611b482f05a5c59c
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-10-17 23:33:41 +02:00
Edward Welbourne 8bb0d2308e tst_QStringApiSymmetry: use startpos to deduplicate indexing data tags
The tests for indexOf() and lastIndexOf() had duplicate data row tags,
due to only using the needle and haystack, although some tests
differed only in start position. Include start position where needed.

Change-Id: I197d415265ab1a805f2d36fb88aec92ea8646f7a
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-10-17 15:53:32 +02:00
Edward Welbourne 5b70546c17 tst_QStringApiSymmetry: avoid repetition in trimmed_data()
Enclosing one string in each substring of another does not need to
repeat the empty substring of the latter. Extracting the empty
substring from different positions doesn't get different results.
In the process, tidy up the code a bit.

Change-Id: Ic66febbdadeaac0c466f4f1174d831a991d31e20
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-10-17 15:53:32 +02:00
Edward Welbourne 781bd075a6 tst_QStringApiSymmetry: eliminate one duplicate of a mid_data() row
There were two copies of the same line in mid_data(), leading to
duplicated data row tags.

Change-Id: Ia21e855ff781b13fe18c932cff48cb0aabd12750
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-10-17 15:53:32 +02:00
Marc Mutz bda3628402 Port qUncompress() to zstream/inflate()
The zlib convenience API we've been using so far has two problems:

- On Windows-64, where sizeof(long) == 4, the use of ulong for sizes
  meant that we could not uncompress data compressed on other 64-bit
  platforms (Unix). While zstream also uses ulong, being a stream API,
  it allows feeding data in chunks. The total_in and total_out members
  are only required for gzip compression and are otherwise just
  informational. They're unsigned, so their overflow does not cause
  UB. In summary, using zstream + inflate() allows us to decompress
  more than 4GiB of data even on Windows-64.

- On all platforms, if the size hint in the header was too short, we'd
  double the output buffer size and try again, from scratch. Using
  zstream + inflate(), we still need to reallocate, but we can then
  let zlib pick up where it left off when it ran out of output buffer
  space. In all but the most pathological cases, copying the
  already-decoded data instead of re-decoding it again should be
  faster, esp. if QArrayData uses realloc() instead of malloc() +
  free() to grow the buffer.

We also now directly allocate at least as much output buffer as we
have input, to cut the first few rounds of reallocations when the
expectedSize was created, as qCompress still does, using modulo
arithmetic mod 4GiB instead of saturation arithmethic.

Factor the growing of the output buffer into a wrapper function,
flate(), which can be reused when porting qCompress().

This completely fixes the uncompression side of QTBUG-106542 and
QTBUG-104972.

Pick-to: 6.4 6.3 6.2
Task-number: QTBUG-104972
Task-number: QTBUG-106542
Change-Id: I97f55ea322c24db1ac48b31c16855bc91708e7e2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-10-16 23:01:36 +02:00
Yuhang Zhao 8cb832090a MSVC: Enable all possible conformance checks
For the full list, please refer to [1].

Needed to change the qstringapisymmetry unit test:
In theory we don't need the array to be static and it did compile
without any problems so far, indeed. However, with this patch applied,
MSVC complains that the lambda function below can't access the array.
I don't understand why, because we use [&] in the lambda and it should
capture all the variables in theory, but in reality it failed to
capture this variable in the end. And making the variable static
solves this issue. Maybe it's a MSVC bug.

Already tested locally. Most Qt repos build without any issues,
only very few repos are not tested, as my local environment
can't build them.

[1] https://docs.microsoft.com/en-us/cpp/build/reference/zc-conformance?view=msvc-170

Change-Id: I658427aa171ee1ae26610d0c68640b2f50789f15
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-10-15 13:34:20 +00:00
Marc Mutz aa37e67ef7 Port from qAsConst() to std::as_const()
We've been requiring C++17 since Qt 6.0, and our qAsConst use finally
starts to bother us (QTBUG-99313), so time to port away from it
now.

Since qAsConst has exactly the same semantics as std::as_const (down
to rvalue treatment, constexpr'ness and noexcept'ness), there's really
nothing more to it than a global search-and-replace, with manual
unstaging of the actual definition and documentation in dist/,
src/corelib/doc/ and src/corelib/global/.

Task-number: QTBUG-99313
Change-Id: I4c7114444a325ad4e62d0fcbfd347d2bbfb21541
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
2022-10-11 23:17:18 +02:00
Ievgenii Meshcheriakov c4e550703c Update UCD to Revision 30
This corresponds to Unicode version 15.0.0.

Added the following scripts:

    * Kawi
    * Nag Mundari

Full support of these scripts requires harfbuzz version 5.2.0,
this version adds support for Unicode 15.0:

    https://github.com/harfbuzz/harfbuzz/releases/tag/5.2.0

Fixes: QTBUG-106810
Change-Id: Ib06c526e49b0f01ef9f21123bcf875c6b19f2601
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-10-11 14:10:59 +00:00
Edward Welbourne 35ff456407 tst_QString::compare(): rename a test tag to be unique
There were two data8 rows; and no data9, so that was easy to fix.

Change-Id: I8191de142e1a3be57bf1ad97e63d5780f2859fea
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-10-11 11:24:51 +02:00
Edward Welbourne 3af2747c46 tst_QString::number_base(): distinguish negative binary test-cases
Two test cases were called "base  2, negative"; one of them use -1 as
value, so s/negative/minus 1/ for it.

Change-Id: Ia5da3952d93976262cc8423d4e75ec19dab9a088
Reviewed-by: Mate Barany <mate.barany@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-10-11 11:24:39 +02:00
Edward Welbourne 167da771f6 tst_QRE::wildcard(): use more informative data row tags
Using simply the pattern didn't work so well when some patters are
used repeatedly, on different haystacks. So include the haystack
in the tag name. Remove one straight up duplicate row.

Change-Id: Ib46364581f23c493e83d75e6d04ab09e4329a3a5
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
Reviewed-by: Samuel Gaist <samuel.gaist@idiap.ch>
2022-10-11 11:24:11 +02:00
Edward Welbourne 042702d961 tst_QBAApiSymmetry::toInt_data(): deduplicate a data-row name
One "empty" test was base ten, the other left the code to work out the
base. Change the latter's name to reflect that difference.

Change-Id: I4918eb0d293420df315d86e532787950b8f05be8
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
2022-10-11 11:24:01 +02:00
Edward Welbourne 11997578ac Provide optTitle for a NumberDoubleTestData testcase that needs it
The test-case for 0.0001 with precision 0 has the same expected text
as that for 0.0 with the same precision; which lead to QBA's test of
it getting a duplicated data tag. Add an optTitle for the one that
isn't precise to deduplicate.

Change-Id: I03600e2af43f6d11b53e05e8027924c92ed4db89
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-10-11 11:21:38 +02:00
Edward Welbourne 3f43ddbd10 Rename tst_QBA::toFromHex()'s longer no-leading-zero data-row
No two data-rows should have the same name.

Change-Id: I0e222498ca5bb973f703b7e6bd314fc38bb4db4d
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
2022-10-11 11:21:30 +02:00
Edward Welbourne 5c4dcd0697 Rename tst_QBA::movablity() to fix the typo
There's an i between b and l, as well as after l.

Change-Id: I9252935687e25f1f11aacb11286edf666ff9af8a
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
2022-10-11 11:21:15 +02:00
Edward Welbourne e61dc8a83b Don't duplicate a column name in tst_QByteArray::movablity_data()
It first added a column, then some rows, then called
prependExtended_data(), which expects to be called first in a data
function and starts by adding the same column. So put that first and
drop the duplicate addition of the column.

Change-Id: Ia5cf86f821608e78f0e4872db2b3167ef81cc59e
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-10-11 11:21:06 +02:00
Marc Mutz df9d882d41 Port from container.count()/length() to size()
This is semantic patch using ClangTidyTransformator:

  auto QtContainerClass = expr(hasType(namedDecl(hasAnyName(<classes>)))).bind(o)
  makeRule(cxxMemberCallExpr(on(QtContainerClass),
                             callee(cxxMethodDecl(hasAnyName({"count", "length"),
                                                  parameterCountIs(0))))),
           changeTo(cat(access(o, cat("size"), "()"))),
           cat("use 'size()' instead of 'count()/length()'"))

a.k.a qt-port-to-std-compatible-api with config Scope: 'Container'.

<classes> are:

    // sequential:
    "QByteArray",
    "QList",
    "QQueue",
    "QStack",
    "QString",
    "QVarLengthArray",
    "QVector",
    // associative:
    "QHash",
    "QMultiHash",
    "QMap",
    "QMultiMap",
    "QSet",
    // Qt has no QMultiSet

Change-Id: Ibe8837be96e8d30d1846881ecd65180c1bc459af
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-10-04 07:40:08 +02:00
Øystein Heskestad d8387fc538 Reduce foldCasing of the needle in Boyer-Moore QString searches
Before searching, foldCase the first up to 256 characters, and use this
buffer to compare against the haystack. If the needle is larger than the
buffer, compare the rest of the needle against the rest of the haystack
for every potential match. The buffer is placed on the stack and must be
refolded for each search, but this change does not break the API.

This is faster than the old implementation, except if the needle is long
and it is found near the beginning of the haystack, or if the needle is
long and it is not found in a short haystack where few comparisons are
done and hence few case foldings were needed in the old implementation.

Benchmarking using tst_bench_qstringtokenizer tokenize_qstring_qstring
shows an improvement for the the total testcase and usually for each
individual test.

Fixes: QTBUG-100239
Change-Id: Ie61342eb5c19f32de3c1ba0a51dbb0db503bdf3a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-09-22 18:05:44 +02:00
Mårten Nordheim 7447d73327 tst_QLocale: use fenv everywhere
And remove the platform-specific code.
fenv is available since c++11.

Change-Id: Ia5540be93b54117d4b5e9c7579100039c151dcc5
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-09-15 23:41:51 +02:00
Edward Welbourne 6a14ad4993 Perform uiLanguages() likely-adjusted processing purely on IDs
The collection of translations available to us need not have anything
to do with whether CLDR has matching data, so preserve the system UI
language list's entries as they are, rather than forcing them through
the QLocale constructor's exercise of likely sub-tag rules.

Instead, simply parse the given locale tags to QLocaleId instances and
use these in the likely-subtag processing to determine what other
entries to add to the list in addition to those supplied by the
operating system. Since going via QLocale did usually supply a
territory, that was included in the BCP 47 name, it's now possible for
the given entry to lack the language_territory name, so be sure to add
that if missing.

This incidentally reduces heap traffic and saves a fair deal of hidden
likely-subtag processing in calls to the constructor and bcp47Name().

Expand testing of QLocale::uiLanguages(), both plain and system.  In
the process, cross-link the two closely-related tests, move a comment
on one's _data() to the other's, where it really belongs, and add
reporting of the actual lists on failure. Enable MySystemLocale to
remember the requested locale's ID, before likely sub-tag processing,
so that we can make query() report results for language, script and
territory as requested, to ensure the fake system locale really does
match what was requested. The new german-britain test failed without
it, because there is no de-GB locale in CLDR.

Task-number: QTBUG-99531
Change-Id: Ide041577772c442a4413e3b9a590e11140c48f49
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-09-07 15:29:39 +02:00
Edward Welbourne ebe9aca900 Simplify system locale management
Have QSystemLocale manage a stack, so that tests can install an
over-ride for the actual system-specific one reliably and restore the
system-specific one when finished. Leave the QSystemLocaleSingleton
out of the stack, all the same. In the process, mark the QDoc comments
for QSystemLocale all as \internal, since this is not public API.

Change-Id: I8faed49780215e42f32be10cf936c32bb46105bf
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-09-07 15:29:39 +02:00
Edward Welbourne 753bfdf6a1 Work round macOS's omission of en_DE from its own uiLanguages()
When the system locale is en_DE, macOS seems to think we should use
en_GB as the right translation. While that probably is a sensible
choice in the absence of an en_DE translation, we should definitely
use the en_DE translation if available, especially if en_GB isn't
available (which lead to a fall-back to de_DE, given later entries in
macOS's list). So prepend the system locale's own pcp47Name() if it
(isn't the C locale and) is missing from what we would otherwise have
used for uiLanguages(), after likely sub-tag perturbations.

Add a test simulating (some approximation to) what macOS was doing
that would have caught this case; and add a scope-guard reporter to
the test to report what shows up when lists don't match.

Fixes: QTBUG-104930
Pick-to: 6.4 6.4.0 6.3 6.2 5.15
Change-Id: I116234708067e1717d9157aebc84da76e04a9f38
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-09-07 13:29:39 +00:00
Mårten Nordheim 070446d5eb tst_qlocale: only restore the fp control masks we change
Certain masks are not supported outside 32-bit x86, and will assert on
x64.

Pick-to: 6.2 6.3 6.4
Fixes: QTBUG-106000
Change-Id: Ic9f58e5a19c1db3309edeb5ec529e7a78c929665
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-09-07 13:15:27 +00:00
Volker Hilsheimer 3b91bab465 Fix compiler warning: don't copy in ranged-for loop
Change-Id: If21f4e34324f33300009a427fb81e23ab49d4ad0
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-09-03 19:10:42 +02:00
Marc Mutz 812a0d3125 QAnyStringView: construct from any T implicitly convertible to QString/QByteArray
This includes QDBusReply, QProperty, and QStringBuilder expressions.

The new constructor subsumes the QStringBuilder case without requiring
jumping though hoops to delay the definition of the ctor the way we
had to for the explicit QStringBuilder constructor, so remove the
explicit QStringBuilder one again.

[ChangeLog][QtCore][QAnyStringView] Can now be constructed from
anything that implicitly converts to either QString or QByteArray.

Fixes: QTBUG-105389
Change-Id: I0e584dd3e20d591381609a3329ef47cec7356ecc
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
2022-09-01 15:51:48 +02:00
Sona Kurazyan b077c419ea Move QMacAutoReleasePool from qglobal.h to qcore_mac_p.h
And include qcore_mac_p.h where needed.

Task-number: QTBUG-99313
Change-Id: Idb1b005f1b5938e8cf329ae06ffaf0d249874db2
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-09-01 13:26:30 +02:00
Ivan Solovev aafd2c3fba tst_qstring: port away from deprecated APIs
* QVariant::Type -> QMetaType::Type.
* Guard the test for deprecated fromUtf16(const ushort *) overload with
  QT_DEPRECATED_SINCE check.
* Use fromUtf16(const char16_t *) overload in other places.

As a drive-by: fix formatting in the affected lines.

Task-number: QTBUG-104858
Change-Id: I9fa3a935bca36e97f934f673e2fc07b451c72872
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-08-27 02:07:54 +02:00
Ivan Solovev c6c41cc1a9 QtBase tests: remove QT_DISABLE_DEPRECATED_UP_TO defines
The value will be propagated from Qt build.

Task-number: QTBUG-104858
Change-Id: Iae2c32c3037438f41b92f9ee28004f30eb4e3210
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-08-24 22:08:49 +02:00
Lucie Gérard 32df595275 Change the license of all CMakeLists.txt and *.cmake files to BSD
Task-number: QTBUG-105718
Change-Id: I5d3ef70a31235868b9be6cb479b7621bf2a8ba39
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-23 23:58:42 +02:00
Ivan Solovev 3226c82740 Rename QT_DISABLE_DEPRECATED_BEFORE -> QT_DISABLE_DEPRECATED_UP_TO
The new name describes the behavior in a better way.

[ChangeLog][Build System] The QT_DISABLE_DEPRECATED_BEFORE macro is
renamed to QT_DISABLE_DEPRECATED_UP_TO. The old name is deprecated, but
is still recognized if it is defined during configuration and the new
name is not defined.

Task-number: QTBUG-104944
Change-Id: Ifc34323e0bbd9e3dc2f86c3e80d4d0940ebccbb8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-08-19 23:52:05 +02:00
Tatiana Borisova 6a3509baa2 Put code related to timezone functionality under QT_CONFIG(timezone)
Pick-to: 6.4 6.3
Change-Id: I0d1573dbb4ae4f9740fc19546950ae4316aa4c0c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-08-18 18:05:09 +00:00
Assam Boudjelthia cf4f147c23 Android: tst_QByteArray::base64_2GiB() times out or gets terminated
Android kills this test case which tries to use too much memory,
or it times out.

Pick-to: 6.4 6.3 6.2
Task-number: QTQAINFRA-4748
Change-Id: Ifce92533d50f4c463ee10fe80e7654ad16172a35
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
2022-08-18 13:52:02 +03:00
Marc Mutz b5d5385201 tst_qbytearray: remove qCompress_data for QT_NO_COMPRESS
The _data function is useless without its test function (and it's not
used in other _data functions).

Pick-to: 6.4 6.3 6.2 5.15
Change-Id: I7aa6006ed1a9d89008577b750af4ea717dae237f
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-08-15 23:02:32 +02:00
Øystein Heskestad 15368fb31b Add Latin 1 case-insensitive Boyer-Moore searcher
The std::boyer_moore_searcher is buggy for older verions of Microsoft's
STL, and missing in AppleClang's libc++ with an inefficient fall back.

Fixes: QTBUG-100236
Change-Id: Ic3cc916946546d2ef78456cd15e1425d957b989d
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-08-04 23:12:39 +02:00
Edward Welbourne 0a36a7c1db Make 'zz' format an alias for 'z' in time format strings
Also document the (seldom helpful) handling of over-long repeats of a
format. Add test to QDateTime and amend QLocale test.

[ChangeLog][QtCore][QDateTime] Doubling the 'z' format in a date-time
or time format string now produces the same output as a single 'z'.
Previously, this would have produced two copies of the milliseconds
field (eliding any trailing zeros in each). Contrast with 'zzz', which
produces the full milliseconds field, including any trailing zeros.

Change-Id: I4c60462b062fee4079370096d745c191c1939506
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-08-03 18:38:53 +02:00
Lucie Gérard fb1b20eab3 Add license headers to cmake files
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.

Existing copyright statements remain intact

Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-03 17:14:55 +02:00
Giuseppe D'Angelo e540d4a864 QRegularExpression: introduce (global)matchView
QRegularExpression::match (and globalMatch) is currently overloaded
for QString and QStringView. This creates a subtle API asymmetry:

  QRegularExpression re;

  auto m1 = re.match(getQString());       // OK
  auto m2 = re.match(getStdU16String());  // Dangling

This goes against our decision that every time that there's a possible
lifetime issue at play, it should be "evident". Solving the lifetime
issue here is possible, but tricky -- since QRegularExpression
is out-of-line, one needs a type-erased container for the input
string (basically, std::any) to keep it alive and so on.

Instead I went for the simpler solution: deprecate match(QStringView)
and introduce matchView(QStringView) (same for globalMatch). This
makes it clear that the call is matching over a view and therefore
users are supposed to keep the source object alive.

Drive-by, remove the documentation that says that the QString
overloads might not keep the string alive: they do and forever will.

[ChangeLog][QtCore][QRegularExpression] Added the matchView()
and globalMatchView() functions that operate on string views.
The match(QStringView) and globalMatch(QStringView) overloads
have been deprecated.

Change-Id: I054b8605c2fdea59b556dcfea8920ef4eee78ee9
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-07-29 23:04:27 +02:00
Alexandru Croitor 4d22405e48 CMake: Don't use PUBLIC_LIBRARIES for tests and test helpers
Change-Id: I9b7404e1d3a78fe0726ec0f5ce1461f6c209e90d
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2022-07-28 14:46:53 +02:00
Marc Mutz d76bf64531 QByteArray: fix base64 round-trip w/more than 2GiB data
There was an explicit int cast in fromBase64Encoding() which was never
ported to qsizetype and therefore truncated the result.

Fix by removing the int cast.

Add a test, optimize it for as low memory usage as possible, given we
need to work in input and output data each in excess of 2GiB.

Fixes: QTBUG-104985
Pick-to: 6.4 6.3 6.2
Change-Id: I9c0924957e62e5cb3003132cd811b8b0315d8ac1
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-07-26 02:06:17 +02:00
Marc Mutz cb9715557c tst_QByteArray: fix custom QCOMPARE for QBAs > 2GiB
The tst_QByteArray test redefines the QCOMPARE macro to check the LHS
to be NUL-terminated. Because the code was never ported from int to
qsizetype, it fails for QByteArrays of size > 2GiB.

Fix by porting to qsizetype.

Pick-to: 6.4 6.3 6.2
Task-number: QTBUG-104985
Change-Id: Ib3951b0efed5f734ae1324ea2d455bb7762fb9c4
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-07-20 12:56:18 +02:00
David Skoland 281d915ba4 Add wasm exception for tst_qchar
Since wasm doesn't like filesystems that well, we omit this test in
tst_qchar for wasm, this allows us to still run the test without the
case where it needs to read the normalization file.

Change-Id: I37e54d97e119f94e1a9ca53917d0b93183321899
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2022-07-11 17:21:27 +02:00
Giuseppe D'Angelo 6f852935e0 QString: fix lifetime issues with QRegularExpression APIs
QString has several functions taking a QRegularExpression: indexOf(),
contains(), and so on. Some of those have an out-argument of type
QRegularExpressionMatch, to report the details of the match (if any).
For instance:

  QRegularExpression re(...);
  QRegularExpressionMatch match;

  if (string.contains(re, &match))
    use(match);

The code used to route the implementation of these functions through
QStringView (which has the very same functions). This however opens
up a lifetime problem with temporary strings:

  if (getString().contains(re, &match))
    use(match); // match is dangling

Here `match` is dangling because it is referencing data into the
destroyed temporary -- nothing is keeping the string alive. This is
against the rules we've decided for Qt, and it's also asymmetric with
the corresponding code that uses QRegularExpression directly instead:

  match = re.match(getString());
  if (match.hasMatch())
    use(match); // not dangling

... although we've documented not to do this. (In light of the decision
we've made w.r.t. temporaries, the documentation is wrong anyways.)
Here QRE takes a copy of the string and stores it in the match object,
thus keeping it alive.

Hence, extend the implementation of the QString functions to keep a
(shallow) copy of the string. To keep the code shared as much as
possible with QStringView, in theory one could have a function taking a
std::variant<QString, QStringView> and that uses the currently active
member. However I've found that std::variant here creates some abysmal
codegen, so instead I went for a simpler approach -- pass a QStringView
and an optional pointer to a QString. Use the latter if it's loaded.

QStringView has some inline code that calls into exported functions, so
I can't change the signature of them without breaking BC; I'm instead
adding new unexported functions and a Qt 7 note to unify them.

Change-Id: I7c65885a84069d0fbb902dcc96ddff543ca84562
Fixes: QTBUG-103940
Pick-to: 6.2 6.3 6.4
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-07-10 23:40:24 +02:00
Lars Knoll 9c1f3b6d4d Add QStringDecoder::decoderForHtml()
Now that QStringConverter can handle non UTF encodings through ICU,
add a way to get a decoder for arbitrary HTML code.

Opposed to QStringConverter::encodingForHtml(), this method will
try to create a valid string decoder also for non unicode codecs.

Pick-to: 6.4
Change-Id: I343584da1b114396c744f482d9b433c9cedcc511
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-06-21 01:04:12 +02:00
Fabian Kosmale 122270d6be Long live the ICU-based QStringConverter interface!
This adds support for additional codecs to QStringConverter when ICU is
available.

We store the converter in the state (d[0]), and its canonical name in
d[1]. We need the name there, as in the clear function we close the
UConverter, and set the pointer to null. Consequently, the actual
conversion functions might need to re-open the converter again. The
advantage of this approach is that clear is used in the destructor of
State, and with this approach we properly clean up the state.
There is however a disadvantage: The clear function was so far also used
for resetting the state when QStringConverter::resetState . Discarding
the whole Uconverter for that is however rather costly. For that reason
we modify resetState to call a new function, State::reset. For existing
converters, it behaves the same as clear; for the ICU based converter,
we call the more efficient ucnv_reset. Code compiled against Qt 6.4 can
benefit from this more efficient version; code compiled against older Qt
versions will continue to work, as the conversion functions can just
recretate the converter from the name.
We can distinguish between ICU and non-ICU converters by checking if the
UsesIcu flag is set.

QStringConverter::name is changed to return the name stored in d[1]. The
interface of the ICU converter has a dummy name, so code using the old
name function from QT < 6.4 still returns something, namely a message
asking the user to recompile.
The function is moved out of line, as we need to check for the private
ICU feature, and want to avoid having that check in the public header.

As the QStringConverter ctor taking a name now can allocate memory, it
can no longer be noexcept. Removing the noexceptness is safe, as it was
only added after Qt 6.3.

Note that we cannot extend the API consuming or returning Encoding, as
we use Encoding values to index into an array of converter interfaces in
inline API.

Further API to support getting an ICU converter for HTML will be added
in a future commit.

Currently, the code depending on ICU is enabled at compile time if ICU
is found. However, in the future it could be moved into a plugin to
avoid a hard dependency on ICU in Core.

[ChangeLog][Corelib][Text] QStringConverter and API using it now
supports more text codecs if Qt is compiled with ICU support.

Fixes: QTBUG-103375
Change-Id: I7afb92fc68ef994179ebc7a3aa73beebb1386204
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-06-19 00:41:12 +02:00
Thiago Macieira 2a1122f465 QStringConverter: fix use-after-free in the stack in the test
Detected by ASan. Introduced by aef27c5aa2

Pick-to: 6.3 6.4
Fixes: QTBUG-104261
Change-Id: Id0fb9ab0089845ee8843fffd16f88bdeb4f42c7c
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2022-06-15 15:43:32 -07:00
Marc Mutz 1b8b802455 tst_QAnyStringView: explicitly check the spaceship operator
We implicitly checked it, because, in C++20 builds, the non-equality
relational operators are synthesized from it by the compiler, and we
test those, but we didn't check that <=> returns strong_ordering.

We now do.

Pick-to: 6.4 6.3 6.2
Task-number: QTBUG-104108
Change-Id: Ieb19a2d4cb2d600d884f4e2e89e98c6187e23872
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-06-14 01:30:48 +00:00
Ivan Solovev 1e36eedb7f tst_QStringApiSymmetry: use QCOMPARE_EQ instead of QCOMPARE
This allows to defer the toString() invocation until it is really
needed, and so allows to speed-up the test execution.

I was testing a release shared developer build, running
tst_QStringApiSymmetry 10 times before the change, and 10 times after
the change, and then taking the average execution duration, as
reported in the log.

Before the change (using QCOMPARE): 51ms
After the change (using QCOMPARE_EQ): 45ms

As we see from the results, the benefit is around 10%.

Task-number: QTBUG-98873
Task-number: QTBUG-98874
Change-Id: Ifcfbcca1f2c1eaf82c7f1a9098fa1512a269cbf8
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-06-03 20:37:51 +00:00
Fabian Kosmale c46ee7df57 QStringConverter: Do not crash if invalid
Attempting to use an invalid QStringConverter would so far have resulted
in a crash, as we would dereference the null iface pointer.
Fix this by inserting adequate checks, and ensure that hasError returns
true if we attempt to en/decode with an invalid converter.

Pick-to: 6.2 6.3
Change-Id: Icf74bb88cd8c95685481cc0bd512da99b62f33e6
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2022-05-25 16:38:35 +02:00
Ievgenii Meshcheriakov 0e64d97cc1 tst_qtextboundaryfinder: Remove full copies of data files
There are no commented out test cases remaining, so the normal
test vectors are identical to full test vectors.

Fixes: QTBUG-97537
Pick-to: 6.3
Change-Id: I987f178f192e1c8e2d998d36499fdce84f237e77
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:43 +02:00
Ievgenii Meshcheriakov 4d4d8dac94 QUnicodeTools: Fix line breaking before open parentheses
UAX #14, revision 45 (Unicode 13) has changed rule LB30 to only
trigger if the open parentheses is non-wide:

    (AL | HL | NU) × [OP-[\p{ea=F}\p{ea=W}\p{ea=H}]]

This fixes the remaining 24 line break tests.

Task-number: QTBUG-97537
Pick-to: 6.3
Change-Id: I9870588c04bf0f6ae0a98289739bef8490f67f69
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:43 +02:00