Then we will automatically handle invalid leading characters instead
of throwing away the whole sequence when it cannot be converted.
Added a test that was failing before.
Drive-by change: add a comment explaining why we
have the stack allocated buffer.
Task-number: QTBUG-118834
Change-Id: I647a58f2ba95e2e7ed4ea6a964d99ecc0c91fad3
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Also extend unit-test to use new test helper functions.
Remove the now-redundant test for three-way comparison, because it is
covered by the test helper functions.
Task-number: QTBUG-117661
Change-Id: I242b560c281245e04e34353c80000a20998fc677
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Replace the existing friend relational operators with the macros.
Add the previously-missing relational opertors with QChar and char16_t.
This allows to remove the last dummy relational operators and the
macros to generate them in tst_qstringapisymmetry.
Because of a bug in libstdc++[0], we have to explicitly keep the
QBA vs QBA relational operators even in C++20 mode. This problem is
specific to QByteArray, because it is convertible to const void *.
[0]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114153
Task-number: QTBUG-117661
Change-Id: Iac7f81cd3274331b7c7695a51803321b511361c0
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Replace the existing friend relational operators with the macros.
Add the previously-missing QChar vs `const char *` relational operators.
These require out-of-line helper methods, because we need to interpret
the `const char *` array as utf-8.
The `const char *` relational operators cause ambiguities when
comparing QChar with 0 (previously it was only handled by the nullptr_t
overloads). As we have it in several places in our tests, I'd assume
that the users can also do it. To resolve the ambiguities, mark the
new relational operators as Q_WEAK_OVERLOADs.
This allows to remove the dummy QChar vs `const char *` relational
operators from tst_qstringapisymmetry, but at the same time requires
that we introduce new dummy operators for QByteArray vs char16_t
comparison. These will be fixed in a follow-up patch.
For QLatin1Char - convert to uchar before doing the comparison, to
match the behavior of QLatin1StringView and QChar. Extend QChar's
unit tests.
Task-number: QTBUG-117661
Change-Id: I9213fe05a5efdb96d48688f07bec9519f9887a77
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This allows to remove the dummy relational operators from
tst_qstringapisymmetry.
Task-number: QTBUG-108805
Change-Id: I7cb3154d6fcb571cafab6b318806f74bc8300448
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Add QU8SV vs QSV, QU8SV vs QChar, and QU8SV vs char16_t relational
operators.
This allows to get rid of the dummy relational operators in
tst_qstringapisymmetry.
Task-number: QTBUG-117661
Change-Id: If95d7418efd13c505ed0e3bef748b86ff55e623a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The comparison with QStringView works as is, there is no need to add
explicit operators.
Add the missing relational operators with QUtf8StringView. Once it
is added, comparison of QString with u8"string literal" becomes
ambiguous, so explicitly add operators for `const char8_t*` as well.
This also makes the comparison with u8 string literals faster,
because it now uses a view instead of constructing a QString.
Adding QUtf8StringView overloads also makes comparison with
`const char *` ambiguous if QT_RESTRICTED_CAST_FROM_ASCII is defined.
To fix that, mark the overload as Q_WEAK_OVERLOAD. Luckily, we can
just use the third Attributes parameter of the macro for that.
Provide more unit-tests to cover the new relational operators.
Task-number: QTBUG-117661
Change-Id: I60d1f4ad7ea607472deeb5c250e62f2bb7019268
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Use the comparison helper macros to replace the member relational
operators for comparison with QByteArray and const char *.
As QString and QByteArray are exported, we cannot simply remove the
inline methods, so wrap them into QT_CORE_REMOVED_SINCE.
Add relational operators with QByteArrayView.
Provide more unit-tests for the comparison with the byte array types.
This enables operator<=> for QString vs byte arrays in C++20 builds.
Task-number: QTBUG-117661
Change-Id: I305343e1b6c5d78b10f2976573db4e904ba6b44b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Replace the hidden friend relational operators with hidden friend
helper functions and comparison helper macros.
Provide more unit-tests for the updated types.
This enables operator<=> in C++20 builds.
Task-number: QTBUG-117661
Change-Id: I17329cd6422f272a435fc1da241203581eef7fbb
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Even though undocumented, it's public API, doesn't hurt to carry
along, and improves compiler coverage in the test, so let's not remove
it.
Found in 6.7 API review
Amends 95e6fac0a5.
Pick-to: 6.7
Change-Id: Ia935036a69e0e678f22ac86b48a2c1c5e8c46733
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Both QByteArrayView and std::string_view are Literal Types, so the
conversion between them should be constexpr.
Amends 96d67da420.
Found in API-review.
Pick-to: 6.7
Change-Id: Ic513ce32aa2a743ca890dc05a683a62c0f3a7d50
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Juha Vuolle <juha.vuolle@qt.io>
[ChangeLog][QtCore][QString/QByteArray] Added slice() methods that work
like sliced(), but modify the string/byte-array they are called on.
Task-number: QTBUG-99218
Change-Id: I3075562983ef123d9aa022a2304c7e774cf2ea42
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Change type of variables `int` -> `size_t`, to match assigned value.
Fixes: QTBUG-122300
Change-Id: I5b99bd6a3b307ba2ec4ef79bcc517da60ae36413
Reviewed-by: Matthias Rauter <matthias.rauter@qt.io>
Now when QLatin1StringView implements relational operators with
QByteArrayView in terms of new comparison helper macros and helper
methods taking QByteArrayView, we can easily re-use these helper
methods to provide comparison with QByteArray and const char *.
QLatin1StringView already provided almost all of these operations,
partly as hidden friend functions, partly as inline methods.
Since the class is not exported, and the methods were inline, we
can just remove all of them and replace them with the comparison
helper macros.
This should speed up the relational operators, because they do not
construct string objects using QString::fromUtf8() anymore, but use
QUtf8StringView instead.
This also adds the previously missing QByteArray vs QLatin1StringView
relational operators.
Task-number: QTBUG-117661
Change-Id: I17a9185127ae130dab9409c6340a58f5d39f5a10
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
... by using the new comparison helper macors.
Note that by providing helper functions for QByteArrayView,
we can support all three types: QByteArray, QByteArrayView, and
const char *.
Use the regular QT_NO_CAST_FROM_ASCII and
QT_RESTRICTED_CAST_FROM_ASCII guards to disable the operators
if the cast from ASCII is forbidden.
Also use QT_ASCII_CAST_WARN on each operator.
This allows to enable related tests in tst_qstringapisymmetry.
Task-number: QTBUG-117661
Task-number: QTBUG-108805
Change-Id: I0d77c30245d8b5ac4b8cfd98d650c1885aca2005
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
... by using the new comparison helper macros.
Also, convert the existing QByteArrayView relational operators to use
these macros.
An attempt to define the helper functions comparesEqual() and
compareThreeWay() as hidden friends leads to compilation errors, because
the QByteArrayView(QLatin1StringView) constructor gets somehow disabled.
Apparently, it cannot satisfy the
QtPrivate::IsContainerCompatibleWithQByteArrayView trait anymore.
I could not find a reason for that, so I just defined the helper
functions as static inline private members of QByteArrayView. This fixes
the issue.
This allows to enable related tests in tst_qstringapisymmetry.
Task-number: QTBUG-108805
Change-Id: I35a69e99db8c61531ec726dab5b242b857f69e85
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
... by using the new comparison helper macors.
This allows to enable related tests in tst_qstringapisymmetry.
Task-number: QTBUG-108805
Change-Id: I2cef8f4a25889b74a921fea47995d59c3a49d368
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
... by using the new comparison helper macros.
First the relational operators between QU8SV and QBAV where added,
and this resulted in the ambiguities when comparing QU8SV vs QBA.
So, the relational operators between QU8SV and QBA are also added
in the same commit. This, in turn, resulted in ambiguities when
comparing QU8SV and const char *, so add these relational operators
as well.
Use the regular QT_NO_CAST_FROM_ASCII and
QT_RESTRICTED_CAST_FROM_ASCII guards to disable the operators if
the cast from ASCII is forbidden. Also use QT_ASCII_CAST_WARN on
each operator.
This allows to enable related tests in tst_qstringapisymmetry.
Task-number: QTBUG-117661
Task-number: QTBUG-108805
Change-Id: If7919496fdf4519fd2a9398397a39210aadba077
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
... by using the new comparison helper macros.
The operators are marked as QT_ASCII_CAST_WARN, like the pre-existing
relational operators with QByteArray and const char *.
This allows to enable related tests in tst_qstringapisymmetry.
Task-number: QTBUG-117661
Task-number: QTBUG-108805
Change-Id: Ic9bcddffc25585edb7375c3e651d49d040a60454
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
... by using the new comparison helper macros.
This allows to remove dummy comparison operators from
tst_qstringapisymmetry.
This is also a pre-requisite for a follow-up commit that introduces
relational operators between QLatin1StringView and QByteArrayView.
Task-number: QTBUG-117661
Task-number: QTBUG-108805
Change-Id: I5837b457a777fddff1071bc252982e68d004fa94
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
As we did for convertToUnicode. To support more than 2Gi input, we
need to handle the input in chunks because of the `int` parameter in the
Windows API. Testing also revealed some corner cases we also need to
handle, which is mostly happening when there is an incomplete surrogate
pair at the end of the current input window.
The test takes between 3 (plain MinGW) and 8 (MSVC with ASAN) seconds
to run on my machine.
Pick-to: 6.7 6.6 6.5
Fixes: QTBUG-105105
Change-Id: I4fb0420b88ca41dfa8b561a35c6d96659bd81468
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
To properly support more than 2Gi input we have to support being asked
to resize more than once. Previously we would only have to resize the
one time because we went from our 4K stack buffer to the final size
heap buffer. But now, since our input size can only be specified in
int, we have to deal with looping over the input and resizing the buffer
as needed.
We also have to deal with trailing data at the end of our sliding window
potentially causing issues for the encoding. So we try to shrink our
window when it causes issues, or store the trailing data for the next
call.
The >2Gi test takes about 6-8 seconds on my machine.
Pick-to: 6.7 6.6 6.5
Task-number: QTBUG-105105
Change-Id: I9a44b8f379bf2c2c58183f961544ed2f4c8c7215
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Qt currently has custom implementation for Indic grapheme breaks.
QTBUG-121907 was created to evaluate whether it is better to use
Unicode implementation instead.
Remove BLACKLIST. The remaining tests pass.
Fixes: QTBUG-121529
Task-number: QTBUG-121907
Change-Id: Ifd8fd76aeeba51d08c9f23b867abbf39750fc7d9
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Add enumerator for the new Unicode version to QChar::UnicodeVersion.
Remap new line breaking classes to their Unicode 15.0 values:
* AK, AP and AS to AL,
* VI and VF to CM.
These are classes for new line breaking support for Indic scripts
that require more work.
Blacklist failing tests for now:
* tst_QUrlUts46::idnaTestV2
* tst_QTextBoundaryFinder::lineBoundariesDefault
* tst_QTextBoundaryFinder::graphemeBoundariesDefault
Regenerate the source files.
Task-number: QTBUG-121529
Change-Id: I869cc9fbaa53765d8ae6265c22cdbef9f19d05bf
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
According to QUIP-18 [1], all tests file should be
LicenseRef-Qt-Commercial OR GPL-3.0-only
[1]: https://contribute.qt-project.org/quips/18
Pick-to: 6.7
Task-number: QTBUG-121787
Change-Id: I9657df5d660820e56c96d511ea49d321c54682e8
Reviewed-by: Christian Ehrlicher <ch.ehrlicher@gmx.de>
QCOMPARE() can report enum values by name just fine, no need to
laboriously convert them to strings. While comparing all tags in one
go did allow a more comprehensive report, it's enough to know we
failed; this is testing cross-platform code, so a debugger can tell us
all those extra details if we get a failure.
Testing qHash() doesn't distinguish equal things is fairly low value;
at least avoid duplicating the construction of the reference value.
Replace a bunch of other QVERIFY()s with the new cousins of QCOMPARE()
for ordered and different comparisons.
In the process, mark some of the QLocale objects as const.
Change-Id: Ic93b8ed60c6f2cc846fbba428983778896d61291
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
They were expanding as simple blocks, so their uses didn't end in
semicolon, which looks wrong when reading the code.
Pick-to: 6.7 6.6 6.5
Change-Id: Ibea7b01ac165045604b6eb7a838765b2061c368a
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
(This turns out to be identical to v44, for our purposes.)
The CLDR license has been revised at v44 to "UNICODE LICENSE V3",
which is now included (as LICENSES/UNICODE-3.0.txt) in addition to the
old license (still in use, presumably, by UCD - at least until its
next update). Some new QLocale::Language entries are needed. There is
no change to the time-zone data.
Some tests needed changes:
* Various Arabic locales now use U+0623 (Arabic letter aleph with
hamza above) in exponent separator, replacing plain U+0627 (Arabic
letter aleph); it is still followed by U+0633 (Arabic letter seen).
* Where likely sub-tags used to fill in world, 001, as territory for a
language, they now (e.g. for Prussian and Yiddish) give specific
countries.
* Tamil locales now have something of a mix of inherited and localized
forms for AM/PM, which looks a lot like a mistake in CLDR.
* New likely sub-tag rules fix ctor(und_US) and ctor(und_GB), which
previously failed.
[ChangeLog][Third-Party Code] Updated QLocale's data extracted from
the Unicode Common Locale Data Repository (CLDR) to v44.1. The license
changed to Unicode License V3.
Pick-to: 6.7 6.6 6.5
Fixes: QTBUG-121485
Task-number: QTBUG-121325
Change-Id: Ide1a68016129526d7a5aa3fc67f1a674858696bc
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Upgrading emscripten to 3.1.50 breaks this test, so we disable the time
for time being. After emscripten update this test is to be enabled
and fixed.
Change-Id: Ic48d81e2285ed8f7639bf20c6c29b2b9e402a591
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
These exhibit the problem described by a recent bug report.
Task-number: QTBUG-121485
Change-Id: Ia09acfa22e687ba096091a73f30df1ffd22a6e32
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
There were three things going on in this test (itself a sufficient
reason to split it up):
* Some reporting and checking of the default locale; the reporting is
duplicated in defaulted_ctor() and the check fitted more naturally
there.
* Checks that various combinations of language, script and territory
got resolved according to likely-subtag rules. These were handled
via a macro and natural candidates to become data-driven.
* A test that territory is preserved when it's the only given tag
(with a few known exceptions); broken out as a steparate test.
In the process, give the data-rows of the likely-subtag parts names
that let me extend their testing to also test construction from
string. The territory-only cases can't support that, as QLocale
doesn't support und_* forms of tags for unspecified language.
Change-Id: Id9f0fc46f30eb887b47931bad1619255acb44266
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Replace various QFETCH()/QCOMPARE() pairs with QTEST().
Just because it's terser.
Change-Id: I8496a293e3634991dcb33b8c7939f1c3028a63c5
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Actually triggered by investigating QTBUG-121485, where the issue
turns out to be at construction time, but the complete lack of any
testing of endonyms clearly needed addressed in any case. In the
process, break up the long list of private slots in the test-class
declaration.
Task-number: QTBUG-121485
Change-Id: I49021f78d3bea2e1e55b2755a45943ab3fc23722
Reviewed-by: Øystein Heskestad <oystein.heskestad@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Every test row was following its locale-specific test with tests for
Irish and Greek day names, repeated regardless of the locale-specific
data. Express these tests as rows in their own right and shorten the
per-row testing, so we only do each once.
Change-Id: I8f919b50ac54423bacab6e5a9d34254b7db59a55
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
In the process, change it to use QCOMPARE(A, B) rather than QVERIFY(A == B).
Change-Id: I26c64100b08aef43c56f5266c0de71f5cde12816
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
A null QString / QStringView has a null begin pointer stored as its
array beginning (something we hide a little in the QString::data()
function, but not in QStringView::data()). We've been passing a null
pointer to memcpy() every time someone passed a null QStringView for
QString's single-argument arg() call, though not the multi-string arg()
version (which is the only one QStringView offers).
Commit f5021835df made this worse by
making QStringViews created from null QStrings retain the nullness (as
was intended).
Fixes: QTBUG-120624
Pick-to: 6.2 6.5 6.6 6.7
Change-Id: I6e2677aad2ab45759db2fffd17a870639b19340b
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Commit ba5db13c8d ("QStringView: add
internal qToStringViewIgnoringNull()") said:
As long as a null QString still returns non-null data(), QStringView's
QString constructor needs to call isNull(). That's a branch we often
can do without, so add an internal function that bypasses this
correctness check.
It's internal, since we might have a Q6String that returns nullptr
data() when null, which will remove the need for this function.
For Qt 6, we made QString and QByteArray be able to return nullptr from
data() when null... but that's not enabled by default yet. However, the
begin() functions do return nullptr, so we can avoid the extra branch
the commit was talking about.
Task-number: QTBUG-119750
Change-Id: Ica7a43f6147b49c187ccfffd179e4cf032bc8565
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
A default-constructed QLocale gets initialized with the
currently-active default locale, and apparently retains that setting
henceforth. That is why the `prior` member is not explicitly
initialized, which is confusing at face value.
Explain the mechanism better, and explicitly default-initialize the
member, so the next reader of the code saves the time to research
this.
Amends 76dfda1ad1.
Pick-to: 6.6 6.5 6.2 5.15
Change-Id: I1d1171f8564c70a971938b92b809f63ba5637d3a
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The (internal) docs say that - and _ are ignored, and they're ignored
everywhere, except as suffixes. If the old code only ignored them as
infixes, fine, that would make some sense, but it ignored infixes and
prefixes, so there's no reason for it to not ignore suffixes, too.
Fix by continuing the loop until both input ranges are exhausted, or a
mismatch was found.
[ChangeLog][QtCore][QStringConverter] Fixed a bug where
encodingForName() failed due to trailing characters (`_`, `-`) that
ought to have been ignored.
Pick-to: 6.6 6.5
Change-Id: Iec21489d988eda7d33c744c170f88cd665b73f34
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Add more tests and DRY the code using the usual row() lambda trick.
Pick-to: 6.6 6.5
Change-Id: I810fba0ab2a96e740e67392155f9507675fe57ae
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
The nameMatch() function has an implicit precondition that neither
argument is nullptr: it immediately dereferences both arguments.
Prevent the crash by checking for name == nullptr early, before
passing to nameMatch().
Add tests for null and empty.
As a drive-by, make variables in the test const (needed for the
QByteArray to avoid detaching, peer pressure for the others).
Amends a639bcda1e.
Pick-to: 6.6 6.5 6.2
Change-Id: I4a30f6c130310eb701ba7c7251168294489c34db
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
By just setting state to nullptr.
Pick-to: 6.6 6.5
Task-number: QTBUG-105105
Change-Id: I6b4f8fe39f1ba51dcfaf98ce7e42c2acd4c4cf98
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The win32 API doesn't give us much choice. _Some_ code pages have
support for returning some error if we pass a specific flag, but not
all of them.
Anyway, since the code pages might not support all that UTF-16 provides,
we can't reasonably make it error out on characters that cannot be
converted.
So, the most reasonable thing we can handle is a unpaired high surrogate
at the end of a string, assume that the rest of the string was fine, and
that the low surrogate will be provided in the next call.
Pick-to: 6.6 6.5
Fixes: QTBUG-118185
Task-number: QTBUG-105105
Change-Id: I1f193c9d8e04bec769d885d32440c759d9dff0c2
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Both to store and to restore.
Without this a 3 or more octet sequence would cause errors or wrong
output. This can be seen with GB 18030.
Pick-to: 6.6 6.5
Fixes: QTBUG-118318
Task-number: QTBUG-105105
Change-Id: Id1f7f5f2fba4633b9f888add2186f4d8d21b7293
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
A few code pages do not support this flag[0]. It's also deprecated[1]
and is what Windows prefers to generate by default. So let's drop it.
[0] https://learn.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-multibytetowidechar
See note at the end for the dwFlags parameter.
[1] It's mentioned in the header files, but not online...
Pick-to: 6.6 6.5
Task-number: QTBUG-118185
Task-number: QTBUG-105105
Change-Id: I798c387170c73a953be874de139868543b2d775e
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Mårten Nordheim
Ivan Solovev
Marc Mutz
Ahmad Samir
Isak Fyksen
Ievgenii Meshcheriakov
Tasuku Suzuki
Lucie Gérard
Edward Welbourne
Piotr Wierciński
Thiago Macieira
Kai Köhne
Anton Kudryavtsev