There was a QStringView::compare(QString, CaseSensitivity) but it's
good that the symmetric also exists
Change-Id: Ic789f11d41eb8cfa393cb51c19bd1f89bb87d912
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
When starting further into the string than index 0, do correct the
length too. This shows up in tst_qurl and tst_qurlinternal.
Kindly pointed out by ASAN:
==5513==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600002bf27 at pc 0x000100654de3 bp 0x7ffeefbfad10 sp 0x7ffeefbfad08
READ of size 16 at 0x60600002bf27 thread T0
#0 0x100654de2 in QtNS::simdTestMask(char const*&, char const*, unsigned int) qstring.cpp:395
#1 0x1005f9777 in QtNS::isAscii(QtNS::QChar const*&, QtNS::QChar const*) qstring.cpp:491
#2 0x100638642 in QtNS::qt_string_normalize(QtNS::QString*, QtNS::QString::NormalizationForm, QtNS::QChar::UnicodeVersion, int) qstring.cpp:7999
Change-Id: I44ad65b47eb98c6085c77b56dc2da50ef5659d25
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The character 'g' is only a valid format when converting numbers
to strings, but not other way round.
Change-Id: Ie772886e7a45a5067c0a3e4eaa3a6ccef8e69426
Reviewed-by: Martin Smith <martin.smith@qt.io>
The character search in the findChar() static function in qstring.cpp is
more efficient than what we had in qurlrecode.cpp and there's no point
in duplicating it. It also has a Neon implementation. So make the
implementation available for use in QtPrivate::qustrchr().
This also simplifies the implementation.
Change-Id: Ib48364abee9f464c96c6fffd152eedd0cd8ad7f8
Reviewed-by: Samuel Gaist <samuel.gaist@idiap.ch>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
It's unclear when the reference to Unicode 4.0 standard
got added - it certainly predates the qt 4 git repository.
Anyhow, nowadays we're using later versions, and it doesn't
make much sense to highlight one specific version here.
Instead, use the correct technical term - UTF-16 code unit.
Also I added a 'correspond _to_', which sounds more common
to me.
Task-number: QTBUG-56699
Change-Id: I4bdcd9060cb2b11521638019c15ef7ab67aa768b
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
Similar to commit cd64a96b31 we also need
to normalize the strings before comparison in order to be compliant with
the ECMAScript test suite.
This patch also adds the remaining test cases from
built-ins/String/prototype/localeCompare/15.5.4.9_CE.
Since the same tests are also failing with strcoll/qt_compare_strings,
this simplifies the code to always normalize except when using ICU
(which gets it right by default).
Change-Id: I16b32da7fc70dc7e6725c49f66fe9941d0bf3a47
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
The compiler was generating some vectorized code for qresource.cpp but
it wasn't very efficient. So improve upon it and make use in other
places where we read UTF-16BE strings.
[ChangeLog][QtCore] Added an overload of q{To,From}{Big,Little}Endian
that operates on a memory region.
Change-Id: I6a540578e810472bb455fffd1531fa2f1d724dfc
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
With ICU and on macOS it appears that the comparison is done on a
canonical form, while CompareString(Ex) does not do that, as the added
test verifies. Explicit normalization fixes that.
As a bonus, this also unifies the code path between regular Windows
and UWP by unconditionally using CompareStringEx (which requires
Vista or later).
This issue surfaced while running the ECMASCript 6 Conformance Test
Suite in QtQml.
This re-uses the existing test for localeAwareCompare, which was
disabled on Windows, macOS and Linux with ICU (the common case).
Change-Id: I52440fce60b54745ead1eff005ec51e98e2a79ec
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
Need to do the same for startsWith() and endsWith(). indexOf() is a lot
harder.
[ChangeLog][QtCore][QByteArray] Added compare(), which takes
Qt::CaseSensitivity as one of the parameters. This function is more
efficient than using toLower() or toUpper() and then comparing.
Change-Id: Ib48364abee9f464c96c6fffd152e69bde4194df7
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
While at it, fix some more issues in the sentences to
harmonize the description between the different classes.
Change-Id: Iee1c3ffe6fd71e82504bfb003d927c4db3b2a065
Reviewed-by: Martin Smith <martin.smith@qt.io>
Both use QLocale::toDouble behind the scenes, so the same limitations
apply. Document them for toFloat() also.
Change-Id: I954362a0db203630685c034df6a921fa6447a509
Reviewed-by: Martin Smith <martin.smith@qt.io>
Failed on my Sandybridge Mac with -march=native:
qstring.cpp:363:19: error: redefinition of 'mask' with a different type: 'const __m128i'
Change-Id: I6efb28c3145047559ec0fffd15386aeb8d36d681
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Commit 5dc1e08c8c changed
QString::toLocal8Bit_helper to use qt_convert_to_latin1 so it became
unused. It was never used in inline functions in the headers.
Change-Id: I6efb28c3145047559ec0fffd15382f9d08efdfeb
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
QString::sprintf(), like the C printf-family, always includes two
digits in any exponent it outputs. Up to 5.6, number() and arg()
taking a double did the same; but changes at 5.7 to enable opting out
of the leading zero this implies for a single-digit exponent
accidentally opted out of it in args() and number(). This commit
fixes number() and arg() to include QLocaleData::ZeroPadExponent in
the flags they pass down to the C locale's doubleToString(), restoring
the prior behavior, including consistency with sprintf().
[ChangeLog][QtCore][QString] Formatting of doubles with single-digit
exponent, by number() or args(), now includes a leading zero in that
exponent, consistently with sprintf(), as it did up to 5.6.
Task-number: QTBUG-63620
Change-Id: I10c491902b8556e9f19e605177ead8d9fd32abd9
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
Turns out that the non-AVX2 code was beating the performance of the AVX2
because the simdTestMask function did a little too much. So just use the
same VPMOVMSKB technique for it.
Change-Id: I0825ff5b5f6f4c85939ffffd152f3b636ab998db
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Instead of two 16-byte loads, if AVX2 is present. Otherwise, it's
exactly the same.
Because of the way the SIMD instructions were extended to 256-bit in
AVX2, we gain nothing doing two 32-byte loads, aside from the loop
unrolling.
Change-Id: Ib48364abee9f464c96c6fffd152e531925814ac2
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
This instruction is somewhat slow and requires a lot of inputs to be
correctly set. Instead, use the PMIN trick, which does have unsigned
comparison support.
This commit moves the helper function to a lambda inside qt_to_latin1,
to make it easier to reuse the constants in the next commit and to avoid
warnings of unused static functions.
Change-Id: Ib48364abee9f464c96c6fffd152e9e84f4ad3ae8
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
If the input is already known to be Latin 1, we don't need to check and
merge in question marks. QJsonObject already needed this code, now we
can make it more efficient.
I'll need the same code in CBOR.
Change-Id: Ib48364abee9f464c96c6fffd152e508f078404e5
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
We don't have _mm_cvtsi64_si128() (the REX.W expansion of MOVD [0F 6E]),
but we do have _mm_loadl_epi64(), the SSE2 expansion of the MMX MOVQ at
opcode 0F 7E. Ditto for _mm_cvtsi128_si64() and _mm_storel_epi64(). And
those work even in 32-bit mode. By doing this, we can reduce the tail
unrolled loops by half, reducing code size.
I'm not adding these new SIMD sections to -Os builds.
Change-Id: Ib48364abee9f464c96c6fffd152e405310ef67be
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Improves performance a little. This is just because I can and the
function is right there for the taking, as this qt_urlRecodeByteArray
function is only used in deprecated QUrl code.
Change-Id: I5d0ee9389a794d80983efffd152d290e570af387
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
We can just reuse the code I added for QtPrivate::isAscii(), adding the
update to the ptr parameter in the failed case.
Change-Id: I5d0ee9389a794d80983efffd152d277e2adf444d
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
First of all, this removes the UB that used to try and calculate the
distance between the two strings. That's a valid technique in assembly,
but dangerous in C++ and totally unnecessary. The compiler is perfectly
able to generate loops with a single induction variable all on its own.
Second, this commit makes the main loop use 32-byte comparisons (16
characters at a time), which is a reasonable size for strings. We use
AVX2 if that's available, or an unrolled pair of 16-byte loads
otherwise. After the existing 16-byte comparison, this commit inserts an
8-byte (4-character) comparison and then reduces the final, unrolled
comparison to just 3 characters.
Change-Id: Ib48364abee9f464c96c6fffd152e474b39e1f293
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
The check for having N valid characters is ptr + N <= end, because
ptr + N == end indicates that we have exactly N characters in the
string.
Change-Id: I5d0ee9389a794d80983efffd152d28d5aa485ce4
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Use QT_CONFIG(regularexpression), disentangle it from QT_BOOTSTRAPPED,
switch it off in the bootstrap build, remove the #ifdefs from
qregularexpression.{h|cpp}, and add QT_REQUIRE_CONFIG(regularexpression)
to the header.
qregularexpression.{h|cpp} are already correctly excluded in tools.pri
if !qtConfig(regularexpression).
Change-Id: I21de154a6a118b76f99003d3acb72ac1e220d302
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
[ChangeLog][QtCore][QString] Added the functions QString::isUpper
and QString::isLower to check if a string contains only uppercase
or only lowercase letters.
Change-Id: I12d3a47d4605eb4514842071e80a9ba0723d4e01
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Commit 8f52ad9fe0 ("ucstricmp: compare
null and empty strings equal") made sure empties and nulls would compare
equally, but may have broken the null vs non-empty comparison (which was
not tested). The commit message also said that it expected all callers
to handle null before calling into those functions, but that's not the
case for QStringView created from a null QString: the incoming "a"
pointer was null.
So just remove the checks for null pointers and rely on the size checks
doing the right thing.
[ChangeLog][QtCore][QString] Fixed a regression from 5.9 that caused
comparing default-constructed QStrings to be sorted after non-empty
strings.
Task-number: QTBUG-65939
Change-Id: I56b444f9d6274221a3b7fffd150c83ad46c599b6
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
In remove(const QString &str, Qt::CaseSensitivity cs)
call remove(QChar c, Qt::CaseSensitivity cs)
if str.size() is equal 1. It prevents quadratic behavior
for that case.
Change-Id: I9a7ab3019c580343533c8c6c6a04b6b0c8c1fb55
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
isLatin1(QLatin1String) is provided for completeness sake, in case some
generic code operates on both QLatin1String and QString/QStringView.
Change-Id: I5e421e32396d44e4b39efffd150b99a18eedf648
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Both ICC and MSVC have bugs concatenating strings and that's despite the
standard giving an example ([lex.string] table 9) that says that
u"a" "b"
is the same as
u"ab"
We can't change to preprocessor concatenation (u ## STR) because that
fails when QStringLiteral is used with a macro instead of an actual
string literal.
Task-number: QTBUG-65479
Change-Id: I39332e0a867442d58082fffd1504a0a868c291ef
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Added missing template clause to \fn commands required by clang-qdoc.
Removed several obsolete \sa targets. Added a few missing return types
to \fn commands.
Also changed a } to ] in a \fn command in qrandom.cpp.
Change-Id: I0f004a1e57507ff5fb035e822ca8b1baa7faad9b
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
Make sure we properly handle both directional embeddings as well as
directional isolates to determine the direction of the string.
According to the latest version of the Unicode bidi algorithm,
parts of the string contained inside an directional isolate is
to be ignored when determining the paragraph direction. Embedding
markers themselves are to be ignored as well, but not the characters
inside an explicit directional embedding or override.
This is also some required pre-work to get our BiDi algorithm
updated to the latest version of the standard.
Move the implementation to QStringView and implement the methods
in QString and QStringRef through that implementation.
Task-number: QTBUG-57743
Change-Id: I7f24e09198e22d6359c6534c9ae40a904e94c46e
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
Samuel Gaist
Albert Astals Cid
Liang Qi
Qt Forward Merge Bot
Erik Verbruggen
Andre Hartmann
Thiago Macieira
Kai Koehne
Simon Hausmann
Edward Welbourne
Ulf Hermann
Ville Voutilainen
Anton Kudryavtsev
Martin Smith
Lars Knoll