It's the same aeshash() as before, except we're passing a template
parameter to indicate whether to read half and then zero-extend the
data. That is, it will perform a conversion from Latin1 on the fly.
When running in zero-extending mode, the length parameters are actually
doubled (counting the number of UTF-16 code units) and we then divide
again by 2 when advancing.
The implementation should have the following performance
characteristics:
* QLatin1StringView now will be roughly half as fast as Qt 6.7
* QLatin1StringView now will be roughly as fast as QStringView
For the aeshash128() in default builds of QtCore (will use SSE4.1), the
long loop (32 characters or more) is:
QStringView QLatin1StringView
movdqu -0x20(%rax),%xmm4 | pmovzxbw -0x10(%rdx),%xmm2
movdqu -0x10(%rax),%xmm5 | pmovzxbw -0x8(%rdx),%xmm3
add $0x20,%rax | add $0x10,%rdx
pxor %xmm4,%xmm0 | pxor %xmm2,%xmm0
pxor %xmm5,%xmm1 | pxor %xmm3,%xmm1
aesenc %xmm0,%xmm0 aesenc %xmm0,%xmm0
aesenc %xmm1,%xmm1 aesenc %xmm1,%xmm1
aesenc %xmm0,%xmm0 aesenc %xmm0,%xmm0
aesenc %xmm1,%xmm1 aesenc %xmm1,%xmm1
The number of instructions is identical, but there are actually 2 more
uops per iteration. LLVM-MCA simulation shows this should execute in the
same number of cycles on older CPUs that do not have support for VAES
(see <https://analysis.godbolt.org/z/x95Mrfrf7>).
For the VAES version in aeshash256() and the AVX10 version in
aeshash256_256():
QStringView QLatin1StringView
vpxor -0x40(%rax),%ymm1,%ym | vpmovzxbw -0x20(%rax),%ymm3
vpxor -0x20(%rax),%ymm0,%ym | vpmovzxbw -0x10(%rax),%ymm2
add $0x40,%rax | add $0x20,%rax
| vpxor %ymm3,%ymm0,%ymm0
| vpxor %ymm2,%ymm1,%ymm1
vaesenc %ymm1,%ymm1,%ymm1 <
vaesenc %ymm0,%ymm0,%ymm0 vaesenc %ymm0,%ymm0,%ymm0
vaesenc %ymm1,%ymm1,%ymm1 vaesenc %ymm1,%ymm1,%ymm1
vaesenc %ymm0,%ymm0,%ymm0 vaesenc %ymm0,%ymm0,%ymm0
> vaesenc %ymm1,%ymm1,%ymm1
In this case, the increase in number of instructions matches the
increase in number of uops. The LLVM-MCA simulation says that the
QLatin1StringView version is faster at 11 cycles/iteration vs 14 cyc/it
(see <https://analysis.godbolt.org/z/1Gv1coz13>), but that can't be
right.
Measured performance of CPU cycles, on an Intel Core i9-7940X (Skylake,
no VAES support), normalized on the QString performance (QByteArray is
used as a stand-in for the performance in Qt 6.7):
aeshash | siphash
QByteArray QL1SV QString QByteArray QString
dictionary 94.5% 79.7% 100.0% 150.5%* 159.8%
paths-small 90.2% 93.2% 100.0% 202.8% 290.3%
uuids 81.8% 100.7% 100.0% 215.2% 350.7%
longstrings 42.5% 100.8% 100.0% 185.7% 353.2%
numbers 95.5% 77.9% 100.0% 155.3%* 164.5%
On an Intel Core i7-1165G7 (Tiger Lake, capable of VAES and AVX512VL):
aeshash | siphash
QByteArray QL1SV QString QByteArray QString
dictionary 90.0% 91.1% 100.0% 103.3%* 157.1%
paths-small 99.4% 104.8% 100.0% 237.5% 358.0%
uuids 88.5% 117.6% 100.0% 274.5% 461.7%
longstrings 57.4% 111.2% 100.0% 503.0% 974.3%
numbers 90.6% 89.7% 100.0% 98.7%* 149.9%
On an Intel 4th Generation Xeon Scalable Platinum (Sapphire Rapids, same
Golden Cove core as Alder Lake):
aeshash | siphash
QByteArray QL1SV QString QByteArray QString
dictionary 89.9% 102.1% 100.0% 158.1%* 172.7%
paths-small 78.0% 89.4% 100.0% 159.4% 258.0%
uuids 109.1% 107.9% 100.0% 279.0% 496.3%
longstrings 52.1% 112.4% 100.0% 564.4% 1078.3%
numbers 85.8% 98.9% 100.0% 152.6%* 190.4%
* dictionary contains very short entries (6 characters)
* paths-small contains strings of varying length, but very few over 32
* uuids-list contains fixed-length strings (38 characters)
* longstrings is the same but 304 characters
* numbers also a lot contains very short strings (1 to 6 chars)
What this shows:
* For short strings, the performance difference is negligible between
all three
* For longer strings, QLatin1StringView now costs between 7 and 17% more
than QString on the tested machines instead of up to ~50% less, except on
the older machine (where I think the main QString hashing is suffering
from memory bandwidth limitations)
* The AES hash implementation is anywhere from 1.6 to 11x faster than
Siphash
* Murmurhash (marked with asterisk) is much faster than Siphash, but it
only managed to beat the AES hash in one test
Change-Id: I664b9f014ffc48cbb49bfffd17b045c1811ac0ed
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
So that it hashes to the same value as QString{,View}.
In order to test this, you must either run on a CPU other than ARM and
x86, or disable the AES hasher. I did that and can confirm siphash and
murmurhash do work with on-the-fly conversion from Latin-1.
Change-Id: I664b9f014ffc48cbb49bfffd17b03e5e62ec4e89
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This is supposed to be a no-op change: I simply split the three phases
of the siphash algorithm into separate functions. This will be needed
for hashing of QLatin1StringView equal to QString's.
Drive-by slight modernization of the code too and made the 32-bit code
be compiled (but not used) on 64-bit, so we don't accidentally let it
bit-rot.
Change-Id: I664b9f014ffc48cbb49bfffd17b03d10c8bd55b2
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
If we add the typedefs, we also need to add the qHash() overloads. See
code comments for the explanation of how it works. I chose to use an
addition to merge the upper and lower parts because of the comment in
QHashCombineCommutative's operator(), introduced by commit
91b44afdcb.
Found in 6.6 API-review, but didn't make the cut and then was
forgotten.
Drive-by fix long line nearby.
[ChangeLog][QtCore] Added qHash() overloads for quint128 and qint128.
Fixes: QTBUG-116054
Task-number: QTBUG-116080
Change-Id: If484aed08ba476e0eace800b719f435203100f3e
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Commit 8179d7edf6 added the check in two
places. Move to a central one, disabling the bits in
qCompilerCpuFeatures.
Change-Id: I50e2158aeade4256ad1dfffd17b117afcb93c126
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
It wasn't getting inlined in aeshash256_lt32_avx256() (used by VAES +
AVX512VL variant) due to a GCC __attribute__((target())) mismatch,
causing a major loss of performance compared to the VAES + AVX2 variant.
Comparing the throughput after this fix on an Intel Core i7-1165G7
(Tiger Lake) laptop, with qHashBits modified to statically select either
[A] aeshash256() or [B] aeshash256_avx256(), out of 5 runs:
dictionary numbers paths-small uuids longstrings
A/B (avg) 103.7% 101.1% 103.5% 104.5% 100.3%
A/B (best) 103.4% 100.9% 103.2% 103.6% 100.8%
Considering that a string representation of a UUID is 37 characters (74
bytes), neither "uuids" nor "longstrings" are directly affected by this
change. However, the overhead does change, with the aeshash256_avx256()
needing slightly fewer instructions to reach aeshash256_ge32().
Benchmarking on an Intel Xeon Scalable 4th Generation (Sapphire Rapids),
the "uuids" data set has a 10% performance loss for some reason.
Pick-to: 6.5 6.6 6.7
Change-Id: I50e2158aeade4256ad1dfffd17b1b105d3cab482
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Commit c0791ac76e didn't explain why it
was #ifdef'ed out. It's just an alias for double. Maybe compilers at the
time used to complain if you used it, but I can't make Apple's clang
produce a warning now.
Pick-to: 6.7
Change-Id: I664b9f014ffc48cbb49bfffd17b02293403e9571
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
It appears to be used only in qlalr, which is, however, not
bootstrapped.
Pick-to: 6.7 6.6 6.5 6.2
Change-Id: Idc16d957bf687238c7b0ee603d8b092e2048ef18
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QVersionNumber, e.g., was added for Qt 5.6, the last Qt version that
didn't require C++11. So it made sense that the original documentation
stated that certain functions were only available in C++11 mode.
But already Qt 5.7 required C++11, so these historical anecdotes are
no longer pertient to today's Qt users, so remove them from the docs.
Pick-to: 6.7 6.6 6.5 6.2 5.15
Change-Id: I5c732d3b9b33e1fb6947eff4fac546476c8379f2
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Also port from qMakePair to just braced initialization using CTAD.
Task-number: QTBUG-115841
Pick-to: 6.7
Change-Id: Ib0ad55d7110521e34004dc9050022f9c0046722e
Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
When QDoc parses a project, it parses the source code to extract the
user-provided documentation and perform sanity checkings based on the
code itself on it.
When QDoc parses an "\fn" command as part of this process, it tries to
understand, based on its intermediate representation built on the
information extracted from the code-base, which "documentable element"
the "\fn" refers to.
When QDoc performs this "matching" process, it takes into consideration
only a certain amount of information.
For example, no checking is performed over the template declaration of a
callable.
Due to some upcoming documentation, where two callables are
indistinguishable to the current process, as they differ only in their
template declaration, QDoc will start to take into consideration the
template declaration of a callable when matching.
This implies that an "\fn" command should now provide information
parity, with regards to template declaration for callables, with the
code-base so that QDoc can perform the match correctly.
The documentation for `qHash(std::nullptr_t, size_t)` was providing a
unnecessary template declaration.
Hence, the incorrect additional information was removed from the `\fn`
command.
Task-number: QTBUG-118080
Change-Id: Iea406abbeb7aabd66e5b2ab092212d56b03cf42f
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
Synchronize the documentation of the four container classes:
- document the return type of insert() and replace()
- don't reference QMultiHash/Map from QHash/Map except in the details
paragraph
Task-number: QTBUG-117757
Change-Id: I93ee7eec0c298854e05e83a43f1c7cffd0610d72
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
- Bring iterator loops into a consistent form using auto and
creating and end variable, use cbegin()/cend() where suitable
- Use (std::)endl instead of Qt::endl for iostreams
- Fix removed container conversion API (QList::fromSet, QSet::toList())
- Use range-based for instead of foreach
- Use initializer lists
- Use qPrintable(QString) for output to std::ostream
- Use qsizetype
- Remove some unused snippets
Complements f6b137bdc4.
Pick-to: 6.5
Change-Id: I8a167099cdb224f45b984fa834d46269144a7ef0
Reviewed-by: Christian Ehrlicher <ch.ehrlicher@gmx.de>
Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
QHash/MultiHash & QMap/MultiMap::erase() do no longer take an iterator
in Qt6. Clean up the examples by only providing one example, the rest is
common c++ which should not be handled in the Qt documentation. Also
mention erase_if() and remove references to the (soon to be deprecated)
Java-style iterators.
Pick-to: 6.5 6.2
Fixes: QTBUG-105109
Change-Id: I47b11f3b5dcc173494e5c6f9ad0167c613b12209
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
It was confusing entry capacity with the bucket capacity. The value
maxNumBuckets() returned was the maximum number of entries. This issue
was harmless: we would just fail to cap the maximum to an allocatable
size. But the array new[] in the Data constructors would have capped the
maximum anyway (by way of throwing std::bad_alloc).
So instead of trying to calculate what the maximum bucket count is so we
can cap at that, simplify the calculation of the next power of 2 while
preventing it from overflowing in our calculations. We continue to rely
on new[] throwing when we return count that is larger than the maximum
allocatable.
This commit changes the load factor for QHashes containing exactly a
number of elements that is exactly a power of two. Previously, it would
be loaded at 50%, now it's at 25%. For this reason, tst_QSet::squeeze
needed to be fixed to depend less on the implementation details.
Pick-to: 6.5
Change-Id: I9671dee8ceb64aa9b9cafffd17415f3856c358a0
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Rework QSimpleParsedNumber to store a qsizetype whose sign serves as
ok flag (positive is ok, zero and negative are not) and magnitude is
the number of characters used. This replaces an endptr that was set to
null to indicate !ok, but that deprived us of end-of-parse
information, which is needed for number-parsing. In particular, JS's
parsing of numbers accepts overflow (where qstrntod() flags it as
invalid) as infinity; so qstrntod() does need to say how long the
overflowing (but JS-valid, none the less) number-text was.
Modify all callers of functions using this (recently-introduced) type
and add tests that fail without this fix.
Fixes: QTBUG-108628
Change-Id: I416cd213e1fb8101b1af5a6d43615b970a5db9b4
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
That is, return everything in the return argument. On the SysV ABI, that
means everything gets returned in registers, in both 32- and 64-bit
platforms (unlike QtPrivate::ParsedNumber). There's a minor but
perceptible performance improvement in parsing strings and byte arrays.
Before:
Parsed string "42" "1234" "-1548860221"
Clock (ns) 16.673 18.878 25.517
CPU cycles 46.548 52.704 71.243
Instructions 201 233 331
After:
Parsed string "42" "1234" "-1548860221"
Clock (ns) 15.577 17.998 24.198
CPU cycles 43.491 49.942 67.552
Instructions 179 211 308
On my Core i7-1165G7 @ 2.80 GHz, the 22-23 instruction gain per
iteration results in half the expected clock gain in runtime (22 /
2.8 GHz = 7.8 ns) because of a slightly lower instruction per cycle
rate. That's acceptable because we need less speculative execution.
Pick-to: 6.4
Task-number: QTBUG-107788
Change-Id: I07ec23f3cb174fb197c3fffd17220fd64d473cc0
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
qhashfunctions.h defines a catch-all 2-arguments qHash(T, seed)
in order to support datatypes that implement a 1-argument overload
of qHash (i.e. qHash(Type)). The catch-all calls the 1-argument
overload and XORs the result with the seed.
The catch-all is constrained on the existence of such a 1-argument
overload. This is done in order to make the catch-all SFINAE-friendly;
otherwise merely instantiating the catch-all would trigger a hard error.
Such an error would make it impossible to build a type trait that
detects if one can call qHash(T, size_t) for a given type T.
The constraint itself is called HasQHashSingleArgOverload and lives in a
private namespace.
It has been observed that HasQHashSingleArgOverload misbehaves for
some datatypes. For instance, HasQHashSingleArgOverload<int> is actually
false, despite qHash(123) being perfectly callable. (The second argument
of qHash(int, size_t) is defaulted, so the call *is* possible.)
--
Why is HasQHashSingleArgOverload<int> false?
This has to do with how HasQHashSingleArgOverload<T> is implemented: as
a detection trait that checks if qHash(declval<T>()) is callable.
The detection itself is not a problem. Consider this code:
template <typename T>
constexpr bool HasQHashSingleArgOverload = /* magic */;
class MyClass {};
size_t qHash(MyClass);
static_assert(HasQHashSingleArgOverload<MyClass>); // OK
Here, the static_assert passes, even if qHash(MyClass) (and MyClass
itself) were not defined at all when HasQHashSingleArgOverload was
defined.
This is nothing but 2-phase lookup at work ([temp.dep.res]): the
detection inside HasQHashSingleArgOverload takes into account the qHash
overloads available when HasQHashSingleArgOverload was declared, as well
as any other overload declared before the "point of instantiation". This
means that qHash(MyClass) will be visible and detected.
Let's try something slightly different:
template <typename T>
constexpr bool HasQHashSingleArgOverload = /* magic */;
size_t qHash(int);
static_assert(HasQHashSingleArgOverload<int>); // ERROR
This one *does not work*. How is it possible? The answer is that 2-phase
name lookup combines the names found at definition time with the names
_found at instantiation time using argument-dependent lookup only_.
`int` is a fundamental type and does not participate in ADL. In the
example, HasQHashSingleArgOverload has actually no qHash overloads to
even consider, and therefore its detection fails.
You can restore detection by moving the declaration of the qHash(int)
overload *before* the definition of HasQHashSingleArgOverload, so it's
captured at definition time:
size_t qHash(int);
template <typename T>
constexpr bool HasQHashSingleArgOverload = /* magic */;
static_assert(HasQHashSingleArgOverload<int>); // OK!
This is why HasQHashSingleArgOverload<int> is currently returning
`false`: because HasQHashSingleArgOverload is defined *before* all the
qHash(fundamental_type) overloads in qhashfunctions.h.
--
Now consider this variation of the above, where we keep the qHash(int)
overload after the detector (so, it's not found), but also prepend an
Evil class implicitly convertible from int:
struct Evil { Evil(int); };
size_t qHash(Evil);
template <typename T> constexpr bool HasQHashSingleArgOverload = /* magic */;
size_t qHash(int);
static_assert(HasQHashSingleArgOverload<int>); // OK
Now the static_assert passes. HasQHashSingleArgOverload is still not
considering qHash(int) (it's declared after), but it's considering
qHash(Evil). Can you call *that* one with an int? Yes, after a
conversion to Evil.
This is extremely fragile and likely an ODR violation (if not ODR, then
likely falls into [temp.dep.candidate/1]).
--
Does this "really matter" for a type like `int`? The answer is no. If
HasQHashSingleArgOverload<int> is true, then a call like
qHash(42, 123uz);
will have two overloads in its overloads set:
1) qHash(int, size_t)
2) qHash(T, size_t), i.e. the catch-all template. To be pedantic,
qHash<int>(const int &, size_t), that is, the instantiation of the
catch-all after template type deduction for T (= int)
([over.match.funcs.general/8]).
Although it may look like this is ambiguous as both calls have perfect
matches for the arguments, 1) is actually a better match than 2) because
it is not a template specialization ([over.match.best/2.4]).
In other words: qHash(int, size_t) is *always* called when the argument
is `int`, no matter the value of HasQHashSingleArgOverload<int>. The
catch-all template may be added or not to the overload set, but it's
a worse match anyways.
--
Now, let's consider this code:
enum MyEnum { E1, E2, E3 };
qHash(E1, 42uz);
This code compiles, although we do not define any qHash overload
specifically for enumeration types (nor one is defined by MyEnum's
author).
Which qHash overload gets called? Again there are two possible
overloads available:
1) qHash(int, size_t). E1 can be converted to `int` ([conv.prom/3]),
and this overload selected.
2) qHash(T, size_t), which after instantiation, is qHash<MyEnum>(const
MyEnum &, size_t).
In this case, 2) is a better match than 1), because it does not require
any conversion for the arguments.
Is 2) a viable overload? Unfortunately the answer here is "it depends",
because it's subject to what we've learned before: since the catch-all
is constrained by the HasQHashSingleArgOverload trait, names introduced
before the trait may exclude or include the overload.
This code:
#include <qhashfunctions.h>
enum MyEnum { E1, E2, E3 };
qHash(E1, 42uz);
static_assert(HasQHashSingleArgOverload<MyEnum>); // ERROR
will fail the static_assert. This means that only qHash(int, size_t) is
in the overload set.
However, this code:
struct Evil { Evil(int); };
size_t qHash(Evil);
#include <qhashfunctions.h>
enum MyEnum { E1, E2, E3 };
qHash(E1, 42uz);
static_assert(HasQHashSingleArgOverload<MyEnum>); // OK
will pass the static_assert. qHash(Evil) can be called with an object of
type MyEnum after an user-defined conversion sequence
([over.best.ics.general], [over.ics.user]: a standard conversion
sequence, made of a lvalue-to-rvalue conversion + a integral promotion,
followed by a conversion by constructor [class.conv.ctor]).
Therefore, HasQHashSingleArgOverload<MyEnum> is true here; the catch-all
template is added to the overload set; and it's a best match for the
qHash(E1, 42uz) call.
--
Is this a problem? **Yes**, and a huge one: the catch-all template does
not yield the same value as the qHash(int, size_t) overload. This means
that calculating hash values (e.g. QHash, QSet) will have different
results depending on include ordering!
A translation unit TU1 may have
#include <QSet>
#include <Evil>
QSet<MyEnum> calculateSet { /* ... */ }
And another translation unit TU2 may have
#include <Evil>
#include <QSet> // different order
void use() {
QSet<MyEnum> set = calculateSet();
}
And now the two TUs cannot exchange QHash/QSet objects as they would
hash the contents differently.
--
`Evil` actually exists in Qt. The bug report specifies QKeySequence,
which has an implicit constructor from int, but one can concoct infinite
other examples.
--
Congratulations if you've read so far.
=========================
=== PROPOSED SOLUTION ===
=========================
1) Move the HasQHashSingleArgOverload detection after declaring the
overloads for all the fundamental types (which we already do anyways).
This means that HasQHashSingleArgOverload<fundamental_type> will now
be true. It also means that the catch-all becomes available for all
fundamental types, but as discussed before, for all of them we have
better matches anyways.
2) For unscoped enumeration types, this means however an ABI break: the
catch-all template becomes always the best match. Code compiled before
this change would call qHash(int, size_t), and code compiled after this
change would call the catch-all qHash<Enum>(Enum, size_t); as discussed
before, the two don't yield the same results, so mixing old code and new
code will break.
In order to restore the old behavior, add a qHash overload for
enumeration types that forwards the implementation to the integer
overloads (using qToUnderlying¹).
(Here I'm considering the "old", correct behavior the one that one gets
by simply including QHash/QSet, declaring an enumeration and calling
qHash on it. In other words, without having Evil around before including
QHash.)
This avoids an ABI break for most enumeration types, for which one
does not explicitly define a qHash overload. It however *introduces*
an ABI break for enumeration types for which there is a single-argument
qHash(E) overload. This is because
- before this change, the catch-all template was called, and that
in turn called qHash(E) and XOR'ed the result with the seed;
- after this change, the newly introduced qHash overload for
enumerations gets called. It's very likely that it would not give
the same result as before.
I don't have a solution for this, so we'll have to accept the ABI
break.
Note that if one defines a two-arguments overload for an enum type,
then nothing changes there (the overload is still the best match).
3) Make plans to kill the catch-all template, for Qt 7.0 at the latest.
We've asked users to provide a two-args qHash overload for a very long
time, it's time to stop working around that.
4) Make plans to switch from overloading qHash to specializing std::hash
(or equivalent). Specializations don't overload, and we'd get rid of
all these troubles with implicit conversions.
--
¹ To nitpick, qToUnderlying may select a *different* overload than
the one selected by an implicit conversion.
That's because an unscoped enumeration without a fixed underlying type
is allowed to have an underlying type U, and implicitly convert to V,
with U and V being two different types (!).
U is "an integral type that can represent all the enumerator values"
([dcl.enum/7]). V is selected in a specific list in a specific order
([conv.prom]/3). This means that in theory a compiler can take enum E {
E1, E2 }, give it `unsigned long long` as underlying type, and still
allow for a conversion to `int`.
As far as I know, no compiler we use does something as crazy as that,
but if it's a concern, it needs to be fixed.
[ChangeLog][Deprecation Notice] Support for overloads of qHash with only
one argument is going to be removed in Qt 7. Users are encouraged to
upgrade to the two-arguments overload. Please refer to the QHash
documentation for more information.
[ChangeLog][Potentially Binary-Incompatible Changes] If an enumeration
type for which a single-argument qHash overload has been declared is
being used as a key type in QHash, QMultiHash or QSet, then objects of
these types are no longer binary compatible with code compiled against
an earlier version of Qt. It is very unlikely that such qHash overloads
exist, because enumeration types work out of the box as keys Qt
unordered associative containers; users do not need to define qHash
overloads for their custom enumerations. Note that there is no binary
incompatibity if a *two* arguments qHash overload has been declared
instead.
Fixes: QTBUG-108032
Fixes: QTBUG-107033
Pick-to: 6.2 6.4
Change-Id: I2ebffb2820c553e5fdc3a341019433793a58e3ab
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
To allow the user to customize the C++ code that QDoc sees, so as to be
able to work-around some limitations on QDoc itself, QDoc defines two
symbols: Q_QDOC and Q_CLANG_QDOC, both of which are "true" during an
entire execution of QDoc.
At a certain point in time, QDoc allowed the user the choice between a
custom C++ parser and a Clang based one.
The Q_QDOC symbol would always be defined while the Q_CLANG_QDOC symbol
would be defined only when the Clang based parser was chosen.
In more recent times, QDoc always uses a Clang based parser, such that
both Q_CLANG_QDOC and Q_QDOC are always defined, making them equivalent.
To avoid using different symbols, and the possible confusion and
fragmentation that derives from it, all usages of Q_CLANG_QDOC are now
replaced by the equivalent usages of Q_QDOC.
Change-Id: I5810abb9ad1016a4c5bbea99acd03381b8514b3f
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
By not splitting Q_UNREACHABLE() and the following return, we can get
rid of the NOLINT(qt-use-unreachable-return).
Change-Id: I3322843e38dabdadb38eea38a6d91b301257fd23
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This is a combination of Q_UNREACHABLE() with a return statement.
ATM, the return statement is unconditionally included. If we notice
that some compilers warn about return after __builtin_unreachable(),
then we can map Q_UNREACHABLE_RETURN(...) to Q_UNREACHABLE() without
having to touch all the code that uses explicit Q_UNREACHABLE() +
return.
The fact that Boost has BOOST_UNREACHABLE_RETURN() indicates that
there are compilers that complain about a lack of return after
Q_UNREACHABLE (we know that MSVC, ICC, and GHS are among them), as
well as compilers that complained about a return being present
(Coverity). Take this opportunity to properly adapt to Coverity, by
leaving out the return statement on this compiler.
Apply the macro around the code base, using a clang-tidy transformer
rule:
const std::string unr = "unr", val = "val", ret = "ret";
auto makeUnreachableReturn = cat("Q_UNREACHABLE_RETURN(",
ifBound(val, cat(node(val)), cat("")),
")");
auto ignoringSwitchCases = [](auto stmt) {
return anyOf(stmt, switchCase(subStmt(stmt)));
};
makeRule(
stmt(ignoringSwitchCases(stmt(isExpandedFromMacro("Q_UNREACHABLE")).bind(unr)),
nextStmt(returnStmt(optionally(hasReturnValue(expr().bind(val)))).bind(ret))),
{changeTo(node(unr), cat(makeUnreachableReturn,
";")), // TODO: why is the ; lost w/o this?
changeTo(node(ret), cat(""))},
cat("use ", makeUnreachableReturn))
);
where nextStmt() is copied from some upstream clang-tidy check's
private implementation and subStmt() is a private matcher that gives
access to SwitchCase's SubStmt.
A.k.a. qt-use-unreachable-return.
There were some false positives, suppressed them with NOLINTNEXTLINE.
They're not really false positiives, it's just that Clang sees the
world in one way and if conditonal compilation (#if) differs for other
compilers, Clang doesn't know better. This is an artifact of matching
two consecutive statements.
I haven't figured out how to remove the empty line left by the
deletion of the return statement, if it, indeed, was on a separate
line, so post-processed the patch to remove all the lines matching
^\+ *$ from the diff:
git commit -am meep
git reset --hard HEAD^
git diff HEAD..HEAD@{1} | sed '/^\+ *$/d' | recountdiff - | patch -p1
[ChangeLog][QtCore][QtAssert] Added Q_UNREACHABLE_RETURN() macro.
Change-Id: I9782939f16091c964f25b7826e1c0dbd13a71305
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The QBAV one should pass the parameter by value, like QStringView. And
now that we have it, the non-View one should call the View one in an
inline function, like we already do for QString.
The extra, defaulted parameter is there only so we get a different
signature in the new inline function compared to the removed one.
Pick-to: 6.4
Change-Id: If05aeeb7176e4f13af9afffd16e7f08062b1dc86
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
This prevents false-negatives and false-positives, as e.g. Clang
10.0.0 masks as GCC 4.2, so Q_CC_GNU is 402 on that
compiler. Depending on the test (Q_CC_GNU > NNN or Q_CC_GNU < NNN),
the result of the test is almost random.
Q_CC_<comp>_ONLY makes sure we match only GCC or MSVC, not bycatch
such as Clang or ICC.
Pick-to: 6.3 6.2 5.15
Change-Id: I4c550a11ecf85fc9a2216b330b69bd03d45b47e0
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Still not complete. Just grepping for static and thread_local.
Task-number: QTBUG-100486
Change-Id: I90ca14e8db3a95590ecde5f89924cf6fcc9755a3
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Building with Apple clang 13.1.6.13160021 (from command line tools 13.3)
results in
qhash.cpp:754:39: error: passing 16-byte aligned argument to 32-byte
aligned parameter 3 of 'operator()' may result in an unaligned pointer
access [-Werror,-Walign-mismatch]
hash2x32bytes(state0, state1, src, src + 1);
^
Help the compiler with deducing the right type for 'src' and 'srcend'.
Also makes 'src' const explicitly.
Change-Id: Id14a034f0fa4c2a002d9b37729d803a50a0e5e9c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The compiler doesn't understand understand that the only way ok can be
true is for qstrntoll() to have been executed.
qhash.cpp:158:9: error: ‘seed’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
158 | if (seed) {
| ^~
Help poor GCC by initializing the variable unconditionally.
Amends cc5cc3225d.
Change-Id: Ic8387e0add2291d994f7f4d96c4bc614b2a8aa99
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Instead of lazily initializing (with some platform exceptions, see the
equivalent qsimd.cpp commit). The rationale is the same as for
qCpuFeatures(): this the qHashBits() function is hot and the current
code generation needs to save a lot of state because of the possible
call to the C++ runtime functions to enable the thread-safe
initialization of the hash seed.
[ChangeLog][Important Behavior Changes] QtCore now initializes the QHash
global seed before the main() function is run, so it is no longer
possible to use qputenv() to affect the seed value for the current
process. Disabling the random global seed for the current process should
be done programmatically with by calling either the 6.2 function
QHashSeed::setDeterministicGlobalSeed() or, if compatibility with Qt 5
is required, by calling qSetGlobalQHashSeed() with value 0.
Change-Id: I54f205f6b7314351b078fffd16cf7eae93f9e27e
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
We have VAES code in qhash.cpp that isn't getting compiled right now.
Change-Id: Ibf4acec0f166495998f7fffd16d6961261dec361
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Our associative containers' iterator's value_type isn't a destructurable
type (yielding key/value). This means that something like
for (auto [k, v] : map)
doesn't even compile -- one can only "directly" iterate on the
values. For quite some time we've had QKeyValueIterator to allow
key/value iteration, but then one had to resort to a "traditional" for
loop:
for (auto i = map.keyValueBegin(), e = keyValueEnd(); i!=e; ++i)
This can be easily packaged in an adaptor class, which is what this
commmit does, thereby offering a C++17-compatible way to obtain
key/value iteration over associative containers.
Something possibly peculiar is the fact that the range so obtained is
a range of pairs of references -- not a range of references to pairs.
But that's easily explained by the fact that we have no pairs to build
references to; hence,
for (auto &[k, v] : map.asKeyValueRange())
doesn't compile (lvalue reference doesn't bind to prvalue pair).
Instead, both of these compile:
for (auto [k, v] : map.asKeyValueRange())
for (auto &&[k, v] : map.asKeyValueRange())
and in *both* cases one gets references to the keys/values in the map.
If the map is non-const, the reference to the value is mutable.
Last but not least, implement pinning for rvalue containers.
[ChangeLog][QtCore][QMap] Added asKeyValueRange().
[ChangeLog][QtCore][QMultiMap] Added asKeyValueRange().
[ChangeLog][QtCore][QHash] Added asKeyValueRange().
[ChangeLog][QtCore][QMultiHash] Added asKeyValueRange().
Task-number: QTBUG-4615
Change-Id: Ic8506bff38b2f753494b21ab76f52e05c06ffc8b
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Fix a typo from da1720485e,
where mm_cvtsz_si128 was undefined when building for a 32 bit
target. The code would make it seem this was a typo, with the
listing of defines didn't match between 32 and 64 bit.
Change-Id: Ica24d88e7f71ecd1d24ed990773711f9105f3ec8
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
The extra "V" accidentally enabled AVX2 code in the aeshash128()
function.
Fixes: QTBUG-101082
Change-Id: Ibf4acec0f166495998f7fffd16d63e220dbe02d0
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Reviewed-by: Robert Löhning <robert.loehning@qt.io>
Instead of performing decisions inside the single aeshash() function, we
have three implementations instead. Those decisions are permanent for
each CPU, so the branch predictor should be pretty good, but hashing is
somewhat performance-sensitive.
We're only adding three of the four possible combinations of AVX512VL
and VAES. Excluded from the implementation are the CPUs that support
AVX512 but not VAES, which are the Skylake-based ones. Those are mostly
found in server CPUs (Intel Xeon Scalable line) as well as top-end
workstations (Intel Core i9), but never made into general desktop and
laptop parts. For those, the performance will remain what it was in Qt
6.3.
VAES is supported in Intel architectures codenamed Sunny Cove and
Gracemont and their successors. That means it's supported in both the E
and P cores of the Intel Alder Lake (12th Generation Core), as well as
future Atom lines. But neither Atoms nor hybrid CPUS have AVX512 (at
least when the E cores are active).
AVX512+VAES is supported for Ice Lake (10th Generation Core), Tiger Lake
(11th) as well as later generation with AVX512 support enabled. Like in
qstring.cpp, we restricted ourselves to 256-bit operations, which don't
cause performance impact and because the 512-bit VAESENC operates on the
fused Ports 0 and 1, so it has the exact same throughput as two 256-bit
VAESENC.
Change-Id: I6fcda969a9e9427198bffffd16cece9c37dbdbd3
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
It's not worth having both cases, since they run at the same time. The
difference is which CPUs they run for: the non-VAES AVX512 code can run
on the Skylake-based AVX512 CPUs, which aren't a usual target for Qt
(servers and workstations), while the AVX512 and VAES case includes the
Ice Lake and Tiger Lake desktop and laptop CPUs.
This is not a good trade-off, but is an stepping stone for the next
commit.
Change-Id: I6fcda969a9e9427198bffffd16cecbe76e03c884
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
The strategy is explained in the aeshash() function. I've chosen to
exclude the case of simultaneous VAES and AVX512VL support for len <= 32
case. Instead, the aeshash128_lt32_avx256() does not attempt to use
VAES, because we wouldn't be getting sufficient benefit at the cost of
code expansion (AESENC can dispatch 2 per cycle). See simulation at
https://analysis.godbolt.org/z/8Y54PMWGj.
The code is slightly convoluted with unexpected indentation so most of
the important lines in the algorithm aren't changed by this commit.
Change-Id: I6fcda969a9e9427198bffffd16ceca30f5e924b5
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Don't allow the compiler to inline them in qHashBits() because they
require a lot of register use and add to the hot code path to aeshash().
Now all calls in this function are tail calls.
Change-Id: I54f205f6b7314351b078fffd16cf7f2f97d99144
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
The loop for 32 bytes is left unchanged, but the tail operation for 16
to 31 bytes is replaced with an overlapped load-and-scramble. This
should make the operation even faster.
Also updated the key creation back to something similar to what Go does.
This massively improves performance as well as the bit spread. Histogram
for the bits in the hash value for the testcase from QTBUG-91739:
|| Bit || Before || After ||
| 0 | 35.0300% | 50.4800% |
| 1 | 42.5250% | 50.2400% |
| 2 | 46.0100% | 50.0000% |
| 3 | 67.5150% | 49.9400% |
| 4 | 56.5150% | 50.0000% |
| 5 | 51.9950% | 50.0000% |
| 6 | 58.9800% | 50.1400% |
| 7 | 55.9550% | 50.0000% |
| 8 | 41.9850% | 49.9200% |
| 9 | 69.9700% | 49.6400% |
| 10 | 68.4950% | 50.0000% |
| 11 | 37.4950% | 50.3000% |
| 12 | 61.9950% | 49.8200% |
| 13 | 53.4900% | 50.0000% |
| 14 | 63.0200% | 49.9800% |
| 15 | 54.9700% | 50.1000% |
Task-number: QTBUG-91739
Pick-to: 6.2.3 6.2 6.3
Change-Id: Icad7c1bad46a449c8e8afffd16cb7fe7ffd3584f
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
[ChangeLog][QtCore][QHash] Fixed a bug in the qHashBits() function,
which affected the hashing of QByteArray, QString (and their View
classes), QLatin1String and QBitArray, which caused the hash to not
include the final 32 bytes of the data source. As a result, QHash
containers where the initial string was the same had a serious
performance degradation on x86 CPUs with AES support.
Fixes: QTBUG-91739
Pick-to: 6.2.3 6.2 6.3
Change-Id: Icad7c1bad46a449c8e8afffd16cb74dd43440f6c
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
We don't need to test for actually crossing a page boundary. It suffices
to check if we're in the upper half or the lower half of the page. In
the upper half, we load ending at the end; in the lower half, we load
starting at the current position. This way, it can never crash.
Pick-to: 6.2.3 6.2 6.3
Change-Id: Icad7c1bad46a449c8e8afffd16cb743e622b3405
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
QDoc made some assumptions about the module a class/header belongs to,
based on the source file path. This feature is rather error-prone and
unnecessarily complex and will be removed from QDoc.
Define modules explicitly to avoid documentation warnings when this
removal happens.
Pick-to: 6.2 6.3
Change-Id: I7947d197db5ac36c12e816caa19bb2f74eda8849
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Venugopal Shivashankar <Venugopal.Shivashankar@qt.io>
It was added in 5.15 when QHash could have multiple entries per key.
Pick-to: 6.2 6.3
Change-Id: Ifb855b05dd65bbc67bb64bf2567229394525b2aa
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
... by not injecting potentially-expensive temporary objects into the
caller's stack frame.
Default arguments are a convenient way to avoid overloads, but if the
defaulted argument isn't a Trivial Type, and the common use case is
not to pass the extra argument explicitly, the construction of the
temporary can dominate the call's runtime.
Since QHash is generic code, we don't know whether T or Key are
expensive or cheap to construct, so use overloading instead of default
arguments to avoid injecting needless code into call sites.
[ChangeLog][QtCore][Potentially Source-Incompatible
Changes][QHash/QMultiHash] The value(key) and key(value) functions are
now overloaded on presence of the defaultValue (was: defaulted
argument) to avoid injecting temporary objects into the caller's stack
frame.
Task-number: QTBUG-98117
Change-Id: I80fdd5436f3de3e4bbe20242fe45916aef62ff0c
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
After the split of QHash and QMultiHash this function was not documented
since it was previously inherited from QHash.
As a drive-by also update 'int' to 'qsizetype' in docs
Pick-to: 6.2
Change-Id: I5d168886f13c2cdd4482038e66d0cf218789c847
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
[ChangeLog][Important Behavior Changes] The qHash functions operating on
string-like types and the qHashBits function will now mix in a shadow
seed (not available in any API) if the provided main seed is not 0. This
means the hashing value for any particular input has an almost zero
chance of being equal in two different processes, even if processes of
the same application. This unpredictability makes QHash more strongly
resist denial-of-service attacks through degenerate hashing tables.
Change-Id: Id2983978ad544ff79911fffd167240196f7cd5c8
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
There's now another half of the seed which will be used by the hashers.
This is not stored in QHash, so it is never changed for the lifetime of
the application (not even when QHashSeed::setDeterministicGlobalSeed()
is called). However, we will not use it when we're in deterministic
mode.
This commit uses the compiler thread-safe statics to implement the
initialization of more than one atomic word, thus freeing us from having
to have a reserved value. As a bonus, the QT_HASH_SEED warning will only
be printed once.
Change-Id: Id2983978ad544ff79911fffd16723f1673f9a5b4
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* Tag deprecated Q(Multi)Map operators in the header to correctly
match them with documentation \fn commands.
* Add documentation for QByteArrayView comparison operators.
* Add a dummy typedef 'jfieldID' for generating docs correctly
on non-Android platforms
* Fix other minor issues
Pick-to: 6.2
Task-number: QTBUG-95860
Change-Id: I141d2f75d6aa10557aa374201f09ad74b4cd6e81
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
* Add module header wrapper that loads the real QtCore header and
qandroidextras_p.h to generate docs for those types
* Add missing dummy typedefs to doc/include/jni.h
* Use the correct \namespace name (QtAndroidPrivate) and mark it
as \preliminary
* Add missing 'const' specifier for Q[Untyped]Bindable methods
* Drop documentation for removed method QProperty::markDirty()
* qmath.h: Fix \fn commands for qFloor(), qCeil()
* QHashSeed: Drop incorrect usage of \relates
Fixes: QTBUG-93942
Task-number: QTBUG-93995
Change-Id: If76b5aa4b79a64add3cb6275eac82ec44ef10319
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Thiago Macieira
Marc Mutz
Luca Di Sera
Christian Ehrlicher
Friedemann Kleint
Edward Welbourne
Giuseppe D'Angelo
Kai Köhne
Lucie Gérard
Sona Kurazyan
Volker Hilsheimer
Martin Storsjö
Topi Reinio
Mårten Nordheim
Nico Vertriest