Use QMultiHash explicitly, and build list of values in a QSet via the
range constructor.
Task-number: QTBUG-131842
Change-Id: I9cbcddeada0bfd88b11515262f5476e5d59e0fad
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
(cherry picked from commit e1217cc52bd367b7e040edeb86700dc923df39a8)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
The AnyTerritory entries in the zoneDataTable are derived from
territory="ZZ" entries in the upstream CLDR data; the World ones from
territory="001". The latter give the default IANA ID for each MS ID,
the former give an (often legacy) IANA ID for the MS ID, that is not
based on geography. Some of these are being removed at CLDR v46.
The documentation said the ZZ entries have "no known territorial
association", hinting that there may be some (unknown) territorial
association; however, CLDR's inclusion of them is as entries with a
known non-territorial association, so revise the phrasing to reflect
this.
Also document that windowsIdToDefaultIanaId() returns empty when
there is no territory-specific value, and callers can use the
territory-neutral call to get a suitable value in that case. (They
may, however, wish to distinguish this case, to treat it differently,
so I decided not to just return that in place of empty in any case.)
The upstream CLDR tables do have entries for territory 001, so we
should report these if asked for World as territory. Amend the
available zone ID lookup and mapping from MS to IANA functions that
take a territory to duly handle World via the default-data that was
derived from 001 data in CLDR, instead of from the territory-varying
table, from which those were effectively filtered out when generating
the two tables. Update docs to mention this handling of World, for
contrast with that of AnyTerritory.
In the process remove a spurious split-on-space from the MS to default
IANA lookup, asserting there is no space (in a field now stored in the
table for single IANA ID entries, instead of the one for space-joined
lists of them in which it used to be stored, before I noticed it's
always only one ID). There is a matching assertion in the cldr.py code
that extracts the data. Added an assertion to this last, that each
default IANA ID given by CLDR's MS data does in fact also appear as
one of the IANA IDs for at least one territory (potentially ZZ), and
comment in C++ code on why this means we don't need to scan the
windowsDataTable in a few places, where it would just produce
duplicate entries.
On picking to 6.8, removed the timezone_locale addition, only relevant
on 6.9 and later.
[ChangeLog][QtCore][QTimeZone] Corrected handling of QLocale::World
and clarified in docs how QLocale::AnyTerritory is handled when
QTimeZone selects zones by territory.
Task-number: QTBUG-130877
Change-Id: I861c777c68b0cb73a194138fe23fbff839df49e6
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit e23dc7c420297fb62db9834a17c59bbf5992dad7)
Reviewed-by: Mate Barany <mate.barany@qt.io>
The change adds CPE and PURL keys to all qt_attribution.json files in
the repo.
In case if no sensible CPE or PURL exists, a "Comment" field is added
with the text "no relevant CPE or PURL found". If only one of them
does not exist, it is written as such in the Comment field.
This allows filtering for files that haven't had the information added
yet vs those that were looked up but no relevant information was
found.
For sources that are not hosted on github, a generic PURL is used with
a download_url fragment pointing either to the exact location where
the sources can be downloaded, or to the homepage of the project.
The generic package name was chosen based on the 'Id' key of the
attribution entry where it was present, and is not authoritative.
For PURL github packages, the 'git tag' name was specified into the
'version' part of the PURL, rather than the 'version number', because
SBOM processing tooling handle that better than the version number.
For example for the freetype package, we specify the string
'VER-2-13-3' rather than the tag name '2.13.3'.
We might revisit this in the future.
[ChangeLog][Third-Party Code] Added PURL and CPE information to the
attribution files of 3rd party sources.
Task-number: QTBUG-122899
Task-number: QTBUG-129602
Change-Id: Iad126242cafc3ea0b678c5c36b26f857039b1dbd
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
(cherry picked from commit 36dca3c04f759449f74008a3e79021a179b0f35e)
Add some type annotatons to cldr2qlocalexml.py as well. Based on the
default arguments the constructor of CldrReader was expecting callables
that return None, but in reality we are passing in functions that
return integers.
Task-number: QTBUG-129613
Change-Id: I06832240956ea635ca0cc0ec45c466a3b2539ff7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
(cherry picked from commit 812f79e75f2d6f2675c4b032f3a98610721e98de)
This should keep gcc from complaining that they're unused const
variables. Pick to 6.8 required conflict resolutions in non-generated
code and qlocalexml2cpp.py due to dev's updates to handle time zone
name L10n; regeneration of data took care of the other conflicts.
Fixes: QTBUG-128930
Change-Id: I4d0bccc5f158dfc72e4cf38a085b90cb562dc185
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit 08ef4d232df82c056d4f5fda636b62b65fc054ad)
Those files are read by reuse to complement or override the copyright
and licensing information found in file.
The use of REUSE.toml files was introduced in REUSE version 3.1.0a1.
This reuse version is compatible with reuse specification
version 3.2 [1].
With this commit's files,
* The SPDX document generated by reuse spdx conforms to SPDX 2.3,
* The reuse lint command reports that the Qt project is reuse compliant.
[1]: https://reuse.software/spec-3.2/
Task-number: QTBUG-124453
Task-number: QTBUG-125211
Change-Id: I01023e862607777a5e710669ccd28bbf56091097
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
(cherry picked from commit 4c83657448f66ac0ee46900aff2e5a230eda49b0)
Amends commit 52909797ac, whose conflict
resolutions inadvertently incorporated the movement of IANA IDs from
the list table to the IANA table, a change only done on dev (6.9), not
picked back to 6.8, but tagged with a # TODO comment in 6.8 that, none
the less, is not to be done in 6.8.
Restore writing to the correct table.
Change-Id: I70fd133eb54526efecae189306b3ae07c1c638db
Reviewed-by: Mate Barany <mate.barany@qt.io>
A missing update of a "last" variable meant the loop inevitably did
nothing useful. Include type-annotation for last, while doing this.
Thankfully the check still doesn't find any duplications, now that
I've fixed it so that actually would, were any present.
Pick-to: 6.5
Change-Id: I672e6570359a3ff102a364d8af98c5c8c0bdc4d9
Reviewed-by: Mate Barany <mate.barany@qt.io>
(cherry picked from commit 98db7a35d2ee56f5da11b4a8e745f2ee6a965077)
Found these while adding type annotations.
Task-number: QTBUG-129566
Change-Id: I51c8e5676f958094946c0e6f396b98c083fd9de0
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
(cherry picked from commit ba9d6b261b04119b15d27f3f57dcfe0aadad5012)
Add some type hints to unicode2hex as well, it is used by
ByteArrayData.
Task-number: QTBUG-128634
Change-Id: I86b7ce8567483bf8a4d4db78c9585652526cb90d
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
(cherry picked from commit c1ffb6e7515621a953b5ebf19c384b27df785d73)
The lookup into it is done case-insensitively (because user-supplied
names of zones might not have the right case) but I forgot to make the
sorting of the data table case-insensitive in the aliases. Regenerate
data: only the qtimezone*_data_p.h are changed by the reindexing of
zone aliases.
On picking to 6.8, drop the change to QTZlocale_data_p.h, as that's
not active until 6.9; and back-date the QTZprivate_data_p.h to match
the change on dev four days ago, as that might just reduce the risk of
later conflicts.
Change-Id: Id5e95c245c7ca421a77298f23baefe6b7021a396
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit ff58e09c8d5b3f505d5d3fe9a205e35e96de922d)
Reviewed-by: Mate Barany <mate.barany@qt.io>
Intel CPUs have had this since 2013 (Ivy Bridge), but some older
Bulldozer AMD CPUs appear to be missing it. This creates a mismatch
between when the __haswell__ macro gets declared in qsimd_p.h and the
runtime check using the CpuArchHaswell value. That in turn creates a
condition where qInitDrawhelperFunctions() in qdrawhelper.cpp leaves the
memfill pointers set to null.
#elif defined(__SSE2__)
# ifndef __haswell__
qt_memfill32 = qt_memfill32_sse2;
qt_memfill64 = qt_memfill64_sse2;
# endif
...
#if defined(QT_COMPILER_SUPPORTS_AVX2)
if (qCpuHasFeature(ArchHaswell)) {
qt_memfill32 = qt_memfill32_avx2;
qt_memfill64 = qt_memfill64_avx2;
It does this so the qt_memfillXX_sse2 functions don't have to be defined
anywhere, so the QtGui build won't carry unnecessary dead code.
This is old code (from Qt 4.x) and several improvements I've made for
QtCore are not applied yet. My work for qSimdDispatcher[1] isn't
complete: it might have avoided this problem here, but it would also
have required major work for the draw helpers to work in the first
place.
[1] https://codereview.qt-project.org/c/qt/qtbase/+/537384
Pick-to: 6.7 6.5 6.2
Fixes: QTBUG-129193
Change-Id: Ia427a9e502b0fb46b2bdfffda8e2131b7091c9e9
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
(cherry picked from commit 54c24313fe1e4ed58b3260189cb623a7c852ab1d)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
On Unix, if target exists and it is a file, rename silently replaces it
if the user has permission. However, on Windows, if the target exists,
FileExistError will be raised.
With replace, if target points to an existing file or empty directory,
it will be unconditionally replaced.
Change-Id: I2774152fec78a00c4ca6c9d1b927e503df2f2e84
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
(cherry picked from commit eff8e6b885dfa3f26523ca39164725b4932c31bd)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
Using the default system encoding cldr2qlocalexml.py and
qlocalexml2cpp.py may terminate with encoding errors on Windows.
Warn the user to set the PYTHONUTF8 environment variable to 1 before
running those scripts to avoid encoding errors.
Change-Id: I315a45072cb6ea516d3e9bb7613c6f251792ec59
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
(cherry picked from commit b3ef2fdd8b99c6920a382fbbc417209bf436c3f3)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
The variable ianalist is not really used for anything, it was probably
meant to be ianaList.
Change-Id: Ie9f42bf9716da28ee0017319dda96389c415ef4f
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
(cherry picked from commit 67ec126168b81adc2adc9fdb346c0a913c3c501d)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
This amends commit 880d1aef99 and
extends it to cover the testlocales program under util/.
Pick-to: 6.7 6.5
Task-number: QTBUG-121653
Change-Id: I3efadc69ce08810876f8e20aa4636c7624728153
Reviewed-by: Lucie Gerard <lucie.gerard@qt.io>
(cherry picked from commit 00644f4b6d69d82a8d27f3b0619de88c0e4fdd6f)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
This was in fact present in v44, but we overlooked it somehow. The new
version also fixes some inconsistencies in the data, that I reported
against v44.1; in particular, Tamil no longer claims to override the
root AM/PM markers (probably because it uses 24-hour time so doesn't
need them).
Add the test-file under util to the list of files containing generated
content.
Conflict at 6.8 resolved by regenerating the data; this only changed
the date of generation, not the data. Then hand-edited the date to
match the picked upstream commit, to avoid future conflicts.
[ChangeLog][Third-Party Code] Updated CLDR data, used by QLocale, to
v45.
Task-number: QTBUG-126060
Pick-to: 6.7 6.5 6.2
Change-Id: I81a5bcca49519b55091fc541de6b73b606661bb4
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit f79548e268a496698d77d0e78365334d0e507212)
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Move to construction time, instead of passing to each append() call;
the table's field sizes are, after all, the same for all entries.
Add support for larger tables by allowing more than 16-bit indices.
Task-number: QTBUG-115158
Change-Id: I8f1113482e80838c512da6353fa17b9f365f956a
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Mate Barany <mate.barany@qt.io>
It was setting *_code='0' for the Any* forms of language, script and
territory; this is wrong, the codes for these are all empty or other
special tokens (like 'und', 'Zzzz', 'ZZ'). The IDs for them are zero,
as an int not a string, but were omitted. Also add the variant
details, for all that they're currently unused, for consistency.
This makes no difference to the generated data.
Task-number: QTBUG-115158
Change-Id: I339d1b201e50e2bbc510758ffbbaae0fa02277d4
Reviewed-by: Mate Barany <mate.barany@qt.io>
The qlocalexml.py Locale.C() had to replicate a whole lot of data that
isn't really relevant to how C differs from en_US and every addition
to what we support required further additions to it. So pass the en_US
Locale object to the pseudoconstructor so that C can inherit from it
and only override the parts where we care about the difference.
Hand-code shortening for short Jalali month names, to match Soroush's
original contribution, and include the narrow forms in the hard-coded
data to keep the generated data unchanged (for now). Note some of the
departures from CLDR; we may want to drop these overrides later.
In the process, convert the mapping from keys to locales to
consistently use IDs for all members of the key, instead of using the
(empty) code value for (as yet unused) variant; it now gets ID 0 and
is consistent with returns from codesToIdNames(). This makes life
easier for the code that now has to construct an en_US key.
Task-number: QTBUG-115158
Change-Id: I3d7acb6a4059daec1bba341fcf015c39c7a6803b
Reviewed-by: Kai Köhne <kai.koehne@qt.io>
Omit parentheses round what python will form into a tuple anyway.
Include trailing commas on last entries of tuples so adding future
entries don't drag the existing line into their diffs.
Let the writer's tag-opener handle attributes, if supplied.
Clean up spacing in some doc-strings.
This is all preparation for further changes, to limit their diffs.
Change-Id: I989ae28bbd235b2af9c1d72467d4741c4f1f20ae
Reviewed-by: Mate Barany <mate.barany@qt.io>
Future work shall need the timezone alias data to be synchronized
between the (expanded) locale-independent timezone data and the
(coming) locale-dependent timezone data. The latter shall need to come
via QLocaleXml, hence the former now needs to, too.
This makes no change to the generated data, aside from changing the
regeneration instructions for qtimezoneprivate_data_p.h, to use the
same scripts as locale data, instead of cldr2qtimezone.py, which is
now removed.
Task-number: QTBUG-115158
Change-Id: I47ddd95f6af1855cbb1f601e9074c13f213cd61c
Reviewed-by: Mate Barany <mate.barany@qt.io>
It's trivial to do - and done when generating our compiled data
tables, so makes no difference to users - but makes the offset list
table simpler. Reformat the list so that the fragment-of-hour offsets
are clearly distinguished from the whole-hour ones.
Change-Id: I6e0ea23dc317542b3256e88492e4073faedef1d7
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
It was originally (without any comment to this effect, either in the
code or the commit message) just the list of offset-zones
corresponding to known Windows zones' offsets, augmented to include
each whole hour offset out to ±14 hours. Absent documentation, of
course, this was not maintained.
Added the four offset zones implied by that, that hadn't been added
when new entries joined the Windows IDs with novel offsets. Check,
after scanning CLDR for Windows data, that this has been kept up to
date. Updated the generated data.
Change-Id: I3cf3932c320876f7f2f74840d8c3951be49cfe70
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The CLDR's "IANA" IDs may (for the sake of stability) date back to
before IANA's own naming has been updated. As a result, the "IANA" IDs
we were using were in some cases out of date. CLDR does provide a
mapping from its stable IDs to all aliases and the current IANA name
for each (which I shall soon be needing in other work), so use that to
map the CLDR IDs to contemporary IANA ones.
Revise the documentation of CldrAccess.readWindowsTimeZones() to take
this into account, pass it the alias mapping from the table, use that
to map IDs internally and, in passing, rename a variable. Update
cldr2qtimezone.py to match the new CldrAccess methods and regenerate
the data.
Change-Id: I23d8a7d048d76392099d125376b544a41faf7eb3
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mate Barany <mate.barany@qt.io>
There are various legacy IANA IDs that we should recognize as aliases
for their contemporary equivalents. Later work shall also take these
into account in the Windows IDs. Scan CLDR's data about these aliases
and use it when constructing QTimeZone. This adds aliasMappingTable
and aliasIdData arrays to QTZP_data_p.h and an AliasData type to its
QtTimeZoneCldr namespace.
Change-Id: I1bbfce62959a7e1b7a0bc4a320c32f5a174a2ff2
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
For those files, reuse fails to process the copyright and licensing information. The information is written to .reuse/dep5 file and reuse is told to ignore what is in the file.
Task-number: QTBUG-124453
Change-Id: If593c713026d9d349055cb6e8e73500966a05d9b
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
This separates the large slabs of data (and their documentation) from
the code that mixes them with CLDR-derived data and generates the data
we actually use. In the process, put the shorter table before the
longer one, to make it less likely that folk shall fail to notice it's
even there at all.
Change-Id: I8457741911657dac0dad53c2e65b977821bb4e71
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
The QLocale XML reader was passing datetime formats through a format
conversion despite the data being converted at the point where we read
it from CLDR. It turns out this was needed because the long date and
time formats in our hard-coded data for the C Locale object used CLDR
format strings, unlike all other Locale objects. Fix those two formats
in the C locale and remove the redundant processing step. This, in
turn, enables the parser to include the date and time formats in its
general handling of most fields that it reads.
This does not result in any change to the generated data QLocale uses
(although it does change the intermediate QLocale XML file).
Task-number: QTBUG-115158
Change-Id: Iaf9da206158043dda2e9e5a3790f009b100e46b4
Reviewed-by: Mate Barany <mate.barany@qt.io>
Include documentation in both, using common phrasing. Take sys.argv as
a parameter, along with sys.stdout and sys.stderr, so that we can
invoke them from python when importing into a python session to debug
or test. Supply the script name to the argument parser as prog, so it
can correctly report it and forward the rest of argv to parse_args().
Remove comments anticipating one of the several calendars we don't yet
support; the existing entries suffice to make clear what shall be
needed when we get round to adding more.
Change-Id: I2cebd385679e3c84d4ccf899e60091ac823ad10d
Reviewed-by: Mate Barany <mate.barany@qt.io>
After several years unused, it had bit-rotted to the point of not
compiling and failing an assertion. It also appears to have always had
a bad free() on exit, due to passing the address of a static object to
a function that took ownership and later deleted it.
Change-Id: I91856258c3fedf820bf151b5d205d257876a8e13
Reviewed-by: Jason McDonald <macadder1@gmail.com>
This old test program has bitrotted due to not being autogenerated as
part of CLDR updates. Amend qlocalexml2cpp.py to regenerate it and do
such an update. It was still using Qt5's QLocale enum numeric values,
many of which have changed in Qt6. Actually fixing the code so that it
compiles and runs can wait for a later commit.
Inspired by a patch supplied by Kizito Birabwa.
Task-number: QTBUG-124200
Change-Id: I33811313976a4860aad6d7b5b88a40c5b111a4fe
Reviewed-by: Mate Barany <mate.barany@qt.io>
It has many grumbles about spacing, but at least this code is
currently consistent about its departure from PEP8's spacing rules
(and closer to Qt's) for the present. We can review whether to do a
drastic spacing revolution later.
Change-Id: Ife4e8a5b02b63434bd9c7ac7ba4cbc11b6311f9f
Reviewed-by: Mate Barany <mate.barany@qt.io>
Volker Hilsheimer
Edward Welbourne
Mate Barany
Alexandru Croitor
Jani Heikkinen
Lucie Gérard
Thiago Macieira