A comment dated from when variables misleadingly named language_list,
script_list and country_list actually held mappings not lists; they've
been renamed to s/list/map/ a while back, so rephrase.
Use a dict-comprehension rather than the somewhat lisp-ier invocation
of the dict constructor on an iterator over pairs.
Change-Id: Ibcb97122434122dbb1dcb0f621aae02b25a4e1fa
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Introduce namespace QtTimeZoneCldr instead of having a Q prefix on
each class name used for the data.
Change-Id: Icb22a91340b67f9cc93173b77374a70f69f81bbe
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
I needed to know in order to make recent changes.
Save the need to work it out again next time.
Change-Id: Ibc606cbe2e6af16e6820fd753a643331a03cdfb3
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
(This turns out to be identical to v44, for our purposes.)
The CLDR license has been revised at v44 to "UNICODE LICENSE V3",
which is now included (as LICENSES/UNICODE-3.0.txt) in addition to the
old license (still in use, presumably, by UCD - at least until its
next update). Some new QLocale::Language entries are needed. There is
no change to the time-zone data.
Some tests needed changes:
* Various Arabic locales now use U+0623 (Arabic letter aleph with
hamza above) in exponent separator, replacing plain U+0627 (Arabic
letter aleph); it is still followed by U+0633 (Arabic letter seen).
* Where likely sub-tags used to fill in world, 001, as territory for a
language, they now (e.g. for Prussian and Yiddish) give specific
countries.
* Tamil locales now have something of a mix of inherited and localized
forms for AM/PM, which looks a lot like a mistake in CLDR.
* New likely sub-tag rules fix ctor(und_US) and ctor(und_GB), which
previously failed.
[ChangeLog][Third-Party Code] Updated QLocale's data extracted from
the Unicode Common Locale Data Repository (CLDR) to v44.1. The license
changed to Unicode License V3.
Pick-to: 6.7 6.6 6.5
Fixes: QTBUG-121485
Task-number: QTBUG-121325
Change-Id: Ide1a68016129526d7a5aa3fc67f1a674858696bc
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
The list is meant to be sorted in increasing order, requiring
"<anything> (Mexico)" to appear after "<anything>" but in two out of
four cases such pairs were in the wrong order. China sorts after
Chatham Island and lexical sorting of numbers doesn't match sorting by
numeric value.
Assert the expected ordering. (The more important check needs a
QBAV::compare(), which isn't constexpr, so we can't static_assert.)
Later commits shall use binary chop exploiting this ordering. The
assertion failed without the rest of this change.
Also improve the comments describing the data tables these searches
check and the types of their entries. Some were inaccurate, others
merely unclear. Likewise, comment the sorting expectations in the
python code that generates the tables.
Change-Id: I640a3cca8f820c5fd5939a2fe5feb96b04407335
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Giving it a symbolic name is clearer (and saves me the need to
duplicate the comment when I add some more references to it).
This amends commit bcdd51cfae
Task-number: QTBUG-115158
Change-Id: I7577e0cde783fcda840009c7aea46934964c6e4c
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
The existing caller returns early on finding a match, so never ran off
the end of the iteration unless there were no matches. I'll soon be
adding a new caller that wants to iterate all matches, so will run off
the end even when there are some. So only raise the Error if we found
nothing.
Task-number: QTBUG-115158
Change-Id: I1cae4674eb5e83c433554c15ecc4441b756f20eb
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
The Supplement type did the needed mapping (using nodeValue when the
value wasn't a string) and it turns out to be useful to do the same
for the DOM object packaged by a Node, too. Pull out into a helper
function, use dict-comprehension and expose as a method of Node.
Change-Id: Ice6737a54a33372b45cf42152e3fdbf5f2da7ba4
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
We've previously used the zip-file form, but that's not been published
for CLDR v44.1 - the advice on the list was to use github
instead. That, however, has ↑↑↑ as a special value for fields, meaning
to inherit from a prent locale. So special-case that value. I have
verified that v44 from the zip file produces identical results to v44
from github, with this minor fix. As it happens v44.1 also produces
identical results.
Pick-to: 6.7 6.5
Change-Id: I6eb0aedda7556753cdc83bb9d76652fbb68dc669
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
The table was almost sorted by offset - its UTC entry, with offset 0,
was at the front rather than first among the offset 0 entries. The
lookups in it were being done as if the IDs were in space-joined lists
(as for the IANA IDs in the Windows table), splitting on space,
despite the fact that it had separate entries for different IDs at the
same offset (this only arose for offset 0). So actually massage the
input table in python to combine IDs with the same offset using space,
placing UTC first among the offset 0 entries, and ensure the C++ table
is sorted. Regenerated the CLDR data tables using the updated script.
In the process, fix an off-by-one error in the iteration over
space-joined IDs, where the search only advanced to the space, rather
than to just after it. That wasn't a problem before, but now would be.
Change-Id: Ib49c27bac269b557166fa10738c3e396d58456c0
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The former is overtly forbidden by ISO 8601 but should be recognized
due to RFC 3339's use of it. The "Serialising Extended Data About
Times and Events Working Group" (sedate WG) has established how to
resolve this, so document that conclusion and note the consequent
inadvisability of using UTC-00:00 as a zone ID (although it should
still be accepted).
Change-Id: Ib9fbbe6765117bfa9a84e726d0e75f7397a4c813
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QItemDelegate was superseded since Qt4 by QStyledItemDelegate but it
took until Qt6.7 to remove the last occurrences in qtbase.
- remove unused includes / replace with qabstractitemdelegate.h
- replace references in the documentation with QStyledItemDelegate
- adjust the examples and tests to use QStyledItemDelegate
Pick-to: 6.5 6.6
Change-Id: I246755004ce2d01192a726ca0972106c237df0cc
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Now that the "and" is only seen in enumdata.py and comments, we can
s/And/and/ in all the various territory names that used it.
Change-Id: Ic376d5904b6f5ab54931f96230c1dd5b7f357b8d
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Various comments need to continue using the enumdata.py names, as they
associate data with particular enum members, but we can now correctly
use the en.xml versions of their names when we report them, rather
than the enum-friendly names we use in the code. Since this now means
the data may stray outside plain ASCII - it'll be UTF-8-encoded - this
implies replacing the QLatin1StringView()s of the code that formerly
read this data with QString::fromUtf8().
Fixes: QTBUG-94460
Change-Id: Id3b08875a46af58c0555c3e303b0e15a19441509
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
We could already use dashes in some, rather than spaces, and now no
longer need to capitalize each word. This changes the *_name_list[]
entries for affected languages to more closely match what CLDR gives
as their names. It also amends various comments. Added tests for the
QLocale::*ToString() functions to cover the entries changed.
Task-number: QTBUG-94460
Change-Id: I0163795cb282881f15a97be00a5311c1936c3a09
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The former needed the latter's .dupes to do the job, so can now just
take a method as a tool to do the job instead, letting .dupes become
private. In the process refine the munging to free enumdata.py from
having to capitalize each word in its names. This will, in due course,
let us use more natural forms in various comments. This causes no
change to generted data.
Update enumdata.py's introduction doc, mainly to reflect this but also
fixing the out-of-date names (old *_list have long been *_map) and
adding some details to other paragraphs.
Task-number: QTBUG-94460
Change-Id: If195b2e94a53a495fc4f1f216bed07a910439fa7
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Moving it makes it easier to document what it's up to and why, while
leaving __checkEnum() easier to read; and I'm going to need it
elsewhere anyway. This makes no difference to generated data.
Task-number: QTBUG-94460
Change-Id: I684375bc926d5d54928fbf5b5e08978528aef487
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Prefer stand-alone versions of the names when available. This saves
the need for a Han-specific kludge in the check for discrepancies
between our enum names and the en.xml names. Causes no change to
generated locale data.
Pick-to: 6.6 6.5
Change-Id: I162f3107d6ffc1f8b893b206e0b78b61cf7254f6
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
The string data tables have a mix of digit-length tokens, up to
four-digit hex; we can fit 12 of those per line within our margins.
Leave the one-row-per-locale tables as they are, though, despite long
lines.
Change-Id: I655fddecf24133c26d16187b7a5a8fbc25553e07
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
In the Windows zone-ID code, we tokenize() a text extracted from CLDR
data. However, a leading or trailing space (or a repeated internal
space) would then give an empty "IANA ID" for us to match, causing the
empty ID to be mapped to the Windows ID for the entry with the
superfluous space. This was uncovered by an entry with a trailing
space in CLDR v43's data.
Canonicalize spacing in the IANA ID lists extracted from CLDR so as to
ensure this doesn't happen. (We could pass Qt::SkipEmptyParts to the
tokenize() call, but fixing the issue when generating the data is
cheaper and more robust than fixing it at run-time every time it's
consulted.)
Task-number: QTBUG-111550
Change-Id: Ib3883419558d6574141e9ab0bc929ade2d73e020
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Also add a comment to check the locales new additions enable do have
substantial data. Some of those added in the past are more or less
stubs, for all that they're officially present.
Task-number: QTBUG-111550
Change-Id: I04d46ee96303ecec56c056a0deff6a9457b863e9
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Mate Barany <mate.barany@qt.io>
The digit-grouping and fractional-part separators need to be distinct
for parsing to be able to distinguish between two thousand and one vs
two and one thousandth. Thakfully ldml.py asserted this, so caught the
glitch in CLDR v43's data where mn_Mong_MN over-rode mn's decimal, but
not group, and thereby clashed with group. Fortunately the over-ride
is marked as draft="contributed" so we can back out of the collision
and limit the selection to draft="approved" values (but only when
there *is* such a conflict, as plenty of locales have (compatible)
draft data), thereby ignoring the conflicting contribution.
Brought to the attention of cldr-users at:
https://groups.google.com/a/unicode.org/g/cldr-users/c/6kW9kC6fz3g
hopefully that'll lead to a saner resolution at v44.
Task-number: QTBUG-111550
Change-Id: I1332486e60481cb4494446c0c87d89d74bd317d4
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
From CLDR v43, "The parentLocale elements now have an optional
component attribute, with a value of segmentations or
collations. These should be used for inheritance for those respective
elements." Since we aren't extracting collation or segmentation data
for the present, omit these elements from the scan for parentLocale
information.
Task-number: QTBUG-111550
Change-Id: I42871929f539c1852471812801953f2fc8be0e8a
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
The script and territory to exclude from reports about unused ones
were swapped, so we excluded a territory from the script list (which
didn't contain it anyway) and vice versa.
TheTest for whether to report used the non-existend .territories
attribute by mistake for .__territories
Change-Id: I29e9d9f8f34883d7c3a5ac15470d9e7a0366e3db
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Amends commit 9a8b9473d5 - apparently
the enumdata.py entries were tidied up after the data had been
generated, leading to them being inconsistent (and I missed that in
review). That, in turn, meant the next update would have changed the
public API enum members, backwards-incompatibly; so make enumdata.py
consistent with the released public API. We'll be tidying the order up
at Qt 7 in any case.
Task-number: QTBUG-110333
Change-Id: I3eed2924ce8b69deb552e923d9b0dc142c5f3a65
Reviewed-by: Konrad Kujawa <konrad.kujawa@qt.io>
Reviewed-by: Mate Barany <mate.barany@qt.io>
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Pack some of the arrays that contain locale data more tightly. The
AlphaCode struct is a char[4] but always holds only [a-z]{,3} which
could be fit into 16 bits, halving the size of an AlphaCode struct.
With the new constructor the initialization of the AlphaCode struct
also changes - modify qlocalexml2cpp.py to reflect this change and
regenerate the languageCodeList.
Fixes: QTBUG-105050
Change-Id: I2b1e93ab7cc3f2d667bf67b45769b74a15211931
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Revise a comment in ldml.py about Suzhou "digits", since it turns out
they aren't the same as hanidec, which is far from contiguous.
Change-Id: Ia3947dbc5a927772026e55fe197c8ebce2540da2
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
New languages (and one local for each) added with v42
- Haryanvi
- Moksha
- Northern Frisian
- Obolo
- Pijin
- Rajasthani
- Toki Pona
It also appears that Canada has changed its date format. Modify the
relevant test case to reflect this change.
Task-number: QTBUG-110333
Pick-to: 6.5
Change-Id: Ia8975c2866cd54c9e565543d05bacd52f4987909
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
We can easily enough obtain the root of the present source tree using
the value of __file__, so might as well do so.
Change-Id: If14773ac1127278b6018a090c0b376437b9c6eec
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
This commit extends functionality for QLocale::codeToLanguage()
and QLocale::languageToCode() by adding an additional argument
that allows selection of the ISO 639 code-set to consider for
those operations.
The following ISO 639 codes are supported:
* Part 1
* Part 2 bibliographic
* Part 2 terminological
* Part 3
As a result of this change the codeToLanguage() overload without
the additional argument now returns a Language value if it matches
any know code. Previously a valid language was returned only if
the function argument matched the first code defined for that
language from the above list.
[ChangeLog][QtCore][QLocale] Added overloads for codeToLanguage()
and languageToCode() that support specifying which ISO 639 codes
to consider.
Fixes: QTBUG-98129
Change-Id: I4da8a89e2e68a673cf63a621359cded609873fa2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Update the locale generation script to support Kaingang and
Nheengatu languages. These are new in CLDR v40. Regenerate
the locale data.
Task-number: QTBUG-94358
Change-Id: I5195d5161d8c4d9f17129bbcfde39dfd3fcf1cd5
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
FTP is insecure and is not supported by modern browsers anymore.
See also: https://mywiki.wooledge.org/FtpMustDie
Change-Id: Iad65d29912e79a4f3fadb9317bb5d9c5fe9b68d7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
They are not needed. Iterations over the table track their sizes.
The size-of-table constants just needed their -1s removed.
Incidentally use std::size() rather than sizeof(array)/sizeof(element).
Change-Id: Ie20eef9f6f5786d93c10b830a87e006d3c5bcc1a
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Use context manager interface (with statement) to atomically update source
files. This ensures that all files are properly closed and the temporary
file is removed even in case of errors.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I18cd96f1d03e467dea6212f6576a41e65f414ce1
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Using NamedTemporaryFile instead mkstemp + fdopen simplifies the code.
It also makes it easier to switch to using context managers for handling
source file modification.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ibeae840ac6dde3d0b49cd7f985cfa6cd775b7f47
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
pathlib's API is more modern and easier to use than os.path. It
also allows to distinguish between paths and other strings in type
annotations.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ie6d9b4e35596f7f6befa4c9635f4a65ea3b20025
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
arparse is the standard way to parse command line arguments in Python.
It provides help and usage information for free and is easier to extend
than a custom argument parser.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I1e4c9cd914449e083d01932bc871ef10d26f0bc2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Replace most uses of str.format() and string arithmetic by f-strings.
This results in more compact code and the code is easier to read
when using an appropriate editor.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I3409f745b5d0324985cbd5690f5eda8d09b869ca
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The schema is in RelaxNG Compact syntax. It can be used to validate
files produced by the cldr2qlocalexml.py script and also gives an
overview of the file format.
Change-Id: I344978f2201c5e67e236ab580a12ad33262f33cb
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This is the standard way to call base class methods in Python 3 and
it is shorter than the custom one used now.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ifaff591a46e92148fbf514856109ff794a50c9f7
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Instead of implementing all the intricacies of a cmp for the python
sort-function, support for which is due to be dropped at Python 3 in
any case, implement a much simpler key function that achieves the same
result.
In the process, eliminate the ugly kludge of setting an attribute on a
function to, in effect, communicate with it via a global. Instead,
instantiate a class, that wraps the value previously given to the
attribute and whose instance provides the key-function.
Thanks to Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io> for
pointing out that a key function is the way of the future - and
sorted() is a nicer way to sort.
Pick-to: 6.2
Change-Id: Icf1ed5597fedf420d054fbc860e3e7fc6615875c
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
The ordering function used to sort the locale data generated for
QLocale attempted to sort the default territory for a given language
and script before other territories, but was too tangled for it to be
obvious this is what it was doing. The result turned out to be
non-transitive. Replace with code that implements the same preference
but only applies it where the result is compatible with transitivity.
This leads to a shuffling of the order of the Serbian-language
locales, which sorts the Cyrillic ones before the Latin ones. This is
consistent with my reading of the CLDR data, which fills in Cyrillic
and Serbia for Serbian; Serbian/Cyrillic/Serbia did previously sort
before all other Serbian variants.
Thanks to Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io> for
discovering the non-transitivity.
Pick-to: 6.2
Change-Id: I0ce9f78e620e714f980f32b85b7100ed0f92ad74
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
The behavior of StopIteration in generators was changed in Python 3
(see https://www.python.org/dev/peps/pep-0479/). Not raising that
exception makes it easier to port the code to Python 3.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Iac6e3f6f1e1e8ef3a1a0d89b19d2ac2d186434f5
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Edward Welbourne
Christian Ehrlicher
Mate Barany
Yuhang Zhao
Lucie Gérard
Ievgenii Meshcheriakov