Commit Graph

96 Commits

Author SHA1 Message Date
Khaled Hosny f9e1192d58 [ot-tag] Document two deprecated symbols
See https://github.com/harfbuzz/harfbuzz/issues/3957
2022-12-18 23:58:36 +02:00
Behdad Esfahbod 59c45f6deb Use hb_memcpy instead of memcpy consistently 2022-11-22 12:54:50 -07:00
Behdad Esfahbod f73c15ca6c [atomic-int] Add operators for relaxed ops 2022-08-03 13:01:04 -06:00
Behdad Esfahbod f7f6d278bb Add hb_language_matches()
New API:
+ hb_language_matches()
2022-07-17 22:15:42 -06:00
Behdad Esfahbod a7960bdfb0 [config] Add HB_NO_LANGUAGE_LONG and enable in TINY profile
Disables 3letter language tags and more complex ones.

Fixes https://github.com/harfbuzz/harfbuzz/issues/3664
2022-06-20 17:55:28 -06:00
Behdad Esfahbod 315ef83b4e Revert "Revert "[ot-lang] Use atomic int for cache""
This reverts commit c56ce8681c.

The revert was not intentional. Ouch!
2022-05-30 05:09:26 -06:00
Behdad Esfahbod c56ce8681c Revert "[ot-lang] Use atomic int for cache"
This reverts commit d61b207491.
2022-05-28 04:25:51 -06:00
Behdad Esfahbod d61b207491 [ot-lang] Use atomic int for cache
Fixes(?) https://github.com/harfbuzz/harfbuzz/issues/3612
2022-05-26 18:24:43 -06:00
Behdad Esfahbod 9c64bda21d [ot-tag] Whitespace 2022-05-17 17:31:18 -06:00
Behdad Esfahbod 3df8017e9b [ot-tag] Optimize subtag_matches() more 2022-05-17 17:29:39 -06:00
Behdad Esfahbod 7f6e8c5536 [ot-tags] Optimize subtag_matches() further
Part of https://github.com/harfbuzz/harfbuzz/issues/3591

Comparing before to after
Benchmark                                                               Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY                -0.3371         -0.3371            71            47            71            47
2022-05-17 16:58:35 -06:00
Behdad Esfahbod 27c11405a2 [ot-tag] Optimize subtag_matches
Part of https://github.com/harfbuzz/harfbuzz/issues/3591
2022-05-17 16:51:51 -06:00
Behdad Esfahbod a07d818597 [ot-tag] Add a likely() to the cache hit case 2022-05-17 16:46:10 -06:00
Behdad Esfahbod dfca47f419 [ot-tag] Cache last bsearch result
Part of https://github.com/harfbuzz/harfbuzz/issues/3591

Humm. Looks like not all of the fat is bsearch overhead now. I cached
the last bsearch result, but most of the time is still there. I'm
baffled.

Before:
------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations
------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY       8.08 ns         8.05 ns     84500482
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN         42.2 ns         42.1 ns     16722006
BM_hb_ot_tags_from_script_and_language/COMMON en_US         16.1 ns         16.0 ns     43461527
BM_hb_ot_tags_from_script_and_language/LATIN en_US          16.5 ns         16.5 ns     42448505
BM_hb_ot_tags_from_script_and_language/COMMON none          4.34 ns         4.33 ns    161290530
BM_hb_ot_tags_from_script_and_language/LATIN none           4.34 ns         4.33 ns    162339799

After:
------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations
------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY       8.13 ns         8.11 ns     80438134
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN         40.0 ns         39.9 ns     17487939
BM_hb_ot_tags_from_script_and_language/COMMON en_US         12.7 ns         12.7 ns     55124394
BM_hb_ot_tags_from_script_and_language/LATIN en_US          13.1 ns         13.0 ns     53660125
BM_hb_ot_tags_from_script_and_language/COMMON none          4.61 ns         4.60 ns    151394104
BM_hb_ot_tags_from_script_and_language/LATIN none           4.70 ns         4.68 ns    150402847
2022-05-17 16:21:02 -06:00
Behdad Esfahbod 909f00ac6e [ot-tags] Further speed up language bsearch()
Using an integer tag to bsearch, instead of string.

Part of: https://github.com/harfbuzz/harfbuzz/issues/3591

Before:
------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations
------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY       8.11 ns         8.08 ns     87067795
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN         53.6 ns         53.5 ns     13042418
BM_hb_ot_tags_from_script_and_language/COMMON en_US         24.2 ns         24.1 ns     29052731
BM_hb_ot_tags_from_script_and_language/LATIN en_US          24.4 ns         24.3 ns     28736769
BM_hb_ot_tags_from_script_and_language/COMMON none          4.43 ns         4.41 ns    160370413
BM_hb_ot_tags_from_script_and_language/LATIN none           4.35 ns         4.34 ns    160578191

After:
------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations
------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY       7.97 ns         7.95 ns     85208363
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN         41.7 ns         41.6 ns     16945817
BM_hb_ot_tags_from_script_and_language/COMMON en_US         16.1 ns         16.0 ns     43613523
BM_hb_ot_tags_from_script_and_language/LATIN en_US          16.5 ns         16.4 ns     42568107
BM_hb_ot_tags_from_script_and_language/COMMON none          4.30 ns         4.29 ns    164055469
BM_hb_ot_tags_from_script_and_language/LATIN none           4.29 ns         4.27 ns    163793591
2022-05-17 15:51:41 -06:00
Behdad Esfahbod c460cf74ce [ot-tags] Cosmetic 2022-05-17 15:30:11 -06:00
Behdad Esfahbod 1c8226ed14 Fix compiler warning
On Mac compiler:

FAILED: src/libharfbuzz.0.dylib.p/hb-ot-tag.cc.o
c++ -Isrc/libharfbuzz.0.dylib.p -Isrc -I../src -I. -I.. -I/usr/local/opt/freetype/include/freetype2 -I/usr/local/Cellar/graphite2/1.3.14/include -I/usr/local/Cellar/glib/2.72.1/include/glib-2.0 -I/usr/local/Cellar/glib/2.72.1/lib/glib-2.0/include -I/usr/local/opt/gettext/include -I/usr/local/Cellar/pcre/8.45/include -Xclang -fcolor-diagnostics --coverage -pipe -Wall -Winvalid-pch -Wnon-virtual-dtor -std=c++11 -fno-rtti -O2 -g -fno-exceptions -fno-rtti -fno-threadsafe-statics -fvisibility-inlines-hidden -DHAVE_CONFIG_H -Wno-non-virtual-dtor -MD -MQ src/libharfbuzz.0.dylib.p/hb-ot-tag.cc.o -MF src/libharfbuzz.0.dylib.p/hb-ot-tag.cc.o.d -o src/libharfbuzz.0.dylib.p/hb-ot-tag.cc.o -c ../src/hb-ot-tag.cc
In file included from ../src/hb-ot-tag.cc:29:
In file included from ../src/hb.hh:481:
../src/hb-array.hh:359:14: error: missing default argument on parameter 'ds'
              Ts... ds) const
                    ^
../src/hb-ot-tag.cc:292:58: note: in instantiation of function template specialization 'hb_sorted_array_t<const LangTag>::bfind<const char *, unsigned int>' requested here
    if (hb_sorted_array (ot_languages, ot_languages_len).bfind (lang_str, &tag_idx,
                                                         ^
1 error generated.
2022-05-17 15:28:50 -06:00
Behdad Esfahbod c1f4b57c06 [ot-tags] Optimize language comparison
Now that we know both strings are of equal len of 2 or 3, optimize.

Part of https://github.com/harfbuzz/harfbuzz/issues/3591

Before:
------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations
------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY       8.50 ns         8.47 ns     81221549
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN         79.6 ns         79.3 ns      8785804
BM_hb_ot_tags_from_script_and_language/COMMON en_US         40.0 ns         39.9 ns     17462768
BM_hb_ot_tags_from_script_and_language/LATIN en_US          39.2 ns         39.1 ns     17886793
BM_hb_ot_tags_from_script_and_language/COMMON none          4.31 ns         4.30 ns    162805417
BM_hb_ot_tags_from_script_and_language/LATIN none           4.32 ns         4.31 ns    162656688

After:
------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations
------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY       8.27 ns         8.24 ns     81868701
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN         56.1 ns         56.0 ns     12353284
BM_hb_ot_tags_from_script_and_language/COMMON en_US         24.3 ns         24.2 ns     28955030
BM_hb_ot_tags_from_script_and_language/LATIN en_US          24.5 ns         24.4 ns     28664868
BM_hb_ot_tags_from_script_and_language/COMMON none          4.35 ns         4.34 ns    161190014
BM_hb_ot_tags_from_script_and_language/LATIN none           4.36 ns         4.34 ns    161319000
2022-05-17 15:19:40 -06:00
Behdad Esfahbod dde48d78c1 Fix compiler warning 2022-05-17 15:07:49 -06:00
Behdad Esfahbod 15be0deda0 [ot-tags] Optimize lang_matches()
Part of https://github.com/harfbuzz/harfbuzz/issues/3591

Before:
------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations
------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY       8.67 ns         8.64 ns     80324382
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN         91.2 ns         90.9 ns      7674131
BM_hb_ot_tags_from_script_and_language/COMMON en_US         41.1 ns         41.0 ns     17174093
BM_hb_ot_tags_from_script_and_language/LATIN en_US          41.3 ns         41.2 ns     17000876
BM_hb_ot_tags_from_script_and_language/COMMON none          4.56 ns         4.55 ns    153914130
BM_hb_ot_tags_from_script_and_language/LATIN none           4.53 ns         4.52 ns    153830303

After:
------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations
------------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON abcd_XY       8.24 ns         8.21 ns     84078465
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN         77.5 ns         77.2 ns      9059230
BM_hb_ot_tags_from_script_and_language/COMMON en_US         38.8 ns         38.7 ns     17790692
BM_hb_ot_tags_from_script_and_language/LATIN en_US          37.6 ns         37.5 ns     18648293
BM_hb_ot_tags_from_script_and_language/COMMON none          4.50 ns         4.49 ns    155573267
BM_hb_ot_tags_from_script_and_language/LATIN none           4.49 ns         4.47 ns    156456653
2022-05-17 14:57:08 -06:00
Behdad Esfahbod dd3c858f84 [ot-tags] Speed up hb_ot_tags_from_language()
Part of https://github.com/harfbuzz/harfbuzz/issues/3591

"After that, bulk of the time I suppose is spent in binary-searching the
language table. I suggest we split the language table in 2-letter and
3-letter tags, to speed-up the vast majority of cases that are
2-letter."

benchmark-ot, before:

----------------------------------------------------------------------------------------------
Benchmark                                                    Time             CPU   Iterations
----------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN        112 ns          111 ns      6286271
BM_hb_ot_tags_from_script_and_language/COMMON en_US       60.6 ns         60.4 ns     11671176
BM_hb_ot_tags_from_script_and_language/LATIN en_US        61.3 ns         61.1 ns     11442645
BM_hb_ot_tags_from_script_and_language/COMMON none        4.75 ns         4.74 ns    146997235
BM_hb_ot_tags_from_script_and_language/LATIN none         4.65 ns         4.64 ns    150938747

After:

----------------------------------------------------------------------------------------------
Benchmark                                                    Time             CPU   Iterations
----------------------------------------------------------------------------------------------
BM_hb_ot_tags_from_script_and_language/COMMON zh_CN       89.5 ns         89.2 ns      7747649
BM_hb_ot_tags_from_script_and_language/COMMON en_US       38.5 ns         38.4 ns     18199432
BM_hb_ot_tags_from_script_and_language/LATIN en_US        39.0 ns         38.9 ns     18049238
BM_hb_ot_tags_from_script_and_language/COMMON none        4.53 ns         4.52 ns    154895110
BM_hb_ot_tags_from_script_and_language/LATIN none         4.54 ns         4.52 ns    154762105
2022-05-17 14:28:28 -06:00
Alexis King 1bc4bad7a5 [ot-tag] Add HB_SCRIPT_MATH (Zmth) and map it to OT ‘math’ tag
The ISO 15924 code for mathematical notation is ‘Zmth’, but the
OpenType script is ‘math’.
2022-02-08 12:40:15 -06:00
Behdad Esfahbod 2337f0d047 Internally use hb_malloc/.../hb_free instead of malloc/.../free
Redefining those stock names as macros was conflicting with gcc 10
headers.

Fixes https://github.com/harfbuzz/harfbuzz/issues/3044
2021-07-08 10:54:09 -07:00
Khaled Hosny 5151c2b668 [introspection] Replace deprecated allow-none
Use optional or nullable as appropriate.
2020-12-30 19:21:49 -08:00
Khaled Hosny 8e8c3e5878 [docs] Complete hb-ot-layout docs a bit 2020-12-30 19:21:49 -08:00
David Corbett b2e7bb2a7c Don’t map BCP 47 to coincidentally similar OT tag 2020-11-22 19:35:47 -08:00
David Corbett b207eab842 Round-trip OpenType tags through BCP 47 2020-05-15 15:00:15 -07:00
Behdad Esfahbod 8fe15485cb [config] Add HB_NO_OT_TAG
Part of https://github.com/harfbuzz/harfbuzz/issues/1652
2019-06-26 13:49:42 -07:00
Behdad Esfahbod 971330c058 [config] Add HB_NO_LANGUAGE_PRIVATE_SUBTAG
Part of https://github.com/harfbuzz/harfbuzz/issues/1652
2019-06-19 16:37:35 -07:00
Behdad Esfahbod f642a5fa6c Restructure code 2019-06-19 16:36:09 -07:00
David Corbett 4f37c0db9b Remove 'mym3' 2019-06-11 15:41:37 -07:00
Behdad Esfahbod fca2786041 [config] Make HB_DISABLE_DEPRECATED actually compile
Part of https://github.com/harfbuzz/harfbuzz/issues/1652
2019-05-11 00:37:01 -07:00
Behdad Esfahbod 41248cce0e Remove MIN/MAX in favor of hb_min/hb_max 2019-05-07 20:54:31 -07:00
David Corbett 6745a600bf Comment out ot_languages where fallback suffices 2019-04-17 10:28:59 -04:00
David Corbett 1ce11b4437 Reduce LangTag from 3 language system tags to 1 2019-04-16 11:41:01 -04:00
Behdad Esfahbod d2db71fdc4 Use internal bsearch() for language tags
Fixes https://github.com/harfbuzz/harfbuzz/pull/1639
2019-03-28 21:04:34 -07:00
Ebrahim Byagowi e412008599 Remove redundant void from C++ sources (#1486) 2018-12-17 13:01:01 -05:00
Behdad Esfahbod 39bd07aed5 Fix bunch of unused parameter warnings
Show up with gcc -O0.

There's a few more but those are functions that need to be filled in.

Maybe this is a lost battle...
2018-10-26 21:22:26 -07:00
Bruce Mitchener 8d1e479d1d Use bool literals instead of 0/1. 2018-10-19 18:05:46 +02:00
Behdad Esfahbod 3d9a0306eb 2.0.0 2018-10-18 05:58:17 -07:00
HinTak 26092bb3d1 "allow-none" annotation for "out" parameters
Fixes the following warnings:
hb-ot-tag.cc:330: Warning: HarfBuzz: invalid "allow-none" annotation: only valid for pointer types and out parameters
hb-ot-tag.cc:334: Warning: HarfBuzz: invalid "allow-none" annotation: only valid for pointer types and out parameters
2018-10-16 17:36:10 -07:00
David Corbett c55100000b Add missing colons to GObject annotations 2018-10-11 22:47:35 -04:00
Behdad Esfahbod 5646dcbd11 Minor 2018-10-11 19:39:07 -04:00
David Corbett 28d091d045 Parse Indic3 tags 2018-10-11 17:44:13 -04:00
Behdad Esfahbod da591f2a9d Whitespace 2018-10-11 14:30:15 -04:00
Behdad Esfahbod 8061664ad1 Add doc stubs for recently added API
Thanks to David Corbett who revamped our script and language processing
and implemented full BCP 47 support.

https://github.com/harfbuzz/harfbuzz/pull/730

New API:
+hb_ot_layout_table_select_script()
+hb_ot_layout_script_select_language()
+HB_OT_MAX_TAGS_PER_SCRIPT
+HB_OT_MAX_TAGS_PER_LANGUAGE
+hb_ot_tags_from_script_and_language()
+hb_ot_tags_to_script_and_language()

Deprecated API:
-hb_ot_layout_table_choose_script()
-hb_ot_layout_script_find_language()
-hb_ot_tags_from_script()
-hb_ot_tag_from_language()
2018-10-11 14:17:17 -04:00
David Corbett 7f1fbfe2e3 Add hb_ot_tags_to_script_and_language 2018-10-11 13:54:28 -04:00
David Corbett 7c7cb2a989 Match extlang subtags
If the second subtag of a BCP 47 tag is three letters long, it denotes
an extended language. The tag converter ignores the language subtag and
uses the extended language instead.

There are some grandfathered exceptions, which are handled earlier.
2018-10-11 13:54:28 -04:00
David Corbett 2f1f961cc0 Autogenerate the BCP 47 to OpenType mappings
The new script, gen-tag-table.py, generates `ot_languages` automatically
from the [OpenType language system tag registry][ot] and the [IANA
Language Subtag Registry][bcp47] with some manual modifications. If an
OpenType tag maps to a BCP 47 macrolanguage, all the macrolanguage's
individual languages are mapped to the same OpenType tag, except for
individual languages with their own OpenType mappings. Deprecated
BCP 47 tags are canonicalized.

[ot]: https://docs.microsoft.com/en-us/typography/opentype/spec/languagetags
[bcp47]: https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry

Some OpenType tags correspond to multiple ISO 639 codes. The mapping
from ISO 639 codes lists OpenType tags in priority order, such that more
specific or more likely tags appear first.

Some OpenType tags have no corresponding ISO 639 code in the registry so
their mappings use BCP 47 subtags besides the language. For example, any
BCP 47 tag with a fonipa variant subtag is mapped to 'IPPH', and 'IPPH'
is mapped back to und-fonipa.

Other OpenType tags have no corresponding ISO 639 code because it is not
clear what they are for. HarfBuzz just ignores these tags.

One such ignored tag is 'ZHP ' (Chinese Phonetic). It probably means
zh-Latn. However, it is used in Microsoft JhengHei and Microsoft YaHei
with the script tag 'hani', implying that it is not a romanization
scheme after all. It would be simple enough to add this mapping to
gen-tag-table.py once a definitive mapping is determined.

The manual modifications are mainly either obvious mappings that the
OpenType registry omits or mappings for compatibility with previous
versions of HarfBuzz. Some of the old mappings were discarded, though,
for homophonous language names. For example, OpenType maps 'KUI ' to
kxu; previous versions of HarfBuzz also mapped it to kvd, because kvd
and kxu both happen to be called "Kui".

gen-tag-table.py also generates a function to convert multi-subtag tags
like el-polyton and zh-HK to OpenType tags, replacing `ot_languages_zh`
and the hard-coded list of special cases in `hb_ot_tags_from_language`.
It also generates a function to convert OpenType tags to BCP 47,
replacing the hard-coded list of special cases in
`hb_ot_tag_to_language`.
2018-10-11 13:54:28 -04:00
David Corbett 91067716f5 Refactor the selection of script and language tags
The old hb-ot-tag.cc functions, `hb_ot_tags_from_script` and
`hb_ot_tag_from_language`, are now wrappers around a new function:
`hb_ot_tags`. It converts a script and a language to arrays of script
tags and language tags. This will make it easier to add new script tags
to scripts, like 'dev3'. It also allows for language fallback chains;
nothing produces more than one language yet though.

Where the old functions return the default tags 'DFLT' and 'dflt',
`hb_ot_tags` returns an empty array. The caller is responsible for
using the default tag in that case.

The new function also adds a new private use subtag syntax for script
overrides: "x-hbscabcd" requests a script tag of 'abcd'.

The old hb-ot-layout.cc functions,`hb_ot_layout_table_choose_script` and
`hb_ot_layout_script_find_language` are now wrappers around the new
functions `hb_ot_layout_table_select_script` and
`hb_ot_layout_script_select_language`. They are essentially the same as
the old ones plus a tag count parameter.

Closes #495.
2018-10-11 13:54:28 -04:00