Commit Graph

927 Commits

Author SHA1 Message Date
Behdad Esfahbod c0a6814b49 Touch up new API
New API:
+hb_ot_layout_feature_get_name_ids()
+hb_ot_layout_feature_get_characters()
2018-10-12 16:06:39 -04:00
Ebrahim Byagowi dc49bd8d81 Add two APIs for getting stylistic set labels
* hb_ot_layout_feature_get_characters
* hb_ot_layout_feature_get_name_ids

However HarfBuzz currently doesn't expose an API for retrieving the actual
information associated with NameId from the `name` table and that should be
done separately.
2018-10-12 16:06:39 -04:00
Behdad Esfahbod bf8469be9a Attach CursivePositioning backwards, not forward
This is how Uniscribe does it.  So, adjust.  This is only relevant
to fonts that apply cursive positioning from a contextual lookup.

Fixes https://github.com/harfbuzz/harfbuzz/issues/1181
2018-10-11 20:45:40 -04:00
Behdad Esfahbod bdb53ca24a [myanmar] Implement Zawgyi shaper
Enabled if script tag 'Qaag' is passed to HarfBuzz.  Disables mark
advance-zeroing and fallback mark-positioning.

Fixes https://github.com/harfbuzz/harfbuzz/issues/1162
2018-10-11 20:20:29 -04:00
Behdad Esfahbod 788e147855 [test] Add test for USE indic3 2018-10-11 19:24:52 -04:00
Behdad Esfahbod a11972787a Minor 2018-10-11 19:23:36 -04:00
David Corbett 28d091d045 Parse Indic3 tags 2018-10-11 17:44:13 -04:00
Behdad Esfahbod 211cd3691b Remove remains of get-codepoint-fuzzer 2018-10-11 17:31:29 -04:00
Behdad Esfahbod 120ed02725 [fuzzing] Fold get-codepoints-fuzzer into subset-fuzzer 2018-10-11 17:08:12 -04:00
Behdad Esfahbod 1d995a340b Minor 2018-10-11 15:42:54 -04:00
Behdad Esfahbod 57b05210b1 [test] Fix use of deprecated symbols 2018-10-11 15:03:21 -04:00
David Corbett 7f1fbfe2e3 Add hb_ot_tags_to_script_and_language 2018-10-11 13:54:28 -04:00
David Corbett 65d01f7755 Test deprecated tag fallback in a font
The font supports the deprecated tag 'DHV ' instead of 'DIV '. dv is
mapped to 'DIV ' and 'DHV ', in that order. The test specifies
`--language=dv`, demonstrating that if a font does not support the first
OpenType tag mapped to a BCP 47 tag, it will fall back to the next tag.
2018-10-11 13:54:28 -04:00
David Corbett 7c7cb2a989 Match extlang subtags
If the second subtag of a BCP 47 tag is three letters long, it denotes
an extended language. The tag converter ignores the language subtag and
uses the extended language instead.

There are some grandfathered exceptions, which are handled earlier.
2018-10-11 13:54:28 -04:00
David Corbett 2f1f961cc0 Autogenerate the BCP 47 to OpenType mappings
The new script, gen-tag-table.py, generates `ot_languages` automatically
from the [OpenType language system tag registry][ot] and the [IANA
Language Subtag Registry][bcp47] with some manual modifications. If an
OpenType tag maps to a BCP 47 macrolanguage, all the macrolanguage's
individual languages are mapped to the same OpenType tag, except for
individual languages with their own OpenType mappings. Deprecated
BCP 47 tags are canonicalized.

[ot]: https://docs.microsoft.com/en-us/typography/opentype/spec/languagetags
[bcp47]: https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry

Some OpenType tags correspond to multiple ISO 639 codes. The mapping
from ISO 639 codes lists OpenType tags in priority order, such that more
specific or more likely tags appear first.

Some OpenType tags have no corresponding ISO 639 code in the registry so
their mappings use BCP 47 subtags besides the language. For example, any
BCP 47 tag with a fonipa variant subtag is mapped to 'IPPH', and 'IPPH'
is mapped back to und-fonipa.

Other OpenType tags have no corresponding ISO 639 code because it is not
clear what they are for. HarfBuzz just ignores these tags.

One such ignored tag is 'ZHP ' (Chinese Phonetic). It probably means
zh-Latn. However, it is used in Microsoft JhengHei and Microsoft YaHei
with the script tag 'hani', implying that it is not a romanization
scheme after all. It would be simple enough to add this mapping to
gen-tag-table.py once a definitive mapping is determined.

The manual modifications are mainly either obvious mappings that the
OpenType registry omits or mappings for compatibility with previous
versions of HarfBuzz. Some of the old mappings were discarded, though,
for homophonous language names. For example, OpenType maps 'KUI ' to
kxu; previous versions of HarfBuzz also mapped it to kvd, because kvd
and kxu both happen to be called "Kui".

gen-tag-table.py also generates a function to convert multi-subtag tags
like el-polyton and zh-HK to OpenType tags, replacing `ot_languages_zh`
and the hard-coded list of special cases in `hb_ot_tags_from_language`.
It also generates a function to convert OpenType tags to BCP 47,
replacing the hard-coded list of special cases in
`hb_ot_tag_to_language`.
2018-10-11 13:54:28 -04:00
David Corbett 91067716f5 Refactor the selection of script and language tags
The old hb-ot-tag.cc functions, `hb_ot_tags_from_script` and
`hb_ot_tag_from_language`, are now wrappers around a new function:
`hb_ot_tags`. It converts a script and a language to arrays of script
tags and language tags. This will make it easier to add new script tags
to scripts, like 'dev3'. It also allows for language fallback chains;
nothing produces more than one language yet though.

Where the old functions return the default tags 'DFLT' and 'dflt',
`hb_ot_tags` returns an empty array. The caller is responsible for
using the default tag in that case.

The new function also adds a new private use subtag syntax for script
overrides: "x-hbscabcd" requests a script tag of 'abcd'.

The old hb-ot-layout.cc functions,`hb_ot_layout_table_choose_script` and
`hb_ot_layout_script_find_language` are now wrappers around the new
functions `hb_ot_layout_table_select_script` and
`hb_ot_layout_script_select_language`. They are essentially the same as
the old ones plus a tag count parameter.

Closes #495.
2018-10-11 13:54:28 -04:00
Behdad Esfahbod 100e95f48e [trak] Add tests 2018-10-11 11:30:45 -04:00
Behdad Esfahbod 1e8fdd285f Remove HAVE_OT
We never tested compiling without it.  Just kill it.  We always build
our own shaper.
2018-10-10 16:32:35 -04:00
Behdad Esfahbod be2f148da4 [ft] Use mutex to lock access to FT_Face
Makes our FT-backed hb_font_t safe to use from multiple threads.  Still,
the underlying FT_Face should NOT be used from other threads by client
or other libraries.

Maybe I add a lock()/unlock() public API ala PangoFT2 and cairo-ft.
Maybe not.
2018-10-09 16:24:50 -04:00
Behdad Esfahbod 9f79365c3b Fix warning
How come this one is not generated by clang everything bot?!

../../../test/api/test-multithread.c:37:26: warning: initialization discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
 static char *font_path = "fonts/Inconsolata-Regular.abc.ttf";
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../test/api/test-multithread.c:38:21: warning: initialization discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
2018-10-09 14:36:25 -04:00
Chun-wei Fan 4035158de4 test/api/test-subset-*.c: Fix build on pre-C99
Ensure that we have the variables at the beginning of the block.  These
are the only fixes that we need for building HarfBuzz on older
compilers.
2018-10-09 09:43:36 -04:00
Behdad Esfahbod eb2be97f86 Port test off deprecated API 2018-10-09 07:48:52 -04:00
Behdad Esfahbod cc126f2817 Minor 2018-10-09 01:10:15 -04:00
Behdad Esfahbod a353c1768d Remove test for deprecated hb_set_invert() 2018-10-09 01:05:28 -04:00
Behdad Esfahbod fc50916589 Remove deprecated decompose_compatibility stuff 2018-10-09 00:39:42 -04:00
Behdad Esfahbod eed737f672 Remove deprecated eastasian_width 2018-10-09 00:37:10 -04:00
Behdad Esfahbod 9c1bb81f5c [test/text-rendering-tests] Update from upstream 2018-10-08 16:11:14 -04:00
Behdad Esfahbod 2a5cb37fdb Revert "[morx] Fix MORX-35"
This reverts commit f62f6e90ad.
2018-10-07 20:37:15 -04:00
Behdad Esfahbod f62f6e90ad [morx] Fix MORX-35
Fixes https://github.com/harfbuzz/harfbuzz/issues/1224
2018-10-07 18:52:48 -04:00
Behdad Esfahbod c07b91b812 [test/text-rendering-tests] Update from upstream 2018-10-07 18:00:14 -04:00
Behdad Esfahbod f92330b5e0 Minor 2018-10-07 17:51:50 +02:00
Ebrahim Byagowi e9abe33963
Use tempfile in run-tests as a fix for Windows CI fails (#1228) 2018-10-06 12:19:33 +03:30
Behdad Esfahbod 5a41cf6be6 [test/text-rendering-tests] Update from upstream 2018-10-05 18:09:12 +02:00
Behdad Esfahbod 7b3ae5ffd2 More bot fixes 2018-10-04 16:36:42 +02:00
Behdad Esfahbod 0abce58139 [test] Choose 'ot' shaper specifically
Now that we added morx support, our OS X bot is running them through CoreText
and failing (with a DoS / infinite loop no less!).  Always run tests through
our own shaper.
2018-10-04 16:23:42 +02:00
Behdad Esfahbod 55468ca01b [test/text-rendering-tests] Update from upstream 2018-10-04 12:19:30 +02:00
Behdad Esfahbod 6ff8a8a10b Minor 2018-10-04 12:09:45 +02:00
Behdad Esfahbod 14ff3cbe0f Apply morx if there's no GSUB! 2018-10-04 11:34:21 +02:00
Behdad Esfahbod 81f5eb09ec Add emoji test for recent work 2018-10-03 21:32:52 +02:00
Behdad Esfahbod 03fb6dd4c7 Rewrite grapheme-formation in terms of new work
Also, don't attach ZWNJ to previous cluster.  Closer to Unicode graphemes.
2018-10-03 21:11:40 +02:00
Behdad Esfahbod 3f1c741b7a [test] Split a test 2018-10-03 21:11:40 +02:00
Ebrahim Byagowi 4146c00caa
[test] Use an in-repo font for test-multithread (#1218)
As Khaled's suggestion, hard-coded font paths was only for my own testing.
2018-10-03 21:26:58 +03:30
lantw44 7a1ab0464d Fix test-multithread build on FreeBSD (#1217)
Add the default font path used by FreeBSD ports.
2018-10-03 18:52:43 +03:30
Behdad Esfahbod df32eaae42 [indic] Disallow vowel mark combinations that spoof other vowel marks
Fixes https://github.com/harfbuzz/harfbuzz/issues/1019

New numbers:

BENGALI: 353725 out of 354188 tests passed. 463 failed (0.130722%)
DEVANAGARI: 707261 out of 707394 tests passed. 133 failed (0.0188014%)
GUJARATI: 366353 out of 366457 tests passed. 104 failed (0.0283799%)
GURMUKHI: 60729 out of 60747 tests passed. 18 failed (0.0296311%)
KANNADA: 951300 out of 951913 tests passed. 613 failed (0.0643966%)
MALAYALAM: 1048136 out of 1048334 tests passed. 198 failed (0.0188871%)
ORIYA: 42327 out of 42329 tests passed. 2 failed (0.00472489%)
SINHALA: 271596 out of 271847 tests passed. 251 failed (0.0923313%)
TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%)
TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)

Devanagari regressed because Uniscribe doesn't enforce the full set.

Tests added with the *-vowel-letters.txt files in tree and Noto fonts.
2018-10-03 15:53:28 +02:00
Behdad Esfahbod d3d0cbd278 Typo 2018-10-03 15:31:33 +02:00
Behdad Esfahbod 75114e01d2 [use] Add Halant_Or_Vowel_Modifier category
Fixes https://github.com/harfbuzz/harfbuzz/issues/1102
2018-10-03 12:30:23 +02:00
Ebrahim Byagowi 6353cc1f83
[circleci] Fix some of warnings from clang-everything bot (#1211)
* -Wshift-sign-overflow
* -Wmissing-prototypes
2018-10-02 21:39:19 +03:30
Behdad Esfahbod 9f1dee32fb [tests] Allow test-runner to ignore shaping output
Fixes https://github.com/harfbuzz/harfbuzz/issues/1212
2018-10-02 18:56:20 +02:00
Behdad Esfahbod d992982d23 [indic] Add some confusable sequences from Unicode Standard 2018-10-02 17:31:11 +02:00
Behdad Esfahbod 729f634728 Disable joiner-skipping when looking back for base to attach mark to
Fixes https://github.com/harfbuzz/harfbuzz/issues/1189
2018-10-02 14:41:31 +02:00