Before, we were just checking the use_category(). This detects as
halant a ligature that had the halant as first glyph (as seen in
NotoSansBalinese.) Change that to use the is_ligated() glyph prop
bit. The font is forming this ligature in ccmp, which is before
the rphf / pref tests. So we need to make sure the "ligated" bit
survives those tests. Since those only check the "substituted" bit,
we now only clear that bit for them and "ligated" survives.
Fixes https://github.com/behdad/harfbuzz/issues/180
Use the DEFINE_ENUM_FLAG_OPERATORS macro in winnt.h on Visual Studio,
which defines the bitwise operators for the enumerations that we want to
mark as hb_mark_as_flags_t, which will take care of the situation on newer
Visual Studio (>= 2012), where the build breaks with C2057 errors as the
underlying types of the enumerations is not clear to the compiler when we
do a bitwise op within the declaration of the enumerations themselves.
Also disable the C4200 (nonstandard extension used : zero-sized array in
struct/union) and C4800 ('type' : forcing value to bool 'true' or 'false'
(performance warning)) warnings as the C4200 is the intended scenario and
C4800 is harmless but is so far an unavoidable side effect of using
DEFINE_ENUM_FLAG_OPERATORS.
This resurrects the space fallback feature, after I disabled
the compatibility decomposition. Now I can release HarfBuzz
again without breaking Pango!
It also remembers which space character it was, such that later
on we can approximate the width of this particular space
character. That part is not implemented yet.
We normalize all GC=Zs chars except for U+1680 OGHA SPACE MARK,
which is better left alone.
"Fixes" https://bugs.freedesktop.org/show_bug.cgi?id=86300
Based on discussion someone else who had a similar issue, most probably
the user is releasing FT_Face before destructing hb_face_t / hb_font_t.
While that's a client bug, and while we can (and should) use FreeType
refcounting to help avoid that, it happens that we were accessing
the table when we didn't really have to. Avoid that.
Sometimes font designers form half/pref/etc consonant forms
unconditionally and then undo that conditionally. Try to
recover the OT_H classification in those cases.
No test number changes expected.
Normally if you want to, say, conditionally prevent a 'pref', you
would use blocking contextual matching. Some designers instead
form the 'pref' form, then undo it in context. To detect that
we now also remember glyphs that went through MultipleSubst.
In the only place that this is used, Uniscribe seems to only care
about the "last" transformation between Ligature and Multiple
substitions. Ie. if you ligate, expand, and ligate again, it
moves the pref, but if you ligate and expand it doesn't. That's
why we clear the MULTIPLIED bit when setting LIGATED.
Micro-test added. Test: U+0D2F,0D4D,0D30 with font from:
[1]
https://code.google.com/a/google.com/p/noto-alpha/issues/detail?id=186#c29
Before we were just relying on the compiler inlining them and not
leaving a trace in our public API. Try to fix. Hopefully not
breaking anyone's build.
Unicode 6.2.0 Section 16.2 / Figure 16.3 says:
"For backward compatibility, between Arabic characters a ZWJ acts just
like the sequence <ZWJ, ZWNJ, ZWJ>, preventing a ligature from forming
instead of requesting the use of a ligature that would not normally be
used. As a result, there is no plain text mechanism for requesting the
use of a ligature in Arabic text."
As such, we flip internal zwj to zwnj flags for GSUB matching, which
means it will block ligation in all features, unless the font
explicitly matches U+200D glyph. This doesn't affect joining behavior.
If there's a mark ligating forward with non-mark, they were
inheriting the GC of the mark and later get advance-zeroed.
Don't do that if there's any non-mark glyph in the ligature.
Sample test: U+1780,U+17D2,U+179F with Kh-Metal-Chrieng.ttf
Also:
Bug 58922 - Issue with mark advance zeroing in generic shaper
After the Ngapi hackfest work, we were assuming that fonts
won't use presentation features to choose specific forms
(eg. conjuncts). As such, we were using auto-joiner behavior
for such features. It proved to be troublesome as many fonts
used presentation forms ('pres') for example to form conjuncts,
which need to be disabled when a ZWJ is inserted.
Two examples:
U+0D2F,U+200D,U+0D4D,U+0D2F with kartika.ttf
U+0995,U+09CD,U+200D,U+09B7 with vrinda.ttf
What we do now is to never do magic to ZWJ during GSUB's main input
match for Indic-style shapers. Note that backtrack/lookahead are still
matched liberally, as is GPOS. This seems to be an acceptable
compromise.
As to the bug that initially started this work, that one needs to
be fixed differently:
Bug 58714 - Kannada u+0cb0 u+200d u+0ccd u+0c95 u+0cbe does not
provide same results as Windows8
https://bugs.freedesktop.org/show_bug.cgi?id=58714
New numbers:
BENGALI: 353689 out of 354188 tests passed. 499 failed (0.140886%)
DEVANAGARI: 707305 out of 707394 tests passed. 89 failed (0.0125814%)
GUJARATI: 366349 out of 366457 tests passed. 108 failed (0.0294714%)
GURMUKHI: 60706 out of 60747 tests passed. 41 failed (0.067493%)
KANNADA: 951030 out of 951913 tests passed. 883 failed (0.0927606%)
KHMER: 299070 out of 299124 tests passed. 54 failed (0.0180527%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048102 out of 1048334 tests passed. 232 failed (0.0221304%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%)
TAMIL: 1091753 out of 1091754 tests passed. 1 failed (9.15957e-05%)
TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
Not for Arabic, but for Indic-like scripts. ZWJ/ZWNJ have special
meanings in those scripts, so let font lookups take full control.
This undoes the regression caused by automatic-joiners handling
introduced two commits ago.
We only disable automatic joiner handling for the "basic shaping
features" of Indic, Myanmar, and SEAsian shapers. The "presentation
forms" and other features are still applied with automatic-joiner
handling.
This change also changes the test suite failure statistics, such that
a few scripts show more "failures". The most affected is Kannada.
However, upon inspection, we believe that in most, if not all, of the
new failures, we are producing results superior to Uniscribe. Hard to
count those!
Here's an example of what is fixed by the recent joiner-handling
changes:
https://bugs.freedesktop.org/show_bug.cgi?id=58714
New numbers, for future reference:
BENGALI: 353892 out of 354188 tests passed. 296 failed (0.0835714%)
DEVANAGARI: 707336 out of 707394 tests passed. 58 failed (0.00819911%)
GUJARATI: 366262 out of 366457 tests passed. 195 failed (0.0532122%)
GURMUKHI: 60706 out of 60747 tests passed. 41 failed (0.067493%)
KANNADA: 950680 out of 951913 tests passed. 1233 failed (0.129529%)
KHMER: 299074 out of 299124 tests passed. 50 failed (0.0167155%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047983 out of 1048334 tests passed. 351 failed (0.0334817%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271539 out of 271847 tests passed. 308 failed (0.113299%)
TAMIL: 1091753 out of 1091754 tests passed. 1 failed (9.15957e-05%)
TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
Behdad Esfahbod
Chun-wei Fan