Commit Graph

349 Commits

Author SHA1 Message Date
Behdad Esfahbod 45b7ec3652 [indic] Followup fix for Malayalam context matching
We regeressed Malayalam in 508cc3d3cf
This brings down the failures to 198 (from 750).

BENGALI: 353725 out of 354188 tests passed. 463 failed (0.130722%)
DEVANAGARI: 707307 out of 707394 tests passed. 87 failed (0.0122987%)
GUJARATI: 366349 out of 366457 tests passed. 108 failed (0.0294714%)
GURMUKHI: 60732 out of 60747 tests passed. 15 failed (0.0246926%)
KANNADA: 951190 out of 951913 tests passed. 723 failed (0.0759523%)
KHMER: 299070 out of 299124 tests passed. 54 failed (0.0180527%)
MALAYALAM: 1048136 out of 1048334 tests passed. 198 failed (0.0188871%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271662 out of 271847 tests passed. 185 failed (0.068053%)
TAMIL: 1091753 out of 1091754 tests passed. 1 failed (9.15957e-05%)
TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)
MYANMAR: 1123865 out of 1123883 tests passed. 18 failed (0.00160159%)
2015-12-18 13:47:16 +00:00
Behdad Esfahbod 2813e3049a [indic] Update data tables to Unicode 8.0
Test stats remain unchanged, except for Malayalam, which we investigate:

BENGALI: 353725 out of 354188 tests passed. 463 failed (0.130722%)
DEVANAGARI: 707307 out of 707394 tests passed. 87 failed (0.0122987%)
GUJARATI: 366349 out of 366457 tests passed. 108 failed (0.0294714%)
GURMUKHI: 60732 out of 60747 tests passed. 15 failed (0.0246926%)
KANNADA: 951190 out of 951913 tests passed. 723 failed (0.0759523%)
KHMER: 299070 out of 299124 tests passed. 54 failed (0.0180527%)
MALAYALAM: 1047584 out of 1048334 tests passed. 750 failed (0.0715421%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271662 out of 271847 tests passed. 185 failed (0.068053%)
TAMIL: 1091753 out of 1091754 tests passed. 1 failed (9.15957e-05%)
TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)

Myanmar, compared to Windows 10 mmrtext.ttf:

MYANMAR: 1123865 out of 1123883 tests passed. 18 failed (0.00160159%)
2015-12-18 11:05:11 +00:00
Behdad Esfahbod 508cc3d3cf [indic] Allow context when matching for Malayalam new-spec
Test sequence:
	U+0995,U+09CD,U+09B0

With Nirmala shipped on Windows 10, this failed to form the below form.
Works now.

Reported by Sairus.
2015-12-17 17:31:17 +00:00
Behdad Esfahbod ea512f7108 Use C-style casts instead of compare to 0, to convert hb_bool_t to bool 2015-11-26 19:22:22 -05:00
Behdad Esfahbod 766963adba Merge pull request #114 from ThePhD/vc++-fixes
Fix all VC++ warnings and errors
2015-11-24 15:38:43 -06:00
Behdad Esfahbod abadc1717d Try to better handle OOM situations
Fixes assert fail in https://github.com/behdad/harfbuzz/issues/161
with libharfbuzz-fuzzing.
2015-11-18 17:52:08 -08:00
Behdad Esfahbod 136863371c Add new shaper method postprocess_glyphs()
Unused currently.  To be used for Syriac stretch implementation.

https://github.com/behdad/harfbuzz/issues/141
2015-11-05 13:24:15 -08:00
Behdad Esfahbod 6f932bc8f9 Fix a few more -Wshadow-local warnings
https://bugzilla.mozilla.org/show_bug.cgi?id=1215894
2015-10-21 11:16:49 -02:00
Behdad Esfahbod b90cb366d7 Fix one -Wshadow-compatible-local warning
From https://bugzilla.mozilla.org/show_bug.cgi?id=1215894
2015-10-21 11:13:21 -02:00
Behdad Esfahbod 305d2fbf5a Add HB_FALLTHROUGH
Borrowed from https://bugzilla.mozilla.org/show_bug.cgi?id=1215411
2015-10-21 11:04:28 -02:00
Behdad Esfahbod 5828c45d7a [indic] Add comments to merge_clusters calls 2015-09-01 16:34:12 +01:00
Behdad Esfahbod 85846b3de7 Use insertion-sort instead of bubble-sort
Needed for upcoming merge-clusters fix.
2015-09-01 15:07:52 +01:00
ThePhD 5c99cf93d6 Merge branch 'master' into vc++-fixes 2015-08-14 01:02:00 -04:00
Behdad Esfahbod ecb0b24ef3 Use foreach_cluster in Indic shaper 2015-07-22 12:02:09 +01:00
Behdad Esfahbod f8160a4959 Add FLAG_SAFE() for values known to be small-enough
And add check to FLAG()
2015-07-21 15:50:02 +01:00
ThePhD 8e545d5961 Fix all VC++ warnings and errors in the current commit's builds. 2015-06-22 22:29:04 -04:00
Behdad Esfahbod 595d2b96c3 Minor 2014-07-26 18:44:15 -04:00
Behdad Esfahbod fc0daafab0 [indic] Handle old-spec Malayalam reordering with final Halant
See comment.

Micro-tests added.
2014-07-23 16:53:03 -04:00
Behdad Esfahbod 7cd33f2304 Micro optimization 2014-07-17 14:39:07 -04:00
Behdad Esfahbod e7ce50d9eb [indic] Fix access past end of array 2014-07-16 12:30:39 -04:00
Behdad Esfahbod 7627100f42 Mark unsigned integer literals with the u suffix
Simplifies hb_in_range() calls as the type can be inferred.
The rest is obsessiveness, I admit.
2014-07-11 16:22:13 -04:00
Behdad Esfahbod 68f724484b [indic] Remove some more now-unused special-cases 2014-06-30 15:46:53 -04:00
Behdad Esfahbod e79c948980 [indic] Remove special-casing of U+1CF2,1CF3
These were introduced in a498565ced,
but IndicSyllabicCategory has had the correct value already, so the
special code was never needed.
2014-06-30 15:39:39 -04:00
Behdad Esfahbod d743ce78e1 [indic-table] Update to Unicode 7.0 data
Touch code just enough to preserve previous syllable structure
and functionality as closely as possible.  Many further cleanups
coming later.
2014-06-30 15:24:45 -04:00
Behdad Esfahbod 1d634cbb4b Fix base-position when 'pref' is NOT formed
If pre-base reordering Ra is NOT formed (or formed and then
broken up), we should consider that Ra as base.  This is
observable when there's a left matra or dotreph that positions
before base.

Now, it might be that we shouldn't do this if the Ra happend
to form a below form.  We can't quite deduce that right now...

Micro test added.  Also at:

https://code.google.com/a/google.com/p/noto-alpha/issues/detail?id=186#c29
2014-06-12 17:10:35 -04:00
Behdad Esfahbod 04dc52fa15 [indic] Recover OT_H undergone ligation and multiplication
Sometimes font designers form half/pref/etc consonant forms
unconditionally and then undo that conditionally.  Try to
recover the OT_H classification in those cases.

No test number changes expected.
2014-06-09 14:20:06 -04:00
Behdad Esfahbod 39c8201f8e [indic] Improve base re-finding
No test numbers change.
2014-06-09 14:20:06 -04:00
Behdad Esfahbod c04d5f0dd2 [indic] Minor 2014-06-09 14:20:06 -04:00
Behdad Esfahbod 832a6f99b3 [indic] Don't reorder reph/pref if ligature was expanded
Normally if you want to, say, conditionally prevent a 'pref', you
would use blocking contextual matching.  Some designers instead
form the 'pref' form, then undo it in context.  To detect that
we now also remember glyphs that went through MultipleSubst.

In the only place that this is used, Uniscribe seems to only care
about the "last" transformation between Ligature and Multiple
substitions.  Ie. if you ligate, expand, and ligate again, it
moves the pref, but if you ligate and expand it doesn't.  That's
why we clear the MULTIPLIED bit when setting LIGATED.

Micro-test added.  Test: U+0D2F,0D4D,0D30 with font from:

[1]
https://code.google.com/a/google.com/p/noto-alpha/issues/detail?id=186#c29
2014-06-05 20:36:01 -04:00
Behdad Esfahbod 17c3b809f4 [indic] Treat U+A8E0..A8F1 as OT_A instead of OT_VD
Apparently they can intermix with other OT_A.

Test: U+0915,A8E2,1CD0
2014-06-02 15:08:18 -04:00
Behdad Esfahbod 7977ca17aa [indic] Allow decimal and Brahmi digits as placeholders
Tests: U+0967,0951 U+0031,093F
2014-05-29 15:34:26 -04:00
Behdad Esfahbod 52b562a6a0 [indic] Clean up a bit
No functional change intended.
2014-05-27 18:19:52 -04:00
Behdad Esfahbod 3bf652b907 [indic] Treat U+002D and U+2010..2014 as placeholders 2014-05-27 18:07:26 -04:00
Behdad Esfahbod e0de95f402 [indic] Treat U+00D7 MULTIPLICATION SIGN as placeholder 2014-05-27 17:58:34 -04:00
Behdad Esfahbod cf78dd483c [indic/myanmar] Rename OT_NBSP to OT_PLACEHOLDER 2014-05-27 17:53:37 -04:00
Behdad Esfahbod cf71d28c38 [indic/myanmar] Refactor a few macros 2014-05-27 17:47:43 -04:00
Behdad Esfahbod 2307268e01 [indic] Treat U+0A72..0A73 like regular consonants
Unicode 6.x IndicSyllableCategory categorizes them as
placeholders, but they can subjoin.
2014-05-27 17:39:01 -04:00
Behdad Esfahbod e9b2a4cfe5 [indic] Support U+1CED 2014-05-23 15:49:10 -04:00
Behdad Esfahbod d19f8e8570 [indic] Support U+A8F2..A8F7,1CE9..1CEC,1CEE..1CF1 2014-05-23 15:47:36 -04:00
Behdad Esfahbod 9f9bd9bf31 [indic] Rename avagraha cluster to symbol cluster
In anticipation of adding more characters to that class of clusters.
2014-05-23 15:35:38 -04:00
Behdad Esfahbod a498565ced [indic] Support U+1CF2,U+1CF3 2014-05-22 19:39:56 -04:00
Behdad Esfahbod ecb98babba [indic] Support U+1CE2..U+1CE8 2014-05-22 19:36:21 -04:00
Behdad Esfahbod 37bf2c9224 Minor 2014-05-22 19:35:17 -04:00
Behdad Esfahbod 131e17ff9a [indic] Support U+1CF5,1CF6 2014-05-22 19:33:10 -04:00
Behdad Esfahbod 72ead0cc72 [indic] Treat U+1CE1 as a tone-mark too
It's spacing, but otherwise the same as the other ones.
2014-05-22 19:12:10 -04:00
Behdad Esfahbod e848bfae7c [indic] Recategorize U+A8E0..A8F1 as OT_VD
Up to two of them come after all OT_A characters.
2014-05-22 18:50:34 -04:00
Behdad Esfahbod c11fc68339 [indic] Support more extended Devanagari tone marks
Also adjust U+0953,0954 handling.
2014-05-22 18:41:49 -04:00
Behdad Esfahbod 26c836e53d [indic] Handle "Cantillation marks for the Samaveda" 2014-05-21 18:35:48 -04:00
Behdad Esfahbod 29531128f2 [indic] Improve reph formation of Sinhala and Telugu
Sinhala and Telugu use "explicit" reph.  That is, the reph is formed by
a Ra,H,ZWJ sequence.  Previously, upon detecting this sequence, we were
checking checking whether the 'rphf' feature applies to the first two
glyphs of the sequence.  This is how the Microsoft fonts are designed.
However, testing with Noto shows that apparently Uniscribe also forms
the reph if the lookup ligates all three glyphs.  So, try both
sequences.

Doesn't affect test results for Sinhala or Telugu.

https://code.google.com/a/google.com/p/noto-alpha/issues/detail?id=232
2014-05-15 14:04:02 -06:00
Behdad Esfahbod b082ef373c Typo 2014-04-25 11:48:10 -07:00