Commit Graph

2520 Commits

Author SHA1 Message Date
Behdad Esfahbod aa7141efe4 [Indic] Fix Khmer syllable-final coeng-consonant
Brings down Khmer failures from 162 to 47.

KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%)

Also rebaselined some of the test files that had only-inherited lines.
Removing those, the stats are:

BENGALI: 353893 out of 354188 tests passed. 295 failed (0.0832891%)
DEVANAGARI: 707315 out of 707394 tests passed. 79 failed (0.0111678%)
GUJARATI: 366438 out of 366457 tests passed. 19 failed (0.00518478%)
GURMUKHI: 60704 out of 60747 tests passed. 43 failed (0.0707854%)
KANNADA: 951042 out of 951913 tests passed. 871 failed (0.0915%)
KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047994 out of 1048334 tests passed. 340 failed (0.0324324%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%)
TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%)
TELUGU: 970553 out of 970573 tests passed. 20 failed (0.00206064%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)

Still some regressions, but some of the more egregious cases are
addressed.
2012-09-05 17:14:52 -04:00
Behdad Esfahbod efb8d3eb71 Fixup test failure reporting
After we implemented dotted-circle, we were still ignoring any tests
that had dottedcircle in it for any of the shapers.  That meant that if
we wrongly outputted dottedcircle, the test was being ignored.  Ouch!

Fixing that shows regressions across the board.  Most are Uniscribe
bugs: NOT inserting dotted-circle when it should.  Some are arou
machine bugs.  This is in fact a nice way to catch Indic-machine
deficiencies and when I fix the regressions, our clusters should be
much closer to Uniscribe.  For now, we regressed from:

BENGALI: 353997 out of 354285 tests passed. 288 failed (0.0812905%)
DEVANAGARI: 707339 out of 707394 tests passed. 55 failed (0.00777502%)
GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%)
GURMUKHI: 60769 out of 60809 tests passed. 40 failed (0.0657797%)
KANNADA: 951086 out of 951913 tests passed. 827 failed (0.0868777%)
KHMER: 299106 out of 299124 tests passed. 18 failed (0.00601757%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048104 out of 1048416 tests passed. 312 failed (0.0297592%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271747 out of 271847 tests passed. 100 failed (0.0367854%)
TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%)
TELUGU: 970558 out of 970573 tests passed. 15 failed (0.00154548%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)

To:

BENGALI: 353990 out of 354285 tests passed. 295 failed (0.0832663%)
DEVANAGARI: 707315 out of 707394 tests passed. 79 failed (0.0111678%)
GUJARATI: 366447 out of 366506 tests passed. 59 failed (0.016098%)
GURMUKHI: 60707 out of 60809 tests passed. 102 failed (0.167738%)
KANNADA: 951042 out of 951913 tests passed. 871 failed (0.0915%)
KHMER: 298962 out of 299124 tests passed. 162 failed (0.0541581%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048074 out of 1048416 tests passed. 342 failed (0.0326206%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%)
TAMIL: 1091835 out of 1091837 tests passed. 2 failed (0.000183178%)
TELUGU: 970553 out of 970573 tests passed. 20 failed (0.00206064%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)

Investigating.
2012-09-05 15:57:38 -04:00
Behdad Esfahbod 27bd55bd2c [Indic] Tamil does not have half-forms either
The Win7 Tamil font does not realy on this behavior, but the WinXP
version does.  Handle Tamil like Malayalam: Matras always move to
before base.

WinXP Tamil failures went down from 168964 (15.4752%) to 167
(0.0152953%) (two orders of magnitude reduction!).

Included in this is a minor fixup that actually fixed a few tests
with non-Tamil too.  Numbers at:

BENGALI: 353997 out of 354285 tests passed. 288 failed (0.0812905%)
DEVANAGARI: 707339 out of 707394 tests passed. 55 failed (0.00777502%)
GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%)
GURMUKHI: 60769 out of 60809 tests passed. 40 failed (0.0657797%)
KANNADA: 951086 out of 951913 tests passed. 827 failed (0.0868777%)
KHMER: 299106 out of 299124 tests passed. 18 failed (0.00601757%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048104 out of 1048416 tests passed. 312 failed (0.0297592%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271747 out of 271847 tests passed. 100 failed (0.0367854%)
TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%)
TELUGU: 970558 out of 970573 tests passed. 15 failed (0.00154548%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2012-09-05 15:22:02 -04:00
Behdad Esfahbod 87b75d0a4a [OT] Allow adding features with fallback implementation 2012-09-04 23:06:38 -04:00
Behdad Esfahbod 1d3947a6bd Minor 2012-09-04 22:42:17 -04:00
Behdad Esfahbod b3b89b6658 [OT] Add SubstLookup serialize API 2012-09-04 21:28:33 -04:00
Behdad Esfahbod 715e03bc21 Minor 2012-09-04 20:10:17 -04:00
Behdad Esfahbod 652d1e0d64 [OT] Start adding Lookup-level serialize API 2012-09-04 20:00:44 -04:00
Behdad Esfahbod a930c68e9c [OT] More serialize. Implements all basic GSUB subtables 2012-09-04 19:16:09 -04:00
Behdad Esfahbod 1b38b4e817 Minor 2012-09-04 18:17:21 -04:00
Behdad Esfahbod f7e81ce0b8 0.9.4 2012-09-04 15:32:37 -04:00
Behdad Esfahbod 2bd9fe3598 Refactor 2012-09-04 15:15:19 -04:00
Behdad Esfahbod a5ddd9e31c [OT] Really fix possible NULL dereference this time 2012-09-04 14:55:00 -04:00
Behdad Esfahbod 2941683358 [OT] Implement serialize() for AlternateSubst 2012-09-03 23:31:14 -04:00
Behdad Esfahbod 1f07e3382a [OT] Implement serialize() for MultiSubst 2012-09-03 23:28:34 -04:00
Behdad Esfahbod 4912030dfb Minor 2012-09-03 21:00:48 -04:00
Behdad Esfahbod f8fa2b5cf6 Fix possible NULL dereference
As reported by Kenichi Ishibashi.
2012-09-03 20:19:46 -04:00
Behdad Esfahbod 4b312fb288 [OT] Remove serialize alignment
Will reintroduce in a different way when we actually need it.
2012-09-01 21:56:06 -04:00
Behdad Esfahbod c61be03d6d [OT] A bit more serialize 2012-09-01 21:49:44 -04:00
Behdad Esfahbod abcc5ac1fd [OT] Improve serialize syntax
For some definition of improvement...
2012-09-01 21:30:17 -04:00
Behdad Esfahbod bc5be24014 [OT] Restart work on serialize() 2012-09-01 21:25:20 -04:00
Behdad Esfahbod 6912e476dd [OT] Insert dotted-circle for run-initial marks
Unfortunately if the font has GPOS and 'mark' feature does
not position mark on dotted-circle, our inserted dotted-circle
will not get the mark repositioned to itself.  Uniscribe cheats
here.

If there is no GPOS however, the fallback positioning kicks in
and sorts this out.

I'm not willing to address the first case.
2012-09-01 20:38:45 -04:00
Behdad Esfahbod 1d581ec384 [OT] Fallback-position ccc=0 Thai / Lao marks
Not perfect, but so is fallback positioning in 2012...
2012-09-01 20:06:26 -04:00
Behdad Esfahbod 3992b5ec4c Move code around 2012-09-01 19:20:41 -04:00
Behdad Esfahbod b85800f9de [Indic] Implement dotted-circle insertion for broken clusters
No panic, we reeally insert dotted circle when it's absolutely broken.

Fixes most of the dotted-circle cases against Uniscribe. (for Devanagari
fixes 80% of them, for Khmer 70%; the rest look like Uniscribe being
really bogus...)

I had to make a decision.  Apparently Uniscribe adds one dotted circle
to each broken character.  I tried that, but that goes wrong easily with
split matras.  So I made it add only one dotted circle to an entire
broken syllable tail.  As in: "if there was a dotted circle here, this
would have formed a correct cluster."  That works better for split
stuff, and I like it more.
2012-08-31 19:18:20 -04:00
Behdad Esfahbod 327d14ef18 [Indic] Start adding dotted-circle instrastructure 2012-08-31 16:49:34 -04:00
Behdad Esfahbod 1be368e96f Minor 2012-08-31 16:29:17 -04:00
Behdad Esfahbod 784f29d061 Minor 2012-08-31 14:06:26 -04:00
Behdad Esfahbod 5a7f18767a [OT] Better fallback-position Thai / Lao ccc!=0 marks 2012-08-30 22:53:29 -04:00
Behdad Esfahbod 9f2348de58 [OT] Add serialize() for Coverage 2012-08-29 21:08:59 -04:00
Behdad Esfahbod e901b954c6 [OT] Start adding serialize() API 2012-08-29 20:26:08 -04:00
Behdad Esfahbod 965c280de0 Add HB_BUFFER_ASSERT_VAR
To be used in places we access buffer vars...
2012-08-29 14:02:37 -04:00
Behdad Esfahbod 0ccf9b6473 Move code around 2012-08-29 14:02:37 -04:00
Behdad Esfahbod 2fcbbdb41a Port Arabic fallback ligating to share code with GSUB
This will eventually allow us to skip marks, as well as (fallback)
attach marks to ligature components of fallback-shaped Arabic.
That would be pretty cool.  I kludged GDEF props in, so mark-skipping
works, but the produced ligature id/components will be cleared later
by substitute_start() et al.

Perhaps using a synthetic table for Arabic fallback shaping was a better
idea.  The current approach has way too many layering violations...
2012-08-29 14:01:22 -04:00
Behdad Esfahbod 5e399a8a45 Minor 2012-08-29 10:40:49 -04:00
Behdad Esfahbod a177d027d1 [GSUB] Move ligation logic over 2012-08-28 23:18:22 -04:00
Behdad Esfahbod 191fa885d9 [GSUB] Merge Ligature and context input matching
Looks better now...
2012-08-28 22:58:55 -04:00
Behdad Esfahbod 93814ca7dc Start converging Ligature and match_input 2012-08-28 22:39:10 -04:00
Behdad Esfahbod 2eef71737e [hb-icu-le] Add visbility 2012-08-28 19:16:38 -04:00
Behdad Esfahbod d59e28e492 Minor 2012-08-28 19:08:36 -04:00
Behdad Esfahbod af169d2813 Minor 2012-08-28 19:08:22 -04:00
Behdad Esfahbod 52ff2681d8 Use VisualStudio-style atomic intrinsics on mingw32 2012-08-28 18:03:35 -04:00
Behdad Esfahbod 7c8e844d92 Use namespace for OpenType tables
Avoids USHORT, SHORT, ULONG, LONG clashes with Windows API.
2012-08-28 17:57:49 -04:00
Behdad Esfahbod dc5df5af6b Revert "Minor"
This reverts commit 3e0a03978b.

I know remember why that line is there :).
2012-08-28 16:31:23 -04:00
Behdad Esfahbod 3e0a03978b Minor 2012-08-27 17:10:02 -04:00
Behdad Esfahbod 667218a5b1 Minor 2012-08-27 17:00:44 -04:00
Behdad Esfahbod 30dd62251f Only fallback-position glyphs if we have the ccc
Previously, ccc=0 Thai / Lao marks were being
mispositioned.  Don't touch them.
2012-08-27 16:54:34 -04:00
Behdad Esfahbod e1ba62811a Center unknown marks horizontally 2012-08-27 16:28:05 -04:00
Behdad Esfahbod a4e75e4128 Minor 2012-08-27 15:54:15 -04:00
Behdad Esfahbod 23b0e9d7dc [Indic] Fix switch
D'oh.  Was working by pure chance :)).
2012-08-26 14:30:38 -04:00