34c215036f
[Indic] Improve Sinhala base algorithm and reph positioning
...
Sinhala does not have half forms. And most (all?) consonants can be
base, except when preceded by ZWJ, which would request a subjoined form.
Hence switch the base algorithm to categorize with Khmer, start search
at start, and stop at a ZWJ.
Also, mark all pos=base consonants after base to be subjoined. Mark
base itself to have pos=base.
Finally, adjust Sinhala's reph position to after-main.
Brings down Sinhala failures from 455 to 328 (0.120656%).
2012-07-23 23:51:29 -04:00
49c5ec5144
Minor refactoring
2012-07-23 20:14:13 -04:00
c3e6fdc379
[Indic] Improve check on ligatures
...
Only skip actual ligatures, not marks in-between ligature components.
2012-07-23 20:11:42 -04:00
771a8f5028
[Indic] exclude ligatures when matching on Indic category
...
If, say, a H,ZWJ,C ligature was formed, we don't want the code to detec
that as a Halant. So, ignore ligatures when matching category in
final_reordering.
Sinhala failures down from 514 to 455 (0.167374%).
2012-07-23 20:09:30 -04:00
baacd090df
[Indic] Minor refactoring
2012-07-23 19:51:48 -04:00
c7c4de2fb9
[Indic] Remove syllable length check before sorting
...
We now limit syllable lengths in the machine. No need to match here.
2012-07-23 18:25:02 -04:00
2cc933aff9
[Indic] Fix cluster formation with left-matras and conjunct forms
...
Test case was: <U+0D15,U+0D4D,U+0D15,U+0D4A>.
2012-07-23 08:23:44 -04:00
e6b01a878c
[Indic] Further streamline cluster formation
...
This should address all possible cluster misformations that I had in
mind.
2012-07-23 00:11:26 -04:00
7b2a7dadd6
[Indic] Merge clusters before sorting
...
This should fix any instabilities in cluster formation that we were
speculating may happen with surrounding syllables. Or most of it
perhaps.
2012-07-22 23:58:55 -04:00
abb3239ef9
[Indic] Update clusters for left-matra even if matra didn't move
...
Fixes crashes reported with left matra under
non-uniscribe-bug-compatibilty mode.
2012-07-22 23:55:19 -04:00
92a1ad7bef
[Indic] Stop searching for base if a post form is found before below form
...
Improves Bengali and Gurmukhi. Malayalam regressed a bit. We will deal
with that later.
2012-07-20 18:55:15 -04:00
4c450c703f
[Indic] Recompose Bengali Ya,Nukta
...
This is a bunch of hacks for now.
Improves Bengali a bit.
2012-07-20 18:13:04 -04:00
34ae336f3f
[Indic] Improve Reph AfterMain positioning
...
Fixes 20 out of 48 failing Oriya tests. Failure rate down to 0.066% now.
2012-07-20 16:17:28 -04:00
bdd080431a
[Indic] Reposition Oriya Candrabindu
...
Oriya failures down from 0.65% to 0.20%.
2012-07-20 16:03:09 -04:00
5f0eaaad12
[Indic] Fix base search in final_reordering
...
Fixes most Malayalam failures. Down from 1.6% to 0.38% now. Fixes a
few more in other scripts too.
2012-07-20 15:47:24 -04:00
81202bd860
[Indic] Don't attach SM/VD to other characters
2012-07-20 15:14:51 -04:00
f31d97e44e
[Indic] Form Telugu Reph out of Ra,Virama,ZWJ
...
Apparently this was approved in Feb 2012. No font yet.
2012-07-20 14:13:35 -04:00
30c3d5e9fc
[Indic] Simplify Uniscribe cluster emulation
...
Now that we break syllables on Halant,ZWNJ, this code can be simplified.
2012-07-20 13:56:32 -04:00
decf6ffca4
[Indic] Minor!
2012-07-20 13:51:31 -04:00
9e4f94a72c
[Indic] Break syllables at Halant,ZWNJ
...
That's really what Uniscribe does, and explains a lot of pecularities of
Halant,ZWNJ before the base.
Sent Telugu from 1% failures to 0.03%. Improved Kannada and Malayalam
slightly. Fixed half of Bengali, and did NOT break anything!
2012-07-20 13:48:03 -04:00
2c372b80f6
[Indic] Better check for applying 'init'
...
Specifically, don't apply 'init' if previous char is a joiner.
Fixes some more of Bengali.
2012-07-20 13:37:48 -04:00
8ed248de77
[Indic] Minor
2012-07-20 11:42:24 -04:00
d0e68dbd0b
[Indic] Implement reph positioning step 5
...
Not tuned, just copied from step 2. Fixes another 0.5% of Kannada
failures. 1% to go.
2012-07-20 11:25:41 -04:00
a9e45c32e4
[Indic] Don't let ZWNJ at the end of syllable affect base search
...
Fixes a few Devanagari, half of remaining Kannada failures, quarter for
Telugu, and others slightly improved or unchanged.
2012-07-20 11:04:15 -04:00
20b68e699f
[Indic] Apply 'cjct' globally
...
Fixes 5 Devanagari failures, and no regressions.
2012-07-20 10:47:46 -04:00
51e764de44
[Indic] Unbreak old scriptures
...
Brings down failures with Lohit-Telugu from 57% to 1.40%.
2012-07-20 10:30:24 -04:00
900cf3d449
Minor
2012-07-20 10:18:23 -04:00
87cd63266e
[Indic] Recategorize some Kannada right matras
...
Kannada failures down from 3.5% to 2.93%.
2012-07-19 21:25:46 -04:00
3604d64ced
[Indic] Recategorize GURMUKHI ADDAK
...
It's not in IndicSyllabicCategory.txt. Fixes most of Gurmukhi failures.
Failures down from 7.7% to 0.222%!
2012-07-19 21:13:04 -04:00
5249f3aee1
[Indic] Unbreak Khmer
...
For Khmer, all consonants are subjoining. No need to look in the font.
We were looking in the wrong order anyway.
2012-07-19 20:30:22 -04:00
e0475345d5
[Indic] Apply 'akhn' globally
...
Fixes 1.5% more failures for Telugu, 2% for Kannada.
Breaks one test in Devanagari.
2012-07-19 20:24:14 -04:00
fa247ebe52
[Indic] Better position U+0CD5
...
Fixes another 5% of Kannada failures.
2012-07-19 19:52:19 -04:00
f055442716
[Indic] Lookup consonant position in the font
...
Fixes most failures of Oriya, and improves others a bit.
2012-07-19 16:20:21 -04:00
8c973ebf0f
[Indic] Implement per-script matra positioning
...
Following what the spec says.
Brings down Telugu failures from 40% to 3.75%, and Kannada failures from
44% to 10%. Does NOT affect other scripts' test results.
2012-07-19 13:25:08 -04:00
8bb32458f9
[Indic] More refactoring
2012-07-19 13:04:44 -04:00
9ccc6382ba
[Indic] Minor refactoring
2012-07-19 12:45:31 -04:00
be8b9f5f71
[Indic] Start refactoring different matra positions per script
2012-07-19 12:11:12 -04:00
10cdc94eee
[Indic] In final reordering, find base, even if it disappeared
...
POS_BASE can disappear if base ligated backward. Define base as last
with position not after base.
Fixes a few hundred of Sinhala failures with Iskoola Pota.
2012-07-18 17:43:23 -04:00
9c4d24a3a6
[Indic] Minor
2012-07-18 17:29:10 -04:00
3285e107c9
[Indic] Implement Sinhala "Al Lakuna" Reph behavior
...
In Sinhala, Reph is formed only explicitly, by the presence of a ZWJ.
2012-07-18 17:22:14 -04:00
552d19b7a1
[Indic] Treat Register Shifters like Nukta
...
Really this time.
Fixes another 18 Khmer tests.
2012-07-18 16:02:33 -04:00
e8cd81f76d
[Indic] Minor
2012-07-18 16:00:20 -04:00
69f26bf39c
[Indic] Fix Matra reordering when base is at end of syllable
...
For example: U+915,U+200c,U+93f
Fixes last Tamil failure!
2012-07-18 15:47:51 -04:00
075d671f10
[Indic] Fix out-of-bounds array access
2012-07-18 15:41:53 -04:00
14dbdd9e39
[Indic] Unbreak Tamil
...
Tamil has only about 150 failures now!
2012-07-18 13:13:03 -04:00
db8981f1e0
[Indic] Position Khmer Robat
...
It's a visual Repha.
Still not positioning logical Repha as occurs in Malayalam.
Another 200 Khmer failures fixed. 547 to go. That's better than
Devanagari!
2012-07-17 23:42:04 -04:00
25bc489498
[Indic] Better categorize Register Shifters and Khmer Various signs
...
Down another 500 or so Khmer failures!
2012-07-17 17:53:03 -04:00
25e302da9a
[Indic] Minor
2012-07-17 14:25:14 -04:00
5d32690a34
[Indic] For scripts without Half forms, always choose first consonant as base
...
In such scripts (ie. Khmer), a ZWJ/ZWNJ shouldn't stop the search for
base. So, instead just choose the first consonant as base directly.
Test sequence:
U+1798,200c,U+17C9,U+17D2,U+179B,U+17C1,U+17C7
2012-07-17 14:23:28 -04:00
0201e0a464
[Indic] Apply 'cfar' for Khmer
...
Mark stuff after a pre-base reordering Ro 'cfar'. Used in Khmer.
This allows distinguishing the following cases with MS Khmer fonts:
U+1784,U+17D2,U+179A,U+17D2,U+1782
U+1784,U+17D2,U+1782,U+17D2,U+179A
2012-07-17 13:56:24 -04:00
55f70ebfb9
[Indic] Position final subjoined consonants (and vowels) after matras
...
In Khmer, a final subjoined consonant or independent vowel can occur
after matras. This final subjoined thing should NOT be reordered to
before the matra even though it's subjoined.
Fixes another 1k of the Khmer failures. Not much left really.
2012-07-17 12:50:13 -04:00
c50ed71e9a
[Indic] Recategorize Khmer coeng sign as a separate category OT_Coeng
...
Amend the syllable structure to allow a final subscripted consonant
(Coeng+C) and a final subscripted independent vowel (Coeng+V).
Fixes another 2k of Khmer failures.
2012-07-17 11:54:28 -04:00
deb521dee4
[Indic] Add a separate Coeng class
...
No characters recategorized yet. No semantic change.
2012-07-17 11:37:32 -04:00
74ccc6a132
[Indic] Move Halant with after-base consonants
...
Normally, we attach the Halant to the previous character and move it
with it. For after-base consonants however, the Halant "belongs" to the
consonant after, so attach it so.
This fixes Bengali sequences involving post-base consonant Ya, which
should ligate with the Halant to form Ya Phala, but previously a
reordered matras was blocking the ligation.
2012-07-17 11:16:19 -04:00
d5c4edcdd6
[Indic] Apply presentation-forms features all at once
...
Seems like this is what Uniscribe is doing, and does not break any fonts
we tested (with Devanagari, Malayalam, Khmer, and Bengali), while fixing
some Ra Phala sequences for Bengali with Vrinda. Fixes another 2% of
Bengali failures (a couple more to go).
2012-07-17 10:40:59 -04:00
af92b4cc90
[Indic] Disable 'kern' in Uniscribe bug compatibility mode
...
Uniscribe does not apply 'kern' in the Indic module. Some of the Khmer
fonts they ship have small adjustments in the 'kern' table. Disable
'kern' in the Indic module under Uniscribe bug compatibility mode.
Fixes some 10% of the Khmer failures. Remains under 3% (excluding
dotted-circle ones).
2012-07-16 20:31:24 -04:00
d96838ef95
Allow complex shapers overriding common features
...
In a new callback... Currently unused by all complex shapers.
2012-07-16 20:26:57 -04:00
df50b84740
[Indic] Categorize other Khmer marks
...
Mark them the same as the Register Shifters for now. Need to rename
that category to something more sensible after all is settled.
Fixes another percent of Khmer failures. Down to under 3%!
2012-07-16 20:14:50 -04:00
8e7b5882fb
[Indic] Recognize pre-base reordering Ra anywhere in the syllable
...
We were doing that only immediately after base.
Fixes another percent in the Khmer failures. About three more to go...
2012-07-16 17:04:46 -04:00
7d09c98a1f
[Indic] Recognizer Register Shifter marks
...
Fixes another 6% of the Khmer failures.
2012-07-16 16:45:22 -04:00
78818124b1
[Indic] Reoder pre-base reordering Ra
...
Brings down Malayalam failures from 14% down to 3%.
2012-07-16 15:49:08 -04:00
1a1dbe9a27
[Indic] Rename
2012-07-16 15:41:33 -04:00
46e645ec4b
[Indic] Start implementing pre-base reordering
2012-07-16 15:30:05 -04:00
921ce5b17d
[Indic] Rename
...
No semantic change.
2012-07-16 15:26:56 -04:00
b504e060f0
[Indic] Implement After-Main Reph positioning
...
Almost...
2012-07-16 15:21:12 -04:00
17d7de91d7
[Indic] Apply 'pref' to pre-base reodering Ra
...
No reordering yet.
2012-07-16 15:20:15 -04:00
362d3db8d3
[Indic] Minor
...
Should not be any semantic change. In preparation for implementing
pre-base reordering Ra.
2012-07-16 15:15:28 -04:00
70fe77bb9a
Minor
2012-07-16 14:52:18 -04:00
2f903215c5
Minor
2012-07-16 13:54:43 -04:00
a3e04bee2c
[Indic] Reorder virama only for old Indic spec
2012-07-16 13:47:19 -04:00
0de771b72d
[Indic] Categorize Khmer consonants
2012-07-16 13:39:36 -04:00
29f106d7fb
[Indic] Apply Above Forms
2012-07-16 12:05:35 -04:00
a2b471df82
Remove static initializers from indic
2012-06-05 15:17:44 -04:00
9fc7a11469
Remove comma at the end of enum
...
As reported by Jonathan Kew on the list.
2012-06-04 08:28:19 -04:00
27aba594c9
Minor
2012-05-24 15:00:01 -04:00
1d6846db9e
[Indic] Apply vatu feature after cjct
...
Testing with old Deva spec this reduces failures.
Test sequence: U+0915,U+094D,U+0930.
2012-05-13 18:09:29 +02:00
617f4ac46f
Refactor
2012-05-13 16:48:03 +02:00
5e4e21fce4
Revert "[Indic] Refactoring"
...
This reverts commit 0831061efb
2012-05-13 16:46:08 +02:00
3f18236a03
Fix more warnings
2012-05-13 16:20:10 +02:00
9f377ed321
Fix more unused-var warnings
2012-05-13 16:13:44 +02:00
eace47b173
Minor
2012-05-13 15:54:43 +02:00
737dded2e0
Fix compiler warnings
2012-05-12 15:40:11 +02:00
7f852b644b
Fix compiler warnings
2012-05-11 23:10:31 +02:00
6a091df9b4
[Indic] Disambiguate sub vs post vs above matras
...
Bengali is at *just* above 5% now.
2012-05-11 21:42:27 +02:00
9d0d319a4a
[Indic] Position Bengali Reph before matras
2012-05-11 21:36:32 +02:00
f893672511
[Indic] Start categorizing Reph per script
2012-05-11 21:10:03 +02:00
a913b024d8
[Indic] Apply 'init' feature for Bengali
...
Error down from 20% to 7%.
2012-05-11 20:59:26 +02:00
eed903b164
[Indic] Refactor for the arrival of 'init' feature
...
Yep, on Bengali now!
2012-05-11 20:50:53 +02:00
18c06e189b
[Indic] Add Uniscribe bug feature for dotted circle
...
For dotted-circle independent clusters, Uniscribe does no Reph shaping
for the exact sequence Ra+Halant+25CC. Which also is the only possible
sequence with 25CC at the end.
2012-05-11 20:02:14 +02:00
0831061efb
[Indic] Refactoring
2012-05-11 19:07:58 +02:00
7ea58db311
Minor
2012-05-11 18:58:57 +02:00
3399a06e70
[Indic] Fix U+0952 and similar classification to match Uniscribe
...
See comments.
2012-05-11 17:54:26 +02:00
11aa3ef18d
[Indic] Treat U+0951..U+0954 all similar to U+0952
2012-05-11 17:30:48 +02:00
892eb78782
[Indic] Implement Uniscribe Reph+Matra+Halant bug feature
2012-05-11 16:54:40 +02:00
67ea29af49
[Indic] Add example of different Uniscribe behavior
2012-05-11 16:51:23 +02:00
ebe29733d4
[Indic] Add runtime Uniscribe bug compatibility mode!
...
Enable by setting envvar:
HB_OT_INDIC_OPTIONS=uniscribe-bug-compatible
Plus, LeftMatra+Halant "feature".
2012-05-11 16:43:12 +02:00
616e692e29
[Indic] Add #define UNISCRIBE_BUG_COMPATIBLE 1
2012-05-11 16:25:02 +02:00
6782bdae3b
[Indic] Fix Left Matra + Halant reordering
...
As can be seen in: U+092B,U+093F,U+094D
2012-05-11 16:23:43 +02:00
3c2ea9481b
Minor
2012-05-11 16:23:38 +02:00
668c6046c1
[Indic] Apply Reph mask to all POS_REPH glyphs
...
Needed for upcoming changes to GSUB/GPOS mask matching.
2012-05-11 15:34:13 +02:00
Behdad Esfahbod