4a7f4f3e56
[Thai] Adjust SARA AM reordering to match Uniscribe
...
Adjust the list of marks before SARA AM that get the reordering
treatment. Also adjust cluster formation to match Uniscribe.
With Wikipedia test data, now I see:
- For Thai, with the Angsana New font from Win7, I see 54 failures out
of over 4M tests (0.00129107%). Of the 54, two are legitimate
reordering issues (fix coming soon), and the other 52 are simply
Uniscribe using a zero-width space char instead of an unknown
character for missing glyphs. No idea why. The missing-glyph
sequences include one that is a Thai character followed by an Arabic
Sokun. Someone confused it with Nikhahit I assume!
- For Lao, with the Dokchampa font from Win7, 33 tests fail out of
54k (0.0615167%). All seem to be insignificant mark positioning
with two marks on a base. Have to investigate.
2012-07-23 13:15:33 -04:00
60554f14d8
[Indic] Merge in Malayalam tests
...
From:
http://silpa.org.in/pub/tests/hb/ml/ml-harfbuzz-testdata.txt
2012-07-22 23:23:56 -04:00
5c7081770c
[Indic] Add extensive Sinhala tests
...
Generated by:
http://git.savannah.gnu.org/cgit/sinhala.git/plain/utils/gen-unicode-sinhala.py
2012-07-22 23:20:27 -04:00
2efe4707b1
[Indic] Add Sinhala tests
...
Merge tests from:
http://git.savannah.gnu.org/cgit/sinhala.git/plain/patches/icu-sinhala-rendering.txt
2012-07-22 23:17:59 -04:00
3d4c111b7a
Add a test case
2012-07-20 19:34:39 -04:00
bdd080431a
[Indic] Reposition Oriya Candrabindu
...
Oriya failures down from 0.65% to 0.20%.
2012-07-20 16:03:09 -04:00
87cd63266e
[Indic] Recategorize some Kannada right matras
...
Kannada failures down from 3.5% to 2.93%.
2012-07-19 21:25:46 -04:00
c87bcddb10
[Indic] Add failing test for Kannada
2012-07-19 20:03:25 -04:00
deeb540a74
[test] Ignore tests with DOTTED CIRCLE in the output
2012-07-19 11:30:48 -04:00
422ecd2d3c
[Indic] Accept a forced Rakar sequence at the end of syllable
...
In Sinhala, Rakar is formed by Al-Lakuna,ZWJ,Ra. If you put that at the
end of a Consonant,Matra syllable, you get a dotted-circle from
Uniscribe. Apparently adding a ZWJ before the Al-Lakuna "fixes" that.
And people have been encoding that sequence... So, allow a forced
"ZWJ,Virama,ZWJ,Ra" sequence at the of syllables.
Fixes some 100 or more of Sinhala failures. Now at 622 only (0.23%).
2012-07-18 23:25:58 -04:00
10cdc94eee
[Indic] In final reordering, find base, even if it disappeared
...
POS_BASE can disappear if base ligated backward. Define base as last
with position not after base.
Fixes a few hundred of Sinhala failures with Iskoola Pota.
2012-07-18 17:43:23 -04:00
3285e107c9
[Indic] Implement Sinhala "Al Lakuna" Reph behavior
...
In Sinhala, Reph is formed only explicitly, by the presence of a ZWJ.
2012-07-18 17:22:14 -04:00
552d19b7a1
[Indic] Treat Register Shifters like Nukta
...
Really this time.
Fixes another 18 Khmer tests.
2012-07-18 16:02:33 -04:00
69f26bf39c
[Indic] Fix Matra reordering when base is at end of syllable
...
For example: U+915,U+200c,U+93f
Fixes last Tamil failure!
2012-07-18 15:47:51 -04:00
391cc03317
[Indic] Allow halant group in Vowel and placeholder syllables
...
Fixes 2 out of 560 Devanagari failures. AND:
Fixes 1 out of 2 Tamil failures.
2012-07-18 15:12:49 -04:00
418d00dffd
[Indic] Minor
2012-07-18 14:57:28 -04:00
25bc489498
[Indic] Better categorize Register Shifters and Khmer Various signs
...
Down another 500 or so Khmer failures!
2012-07-17 17:53:03 -04:00
34b5714906
[Indic] Treat Khmer Register Shifters more like Nuktas
...
Except that there may be a ZWNJ before a Register Shifter.
2012-07-17 14:09:32 -04:00
0201e0a464
[Indic] Apply 'cfar' for Khmer
...
Mark stuff after a pre-base reordering Ro 'cfar'. Used in Khmer.
This allows distinguishing the following cases with MS Khmer fonts:
U+1784,U+17D2,U+179A,U+17D2,U+1782
U+1784,U+17D2,U+1782,U+17D2,U+179A
2012-07-17 13:56:24 -04:00
55f70ebfb9
[Indic] Position final subjoined consonants (and vowels) after matras
...
In Khmer, a final subjoined consonant or independent vowel can occur
after matras. This final subjoined thing should NOT be reordered to
before the matra even though it's subjoined.
Fixes another 1k of the Khmer failures. Not much left really.
2012-07-17 12:50:13 -04:00
c50ed71e9a
[Indic] Recategorize Khmer coeng sign as a separate category OT_Coeng
...
Amend the syllable structure to allow a final subscripted consonant
(Coeng+C) and a final subscripted independent vowel (Coeng+V).
Fixes another 2k of Khmer failures.
2012-07-17 11:54:28 -04:00
74ccc6a132
[Indic] Move Halant with after-base consonants
...
Normally, we attach the Halant to the previous character and move it
with it. For after-base consonants however, the Halant "belongs" to the
consonant after, so attach it so.
This fixes Bengali sequences involving post-base consonant Ya, which
should ligate with the Halant to form Ya Phala, but previously a
reordered matras was blocking the ligation.
2012-07-17 11:16:19 -04:00
d5c4edcdd6
[Indic] Apply presentation-forms features all at once
...
Seems like this is what Uniscribe is doing, and does not break any fonts
we tested (with Devanagari, Malayalam, Khmer, and Bengali), while fixing
some Ra Phala sequences for Bengali with Vrinda. Fixes another 2% of
Bengali failures (a couple more to go).
2012-07-17 10:40:59 -04:00
6de103547e
[test/arabic] Add Arabic tests for mark skipping
...
Expose a bug with Khaled's Hussaini Nastaleeq font.
2012-07-16 22:46:52 -04:00
1167c7bfc9
Minor
2012-07-11 18:00:28 -04:00
aa116582e6
Minor
2012-07-11 18:00:28 -04:00
b0a6e58bb3
s/script-punjabi/script-gurmukhi/
2012-06-04 10:21:22 -04:00
4efdffec09
Minor Malayalam test case
...
From https://bugs.freedesktop.org/show_bug.cgi?id=45166
2012-05-28 10:45:50 -04:00
dfff5b3021
Add Myanmar test case
2012-05-28 10:45:50 -04:00
ff3524c21a
Add Arabic diacritics tests
2012-05-23 21:50:43 -04:00
a6de53664d
Add CJK Compatibility Ideographs tests
...
From:
http://people.mozilla.org/~jdaggett/tests/cjkcompat.html
2012-05-18 15:04:35 -04:00
f538fcb538
[test] Make tool usage easier by not requiring "--stdin"
...
Just default to it. Added "--help" instead to get usage.
2012-05-12 15:34:40 +02:00
a3273e30bb
[Indic] Add more Malayalam tests
2012-05-12 13:34:18 +02:00
5b16de97bc
[Indic] Add tests for dottedcircle
2012-05-11 19:55:42 +02:00
c071b99f15
[Indic] Add test for Left Matra with Halant
...
Uniscribe doesn't move the Halant, we do. And do a broken job of it now.
2012-05-11 16:22:46 +02:00
b20c9ebaf5
[Indic] Add test for matra group
...
The spec says: "[{M}+[N]+[H]]", and that's what Uniscribe implements.
We instead do: "{M+[N]+[H]}", which means we allow Nukta and Halant
after all Matras, not just the last one. It makes more sense.
2012-05-10 18:31:17 +02:00
61a58e26a5
[Indic] Add tricky reordering test cases
...
In the case of Consonant,LeftMatra,Halant, Uniscribe leaves the Halant
where it is, but we want to move it with the Matra as that makes more
logical sense.
2012-05-10 14:43:53 +02:00
3943293a99
[Indic] Add joiner test cases for Devanagari
2012-05-09 15:27:56 +02:00
2214a03900
Add hb-diff-ngrams
2012-05-09 09:54:54 +02:00
178e6dce01
Add N-gram generator
2012-05-09 08:57:29 +02:00
98669ceb77
Use groupby()
2012-05-09 08:16:15 +02:00
c438a14b62
Add hb-diff-stat
2012-05-09 07:45:17 +02:00
1058d031e2
Make hb-diff-filter-failtures retain all test info for failed tests
2012-05-09 07:35:28 +02:00
f1eb008cc7
Add hb-diff-colorize
...
Accepts --format=html now.
2012-05-09 00:01:50 +02:00
9155e4ffe0
Cleanup diff
...
Doesn't do --color anymore. That will go into a new hb-diff-colorize
tool.
2012-05-08 22:44:21 +02:00
7d22135b4c
Make hb-diff faster
2012-05-08 19:38:49 +02:00
a93e238e05
More tests
2012-05-08 18:55:29 +02:00
585b107cde
Add test caes for a minority language using Bengali
...
U+0985 BENGALI LETTER A followed by U+09D7 BENGALI AU LENGTH MARK.
According to Bobby de Vos on the mailing list, this results in a dotted
circle with most shaping engines, but is a legitimate sequence in this
minority language.
We reached the consensus on the list to NOT implement dotted-circle
in HarfBuzz.
2012-04-24 16:00:50 -04:00
0290bbf861
Add another Thai test
2012-04-17 10:28:21 -04:00
4d85252bda
Add Japanese test data from Adobe's Kazuraki font ligatures
2012-04-16 15:54:26 -04:00
f9746b600a
Minor
2012-04-12 09:59:26 -04:00
7470b0ff80
Add Mongolian test case
2012-04-12 09:44:27 -04:00
a4976447cd
Add Hangul test
2012-04-11 17:48:40 -04:00
e95d912b3b
Fix diff tool
2012-04-11 17:33:02 -04:00
e099dd6592
Add Thai test case for SARA AM decomposition
2012-04-10 10:47:33 -04:00
4450dc9354
Move around
2012-04-07 22:07:23 -04:00
aaa25d5f45
Add Hangul test case
...
Composed, and decomposed, of the same text.
2012-04-05 17:27:23 -04:00
406044986a
Add Hebrew diacritics test cases
...
From:
https://bugzilla.mozilla.org/show_bug.cgi?id=662055
2012-03-06 20:24:31 -05:00
7a70ca78e0
Add test case from https://bugzilla.mozilla.org/show_bug.cgi?id=714067
2012-02-21 11:31:47 -05:00
1a5a91dc0d
Add a few more tests
2012-01-22 19:58:23 -05:00
1795f3a222
Add a couple Thai test cases from Thep
2012-01-22 19:29:45 -05:00
ec3f506682
Add Devanagari test from Tom Hacohen
2012-01-22 19:10:55 -05:00
71be4ca3dd
Also ignore "ChangeLog" in manifests
2012-01-22 16:26:49 -05:00
3c9a39ecd6
Remove newline
2012-01-22 16:21:19 -05:00
e4ccbfe276
Allow --color=html in hb-diff
...
Not that useful right now as we don't escape < and >. Perhaps
another tool can be added to convert the ANSI output to HTML.
2012-01-22 16:07:32 -05:00
8f80f93491
More shoveling around
2012-01-21 20:03:25 -05:00
c78c6e9844
Cleanup
2012-01-21 19:55:16 -05:00
ab94a9c542
Distribute testing tools
2012-01-21 19:43:58 -05:00
3e86feb54c
Speed up colorless diff
2012-01-21 19:40:30 -05:00
1e58df6034
Cleanup manifest code
2012-01-21 19:37:31 -05:00
956d552e10
Port hb-manifest-update to Python
2012-01-21 19:31:51 -05:00
3a34e9e351
Ignore Broken Pipe errors
2012-01-21 19:15:41 -05:00
f22089ac24
Misc fixes
2012-01-20 21:22:14 -05:00
96968bfae5
Port hb-manifest-read to Python
2012-01-20 21:16:34 -05:00
a59ed46fa4
Add final residues from test-shape-complex
2012-01-20 20:56:32 -05:00
820e0ed318
Add Punjabi tests from test-shape-complex also
2012-01-20 20:51:52 -05:00
a7d71c1057
Add Tamil test data from Muguntharaj Subramanian
2012-01-20 20:50:09 -05:00
5992a9941e
Import test data from late test-shape-complex
2012-01-20 20:48:14 -05:00
46ac456477
Fix Unicode encoding issue
2012-01-20 19:32:17 -05:00
ad34e39a4a
Make test tools interactive
...
By bypassing readlines() buffering.
2012-01-20 18:40:25 -05:00
91540a7d97
Move most testing logic into hb_test_tools.py
...
The actual utils are one-liners now.
2012-01-20 18:28:10 -05:00
66aa080033
Remove test-shape-complex
...
New shaping testsuite and framework coming.
2012-01-20 17:36:10 -05:00
ed459bfb63
Add hb-unicode-encode
2012-01-20 17:24:05 -05:00
b12c4d4361
Add hb-diff-filter-failures
2012-01-20 17:17:44 -05:00
d4bffbc55b
Move
2012-01-20 17:16:35 -05:00
45f640c98d
Minor
2012-01-20 14:24:21 -05:00
47ca766a9c
Minor
2012-01-20 14:21:53 -05:00
8f1db07894
[test/shaping] Add some Indic test data for the new test suite
...
Imported from UTRRS.
2012-01-20 14:00:44 -05:00
11267aef36
Fix
2012-01-20 13:57:14 -05:00
4e84ce48d5
Move hb-diff to test/shaping/
2012-01-20 13:51:22 -05:00
f868e1b84d
Add hb-unicode-decode
2012-01-20 13:50:05 -05:00
9ab23ef474
Minor
2012-01-20 13:49:56 -05:00
c8d81db033
Recognize more characters
2012-01-20 13:39:27 -05:00
0016d4662d
[test] Make hb-unicode-prettyname take a --stdin option
2012-01-20 13:31:59 -05:00
ad8c6446f2
[test/shaping] Add hb-unicode-prettyname
2012-01-20 13:27:40 -05:00
e900869b0f
[test/shaping] Add hb-read-manifest
2012-01-19 20:28:15 -05:00
a211cd3ffc
Ignore AUTHORS also
2012-01-19 20:27:53 -05:00
a33e46cf7d
[test/shaping] Add hb-update-manifests
2012-01-19 15:44:55 -05:00
d4de562adf
Start adding new shaping test suite together
2012-01-19 15:21:04 -05:00
Behdad Esfahbod