Commit Graph

1960 Commits

Author SHA1 Message Date
Simon Cozens fc5354c5e5 Add test for #2516 2022-01-19 16:04:58 +02:00
luz paz e2e305066a Fix various typos
Found via `codespell -q 3 -S ./perf/texts -L actualy,ba,beng,fo,gir,inout,nd,ot,pres,ro,te,teh,timne`
2022-01-16 05:39:03 -08:00
Garret Rieger 87496bf63e [subset] fix fuzzer timeout if visisted_paint goes into error. 2022-01-13 11:08:24 -08:00
Garret Rieger 067f90a820 [subset] Fix for fuzzer timeout.
Fixes https://oss-fuzz.com/testcase-detail/5549945449480192

In prune_langsys: move LangSys visited check up before any work is done for a LangSys. In this particular case the compare() method is responsible for the majority of the time spent and wasn't being guarded with a visisted check.
2021-12-14 17:34:21 -07:00
Garret Rieger c4573c2ec7 [repacker] don't infinite loop if visited or roots is in error.
Fixes https://oss-fuzz.com/testcase-detail/5205038086094848
2021-12-14 15:57:48 -07:00
Behdad Esfahbod 23159084b4 [morx] Reverse graphemese, not whole buffer
Fixes https://github.com/harfbuzz/harfbuzz/issues/3314
Supersedes https://github.com/harfbuzz/harfbuzz/pull/3315
2021-12-10 15:02:57 -07:00
Qunxin Liu 706014f69d [subset] (Chain)ContextSubst glyph_closure fix
- When pos_glyphs is empty, use current full glyphs set as input for
subsequent recursive closure process
- Also increase max_lookup_visit_count to 35000 cause a real font file hit
previous limit 20000 and some lookups are dropped unexpectedly
2021-12-10 11:08:36 -07:00
Qunxin Liu 51655a078e [subset] COLR : only include glyphs after COLR closure 2021-12-06 13:29:21 -07:00
Qunxin Liu 74b46b29e7 [subset] MATH: don't serialize coverage table when iterator is empty
when iterator is empty, just set coverage offset to 0.
serialize() in coverage will at lease write out a 16-bit format header.
2021-12-03 08:00:33 -07:00
Khaled Hosny 6c81cd9543 [tests] Add tests for platform shapers
Fixes https://github.com/harfbuzz/harfbuzz/issues/3115
2021-12-01 19:55:48 +02:00
Garret Rieger 9121ed0cec [subset] Improve sharing of Ligature subtables.
Ligature subtables use virtual links to enforce an ordering constraint between the subtables and the coverage table. Unfortunately this has the sideeffect of prevent the subtables from being shared by another Ligature with a different coverage table since object equality compares all links real and virtual. This change makes virtual links stored separately from real links and updates the equality check to only check real links. If an object is de-duped any virtual links it has are merged into the object that replaces it.
2021-12-01 08:37:04 -07:00
Garret Rieger 95329081c2 [subset] further optimize cmap4 packing. 2021-11-28 07:47:49 -07:00
Garret Rieger 599143824c [subset] Don't pad glyphs when using long loca. 2021-11-26 18:01:04 -07:00
Garret Rieger d9660fd58a [subset] Make cmap4 packing more optimal.
The current CMAP4 implementation uses whatever the current codepoint ranges are and then encodes them as indivudal glyph ids or as a delta if possible. However, it's often possible to save bytes by splitting up existing ranges and encoding parts of them using deltas where the cost of splitting the range is less than encoding each glyph individual.
2021-11-26 13:21:50 -07:00
Behdad Esfahbod 720ab0883b [util] Add --single-par
Use it in aots tests

Fixes https://github.com/harfbuzz/harfbuzz/issues/3129
Related https://github.com/harfbuzz/harfbuzz/issues/3298
2021-11-25 11:49:16 -07:00
Qunxin Liu 903a6baece [subset] layout_features filtering fix
we should not use get_size (), which returns length * item_size
2021-11-25 11:24:41 -07:00
Khaled Hosny 69d8f27c69 [meson] Require 0.55.0
We implicitly require it for building ragel subproject. This new version
requirement should satisfied in both Fedora 33 and Debian bullseye, and
not be too cutting edge for us.
2021-11-22 03:11:36 +02:00
Qunxin Liu 84dc4e85e8 [subset] avoid writing out duplicate extra glyph names in post table
Add check for possible duplicate with other name index
2021-11-19 17:22:35 -07:00
Qunxin Liu ca418cac74 [subset] keep features that have FeatureParams and the tag is "size" 2021-11-19 13:31:25 -07:00
Qunxin Liu e88fc41ef3 [subset] inputSequence could be empty, change the sanity check 2021-11-19 12:57:43 -07:00
Garret Rieger ace98cc65f [subset] Only sanitize recursion depth in COLR. 2021-11-10 10:34:46 -07:00
Garret Rieger f51b48c8e7 [subset] Fix fuzzer found memory leak.
Happens because an insert into a map with an invalid key reports successful, but this causes the set being inserted to be lost.
2021-11-02 17:00:07 -07:00
Qunxin Liu 540f19b6fe [subset] fix bug in (Chain)ContextFormat2
Only keep rulesets for glyphs class numbers that survived in coverage
2021-11-02 16:05:55 -07:00
Garret Rieger e260eeb9be [subset] Update test goldens for gdef.glyphset. 2021-11-02 16:56:52 -06:00
Qunxin Liu 60e203644b [subset] use glyphset_gsub instead of glyphset for GDEF 2021-11-02 16:56:52 -06:00
Garret Rieger e39647c0f1 [subset] update subset goldens to fix tests. 2021-11-02 16:26:02 -06:00
Qunxin Liu 0a7563a53f [subset] fuzzer fix: https://oss-fuzz.com/testcase?key=6254792024915968
Make sure input is valid, each gid has a corresponding offset value in
the map
2021-11-02 15:25:18 -07:00
Qunxin Liu 364b6b3989 [subset] Don't assume FeatureList is sorted
Though the spec said FeatureRecords are sorted alphabetically by feature
tag, there're font files with unsorted FeatureList. And harfbuzz is not
able to subset these files correctly because we use binary search in
finding featureRecords when collecting lookups. Also
find_duplicate_features needs to be updated to handle this.
2021-11-02 15:15:02 -07:00
Garret Rieger 49c9392412 [subset] During LigatureSubstFormat1 subsetting always place Coverage last.
In Windows 7 on Chrome if the coverage table comes before any of the LigatureSet or Ligature subtables the font won't load. This changes the packing order to always place the Coverage table last. Virtual links are used to ensure the repacker maintains the desired ordering.

Coincidentally fontTools also does the same thing (a3f988fbf6/Lib/fontTools/ttLib/tables/otTables.py (L1137)) to reduce overflows during packing.
2021-11-02 12:19:13 -06:00
Matthias Clasen 8dc86d0703 Add a test font scale rounding behavior
This test verifies the fix for
https://github.com/harfbuzz/harfbuzz/issues/3274
2021-11-01 17:59:48 -07:00
Qunxin Liu 85deddb16e [subset] fuzzer fix: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=40406 2021-10-27 16:16:15 -07:00
Behdad Esfahbod 4262636926
Merge pull request #3248 from googlefonts/connected_components
[repacker] Keep connected subgraphs in the same space.
2021-10-23 13:18:22 -07:00
Qunxin Liu 8c583db9b5 [subset] COLRv1: update subset() method for new strutc ClipList and VarIdxMap
Also fix issues in struct PaintTransform definition
2021-10-23 11:22:39 -07:00
David Corbett 8eaee2f887 [USE] Only skip default ignorables within CCSes 2021-10-08 10:15:08 -06:00
Garret Rieger 79937d24b7 [repacker] Add repacker test that requires space splitting. 2021-10-05 12:53:21 -07:00
Khaled Hosny bb57df7a4f [test] Add test for https://github.com/harfbuzz/harfbuzz/issues/2757#issuecomment-933531978 2021-10-05 16:04:10 +02:00
Khaled Hosny 1624550377 [test] Add test for https://github.com/harfbuzz/harfbuzz/issues/3244 2021-10-05 16:04:10 +02:00
Qunxin Liu 794b00db4b [subset] fuzzer fix: https://oss-fuzz.com/testcase-detail/6616166961905664 2021-09-29 10:29:06 -06:00
Garret Rieger d5beb96e06 [repacker] add repacker isolation test on real font. 2021-09-22 18:18:50 -06:00
Qunxin Liu ca7b9daef0 [subset] subset MATH table 2021-09-22 18:08:30 -04:00
Garret Rieger c2cc566c9d [subset] Fix subset_offset_array adding unused space to serializer.
ArrayOf.serialize_append allocates space for the new item, but ArrayOf.pop() does not recover the allocated space. So in the case where the revert path was entered the extra space added by serialize_append gets left in the serialization buffer. This moves the snapshot to before ArrayOf.serialize_append is called so that revert cleans up the buffer extend.
2021-09-22 16:07:18 -06:00
Qunxin Liu 8c5c81746d [subset] fix find_duplicate_features
We should check each feature against all other features with the same
tag for duplicates.
2021-09-22 15:35:38 -06:00
Garret Rieger 74f96d9d4b [repacker] fix heap use after free in repacker.
Don't store a reference to the link in overflow records as the link object may be freed if the sorted graph vector is resized.
2021-09-19 09:06:17 -06:00
Garret Rieger d700646c33 [subset] In preperation for 3.0.0 release remove legacy subsetting api methods. 2021-09-16 15:31:29 -06:00
David Corbett a0101a31b5 [Unicode 14] Add tests 2021-09-14 18:38:50 -04:00
Khaled Hosny 4d9b7de0b6 [style] Prepare for release 2021-09-13 10:10:10 -06:00
Behdad Esfahbod 201cbe3c66 [style] Rename HB_STYLE_TAG_SLANT to HB_STYLE_TAG_SLANT_ANGLE 2021-09-10 04:59:16 -06:00
Qunxin Liu 37379f8f7d [subset] fox for (Chain)ContextFormat3: subset lookupRecord
skip copying a lookupRecord if lookup referenced in the lookupRecord is
not retained after subset
2021-09-08 13:55:21 -06:00
Khaled Hosny 9fddbb3e24 [test] Rename test and split into smaller subtests 2021-09-03 18:24:40 +02:00
Khaled Hosny c6bb719e65 [test] Add tests for Arabic decomposition
Test that both NFC and NFD input produces identical results for fonts
that used composed fonts internally (Amiri here) and fonts that
decompose internally (Noto Nastaliq Urdu here) and that for the former
composed forms are used.

See https://github.com/harfbuzz/harfbuzz/issues/3179
2021-09-03 08:58:35 -06:00
Khaled Hosny a9dc4fbeac [test] Make shape test names match file names
No idea why test names are underscorified but it it just makes calling
meson test testname harder than it should being not able to copy file
name directly.
2021-09-03 08:56:34 -06:00
Garret Rieger 102666dab7 [subset] add test for hb_subset_input_set (...). 2021-08-27 10:36:42 -07:00
Behdad Esfahbod 99356ea5f0
Merge pull request #3150 from harfbuzz/item-context-tests
[test] Add tests for item context
2021-08-25 17:19:06 -06:00
Behdad Esfahbod 955f86a034 [test-set] Fix compiler warnings 2021-08-24 23:16:24 -06:00
Garret Rieger f2441a4b65 [subset] Remove retain all layout features flag.
Instead use inverted sets to handle requesting all features. Modifies feature collection in subset plan to intersect the set of requested features against the features in the font. This prevents iterating a fully filled feature tag set.
2021-08-24 18:17:07 -06:00
Garret Rieger f84daccb4f [set] include null pool sets in the permutations tested by inverted_operations. 2021-08-24 16:51:11 -06:00
Garret Rieger fb07f8f876 During subset input creation check for set alloc failures and fail if encountered. 2021-08-24 10:59:14 -06:00
Khaled Hosny 33bfe9edd6 [test] Add tests for item context
Test pre-context and post-context in Arabic shaper, as well as
interaction of pre-context with BOT flag.
2021-08-24 02:12:34 +02:00
Qunxin Liu 34e0b28fae [subset] make glyph-names option match fonttools behavior
Don't encode psNames that are contained in standard glyph names set
2021-08-20 01:30:23 -06:00
Behdad Esfahbod 829b0f3353
Merge pull request #3154 from harfbuzz/set-invert
Resurrect hb_set_invert()
2021-08-20 01:28:15 -06:00
Garret Rieger 915550ab19 [set] remove inverted test TODO. 2021-08-19 17:48:38 -07:00
Garret Rieger c4ed582997 [set] add test for inverted set operations.
This test checks all possible set configurations against each operation type.
2021-08-19 17:47:41 -07:00
Garret Rieger 325fd6ddb8 [set] add tests for inverted set equality. 2021-08-19 15:54:31 -07:00
Garret Rieger 287032af6c [set] update set test TODO list. 2021-08-19 15:45:28 -07:00
Garret Rieger 5c003d80a6 [set] add tests for inverted previous iteration. 2021-08-19 15:41:12 -07:00
Garret Rieger 3f2cc582f2 [set] add basic and iteration set inverion tests. 2021-08-19 15:05:10 -07:00
Garret Rieger dc31920bbe Don't serialize null offsets in CPAL.
Fixes https://oss-fuzz.com/testcase-detail/5443213648330752
2021-08-18 16:52:45 -06:00
Garret Rieger cb05c363d7 [subset] add some basic tests for the legacy subset methods. 2021-08-14 10:37:02 -06:00
Garret Rieger 44802c8ec4 [subset] use layout-features=* for layout subsetting tests w/ synthetic test fonts. 2021-08-12 14:57:18 -06:00
Garret Rieger c0f3af91b8 [subset] speed up add_gid_and_children and adjust op limit.
Fix for fuzzer timeout: https://oss-fuzz.com/testcase-detail/5001604901240832.

- Operation limit is per glyph, so 100,000 should still be far more than needed.
- Switches from for(...) to while(...) loop for iteration. for(...) calls it.end() which in this case triggers a complete iteration.
- Cache CompositeGlyph size in the iterator to avoid needing to recalculate it.
2021-08-12 14:50:42 -06:00
Behdad Esfahbod 23282c9c9d [util/hb-subset] Support ranges in --unicodes / --unicodes-file
Fixes https://github.com/harfbuzz/harfbuzz/issues/3109
2021-08-11 22:49:47 -06:00
Behdad Esfahbod 58bfe40794 [util] Move hb-subset away from main-font-text 2021-08-11 20:12:36 -06:00
Behdad Esfahbod 3147e081da Merge branch 'main' into post-parse 2021-08-11 19:33:00 -06:00
Khaled Hosny 43722cc9f4 [test] Add some real reverse sub tests 2021-08-11 19:30:16 -06:00
Behdad Esfahbod e0d7060f80 [test] Rename shaping/ to shape/ 2021-08-11 18:49:59 -06:00
Khaled Hosny f122698810 [test] Improve generate-expected-outputs.py diff output 2021-08-12 02:02:10 +02:00
Khaled Hosny 6ac6a2b27b [test] hb_ot_layout_language_get_feature_tags() 2021-08-11 16:33:37 -06:00
Khaled Hosny e9b743b26d [test] hb_ot_layout_script_get_language_tags() 2021-08-11 16:33:37 -06:00
Khaled Hosny bf438b815f [test] hb_ot_layout_table_get_feature_tags() 2021-08-11 16:33:37 -06:00
Khaled Hosny 2f88ea5a20 [test] hb_ot_layout_table_find_script() 2021-08-11 16:33:37 -06:00
Khaled Hosny f60caad7ff [test] hb_ot_layout_table_get_script_tags() 2021-08-11 16:33:37 -06:00
Garret Rieger c08f1b8903 [map] fix incorrect population count in hash map.
If the same key was set twice the population was being incorrectly incremented.
2021-08-10 14:00:55 -06:00
Garret Rieger e39c3bde7b [subset] fix bug in parsing glyf flags.
the bytes of the flag stream were being treated as signed integers instead of unsigned as specified in the spec.
2021-08-10 12:57:07 -06:00
Behdad Esfahbod f3acb977ec [test/shaping;util] Use ';' instead of ':' to separate test fields
Accept that in --batch mode. Also in batch mode don't send the 0th arg.

Related discussion: https://github.com/harfbuzz/harfbuzz/pull/3102
2021-08-10 11:08:36 -06:00
Behdad Esfahbod b83fd3a564 [util] Refactor batch-processing code into batch.hh 2021-08-10 02:32:55 -06:00
Garret Rieger 8c0c217b5a [subset] fail reference blob in face builder if allocation for table sorting fails.
Fixes https://oss-fuzz.com/testcase-detail/5041767803125760
2021-08-06 15:54:41 -06:00
Garret Rieger c2ee1fdd40 [subset] remove unsed table_entry struct. 2021-08-05 11:47:07 -06:00
Garret Rieger dea0fe5717 [subset] discard extra copies of a table in face builder.
Fixes #2361. Stores tables in the builder in a hashmap so you end up with at most one copy of each table. Table serialization order is now based on tag sort order instead of order of insertion into the builder.
2021-08-05 11:47:07 -06:00
Garret Rieger 2c024dc3cb [subset] prune redundant cmap12 subtables.
If the post subset cmap12 table is equivalent to another cmap subtable don't include the 12 table in the final subset. Matches change https://github.com/fonttools/fonttools/pull/2146 from fontTools.
2021-08-04 17:36:24 -06:00
Khaled Hosny 84946e4d2c [test] Suggest updating the expectation if ttx matches
https://github.com/harfbuzz/harfbuzz/issues/3089#issuecomment-892208892
2021-08-04 17:35:19 -06:00
Behdad Esfahbod 5086e10538 [test] Add failing fuzzer test case
From https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=36236
https://oss-fuzz.com/testcase-detail/5061207689134080
2021-08-04 11:55:53 -06:00
Khaled Hosny f698fe5aee [test] Always fail subset tests if hashes don’t match
Regardless of the pre sentience or absence of fonttools.
2021-08-04 10:26:14 -06:00
Khaled Hosny 9f544e5008 [test] Don’t skip subset tests early
Check for FontTools only when the checksums are mismatching.
2021-08-04 04:20:14 +02:00
Khaled Hosny b084153317 [test] Remove subset tests from the slow suite 2021-08-04 04:12:31 +02:00
Khaled Hosny 7ccc52b075 [test] Compare sha256 hash before TTX dumps
Most of time the files are identical, so instead of comparing the TTX
dump we can check sha256 hashes of the files first and if they match, we
don’t have to check the TTX dumps at all, making the subset tests orders
of magnitude faster.

time meson test --suite=subset down from:
real    0m19.418s
user    0m38.171s
sys     0m3.587s

to:
real	0m3.102s
user	0m8.622s
sys	0m1.701s

The expected files have been replaced by hb-subset output so they are
bit-identical where FontTools output might not.

The generate-expected-outputs.py now compares the hb-subset output with
fontttols subset and errors of they don’t match.
2021-08-04 04:10:38 +02:00
Khaled Hosny 770fbd5aa6 Revert "[test] Speed-up subset tests by saving TTX dump"
This reverts commit 278f44dcee.
2021-08-04 04:10:37 +02:00
Khaled Hosny 2cc9ed2b9e [test] More lookup_collect_glyphs() tests 2021-08-03 10:06:27 -06:00
Khaled Hosny 0664037401 [test] More lookup_collect_glyphs() tests 2021-08-03 10:06:27 -06:00
Khaled Hosny ecdd8e3f18 [test] More lookup_collect_glyphs() tests 2021-08-03 10:06:27 -06:00
Khaled Hosny f998d7e1ce [test] Add API test for lookup_collect_glyphs()
We don’t seem to have any tests exercising this API at all, and it is a
good check of GSUB/GPOS code.
2021-08-03 10:06:27 -06:00