The rest of layout subsetting depends on lookup indices being consistent with those computed during planning. So if an empty lookup is discarded during the subset phase it will invalidate all subsequent lookup indices. Generally we shouldn't end up with an empty lookup as we pre-prune them during the planning phase, but it can happen in rare cases such as when a subtable is considered degenerate (eg. #3853)
During splitting of PairPosFormat2 the code was assuming the maximum size of the generated coverage table would be equal too the current size. This is incorrect size the new coverage table may not preserve the ranges found in the original coverage table (since we are splitting based on class, not coverage) and in the worst case may convert from format2 to format1. So use the size of a format1 table as the max size.
When retain gids is enabled the subset plan may require the output of many empty glyphs. This change optimizes the glyf subsetting code when the number of retained glyphs << number of output glyphs. Unnessecary lookups to the glyph map are reduced by iterating through the glyph map instead of the output glyph set.
Also updated the script that is used to generate tests.With fonttools,
we now do instancing first and then subsetting.
With different order of subsetting and instancing operations on the same
VF file, fonttools seems to generate 2 different font files with different
glyph set.
1. do subsetting and then instancing: this seems result in a larger glyph
set in the font file. Lookups are collected from both retained features
and all possible alternate featurevariations, this leads to a larger
glyph set after glyph closurei. And instancer doesn't redo glyph
closure, it does lookups pruning only.
2. do instancing and then subsetting: lookups are collected from
features that are replaced already and possible alternate feature
variations
we need pass in source glyph's outline data to calculate boundaries, and
then drop the outline data if notdef-outline is specified.
Also disable shifting points for instancing in get_points () API