Commit Graph

115 Commits

Author SHA1 Message Date
Garret Rieger 5914acb3cb [repacker] Clear distance and position cache when assigning a new space.
A change in space will effect the distance assigned to the node and any of it's children so clear the distance cache.
2021-12-10 11:09:03 -07:00
Garret Rieger be2c488e5d [repacker] Improve vertex priority packing.
Previous priority implementation would move a node further back within it's layer, but at max priority was unable to move any further up than that. This updates the implementation to have 3 priority levels:
1. Distance is reduced by half of table size.
2. Distance is reduced by full table size (move to beginning of the layer).
3. Distance is set to 0. Vertex will be packed as soon as possible.

Also makes the iterative resolutions aware of max priority, so it won't keep trying to raise priority beyond the maximum.
2021-12-09 17:46:52 -07:00
Garret Rieger 3e4a25098d [repacker] add a maximum number of roots that can be moved in one iteration.
Set to half of the roots in a space. This prevents the repacker from moving all roots in a space to a new space if their are overflows in every root.
2021-12-06 21:27:28 -07:00
Garret Rieger 02b12d7965 [repacker] Move all overflowing roots to a new space simultaneously. 2021-12-06 21:27:28 -07:00
Garret Rieger fa966bcc29 [repacker] create repacker output buffer after final length is known.
Don't rely on a buffer provided by the caller, as it may not be large enough.
2021-12-06 21:27:28 -07:00
Garret Rieger 39e76af19e [subset] add all_links () to object_t.
Helper to provide easy access to concatenated real and virtual links iterator.
2021-12-01 08:37:04 -07:00
Garret Rieger 9121ed0cec [subset] Improve sharing of Ligature subtables.
Ligature subtables use virtual links to enforce an ordering constraint between the subtables and the coverage table. Unfortunately this has the sideeffect of prevent the subtables from being shared by another Ligature with a different coverage table since object equality compares all links real and virtual. This change makes virtual links stored separately from real links and updates the equality check to only check real links. If an object is de-duped any virtual links it has are merged into the object that replaces it.
2021-12-01 08:37:04 -07:00
Garret Rieger 7615b94ecf [repacker] add 'virtual links' to the serializer.
These aren't associated with an offset field, but instead exist solely to add an ordering constraint to the object graph.
2021-11-02 12:19:13 -06:00
Behdad Esfahbod 8c05569930 [algs] Add hb_swap() ala, and using, std::swap()
Use it in vector.

Use ADL idiom.
2021-11-01 17:59:17 -06:00
Behdad Esfahbod 4262636926
Merge pull request #3248 from googlefonts/connected_components
[repacker] Keep connected subgraphs in the same space.
2021-10-23 13:18:22 -07:00
Garret Rieger d17155f5d2 [repacker] use possibly updated root idx after isolate_subgraph.
isolate_subgraph can change the root indices in some cases. So operations after the isolation need to use the roots from the output of isolate_subgraph.
2021-10-13 14:40:00 -07:00
Behdad Esfahbod a7a3608553 [docs] Rename overflow_resolution to repacker 2021-10-12 16:11:25 -07:00
Garret Rieger 6bc64317ca Add a writeup of the overflow resolution algorithm in harfbuzz. 2021-10-12 17:10:07 -06:00
Garret Rieger 5b882c420a [repacker] performance optimizations for topological sorting.
- Presize the output sorted graph and write it once in the correct order to avoid needing to reverse.
- Swap the old/new graph vectors instead of copying.
- Use a boolean vector for tracking visited instead of a set.
2021-10-06 11:43:47 -07:00
Garret Rieger ff7a86e9b0 [repacker] remove clone buffer, they are unnessecary.
When nodes are duplicated it's fine to just reuse head, tail from the node being cloned since we don't modify the contents.
2021-10-06 10:51:45 -07:00
Garret Rieger 8cae8b653d [repacker] add missing fini for parents vector. 2021-10-05 14:03:02 -07:00
Garret Rieger 7883b7ed95 [repacker] Add additional splitting spaces test.
Fix issues it uncovered.
2021-10-05 12:46:59 -07:00
Garret Rieger d97bd4268a [repacker] when assigning spaces use post isolation node indices.
isolate_subgraph can result in some of the roots being duplicated and moved to new indices, so do subgraph isolation before assign roots to spaces.
2021-10-05 10:53:05 -07:00
Garret Rieger 375a6c8f81 [repacker] add the ability to move subgraphs from a shared space into their own space.
Used to resolve overflows during manual resolution.
2021-09-29 18:14:57 -07:00
Garret Rieger 0dccbf368f [repacker] Handle the case where a subgraph root has an incoming 32 and 16 bit edge.
In this case the entire subgraph from that root will be duplicated.
2021-09-29 14:28:27 -07:00
Garret Rieger 816c5302a7 [repacker] restrict 32 bit subgraph connected component search to only nodes reachable via directed links. 2021-09-28 16:04:27 -07:00
Garret Rieger 67eb222b8e [repacker] when assigning each connected subgraph a space, also isolate it.
This will break any links coming from space 0 (ie. the 16 bit offset only space).
2021-09-28 13:54:54 -07:00
Garret Rieger 307acf7fb0 [repacker] add space assignment based on connected components.
Assign each connected component that is underneath one or more 32 bit offsets into a unique space. This ensures that 32 bit subgraphs which are connected are packed into the same space.
2021-09-28 12:08:18 -07:00
Garret Rieger c77bdf1dcf [repacker] begin storing each nodes parents.
Will be used for connected component search.
2021-09-27 14:42:54 -07:00
Garret Rieger efda2f14e1 [repacker] fix bug in subgraph isolation.
Prior to this fix id remapping at the end of the isolation operation was fed the old subgraph instead of the new one. Which results in object indices being remapped for the nodes outside of the new subgraph. Adds a test which detects this problem.
2021-09-24 17:53:08 -06:00
Garret Rieger fe155de989 [repacker] handle a couple of duplication edge cases.
- Detect cases where there are multiple links from a parent to a child. Don't duplicate that child if those are the only remaining links to the child.
- Correctly handle isolating a subgraph where the root idx has multiple incoming links.
2021-09-22 18:18:50 -06:00
Garret Rieger c19ec97da7 [repacker] reduce the bits used by order by 2 to give more bits to distance. 2021-09-22 18:18:50 -06:00
Garret Rieger d0daa7a59b [repacker] add a couple more complex isolation tests. 2021-09-22 18:18:50 -06:00
Garret Rieger 62c502cd9d [repacker] correctly update incoming_edges in duplicate. 2021-09-22 18:18:50 -06:00
Garret Rieger a57ef8df7e [repacker] default space to 0.
Since vector push() init's new objects to all zeros.
2021-09-22 18:18:50 -06:00
Garret Rieger 58facaade1 [repacker] put each 32 bit subgraph into it's own packing space.
Each subgraph pointed to by a 32 bit offset should be packed into it's own space. This adds a space property to vertices which affects the distance calculation. This effectively places the distances for all of the nodes of a 32 bit subgraph into a distinct range. Thus all of the nodes of the subgraph will be packed together.
2021-09-22 18:18:50 -06:00
Garret Rieger 543a3f9733 [repacker] Add repacker test for subgraph isolation. 2021-09-22 18:18:50 -06:00
Garret Rieger 7147f169d6 [repacker] recursively duplicate nodes during isolation.
If a node is duplicated during isolation then any children it has will have incoming links from outside the subgraph (from the duplicated node and the original node), so they must be duplicated too.
2021-09-22 18:18:50 -06:00
Garret Rieger 41bbf2812e [repacker] do extension subtable isolation before starting resolution attempts. 2021-09-22 18:18:50 -06:00
Garret Rieger 8d8b7458a4 [repacker] extract overflows processing into its own method. 2021-09-22 18:18:50 -06:00
Garret Rieger b14b3f13ba [repacker] begin implementing the ability to isolate extension subtables.
Adds isolate_subgraph operation to the repacker. This severs any links from outside a subgraph by duplicating the affected vertices. This will be used to isolate the subgraphs of a extension subtable from the rest of object graph. Thus allowing the extension subtable to be packed far away from the rest of the objects.
2021-09-22 18:18:50 -06:00
Garret Rieger 02c4a516db Add a debug message when offset overflow resolution fails. 2021-09-22 18:18:50 -06:00
Garret Rieger 74f96d9d4b [repacker] fix heap use after free in repacker.
Don't store a reference to the link in overflow records as the link object may be freed if the sorted graph vector is resized.
2021-09-19 09:06:17 -06:00
Behdad Esfahbod 2337f0d047 Internally use hb_malloc/.../hb_free instead of malloc/.../free
Redefining those stock names as macros was conflicting with gcc 10
headers.

Fixes https://github.com/harfbuzz/harfbuzz/issues/3044
2021-07-08 10:54:09 -07:00
Garret Rieger 5e0ec33b3d Error when link width not in [2, 4] 2021-05-12 16:05:11 -06:00
Qunxin Liu b23f29bf05 [subset] Add subset () method for COLRv1 Paint tables, BaseGlyphV1List and LayerV1List
Also add support for Offset24 in serializer and repacker
2021-05-12 16:05:11 -06:00
Garret Rieger ec4321068b [subset] fix infinite loop caused by alloc failure in repacker.
Fixes: https://oss-fuzz.com/testcase-detail/5609112151916544.
2021-04-20 13:18:07 -06:00
Garret Rieger 0e845d973e [subset] fix memory leak in repacker caused by failed alloc.
Fixes: https://oss-fuzz.com/testcase-detail/5616763250278400.
2021-04-20 13:18:07 -06:00
Garret Rieger 71d6d15600 [subset] clamp distance to prevent shifting outside of the limits of int64.
Fixes https://oss-fuzz.com/testcase-detail/4961171477233664.
2021-04-06 11:48:39 -06:00
Garret Rieger c5c13006a1 [subset] fix memory leaks found in https://oss-fuzz.com/testcase-detail/5179935334465536 2021-03-31 12:37:45 -06:00
Garret Rieger 3827a3eb56 [subset] rename serializer::set_error() to err(). 2021-03-18 11:20:03 -07:00
Garret Rieger f561fa6e4c Change priority queue to use (priority, value) instead of (value, priority). 2021-03-18 11:13:47 -07:00
Garret Rieger b14475d2ae [subset] further changes to serializer error handling.
- Rename enum type and enum members.
- in_errors() now returns true for any error having been set. hb-subset now looks for offset overflow only errors to divert to repacker.
- Added INT_OVERFLOW and ARRAY_OVERFLOW enum values.
2021-03-18 10:51:26 -07:00
Garret Rieger 73ed59f7a6 [subset] store errors in the serializer as a flag set.
Make check_assign/check_equal specify the type of error to set.
2021-03-17 15:58:34 -07:00
Garret Rieger cf79fc342d [subset] limit priority bumps to 16. 2021-03-17 15:53:58 -07:00
Garret Rieger d3e2ba7c01 [subset] comment cleanup in hb-repacker.hh 2021-03-17 15:53:58 -07:00
Garret Rieger bb5c80a7c2 [subset] add error tracking to the repacker.
Also check for allocation failures as needed.
2021-03-17 15:53:58 -07:00
Garret Rieger 6e9468fcfb [subset] cleanup memory leaks in the repacker. 2021-03-17 15:53:58 -07:00
Garret Rieger a7a86a6eb4 [subset] Add prioritization offset resolution.
Vertices can now be prioritized to force them to sort closer to their parent. The resolver will attempt to use this for overflows on non-shared vertices.
2021-03-17 15:53:58 -07:00
Garret Rieger b452b2c76c [subset] refactor repacker graph to cache edge count and distances of vertices. 2021-03-17 15:53:57 -07:00
Garret Rieger 75414e82b5 [subset] Add table duplication overflow resolution. 2021-03-17 15:53:57 -07:00
Garret Rieger 8286bd8094 [subset] use vectors instead of hashmaps throughout the repacker since all keys will be mapped for these use cases. 2021-03-17 15:53:57 -07:00
Garret Rieger 519ae96617 [subset] switch sort_shortest_distance() to use priority queue. 2021-03-17 15:53:57 -07:00
Garret Rieger 5d3511e5b1 [subset] Change compute_distances() to use a priority queue. 2021-03-17 15:53:57 -07:00
Garret Rieger 4c8dd41ed9 [subset] re-write compute distances to use an array lookup for the distance map. 2021-03-17 15:53:57 -07:00
Garret Rieger aaa7873d42 [subset] add topological sort by closest distance via Dijkstra's algorithm. 2021-03-17 15:53:57 -07:00
Garret Rieger 8ebe5d734f Implement will_overflow (). 2021-03-17 15:53:57 -07:00
Garret Rieger f4c78cc7dd [subset] Implement Kahn's algo for topological sorting instead of BFS. 2021-03-17 15:53:57 -07:00
Garret Rieger 00f393dc3f [subset] finish up BFS sort implementation. 2021-03-17 15:53:57 -07:00
Garret Rieger 1584d3cb8f [subset] Start a proof of concept implementation of the GSUB/GPOS offset overflow resolver. 2021-03-17 15:53:57 -07:00