Basically we trust the font cmap table now...
New numbers:
behdad:src 0$ time fc-scan ~/fonts/ > after-noloca
real 0m55.788s
user 0m15.836s
sys 0m17.008s
behdad:src 0$
behdad:src 0$ time fc-scan ~/fonts/ > after-noloca
real 0m24.794s
user 0m12.164s
sys 0m12.420s
Before this change it was:
behdad:src 130$ time fc-scan ~/fonts/ > after
real 0m24.825s
user 0m12.408s
sys 0m11.356s
Not any faster! I suppose most time is being spent in loading cmap and advances now.
I'll see about loading hmtx ourselves.
With I/O numbers. Before:
behdad:src 0$ \time fc-scan ~/fonts/ > after
11.66user 12.17system 0:24.03elapsed 99%CPU (0avgtext+0avgdata 487684maxresident)k
2320inputs+50480outputs (21major+11468549minor)pagefaults 0swaps
after:
behdad:src 130$ \time fc-scan ~/fonts/ > after-noloca
11.94user 11.99system 0:24.11elapsed 99%CPU (0avgtext+0avgdata 487704maxresident)k
16inputs+50688outputs (0major+11464386minor)pagefaults 0swaps
We are definitely doing a lot less I/O. Surprisingly less in fact. I don't get it.
Part of https://bugs.freedesktop.org/show_bug.cgi?id=64766#c47
This is the approach introduced in
https://bugs.freedesktop.org/show_bug.cgi?id=64766#c30
Testing it with 11GB worth of stuff, before/after:
behdad:src 130$ time fc-scan ~/fonts/ > before
real 2m18.428s
user 1m17.008s
sys 0m20.576s
behdad:src 0$ time fc-scan ~/fonts/ > after
real 1m12.130s
user 0m18.180s
sys 0m19.952s
Running the after case a second time is significantly faster:
behdad:src 130$ time fc-scan ~/fonts/ > after
real 0m24.825s
user 0m12.408s
sys 0m11.356s
Next I'm going to try to not even read loca...
As written at:
https://lists.freedesktop.org/archives/fontconfig/2017-June/005929.html
I think FcCharSetFreezeOrig() and FcCharSetFindFrozen() should use the %
operator instead of & when computing the bucket index for
freezer->orig_hash_table, otherwise at most 8 buckets among the 67
available (FC_CHAR_SET_HASH_SIZE) are used.
Another way would be to change FC_CHAR_SET_HASH_SIZE to be of the form
2**n -1 (i.e., a power of two minus one). In such a case, the & and %
operators would be equivalent.
In fcLangCountrySets, it may happen that two charsets for the same
language but different territories are found in different FcChar32
"buckets" (different "columns" on the same line). This is currently the
case for the following pairs:
mn-cn and mn-mn
pap-an and pap-aw
The FcLangSetCompare() code so far used to return FcLangDifferentLang
instead of FcLangDifferentTerritory when comparing:
an FcLangSet containing only mn-cn with one containing only mn-mn
or
an FcLangSet containing only pap-an with one containing only pap-aw
This commit fixes this problem.
FcLangSetIndex() indicates "not found" with a non-negative return value.
Return value 0 doesn't imply "not found", it rather means "language
found at index 0 in fcLangCharSets".
This commit fixes a bug that can be reproduced like this:
- remove all languages starting with 'a' in fc-lang/Makefile.am (in
ORTH's definition);
- rebuild fontconfig with this change (-> new fc-lang/fclang.h);
- create an FcLangSet 'ls1' that contains at least the first language
from fcLangCharSets (i.e., the first *remaining* in lexicographic
order); let's assume it is "ba" for the sake of this description;
- create an FcLangSet 'ls2' that only contains the language "aa" (any
language starting with 'a' should work as well);
- check the return value of FcLangSetContains(ls1, ls2);
The expected return value is FcFalse, however it is FcTrue if you use
the code before this commit.
What happens is that FcLangSetIndex() returns 0, because this is the
index of the first slot after the not-found language "aa" in
fcLangCharSets (since we removed all languages starting with 'a').
However, this index happens to be non-negative, therefore
FcLangSetContainsLang() mistakenly infers that the language "aa" was
found in fcLangCharSets, and thus calls FcLangSetBitGet(ls1, 0), which
returns FcTrue since we've put the first remaining language "ba" in the
'ls1' language set.
The "return -low;" statement previously in FcLangSetIndex() was
inconsistent with the final return statement. "return -(low+1);" fixes
this inconsistency as well as the incorrect behavior described above.
Before this commit, FcCharSetHash() repeatedly used the address of the
'numbers' array of an FcCharSet to compute the FcCharSet hash, instead
of the value of each array element. This is not good for even spreading
of the FcCharSet objects among the various buckets of the hash table
(and should thus reduce performance). This bug appears to have been
mistakenly introduced in commit
cd2ec1a940 (June 2005).
On Windows, opened or locked files cannot be removed.
Since fontconfig locked an old cache file while updating the file,
fontconfig failed to replace the file with updated file on Windows.
This patch makes fontconfig does not lock the old cache file
while updating it on Windows.
To support the one of changes in gperf 3.1:
* The 'len' parameter of the hash function and of the lookup function is now
of type 'size_t' instead of 'unsigned int'. This makes it safe to call these
functions with strings of length > 4 GB, on 64-bit machines.
Validation fails when the FcValueList contains more than font->num.
this logic was wrong because font->num contains a number of the elements
in FcPatternElt but FcValue in FcValueList.
This corrects 7a4a5bd7.
Patch from Tobias Stoeckmann
The cache files are insufficiently validated. Even though the magic
number at the beginning of the file as well as time stamps are checked,
it is not verified if contained offsets are in legal ranges or are
even pointers.
The lack of validation allows an attacker to trigger arbitrary free()
calls, which in turn allows double free attacks and therefore arbitrary
code execution. Due to the conversion from offsets into pointers through
macros, this even allows to circumvent ASLR protections.
This attack vector allows privilege escalation when used with setuid
binaries like fbterm. A user can create ~/.fonts or any other
system-defined user-private font directory, run fc-cache and adjust
cache files in ~/.cache/fontconfig. The execution of setuid binaries will
scan these files and therefore are prone to attacks.
If it's not about code execution, an endless loop can be created by
letting linked lists become circular linked lists.
This patch verifies that:
- The file is not larger than the maximum addressable space, which
basically only affects 32 bit systems. This allows out of boundary
access into unallocated memory.
- Offsets are always positive or zero
- Offsets do not point outside file boundaries
- No pointers are allowed in cache files, every "pointer or offset"
field must be an offset or NULL
- Iterating linked lists must not take longer than the amount of elements
specified. A violation of this rule can break a possible endless loop.
If one or more of these points are violated, the cache is recreated.
This is current behaviour.
Even though this patch fixes many issues, the use of mmap() shall be
forbidden in setuid binaries. It is impossible to guarantee with these
checks that a malicious user does not change cache files after
verification. This should be handled in a different patch.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Behdad Esfahbod
Akira TAGOH
Josselin Mouette
Florent Rougon
Masamichi Hosoda
Khem Raj
Tobias Stoeckmann
Erik de Castro Lopo
Petr Filipsky
Patrick Haller