GPerf seems not allowing the empty lines though, current recipes are supposed to drop them.
but seems not working on some env.
So taking the proper way to do that instead of incompatible things against platforms.
FcFontSetFont() accesses fs->fonts in that macro though, there was no error checks
if it is null or not.
As a result, there was a code path that it could be a null.
Even though this is unlikely to see in usual use, it might be intentionally created
in a cache.
So if fs->fonts is a null, we should consider a cache is invalid.
FcConfigRealFilename() follows symlinks, but the link may be relative to the
directory containing the link. For example, on my system, I have this file:
/etc/fonts/conf.d/99-language-selector-zh.conf ->
../conf.avail/99-language-selector-zh.conf
Since /etc/fonts/conf.d is probably not in PATH, open()ing the file would fail.
This change makes FcConfigRealFilename() return the canonicalized filename
instead. So for the example above, it would return:
/etc/fonts/conf.avail/99-language-selector-zh.conf
This was causing bad font rendering in Chromium [1] after the regression I
introduced in 7ad010e80b.
[1] https://bugs.chromium.org/p/chromium/issues/detail?id=857511
CFI [1] is a dynamic analysis tool that checks types at runtime. It reports an
error when using a function with signature eg. (void (*)(char*)) as
(void (*)(void*)). This change adds some wrapper functions to avoid this issue.
In optimized builds, the functions should get optimized away.
[1] https://clang.llvm.org/docs/ControlFlowIntegrity.html
This changes the rewriting of the FC_FILE values for relocated caches to an earlier stage
while reading the cache. This is better, because it means all APIs will report the
rewritten paths, not just the once that use the list apis.
We do this by detecting the relocated case and duplicating the FcPattern and FcPatternElm
in an cache allocation (which will die with the cache) and then reusing the FcValueLists
from the cache.
This means that in the rewritten case we will use some more memory, but not the full
size of the cache. In a test here I had 800k of relocated caches, but ~200k of wasted
on duplicating the objects.
This should fix https://bugs.freedesktop.org/show_bug.cgi?id=106618
Whilst working on the Reproducible Builds[0] effort, we noticed that
fontconfig generates unreproducible cache files.
This is due to fc-cache uses the modification timestamps of each
directory in the "checksum" and "checksum_nano" members of the _FcCache
struct. This is so that it can identify which cache files are valid
and/or require regeneration.
This patch changes the behaviour of the checksum calculations to prefer
the value of the SOURCE_DATE_EPOCH[1] environment variable over the
directory's own mtime. This variable can then be exported by build
systems to ensure reproducible output.
If SOURCE_DATE_EPOCH is not set or is newer than the mtime of the
directory, the existing behaviour is unchanged.
This work was sponsored by Tails[2].
[0] https://reproducible-builds.org/
[1] https://reproducible-builds.org/specs/source-date-epoch/
[2] https://tails.boum.org/
realfilename is the file name after sysroot adjustments. It should be used
instead of filename in the call to FcOpen() which forwards the name directly to
open().
Though I don't explicitly request a sysroot, I was getting error messages saying
"failed reading config file". This CL fixes the error spam.
The expression "1 << 31" will cause UBSAN to complain with this error message:
runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
The same operation on unsigned types is fine, however. This CL replaces the
strings "1 <<" with "1U <<".
In 161c738 I switched from linear name scanning to binary searching.
That, however, ignored the fact that there might be more than one
name table entry for each pair we want to query.
To fix that and retain bsearch, I now get all name entries first,
sort them, and use for bsearching.
This fixes https://bugs.freedesktop.org/show_bug.cgi?id=105756
This makes scaning Voto Serif GX twice slower though, since we are
creating and sorting the list for each instance. In the next commit,
I'll share this list across different instances to fix this.
We were comparing the passed in key with the ready-to-insert key
rather than the key in the hashtable, so if you ever had a hash
conflicts we'll never insert the new item.
https://bugs.freedesktop.org/show_bug.cgi?id=101889
Fontconfig symbols were hardcoded to be either hidden or exported. This patch
adds configurable symbol visibility. This is useful for projects that want to
do in-tree fontconfig builds and not export any symbols, otherwise they would
conflict with the system library's symbols
Chromium is a project that does in-tree fontconfig builds, and the workaround
currently used is "#define visibility(x) // nothing" [1] and building with
"-fvisibility=hidden".
[1] https://cs.chromium.org/chromium/src/third_party/fontconfig/BUILD.gn?rcl=ce146f1f300988c960e1eecf8a61b238d6fd7f7f&l=62
This is the last piece of the puzzle for variable-font support in
fontconfig. This takes care of automatically setting the variation
settings when user requests a weight / width / size that has variation
in the matched font.
When a cache contains no fonts, it will be unloaded immediately.
Previously the certain alias and UUID entries will be purged at that time though,
this doesn't work when the targeted directory has sub-directories.
To avoid the unnecessary cache creation with the md5-based naming, try to keep them
as far as possible.
Although this way seems not perfectly working if the first directory to look up is like that
Reject caches when FcPattern isn't a constant.
This is usually unlikely to happen but reported.
I've decided to add more validation since this isn't reproducible
and easy to have a workaround rather than investigating 'why'.
https://bugs.freedesktop.org/show_bug.cgi?id=103237
Trying to address what these configuration files really do.
This change allows to see the short description that mention
the purpose of the content in the config file and obtain
them through API.
This change also encourage one who want to make some UI for
the user-specific configuration management. it is the main
purpose of this change for me though.
Aside from that, I've also made programs translatable. so
we see more dependencies on the build time for gettext,
and itstool to generate PO from xml.
VotoSerifGX has over 500 named instances, which means it also has over a thousand
name table entries. So we were looking for names for over 500 pattern, looking for
some thirty different name-ids, and using linear search across the 1000 entries!
Makes scanning VotoSerifGX three times faster. The rest is probably the lang
matching, which can also be shared across named-instances. Upcoming.
This reverts commit 57764e3a36449da25bb829c34cb08c54e9e5de90.
For regular font pattern we don't look into fvar, so it doesn't make sense to
get non-variation from it either.
Has two distinctions from FcCompareRange():
1. As best value, it returns query pattern size, even if it's out of font range,
2. Implements semi-closed interval, as that's what OS/2 v5 table defines
If font claims to support range [100,900], and request is for [250], then
return [250] in "rendered" pattern. Previously was returning [100,900].
This is desirable for varfonts weight and width, but probably not for size.
Will roll back size to return request size always, for non-empty ranges.
If "instance-number" part of face id is set to 0x8000, return a pattern
for variable font as a whole. This might have a range for weight, width,
and size.
If no variation is found, NULL is returned.
Not hooked up to FcQueryFaceAll() yet. For now, can be triggered using
fc-query -i 0x80000000
There's nothing assymetrical about how we match them. Previously we "considered"
them half-open because the OS/2 spec had usLowerOpticalPointSize as inclusive
and usUpperOpticalPointSize as exclusive. But we do not respect that.
Note that the parsing code accepts both anyway, because of the way our sscanf()
usage is written...
This is for clients to passthru font variation settings. Modeled
similar to FC_FONT_FEATURES. Each element value is for one axis
settings, eg. "abcd=2.3" where 'abcd' is the OpenType Font Variations
axis tag.
Needs docs update.
Use FcCompareNumber(). The FcCompareSize() returns 0 ("perfect match")
if v2 is zero. I cannot think of a use-case for this. The code has been
there from initial commit in 2002. I suppose back then Keith had a use
for size=0 to mean scalable or something. Anyway, remove and see.
Like FcFreeTypeQuery(), but adds patterns for all fonts found, including named
instances of variable fonts. If id is -1, then all collection faces are queried.
Returns number of fonts added.
This merges the same face loop that was in fc-query. and fcdir.c.
Needs documentation update.
Ignore control chars for purpose of emptiness check. I *think* U+0000 and U+000D
are rendered empty, but since they are not in blanks, for now just ignore them.
The FT_Select_Size() call is important for bitmap-only fonts.
Put it back. It was removed in e327c4e54544dac5415e8864e80d6b75a0c900fd
Remove some unused abstractions.
Basically we trust the font cmap table now...
New numbers:
behdad:src 0$ time fc-scan ~/fonts/ > after-noloca
real 0m55.788s
user 0m15.836s
sys 0m17.008s
behdad:src 0$
behdad:src 0$ time fc-scan ~/fonts/ > after-noloca
real 0m24.794s
user 0m12.164s
sys 0m12.420s
Before this change it was:
behdad:src 130$ time fc-scan ~/fonts/ > after
real 0m24.825s
user 0m12.408s
sys 0m11.356s
Not any faster! I suppose most time is being spent in loading cmap and advances now.
I'll see about loading hmtx ourselves.
With I/O numbers. Before:
behdad:src 0$ \time fc-scan ~/fonts/ > after
11.66user 12.17system 0:24.03elapsed 99%CPU (0avgtext+0avgdata 487684maxresident)k
2320inputs+50480outputs (21major+11468549minor)pagefaults 0swaps
after:
behdad:src 130$ \time fc-scan ~/fonts/ > after-noloca
11.94user 11.99system 0:24.11elapsed 99%CPU (0avgtext+0avgdata 487704maxresident)k
16inputs+50688outputs (0major+11464386minor)pagefaults 0swaps
We are definitely doing a lot less I/O. Surprisingly less in fact. I don't get it.
Part of https://bugs.freedesktop.org/show_bug.cgi?id=64766#c47
This is the approach introduced in
https://bugs.freedesktop.org/show_bug.cgi?id=64766#c30
Testing it with 11GB worth of stuff, before/after:
behdad:src 130$ time fc-scan ~/fonts/ > before
real 2m18.428s
user 1m17.008s
sys 0m20.576s
behdad:src 0$ time fc-scan ~/fonts/ > after
real 1m12.130s
user 0m18.180s
sys 0m19.952s
Running the after case a second time is significantly faster:
behdad:src 130$ time fc-scan ~/fonts/ > after
real 0m24.825s
user 0m12.408s
sys 0m11.356s
Next I'm going to try to not even read loca...
As written at:
https://lists.freedesktop.org/archives/fontconfig/2017-June/005929.html
I think FcCharSetFreezeOrig() and FcCharSetFindFrozen() should use the %
operator instead of & when computing the bucket index for
freezer->orig_hash_table, otherwise at most 8 buckets among the 67
available (FC_CHAR_SET_HASH_SIZE) are used.
Another way would be to change FC_CHAR_SET_HASH_SIZE to be of the form
2**n -1 (i.e., a power of two minus one). In such a case, the & and %
operators would be equivalent.
In fcLangCountrySets, it may happen that two charsets for the same
language but different territories are found in different FcChar32
"buckets" (different "columns" on the same line). This is currently the
case for the following pairs:
mn-cn and mn-mn
pap-an and pap-aw
The FcLangSetCompare() code so far used to return FcLangDifferentLang
instead of FcLangDifferentTerritory when comparing:
an FcLangSet containing only mn-cn with one containing only mn-mn
or
an FcLangSet containing only pap-an with one containing only pap-aw
This commit fixes this problem.
FcLangSetIndex() indicates "not found" with a non-negative return value.
Return value 0 doesn't imply "not found", it rather means "language
found at index 0 in fcLangCharSets".
This commit fixes a bug that can be reproduced like this:
- remove all languages starting with 'a' in fc-lang/Makefile.am (in
ORTH's definition);
- rebuild fontconfig with this change (-> new fc-lang/fclang.h);
- create an FcLangSet 'ls1' that contains at least the first language
from fcLangCharSets (i.e., the first *remaining* in lexicographic
order); let's assume it is "ba" for the sake of this description;
- create an FcLangSet 'ls2' that only contains the language "aa" (any
language starting with 'a' should work as well);
- check the return value of FcLangSetContains(ls1, ls2);
The expected return value is FcFalse, however it is FcTrue if you use
the code before this commit.
What happens is that FcLangSetIndex() returns 0, because this is the
index of the first slot after the not-found language "aa" in
fcLangCharSets (since we removed all languages starting with 'a').
However, this index happens to be non-negative, therefore
FcLangSetContainsLang() mistakenly infers that the language "aa" was
found in fcLangCharSets, and thus calls FcLangSetBitGet(ls1, 0), which
returns FcTrue since we've put the first remaining language "ba" in the
'ls1' language set.
The "return -low;" statement previously in FcLangSetIndex() was
inconsistent with the final return statement. "return -(low+1);" fixes
this inconsistency as well as the incorrect behavior described above.
Before this commit, FcCharSetHash() repeatedly used the address of the
'numbers' array of an FcCharSet to compute the FcCharSet hash, instead
of the value of each array element. This is not good for even spreading
of the FcCharSet objects among the various buckets of the hash table
(and should thus reduce performance). This bug appears to have been
mistakenly introduced in commit
cd2ec1a940 (June 2005).
On Windows, opened or locked files cannot be removed.
Since fontconfig locked an old cache file while updating the file,
fontconfig failed to replace the file with updated file on Windows.
This patch makes fontconfig does not lock the old cache file
while updating it on Windows.
To support the one of changes in gperf 3.1:
* The 'len' parameter of the hash function and of the lookup function is now
of type 'size_t' instead of 'unsigned int'. This makes it safe to call these
functions with strings of length > 4 GB, on 64-bit machines.
Validation fails when the FcValueList contains more than font->num.
this logic was wrong because font->num contains a number of the elements
in FcPatternElt but FcValue in FcValueList.
This corrects 7a4a5bd7.
Patch from Tobias Stoeckmann
The cache files are insufficiently validated. Even though the magic
number at the beginning of the file as well as time stamps are checked,
it is not verified if contained offsets are in legal ranges or are
even pointers.
The lack of validation allows an attacker to trigger arbitrary free()
calls, which in turn allows double free attacks and therefore arbitrary
code execution. Due to the conversion from offsets into pointers through
macros, this even allows to circumvent ASLR protections.
This attack vector allows privilege escalation when used with setuid
binaries like fbterm. A user can create ~/.fonts or any other
system-defined user-private font directory, run fc-cache and adjust
cache files in ~/.cache/fontconfig. The execution of setuid binaries will
scan these files and therefore are prone to attacks.
If it's not about code execution, an endless loop can be created by
letting linked lists become circular linked lists.
This patch verifies that:
- The file is not larger than the maximum addressable space, which
basically only affects 32 bit systems. This allows out of boundary
access into unallocated memory.
- Offsets are always positive or zero
- Offsets do not point outside file boundaries
- No pointers are allowed in cache files, every "pointer or offset"
field must be an offset or NULL
- Iterating linked lists must not take longer than the amount of elements
specified. A violation of this rule can break a possible endless loop.
If one or more of these points are violated, the cache is recreated.
This is current behaviour.
Even though this patch fixes many issues, the use of mmap() shall be
forbidden in setuid binaries. It is impossible to guarantee with these
checks that a malicious user does not change cache files after
verification. This should be handled in a different patch.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Akira TAGOH
Tom Anderson
Behdad Esfahbod
Matthieu Herrb
Alexander Larsson
Chris Lamb
Olivier Crête
Alexander Kanavin
Josselin Mouette
Florent Rougon
Masamichi Hosoda
Khem Raj
Tobias Stoeckmann