Windows does not update mtime of directory on FAT filesystem when
file is added to it or removed from it. Fontconfig uses mtime of
directory to check cache file aging and hence fails to detect
newly added or recently removed files.
This changeset detects FAT filesystem (currently implemented for
Linux) and adds generating checksum of directory entries instead
of using mtime which guarantees proper cache rebuild.
For non-FAT filesystems this patch adds single syscall per directory
which is negligeable overhead.
This fixes bug https://bugs.freedesktop.org/show_bug.cgi?id=25535
Signed-off-by: Mikhail Gusarov <dottedmag@dottedmag.net>
FcStat() logic is quite complicated in presence of various semi-broken operating
systems and filesystems, split it out in order to make it a bit easier.
Signed-off-by: Mikhail Gusarov <dottedmag@dottedmag.net>
link(2) might be failed on the filesystem that doesn't support
the hard link. e.g. FcAtomicLock() always fails on FAT filesystem
when link(2) is available.
So that may be a good idea to fallback if link(2) is failed.
Warn if the multiple values is set to <test>, including the case of
in <alias> because the behavior isn't intuitive since so many users
is asking for a help to get things working for their expectation.
Use multiple <match>s or <alias>es for OR operator and
multiple <test>s for AND operator.
Allows reading configuration files, fonts and cache files from
the directories where the XDG Base Directory Specification defines.
the old directories are still in the configuration files for
the backward compatibility.
Add a new attribute `ignore-blanks' to <test>.
When this is set to "true", any blanks in the string will be ignored
on comparison. This takes effects for compare="eq" or "not_eq" only.
Also changed the behavior of the comparison on <alias> too.
This patch isn't really tested as I don't have such a machine, but I
have a bug report that on m68k machines, double values are aligned on
smaller than 4 byte boundaries. If ALIGNOF_DOUBLE < sizeof(int),
the "expected" sizeof of FcValue is miscomputed. Use the maximum of 4
(sizeof (int)) and ALIGNOF_DOUBLE when computing the expected size of
FcValue.
Signed-off-by: Keith Packard <keithp@keithp.com>
Stop using mmap() if the cache file is stored on NFS.
also added FONTCONFIG_USE_MMAP environment variable to enforce the use of
or not the use of mmap(2) regardless of what the filesystem the cache files
are stored on.
Add "namelang" object to obtain the localized name in the font regardless
of the lang object. it's applied to "familylang", "stylelang" and
"fullnamelang" alltogether. this would helps if one wants to enforce
selecting them in the specific language if any. the default value for
the namelang object is determined from current locale.
Given that fontconfig will scan all of the cache file data during the
first font search, ask the kernel to start reading the pages right away.
Signed-off-by: Keith Packard <keithp@keithp.com>
Add configure options to set the directory to be installed:
--with-templatedir for the configuration files a.k.a.
/etc/fonts/conf.avail
--with-baseconfigdir for fonts.conf etc a.k.a. /etc/fonts
--with-configdir for the active configuration files a.k.a.
/etc/fonts/conf.d
--with-xmldir for fonts.dtd etc
and the default path for templatedir is changed to
${datadir}/fontconfig/conf.avail
This is a reasonably conservative increase in the number of buckets in the hash
table to 251. After FcInit(), there are 240 shared strings in use on my system
(from configuration files I assume). The hash value is stored in each link in
the chains so comparison are actually not very expensive. This change should
reduce the average length of chains by a factor of 8. With the reference
counted strings, it should keep the average length of chains to about 2. The
number of buckets is prime so as not to rely too much on the quality of the
hash function.
https://bugs.freedesktop.org/show_bug.cgi?id=17832#c5
Patch from Karl Tomlinson
In the previous code, the result of 'result' in the argument for
FcFontSetSort() and FcFontSetMatch() wasn't predictable and not reliable to
check if the return value is valid or not. this change is to ensure if it's
performed successfully.
$< isn't supported in BSD make say. $(RM) is pre-defined in GNU make
though, not in BSD make say. so changed to check on configure if it's
pre-defined by make, otherwise set the appropriate command to $(RM).
This would be a workaround until it has the certain pre-defined value.
When adding new functions, if the actual definition doesn't match the
header (say due to a typo), the regeneration of the internal headers
get confused and output bad cpp logic. This causes gcc to barf due
to mismatched #ifdef/#endif. Which is a pain to figure out due to
the sheer voulme of generated code.
So tweak the makealias script to detect this case and error out.
While we're here, improve the cpp output a bit to indent, include
comments, and merge similar ifdef blocks.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Newer gcc doesn't like when you switch on an enum and use a value
that isn't declared:
fcname.c: In function 'FcObjectValidType':
fcname.c:299:2: warning: case value '4294967295'
not in enumerated type 'FcType' [-Wswitch]
So tweak the logic to avoid this warning.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Latest configure code will setup FC_ARCHITECTURE directly rather than
going through ARCHITECTURE, so update fcarch.h accordingly.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
These funcs don't modify the incoming string, so add const markings.
This is the "right thing", shouldn't change the ABI, and fixes some
gcc warnings:
fccfg.c: In function 'FcConfigEvaluate':
fccfg.c:916:2: warning: passing argument 1 of 'IA__FcNameConstant'
discards 'const' qualifier from pointer target type [enabled by default]
fcalias.h:253:34: note: expected 'FcChar8 *' but
argument is of type 'const FcChar8 *'
fcxml.c: In function 'FcTypecheckExpr':
fcxml.c:604:2: warning: passing argument 1 of 'IA__FcNameGetConstant'
discards 'const' qualifier from pointer target type [enabled by default]
fcalias.h:251:37: note: expected 'FcChar8 *' but
argument is of type 'const FcChar8 *'
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
This shouldn't affect the ABI, makes FcStat more like the rest of the
fontconfig API, and fixes warnings where we pass FcChar8* pointers in
to this func from other places.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
We've already calculated the lengths of these strings, so re-use those
values to avoid having to rescan the strings multiple times.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
I broke FcFontSort() language handling at the end of 2008 with this
commit: c7641f28
G-d knows how many of the lang-matching bugs in bugzilla will be
fixed by this changed...
I'm really sorry, everyone!
- Do not throw away FC_FILE in FcNameUnparse
- Update the builtin "fclist" format to remove FC_FILE properly instead
- Switch fc-list to use FcPatternFormat()
Note that I had previously broken fc-list and it was not showing the
file name anymore. No one noticed that it seems! Now fixed.
Do not remove blacklisted fonts during cache generation. We already
apply the blacklist when reading the caches. The idea always has been
that the config should not affect caches built, although that design
was tarnished with the introduction of target="scan" configurations.
The syntax to add any characters to the charset table looks like:
<match target="scan">
<test name="family">
<string>Buggy Sans</string>
</test>
<edit name="charset" mode="assign">
<plus>
<name>charset</name>
<charset>
<int>0x3220</int> <!-- PARENTHESIZED IDEOGRAPH ONE -->
</charset>
</plus>
</edit>
</match>
To remove any characters from the charset table:
<match target="scan">
<test name="family">
<string>Buggy Sans</string>
</test>
<edit name="charset" mode="assign">
<minus>
<name>charset</name>
<charset>
<int>0x06CC</int> <!-- ARABIC LETTER FARSI YEH -->
<int>0x06D2</int> <!-- ARABIC LETTER YEH BARREE -->
<int>0x06D3</int> <!-- ARABIC LETTER YEH BARREE WITH HAMZA ABOVE -->
</charset>
</minus>
</edit>
</match>
You could also use the range element for convenience:
...
<charset>
<int>0x06CC</int> <!-- ARABIC LETTER FARSI YEH -->
<range>
<int>0x06D2</int> <!-- ARABIC LETTER YEH BARREE -->
<int>0x06D3</int> <!-- ARABIC LETTER YEH BARREE WITH HAMZA ABOVE -->
</range>
</charset>
...
The OT spec says:
"When building a Unicode font for Windows, the platform ID should be 3 and the
encoding ID should be 1. When building a symbol font for Windows, the platform
ID should be 3 and the encoding ID should be 0."
We were ignoring the SYMBOL_CS entry before. It's UTF-16/UCS-2 like the
UNICODE_CS.
Also, always use UTF-16BE instead of UCS-2BE. The conversion was doing
UTF-16BE anyway.
Last night in between my dreams I also noticed that we support Unicode
values up to 0x01000000 and not 0x00100000 which I thought before.
This covers the entire Unicode range.
Protect cache against future expansions of FcLangSet (adding new
orth files). Previously, doing so could change the size of
that struct. Indeed, that happened between 2.6.0 and 2.7.3, causing
crashes. Unfortunately, sizeof(FcLangSet) was not checked in fcarch.c.
This changes FcLangSet code to be able to cope with struct size changes.
And change cache format, hence bumping from 2 to 3.
Before a NULL config was passed down adn essentially FcFileScan was
equivalent to FcFreeTypeQuery. Now fc-scan tool correctly applies
the configuration to the scanned patterns.
The East Asian double-byte codepages have characters with backslash as
the second byte, so we must use _mbsrchr() instead of strrchr() when
looking at pathnames in the system codepage.
Must not call FcStrFree() on a value returned by
FcStrBufDoneStatic(). In the Windows code don't bother with dynamic
allocation, just use a local buffer.
Fontconfig assigns an index number to each language it knows about.
The index is used to index a bit in FcLangSet language map. The bit
map is stored in the cache.
Previously fc-lang simply sorted the list of languages and assigned
them an index starting from zero. Net effect is that whenever new
orth files were added, all the FcLangSet info in the cache files would
become invalid. This was causing weird bugs like this one:
https://bugzilla.redhat.com/show_bug.cgi?id=490888
With this commit we fix the index assigned to each language. The index
will be based on the order the orth files are passed to fc-lang. As a
result all orth files are explicitly listed in Makefile.am now, and
new additions should be made to the end of the list. The list is made
to reflect the sorted list of orthographies from 2.6.0 released followed
by new additions since.
This fixes the stability problem. Needless to say, recreating caches
is necessary before any new orthography is recognized in existing fonts,
but at least the existing caches are still valid and don't cause bugs
like the above.
The format '%{[]family,familylang{expr}}' expands expr once for the first
value of family and familylang, then for the second, etc, until both lists
are exhausted.
The '%{=unparse}' format expands to the FcNameUnparse() result on the
pattern. Need to add '%{=verbose}' for FcPatternPrint() output but
need to change that function to output to a string first.
Also added the '%{=fclist}' and '%{=fcmatch}' which format like the
default format of fc-list and fc-match respectively.
The format '%{family|delete( )}' expands to family values with space removed.
The format '%{family|translate( ,-)}' expands to family values with space
replaced by dash. Multiple chars are supported, like tr(1).
The format '%{family|escape(\\ )}' expands to family values with space
escaped using backslash.
The format '%{family|downcase}' for example prints the lowercase of
the family element. Three converters are defined right now:
'downcase', 'basename', and 'dirname'.
The conditional '%{?elt1,elt2,!elt3{expr1}{expr2}}' will evaluate
expr1 if elt1 and elt2 exist in pattern and elt3 doesn't exist, and
expr2 otherwise. The '{expr2}' part is optional.
The filtering, '%{+elt1,elt2,elt3{subexpr}}' will evaluate subexpr
with a pattern only having the listed elements from the surrounding
pattern.
The deletion, '%{-elt1,elt2,elt3{subexpr}}' will evaluate subexpr
with a the surrounding pattern sans the listed elements.
Diego Santa Cruz pointed out that we are using that API wrongly.
The forth argument is a pointer to a pointer. Turns out we don't
need that arugment and it accepts NULL, so just pass that.
To only work on writable charsets. Also, return a bool indicating whether
the merge changed the charset.
Also changes the implementation of FcCharSetMerge and FcCharSetIsSubset
Previously an index j was added to element score to prefer matches earlier
in the value list to the later ones. This index started from 0, meaning
that the score zero could be generated for the first element. By starting
j from one, scores for when the element exists in both pattern and font
can never be zero. The score zero is reserved for when the element is
NOT available in both font and pattern. We will use this property later.
This shouldn't change matching much. The only difference I can think of
is that if a font family exists both as a bitmap font and a scalable
version, and when requesting it at the size of the bitmap version,
previously the font returned was nondeterministic. Now the scalable
version will always be preferred.
Previously the matcher multiplied comparison results by 100 and added
index value to it. With long lists of families (lots of aliases),
reaching 100 is not that hard. That could result in a non-match early
in the list to be preferred over a match late in the list. Changing
the multiplier from 100 to 1000 should fix that.
To keep things relatively in order, the lang multiplier is changed
from 1000 to 10000.
Previously fc-match "xxx,nazli" matched Nazli, but "xxx, nazli" didn't.
This was because of a bug in FcCompareFamily's short-circuit check
that forgot to ignore spaces.
I can't understand why the special case is needed. Indeed, removing it
does not make any difference in the "fc-match --verbose" output, and
that's the only time fc-match uses FcPatternPrint.
Two changes:
- after mkdir(), we immediately chmod(), such that we are not affected
by stupid umask's.
- if a directory we want to use is not writable but exists, we try a
chmod on it. This is to recover from stupid umask's having affected
us with older versions.
The current behaviour of FcSortWalk() is to create a new FcCharSet on
each iteration that is the union of the previous iteration with the next
FcCharSet in the font set. This causes the existing FcCharSet to be
reproduced in its entirety and then allocates fresh leaves for the new
FcCharSet. In essence the number of allocations is quadratic wrt the
number of fonts required.
By introducing a new method for merging a new FcCharSet with an existing
one we can change the behaviour to be effectively linear with the number
of fonts - allocating no more leaves than necessary to cover all the
fonts in the set.
For example, profiling 'gedit UTF-8-demo.txt'
Allocator nAllocs nBytes
Before:
FcCharSetFindLeafCreate 62886 2012352
FcCharSetPutLeaf 9361 11441108
After:
FcCharSetFindLeafCreate 1940 62080
FcCharSetPutLeaf 281 190336
The savings are even more significant for applications like firefox-3.0b5
which need to switch between large number of fonts.
Before:
FcCharSetFindLeafCreate 4461192 142758144
FcCharSetPutLeaf 1124536 451574172
After:
FcCharSetFindLeafCreate 80359 2571488
FcCharSetPutLeaf 18940 9720522
Out of interest, the next most frequent allocations are
FcPatternObjectAddWithBinding 526029 10520580
tt_face_load_eblc 42103 2529892
Note that this also fixes a bug with FcFontList() where previously
it was NOT checking whether the config is up-to-date. May want to
keep the old behavior and document that ScanInterval is essentially
unused internally (FcFontSetList uses it, but we can remove that
too).
A private FcObjectGetSet() is implemented that provides an
FcObjectSet of all registered elements. FcFontSetList() is
then modified to use the object set from FcObjectGetSet() if
provided object-set is NULL.
Alternatively FcObjectGetSet() can be made public. In that
case fc-list can use that as a base if --verbose is included,
and also add any elements provided by the user (though that has
no effect, as all elements from the cache are already registered).
Currently fc-list ignores user-provided elements if --verbose
is specified.
The fact that we now drop final slashes from all filenames without
checking that the file name represents a directory may surprise some,
but it doesn't bother me really.
At OLPC, we came across a bug where the Browse activity (based on xulrunner)
took 100% CPU after an upgrade/. It turns out the Mozilla uses
FcConfigUptoDate() to check if new fonts have been added to the system, and
this function was always returning FcFalse since we have the mtimes of some
font directories set in the future. The attached patch makes
FcConfigUptoDate() print a warning and return FcTrue if mtime of directories
are in the future.
It seems indices in _FcMatchers array are slightly mixed up, MATCH_DECORATIVE
should be 10, not 11.
And MATCH_RASTERIZER_INDEX should be 13, not 12, right?
Akira TAGOH
Mikhail Gusarov
Keith Packard
Jeremy Huddleston
Mike Frysinger
Behdad Esfahbod
Tor Lillqvist
Karl Tomlinson
Serge van den Boom
Karl Tomlinson
Harald Fernengel
Chris Wilson
Sayamindu Dasgupta
Evgeniy Stepanov