Compare commits

...

441 Commits

Author SHA1 Message Date
George Sokianos 92d3ef3d19 Changes in the Makefile.os4 to fully compile 2023-01-11 00:23:36 +00:00
George Sokianos 9098ed0e31 Added OS4 makefile 2023-01-10 23:51:34 +00:00
Chun-wei Fan c47cf796fc tools/psl.c: Fix build on Windows
localtime_r() is not available on Windows but a more-secure variant
of localtime(), localtime_s() is provided on Windows.

Define localtime_r() on Windows as its arguments are reversed as compared
to localetime_s(), to achive more or less the same purpose.
2022-12-27 11:09:14 +01:00
Chun-wei Fan 86923341a3 meson: Clean up ICU dependency search
Since we are using Meson 0.60.0 or later, use CMake to help us find ICU,
which is cleaner and more comprehensive, and allows us to use libicu_dep
= depedency(...) even on Visual Studio, where CMake will find the
appropriate ICU headers and libraries for us internally.

Sadly, due to peculiarities, we have to live with the warning that comes
up as we needed to look for ICU in the older method instead of the
recommended modules: ['ICU::uc'] method.
2022-12-27 11:07:54 +01:00
Tim Rühsen 927b202c69 Release v0.21.2 2022-12-26 13:46:23 +01:00
Tim Rühsen f2bdc1709d tools/psl.c: Use localtime_r instead of localtime 2022-12-09 13:38:14 +01:00
Tim Rühsen 5eaa8b4796 CI: Install missing autopoint 2022-12-09 13:28:57 +01:00
Tim Rühsen c03f48aa24 Update PSL 2022-12-09 13:28:57 +01:00
LGTM Migrator 4cb5c0cea0 Add CodeQL workflow for GitHub code scanning 2022-12-09 13:10:11 +01:00
Matt Turner 182bfd0e5f Use AC_SYS_LARGEFILE
Ensures that calls to fopen() and stat() can handle largefiles.
2022-11-06 19:31:50 +01:00
Cœur c546aafc0a Fix "Although the value stored to 'pos' is used in the enclosing expression, the value is never actually read from 'pos'" 2022-10-15 19:06:05 +02:00
Eli Schwartz b9503b64c3 meson: fix regression that made it impossible to build
In commit aa4909766c an option was
modified and changed its type from a list of possible strings to a
true/false value.

However, the build wasn't updated to correspond to that change, so it
didn't logically work -- and Meson even threw a type error saying you
cannot compare strings to booleans.

The actual change was basically "do not use this option anymore except
to pass a macro to the source file and embed the data at all". So we can
simply get rid of a bunch of complex dependency conditionals.

Fixes #185
2022-07-26 12:16:13 +02:00
Tim Rühsen a7b63cbb35
Merge pull request #183 from neheb/patch-1
meson: switch iconv to meson's handling
2022-06-10 15:54:38 +02:00
Tim Rühsen cf6878fd24 Update PSL submodule 2022-06-10 15:51:21 +02:00
Tim Rühsen 7a4cbdd386
Merge pull request #181 from hwti/builtin-option
Make builtin option independant from runtime one
2022-06-10 15:49:58 +02:00
Rosen Penev b27d4b69ce
meson: switch iconv to meson's handling
Fixes compilation under OpenWrt.
2022-06-09 17:01:18 -07:00
Loïc Yhuel aa4909766c Make builtin option independant from runtime one
Since commit deabd4a, psl2c has been replaced by psl-make-dafsa.
The builtin PSL data is always generated the same way, and the value of the builtin
configure option does not matter anymore.

The built-in data is now embedded by default (instead of the previous "auto"), unless
explicitly disabled by --disable-builtin/-Dbuiltin=false.

When not using an run-time runtime IDNA library (--disable-runtime/-Druntime=no), libpsl can
now include the built-in data without any build-time dependency.
2022-06-06 22:48:32 +02:00
Tim Rühsen 6a82bcbc2a Fix comments from C++ to C/C++ in psl-make-dafsa 2022-06-06 12:32:17 +02:00
Tim Rühsen 664f3dc852
Merge pull request #180 from eugmes/remove-macro
Remove definition of _GCC_VERSION_AT_LEAST
2022-02-18 19:39:29 +01:00
Ievgenii Meshcheriakov 9d39483d42 Remove definition of _GCC_VERSION_AT_LEAST
This macro is unused and is the only piece for compiler-specific
code in this file.
2022-02-18 16:57:35 +01:00
Tim Rühsen 67aadbe967 Remove obsolete fuzzer builds from fuzz/run-clang 2022-01-16 19:36:07 +01:00
Tim Rühsen 54038ae4d9 Require gettext version 0.19.3 2022-01-16 19:14:33 +01:00
Tim Rühsen 97f8ae52c1 Update fuzzing tools and build scripts 2022-01-16 12:59:23 +01:00
Tim Rühsen 21d2d51911 Avoid 'NULL + 1' as it is UB 2022-01-16 12:55:51 +01:00
Tim Rühsen 55d0ae04de Avoid 8bit overflow in is_public_suffix() 2022-01-16 12:51:33 +01:00
Tim Rühsen 042c586371 Update corpora from OSS-FUZZ 2022-01-16 12:08:01 +01:00
Tim Rühsen 1023a9ad12 Fix stack buffer overflow WRITE 1 in domain_to_punycode()
Reported-by: oss-fuzz (issue 39424 and issue 39226)

The affected code would only be built into the library when
configured to build without any IDNA library.
2022-01-15 22:38:36 +01:00
Tim Rühsen 044b5ed5d4 Bump gettext version to 0.19.3 2022-01-15 14:26:06 +01:00
Tim Rühsen be03c8ef97 Update m4/ax_check_compile_flag.m4 2022-01-15 14:18:37 +01:00
Tim Rühsen 864d938f39 Add brackets for AC_INIT argument 2022-01-15 14:18:18 +01:00
Tim Rühsen 5f796be788
Merge pull request #177 from equeim/meson-static
meson: automatically add PSL_STATIC define when building Windows static library and fix tests/tools link errors
2022-01-15 13:54:28 +01:00
Alexey Rochev 1159bd46c3 meson: fix tests/tools link errors when building shared library for Windows
We need to link them with ws2_32 explicitly.
2022-01-13 01:55:03 +03:00
Alexey Rochev ba6f11f751 meson: automatically add PSL_STATIC define when building Windows static library 2022-01-13 01:54:38 +03:00
Tim Rühsen 4a10a1698b
Merge pull request #179 from equeim/meson-libicu-static
Use C++ linker when building with libicu
2022-01-08 19:43:10 +01:00
Alexey Rochev 02c8476071 Use C++ linker when building with libicu
ICU is a C++ library, event though it has C interface.
Therefore we have to use C++ linker when linking with it.
2022-01-07 01:12:17 +03:00
Tim Rühsen 88a75d4711
Merge pull request #178 from equeim/tests-libicu
Fix Meson build
2022-01-06 19:29:39 +01:00
Alexey Rochev 5a29a64444 Remove libicu includes from test-registrable-domain.c
It doesn't seem like they are used,
and they cause build failures with meson.
2022-01-06 03:03:43 +03:00
Alexey Rochev 43fc4f0631 meson: bring back libiconv dependency lookup 2022-01-06 03:03:43 +03:00
Alexey Rochev 0ecc788797 Remove meson.build generation and separate project and library versions in their own files
Project and library version are now stored in
version.txt and libtool_version_info.txt files respectively.
Both autotools and meson get their versioning information from this files.
Meson additionally automatically transforms library version
from libtool's own format.

Also raise meson version requirement from 0.47 to 0.57 which
is required by these changes.
2022-01-06 03:02:33 +03:00
Tim Rühsen 4afd9e705b
Merge pull request #174 from rockdaboot/tmp-wsa-startup
Add WSAStartup() for Windows psl tool and tests.
2021-11-26 19:18:19 +01:00
Tim Rühsen f5035b854b Add WSAStartup() for Windows psl tool and tests.
As reported at https://github.com/rockdaboot/libpsl/issues/173,
psl_is_cookie_domain_acceptable() doesn't work properly without
WSAStartup() on Windows.

Co-authored-by: gvanem@yahoo.no
2021-11-26 19:10:37 +01:00
Tim Rühsen b2625f93f2 Fix write buffer overflow by 1 in domain_to_punycode()
This issue has been triggered after the previous commit increased
the size of label_buf.

It has been found by OSS-Fuzz (issue 39226).
The testcase is included into the unit tests.
2021-09-26 18:02:04 +02:00
Tim Rühsen 304ca77522 Increase label size from 48 -> 128 2021-09-18 12:24:45 +02:00
Tim Rühsen 50ff7f2edf Update upstream PSL 2021-09-18 12:22:16 +02:00
Tim Rühsen 6ce6c4e399
Merge pull request #168 from nacho/nacho/gtk-doc
meson: build reference manual with gtk_doc
2021-01-31 17:38:23 +01:00
Ignacio Casal Quinteiro a1b91e86e4 meson: build reference manual with gtk_doc 2021-01-26 12:38:01 +01:00
Tim Rühsen 6f4f6d766c Replace obsolete AC_HELP_STRING with AS_HELP_STRING 2021-01-16 18:16:35 +01:00
Tim Rühsen 99e7e6a2af
Merge pull request #166 from quink-black/fix-build-iconv
Add libiconv dep to meson build again
2021-01-16 18:07:35 +01:00
Tim Rühsen 44443ad3a4 Update PSL data from upstream 2021-01-16 17:56:23 +01:00
Tim Rühsen fef1568bef
Merge pull request #167 from SteelPangolin/meson-install-more
Meson: install libpsl man page, DAFSA builder
2021-01-16 17:46:39 +01:00
Erica Ehrhardt 62081a021f Meson: install libpsl man page, DAFSA builder
Partially addresses issue #136,
at least when building from a distribution tarball.
2021-01-14 17:13:46 -08:00
Zhao Zhili 5b63816e82 Add libiconv dep to meson build again
Build success on macOS and ubuntu.
2021-01-07 20:59:17 +08:00
Tim Rühsen 6a5e41edc5
Merge pull request #164 from tp-m/fix-meson-build
meson: fix build
2020-10-26 21:27:11 +01:00
Tim-Philipp Müller 8e02c2c4f1 meson: fix build
src/meson.build:19:0: ERROR: Unknown variable "libiconv".
2020-10-26 10:46:53 +00:00
Tim Rühsen d61859f4b5
Merge pull request #163 from rockdaboot/tmp-doc-buildfromtarball
Add 'Building from tarball' to README.md
2020-10-25 11:56:20 +01:00
Tim Rühsen 5ccf1e85fe Add 'Building from tarball' to README.md 2020-10-25 11:55:17 +01:00
Tim Rühsen 03df58fc96
Merge pull request #162 from thinkski/master
Add libiconv dep to meson build
2020-09-21 16:22:17 +02:00
Chris Hiszpanski 079e8d0f6c Add libiconv dep to meson build 2020-09-20 04:57:12 -07:00
Tim Rühsen 17cce94216
Merge pull request #158 from kloczek/master
Install psl executable and man pages.
2020-08-05 20:06:12 +02:00
Tomasz Kłoczko 47bab910ee Install psl executable and man pages.
Resolve #156
2020-08-05 18:38:11 +01:00
Tim Rühsen a5e6f26cce Release V0.21.1 2020-07-18 13:15:00 +02:00
Tim Rühsen 0073dd737d
Merge pull request #155 from damz/pr/fix-coveralls
Upgrade Ubuntu to fix the coveralls issue
2020-07-02 18:52:47 +02:00
Damien Tournoud c471792664 Try if upgrading ubuntu helps with the coveralls issue 2020-07-02 09:43:52 -07:00
Tim Rühsen f5142207aa
Merge pull request #154 from kotnik/psh-test-update
Update Platform.sh example in test_data
2020-07-02 18:38:52 +02:00
Nikola Kotur f364cea73e Update Platform.sh example in test data 2020-07-02 17:04:27 +02:00
Tim Rühsen 72365d3036
Merge pull request #153 from nirbheek/fix-build-windows
meson: Fix running of psl-make-dafsa on Windows
2020-04-06 17:03:17 +02:00
Nirbheek Chauhan 0c11bf19df meson: Fix running of psl-make-dafsa on Windows
`find_program()` evaluates to `python psl-make-dafsa` on Windows, so
running it with `python` evaluates to `python python psl-make-dafsa`,
causing this error on Windows:

```
"c:/python38/python.exe" "python" "C:/projects/repos/libpsl/src/psl-make-dafsa" "--output-format=binary" "C:/projects/repos/libpsl/list/public_suffix_list.dat" "tests/psl.dafsa"
c:/python38/python.exe: can't open file 'python': [Errno 2] No such file or directory
```
2020-04-06 20:18:45 +05:30
Tim Rühsen 20ad8fb106
Merge pull request #152 from nirbheek/fix-typo
meson: Fix typo
2020-04-03 17:29:05 +02:00
Nirbheek Chauhan 116b5fe7b9 meson: Fix typo 2020-04-03 20:50:07 +05:30
Tim Rühsen 8d9e490ee7
Merge pull request #151 from nirbheek/fix-buildtype-usage
meson: Use the b_vscrt option for selecting the CRT
2020-04-03 15:22:51 +02:00
Nirbheek Chauhan a582a9c142 meson: Use the b_vscrt option for selecting the CRT
This option has been available since 0.48, and we should use it
instead of only guessing based on buildtype.
2020-04-03 16:45:11 +05:30
Tim Rühsen 96412ad0e0
Merge pull request #150 from ystreet/meson-python
meson: run python script against the meson version of python
2020-04-02 10:22:12 +02:00
Matthew Waters a10c374dad meson: run python script against the meson version of python
Avoids using a python that may not exist in a python3 only distribution:

/usr/bin/env: ‘python’: No such file or directory
2020-04-02 15:16:20 +11:00
Tim Rühsen d54ecbd05e Add meson check for strings.h 2020-03-10 15:57:33 +01:00
Tim Rühsen 55889b1d9e Add recent changes from meson.build to meson.build.in 2020-03-10 15:47:55 +01:00
Tim Rühsen b9a190264f psl.c: Include strings.h only if it is available 2020-03-10 15:16:25 +01:00
Tim Rühsen a2411a7207 Add Github FUNDING.yml 2020-03-07 20:42:04 +01:00
Tim Rühsen d26a0acfd7
Merge pull request #148 from nirbheek/meson-port
meson: Use libtool library versioning, same as autotools
2020-02-28 14:01:36 +01:00
Nirbheek Chauhan 841ae6dddd meson: Use libtool library versioning, same as autotools
This helps maintain ABI compatibility with the Autotools build so it's
a drop-in replacement.
2020-02-28 13:20:23 +05:30
Tim Rühsen fca622f587 Include string.h in src/psl.c 2020-02-01 17:04:36 +01:00
Tim Rühsen 39c2990840
Merge pull request #146 from fanc999/master.msv
Improve Visual Studio builds Add instructions
2019-10-30 14:32:02 +01:00
Chun-wei Fan 0e8addfe20 NMake builds: Make PDB generation cleaner
Separate the locations where the intermediate PDB files are generated,
to make things cleaner.
2019-10-30 18:09:00 +08:00
Chun-wei Fan 28319ed36f NMake builds: Include Visual Studio version in output dir
This avoids builds done with other Visual Studio versions being
confused with the one that is being used now.
2019-10-30 18:07:19 +08:00
Chun-wei Fan 3e92891d8c msvc/detectenv-msvc.mak: Accomdate Visual Studio 2019
This adds detection logic for Visual Studio 2019.
2019-10-30 17:36:38 +08:00
Chun-wei Fan f277af5bd8 Add instructions on building with MSVC
This includes building via NMake and Meson.
2019-10-30 17:36:06 +08:00
Chun-wei Fan 978e2989ca NMake: Fix building test programs
Define PSL_DAFSA and PSL_ASCII_DAFSA to point to the corresponding DAFSA
files that we generated during the build.
2019-10-30 15:26:02 +08:00
Tim Rühsen c6114a690c
Merge pull request #141 from blino/flags-fix
Fix auto flags for icu runtime and libidn builtin
2019-08-28 14:17:54 +02:00
Olivier Blin 75603d131d Fix auto flags for icu runtime and libidn builtin 2019-08-28 13:59:26 +02:00
Tim Rühsen e497735e45
Merge pull request #140 from rossburton/py
Makefile.am: use PYTHON when invoking psl-make-dafsa
2019-07-12 09:59:34 +02:00
Ross Burton b4fec5d0dd Makefile.am: use PYTHON when invoking psl-make-dafsa
In an environment where only Python 3 is installed, configure.ac finds and sets
PYTHON=python3 correctly but src/psl-make-dafsa is called directly, so the hashbang of
`#!/usr/bin/env python` is used which doesn't exist.

Fix this by explicitly running $(PYTHON) when using the tool.
2019-07-11 17:47:11 +01:00
Tim Rühsen 22a7e4045c
Merge pull request #139 from xhaakon/win32-build
Fix build when dirent.h is not available
2019-06-05 09:56:09 +02:00
Jakub Adam 49ce76a8cf Fix build when dirent.h is not available 2019-06-05 09:45:50 +02:00
Tim Rühsen f850de85ab
Merge pull request #138 from xclaesse/langinfo
Fix build when langinfo.h is not found
2019-06-03 22:06:14 +02:00
Xavier Claessens 93e3bb3e29 Fix build when langinfo.h is not found 2019-06-03 15:52:48 -04:00
Tim Rühsen 867872e80d Include stdio.h for fmemopen in fuzzers 2019-06-02 13:39:15 +02:00
Tim Rühsen f32c240cc0 Remove alloca from fuzz/main.c 2019-06-02 13:07:35 +02:00
Tim Rühsen 5131ef0d5d
Merge pull request #137 from kanavin/fix-tree-index
gtk-doc: do not include tree_index.sgml
2019-06-01 14:27:44 +02:00
Alexander Kanavin 87d1add318 gtk-doc: do not include tree_index.sgml
gtk-doc 1.30 no longer generates the file if the object tree is empty.
2019-06-01 13:13:50 +02:00
Tim Rühsen 999f28cc15 Use %NULL and %FILE overall in gtk-docs 2019-05-26 18:01:50 +02:00
Tim Rühsen 74ee95079d Add missing api-indeces for gtk-doc 2019-05-26 18:00:29 +02:00
Tim Rühsen ef352f8923 Add PSL_API to libpsl-sections.txt 2019-05-26 17:59:44 +02:00
Tim Rühsen 87715ba378
Merge pull request #131 from sunpoet/master
Fix build when configured with --with-psl-file
2019-04-17 15:57:06 +02:00
Po-Chuan Hsieh 9347024221
Fix build when configured with --with-psl-file
The error message is as follows:
Making all in tests
make: don't know how to make ../list/public_suffix_list.dat. Stop

make: stopped in /usr/ports/works/usr/ports/dns/libpsl/work/libpsl-libpsl-0.21.0/tests
*** Error code 1

Stop.
2019-04-17 13:25:48 +00:00
Tim Rühsen 8b913fe484 Dynamically create version in meson.build 2019-04-17 12:46:46 +02:00
Tim Rühsen 082ee9740e Fix patch version in include/meson.build 2019-04-17 12:33:03 +02:00
Tim Rühsen 94a7e7c6ed Include meson build files into tarball 2019-04-17 12:05:00 +02:00
Tim Rühsen 1eaf180359 Release V0.21.0 2019-04-16 15:10:17 +02:00
Tim Rühsen e02ac83d93 Add building of an lzip tarball archive 2019-04-16 14:42:34 +02:00
Tim Rühsen 46a990736c Create test DAFSAs if older than public_suffix_list.dat 2019-04-05 15:08:32 +02:00
Tim Rühsen 35c781924f Update PSL data from upstream 2019-04-05 15:04:57 +02:00
Tim Rühsen 0c85c95dd1
Merge pull request #126 from xhaakon/master
Windows: Define ssize_t for MSVC
2019-04-05 15:01:46 +02:00
Jakub Adam c230a7b48b Windows: MSVC doesn't define ssize_t 2019-04-05 13:15:34 +02:00
Tim Rühsen 19359c6733
Merge pull request #124 from carlo-bramini/fix-older-windows
Fix older windows
2019-03-26 12:55:25 +01:00
carlo-bramini 7dcb69eb1d Windows: decrease minimum version to NT2000
Use WSAStringToAddressW() instead of inet_pton()

Undefine _WIN32_WINNT before (re)define it.
Fix if this macro is already declared somewhere.

Just add Winsock, no need to test anymore.

The number of characters must be used.

Use countof() macro instead

Improved fixes for Windows
2019-03-26 12:04:22 +01:00
Tim Rühsen b32e81367c
Merge pull request #125 from nacho/fix-msvc-icu-debug
meson: on msvc icu installs the libraries with a "d" suffix
2019-01-21 13:18:04 +01:00
Ignacio Casal Quinteiro 0bd3ae2a0c meson: on msvc icu installs the libraries with a "d" suffix 2019-01-21 12:20:53 +01:00
Tim Rühsen c6fdf56b20 msvc/*.py: Let main() return 0 2019-01-11 21:59:06 +01:00
Ignacio Casal Quinteiro 726d6773d4 meson: install the library 2018-12-31 17:01:00 +01:00
Ignacio Casal Quinteiro b61285e74d meson: install the header file 2018-12-31 16:57:56 +01:00
Tim Rühsen ad895e382a Silence format warning on printing time_t 2018-12-31 16:51:48 +01:00
Tim Rühsen b38320fd42 Update PSL data from upstream 2018-12-29 19:26:50 +01:00
Tim Rühsen 80c0d5ff22 Add contrib/spell-checker 2018-12-28 19:45:30 +01:00
Tim Rühsen c9315f26bc Fix typos 2018-12-28 19:44:39 +01:00
Chun-wei Fan df873db543 meson: Ensure things build with Windows
We must link to ws2_32.lib for all Windows builds, and we need to use
workarounds for alloca() and snprintf() since Visual Studio uses
_alloca() for alloca() and provides snprintf() only on Visual Studio
2015 or later, meaning that we need to use _snprintf() here.

Also fix the parts where we run the Python scripts to generate items,
since those scripts already have shebang lines for Meson to pick up.
2018-12-11 09:28:51 +08:00
Tim Rühsen 6a47e7c527 Meson: Build libpsl version number without external command 2018-12-10 21:02:33 +01:00
Tim Rühsen c3133c7fd6 Meson: Add ASAN build to contrib/check-hard-meson 2018-12-09 18:30:41 +01:00
Tim Rühsen c8c9c5a391 Meson: Add contrib/check-hard-meson 2018-12-09 14:34:45 +01:00
Tim Rühsen 04da21e2e9 Meson: Include config.h in tests 2018-12-09 14:33:56 +01:00
Tim Rühsen 73b734c364 Fix C89 warnings in fuzz/main.c 2018-12-08 23:38:45 +01:00
Tim Rühsen 6ef129be93 Fix include path in fuzz/main.c 2018-12-08 23:27:28 +01:00
Tim Rühsen 277b82bca2 Fix C++ comment to C 2018-12-08 23:22:06 +01:00
Xavier Claessens a9c3b6df39 Meson: Fallback to find_library if no pkg-config file is found 2018-12-08 14:13:51 -05:00
Xavier Claessens ed5f23d69a Meson: Fix srcdir pointing to root source dir instead of current subdir 2018-12-08 12:49:20 -05:00
Tim Rühsen 22f3ecdb12
Merge pull request #117 from xclaesse/meson-fuzz
Meson: Fix fuzz test names
2018-12-08 16:57:24 +01:00
Xavier Claessens 81a793346e Meson: Fix fuzz test names 2018-12-08 10:11:17 -05:00
Xavier Claessens 13bf8e102e Meson: Fix link of fuzzer tests
libicu is an internal dependency of libpsl, so if an executable
uses libicu directly, linking on libpsl is not enough, it must also
link on libicu.

Closes: #115
2018-12-08 08:12:17 -05:00
Tim Rühsen 77971566d3 Meson: Fix issues in meson.build, remove building fuzz/ 2018-12-08 13:50:36 +01:00
Tim Rühsen 2483cb608d Mention meson and nmake build in README.md 2018-12-08 10:59:07 +01:00
Xavier Claessens 77c10beab7 Add initial Meson build system 2018-12-07 09:54:20 -05:00
Tim Rühsen aae1995ca7
Merge pull request #114 from xclaesse/prepare-for-meson
Prepare for meson
2018-12-06 16:05:03 +01:00
Xavier Claessens aff5cef20e tests: Do not hardcode psl.dafsa and psl_ascii.dafsa file names 2018-12-06 09:11:56 -05:00
Xavier Claessens 1653897a49 Rename suffixes_dafsa.c to suffixes_dafsa.h
It is a generated file that we #include, so it makes more sense to have
.h extension. Also Meson tries to compile that file if it has .c
extension.
2018-12-06 09:11:02 -05:00
Tim Rühsen 808dbaf1f1 Update PSL from upstream 2018-11-06 14:53:07 +01:00
Tim Rühsen 3687c2b97b Revert "Disable build without runtime for Travis"
This reverts commit b6fb9815f6.

Upstream accepted my patches :-)
2018-11-06 13:27:09 +01:00
Tim Rühsen 8fbe3f3130
Merge pull request #111 from rockdaboot/tmp-fix-travis-build
Disable build without runtime for Travis
2018-10-30 13:18:29 +01:00
Tim Rühsen b6fb9815f6 Disable build without runtime for Travis
When/If publicsuffix/list changes to NFC format, this commit
can be reverted.
2018-10-30 12:48:20 +01:00
Tim Rühsen a8b4b68959
Merge pull request #109 from rockdaboot/define-comply-c99
src/psl.c: Use C99 compliant defines
2018-10-29 15:27:45 +01:00
Tim Rühsen 9eedf5f246 Fix code broken by 2nd last patch 2018-10-29 14:56:27 +01:00
Tim Rühsen 1e1b9f5748 src/psl.c: Use C99 compliant defines 2018-10-29 13:17:59 +01:00
Tim Rühsen e21c174e86
Merge pull request #110 from rockdaboot/use-more-c99
Use more C99 compliant file-scope variables
2018-10-29 13:13:47 +01:00
Tim Rühsen 9dba7e4f4c Use more C99 compliant file-scope variables 2018-10-29 12:16:52 +01:00
Tim Rühsen 8ee5c0993f Fix some issues found by lgtm.com 2018-10-15 17:40:29 +02:00
Tim Rühsen 6938d1f513 Add .lgtm.yml 2018-10-15 17:29:23 +02:00
Tim Rühsen ab893d4e23 Rename _PSL_* macros to be POSIX compliant
Reported-by: Marc Lehmann
2018-10-14 20:24:45 +02:00
Tim Rühsen e87d463d04 Remove NLS / gettext from library code 2018-10-13 22:37:00 +02:00
Tim Rühsen e1104c58f3 Remove redundant code in _psl_is_public_suffix()
Reported-by: Marc Lehmann
2018-10-13 22:24:05 +02:00
Tim Rühsen 68b1b0c7e5 Change download link of the PSL 2018-10-13 22:01:36 +02:00
Tim Rühsen 85ccc92281 Change ML archive link 2018-10-13 21:57:41 +02:00
Tim Rühsen 9d064cace8 Update to latest Public Suffix List 2018-10-07 21:15:07 +02:00
Don 02648135c9 Use __has_declspec_attribute for shared builds
Clang compilation targets may support `__declspec` attributes. Because of that it has a `__has_declspec_attribute` [check](https://clang.llvm.org/docs/LanguageExtensions.html#has-declspec-attribute).

Currently libpsl just assumes that `__declspec` is Windows only. This adds a compatibility macro and checks whether `dllimport` and `dllexport` are available.
2018-10-07 11:32:45 +02:00
Tim Rühsen a3bd08fbf2 Add script to build with MinGW 2018-10-05 20:49:21 +02:00
Tim Rühsen d691a3b941 Add Windows compatibility to fopen 2018-10-05 20:47:26 +02:00
Tim Rühsen 523397fb83 psl tool: Fix possible crashes with printf %s, NULL
Reported-by: Mike Schiffman
2018-07-04 09:37:25 +02:00
Tim Rühsen 4c68696eb1 Add -b/--batch to suppress printing the domain 2018-05-18 15:37:53 +02:00
Tim Rühsen b77d54469e Fix and cleanup build 2018-04-30 15:14:17 +02:00
Tim Rühsen b7f5c1414d Release V0.20.2 2018-04-26 20:18:39 +02:00
Tim Rühsen 3695113cf6 Add msvc/ autogen files to .gitignore 2018-04-26 19:27:15 +02:00
Tim Rühsen 5fd44f138b Fix several configure --enable options 2018-04-26 19:20:46 +02:00
Tim Rühsen a6397a0d62 Remove _HIDDEN from src/lookup_string_in_fixed_set.c 2018-04-26 16:45:15 +02:00
Tim Rühsen c93175c32d
Merge pull request #99 from fanc999/build.win
Add NMake Makefiles for Visual Studio builds
2018-04-26 10:46:50 +00:00
Chun-wei Fan 6a5da68ccf Add NMake Makefiles for Visual Studio builds
This adds a set of Visual Studio NMake Makefiles that can be used to
build libpsl, either as a DLL or as a static .lib.  Building of the psl
tool and the tests in tests/ (but not fuzz, since those tests use
fmemopen(), that is not supported on Windows) are also supported.

A simplified "install" target is provided that copies the build
results to a default or set prefix with the pkg-config file so that
other packages (such as libsoup) with build systems that depend on this
.pc file can be built.

These files make use of autotools stuff so that they have the up-to-date
version info upon a release.

Currently, only builds using the ICU runtime/builtin or with no
runtime/builtin are supported with Visual Studio builds.
2018-04-26 17:09:28 +08:00
Tim Rühsen 724fbd0098 Fix make distcheck 2018-04-25 17:06:44 +02:00
Tim Rühsen 7cb146765e Fix when built as a projects subdir 2018-04-25 16:50:42 +02:00
Tim Rühsen 1a59430862 Add LTLIBICONV and LTLIBINTL to libpsl build 2018-04-25 16:00:12 +02:00
Tim Rühsen 64db5a57da Add support for native Win32 builds 2018-04-25 15:56:00 +02:00
Tim Rühsen b220ffb0b9 Fix MinGW cros build on Linux 2018-04-21 11:30:22 +02:00
Tim Rühsen 8595ebb0ce Fix previous commit 2018-04-21 11:26:20 +02:00
Tim Rühsen 1a8e3e01f5 Fallbacks for IDN library detection
When not explictly given --enable-runtime/--enable-builtin,
./configure tries to first detect libidn2, then libicu, then
libidn. If none found, it fals back to --disable-runtime and
--disable-builtin.

Reported-by: Chun-wei Fan
2018-04-20 15:36:36 +02:00
Chun-wei Fan 5f85085d76 src/psl.c: Fix build on Visual Studio
Visual Studio does not come with unistd.h, so only include it when we
have HAVE_UNISTD_H.  It also does not like including locale.h within
main() for some reason, so include that with the other system headers.
2018-04-20 14:46:26 +02:00
Chun-wei Fan 9620b13374 fuzz: Skip the tests if fmemopen() is not found
fmemopen() is a function that is only provided with *NIX systems, so we
ought to check for its presence in order to build and run the tests
in fuzz/ fully, otherwise, we just skip the tests.

Also include headers according to how they are found, and add fallbacks
for Visual Studio that do not have stdint.h yet.
2018-04-20 14:46:02 +02:00
Tim Rühsen 44256b1a3a Make global vars static in test-is-public-all.c 2018-04-19 10:21:25 +02:00
Tim Rühsen 01b6cba78f Remove leading _ from header guard 2018-04-19 10:16:45 +02:00
Tim Rühsen c4a2698e22 Remove unused vars in test-is-public-all.c 2018-04-19 10:15:53 +02:00
Tim Rühsen 6244c731b9 Extend visibility support
Also renamed PSL_PUBLIC -> PSL_API (conforming to other known libraries).
2018-04-19 10:07:10 +02:00
Chun-wei Fan 8e91838fd5 libpsl.h.in: Decorate public symbols for export
Introduce a macro, PSL_PUBLIC, which is defined as nothing by default,
which can be used by the compiler to instruct the linker to export the
public symbols, such as __declspec (dllexport) on Visual Studio.
2018-04-19 09:37:57 +02:00
Chun-wei Fan 859b307593 src/psl.c: Make code compilable on native Windows
Include the Windows/Winsock2 counterparts of the networking headers on
Windows and avoid including *NIX-specific headers on Windows.

Also remove the small bits of C99isms from the code.
2018-04-19 09:37:37 +02:00
Tim Rühsen 84536ee21d
Merge pull request #96 from ebassi/non-srcdir-build
Fix non-srcdir builds
2018-04-06 18:22:59 +00:00
Emmanuele Bassi c9a550e93c Fix non-srcdir builds
We need to modify the various paths we reference to include the build
directory, in the case when the build directory is not the source
directory.
2018-04-06 18:26:07 +01:00
Tim Rühsen 8aba48aa27
Merge pull request #94 from dkg/speling
fix spelling errors
2018-03-23 13:31:26 +00:00
Daniel Kahn Gillmor 2c66c15a99 fix spelling errors 2018-03-23 11:33:55 +00:00
Tim Rühsen fe2042fea8 Fix make distcheck without enabled docs 2018-03-06 10:21:45 +01:00
Tim Rühsen 32f3861216 Don't require xsltproc when not building man pages 2018-03-06 09:40:11 +01:00
Tim Rühsen f7e0d9441a Fix --disable-builtin configure option 2018-03-05 11:25:06 +01:00
Tim Rühsen 6220e02ebd Release V0.20.1 2018-02-26 11:47:27 +01:00
Tim Rühsen e0c1ed0e7a Improve docs for PSL_TYPE_NO_STAR_RULE 2018-02-26 11:45:57 +01:00
Tim Rühsen a9ccf7ab23 Add man page text about effect of --no-star-rule 2018-02-26 11:35:18 +01:00
Tim Rühsen 337e189979 Add --no-star-rule to psl utility 2018-02-26 11:23:01 +01:00
Tim Rühsen 35d96246e7 Release v0.20.1 2018-02-23 14:50:55 +01:00
Tim Rühsen 8fd480584e Fix PSL_TYPE_NO_STAR_RULE and improve test suite
Reported-by: Daniel Kahn Gillmor
2018-02-23 12:09:07 +01:00
Tim Rühsen 9542813a8f Release v0.20.0 2018-02-22 10:04:00 +01:00
Tim Rühsen 43ec750b40 Update copyrights 2018-02-22 10:04:00 +01:00
Tim Rühsen 2e15e597ca Update fuzzer to use PSL_TYPE_NO_STAR_RULE 2018-02-22 09:54:50 +01:00
Tim Rühsen 53326dc098 Update PSL submodule 2018-02-21 20:51:29 +01:00
Tim Rühsen aaacdae977 Add TLDs to (DAFSA) data to allow skipping the star rule 2018-02-21 20:49:26 +01:00
Claudio Saavedra 49707dc3b3 Add tests for PSL_TYPE_NO_STAR_RULE 2018-02-21 20:48:05 +01:00
Claudio Saavedra 9e9341f5b9 psl_is_public_suffix2(): allow checking for suffixes not in the list
Add a PSL_TYPE_NO_STAR_RULE type to check for suffixes without the '*'
rule. This allows checking for suffixes that are not in the PSL.
2018-02-21 17:11:01 +02:00
Tim Rühsen 819486edd1 Remove C99 loop construct 2018-02-21 15:56:58 +01:00
Tim Rühsen 179ca703b2 Limit CPU wasting on large inputs
Large inputs on psl_registrable_domain() and psl_unregistrable_domain()
suffer from a O(N^2) behavior. This change limits N to avoid excessive
CPU usage.

At the same time we limit the fuzz corpora size to 64k which is far more
then we expect any real life domain to be.

Reported-by: OSS-Fuzz
2018-02-13 15:42:17 +01:00
Dagobert Michelsen 27614b6b35 Remove unconditional setting of GCC-specific flag 2018-01-16 15:56:24 +01:00
Tim Rühsen f9f256dbcc Update PSL submodule 2018-01-16 15:32:12 +01:00
Jeffrey Walton 8b9f5c204d Expand defintion of PSL; add ICANN reference 2018-01-05 10:16:11 +01:00
Tim Rühsen 4491895e6c Add missing include alloca.h in fuzz/main.c 2017-11-11 15:18:37 +01:00
Tim Rühsen cb5933d981 Fix printf warnings in fuzz/main.c 2017-11-11 15:10:25 +01:00
Tim Rühsen 9bd2c6ad37 Release v0.19.1 2017-11-09 15:37:26 +01:00
Tim Rühsen 1c44781718 Fix unsigned integer overflow in _mem_is_ascii()
Found by OSS-Fuzz. It has no impact.
2017-11-03 12:10:05 +01:00
Tim Rühsen 16bf63a6bf Update corpora from oss-fuzz 2017-09-15 17:49:19 +02:00
Tim Rühsen 4e51142022 psl_*_count() return -1 if information is not available 2017-09-15 17:14:32 +02:00
Tim Rühsen 192b3f06d2 Add psl_free_string to libpsl-sections.txt 2017-09-15 17:14:32 +02:00
Tim Rühsen c7a48a0bf8 Amend start of comments in lookup_string_in_fixed_set.c 2017-09-15 17:14:32 +02:00
Tim Rühsen a12bd1d2a6 Fix input encoding for python3 2017-09-14 20:25:59 +02:00
Tim Rühsen 964e90a43e Merge pull request #79 from darnir/free_string
Add new function psl_free_string()
2017-08-30 12:23:28 +02:00
Darshit Shah 4d5982ed98 Add new function psl_free_string()
When writing a wrapper around LibPSL in a different language it is
important that libpsl provide functions to free any memory that it
allocates. Without this, it is impossible to correctly free the memory
allocated by psl_str_to_utf8lower() function since in other languages
one may not have access to the same free() call from libc.
2017-08-30 11:07:04 +02:00
Tim Rühsen d6c1051b37 Add m4/ax_check_compile_flag.m4 2017-07-27 11:55:29 +02:00
Tim Rühsen 0ff626f6b6 Update corpora from oss-fuzz 2017-07-27 11:36:18 +02:00
Tim Rühsen 4abb701b2c Improve fuzz/get_all_corpora 2017-07-27 11:29:29 +02:00
Tim Rühsen c1bf52dbc3 Add -fsanitize-address-use-after-scope to --enable-asan if available 2017-07-27 11:29:02 +02:00
Tim Rühsen c10ef083d6 Cleanup .travis_coveralls.sh 2017-07-27 11:07:00 +02:00
Tim Rühsen 05d957ffae Update .travis_coveralls.sh 2017-07-20 16:40:40 +02:00
Tim Rühsen 0d2649684f Release v0.18.0 2017-07-20 14:48:12 +02:00
Tim Rühsen 0e0ddc4cec Update contrib/check-hard 2017-07-20 14:46:55 +02:00
Tim Rühsen 659ee4391e Remove compiler warnings 2017-07-20 11:36:13 +02:00
Tim Rühsen 9ec54644d0 Update PSL submodule 2017-07-19 16:54:35 +02:00
Tim Rühsen 280c130f73 Add fuzz/git_all_corpora 2017-07-19 15:02:03 +02:00
Tim Rühsen bf0f1cda52 Update fuzzer corpora 2017-07-19 15:01:40 +02:00
Tim Rühsen ecc3914172 Fix link libraries for oss-fuzz static building 2017-07-15 12:31:53 +02:00
Tim Rühsen 4306c0cfc0 Fix pkg-config support for libidn and libidn2 2017-07-15 12:31:01 +02:00
Tim Rühsen 0a48980efc Use libidn2 as default 2017-07-14 15:53:31 +02:00
Tim Rühsen 083c8c6801 Fix a few C89 issues 2017-07-14 15:53:31 +02:00
Tim Rühsen 1076ec178c Update internal IDNA corpora from OSS-Fuzz 2017-07-14 15:53:31 +02:00
Tim Rühsen 2cb2266cb1 Update libidn corpora from OSS-Fuzz 2017-07-14 15:53:31 +02:00
Tim Rühsen 5f1176485e Update libidn2 corpora from OSS-Fuzz 2017-07-14 15:53:31 +02:00
Tim Rühsen 9015ae5fc0 Update libicu corpora from OSS-Fuzz 2017-07-14 15:53:31 +02:00
Tim Rühsen 3541a7cb80 Update libicu corpora from OSS-Fuzz 2017-07-13 15:50:04 +02:00
Tim Rühsen a6e4703318 Fix oss-fuzz issue #2600 (buffer overflow in libicu code)
Added a reproducer corpus and fixed the broken libicu code.
The buffer overflow could be triggered by psl_load(), psl_load_fp(),
psl_is_public_suffix(), psl_is_public_suffix2(), psl_unregistrable_domain(),
and psl_registrable_domain().
2017-07-13 15:40:58 +02:00
Tim Rühsen 3f03987897 Fix fuzz/ regression tester building 2017-07-13 15:39:46 +02:00
Tim Rühsen 631362d3d0 Fuzz all types of builds (idn2, idn, icu, none) 2017-07-12 15:37:35 +02:00
Tim Rühsen 2abce22058 Add UBSAN testing for Travis CI 2017-07-09 20:53:56 +02:00
Tim Rühsen 926cc34ade Fix uninitialised value created by stack allocation
Using valgrind testing the fuzz corpora revealed a missing
check in _add_punycode_if_needed() which lead to a
"Uninitialised value was created by a stack allocation".

Thanks to OSS-fuzz for the corpora, thanks valgrind to find this
issue (asan and ubsan didn't find it).
2017-07-09 20:21:55 +02:00
Tim Rühsen d583db99b8 Update oss-fuzz target for static linking 2017-07-08 18:02:52 +02:00
Tim Rühsen 467f035f0a Add one new corpus for libpsl_load_fuzzer 2017-07-08 12:11:50 +02:00
Tim Rühsen 90a1198939 Find static libidn2 via configure 2017-07-06 11:08:20 +02:00
Tim Rühsen 49c9fade7a Fix fuzz/run-clang.sh for CTRL-C 2017-07-02 21:23:10 +02:00
Tim Rühsen f2b9b2447a Remove C++ settings from fuzz/README.md 2017-07-02 21:11:43 +02:00
Tim Rühsen 5e562e6f3b Update fuzzer corpora and scripts 2017-06-30 19:55:55 +02:00
Tim Rühsen 61824edc3d Add oss-fuzz corpora downloader 2017-06-26 19:55:04 +02:00
Tim Rühsen 0cbb7089f6 Enable ASan for Travis CI testing 2017-06-20 21:08:35 +02:00
Tim Rühsen b858480f0d Add --enable-asan --enable-ubsan to contrib/check-hard 2017-06-20 16:44:41 +02:00
Tim Rühsen 492c884d7d Fix memory overflow in LIBICU code of psl_str_to_utf8lower()
Immediately discovered with the new --enable-asan / --enable-ubsan
configure options, thanks to the fuzz corpora.
2017-06-20 16:30:29 +02:00
Tim Rühsen 47734a6f1f Add --enable-asan and --enable-usan configure options 2017-06-20 16:27:18 +02:00
Tim Rühsen e20e6f369b Add clang's CFI instrumentation
Add --enable-cfi ./configure option to enable
LLVM/Clang's Control Flow Integrity for builds.

CFI aborts a program upon detecting certain forms of undefined behavior
that can potentially allow attackers to subvert the program’s control flow.
2017-06-19 17:14:29 +02:00
Tim Rühsen d417badedb New corpora for libpsl_load_fuzzer 2017-06-19 10:49:26 +02:00
Tim Rühsen 81737c9260 New OSS-Fuzz corpora for libpsl_load_dafsa_fuzzer 2017-06-19 10:44:20 +02:00
Tim Rühsen fa69455d07 Add fuzz/ stuff to .gitignore 2017-06-14 14:37:23 +02:00
Tim Rühsen f304dbe324 More fuzzer improvements 2017-06-14 12:48:58 +02:00
Tim Rühsen 43f460d4c5 New OSS-Fuzz corpora for libpsl_fuzzer 2017-06-14 11:40:56 +02:00
Tim Rühsen 44db84e6ec More corpora for libpsl_load_dafsa_fuzzer 2017-06-14 10:00:49 +02:00
Tim Rühsen 666c07b364 New fuzz/libpsl_load_fuzzer.c plus corpus directory 2017-06-14 09:49:07 +02:00
Tim Rühsen 6ca1304bd4 Fix libpsl_load_dafsa_fuzzer for libc6 < 2.22 2017-06-14 09:07:54 +02:00
Tim Rühsen d686c1fff7 Fix memleak in _psl_is_public_suffix() (found by fuzzing) 2017-06-13 22:24:18 +02:00
Tim Rühsen 812e887530 New fuzz corpora 2017-06-13 22:17:09 +02:00
Tim Rühsen 896a5e299a Enhance fuzzing 2017-06-13 22:16:14 +02:00
Tim Rühsen e8f083e867 Add function calls to fuzz/libpsl_fuzzer.c 2017-06-12 17:07:54 +02:00
Tim Rühsen 9da4e94c1a Update fuzz/README.md 2017-06-12 17:07:07 +02:00
Tim Rühsen e584007f42 * src/psl.c (psl_str_to_utf8lower): Fix docs 2017-06-12 17:00:53 +02:00
Tim Rühsen 045bf63031 Fix double free in psl_load_fp(), found by fuzzing 2017-06-09 22:53:19 +02:00
Tim Rühsen bce2e9acb0 fuzz/fuzzer.h: Add extern 'C' 2017-06-09 20:35:46 +02:00
Tim Rühsen f41c6aaf63 Add fuzzing architecture 2017-06-09 16:27:37 +02:00
Tim Rühsen fa2985a535 Update PSL submodule 2017-06-09 11:01:17 +02:00
Tim Rühsen a33feb8ff4 Fix typos found by ka7/misspell_fixer 2017-04-19 11:46:27 +02:00
Tim Rühsen d23cf2bf17 Update PSL submodule 2017-04-19 11:43:11 +02:00
Tim Rühsen 448f6e4564 Fix order of files in psl_latest()
If 'dist_filename' and 'filename' are given and both have the same
age, we want 'dist_filename' (expected DAFSA) being loaded.
2017-02-21 12:18:29 +01:00
Tim Rühsen 8fddb72033 Release v0.17.0 2017-01-16 12:57:25 +01:00
Tim Rühsen 2aa06e17c8 Update PSL submodule 2017-01-16 12:14:37 +01:00
Tim Rühsen eda8276b5f Use NON-TRANSITIONAL toASCII() with libicu 2017-01-16 10:47:21 +01:00
Tim Rühsen 26d0856d0a Fix typo 2017-01-16 10:26:12 +01:00
Tim Rühsen 526768cc5d Use TR46 non-transitional with libidn2 >= 0.14
I changed my mind after talking with the cURL
maintainer Daniel Stenberg.
See https://github.com/curl/curl/pull/1207
2017-01-14 15:47:44 +01:00
Tim Rühsen 71059c2c6d Work around https://github.com/eddyxu/cpp-coveralls/issues/108 2017-01-04 20:25:38 +01:00
Tim Rühsen 1a06a6c888 Add coveralls token to .travis_coveralls.sh 2017-01-04 17:19:05 +01:00
Tim Rühsen 31684dc7fd Add psl_latest() and psl_dist_filename() to tests 2017-01-04 17:10:57 +01:00
Tim Rühsen a3f6134a5e Amend .travis_coveralls.sh 2017-01-04 16:54:09 +01:00
Tim Rühsen 2c17d56234 Use TR46 transitional with libidn2 >= 0.14 2017-01-03 12:30:43 +01:00
Tim Rühsen 7d5f3a67af Release v0.16.1 2016-12-15 13:01:45 +01:00
Tim Rühsen d4ea513f27 Add --use-latest-psl to tools/psl, now default
Fixes #76
2016-12-15 12:51:35 +01:00
Tim Rühsen 3c3320180e Release v0.16.0 2016-12-10 18:06:44 +01:00
Zhang Hai 00f628d802 s/Publix/Public/ 2016-12-07 16:14:04 +08:00
Tim Rühsen ff29f13d8f Add functions psl_latest() and psl_dist_filename()
Also add a new ./configure function to set a distribution wide
PSL file used by psl_latest(): --with-psl-distfile
If possible that filename should point to a DAFSA PSL file that
becomes updated regularly.
2016-12-06 20:16:12 +01:00
Tim Rühsen deabd4a546 Replace psl2c by psl-make-dafsa
Removed --input-format from psl-make-dafsa.
Added --output-format=cxx+ to psl-make-dafsa.
Removed psl2c.
2016-12-06 15:22:18 +01:00
Tim Rühsen 6490b8214b Don't taint out variable on error in psl_str_to_utf8lower()
Fixes #71
2016-12-05 16:28:47 +01:00
Tim Rühsen b9e04d6958 Update copyright year 2016-12-05 15:03:27 +01:00
Tim Rühsen 2a3a743643 Fix typo Publix -> Public 2016-12-05 15:01:27 +01:00
Frederic Cambus 9f0b09e830 Missing includes for in6_addr / AF_INET*, fixes compilation on OpenBSD 2016-12-02 19:11:18 +01:00
Tim Rühsen 65e785e1ca Replace NFCK -> NFKC in the docs 2016-11-29 14:49:35 +01:00
Tim Rühsen d83bc6d523 Release v0.15.0 2016-11-14 12:50:48 +01:00
Tim Rühsen 3909351697 Update PSL submodule 2016-11-14 12:36:16 +01:00
Tim Rühsen 5d32b80077 Make API docs more detailed 2016-11-14 12:08:20 +01:00
Olle Liljenzin 3f276c7d1e Fix psl-make-dafsa to work with python3 2016-11-12 21:21:28 +01:00
Tim Rühsen 1b36fb0012 Remove -Wall from automake options to reduce verbosity 2016-11-10 11:56:19 +01:00
Tim Rühsen f01a81472b Fix list email address in README.md 2016-11-10 11:05:00 +01:00
Tim Rühsen 761d938d2a Fix name of Olle Liljenzin in src/psl-make-dafsa.1 2016-11-06 22:47:33 +01:00
Olle Liljenzin 3a4dff8805 Fixed documentation and error message to match the actual code. 2016-11-06 16:26:44 +01:00
Tim Rühsen 514aa0163c Add Daurnimator to AUTHORS 2016-11-06 16:22:52 +01:00
Tim Rühsen 2c871b1306 Skip conversion in _psl_is_public_suffix() for builtin psl context 2016-11-06 11:59:36 +01:00
Tim Rühsen 44e6bd4eb8 src/psl2c.c: Also include UTF-8 into DAFSA output 2016-11-06 11:30:20 +01:00
Tim Rühsen 2677621b62 Add test DAFSA files to .gitignore 2016-11-06 11:30:20 +01:00
Tim Rühsen 70661c6807 Add checking with ASCII DAFSA in test-is-public-all.c 2016-11-06 11:30:20 +01:00
Tim Rühsen 3211a66f00 Put punycode + UTF-8 rules into DAFSA in utf-8 mode 2016-11-06 11:30:20 +01:00
Tim Rühsen e30e77ef12 Create ASCII and UTF-8 binaries for testing 2016-11-05 11:17:11 +01:00
Tim Rühsen 3ac807d987 Add --encoding to psl-make-dafsa man page 2016-11-05 10:37:01 +01:00
Tim Rühsen 4b42762cbf Skip punycode conversion for _psl_is_public_suffix() if data contains UTF-8 rules 2016-11-05 10:37:01 +01:00
Olle Liljenzin 86034ac7c9 Added function to the parser for reading DAFSA encoding mode. 2016-11-05 10:37:01 +01:00
Olle Liljenzin 8c2bcd5a24 Added version info into generated DAFSA.
psl-make-dafsa got a mode switch so that the old version can be
generated for testing.
2016-11-05 10:01:54 +01:00
Olle Liljenzin e03953e27a Updated DAFSA generator and parser to support UTF-8 encoding 2016-11-05 10:01:54 +01:00
Tim Rühsen e126a67354 Add Solaris OpenCSW badges 2016-11-04 14:28:30 +01:00
Tim Rühsen 8a62205004 Add Travis script for Coveralls.io 2016-09-28 12:31:49 +02:00
Tim Rühsen 598a78b2de Add better test code coverage 2016-09-26 15:15:34 +02:00
Tim Rühsen 5ebc24f0e0 Code cleanup in libidn2 branch of _psl_idna_toASCII()
Reported-by: https://github.com/daurnimator
2016-09-26 10:13:43 +02:00
Tim Rühsen 7eb8592035 Let u8_tolower() allocate the result buffer.
Reported-by: https://github.com/daurnimator
2016-09-25 19:44:33 +02:00
Tim Rühsen 32543dd5a5 Avoid unneeded memory allocactions in psl_str_to_utf8lower()
Reported-by: https://github.com/daurnimator
2016-09-25 12:49:56 +02:00
Tim Rühsen 069c6ff091 Fix check-coverage target (CFLAGS+LDFLAGS not set properly)
Reported-by: https://github.com/darnir
2016-09-25 12:48:32 +02:00
Tim Rühsen 9b2d7b7a9c Add gcov test coverage report 2016-09-23 17:13:32 +02:00
Tim Rühsen 1baaacccd5 Fix libidn/libidn2 code path of psl_str_to_utf8lower()
* fixing memory leaks
* proper handling of unterminated results of u8_tolower()
* second call to iconv() ensures flush of internal memory
* check more code paths of psl_str_to_utf8lower() via
  tests/test-registrable-domain.c
2016-09-23 12:35:08 +02:00
Tim Rühsen e2812e8c4c Check return value for strdup and strndup
Fixes #60
Reported-by: https://github.com/daurnimator
2016-09-22 15:53:31 +02:00
Tim Rühsen 351b3fb912 Remove redundant define of countof() 2016-09-22 11:37:23 +02:00
Tim Rühsen 9e1ca81be4 Remove memory allocations from _utf8_to_utf32()
Reported-by: https://github.com/daurnimator
2016-09-22 11:19:52 +02:00
Tim Rühsen b2fcafcfdd Add 'make clean' to contrib/check-hard 2016-09-22 11:19:30 +02:00
Tim Rühsen 0264454ea2 Add Coverity badge 2016-09-21 12:39:40 +02:00
Tim Rühsen 6cfb33e530 Amend API docs to be more precise about invalid input.
Fixes #59
Reported-by: https://github.com/daurnimator
2016-09-21 12:03:00 +02:00
Tim Rühsen 10f7b5fe7c Fallback to malloc from alloca for larger memory chunks
Fixes #58
Reported-by: https://github.com/daurnimator
2016-09-21 11:54:39 +02:00
Tim Rühsen 1ab7be5641 Check malloc/realloc results in src/psl.c
Fixes #57
Reported-by: https://github.com/daurnimator
2016-09-21 11:15:43 +02:00
Tim Rühsen 01d3f53321 Add src/suffixes_dafsa.c to .gitignore 2016-09-21 10:00:22 +02:00
Tim Rühsen c9de2de301 Fix error msg when libunistring is missing 2016-09-17 14:46:09 +02:00
Dagobert Michelsen 7983f86820 Use proper library path and libs for ICU 2016-09-17 14:46:06 +02:00
Tim Rühsen 126d2dca9c Package and install psl.1 and psl-make-dafsa.1
Fixes #53
Reported-by: https://github.com/yselkowitz
2016-09-17 14:46:00 +02:00
Tim Rühsen ce8e24c509 Merge pull request #56 from SteelPangolin/master
Change src/psl-make-dafsa shebang so it'll run on OS X
2016-09-17 11:46:15 +02:00
Jeremy Ehrhardt 003dec4203 Change src/psl-make-dafsa shebang so it'll run on OS X 2016-09-16 18:42:54 -07:00
Tim Rühsen 0486815dbd Release v0.14.0 2016-07-30 13:53:17 +02:00
Tim Rühsen 23ae4a2ba1 Update PSL submodule 2016-07-30 13:42:46 +02:00
Tim Rühsen 583cf6ae56 Merge pull request #52 from dkg/dafsa-reorg
rename src/make_dafsa.py to src/psl-make-dafsa, add documentation
2016-07-14 12:05:13 +02:00
Daniel Kahn Gillmor dc7bf5bbae rename src/make_dafsa.py to src/psl-make-dafsa, add documentation
I've talked to the good people on #debian-bootstrap who would be most
affected by the possible build-dep cycle, and i think the simplest
approach is actually to split out make_dafsa.py into its own
architecture-independent package.

I'm thinking i'll call the package psl-make-dafsa, and in the course of
shipping it, i'll place src/make_dafsa.py as /usr/bin/psl-make-dafsa.

This is because:

 * debian discourages scripts on the $PATH from having language-specific
   suffixes like .py:

    https://lintian.debian.org/tags/script-with-language-extension.html

 * "-" appears to be a more common delimiter in command names than "_":

    0 dkg@alice:~$ for x in - _; do printf "%s: %d " "$x" $(ls -1 ${PATH//:/ } | grep -c "$x"); done; echo
    -: 1235 _: 368
    0 dkg@alice:~$

 * i'd prefer to prefix the command with "psl-" since it really is
   producing and interpreting PSL-specific data structures.

Accepting this patch would mean i'd have fewer changes to make in the
debian packaging, and would allow other distributors to take a similar
approach if they want to.
2016-07-14 11:55:04 +02:00
Tim Rühsen 8dba092c73 Add magic header to DAFSA binary files 2016-07-13 11:14:18 +02:00
Tim Rühsen 852931571f Fixed invocation of make_dafsa.py in psl2c.c 2016-07-13 11:13:04 +02:00
Tim Rühsen 972cb789dd Update PSL submodule 2016-07-13 10:49:22 +02:00
Tim Rühsen 10fe88b22b Amend formatting of psl man page 2016-07-06 15:32:51 +02:00
Tim Rühsen 45cce63142 Remove auto-generated ABOUT-NLS from git 2016-07-06 15:32:51 +02:00
Daniel Kahn Gillmor a7c2c16585 add a man page 2016-07-06 15:32:51 +02:00
Daniel Kahn Gillmor dc9cc02982 s/publix/public/ 2016-07-06 15:32:51 +02:00
Daniel Kahn Gillmor 248327e4aa use https where possible 2016-07-06 15:32:51 +02:00
Tim Rühsen 514488d9b3 Remove unneeded libraries from tools/psl link step 2016-07-06 13:10:16 +02:00
Tim Rühsen a6ed60e57f Release v0.13.0 2016-03-02 12:25:35 +01:00
Tim Rühsen be9b3fd3f4 Merge remote-tracking branch 'origin/newfmt' into develop 2016-03-02 11:55:17 +01:00
Tim Rühsen f097290c90 Remove python linter
It has been merged into https://github.com/publicsuffix/list
and can be found in list/linter.
2016-03-02 11:45:28 +01:00
Tim Rühsen efdd800cb3 Use tests.txt as PSL test file by default 2016-02-20 23:06:30 +01:00
Tim Rühsen cb3fc3b502 Travis CI only for branches develop and master 2016-02-20 23:05:34 +01:00
Tim Rühsen 4798ee88a2 Add test suite support for list/tests/tests.txt 2016-02-20 15:06:48 +01:00
Tim Rühsen 2914afa8c7 New linter/ dir with pslint.py selftest 2016-02-18 16:40:06 +01:00
Tim Rühsen 811513f17e Print message and exit when no suffixes are found 2016-02-12 12:27:25 +01:00
Tim Rühsen d19c46c003 Make a few enhancements to pslint 2016-02-08 14:11:52 +01:00
Tim Rühsen 36609787d5 Fix python3 UTF-8 runtime error and section detection 2016-02-08 09:40:43 +01:00
Tim Rühsen 568394438d Add disabled code for 'Group Order' checking
The check has been disabled since it turned out that those
'groupings' of PSL entries are not really ordered in the way
(# of labels, TLD, sublabel#1, sublabel#2, ...)

This commit also fixes section detection / verification
2016-02-05 12:16:50 +01:00
Tim Rühsen 0e2da86eeb Update PSL submodule 2016-02-02 22:52:29 +01:00
Tim Rühsen aa028e606b Adjust text in doublette comment in src/pslint.py 2016-02-02 22:49:02 +01:00
Tim Rühsen a46af675b4 Fix indentation multi-line comment in src/pslint.py 2016-02-02 22:41:18 +01:00
Tim Rühsen bd70c79c18 Indent src/pslint.py with tabs 2016-02-02 22:20:58 +01:00
Tim Rühsen 98aed19c3a Convert copyright line to UTF-8 in pslint.py 2016-02-02 19:59:45 +01:00
Tim Rühsen 3ba8903915 Add PSL linter written in Python 2016-02-02 16:43:03 +01:00
Tim Rühsen 8c39291f55 Slightly shorter DAFSA array when sorting input 2016-01-05 10:57:07 +01:00
Tim Rühsen 1bd9347af9 Fix for commit fd928da46e 2016-01-04 22:15:43 +01:00
Tim Rühsen f86094972a Check for python 2.7+ in configure.ac 2016-01-04 20:41:24 +01:00
Tim Rühsen fd928da46e Fix python3 incompatibilities in make_dafsa.py 2016-01-04 20:22:13 +01:00
Tim Rühsen c69a18ff9b Release v0.12.0 2016-01-02 14:38:37 +01:00
Tim Rühsen 5bd2db75d2 Release v0.12.0 2016-01-02 14:25:44 +01:00
Tim Rühsen 95a5152e56 Update copyright year to 2016 2016-01-02 13:36:49 +01:00
Tim Rühsen 96e0848d81 Release unused memory after loading DAFSA data 2016-01-02 13:31:53 +01:00
Tim Rühsen f829668e0d Let autogen.sh work on Solaris and without gtkdocize 2016-01-02 13:01:19 +01:00
Tim Rühsen 9f420e2d91 Add tests against the built-in PSL to test-is-public-all.c 2016-01-01 23:26:44 +01:00
Tim Rühsen 748e3ae9cc Load DAFSA precompiled files (auto-detection) 2016-01-01 22:38:21 +01:00
Tim Rühsen 7a859d11c8 Add PSL_TYPE* defines to docs 2016-01-01 22:35:35 +01:00
Tim Rühsen 5c159a3a09 Add few more tests to test-is-public-builtin.c 2016-01-01 22:34:55 +01:00
Tim Rühsen 1604cb3dca Fix make_dafsa.py to generate 4 bit return values 2016-01-01 22:32:11 +01:00
Tim Rühsen 23345f5f37 Convert lookup_string_in_fixed_set.c into UTF-8 2016-01-01 22:31:01 +01:00
Tim Rühsen c9d76e4898 Remove unused variable source_date_epoch 2016-01-01 17:20:30 +01:00
Tim Rühsen cde5e53ea6 Remove psl_builtin_compile_time() for reproducable builds 2016-01-01 15:44:24 +01:00
Tim Rühsen c699e3c441 Add --input-format and --output-format to make_dafsa.py 2015-12-30 17:52:48 +01:00
Tim Rühsen 355edc152f Fix for previous commit 2015-12-29 17:20:28 +01:00
Tim Rühsen 82e9445493 Add psl2c --binary to create DAFSA binary file from PSL 2015-12-29 16:53:47 +01:00
Tim Rühsen e63ff8abfc Avoid libicu dependency with --enable-runtime=no 2015-12-26 14:44:57 +01:00
Tim Rühsen 5363290cbe Remove debugging printf 2015-12-26 14:29:10 +01:00
Tim Rühsen 093d5eac3d Fix ./configure --disable-runtime
Added runtime punycode generation code from
  http://www.nicemice.net/idn/punycode-spec.gz
2015-12-26 14:15:08 +01:00
Tim Rühsen 620e62348a Revert "Remove ./configure --disable-runtime"
This reverts commit cd5a11b7a5.
2015-12-25 19:40:24 +01:00
Tim Rühsen cd5a11b7a5 Remove ./configure --disable-runtime
With DAFSA as builtin data, we need runtime conversions.
2015-12-15 21:21:28 +01:00
Tim Rühsen e252af877f Fix ./configure --disable-builtin 2015-12-15 20:46:25 +01:00
Tim Rühsen c6048b0cdb Merge commit 'df053f91cca36f47d4b994938dbf2bf4f84b6e8e' into develop 2015-12-15 20:19:28 +01:00
Tim Rühsen d9a36e4d58 Merge pull request #39 from dkg/develop
re-fix psl_builtin_outdated()
2015-12-12 23:40:22 +01:00
Daniel Kahn Gillmor 01a3751524 re-fix psl_builtin_outdated() 2015-12-11 22:59:15 -05:00
Darshit Shah df053f91cc Faster, more concurrent travis builds 2015-12-10 23:16:37 +01:00
Tim Rühsen 0ca3741df6 Use DAWG/DAFSA format for builtin data
This data representation reduces the size of the PSL data
drastically and still allows fast lookups.
2015-12-09 09:35:04 +01:00
Tim Rühsen 36139b601d Merge branch 'develop' into dafsa 2015-12-07 10:33:44 +01:00
Tim Rühsen 9d2e93f0b8 New function psl_is_public_suffix2()
The current PSL has two sections, ICANN and PRIVATE.
This new function allows to limit the check for one or both
of these sections.
2015-12-06 21:55:56 +01:00
Tim Rühsen 883e67f008 Create src/suffixes_dafsa.c with DAFSA C array 2015-12-04 21:26:30 +01:00
Tim Rühsen 375aef05ae Add tools/make_dafsa.py 2015-12-04 17:15:55 +01:00
Tim Rühsen aa0593460c Remove .travis.yml from branch 2015-12-04 17:15:03 +01:00
Tim Rühsen da94ada284 Merge pull request #37 from darnir/darnir
Fix Shebang line and other issues with autogen.sh

The shebang line was not being interpreted due to a space between # and !. On
fixing it, smaller issues came to the fore due to the effects of the -e switch.
Fixed them all.
2015-12-03 10:21:08 +01:00
Darshit Shah 7898c22a06 Fix Shebang line and other issues with autogen.sh
The shebang line was not being interpreted due to a space between # and !. On
fixing it, smaller issues came to the fore due to the effects of the -e switch.
Fixed them all.
2015-12-03 10:10:25 +01:00
Tim Rühsen 519b8c9d17 Add time measurement for test-is-public-all.c 2015-12-03 10:08:04 +01:00
Tim Rühsen d14ada235c Use echo without -n in configure.ac (compatibility) 2015-11-19 12:11:27 +01:00
Tim Rühsen b53273d406 Use absolute PSL path to make psl_builtin_outdated() work reliable 2015-11-19 11:18:17 +01:00
Tim Rühsen 583f97f2bf Copy custom PSL file and test file into distribution tarball 2015-11-19 11:16:24 +01:00
Tim Rühsen f3b479fd45 Remove -DDATADIR from DEFS in tests/Makefile.am 2015-11-19 11:14:31 +01:00
Tim Rühsen 71a3f76414 Update publixsuffix/list submodule 2015-11-19 10:06:30 +01:00
Tim Rühsen dbefdb6767 Remove include of bits/stat.h 2015-11-19 10:06:04 +01:00
Christopher Meng db9a3613d6 Correct the present year 2015-10-07 13:28:02 +02:00
Tim Rühsen ac40a6bfc8 Extend tools/psl --print-info and --version 2015-09-27 19:14:58 +02:00
Tim Rühsen 643e523f09 Fix psl_builtin_outdated() 2015-09-27 19:14:13 +02:00
Tim Rühsen d1dec3d16e Release v0.11.0 2015-09-23 14:52:04 +02:00
Tim Rühsen 53c2fe31a8 Update copyright years 2015-09-23 14:50:01 +02:00
Tim Rühsen f1954e5311 Add gtk-doc.m4 and include/libpsl.h to .gitignore 2015-09-23 14:16:52 +02:00
Tim Rühsen a9524b5570 Add PSL_VERSION defines and psl_check_version_number() to docs 2015-09-23 14:15:23 +02:00
Tim Rühsen 00b9cfb119 Add function psl_check_version_number() 2015-09-23 14:04:17 +02:00
Tim Rühsen eabf39c174 Add version numbers to libpsl.h 2015-09-23 12:38:34 +02:00
Tim Rühsen 789d7ad132 Release v0.10.0 2015-09-19 14:12:49 +02:00
Tim Rühsen 3bf7275a17 Add psl_suffix_wildcard_count() to docs 2015-09-19 14:02:49 +02:00
Tim Rühsen 6a8f33ee39 Add new function psl_builtin_outdated() 2015-09-19 14:00:49 +02:00
Tim Rühsen 34289fa59b Add function psl_suffix_wildcard_count() 2015-09-19 10:55:09 +02:00
Tim Rühsen e443d21b61 Code cleanup, faster lookups 2015-09-19 10:50:00 +02:00
Tim Rühsen fb942952ec Release v0.9.0 2015-09-15 15:17:20 +02:00
Tim Rühsen 4c8d3c4d73 Remove unused variable from tests/test-is-public-all.c 2015-09-15 15:16:04 +02:00
Tim Rühsen a9c71d6de5 Update submodule list 2015-09-15 14:50:58 +02:00
Tim Rühsen 597709cb11 Support combination of foo.bar and *.foo.bar 2015-09-15 14:49:53 +02:00
Tim Rühsen fa55bb75c4 Removed test with wrong assumptions 2015-09-15 14:46:07 +02:00
Tim Rühsen 0652af910e Fix SIGSEGV in test-is-public-all.c for TLD exceptions 2015-09-15 12:19:18 +02:00
Tim Rühsen f6a3b96f91 Check PSL entries before generating built-in data 2015-09-15 11:46:21 +02:00
2301 changed files with 108600 additions and 2348 deletions

1
.github/FUNDING.yml vendored Normal file
View File

@ -0,0 +1 @@
github: rockdaboot

53
.github/workflows/codeql.yml vendored Normal file
View File

@ -0,0 +1,53 @@
name: "CodeQL"
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
schedule:
- cron: "49 4 * * 0"
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ python, cpp ]
steps:
- name: Checkout
uses: actions/checkout@v3
with:
submodules: recursive
- name: Install Packages (cpp)
if: ${{ matrix.language == 'cpp' }}
run: |
sudo apt-get update
sudo apt-get install --yes lzip autopoint
- name: Configure (cpp)
if: ${{ matrix.language == 'cpp' }}
run: ./autogen.sh && ./configure --disable-gtk-doc
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
queries: +security-and-quality
- name: Autobuild
uses: github/codeql-action/autobuild@v2
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{ matrix.language }}"

26
.gitignore vendored
View File

@ -1,14 +1,19 @@
*.gz
*.o
*.lo
*.la
*.exe
*.gcda
*.gcno
*.gcov
*.gz
*.la
*.lo
*.log
*.o
*~
*/.deps
*/.libs
*.cache
*.plist
*.stamp
ABOUT-NLS
aclocal.m4
ar-lib
autom4te.cache/
@ -38,12 +43,22 @@ docs/libpsl/libpsl.prerequisites
docs/libpsl/libpsl.signals
docs/libpsl/version.xml
docs/libpsl/xml/
fuzz/*.log
fuzz/*.trs
fuzz/*_fuzzer
fuzz/*_fuzzer.new
gtk-doc.m4
gtk-doc.make
include/libpsl.h
install-sh
lcov/
libpsl.info
libpsl.pc
libtool
ltmain.sh
m4/
msvc/config-msvc.mak
msvc/config.h.win32
Makefile
Makefile.in
missing
@ -64,10 +79,13 @@ po/remove-potcdate.sed
po/stamp-po
src/psl2c
src/suffixes.c
src/suffixes_dafsa.h
stamp-h1
test-driver
tests/*.log
tests/*.trs
tests/psl.dafsa
tests/psl_ascii.dafsa
tests/test-is-cookie-domain-acceptable
tests/test-is-public
tests/test-is-public-all

7
.lgtm.yml Normal file
View File

@ -0,0 +1,7 @@
extraction:
cpp:
prepare:
packages:
- lzip
configure:
command: ./autogen.sh && ./configure --disable-gtk-doc

View File

@ -1,29 +1,59 @@
sudo: false
dist: xenial
language: c
compiler:
- gcc
- clang
# Change this to your needs
- clang-3.8
branches:
only:
- master
- coveralls-text
env:
- RUNTIME=libicu
- RUNTIME=libidn2
- RUNTIME=libidn
- RUNTIME=no
addons:
apt:
packages:
- automake
- autoconf
- autopoint
- libtool
- gtk-doc-tools
- gettext
- libidn11-dev
- libidn2-0-dev
- libicu-dev
- libunistring-dev
- lcov
- clang-3.8
- llvm-3.8-dev
- llvm-dev
- valgrind
script:
- set -e
- sed -i 's/dist-lzip//g' configure.ac
- ./autogen.sh
- ./configure && make -j4 && make check -j4
- ./configure --enable-runtime=libicu --enable-builtin=libicu && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libicu --enable-builtin=libidn2 && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libicu --enable-builtin=libidn && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libicu --disable-builtin && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libidn2 --enable-builtin=libicu && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libidn2 --enable-builtin=libidn2 && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libidn2 --enable-builtin=libidn && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libidn2 --disable-builtin && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libidn --enable-builtin=libicu && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libidn --enable-builtin=libidn2 && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libidn --enable-builtin=libidn && make clean && make -j4 && make check -j4
- ./configure --enable-runtime=libidn --disable-builtin && make clean && make -j4 && make check -j4
- ./configure --disable-runtime --enable-builtin=libicu && make clean && make -j4 && make check -j4
- ./configure --disable-runtime --enable-builtin=libidn2 && make clean && make -j4 && make check -j4
- ./configure --disable-runtime --enable-builtin=libidn && make clean && make -j4 && make check -j4
- ./configure --disable-runtime --disable-builtin && make clean && make -j4 && make check -j4
- ./configure --enable-gtk-doc && make -j4 && make check -j4
# CFI doesn't work out on Ubuntu 14.04 (Trusty). They don't include LLVMgold.so for clang-3.8.
# clang-3.4 doesn't know CFI yet.
# The only chance with Travis-CI is to switch to docker and use Ubuntu 16.04 or whatever.
# - if [[ $CC == "gcc" ]]; then CFI=""; else CFI="--enable-cfi"; fi
- if [[ $CC == "clang-3.8" ]]; then UBSAN="--enable-ubsan"; fi
- ./configure $CFI --enable-asan $UBSAN && make -j4 && make check -j4
- ./configure $CFI --enable-runtime=$RUNTIME --enable-builtin=libicu --enable-asan $UBSAN && make clean && make -j4 && make check -j4
- ./configure $CFI --enable-runtime=$RUNTIME --enable-builtin=libidn2 --enable-asan $UBSAN && make clean && make -j4 && make check -j4
- ./configure $CFI --enable-runtime=$RUNTIME --enable-builtin=libidn --enable-asan $UBSAN && make clean && make -j4 && make check -j4
- ./configure $CFI --enable-runtime=$RUNTIME --disable-builtin --enable-asan $UBSAN && make clean && make -j4 && make check -j4
# libidn2 0.9.0 on Trusty has bugs, valgrinding the fuzz corpora fails
# so we use libicu here
- if [[ $RUNTIME == "libicu" ]]; then VALGRIND="--enable-valgrind-tests"; fi
- ./configure --enable-runtime=$RUNTIME --enable-builtin=libicu --enable-gtk-doc $VALGRIND && make clean && make -j4 && make check -j4 || cat fuzz/*_fuzzer.log
- make distcheck
before_install:
- sudo apt-get -qq update
- sudo apt-get -q install autoconf automake autopoint libtool gtk-doc-tools gettext libidn11 libidn11-dev libidn2-0 libidn2-0-dev libicu48 libicu-dev libunistring0 libunistring-dev
- if [[ $CC == "gcc" && $RUNTIME == "libicu" ]]; then ./.travis_coveralls.sh; fi

6
.travis_coveralls.sh Executable file
View File

@ -0,0 +1,6 @@
#!/bin/bash
make check-coverage-libicu
pip install --user cpp-coveralls
coveralls --include src/

1282
ABOUT-NLS

File diff suppressed because it is too large Load Diff

View File

@ -15,3 +15,10 @@ Dagobert Michelsen (Fixed Solaris building)
Christopher Meng (Fedora building)
Jakub Čajka
Giuseppe Scrivano
Ryan Sleevi (Discussion, Requested DAFSA format and ICANN/PRIVATE support)
Daurnimator (Code review, discussion, reports)
Olle Liljenzin (Original DAFSA implementation and UTF-8 patch)
Claudio Saveedra (Add support for PSL_TYPE_NO_STAR_RULE)
Chun-wei Fan (Add NMake files)
Xavier Claessens (Add Meson build system)
Ignacio Casal Quinteiro (Improve Meson build)

View File

@ -1,4 +1,4 @@
Copyright (C) 2014 Tim Rühsen
Copyright (C) 2014-2022 Tim Rühsen
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),

View File

@ -1,4 +1,4 @@
Copyright (C) 2014 Tim Rühsen
Copyright (C) 2014-2022 Tim Rühsen
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),

View File

@ -1,6 +1,14 @@
# got some hints from https://gitorious.org/openismus-playground/examplelib/source
SUBDIRS = po include src tools $(LIBPSL_DOCS) tests
SUBDIRS = po include src tools
if ENABLE_GTK_DOC
SUBDIRS += docs/libpsl
else
if ENABLE_MAN
SUBDIRS += docs/libpsl
endif
endif
SUBDIRS += fuzz tests msvc
ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
@ -14,4 +22,46 @@ ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = libpsl.pc
EXTRA_DIST = config.rpath LICENSE list/public_suffix_list.dat list/tests/test_psl.txt
EXTRA_DIST = build-aux/config.rpath LICENSE meson.build meson_options.txt
dist-hook:
mkdir -p $(distdir)/list/tests
cp -p $(PSL_FILE) $(distdir)/list
cp -p $(PSL_TESTFILE) $(distdir)/list/tests
clean-local:
rm -rf */*.gc?? */*/*.gc?? libpsl.info lcov
LCOV_INFO=libpsl.info
check-coverage: clean
if test -z "$(XLIB)"; then \
CFLAGS=$$CFLAGS" --coverage -O0" LDFLAGS=$$LDFLAGS" --coverage" ./configure --disable-runtime --disable-builtin; \
else \
CFLAGS=$$CFLAGS" --coverage -O0" LDFLAGS=$$LDFLAGS" --coverage" ./configure --enable-runtime=$(XLIB) --enable-builtin=$(XLIB); \
fi
$(MAKE)
lcov --capture --initial --directory src --output-file $(LCOV_INFO)
$(MAKE) check
lcov --capture --directory src --output-file $(LCOV_INFO)
genhtml --prefix . $(LCOV_INFO) --legend --title "libpsl" --output-directory=lcov
@echo
@echo "You can now view the coverage report with 'xdg-open lcov/index.html'"
check-coverage-libidn:
XLIB=libidn $(MAKE) check-coverage
check-coverage-libidn2:
XLIB=libidn2 $(MAKE) check-coverage
check-coverage-libicu:
XLIB=libicu $(MAKE) check-coverage
fuzz-coverage: clean
$(MAKE) -C src CFLAGS="$(CFLAGS) --coverage" LDFLAGS="$(LDFLAGS) --coverage"
$(MAKE) -C fuzz fuzz-coverage CFLAGS="$(CFLAGS) --coverage" LDFLAGS="$(LDFLAGS) --coverage"
lcov --capture --initial --directory src --directory fuzz --output-file $(LCOV_INFO)
lcov --capture --directory src --directory fuzz --output-file $(LCOV_INFO)
# lcov --remove $(LCOV_INFO) '*/test_linking.c' '*/css_tokenizer.lex' '*/<stdout>' -o $(LCOV_INFO)
genhtml --prefix . --ignore-errors source $(LCOV_INFO) --legend --title "libpsl-fuzz" --output-directory=lcov
@echo
@echo "You can now view the coverage report with 'xdg-open lcov/index.html'"

35
Makefile.os4 Normal file
View File

@ -0,0 +1,35 @@
USE_CLIB2=YES
ifeq ($(USE_CLIB2), YES)
LIBC=clib2
else
LIBC=newlib
endif
all: build
init:
# git submodule update --init
sh ./autogen.sh
autoreconf --install --force --symlink
build: init
CC="ppc-amigaos-gcc" CFLAGS="-mcrt=${LIBC} -fPIC" CXXFLAGS="-mcrt=${LIBC} -fPIC" LDFlags="-mcrt=${LIBC}" LIBS="-lpthread -athread=native" ./configure --host=ppc-amigaos --enable-shared=no --enable-builtin --disable-runtime && \
make -j$(shell nproc)
clean:
@make -f Makefile clean
@rm -rf autom4te.cache .deps
@rm config.log config.status configure libtool Makefile Makefile.in *.m4
release:
@mkdir -p release/local/common/include
@mkdir -p release/local/clib2/lib
# @mkdir -p release/local/newlib/lib
@make -f Makefile.os4 USE_CLIB2=YES
@cp ./src/.libs/libpsl.a ./src/.libs/libpsl.lai ./src/libpsl.la release/local/clib2/lib/
@cp ./include/libpsl.h release/local/common/include/
# @make -f Makefile.os4 clean
# @make -f Makefile.os4 USE_CLIB2=NO
# @cp ./src/.libs/libpsl.a ./src/.libs/libpsl.lai ./src/libpsl.la release/local/newlib/lib/
@lha -aeq libpsl.lha release/

139
NEWS
View File

@ -1,4 +1,141 @@
Copyright (C) 2014 Tim Rühsen
Copyright (C) 2014-2022 Tim Rühsen
26.12.2022 Release V0.21.2
* Increased internal label size
* Meson build improvements (needs meson >= 0.60.0)
* Autoconf build improvements
* Add instructions on how to build from tarball
* Add WSAStartup() for Windows psl tool and tests
* Bump gettext version to 0.19.3
* Fix stack buffer overflow WRITE 1 in domain_to_punycode()
(Relevant only when built without any IDNA library.)
* Fix undefined behavior in library code
* Ensures that calls to fopen() and stat() can handle largefiles
* Several minor (non-functional) changes
18.07.2020 Release V0.21.1
* Fixing a test due to recent changes in upstream PSL
* Meson build improvements
* Documentation cleanups
* Use semantic versioning for git tags
16.04.2019 Release V0.21.0
* Add -b/--batch to 'psl' to suppress printing the domain
* Add support for Meson build system
* Improve build system
* Improve Windows compatibility
* Remove NLS / gettext
* Several cleanups and cosmetics
26.04.2018 Release V0.20.2
* Fix non srcdir builds
* Add API decoration
* Fix for MSVC/Win32 builds
* Detection fallback from libidn2 to libcu, libidn
* Fix MinGW cross builds on Linux
* Add NMake Makefiles for Visual Studio builds
26.02.2018 Release V0.20.1
* Fix issue introduced with PSL_TYPE_NO_STAR_RULE in V0.20.0
* Fix SO_VERSION to 8:0:3
* Improve unit tests
22.02.2018 Release V0.20.0
* Remove hard-coded gcc flag in Makefile.am
* Prevent excessive CPU cycles on large inputs
* New flag PSL_TYPE_NO_STAR_RULE to skip star rule
09.11.2017 Release V0.19.1
* Add the forgotten commit with NEWS and configure.ac
09.11.2017 Release V0.19.0
* New function psl_free_string()
* psl_make_dafsa now works with python2 and python3
* psl_*count() functions now return -1 if info is not available
* Fixed unsigned integer overflow in _mem_is_ascii()
* Add -fsanitize-address-use-after-scope to --enable-asan if available
20.07.2017 Release V0.18.0
* Fix order of files in psl_latest()
* Add fuzzing architecture
* Fix memleak in _psl_is_public_suffix()
* Add configure option --enable-asan (Address sanitizer)
* Add configure option --enable-usan (Undefined sanitizer)
* Add configure option --enable-cfi (Control Flow Integrity)
* Fix finding libidn2 for static builds
* Fix use of uninitialized stack value
* Fix buffer overflow in libicu build
* Use libidn2 as default for builds (former libicu)
* Add pkg-config support for libidn and libidn2
16.01.2017 Release V0.17.0
* Use TR46 non-transitional for IDNA (libicu, libidn2 >= 0.14)
* Fix coverage upload from TravisCI to Coveralls
* New tests to cover psl_latest() and psl_dist_filename()
15.12.2016 Release V0.16.1
* Fix SO_VERSION to 6:0:1
* Add --use-latest-psl to tools/psl as default
16.12.2016 Release V0.16.0
* Add functions psl_latest() and psl_dist_filename()
* Do not taint out variable on error in psl_str_to_utf8lower()
* Replace psl2c by psl-make-dafsa
* Add missing includes for OpenBSD
* Fix typos
* Update copyright year
14.11.2016 Release V0.15.0
* Python3 compatibility for psl-make-dafsa
* Support for UTF-8 in DAFSA data
* Skip punycode conversion if DAFSA has UTF-8
* Better code coverage by test suite
* Code cleanup and enhancements
* Install man pages for psl-make-dafsa and psl
* Enhancements to the documentation
30.07.2016 Release V0.14.0
* Remove unneeded libraries from tools/psl link step
* Use https instead of http where possible
* Add man page for tools/psl
* Add header magic to DAFSA files
* Rename make_dafsa.py to psl-make-dafsa
* Add man page for psl-make-dafsa
02.03.2016 Release V0.13.0
* Use tests.txt as PSL test file by default
* Slightly shorter DAFSA array when sorting input
* Check for python 2.7+ in configure.ac
* Fix python3 incompatibilities in make_dafsa.py
02.01.2016 Release V0.12.0
* Load DAFSA binaries via psl_load_file() via auto-detection
* Add more tests
* Remove psl_builtin_compile_time()
* Compile PSL into DAFSA using make_dafsa.py
* Avoid libicu dependency with --enable-runtime=no
* Test on new Travis-CI build farm
* Use DAFSA format for builtin PSL data
* Add function psl_is_public_suffix2()
* Fix psl_builtin_outdated()
* Fix several bugs
* Cleanup code
23.09.2015 Release V0.11.0
* Add new function psl_check_version_number()
* Add version defines to include file
19.09.2015 Release V0.10.0
* Code simplified
* Less data entries, faster lookups
* Add new function psl_suffix_wildcard_count()
* Add new helper function psl_builtin_outdated()
15.09.2015 Release V0.9.0
* Added semantic checks to PSL entries when generating built-in data
* Fix test suite for TLD exceptions (not used yet in reality)
* Removed wrong assumption from test suite
* Support explicit combination of 'foo.bar' and '*.foo.bar'
14.08.2015 Release V0.8.1
* Fix documentation

138
README.MSVC.md Normal file
View File

@ -0,0 +1,138 @@
Building libpsl with Visual Studio
==================================
Building libpsl for Windows using Visual Studio 2008 or later is
supported with NMake (from release tarballs) or Meson (from GIT
checkouts). The following sections will cover building libpsl with
these methods.
Currently, for builtin/runtime public suffix list (PSL) IDNA handling,
only ICU is supported for Visual Studio builds.
Using NMake (from a release tarball)
===========
You will need a Python 2.7.x or later installation in order to
complete the build successfully.
You will need the ICU (International Components for Unicode)
libraries, headers and DLLs, to build libpsl, unless both
DISABLE_BUILTIN=1 and DISABLE_RUNTIME=1 are passed into the NMake
command line as listed below.
You can also buid libpsl with libiconv and gettext support, please
see the options below for enabling such support.
In a Visual Studio command prompt which matches your desired
configuration (x86/Win32, x64 etc.),
go to $(srcroot)\msvc, and issue the following command:
nmake /f Makefile.vc CFG=[debug|release]
A 'test' target is provided to build the test programs, while a
'clean' target is provided to remove all the compiled and generated
files for the build. An 'install' target is provided to copy the
build PSL DLL, .lib and executables, as well as the related PDB files
and libpsl header, into appropriate locations under PREFIX (please see
below).
This will build the libpsl DLL/LIB and the psl.exe utility in the
vsX\$(CFG)\$(ARCH) subdirectory, where X is the release version
of Visual Studio, such as 9 for 2008 and 16 for 2019, and ARCH is
Win32 for 32-bit builds and x64 for 64-bit (x86_64) builds.
A number of options can be passed into the NMake command, as follows.
Enable by setting each option to 1, unless otherwise indicated:
* PSL_FILE: Location of the PSL data file, which is retrieved from
https://publicsuffix.org/list/public_suffix_list.dat,
or some other custom location (not supported). Default
is in $(srcroot)\list\public_suffix_list.dat. This is
needed to generate the suffixes_dafsa.h header required
for the build, as well as the binary and ascii dafsa
files used for the test programs.
* TEST_PSL_FILE: Location of the test PSL file. Default is in
$(srcroot)\list\tests\tests.txt. This is
required for building and running the test
programs.
* STATIC: Set if building static versions of libpsl is desired.
* USE_LIBTOOL_DLLNAME: Set to use libtool-style DLL naming.
* DISABLE_RUNTIME: Do not use ICU to generate runtime PSL data.
* DISABLE_BUILTIN: Do not use ICU to generate builtin PSL data.
* USE_ICONV: Enable libiconv support, requires libiconv.
* USE_GETTEXT: Enable gettext support for displaying i18n messages.
Implies USE_ICONV, and requires gettext-runtime.
* PYTHON: Full path to a Python 2.7.x (or later) interpreter, if not
already in your PATH.
Required to generate DAFSA headers and data files that is
needed for the build, as well as generating pkg-config
files for NMake builds.
* PREFIX: Base installation path of the build. Note that any dependent
libraries are searched first from the include\ and lib\
sub-directories in PREFIX before searching in the paths
specified by %INCLUDE% and %LIB%. Default is
$(srcroot)\..\vsX\$(PLATFORM), where X is the release version
of Visual Studio, such as 9 for 2008 and 16 for 2019,
$(PLATFORM) is the target platform (Win32/x64) of the build.
Building libpsl with Meson
==========================
Building using Meson is now supported for Visual Studio builds from a
GIT checkout.
Besides the requirements listed in the NMake builds, you will also need
* Python 3.5.x or later
* Meson build system, use PIP to install from Python 3.5.x64
* Ninja build tool (if not involking Meson with --backend=
vs[2010|2015|2017|2019])
* A compatible PSL data file and a test PSL data file. You may
consider using the ones shipped with the latest libpsl release
tarball and place the PSL data file in $(srcroot)/list and the
test PSL data file in $(srcroot)/list/tests. You may also choose
to download the latest PSL data file from
https://publicsuffix.org/list/public_suffix_list.dat and place it
it $(srcroot)/list. Alternatively, specify
-Dpsl_file=<path_to_psl_data_file> and/or
-Dpsl_testfile=<path_to_test_psl_data_file> when invoking Meson.
Open a Visual Studio command prompt and enter an empty build directory.
Your Python interpreter, Meson executable script and Ninja (if used)
need to be in your PATH.
Any dependent libraries that are being used should have their headers
found in paths specified by %INCLUDE% and their .lib files in the
paths specified by %LIB%.
In the empty build directory, run the following:
meson <path_to_libpsl_git_checkout> --buildtype=... --prefix=<some_prefix> [--backend=vs[2010|2015|2017|2019]]
Please see the Meson documentation for the values accepted by
--buildtype. --backend=vsXXXX generates the corresponding versions
of the Visual Studio solution files to build libpsl, which
will elimnate the need to have the Ninja build tool installed.
When the Meson configuration completes, run 'ninja' or open the
generated solution files with Visual Studio and build the projects
to carry out the build. Run 'ninja test' or the test project to
test the build and run 'ninja install' or 'ninja install' to
install the build results.
If building with Visual Studio 2008, run the following after running
'ninja install' in your builddir:
for /r %f in (*.dll.manifest) do if exist $(prefix)\bin\%~nf mt /manifest %f /outputresource:$(prefix)\bin\%~nf;2
for /r %f in (*.exe.manifest) do if exist $(prefix)\bin\%~nf mt /manifest %f /outputresource:$(prefix)\bin\%~nf;1
So that the application manifests get properly embedded.

View File

@ -1,20 +1,33 @@
[![Build Status](https://travis-ci.org/rockdaboot/libpsl.png?branch=master)](https://travis-ci.org/rockdaboot/libpsl)
[![Travis-CI Status](https://travis-ci.org/rockdaboot/libpsl.png?branch=master)](https://travis-ci.org/rockdaboot/libpsl)
[![Coverity Scan](https://img.shields.io/coverity/scan/10227.svg)](https://scan.coverity.com/projects/rockdaboot-libpsl)
[![Coverage Status](https://coveralls.io/repos/github/rockdaboot/libpsl/badge.svg?branch=master)](https://coveralls.io/github/rockdaboot/libpsl?branch=master)
Solaris OpenCSW [![Build Status Solaris amd64](https://buildfarm.opencsw.org/buildbot/png?builder=libpsl-solaris10-amd64)](https://buildfarm.opencsw.org/buildbot/builders/libpsl-solaris10-amd64)
[![Build Status Solaris i386](https://buildfarm.opencsw.org/buildbot/png?builder=libpsl-solaris10-i386)](https://buildfarm.opencsw.org/buildbot/builders/libpsl-solaris10-i386)
[![Build Status Solaris Sparc](https://buildfarm.opencsw.org/buildbot/png?builder=libpsl-solaris10-sparc)](https://buildfarm.opencsw.org/buildbot/builders/libpsl-solaris10-sparc)
[![Build Status Solaris SparcV9](https://buildfarm.opencsw.org/buildbot/png?builder=libpsl-solaris10-sparcv9)](https://buildfarm.opencsw.org/buildbot/builders/libpsl-solaris10-sparcv9)
libpsl - C library to handle the Public Suffix List
===================================================
A "public suffix" is a domain name under which Internet users can directly register own names.
A *Public Suffix List* is a collection of Top Level Domains (TLDs) suffixes.
TLDs include *Global Top Level Domains* (gTLDs) like `.com` and `.net`;
*Country Top Level Domains* (ccTLDs) like `.de` and `.cn`;
and *[Brand Top Level Domains](https://icannwiki.org/Brand_TLD)* like `.apple` and `.google`.
Brand TLDs allows users to register their own top level domain that exist at the same level as ICANN's gTLDs.
Brand TLDs are sometimes referred to as Vanity Domains.
Browsers and other web clients can use it to
Browsers, web clients and other user agents can use a public suffix list to:
- avoid privacy-leaking "supercookies"
- avoid privacy-leaking "super domain" certificates ([see post from Jeffry Walton](http://lists.gnu.org/archive/html/bug-wget/2014-03/msg00093.html))
- avoid privacy-leaking "super domain" certificates ([see post from Jeffry Walton](https://lists.gnu.org/archive/html/bug-wget/2014-03/msg00093.html))
- domain highlighting parts of the domain in a user interface
- sorting domain lists by site
Libpsl...
- has built-in PSL data for fast access
- has built-in PSL data for fast access (DAWG/DAFSA reduces size from 180kB to ~32kB)
- allows to load PSL data from files
- checks if a given domain is a "public suffix"
- provides immediate cookie domain verification
@ -22,11 +35,13 @@ Libpsl...
- finds the shortest private part of a given domain
- works with international domains (UTF-8 and IDNA2008 Punycode)
- is thread-safe
- handles IDNA2008 UTS#46 (libicu is used by psl2c if installed)
- handles IDNA2008 UTS#46 (if libicu is available)
Find more information about the Publix Suffix List [here](http://publicsuffix.org/).
Find more information about the Public Suffix List [here](https://publicsuffix.org/).
Download the Public Suffix List [here](https://hg.mozilla.org/mozilla-central/raw-file/tip/netwerk/dns/effective_tld_names.dat).
Download the Public Suffix List [here](https://github.com/publicsuffix/list/blob/master/public_suffix_list.dat).
The original DAFSA code is from the [Chromium Project](https://code.google.com/p/chromium/).
API Documentation
@ -68,35 +83,80 @@ library API via command line.
prints the usage.
Convert PSL into DAFSA
----------------------
The [DAFSA](https://en.wikipedia.org/wiki/Deterministic_acyclic_finite_state_automaton) format is a compressed
representation of strings. Here we use it to reduce the whole PSL to about 32k in size.
Generate `psl.dafsa` from `list/public_suffix_list.dat`
$ src/psl-make-dafsa --output-format=binary list/public_suffix_list.dat psl.dafsa
Test the result (example)
$ tools/psl --load-psl-file psl.dafsa aeroclub.aero
License
-------
Libpsl is made available under the terms of the MIT license.<br>
See the LICENSE file that accompanies this distribution for the full text of the license.
src/psl-make-dafsa and src/lookup_string_in_fixed_set.c are licensed under the term written in
src/LICENSE.chromium.
Building from tarball
---------------------
Choose a release from https://github.com/rockdaboot/libpsl/tags an download the tarball
named `libpsl-{version}.tar.*`. Unpack with `tar xf <filename>`, cd into the libsl* directory.
Build with
```
./configure
make
make check
```
Install with
```
sudo make install
```
Building from git
-----------------
You should have python2.7+ installed.
Download project and prepare sources with
git clone http://github.com/rockdaboot/libpsl
git clone https://github.com/rockdaboot/libpsl
./autogen.sh
./configure
make
make check
If you prefer a `meson` build
meson builddir
ninja -C builddir
ninja -C builddir test
There is also an unofficial MSVC nmake build configuration in `msvc/`. Please
see README.MSVC.md on building libpsl with Visual Studio via NMake or Meson.
Mailing List
------------
[Mailing List Archive](http://news.gmane.org/gmane.network.dns.libpsl.bugs)
[Mailing List Archive](https://groups.google.com/forum/#!forum/libpsl-bugs)
[Mailing List](https://groups.google.com/forum/#!forum/libpsl-bugs)
To join the mailing list send an email to
<libpsl-bugs+subscribe@googlegroups.com>
libpsl-bugs+subscribe@googlegroups.com
and follow the instructions provided by the answer mail.

View File

@ -1,21 +1,21 @@
# !/bin/sh -e
#!/bin/sh
AUTORECONF=`which autoreconf 2>/dev/null`
AUTORECONF=$(which autoreconf 2>/dev/null)
if test $? -ne 0; then
echo "No 'autoreconf' found. You must install the autoconf package."
exit 1
fi
GIT=`which git 2>/dev/null`
GIT=$(which git 2>/dev/null)
if test $? -ne 0; then
echo "No 'git' found. You must install the git package."
exit 1
fi
# create m4 before gtkdocize
mkdir m4 2>/dev/null
# create m4 before gtkdocize
mkdir -p m4 2>/dev/null
GTKDOCIZE=`which gtkdocize 2>/dev/null`
GTKDOCIZE=$(which gtkdocize 2>/dev/null)
if test $? -ne 0; then
echo "No gtk-doc support found. You can't build the docs."
# rm because gtk-doc.make might be a link to a protected file
@ -24,12 +24,12 @@ if test $? -ne 0; then
echo "CLEANFILES =" >>gtk-doc.make
GTKDOCIZE=""
else
$GTKDOCIZE || exit $?
$GTKDOCIZE
fi
$GIT submodule init
$GIT submodule update
$AUTORECONF --install --force --symlink || exit $?
$AUTORECONF --install --force --symlink
echo
echo "----------------------------------------------------------------"

View File

@ -1,7 +1,7 @@
AC_INIT([libpsl], [0.8.1], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl])
AC_INIT([libpsl],[m4_normalize(m4_include(version.txt))],[tim.ruehsen@gmx.de],[libpsl],[https://github.com/rockdaboot/libpsl])
AC_PREREQ([2.59])
AM_INIT_AUTOMAKE([1.10 -Wall no-define foreign])
AC_CONFIG_AUX_DIR([build-aux])
AM_INIT_AUTOMAKE([1.10 no-define foreign dist-lzip])
# Generate two configuration headers; one for building the library itself with
# an autogenerated template, and a second one that will be installed alongside
@ -10,18 +10,33 @@ AC_CONFIG_HEADERS([config.h])
AC_PROG_CC
m4_ifdef([AM_PROG_AR], [AM_PROG_AR])
#LT_INIT([disable-static])
LT_INIT
LT_INIT([win32-dll])
AC_CONFIG_MACRO_DIR([m4])
m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
AC_SYS_LARGEFILE
dnl Check that compiler understands inline
AC_C_INLINE
dnl Check for visibility support
gl_VISIBILITY
#
# Generate version defines for include file
#
AC_SUBST([LIBPSL_VERSION_MAJOR], [`echo $VERSION|cut -d'.' -f1`])
AC_SUBST([LIBPSL_VERSION_MINOR], [`echo $VERSION|cut -d'.' -f2`])
AC_SUBST([LIBPSL_VERSION_PATCH], [`echo $VERSION|cut -d'.' -f3`])
AC_SUBST([LIBPSL_VERSION_NUMBER], [`printf '0x%02x%02x%02x' $LIBPSL_VERSION_MAJOR $LIBPSL_VERSION_MINOR $LIBPSL_VERSION_PATCH`])
AC_CONFIG_FILES([include/libpsl.h])
#
# Gettext
#
AM_GNU_GETTEXT([external],[need-ngettext])
AM_GNU_GETTEXT_VERSION([0.18.1])
AM_GNU_GETTEXT_VERSION([0.19.3])
AM_GNU_GETTEXT_REQUIRE_VERSION([0.19.3])
#
# check for gtk-doc
@ -37,38 +52,74 @@ AM_CONDITIONAL([GTK_DOC_USE_LIBTOOL], false)
])
if test x"$have_gtk_doc" = xyes -a x"$enable_gtk_doc" = xyes; then
AC_CONFIG_FILES([docs/libpsl/Makefile docs/libpsl/version.xml])
AC_SUBST([LIBPSL_DOCS], [docs/libpsl])
fi
#
# enable creation of man pages
#
AC_ARG_ENABLE(man,[AC_HELP_STRING([--enable-man],
[generate man pages [default=auto]])],enable_man=yes,enable_man=no)
AS_IF([test "$enable_man" != no], [
AC_PATH_PROG([XSLTPROC], [xsltproc])
AS_IF([test -z "$XSLTPROC"], [
AS_IF([test "$enable_man" = yes], [
AC_MSG_ERROR([xsltproc is required for --enable-man])
])
enable_man=no
])
])
AM_CONDITIONAL(ENABLE_MAN, test x$enable_man != xno)
AC_MSG_CHECKING([whether to generate man pages])
AS_IF([ test "$enable_man" != no ], [
AC_MSG_RESULT([yes])
], [
AC_MSG_RESULT([no])
])
AC_ARG_ENABLE([man],
[AS_HELP_STRING([--enable-man], [generate man pages [default=auto]])],
[
if test "$enable_man" = yes; then
AC_PATH_PROG([XSLTPROC], [xsltproc])
AS_IF([test -z "$XSLTPROC"], [
AC_MSG_ERROR([xsltproc is required for --enable-man])
enable_man="no (xsltproc not found)"
])
fi
], [ enable_man=no ])
AM_CONDITIONAL(ENABLE_MAN, test x$enable_man = xyes)
# src/psl-make-dafsa needs python 2.7+
AM_PATH_PYTHON([2.7])
PKG_PROG_PKG_CONFIG
# Define these substitions here to keep all version information in one place.
AC_ARG_ENABLE([fuzzing],
[AS_HELP_STRING([--enable-fuzzing], [Turn on fuzzing build (for developers)])],
[enable_fuzzing=yes;
AC_SUBST([LIB_FUZZING_ENGINE])
AC_DEFINE([FUZZING], 1, [Define to 1 if this is a fuzzing build])
], [enable_fuzzing=no; LIB_FUZZING_ENGINE=""])
AM_CONDITIONAL([FUZZING], [test "$enable_fuzzing" = "yes"])
AC_ARG_ENABLE([cfi],
[AS_HELP_STRING([--enable-cfi], [Turn on clang's Control Flow Integrity (CFI)])],
[
if test "$enable_cfi" = yes; then
CFLAGS=$CFLAGS" -B/usr/bin/gold -fsanitize=cfi -flto -fvisibility=default -fno-sanitize-trap=all"
AC_LINK_IFELSE([
AC_LANG_PROGRAM([], [])
], [], [
AC_MSG_ERROR([clang 3.7+ and the 'gold' linker are required for --enable-cfi])
])
fi
], [ enable_cfi=no ])
AC_ARG_ENABLE([ubsan],
[AS_HELP_STRING([--enable-ubsan], [Turn on Undefined Behavior Sanitizer (UBSan)])],
[
if test "$enable_ubsan" = yes; then
# Set basic UBSAN compiler flags. Add your own to CFLAGS.
CFLAGS=$CFLAGS" -fsanitize=undefined -fno-sanitize-recover=undefined"
fi
], [ enable_ubsan=no ])
AC_ARG_ENABLE([asan],
[AS_HELP_STRING([--enable-asan], [Turn on Address Sanitizer (ASan)])],
[
if test "$enable_asan" = yes; then
# Set basic ASAN compiler flags. Add your own to CFLAGS.
CFLAGS=$CFLAGS" -fsanitize=address -fno-omit-frame-pointer"
AX_CHECK_COMPILE_FLAG([-fsanitize-address-use-after-scope], [CFLAGS="$CFLAGS -fsanitize-address-use-after-scope"])
fi
], [ enable_asan=no ])
# Define these substitutions here to keep all version information in one place.
# For information on how to properly maintain the library version information,
# refer to the libtool manual, section "Updating library version information":
# http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
# https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
#
# 1. Start with version information of 0:0:0 for each libtool library.
# 2. Update the version information only immediately before a public release of your software. More frequent updates are unnecessary, and only guarantee that the current interface number gets larger faster.
@ -76,25 +127,25 @@ PKG_PROG_PKG_CONFIG
# 4. If any interfaces have been added, removed, or changed since the last update, increment current, and set revision to 0.
# 5. If any interfaces have been added since the last public release, then increment age.
# 6. If any existing interfaces have been removed or changed since the last public release, then set age to 0.
AC_SUBST([LIBPSL_SO_VERSION], [2:5:2])
AC_SUBST([LIBPSL_SO_VERSION], m4_normalize(m4_include([libtool_version_info.txt])))
AC_SUBST([LIBPSL_VERSION], $VERSION)
# Check for enable/disable builtin PSL data
# Check for enable/disable runtime PSL data
AC_ARG_ENABLE(runtime,
[
--enable-runtime[[=IDNA library]]
Specify the IDNA library used for libpsl run-time conversions:
libicu [[default]]: IDNA2008 UTS#46 library
libidn2: IDNA2008 library (also needs libunistring)
libidn: IDNA2003 library (also needs libunistring)
--disable-runtime Do not link runtime IDNA functionality
libidn2 [[default]]: IDNA2008 library (also needs libunistring)
libicu: IDNA2008 UTS#46 library
libidn: IDNA2003 library (also needs libunistring)
--disable-runtime Do not link runtime IDNA functionality
], [
if test "$enableval" = "libicu" -o "$enableval" = "yes"; then
enable_runtime=libicu
AC_DEFINE([WITH_LIBICU], [1], [generate PSL data using libicu])
elif test "$enableval" = "libidn2"; then
if test "$enableval" = "libidn2" -o "$enableval" = "yes"; then
enable_runtime=libidn2
AC_DEFINE([WITH_LIBIDN2], [1], [generate PSL data using libidn2])
elif test "$enableval" = "libicu"; then
enable_runtime=libicu
AC_DEFINE([WITH_LIBICU], [1], [generate PSL data using libicu])
elif test "$enableval" = "libidn"; then
enable_runtime=libidn
AC_DEFINE([WITH_LIBIDN], [1], [generate PSL data using libidn])
@ -105,29 +156,20 @@ AC_ARG_ENABLE(runtime,
fi
], [
# this is the default if neither --enable-runtime nor --disable-runtime were specified
enable_runtime=libicu
AC_DEFINE([WITH_LIBICU], [1], [generate PSL data using libicu])
enable_runtime=auto
])
# Check for enable/disable builtin PSL data
AC_ARG_ENABLE(builtin,
[
--enable-builtin[[=IDNA library]]
Specify the IDNA library used for built-in data generation:
libicu [[default]]: IDNA2008 UTS#46 library
libidn2: IDNA2008 library (also needs libunistring)
libidn: IDNA2003 library (also needs libunistring)
--disable-builtin Do not generate built-in PSL data
--enable-builtin Generate built-in PSL data
--disable-builtin Do not generate built-in PSL data
], [
if test "$enableval" = "libicu" -o "$enableval" = "yes"; then
enable_builtin=libicu
AC_DEFINE([BUILTIN_GENERATOR_LIBICU], [1], [generate PSL data using libicu])
elif test "$enableval" = "libidn2"; then
enable_builtin=libidn2
AC_DEFINE([BUILTIN_GENERATOR_LIBIDN2], [1], [generate PSL data using libidn2])
elif test "$enableval" = "libidn"; then
enable_builtin=libidn
AC_DEFINE([BUILTIN_GENERATOR_LIBIDN], [1], [generate PSL data using libidn])
if test "$enableval" = "yes"; then
enable_builtin=yes
elif test "$enableval" = "libidn" -o "$enableval" = "libidn2" -o "$enableval" = "libicu"; then
AC_MSG_WARN([--enable-builtin=$enableval is deprecated, use --enable-builtin (enabled by default)])
enable_builtin=yes
elif test "$enableval" = "no"; then
enable_builtin=no
else
@ -135,18 +177,48 @@ AC_ARG_ENABLE(builtin,
fi
], [
# this is the default if neither --enable-builtin nor --disable-builtin were specified
enable_builtin=libicu
AC_DEFINE([BUILTIN_GENERATOR_LIBICU], [1], [generate PSL data using libicu])
enable_builtin=yes
])
if test "$enable_runtime" = "libicu" -o "$enable_builtin" = "libicu"; then
if test "$enable_builtin" = "yes"; then
AC_DEFINE([ENABLE_BUILTIN], [1], [Generate built-in PSL data])
fi
if test "$enable_runtime" = "libidn2" -o "$enable_runtime" = "auto"; then
# Check for libidn2
PKG_CHECK_MODULES([LIBIDN2], [libidn2], [
HAVE_LIBIDN2=yes
if test "$enable_runtime" = "libidn2" -o "$enable_runtime" = "auto"; then
LIBS="$LIBIDN2_LIBS $LIBS"
CFLAGS="$LIBIDN2_CFLAGS $CFLAGS"
fi
], [
AC_SEARCH_LIBS(idn2_lookup_u8, idn2, HAVE_LIBIDN2=yes,
[
if test "$enable_runtime" = "libidn2"; then
AC_MSG_ERROR(You requested libidn2 but it is not installed.)
fi
], -lunistring)
])
if test "x$HAVE_LIBIDN2" = "xyes"; then
if test "$enable_runtime" = "auto"; then
enable_runtime=libidn2
AC_DEFINE([WITH_LIBIDN2], [1], [generate PSL data using libidn2])
fi
fi
fi
if test "$enable_runtime" = "libicu" -o "$enable_runtime" = "auto"; then
# Check for libicu
# using pkg-config won't work on older systems like Ubuntu 12.04 LTS Server Edition 64bit
# using AC_SEARCH_LIBS also don't work since functions have the library version appended
PKG_CHECK_MODULES([LIBICU], [icu-uc], [
HAVE_LIBICU=yes
LIBS="$LIBICU_LIBS $LIBS"
CFLAGS="$LIBICU_CFLAGS $CFLAGS"
if test "$enable_runtime" = "libicu" -o "$enable_runtime" = "auto"; then
LIBS="$LIBICU_LIBS $LIBS"
CFLAGS="$LIBICU_CFLAGS $CFLAGS"
fi
], [
OLDLIBS=$LIBS
LIBS="-licuuc $LIBS"
@ -155,31 +227,56 @@ if test "$enable_runtime" = "libicu" -o "$enable_builtin" = "libicu"; then
[AC_LANG_PROGRAM(
[[#include <unicode/ustring.h>]],
[[u_strToUTF8(NULL, 0, NULL, NULL, 0, NULL);]])],
[HAVE_LIBICU=yes; AC_MSG_RESULT([yes])],
[AC_MSG_RESULT([no]); AC_MSG_ERROR(You requested libicu but it is not installed.)])
LIBS=$OLDLIBS
[HAVE_LIBICU=yes; LIBICU_LIBS="-licuuc"; AC_MSG_RESULT([yes])],
[ AC_MSG_RESULT([no]);
if test "$enable_runtime" = "libicu"; then
AC_MSG_ERROR(You requested libicu but it is not installed.)
fi
LIBS=$OLDLIBS
])
])
if test "x$HAVE_LIBICU" = "xyes"; then
if test "$enable_runtime" = "auto"; then
enable_runtime=libicu
AC_DEFINE([WITH_LIBICU], [1], [generate PSL data using libicu])
fi
fi
fi
if test "$enable_runtime" = "libidn2" -o "$enable_builtin" = "libidn2"; then
# Check for libidn2
OLDLIBS=$LIBS
AC_SEARCH_LIBS(idn2_lookup_u8, idn2, HAVE_LIBIDN2=yes, AC_MSG_ERROR(You requested libidn2 but it is not installed.))
LIBS=$OLDLIBS
fi
if test "$enable_runtime" = "libidn" -o "$enable_builtin" = "libidn"; then
if test "$enable_runtime" = "libidn" -o "$enable_runtime" = "auto"; then
# Check for libidn
OLDLIBS=$LIBS
AC_SEARCH_LIBS(idna_to_ascii_8z, idn, HAVE_LIBIDN=yes, AC_MSG_ERROR(You requested libidn but it is not installed.))
LIBS=$OLDLIBS
PKG_CHECK_MODULES([LIBIDN], [libidn], [
HAVE_LIBIDN=yes
if test "$enable_runtime" = "libidn" -o "$enable_runtime" = "auto"; then
LIBS="$LIBIDN_LIBS $LIBS"
CFLAGS="$LIBIDN_CFLAGS $CFLAGS"
fi
], [
AC_SEARCH_LIBS(idna_to_ascii_8z, idn, HAVE_LIBIDN=yes,
[
if test "$enable_runtime" = "libidn"; then
AC_MSG_ERROR(You requested libidn but it is not installed.)
fi
])
])
if test "x$HAVE_LIBIDN" = "xyes"; then
if test "$enable_runtime" = "auto"; then
enable_runtime=libidn
AC_DEFINE([WITH_LIBIDN], [1], [generate PSL data using libidn])
fi
fi
fi
# last fallback is noruntime/nobuiltin
if test "$enable_runtime" = "auto"; then
enable_runtime=no
fi
if test "x$HAVE_LIBIDN2" = "xyes" -o "x$HAVE_LIBIDN" = "xyes"; then
# Check for libunistring, we need it for psl_str_to_utf8lower()
OLDLIBS=$LIBS
AC_SEARCH_LIBS(u8_tolower, unistring, HAVE_UNISTRING=yes, AC_MSG_ERROR(You requested libidn2 but libunistring is not installed.))
LIBS=$OLDLIBS
AC_SEARCH_LIBS(u8_tolower, unistring, HAVE_UNISTRING=yes, AC_MSG_ERROR(You requested libidn2|libidn but libunistring is not installed.))
fi
# AM_ICONV sets @LIBICONV@ and @LTLIBICONV@ for use in Makefile.am
@ -189,10 +286,7 @@ AM_ICONV
AM_CONDITIONAL([WITH_LIBICU], test "x$enable_runtime" = "xlibicu")
AM_CONDITIONAL([WITH_LIBIDN2], test "x$enable_runtime" = "xlibidn2")
AM_CONDITIONAL([WITH_LIBIDN], test "x$enable_runtime" = "xlibidn")
AM_CONDITIONAL([BUILTIN_GENERATOR_LIBICU], test "x$enable_builtin" = "xlibicu")
AM_CONDITIONAL([BUILTIN_GENERATOR_LIBIDN2], test "x$enable_builtin" = "xlibidn2")
AM_CONDITIONAL([BUILTIN_GENERATOR_LIBIDN], test "x$enable_builtin" = "xlibidn")
AM_CONDITIONAL([WITH_BUILTIN], test $enable_builtin = yes)
AM_CONDITIONAL([ENABLE_BUILTIN], test "x$enable_builtin" = "xyes")
# Solaris has socket in libsocket and inet_ntop in libnsl, but also needs libsocket, so the order is important here
AC_CHECK_LIB([socket], [socket], [NEEDS_SOCKET=yes], [])
@ -205,8 +299,24 @@ elif test -n "$NEEDS_SOCKET" ; then
LIBS="$LIBS -lsocket"
elif test -n "$NEEDS_NSL" ; then
LIBS="$LIBS -lnsl"
else
# Platform dependant options
case "${host_os}" in
# MinGW / Windows
*mingw*)
# Select Windows NT/2000 and later, for WSAStringToAddressW()
CPPFLAGS="$CPPFLAGS -D_WIN32_WINNT=0x500"
# Needed for network support
LIBS="$LIBS -lws2_32"
;;
*)
;;
esac
fi
# Check for clock_gettime() used for performance measurement
AC_SEARCH_LIBS(clock_gettime, rt)
# Check for valgrind
ac_enable_valgrind=no
AC_ARG_ENABLE(valgrind-tests,
@ -227,23 +337,34 @@ else
TESTS_INFO="Valgrind testing not enabled"
fi
# Check for distribution-wide PSL file
AC_ARG_WITH(psl-distfile,
AS_HELP_STRING([--with-psl-distfile=[PATH]], [path to distribution-wide PSL file]),
PSL_DISTFILE=$withval AC_SUBST(PSL_DISTFILE))
# Check for custom PSL file
AC_ARG_WITH(psl-file,
AC_HELP_STRING([--with-psl-file=[PATH]], [path to PSL file]),
AS_HELP_STRING([--with-psl-file=[PATH]], [path to PSL file]),
PSL_FILE=$withval,
PSL_FILE="\$(top_srcdir)/list/public_suffix_list.dat")
AC_SUBST(PSL_FILE)
# Check for custom PSL test file
AC_ARG_WITH(psl-testfile,
AC_HELP_STRING([--with-psl-testfile=[PATH]], [path to PSL test file]),
AS_HELP_STRING([--with-psl-testfile=[PATH]], [path to PSL test file]),
PSL_TESTFILE=$withval,
PSL_TESTFILE="\$(top_srcdir)/list/tests/test_psl.txt")
PSL_TESTFILE="\$(top_srcdir)/list/tests/tests.txt")
AC_SUBST(PSL_TESTFILE)
# check for alloca / alloca.h
AC_FUNC_ALLOCA
AC_CHECK_FUNCS([strndup])
AC_CHECK_FUNCS([strndup clock_gettime fmemopen nl_langinfo])
# check for dirent.h
AC_HEADER_DIRENT
# strings.h may not exist on WIN32
AC_CHECK_HEADERS([strings.h])
# Override the template file name of the generated .pc file, so that there
# is no need to rename the template file when the API version changes.
@ -252,8 +373,13 @@ AC_CONFIG_FILES([Makefile
src/Makefile
tools/Makefile
po/Makefile.in
fuzz/Makefile
tests/Makefile
libpsl.pc:libpsl.pc.in])
docs/libpsl/Makefile docs/libpsl/version.xml
libpsl.pc:libpsl.pc.in
msvc/Makefile
msvc/config.h.win32
msvc/config-msvc.mak])
AC_OUTPUT
AC_MSG_NOTICE([Summary of build options:
@ -267,7 +393,12 @@ AC_MSG_NOTICE([Summary of build options:
Libs: ${LIBS}
Runtime: ${enable_runtime}
Builtin: ${enable_builtin}
PSL Dist File: ${PSL_DISTFILE}
PSL File: ${PSL_FILE}
PSL Test File: ${PSL_TESTFILE}
Sanitizers: UBSan $enable_ubsan, ASan $enable_asan, CFI $enable_cfi
Docs: $enable_gtk_doc
Man pages: $enable_man
Tests: ${TESTS_INFO}
Fuzzing build: $enable_fuzzing, $LIB_FUZZING_ENGINE
])

View File

@ -6,15 +6,14 @@
set -e
set -u
CORES=$(grep -c ^processor /proc/cpuinfo 2> /dev/null || echo 0)
if [ $CORES -lt 1 ]; then CORES=1; fi
CORES=$(nproc)
echo "Running: make distclean"
make distclean > /dev/null || true
# We define _GNU_SOURCE to avoid warnings with missing prototypes.
# C89 does not know snprintf, strdup, strndup, popen, pclose
CFLAGS="-std=c89 -pedantic -O2 -g -Wall -Wextra -Wstrict-prototypes -Wold-style-definition -Wwrite-strings -Wshadow -Wformat -Wformat-security -Wunreachable-code -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -D_GNU_SOURCE"
CFLAGS="-std=gnu89 -pedantic -g -Wall -Wextra -Wstrict-prototypes -Wold-style-definition -Wwrite-strings -Wshadow -Wformat -Wformat-security -Wunreachable-code -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition"
CACHEFILE=$PWD/config_check.cache
@ -38,27 +37,52 @@ for CC in gcc clang; do
rm -f $CACHEFILE
for options in \
"--enable-runtime=libicu --enable-builtin=libicu" \
"--enable-runtime=libidn2 --enable-builtin=libidn2" \
"--enable-runtime=libidn --enable-builtin=libidn"; do
export DISTCHECK_CONFIGURE_FLAGS="-C --cache-file=$CACHEFILE $options"
echo
echo " *** ./configure $DISTCHECK_CONFIGURE_FLAGS"
./configure $DISTCHECK_CONFIGURE_FLAGS CFLAGS="$CFLAGS" > /dev/null
"--enable-runtime=libicu" \
"--enable-runtime=libidn2" \
"--enable-runtime=libidn" \
"--disable-runtime"; do
export DISTCHECK_CONFIGURE_FLAGS="-q -C --cache-file=$CACHEFILE $options"
if test "$CC" = "clang"; then
echo
echo " *** CC=$CC ./configure $DISTCHECK_CONFIGURE_FLAGS --enable-cfi"
./configure $DISTCHECK_CONFIGURE_FLAGS --enable-cfi CFLAGS="$CFLAGS"
for xLCALL in C tr_TR.utf8; do
make -s clean
make -s check -j$CORES
done
fi
for flags in --enable-ubsan --enable-asan; do
# case "$flags" in *-asan) case $options in *libicu*) continue;; esac;; esac
echo
echo " *** CC=$CC ./configure $DISTCHECK_CONFIGURE_FLAGS $flags"
./configure $DISTCHECK_CONFIGURE_FLAGS $flags CFLAGS="$CFLAGS"
for xLCALL in C tr_TR.utf8; do
make -s clean
make -s check -j$CORES
done
done
echo
echo " *** CC=$CC ./configure $DISTCHECK_CONFIGURE_FLAGS"
./configure $DISTCHECK_CONFIGURE_FLAGS CFLAGS="$CFLAGS"
for xVALGRIND in 0 1; do
for xLCALL in C tr_TR.utf8; do
export TESTS_ENVIRONMENT="LC_ALL=$xLCALL VALGRIND_TESTS=$xVALGRIND"
echo " *** TESTS_ENVIRONMENT=\"$TESTS_ENVIRONMENT\"" make check -j$CORES
make check -j$CORES > /dev/null
make -s clean
make -s check -j$CORES
done
done
unset TESTS_ENVIRONMENT
export TESTS_ENVIRONMENT
echo
echo " *** make distcheck CFLAGS=$CFLAGS -j$CORES"
make distcheck CFLAGS="$CFLAGS" -j$CORES > /dev/null
echo " *** $CC: make distcheck CFLAGS=$CFLAGS -j$CORES"
./configure $DISTCHECK_CONFIGURE_FLAGS CFLAGS="$CFLAGS"
make -s clean
make -s check -j$CORES
make -s distcheck CFLAGS="$CFLAGS" -j$CORES
done
done

69
contrib/check-hard-meson Executable file
View File

@ -0,0 +1,69 @@
#!/bin/sh
#
# Do some checking before 'git push'.
# Set a stricter bash mode
set -e
set -u
# We define _GNU_SOURCE to avoid warnings with missing prototypes.
# C89 does not know snprintf, strdup, strndup, popen, pclose
export CFLAGS="-std=gnu89 -pedantic -g -Wall -Wextra -Wstrict-prototypes -Wold-style-definition -Wwrite-strings -Wshadow -Wformat -Wformat-security -Wunreachable-code -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition"
ASAN_FLAGS="-fsanitize=address -fno-omit-frame-pointer -fsanitize-address-use-after-scope"
# measure time consumed and print it at the end of the script
START=$(date +%s.%N)
for CC in clang gcc; do
export CC
echo
echo "*** Testing with CC=$CC"
for options in \
"-Dbuiltin=libicu -Druntime=libicu" \
"-Dbuiltin=libidn2 -Druntime=libidn2" \
"-Dbuiltin=libidn -Druntime=libidn" \
"-Dbuiltin=no -Druntime=no"; do
# normal build+check+valgrind-check
echo
echo " *** $options"
rm -rf builddir
meson builddir $options
cd builddir
ninja
for xLCALL in C tr_TR.utf8; do
LC_ALL=$xLCALL meson test
LC_ALL=$xLCALL meson test --wrap='valgrind --tool=memcheck'
done
cd ..
# UBSAN build+check
flags="-Db_sanitize=undefined -Db_lundef=false"
echo
echo " *** $options $flags"
rm -rf builddir
LDFLAGS="-lubsan" meson builddir $options $flags
LDFLAGS="-lubsan" ninja -C builddir
for xLCALL in C tr_TR.utf8; do
LC_ALL=$xLCALL ninja test -C builddir
done
# ASAN build+check
flags="-Db_sanitize=address -Db_lundef=false"
echo
echo " *** $options $flags"
rm -rf builddir
CFLAGS="$CFLAGS $ASAN_FLAGS" meson builddir $options $flags
CFLAGS="$CFLAGS $ASAN_FLAGS" ninja -C builddir
for xLCALL in C tr_TR.utf8; do
LC_ALL=$xLCALL CFLAGS="$CFLAGS $ASAN_FLAGS" ninja test -C builddir
done
done
done
END=$(date +%s.%N)
echo
echo "Duration: "$(echo "$END - $START" | bc)

57
contrib/mingw Executable file
View File

@ -0,0 +1,57 @@
#!/bin/bash
set -e
# just in case...
unset CC
unset CXX
#PREFIX=i686-w64-mingw32
PREFIX=x86_64-w64-mingw32
#export PATH="/usr/$PREFIX/bin:$PATH"
export INSTALLDIR="$PWD/$PREFIX"
export PKG_CONFIG_PATH=$INSTALLDIR/lib/pkgconfig:/usr/$PREFIX/lib/pkgconfig
export CPPFLAGS="-I$INSTALLDIR/include"
export LDFLAGS="-L$INSTALLDIR/lib"
# let mingw compiler be less verbose
export CFLAGS="-O2 -Wall -Wno-format"
# build libunistring
if [ ! -d libunistring ]; then
git clone https://git.savannah.gnu.org/git/libunistring.git
cd libunistring
else
cd libunistring
git pull
fi
./autogen.sh
./configure --build=x86_64-pc-linux-gnu --host=$PREFIX --enable-shared --prefix=$INSTALLDIR
make clean
make -j$(nproc)
make install
cd ..
# build libidn2
if [ ! -d libidn2 ]; then
git clone https://gitlab.com/libidn/libidn2.git
cd libidn2
else
cd libidn2
git pull
fi
./bootstrap
./configure --build=x86_64-pc-linux-gnu --host=$PREFIX --enable-shared --disable-doc --disable-gcc-warnings --prefix=$INSTALLDIR
make clean
make -j$(nproc)
make install
cd ..
# build libpsl
export WINEPATH="$INSTALLDIR/bin;$INSTALLDIR/lib;/usr/$PREFIX/bin;/usr/$PREFIX/lib;$PWD/libpsl/.libs;$GCCLIB"
./autogen.sh
./configure --build=x86_64-pc-linux-gnu --host=$PREFIX --enable-shared --disable-gtk-doc --enable-runtime=libidn2 --prefix=$INSTALLDIR
make clean
make -j$(nproc)
make check -j$(nproc) LOG_COMPILER=wine

9
contrib/spell-checker Executable file
View File

@ -0,0 +1,9 @@
#!/usr/bin/env sh
#
# see https://github.com/codespell-project/codespell
#
# in short:
# - pip install codespell
# - add ~/.local/bin to PATH
codespell $* -L "tim" `git ls-files|egrep -v '_fuzzer.in|_fuzzer.repro|^list$'`

View File

@ -68,7 +68,7 @@ HTML_IMAGES=
# e.g. content_files=running.sgml building.sgml changes-2.0.sgml
content_files=
# SGML files where gtk-doc abbrevations (#GtkWidget) are expanded
# SGML files where gtk-doc abbreviations (#GtkWidget) are expanded
# These files must be listed here *and* in content_files
# e.g. expand_content_files=running.sgml
expand_content_files=
@ -86,7 +86,7 @@ include $(top_srcdir)/gtk-doc.make
# Other files to distribute
# e.g. EXTRA_DIST += version.xml.in
EXTRA_DIST +=
EXTRA_DIST += meson.build
# Files not to distribute
# for --rebuild-types in $(SCAN_OPTIONS), e.g. $(DOC_MODULE).types

View File

@ -11,7 +11,7 @@
<releaseinfo>
for Libpsl &version;.
The latest version of this documentation can be found on-line at
<ulink role="online-location" url="http://github.com/rockdaboot/libpsl">GitHub</ulink>.
<ulink role="online-location" url="https://github.com/rockdaboot/libpsl">GitHub</ulink>.
</releaseinfo>
</bookinfo>
@ -22,10 +22,6 @@
</para>
<xi:include href="xml/libpsl.xml"/>
</chapter>
<chapter id="object-tree">
<title>Object Hierarchy</title>
<xi:include href="xml/tree_index.sgml"/>
</chapter>
<index id="api-index-full">
<title>API Index</title>
<xi:include href="xml/api-index-full.xml"><xi:fallback /></xi:include>
@ -34,6 +30,35 @@
<title>Index of deprecated API</title>
<xi:include href="xml/api-index-deprecated.xml"><xi:fallback /></xi:include>
</index>
<index role="0.1">
<title>Index of symbols included in 0.1</title>
<xi:include href="xml/api-index-0.1.xml"><xi:fallback/></xi:include>
</index>
<index role="0.2.5">
<title>New symbols included in 0.2.5</title>
<xi:include href="xml/api-index-0.2.5.xml"><xi:fallback/></xi:include>
</index>
<index role="0.4">
<title>New symbols included in 0.4</title>
<xi:include href="xml/api-index-0.4.xml"><xi:fallback/></xi:include>
</index>
<index role="0.10.0">
<title>New symbols included in 0.10.0</title>
<xi:include href="xml/api-index-0.10.0.xml"><xi:fallback/></xi:include>
</index>
<index role="0.11.0">
<title>Index of symbols included in 0.11.0</title>
<xi:include href="xml/api-index-0.11.0.xml"><xi:fallback/></xi:include>
</index>
<index role="0.16">
<title>Index of symbols included in 0.16</title>
<xi:include href="xml/api-index-0.16.xml"><xi:fallback/></xi:include>
</index>
<index role="0.19">
<title>Index of symbols included in 0.19</title>
<xi:include href="xml/api-index-0.19.xml"><xi:fallback/></xi:include>
</index>
<!--
<xi:include href="xml/annotation-glossary.xml"><xi:fallback /></xi:include>

View File

@ -1,22 +1,38 @@
<SECTION>
<FILE>libpsl</FILE>
<TITLE>Public Suffix List functions</TITLE>
PSL_API
PSL_VERSION
PSL_VERSION_MAJOR
PSL_VERSION_MINOR
PSL_VERSION_NUMBER
PSL_VERSION_PATCH
PSL_TYPE_ICANN
PSL_TYPE_PRIVATE
PSL_TYPE_NO_STAR_RULE
PSL_TYPE_ANY
psl_error_t
psl_ctx_t
psl_load_file
psl_load_fp
psl_latest
psl_builtin
psl_free
psl_is_public_suffix
psl_is_public_suffix2
psl_unregistrable_domain
psl_registrable_domain
psl_suffix_count
psl_suffix_exception_count
psl_builtin_compile_time
psl_suffix_wildcard_count
psl_builtin_file_time
psl_builtin_sha1sum
psl_builtin_filename
psl_builtin_outdated
psl_is_cookie_domain_acceptable
psl_dist_filename
psl_get_version
psl_check_version_number
psl_str_to_utf8lower
psl_free_string
</SECTION>

20
docs/libpsl/meson.build Normal file
View File

@ -0,0 +1,20 @@
gnome = import('gnome')
if get_option('docs')
configure_file(
input : 'version.xml.in',
output : 'version.xml',
configuration : cdata)
gnome.gtkdoc('libpsl',
main_sgml: 'libpsl-docs.sgml',
src_dir: [
'src',
'include',
],
dependencies: libpsl,
scan_args: [
'--ignore-decorators=G_GNUC_PSL_UNUSED',
],
install: true)
endif

99
fuzz/Makefile.am Normal file
View File

@ -0,0 +1,99 @@
AM_CFLAGS = $(WERROR_CFLAGS) $(WARN_CFLAGS)
AM_CPPFLAGS = -I$(top_srcdir)/include -I$(srcdir) -DSRCDIR=\"$(abs_srcdir)\"
#AM_LDFLAGS = -static
LDADD = ../src/libpsl.la $(LIBICU_LIBS) $(LIBIDN_LIBS) $(LIBIDN2_LIBS)
if !FUZZING
MAIN = main.c fuzzer.h
endif
if WITH_LIBICU
XLIBS = $(LIBICU_LIBS)
XTYPE = _icu
libpsl_icu_fuzzer_SOURCES = libpsl_fuzzer.c $(MAIN)
libpsl_icu_load_fuzzer_SOURCES = libpsl_load_fuzzer.c $(MAIN)
libpsl_icu_load_dafsa_fuzzer_SOURCES = libpsl_load_dafsa_fuzzer.c $(MAIN)
PSL_TESTS = \
libpsl_icu_fuzzer$(EXEEXT) \
libpsl_icu_load_fuzzer$(EXEEXT) \
libpsl_icu_load_dafsa_fuzzer$(EXEEXT)
else
if WITH_LIBIDN2
XLIBS = -lidn2 -lunistring
XTYPE = _idn2
libpsl_idn2_fuzzer_SOURCES = libpsl_fuzzer.c $(MAIN)
libpsl_idn2_load_fuzzer_SOURCES = libpsl_load_fuzzer.c $(MAIN)
libpsl_idn2_load_dafsa_fuzzer_SOURCES = libpsl_load_dafsa_fuzzer.c $(MAIN)
PSL_TESTS = \
libpsl_idn2_fuzzer$(EXEEXT) \
libpsl_idn2_load_fuzzer$(EXEEXT) \
libpsl_idn2_load_dafsa_fuzzer$(EXEEXT)
else
if WITH_LIBIDN
XLIBS = -lidn -lunistring
XTYPE = _idn
libpsl_idn_fuzzer_SOURCES = libpsl_fuzzer.c $(MAIN)
libpsl_idn_load_fuzzer_SOURCES = libpsl_load_fuzzer.c $(MAIN)
libpsl_idn_load_dafsa_fuzzer_SOURCES = libpsl_load_dafsa_fuzzer.c $(MAIN)
PSL_TESTS = \
libpsl_idn_fuzzer$(EXEEXT) \
libpsl_idn_load_fuzzer$(EXEEXT) \
libpsl_idn_load_dafsa_fuzzer$(EXEEXT)
else
XLIBS =
XTYPE =
libpsl_fuzzer_SOURCES = libpsl_fuzzer.c $(MAIN)
libpsl_load_fuzzer_SOURCES = libpsl_load_fuzzer.c $(MAIN)
libpsl_load_dafsa_fuzzer_SOURCES = libpsl_load_dafsa_fuzzer.c $(MAIN)
PSL_TESTS = \
libpsl_fuzzer$(EXEEXT) \
libpsl_load_fuzzer$(EXEEXT) \
libpsl_load_dafsa_fuzzer$(EXEEXT)
endif
endif
endif
if FUZZING
bin_PROGRAMS = $(PSL_TESTS)
LDADD += $(LIB_FUZZING_ENGINE)
AM_LDFLAGS = -no-install
else
AM_CPPFLAGS += -DTEST_RUN
AM_TESTS_ENVIRONMENT = export VALGRIND_TESTS"=@VALGRIND_TESTS@";
TESTS = $(PSL_TESTS)
check_PROGRAMS = $(PSL_TESTS)
endif
EXTRA_DIST = meson.build
dist-hook:
find . -name '*.options' -exec cp -v '{}' $(distdir) ';'
find . -name '*.dict' -exec cp -v '{}' $(distdir) ';'
find . -name '*.in' -exec cp -vr '{}' $(distdir) ';'
find . -name '*.repro' -exec cp -vr '{}' $(distdir) ';'
TESTS_ENVIRONMENT = TESTS_VALGRIND="@VALGRIND_ENVIRONMENT@"
clean-local:
rm -rf *.gc?? *.log lcov coverage.info *_fuzzer *.o
fuzz-coverage: $(PSL_TESTS)
find . -name '*_fuzzer' -exec ./coverage.sh '{}' ';'
CXX ?= clang
CXXFLAGS ?= $(CFLAGS)
oss-fuzz:
if test "$$OUT" != ""; then \
for cfile in `ls *_fuzzer.c`; do \
fuzzer="libpsl$(XTYPE)_"$$(basename $$cfile .c|cut -d'_' -f2-); \
$$CXX $$CXXFLAGS $$CPPFLAGS -I$(top_srcdir)/include -I$(top_srcdir) \
"$${cfile}" -o "$${fuzzer}" \
../src/.libs/libpsl.a $${LIB_FUZZING_ENGINE} \
-Wl,-Bstatic \
$(XLIBS) \
-Wl,-Bdynamic; \
done; \
fi
.PHONY: oss-fuzz

108
fuzz/README.md Normal file
View File

@ -0,0 +1,108 @@
# Fuzzers
These are fuzzers designed for use with `libFuzzer` or `afl`. They can
be used to run on Google's OSS-Fuzz (https://github.com/google/oss-fuzz/).
The convention used here is that the initial values for each parser fuzzer
are taken from the $NAME.in directory.
Crash reproducers from OSS-Fuzz are put into $NAME.repro directory for
regression testing with top dir 'make check' or 'make check-valgrind'.
The ./configure runs below are for libidn2.
To test libicu replace 'libidn2' with 'libicu', to test with
libidn replace 'libidn2' by 'libidn'.
To test without IDNA libraries replace `--enable-runtime=...` with `--disable-runtime`
and replace `--enable-builtin=...` with `--disable-builtin`.
# Running a fuzzer using clang
Use the following commands on top dir:
```
export CC=clang
export LIB_FUZZING_ENGINE="-lFuzzer -lstdc++"
export UBSAN_OPTIONS=print_stacktrace=1
export ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer
# set flags for clang asan
CFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address,leak,nonnull-attribute -fsanitize-address-use-after-scope -fsanitize-coverage=trace-pc-guard,trace-cmp"
# set flags for clang ubsan
CFLAGS="$CFLAGS -fsanitize=bool,array-bounds,float-divide-by-zero,function,integer-divide-by-zero,return,shift,signed-integer-overflow,vla-bound,vptr,undefined,alignment,null,enum,integer,builtin,float-divide-by-zero,function,object-size,returns-nonnull-attribute,unsigned-integer-overflow,unreachable -fsanitize=fuzzer-no-link"
# unsigned-integer-overflow is not UB, so recover when we see it.
CFLAGS="$CFLAGS -fno-sanitize-recover=all -fsanitize-recover=unsigned-integer-overflow"
export CFLAGS
./configure --enable-static --disable-gtk-doc --enable-fuzzing --enable-runtime=libidn2 --enable-builtin=libidn2
make -j$(nproc)
cd fuzz
# run libpsl_fuzzer
./run-clang.sh libpsl_fuzzer
```
If you see a crash, then a crash corpora is written that can be used for further
investigation. E.g.
```
==2410==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000004e90 at pc 0x00000049cf9c bp 0x7fffb5543f70 sp 0x7ff
fb5543720
...
Test unit written to ./crash-adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
```
To reproduce the crash:
```
./libpsl_fuzzer < ./crash-adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
```
You can also copy/move that file into libpsl_fuzzer.repro/
and re-build the project without fuzzing for a valgrind run, if you like that better.
Just a `./configure` and a `make check-valgrind` should reproduce it.
# Running a fuzzer using AFL
Use the following commands on top dir:
```
$ CC=afl-clang-fast ./configure --disable-gtk-doc --enable-runtime=libidn2 --enable-builtin=libidn2
$ make -j$(nproc) clean all
$ cd fuzz
$ ./run-afl.sh libpsl_fuzzer
```
# Fuzz code coverage using the corpus directories *.in/
Code coverage reports currently work best with gcc+lcov+genhtml.
In the top directory:
```
CC=gcc CFLAGS="-O0 -g" ./configure --disable-gtk-doc --enable-runtime=libidn2 --enable-builtin=libidn2
make fuzz-coverage
xdg-open lcov/index.html
```
Each fuzzer target has it's own functions to cover, e.g.
`libpsl_fuzzer` covers psl_is_public_suffix.
To work on corpora for better coverage, `cd fuzz` and use e.g.
`./view-coverage.sh libpsl_fuzzer`.
# Enhancing the testsuite for issues found
Each reproducer file should be dropped into the appropriate *.repro/
directory.
# Clang CFI instrumentation
```
CC=clang CFLAGS="-B/usr/bin/gold -O0 -fsanitize=cfi -flto -fvisibility=default -fno-sanitize-trap=all" ./configure
make clean
make
make check
```

5
fuzz/coverage.sh Executable file
View File

@ -0,0 +1,5 @@
#!/bin/sh
for f in $1.in/*; do
$1 < $f >/dev/null
done

38
fuzz/fuzzer.h Normal file
View File

@ -0,0 +1,38 @@
/*
* Copyright(c) 2017-2022 Tim Ruehsen
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*
* This file is part of libpsl.
*/
#include <config.h>
#include <stddef.h> /* size_t */
#ifdef HAVE_STDINT_H
#include <stdint.h> /* uint8_t */
#elif defined (_MSC_VER)
typedef unsigned __int8 uint8_t;
#endif
#ifdef __cplusplus
extern "C"
#endif
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);

18
fuzz/get_all_corpora Executable file
View File

@ -0,0 +1,18 @@
#!/bin/sh -eu
#
#
if ! grep -q FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile; then
echo "The build environment hasn't been set up for fuzzing (likely for regression testing)."
echo "Please built regarding README.md and try again."
exit 1
fi
if $(grep -q '^#define WITH_LIBIDN2 1' ../config.h); then RUNTIME="_idn2";
elif $(grep -q '^#define WITH_LIBIDN 1' ../config.h); then RUNTIME="_idn";
elif $(grep -q '^#define WITH_LIBICU 1' ../config.h); then RUNTIME="_icu";
else RUNTIME=""; fi
for fuzzer in `ls *_fuzzer.c|cut -d'_' -f2-|cut -d'.' -f1`; do
./get_ossfuzz_corpora libpsl${RUNTIME}_$fuzzer
done

46
fuzz/get_ossfuzz_corpora Executable file
View File

@ -0,0 +1,46 @@
#!/bin/sh -eu
# As a first step see README.md and follow the steps under "Running a fuzzer using clang".
# You might need 'gsutil' to download new corpora from the Google cloud:
# Read the docs at https://github.com/google/oss-fuzz/blob/master/docs/corpora.md
# then install 'google-cloud-sdk' and execute 'gcloud init'.
# Now 'gsutil' should be ready to use.
if test -z "$1"; then
echo "Usage: $0 <fuzzer target>"
echo "Example: $0 libpsl_fuzzer"
exit 1
fi
if ! grep -q FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile; then
echo "The fuzzers haven't been built for fuzzing (maybe for regression testing !?)"
echo "Please built regarding README.md and try again."
exit 1
fi
fuzzer=$1
project=$(echo $1 | cut -d'_' -f1)
# sync/copy the OSS-Fuzz corpora into the .new directory
mkdir -p ${fuzzer}.in ${fuzzer}.new
cp -fp ${fuzzer}.in/* ${fuzzer}.new 2>/dev/null || true
gsutil cp $(gsutil ls gs://${project}-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/${fuzzer}|tail -n 1) ${fuzzer}.new/
(cd ${fuzzer}.new && unzip -o *.zip && mv *.zip ..)
# create fuzzer target
BUILD_ONLY=1 ./run-clang.sh ${fuzzer}
# merge the corpora into the .in directory
./${fuzzer} -merge=1 ${fuzzer}.in ${fuzzer}.new
# now clear .new dir and put all corpora there
rm -rf ${fuzzer}.new
mv ${fuzzer}.in ${fuzzer}.new
mkdir ${fuzzer}.in
# now merge again (optimizes number of corpora)
./${fuzzer} -merge=1 ${fuzzer}.in ${fuzzer}.new
echo
echo "If new files have been added, 'git add' and 'git commit' them."

103
fuzz/libpsl_fuzzer.c Normal file
View File

@ -0,0 +1,103 @@
/*
* Copyright(c) 2017-2022 Tim Ruehsen
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*
* This file is part of libpsl.
*/
#include <config.h>
#include <assert.h> /* assert */
#ifdef HAVE_STDINT_H
#include <stdint.h> /* uint8_t */
#elif defined (_MSC_VER)
typedef unsigned __int8 uint8_t;
#endif
#include <stdlib.h> /* malloc, free */
#include <string.h> /* memcpy */
#if defined(WITH_LIBICU)
#include <unicode/uclean.h>
#endif
#include "libpsl.h"
#include "fuzzer.h"
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
static int first_run = 1;
psl_ctx_t *psl;
char *domain, *res;
int rc;
if (size > 64 * 1024 - 1)
return 0;
domain = (char *) malloc(size + 1);
assert(domain != NULL);
/* 0 terminate */
memcpy(domain, data, size);
domain[size] = 0;
psl = (psl_ctx_t *) psl_builtin();
psl_is_public_suffix(psl, domain);
psl_is_public_suffix2(psl, domain, PSL_TYPE_PRIVATE);
psl_is_public_suffix2(psl, domain, PSL_TYPE_ICANN);
psl_is_public_suffix2(psl, domain, PSL_TYPE_NO_STAR_RULE);
psl_is_public_suffix2(psl, domain, PSL_TYPE_NO_STAR_RULE|PSL_TYPE_ANY);
psl_unregistrable_domain(psl, domain);
psl_registrable_domain(psl, domain);
psl_is_cookie_domain_acceptable(psl, "", NULL);
psl_is_cookie_domain_acceptable(psl, "a.b.c.e.com", domain);
if ((rc = psl_str_to_utf8lower(domain, "utf-8", NULL, &res)) == PSL_SUCCESS)
free(res);
if ((rc = psl_str_to_utf8lower(domain, "iso-8859-1", NULL, &res)) == PSL_SUCCESS)
free(res);
if ((rc = psl_str_to_utf8lower(domain, NULL, NULL, &res)) == PSL_SUCCESS)
free(res);
psl_free(psl);
if (first_run) {
psl_is_public_suffix(NULL, domain);
psl_check_version_number(1);
psl_get_version();
psl_dist_filename();
psl_builtin_outdated();
psl_builtin_filename();
psl_builtin_sha1sum();
psl_builtin_file_time();
first_run = 0;
}
free(domain);
#if defined(WITH_LIBICU)
u_cleanup(); /* free all library internal memory to avoid memory leaks being reported */
#endif
return 0;
}

View File

@ -0,0 +1,5 @@
x
x
x
x
x

View File

@ -0,0 +1 @@
x

View File

@ -0,0 +1 @@
......................................................................mmmmmmmmmmm.........................................................................................

View File

@ -0,0 +1 @@
...........................................................................................................................................................................

View File

@ -0,0 +1 @@
XBBBBBBBBBBBBBBBB.BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBxc}m

View File

@ -0,0 +1 @@
............................................

View File

@ -0,0 +1 @@
fbxos.fbxos.

View File

@ -0,0 +1 @@
V.e.

View File

@ -0,0 +1 @@
q.

View File

@ -0,0 +1 @@
...com

View File

@ -0,0 +1 @@
......................................................................................

View File

@ -0,0 +1 @@
<EFBFBD>.

View File

@ -0,0 +1 @@
X:X:ccom

View File

@ -0,0 +1,2 @@
IIIII
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIKIIIIIIIIIIIIIIIIIIIKIIIIIIIIAIIIIIIIIIIIIIIIIIIIIII

View File

@ -0,0 +1 @@
X.BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBxcom

View File

@ -0,0 +1 @@
x.x

View File

@ -0,0 +1 @@
..e.c.c

View File

@ -0,0 +1 @@
X:X:ccoEEEEEEEEm

View File

@ -0,0 +1 @@
;.VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV

View File

@ -0,0 +1 @@
<EFBFBD>.

View File

@ -0,0 +1 @@
.............................................................................................................-...................

View File

@ -0,0 +1 @@
Ã.a

View File

@ -0,0 +1 @@
e.com

View File

@ -0,0 +1 @@
mm.mm.mm.mm.mm.mm.mm.mm.mm.mm.mm.mm.mm.mm.mm.mm.

View File

@ -0,0 +1 @@
ac.'

View File

@ -0,0 +1,5 @@
x
x
x
x
x

View File

@ -0,0 +1 @@
l.cc.co.usc...cc.co.usc....co.us<..co.us<

View File

@ -0,0 +1 @@
0emm.0emm.

View File

@ -0,0 +1 @@
...d<>

View File

@ -0,0 +1 @@
cc.co.us<63>.s.se

View File

@ -0,0 +1 @@
6.6..в.4..в.в.

View File

@ -0,0 +1 @@
lZź...Z..źÎZ.ĂlddŇŃŃŃŇŃŃ

View File

@ -0,0 +1 @@
酶.酶.酶.<2E>

View File

@ -0,0 +1 @@
י.י

View File

@ -0,0 +1 @@
.......com

View File

@ -0,0 +1 @@
ac.7

View File

@ -0,0 +1 @@
o.m.o.mmm

View File

@ -0,0 +1 @@
mm.mm.mm.

View File

@ -0,0 +1 @@
.............................................................................................................................e.com

View File

@ -0,0 +1 @@
www.ck

View File

@ -0,0 +1 @@
......................................................................mmmmmmmmmmm.........................................................................................

View File

@ -0,0 +1 @@
в≥в╗в.в≥в╗в..в≥в╗в..в≥в╗в.в≥вв╗в..в≥в╗в.в≥в╗в..в≥в╗в.в╗в..в≥в╗в.в≥в╗в.

View File

@ -0,0 +1 @@
..mm

View File

@ -0,0 +1 @@
a..и.

View File

@ -0,0 +1 @@
.............................e.com

View File

@ -0,0 +1 @@
6.6.6.в.4..в.4..в.в.6.6.в.4..в.4..в.в.6.в.4..в.4..4..в.4..в.вв.в.

View File

@ -0,0 +1 @@
.в.в≥в.co.keFЪЪX:X:ccoм.dЪIЪУ.dЪм.dЪIЪУ.dЪЪЪ╓ц╤*.ц╓Ю

View File

@ -0,0 +1 @@
mm.

View File

@ -0,0 +1 @@
j<EFBFBD>.j<>

View File

@ -0,0 +1 @@
...m<>

View File

@ -0,0 +1 @@
t.t.t.

View File

@ -0,0 +1 @@
...mb<6D>

View File

@ -0,0 +1 @@
...и..

View File

@ -0,0 +1 @@
...........................................................................................................................................................................

View File

@ -0,0 +1 @@
x.x..x.c..*.x.x..x..x.c...x.c.η.c.η...x.c...x.x..x..x.c...x.c.η.c.η.

View File

@ -0,0 +1 @@
fbxos.fbxos.fbxos.fbxos.fbxos.fbxos.fbxos.fxos.fbxos.fbxos.fbxos.fbxos.fbxos.fbxos.fbxos.fxos.

View File

@ -0,0 +1 @@
...η

View File

@ -0,0 +1 @@
d.d.

View File

@ -0,0 +1 @@
.(..s.se

View File

@ -0,0 +1 @@
XBBBBBBBBBBBBBBBB.BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBxc}m

Some files were not shown because too many files have changed in this diff Show More