From a06ac3334358a28a887f96452830efc953b75342 Mon Sep 17 00:00:00 2001 From: rockdaboot Date: Thu, 20 Mar 2014 08:38:32 -0700 Subject: [PATCH 001/104] Initial commit --- LICENSE | 165 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 2 + 2 files changed, 167 insertions(+) create mode 100644 LICENSE create mode 100644 README.md diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..bde60ce --- /dev/null +++ b/LICENSE @@ -0,0 +1,165 @@ +GNU LESSER GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + + This version of the GNU Lesser General Public License incorporates +the terms and conditions of version 3 of the GNU General Public +License, supplemented by the additional permissions listed below. + + 0. Additional Definitions. + + As used herein, "this License" refers to version 3 of the GNU Lesser +General Public License, and the "GNU GPL" refers to version 3 of the GNU +General Public License. + + "The Library" refers to a covered work governed by this License, +other than an Application or a Combined Work as defined below. + + An "Application" is any work that makes use of an interface provided +by the Library, but which is not otherwise based on the Library. +Defining a subclass of a class defined by the Library is deemed a mode +of using an interface provided by the Library. + + A "Combined Work" is a work produced by combining or linking an +Application with the Library. The particular version of the Library +with which the Combined Work was made is also called the "Linked +Version". + + The "Minimal Corresponding Source" for a Combined Work means the +Corresponding Source for the Combined Work, excluding any source code +for portions of the Combined Work that, considered in isolation, are +based on the Application, and not on the Linked Version. + + The "Corresponding Application Code" for a Combined Work means the +object code and/or source code for the Application, including any data +and utility programs needed for reproducing the Combined Work from the +Application, but excluding the System Libraries of the Combined Work. + + 1. Exception to Section 3 of the GNU GPL. + + You may convey a covered work under sections 3 and 4 of this License +without being bound by section 3 of the GNU GPL. + + 2. Conveying Modified Versions. + + If you modify a copy of the Library, and, in your modifications, a +facility refers to a function or data to be supplied by an Application +that uses the facility (other than as an argument passed when the +facility is invoked), then you may convey a copy of the modified +version: + + a) under this License, provided that you make a good faith effort to + ensure that, in the event an Application does not supply the + function or data, the facility still operates, and performs + whatever part of its purpose remains meaningful, or + + b) under the GNU GPL, with none of the additional permissions of + this License applicable to that copy. + + 3. Object Code Incorporating Material from Library Header Files. + + The object code form of an Application may incorporate material from +a header file that is part of the Library. You may convey such object +code under terms of your choice, provided that, if the incorporated +material is not limited to numerical parameters, data structure +layouts and accessors, or small macros, inline functions and templates +(ten or fewer lines in length), you do both of the following: + + a) Give prominent notice with each copy of the object code that the + Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the object code with a copy of the GNU GPL and this license + document. + + 4. Combined Works. + + You may convey a Combined Work under terms of your choice that, +taken together, effectively do not restrict modification of the +portions of the Library contained in the Combined Work and reverse +engineering for debugging such modifications, if you also do each of +the following: + + a) Give prominent notice with each copy of the Combined Work that + the Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the Combined Work with a copy of the GNU GPL and this license + document. + + c) For a Combined Work that displays copyright notices during + execution, include the copyright notice for the Library among + these notices, as well as a reference directing the user to the + copies of the GNU GPL and this license document. + + d) Do one of the following: + + 0) Convey the Minimal Corresponding Source under the terms of this + License, and the Corresponding Application Code in a form + suitable for, and under terms that permit, the user to + recombine or relink the Application with a modified version of + the Linked Version to produce a modified Combined Work, in the + manner specified by section 6 of the GNU GPL for conveying + Corresponding Source. + + 1) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (a) uses at run time + a copy of the Library already present on the user's computer + system, and (b) will operate properly with a modified version + of the Library that is interface-compatible with the Linked + Version. + + e) Provide Installation Information, but only if you would otherwise + be required to provide such information under section 6 of the + GNU GPL, and only to the extent that such information is + necessary to install and execute a modified version of the + Combined Work produced by recombining or relinking the + Application with a modified version of the Linked Version. (If + you use option 4d0, the Installation Information must accompany + the Minimal Corresponding Source and Corresponding Application + Code. If you use option 4d1, you must provide the Installation + Information in the manner specified by section 6 of the GNU GPL + for conveying Corresponding Source.) + + 5. Combined Libraries. + + You may place library facilities that are a work based on the +Library side by side in a single library together with other library +facilities that are not Applications and are not covered by this +License, and convey such a combined library under terms of your +choice, if you do both of the following: + + a) Accompany the combined library with a copy of the same work based + on the Library, uncombined with any other library facilities, + conveyed under the terms of this License. + + b) Give prominent notice with the combined library that part of it + is a work based on the Library, and explaining where to find the + accompanying uncombined form of the same work. + + 6. Revised Versions of the GNU Lesser General Public License. + + The Free Software Foundation may publish revised and/or new versions +of the GNU Lesser General Public License from time to time. Such new +versions will be similar in spirit to the present version, but may +differ in detail to address new problems or concerns. + + Each version is given a distinguishing version number. If the +Library as you received it specifies that a certain numbered version +of the GNU Lesser General Public License "or any later version" +applies to it, you have the option of following the terms and +conditions either of that published version or of any later version +published by the Free Software Foundation. If the Library as you +received it does not specify a version number of the GNU Lesser +General Public License, you may choose any version of the GNU Lesser +General Public License ever published by the Free Software Foundation. + + If the Library as you received it specifies that a proxy can decide +whether future versions of the GNU Lesser General Public License shall +apply, that proxy's public statement of acceptance of any version is +permanent authorization for you to choose that version for the +Library. \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000..bbe6bac --- /dev/null +++ b/README.md @@ -0,0 +1,2 @@ +libpsl +====== From 6563469fc1f0c62e1c861136651c55f2b9bdfa98 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 20 Mar 2014 17:17:24 +0100 Subject: [PATCH 002/104] inital commit --- AUTHORS | 0 COPYING | 1 + ChangeLog | 2 ++ INSTALL | 1 + Makefile.am | 14 +++++++++++ NEWS | 0 README | 0 README.md | 18 +++++++++++-- autogen.sh | 18 +++++++++++++ configure.ac | 61 +++++++++++++++++++++++++++++++++++++++++++++ include/Makefile.am | 1 + include/libpsl.h | 0 tests/Makefile.am | 32 ++++++++++++++++++++++++ 13 files changed, 146 insertions(+), 2 deletions(-) create mode 100644 AUTHORS create mode 120000 COPYING create mode 100644 ChangeLog create mode 120000 INSTALL create mode 100644 Makefile.am create mode 100644 NEWS create mode 100644 README create mode 100755 autogen.sh create mode 100644 configure.ac create mode 100644 include/Makefile.am create mode 100644 include/libpsl.h create mode 100644 tests/Makefile.am diff --git a/AUTHORS b/AUTHORS new file mode 100644 index 0000000..e69de29 diff --git a/COPYING b/COPYING new file mode 120000 index 0000000..caeca07 --- /dev/null +++ b/COPYING @@ -0,0 +1 @@ +/usr/share/automake-1.14/COPYING \ No newline at end of file diff --git a/ChangeLog b/ChangeLog new file mode 100644 index 0000000..938ace1 --- /dev/null +++ b/ChangeLog @@ -0,0 +1,2 @@ +2014-02-20 Tim Ruehsen + * inital setup diff --git a/INSTALL b/INSTALL new file mode 120000 index 0000000..f812f5a --- /dev/null +++ b/INSTALL @@ -0,0 +1 @@ +/usr/share/automake-1.14/INSTALL \ No newline at end of file diff --git a/Makefile.am b/Makefile.am new file mode 100644 index 0000000..6064291 --- /dev/null +++ b/Makefile.am @@ -0,0 +1,14 @@ +# got some hints from https://gitorious.org/openismus-playground/examplelib/source + +SUBDIRS = po include libpsl src examples tests +ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS} + +# Enable GTK-Doc during make distcheck +#DISTCHECK_CONFIGURE_FLAGS = --enable-gtk-doc --enable-man + +## Install the generated pkg-config file (.pc) into the expected location for +## architecture-dependent package configuration information. Occasionally, +## pkg-config files are also used for architecture-independent data packages, +## in which case the correct install location would be $(datadir)/pkgconfig. +pkgconfigdir = $(libdir)/pkgconfig +pkgconfig_DATA = libpsl-$(LIBPSL_API_VERSION).pc diff --git a/NEWS b/NEWS new file mode 100644 index 0000000..e69de29 diff --git a/README b/README new file mode 100644 index 0000000..e69de29 diff --git a/README.md b/README.md index bbe6bac..2572a4f 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,16 @@ -libpsl -====== +libpsl - C library to handle the Public Suffix List +=================================================== + +Find more information [here](http://publicsuffix.org/). +Download the Public Suffix List [here](https://hg.mozilla.org/mozilla-central/raw-file/tip/netwerk/dns/effective_tld_names.dat). + +Building from git +----------------- + +Download project and prepare sources with + + git clone http://github.com/rockdaboot/mget + ./autogen.sh + ./configure + make + make check diff --git a/autogen.sh b/autogen.sh new file mode 100755 index 0000000..3006b3d --- /dev/null +++ b/autogen.sh @@ -0,0 +1,18 @@ +# !/bin/sh -e + +if test -z `which autoreconf`; then + echo "No autoreconf found. You must install the autoconf package." + exit 1 +fi + +mkdir m4 2>/dev/null + +autoreconf --install --force --symlink || exit $? + +echo +echo "----------------------------------------------------------------" +echo "Initialized build system. For a common configuration please run:" +echo "----------------------------------------------------------------" +echo +echo "./configure" +echo diff --git a/configure.ac b/configure.ac new file mode 100644 index 0000000..972bbc4 --- /dev/null +++ b/configure.ac @@ -0,0 +1,61 @@ + +AC_INIT([LibPSL], [0.1], [tim.ruehsen@gmx.de], [psl], [http://github.com/rockdaboot/libpsl]) +AC_PREREQ([2.59]) +AM_INIT_AUTOMAKE([1.10 -Wall no-define]) + +# Generate two configuration headers; one for building the library itself with +# an autogenerated template, and a second one that will be installed alongside +# the library. +AC_CONFIG_HEADERS([config.h]) +AC_PROG_CXX +LT_INIT([disable-static]) +C_CONFIG_MACRO_DIR([m4]) +m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) + +# Define these substitions here to keep all version information in one place. +# For information on how to properly maintain the library version information, +# refer to the libtool manual, section "Updating library version information": +# http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html +AC_SUBST([LIBPSL_SO_VERSION], [0:0:0]) +AC_SUBST([LIBPSL_API_VERSION], [0.1]) + +# Check for valgrind +ac_enable_valgrind=no +AC_ARG_ENABLE(valgrind-tests, + AS_HELP_STRING([--enable-valgrind-tests], [enable using Valgrind for tests]), + [ac_enable_valgrind=$enableval], [ac_enable_valgrind=no]) + +if test "${ac_enable_valgrind}" = "yes" ; then + AC_CHECK_PROG(HAVE_VALGRIND, valgrind, yes, no) + if test "$HAVE_VALGRIND" = "yes" ; then + VALGRIND_ENVIRONMENT="valgrind --error-exitcode=1 --leak-check=yes --show-reachable=yes --track-origins=yes" + AC_SUBST(VALGRIND_ENVIRONMENT) + TESTS_INFO="Test suite will be run under Valgrind" + else + TESTS_INFO="Valgrind not found" + fi +else + TESTS_INFO="Valgrind testing not enabled" +fi + +# Override the template file name of the generated .pc file, so that there +# is no need to rename the template file when the API version changes. +AC_CONFIG_FILES([Makefile + include/Makefile + src/Makefile + po/Makefile.in + tests/Makefile + libpsl-${LIBPSL_API_VERSION}.pc:libpsl.pc.in]) +AC_OUTPUT + +AC_MSG_NOTICE([Summary of build options: + + Version: ${PACKAGE_VERSION} + Host OS: ${host_os} + Install prefix: ${prefix} + Compiler: ${CC} + CFlags: ${CFLAGS} ${CPPFLAGS} + LDFlags: ${LDFLAGS} + Tests: ${TESTS_INFO} +]) + diff --git a/include/Makefile.am b/include/Makefile.am new file mode 100644 index 0000000..a45de59 --- /dev/null +++ b/include/Makefile.am @@ -0,0 +1 @@ +include_HEADERS = libpsl.h diff --git a/include/libpsl.h b/include/libpsl.h new file mode 100644 index 0000000..e69de29 diff --git a/tests/Makefile.am b/tests/Makefile.am new file mode 100644 index 0000000..0055c82 --- /dev/null +++ b/tests/Makefile.am @@ -0,0 +1,32 @@ +#DEFS = @DEFS@ -DDATADIR=\"$(datadir)/@PACKAGE@\" -DSRCDIR=\"$(srcdir)\" +DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" +AM_CPPFLAGS = -Wno-missing-field-initializers -I$(top_srcdir)/include +AM_LDFLAGS = -static +LDADD = libtest.la ../src/libpsl-@LIBPSL_API_VERSION@.la + +PSL_TESTS = test + +check_PROGRAMS = $(PSL_TESTS) + +test_SOURCES = test.c +test_LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la +test_parse_html_LDADD = ../libmget/libmget-@LIBPSL_API_VERSION@.la + +noinst_LTLIBRARIES = libtest.la +libtest_la_SOURCES = libtest.c +libtest_la_CPPFLAGS = -I$(top_srcdir)/tests -I$(top_srcdir)/include +#libtest_LDADD = libtest.o + +EXTRA_DIST = libtest.h +dist-hook: + rm -f $(distdir)/files/elb_bibel.txt +# cp $(top_srcdir)/data/public_suffixes.txt $(distdir)/files/ +# rm -rf `find $(distdir)/files -name CVS` + +#dist-hook: +# mkdir $(distdir)/random +# cp -p $(srcdir)/random/a1 $(srcdir)/random/a2 $(distdir)/random + +TESTS_ENVIRONMENT = TESTS_VALGRIND="@VALGRIND_ENVIRONMENT@" +#TESTS = test $(PSL_TESTS) +TESTS = $(PSL_TESTS) From dc05276515194cae81eb984af0e6dc1fcdc84f98 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 20 Mar 2014 22:43:04 +0100 Subject: [PATCH 003/104] autoconf first version --- ABOUT-NLS | 1282 ++++++ ChangeLog | 15 + Makefile.am | 6 +- README.md | 2 +- configure.ac | 9 +- data/Makefile.am | 3 + data/effective_tld_names.dat | 8038 ++++++++++++++++++++++++++++++++++ include/libpsl.h | 60 + libpsl.pc.in | 11 + po/Makevars | 53 + po/POTFILES.in | 2 + src/Makefile.am | 14 + src/psl.c | 333 ++ tests/Makefile.am | 25 +- tests/test-is-tld.c | 105 + 15 files changed, 9933 insertions(+), 25 deletions(-) create mode 100644 ABOUT-NLS create mode 100644 data/Makefile.am create mode 100644 data/effective_tld_names.dat create mode 100644 libpsl.pc.in create mode 100644 po/Makevars create mode 100644 po/POTFILES.in create mode 100644 src/Makefile.am create mode 100644 src/psl.c create mode 100644 tests/test-is-tld.c diff --git a/ABOUT-NLS b/ABOUT-NLS new file mode 100644 index 0000000..b1de1b6 --- /dev/null +++ b/ABOUT-NLS @@ -0,0 +1,1282 @@ +1 Notes on the Free Translation Project +*************************************** + +Free software is going international! The Free Translation Project is +a way to get maintainers of free software, translators, and users all +together, so that free software will gradually become able to speak many +languages. A few packages already provide translations for their +messages. + + If you found this `ABOUT-NLS' file inside a distribution, you may +assume that the distributed package does use GNU `gettext' internally, +itself available at your nearest GNU archive site. But you do _not_ +need to install GNU `gettext' prior to configuring, installing or using +this package with messages translated. + + Installers will find here some useful hints. These notes also +explain how users should proceed for getting the programs to use the +available translations. They tell how people wanting to contribute and +work on translations can contact the appropriate team. + +1.1 INSTALL Matters +=================== + +Some packages are "localizable" when properly installed; the programs +they contain can be made to speak your own native language. Most such +packages use GNU `gettext'. Other packages have their own ways to +internationalization, predating GNU `gettext'. + + By default, this package will be installed to allow translation of +messages. It will automatically detect whether the system already +provides the GNU `gettext' functions. Installers may use special +options at configuration time for changing the default behaviour. The +command: + + ./configure --disable-nls + +will _totally_ disable translation of messages. + + When you already have GNU `gettext' installed on your system and run +configure without an option for your new package, `configure' will +probably detect the previously built and installed `libintl' library +and will decide to use it. If not, you may have to to use the +`--with-libintl-prefix' option to tell `configure' where to look for it. + + Internationalized packages usually have many `po/LL.po' files, where +LL gives an ISO 639 two-letter code identifying the language. Unless +translations have been forbidden at `configure' time by using the +`--disable-nls' switch, all available translations are installed +together with the package. However, the environment variable `LINGUAS' +may be set, prior to configuration, to limit the installed set. +`LINGUAS' should then contain a space separated list of two-letter +codes, stating which languages are allowed. + +1.2 Using This Package +====================== + +As a user, if your language has been installed for this package, you +only have to set the `LANG' environment variable to the appropriate +`LL_CC' combination. If you happen to have the `LC_ALL' or some other +`LC_xxx' environment variables set, you should unset them before +setting `LANG', otherwise the setting of `LANG' will not have the +desired effect. Here `LL' is an ISO 639 two-letter language code, and +`CC' is an ISO 3166 two-letter country code. For example, let's +suppose that you speak German and live in Germany. At the shell +prompt, merely execute `setenv LANG de_DE' (in `csh'), +`export LANG; LANG=de_DE' (in `sh') or `export LANG=de_DE' (in `bash'). +This can be done from your `.login' or `.profile' file, once and for +all. + + You might think that the country code specification is redundant. +But in fact, some languages have dialects in different countries. For +example, `de_AT' is used for Austria, and `pt_BR' for Brazil. The +country code serves to distinguish the dialects. + + The locale naming convention of `LL_CC', with `LL' denoting the +language and `CC' denoting the country, is the one use on systems based +on GNU libc. On other systems, some variations of this scheme are +used, such as `LL' or `LL_CC.ENCODING'. You can get the list of +locales supported by your system for your language by running the +command `locale -a | grep '^LL''. + + Not all programs have translations for all languages. By default, an +English message is shown in place of a nonexistent translation. If you +understand other languages, you can set up a priority list of languages. +This is done through a different environment variable, called +`LANGUAGE'. GNU `gettext' gives preference to `LANGUAGE' over `LANG' +for the purpose of message handling, but you still need to have `LANG' +set to the primary language; this is required by other parts of the +system libraries. For example, some Swedish users who would rather +read translations in German than English for when Swedish is not +available, set `LANGUAGE' to `sv:de' while leaving `LANG' to `sv_SE'. + + Special advice for Norwegian users: The language code for Norwegian +bokma*l changed from `no' to `nb' recently (in 2003). During the +transition period, while some message catalogs for this language are +installed under `nb' and some older ones under `no', it's recommended +for Norwegian users to set `LANGUAGE' to `nb:no' so that both newer and +older translations are used. + + In the `LANGUAGE' environment variable, but not in the `LANG' +environment variable, `LL_CC' combinations can be abbreviated as `LL' +to denote the language's main dialect. For example, `de' is equivalent +to `de_DE' (German as spoken in Germany), and `pt' to `pt_PT' +(Portuguese as spoken in Portugal) in this context. + +1.3 Translating Teams +===================== + +For the Free Translation Project to be a success, we need interested +people who like their own language and write it well, and who are also +able to synergize with other translators speaking the same language. +Each translation team has its own mailing list. The up-to-date list of +teams can be found at the Free Translation Project's homepage, +`http://translationproject.org/', in the "Teams" area. + + If you'd like to volunteer to _work_ at translating messages, you +should become a member of the translating team for your own language. +The subscribing address is _not_ the same as the list itself, it has +`-request' appended. For example, speakers of Swedish can send a +message to `sv-request@li.org', having this message body: + + subscribe + + Keep in mind that team members are expected to participate +_actively_ in translations, or at solving translational difficulties, +rather than merely lurking around. If your team does not exist yet and +you want to start one, or if you are unsure about what to do or how to +get started, please write to `coordinator@translationproject.org' to +reach the coordinator for all translator teams. + + The English team is special. It works at improving and uniformizing +the terminology in use. Proven linguistic skills are praised more than +programming skills, here. + +1.4 Available Packages +====================== + +Languages are not equally supported in all packages. The following +matrix shows the current state of internationalization, as of June +2010. The matrix shows, in regard of each package, for which languages +PO files have been submitted to translation coordination, with a +translation percentage of at least 50%. + + Ready PO files af am an ar as ast az be be@latin bg bn_IN bs ca + +--------------------------------------------------+ + a2ps | [] [] | + aegis | | + ant-phone | | + anubis | | + aspell | [] [] | + bash | | + bfd | | + bibshelf | [] | + binutils | | + bison | | + bison-runtime | [] | + bluez-pin | [] [] | + bombono-dvd | | + buzztard | | + cflow | | + clisp | | + coreutils | [] [] | + cpio | | + cppi | | + cpplib | [] | + cryptsetup | | + dfarc | | + dialog | [] [] | + dico | | + diffutils | [] | + dink | | + doodle | | + e2fsprogs | [] | + enscript | [] | + exif | | + fetchmail | [] | + findutils | [] | + flex | [] | + freedink | | + gas | | + gawk | [] [] | + gcal | [] | + gcc | | + gettext-examples | [] [] [] [] | + gettext-runtime | [] [] | + gettext-tools | [] [] | + gip | [] | + gjay | | + gliv | [] | + glunarclock | [] [] | + gnubiff | | + gnucash | [] | + gnuedu | | + gnulib | | + gnunet | | + gnunet-gtk | | + gnutls | | + gold | | + gpe-aerial | | + gpe-beam | | + gpe-bluetooth | | + gpe-calendar | | + gpe-clock | [] | + gpe-conf | | + gpe-contacts | | + gpe-edit | | + gpe-filemanager | | + gpe-go | | + gpe-login | | + gpe-ownerinfo | [] | + gpe-package | | + gpe-sketchbook | | + gpe-su | [] | + gpe-taskmanager | [] | + gpe-timesheet | [] | + gpe-today | [] | + gpe-todo | | + gphoto2 | | + gprof | [] | + gpsdrive | | + gramadoir | | + grep | | + grub | [] [] | + gsasl | | + gss | | + gst-plugins-bad | [] | + gst-plugins-base | [] | + gst-plugins-good | [] | + gst-plugins-ugly | [] | + gstreamer | [] [] [] | + gtick | | + gtkam | [] | + gtkorphan | [] | + gtkspell | [] [] [] | + gutenprint | | + hello | [] | + help2man | | + hylafax | | + idutils | | + indent | [] [] | + iso_15924 | | + iso_3166 | [] [] [] [] [] [] [] | + iso_3166_2 | | + iso_4217 | | + iso_639 | [] [] [] [] | + iso_639_3 | | + jwhois | | + kbd | | + keytouch | [] | + keytouch-editor | | + keytouch-keyboa... | [] | + klavaro | [] | + latrine | | + ld | [] | + leafpad | [] [] | + libc | [] [] | + libexif | () | + libextractor | | + libgnutls | | + libgpewidget | | + libgpg-error | | + libgphoto2 | | + libgphoto2_port | | + libgsasl | | + libiconv | [] | + libidn | | + lifelines | | + liferea | [] [] | + lilypond | | + linkdr | [] | + lordsawar | | + lprng | | + lynx | [] | + m4 | | + mailfromd | | + mailutils | | + make | | + man-db | | + man-db-manpages | | + minicom | | + mkisofs | | + myserver | | + nano | [] [] | + opcodes | | + parted | | + pies | | + popt | | + psmisc | | + pspp | [] | + pwdutils | | + radius | [] | + recode | [] [] | + rosegarden | | + rpm | | + rush | | + sarg | | + screem | | + scrollkeeper | [] [] [] | + sed | [] [] | + sharutils | [] [] | + shishi | | + skencil | | + solfege | | + solfege-manual | | + soundtracker | | + sp | | + sysstat | | + tar | [] | + texinfo | | + tin | | + unicode-han-tra... | | + unicode-transla... | | + util-linux-ng | [] | + vice | | + vmm | | + vorbis-tools | | + wastesedge | | + wdiff | | + wget | [] [] | + wyslij-po | | + xchat | [] [] [] [] | + xdg-user-dirs | [] [] [] [] [] [] [] [] [] | + xkeyboard-config | [] [] | + +--------------------------------------------------+ + af am an ar as ast az be be@latin bg bn_IN bs ca + 6 0 1 2 3 19 1 10 3 28 3 1 38 + + crh cs da de el en en_GB en_ZA eo es et eu fa + +-------------------------------------------------+ + a2ps | [] [] [] [] [] [] [] | + aegis | [] [] [] | + ant-phone | [] () | + anubis | [] [] | + aspell | [] [] [] [] [] | + bash | [] [] [] | + bfd | [] | + bibshelf | [] [] [] | + binutils | [] | + bison | [] [] | + bison-runtime | [] [] [] [] | + bluez-pin | [] [] [] [] [] [] | + bombono-dvd | [] | + buzztard | [] [] [] | + cflow | [] [] | + clisp | [] [] [] [] | + coreutils | [] [] [] [] | + cpio | | + cppi | | + cpplib | [] [] [] | + cryptsetup | [] | + dfarc | [] [] [] | + dialog | [] [] [] [] [] | + dico | | + diffutils | [] [] [] [] [] [] | + dink | [] [] [] | + doodle | [] | + e2fsprogs | [] [] [] | + enscript | [] [] [] | + exif | () [] [] | + fetchmail | [] [] () [] [] [] | + findutils | [] [] [] | + flex | [] [] | + freedink | [] [] [] | + gas | [] | + gawk | [] [] [] | + gcal | [] | + gcc | [] [] | + gettext-examples | [] [] [] [] | + gettext-runtime | [] [] [] [] | + gettext-tools | [] [] [] | + gip | [] [] [] [] | + gjay | [] | + gliv | [] [] [] | + glunarclock | [] [] | + gnubiff | () | + gnucash | [] () () () () | + gnuedu | [] [] | + gnulib | [] [] | + gnunet | | + gnunet-gtk | [] | + gnutls | [] [] | + gold | [] | + gpe-aerial | [] [] [] [] | + gpe-beam | [] [] [] [] | + gpe-bluetooth | [] [] | + gpe-calendar | [] | + gpe-clock | [] [] [] [] | + gpe-conf | [] [] [] | + gpe-contacts | [] [] [] | + gpe-edit | [] [] | + gpe-filemanager | [] [] [] | + gpe-go | [] [] [] [] | + gpe-login | [] [] | + gpe-ownerinfo | [] [] [] [] | + gpe-package | [] [] [] | + gpe-sketchbook | [] [] [] [] | + gpe-su | [] [] [] [] | + gpe-taskmanager | [] [] [] [] | + gpe-timesheet | [] [] [] [] | + gpe-today | [] [] [] [] | + gpe-todo | [] [] [] | + gphoto2 | [] [] () [] [] [] | + gprof | [] [] [] | + gpsdrive | [] [] [] | + gramadoir | [] [] [] | + grep | [] | + grub | [] [] | + gsasl | [] | + gss | | + gst-plugins-bad | [] [] [] [] [] | + gst-plugins-base | [] [] [] [] [] | + gst-plugins-good | [] [] [] [] [] [] | + gst-plugins-ugly | [] [] [] [] [] [] | + gstreamer | [] [] [] [] [] | + gtick | [] () [] | + gtkam | [] [] () [] [] | + gtkorphan | [] [] [] [] | + gtkspell | [] [] [] [] [] [] [] | + gutenprint | [] [] [] | + hello | [] [] [] [] | + help2man | [] | + hylafax | [] [] | + idutils | [] [] | + indent | [] [] [] [] [] [] [] | + iso_15924 | [] () [] [] | + iso_3166 | [] [] [] [] () [] [] [] () | + iso_3166_2 | () | + iso_4217 | [] [] [] () [] [] | + iso_639 | [] [] [] [] () [] [] | + iso_639_3 | [] | + jwhois | [] | + kbd | [] [] [] [] [] | + keytouch | [] [] | + keytouch-editor | [] [] | + keytouch-keyboa... | [] | + klavaro | [] [] [] [] | + latrine | [] () | + ld | [] [] | + leafpad | [] [] [] [] [] [] | + libc | [] [] [] [] | + libexif | [] [] () | + libextractor | | + libgnutls | [] | + libgpewidget | [] [] | + libgpg-error | [] [] | + libgphoto2 | [] () | + libgphoto2_port | [] () [] | + libgsasl | | + libiconv | [] [] [] [] [] | + libidn | [] [] [] | + lifelines | [] () | + liferea | [] [] [] [] [] | + lilypond | [] [] [] | + linkdr | [] [] [] | + lordsawar | [] | + lprng | | + lynx | [] [] [] [] | + m4 | [] [] [] [] | + mailfromd | | + mailutils | [] | + make | [] [] [] | + man-db | | + man-db-manpages | | + minicom | [] [] [] [] | + mkisofs | | + myserver | | + nano | [] [] [] | + opcodes | [] [] | + parted | [] [] | + pies | | + popt | [] [] [] [] [] | + psmisc | [] [] [] | + pspp | [] | + pwdutils | [] | + radius | [] | + recode | [] [] [] [] [] [] | + rosegarden | () () () | + rpm | [] [] [] | + rush | | + sarg | | + screem | | + scrollkeeper | [] [] [] [] [] | + sed | [] [] [] [] [] [] | + sharutils | [] [] [] [] | + shishi | | + skencil | [] () [] | + solfege | [] [] [] | + solfege-manual | [] [] | + soundtracker | [] [] [] | + sp | [] | + sysstat | [] [] [] | + tar | [] [] [] [] | + texinfo | [] [] [] | + tin | [] [] | + unicode-han-tra... | | + unicode-transla... | | + util-linux-ng | [] [] [] [] | + vice | () () | + vmm | [] | + vorbis-tools | [] [] | + wastesedge | [] | + wdiff | [] [] | + wget | [] [] [] | + wyslij-po | | + xchat | [] [] [] [] [] | + xdg-user-dirs | [] [] [] [] [] [] [] [] [] | + xkeyboard-config | [] [] [] [] [] [] | + +-------------------------------------------------+ + crh cs da de el en en_GB en_ZA eo es et eu fa + 5 64 105 117 18 1 8 0 28 89 18 19 0 + + fi fr ga gl gu he hi hr hu hy id is it ja ka kn + +----------------------------------------------------+ + a2ps | [] [] [] [] | + aegis | [] [] | + ant-phone | [] [] | + anubis | [] [] [] [] | + aspell | [] [] [] [] | + bash | [] [] [] [] | + bfd | [] [] [] | + bibshelf | [] [] [] [] [] | + binutils | [] [] [] | + bison | [] [] [] [] | + bison-runtime | [] [] [] [] [] [] | + bluez-pin | [] [] [] [] [] [] [] [] | + bombono-dvd | [] | + buzztard | [] | + cflow | [] [] [] | + clisp | [] | + coreutils | [] [] [] [] [] | + cpio | [] [] [] [] | + cppi | [] [] | + cpplib | [] [] [] | + cryptsetup | [] [] [] | + dfarc | [] [] [] | + dialog | [] [] [] [] [] [] [] | + dico | | + diffutils | [] [] [] [] [] [] [] [] [] | + dink | [] | + doodle | [] [] | + e2fsprogs | [] [] | + enscript | [] [] [] [] | + exif | [] [] [] [] [] [] | + fetchmail | [] [] [] [] | + findutils | [] [] [] [] [] [] | + flex | [] [] [] | + freedink | [] [] [] | + gas | [] [] | + gawk | [] [] [] [] () [] | + gcal | [] | + gcc | [] | + gettext-examples | [] [] [] [] [] [] [] | + gettext-runtime | [] [] [] [] [] [] | + gettext-tools | [] [] [] [] | + gip | [] [] [] [] [] [] | + gjay | [] | + gliv | [] () | + glunarclock | [] [] [] [] | + gnubiff | () [] () | + gnucash | () () () () () [] | + gnuedu | [] [] | + gnulib | [] [] [] [] [] [] | + gnunet | | + gnunet-gtk | [] | + gnutls | [] [] | + gold | [] [] | + gpe-aerial | [] [] [] | + gpe-beam | [] [] [] [] | + gpe-bluetooth | [] [] [] [] | + gpe-calendar | [] [] | + gpe-clock | [] [] [] [] [] | + gpe-conf | [] [] [] [] | + gpe-contacts | [] [] [] [] | + gpe-edit | [] [] [] | + gpe-filemanager | [] [] [] [] | + gpe-go | [] [] [] [] [] | + gpe-login | [] [] [] | + gpe-ownerinfo | [] [] [] [] [] | + gpe-package | [] [] [] | + gpe-sketchbook | [] [] [] [] | + gpe-su | [] [] [] [] [] [] | + gpe-taskmanager | [] [] [] [] [] | + gpe-timesheet | [] [] [] [] [] | + gpe-today | [] [] [] [] [] [] [] | + gpe-todo | [] [] [] | + gphoto2 | [] [] [] [] [] [] | + gprof | [] [] [] [] | + gpsdrive | [] [] [] | + gramadoir | [] [] [] | + grep | [] [] | + grub | [] [] [] [] | + gsasl | [] [] [] [] [] | + gss | [] [] [] [] [] | + gst-plugins-bad | [] [] [] [] [] [] | + gst-plugins-base | [] [] [] [] [] [] | + gst-plugins-good | [] [] [] [] [] [] | + gst-plugins-ugly | [] [] [] [] [] [] | + gstreamer | [] [] [] [] [] | + gtick | [] [] [] [] [] | + gtkam | [] [] [] [] [] | + gtkorphan | [] [] [] | + gtkspell | [] [] [] [] [] [] [] [] [] | + gutenprint | [] [] [] [] | + hello | [] [] [] | + help2man | [] [] | + hylafax | [] | + idutils | [] [] [] [] [] [] | + indent | [] [] [] [] [] [] [] [] | + iso_15924 | [] () [] [] | + iso_3166 | [] () [] [] [] [] [] [] [] [] [] [] | + iso_3166_2 | () [] [] [] | + iso_4217 | [] () [] [] [] [] | + iso_639 | [] () [] [] [] [] [] [] [] | + iso_639_3 | () [] [] | + jwhois | [] [] [] [] [] | + kbd | [] [] | + keytouch | [] [] [] [] [] [] | + keytouch-editor | [] [] [] [] [] | + keytouch-keyboa... | [] [] [] [] [] | + klavaro | [] [] | + latrine | [] [] [] | + ld | [] [] [] [] | + leafpad | [] [] [] [] [] [] [] () | + libc | [] [] [] [] [] | + libexif | [] | + libextractor | | + libgnutls | [] [] | + libgpewidget | [] [] [] [] | + libgpg-error | [] [] | + libgphoto2 | [] [] [] | + libgphoto2_port | [] [] [] | + libgsasl | [] [] [] [] [] | + libiconv | [] [] [] [] [] [] | + libidn | [] [] [] [] | + lifelines | () | + liferea | [] [] [] [] | + lilypond | [] [] | + linkdr | [] [] [] [] [] | + lordsawar | | + lprng | [] | + lynx | [] [] [] [] [] | + m4 | [] [] [] [] [] [] | + mailfromd | | + mailutils | [] [] | + make | [] [] [] [] [] [] [] [] [] | + man-db | [] [] | + man-db-manpages | [] | + minicom | [] [] [] [] [] | + mkisofs | [] [] [] [] | + myserver | | + nano | [] [] [] [] [] [] | + opcodes | [] [] [] [] | + parted | [] [] [] [] | + pies | | + popt | [] [] [] [] [] [] [] [] [] | + psmisc | [] [] [] | + pspp | | + pwdutils | [] [] | + radius | [] [] | + recode | [] [] [] [] [] [] [] [] | + rosegarden | () () () () () | + rpm | [] [] | + rush | | + sarg | [] | + screem | [] [] | + scrollkeeper | [] [] [] [] | + sed | [] [] [] [] [] [] [] [] | + sharutils | [] [] [] [] [] [] [] | + shishi | [] | + skencil | [] | + solfege | [] [] [] [] | + solfege-manual | [] [] | + soundtracker | [] [] | + sp | [] () | + sysstat | [] [] [] [] [] | + tar | [] [] [] [] [] [] [] | + texinfo | [] [] [] [] | + tin | [] | + unicode-han-tra... | | + unicode-transla... | [] [] | + util-linux-ng | [] [] [] [] [] [] | + vice | () () () | + vmm | [] | + vorbis-tools | [] | + wastesedge | () () | + wdiff | [] | + wget | [] [] [] [] [] [] [] [] | + wyslij-po | [] [] [] | + xchat | [] [] [] [] [] [] [] [] [] | + xdg-user-dirs | [] [] [] [] [] [] [] [] [] [] [] [] [] | + xkeyboard-config | [] [] [] [] [] | + +----------------------------------------------------+ + fi fr ga gl gu he hi hr hu hy id is it ja ka kn + 105 121 53 20 4 8 3 5 53 2 120 5 84 67 0 4 + + ko ku ky lg lt lv mk ml mn mr ms mt nb nds ne + +-----------------------------------------------+ + a2ps | [] | + aegis | | + ant-phone | | + anubis | [] [] | + aspell | [] | + bash | | + bfd | | + bibshelf | [] [] | + binutils | | + bison | [] | + bison-runtime | [] [] [] [] [] | + bluez-pin | [] [] [] [] [] | + bombono-dvd | | + buzztard | | + cflow | | + clisp | | + coreutils | [] | + cpio | | + cppi | | + cpplib | | + cryptsetup | | + dfarc | [] | + dialog | [] [] [] [] [] | + dico | | + diffutils | [] [] | + dink | | + doodle | | + e2fsprogs | | + enscript | | + exif | [] | + fetchmail | | + findutils | | + flex | | + freedink | [] | + gas | | + gawk | | + gcal | | + gcc | | + gettext-examples | [] [] [] [] | + gettext-runtime | [] | + gettext-tools | [] | + gip | [] [] | + gjay | | + gliv | | + glunarclock | [] | + gnubiff | | + gnucash | () () () () | + gnuedu | | + gnulib | | + gnunet | | + gnunet-gtk | | + gnutls | [] | + gold | | + gpe-aerial | [] | + gpe-beam | [] | + gpe-bluetooth | [] [] | + gpe-calendar | [] | + gpe-clock | [] [] [] [] [] | + gpe-conf | [] [] | + gpe-contacts | [] [] | + gpe-edit | [] | + gpe-filemanager | [] [] | + gpe-go | [] [] [] | + gpe-login | [] | + gpe-ownerinfo | [] [] | + gpe-package | [] [] | + gpe-sketchbook | [] [] | + gpe-su | [] [] [] [] [] [] | + gpe-taskmanager | [] [] [] [] [] [] | + gpe-timesheet | [] [] | + gpe-today | [] [] [] [] | + gpe-todo | [] [] | + gphoto2 | | + gprof | [] | + gpsdrive | | + gramadoir | | + grep | | + grub | | + gsasl | | + gss | | + gst-plugins-bad | [] [] [] [] | + gst-plugins-base | [] [] | + gst-plugins-good | [] [] | + gst-plugins-ugly | [] [] [] [] [] | + gstreamer | | + gtick | | + gtkam | [] | + gtkorphan | [] [] | + gtkspell | [] [] [] [] [] [] [] | + gutenprint | | + hello | [] [] [] | + help2man | | + hylafax | | + idutils | | + indent | | + iso_15924 | [] [] | + iso_3166 | [] [] () [] [] [] [] [] | + iso_3166_2 | | + iso_4217 | [] [] | + iso_639 | [] [] | + iso_639_3 | [] | + jwhois | [] | + kbd | | + keytouch | [] | + keytouch-editor | [] | + keytouch-keyboa... | [] | + klavaro | [] | + latrine | [] | + ld | | + leafpad | [] [] [] | + libc | [] | + libexif | | + libextractor | | + libgnutls | [] | + libgpewidget | [] [] | + libgpg-error | | + libgphoto2 | | + libgphoto2_port | | + libgsasl | | + libiconv | | + libidn | | + lifelines | | + liferea | | + lilypond | | + linkdr | | + lordsawar | | + lprng | | + lynx | | + m4 | | + mailfromd | | + mailutils | | + make | [] | + man-db | | + man-db-manpages | | + minicom | [] | + mkisofs | | + myserver | | + nano | [] [] | + opcodes | | + parted | | + pies | | + popt | [] [] [] | + psmisc | | + pspp | | + pwdutils | | + radius | | + recode | | + rosegarden | | + rpm | | + rush | | + sarg | | + screem | | + scrollkeeper | [] [] | + sed | | + sharutils | | + shishi | | + skencil | | + solfege | [] | + solfege-manual | | + soundtracker | | + sp | | + sysstat | [] | + tar | [] | + texinfo | [] | + tin | | + unicode-han-tra... | | + unicode-transla... | | + util-linux-ng | | + vice | | + vmm | | + vorbis-tools | | + wastesedge | | + wdiff | | + wget | [] | + wyslij-po | | + xchat | [] [] [] | + xdg-user-dirs | [] [] [] [] [] [] [] [] | + xkeyboard-config | [] [] [] | + +-----------------------------------------------+ + ko ku ky lg lt lv mk ml mn mr ms mt nb nds ne + 20 5 10 1 13 48 4 2 2 4 24 10 20 3 1 + + nl nn or pa pl ps pt pt_BR ro ru rw sk sl sq sr + +---------------------------------------------------+ + a2ps | [] [] [] [] [] [] [] [] | + aegis | [] [] [] | + ant-phone | [] [] | + anubis | [] [] [] | + aspell | [] [] [] [] [] | + bash | [] [] | + bfd | [] | + bibshelf | [] [] | + binutils | [] [] | + bison | [] [] [] | + bison-runtime | [] [] [] [] [] [] [] | + bluez-pin | [] [] [] [] [] [] [] [] | + bombono-dvd | [] () | + buzztard | [] [] | + cflow | [] | + clisp | [] [] | + coreutils | [] [] [] [] [] [] | + cpio | [] [] [] | + cppi | [] | + cpplib | [] | + cryptsetup | [] | + dfarc | [] | + dialog | [] [] [] [] | + dico | [] | + diffutils | [] [] [] [] [] [] | + dink | () | + doodle | [] [] | + e2fsprogs | [] [] | + enscript | [] [] [] [] [] | + exif | [] [] [] () [] | + fetchmail | [] [] [] [] | + findutils | [] [] [] [] [] | + flex | [] [] [] [] [] | + freedink | [] [] | + gas | | + gawk | [] [] [] [] | + gcal | | + gcc | [] | + gettext-examples | [] [] [] [] [] [] [] [] | + gettext-runtime | [] [] [] [] [] [] [] [] [] | + gettext-tools | [] [] [] [] [] [] | + gip | [] [] [] [] [] | + gjay | | + gliv | [] [] [] [] [] [] | + glunarclock | [] [] [] [] [] | + gnubiff | [] () | + gnucash | [] () () () | + gnuedu | [] | + gnulib | [] [] [] [] | + gnunet | | + gnunet-gtk | | + gnutls | [] [] | + gold | | + gpe-aerial | [] [] [] [] [] [] [] | + gpe-beam | [] [] [] [] [] [] [] | + gpe-bluetooth | [] [] | + gpe-calendar | [] [] [] [] | + gpe-clock | [] [] [] [] [] [] [] [] | + gpe-conf | [] [] [] [] [] [] [] | + gpe-contacts | [] [] [] [] [] | + gpe-edit | [] [] [] | + gpe-filemanager | [] [] [] | + gpe-go | [] [] [] [] [] [] [] [] | + gpe-login | [] [] | + gpe-ownerinfo | [] [] [] [] [] [] [] [] | + gpe-package | [] [] | + gpe-sketchbook | [] [] [] [] [] [] [] | + gpe-su | [] [] [] [] [] [] [] [] | + gpe-taskmanager | [] [] [] [] [] [] [] [] | + gpe-timesheet | [] [] [] [] [] [] [] [] | + gpe-today | [] [] [] [] [] [] [] [] | + gpe-todo | [] [] [] [] [] | + gphoto2 | [] [] [] [] [] [] [] [] | + gprof | [] [] [] | + gpsdrive | [] [] | + gramadoir | [] [] | + grep | [] [] [] [] | + grub | [] [] [] | + gsasl | [] [] [] [] | + gss | [] [] [] | + gst-plugins-bad | [] [] [] [] [] [] | + gst-plugins-base | [] [] [] [] [] | + gst-plugins-good | [] [] [] [] [] | + gst-plugins-ugly | [] [] [] [] [] [] | + gstreamer | [] [] [] [] [] | + gtick | [] [] [] | + gtkam | [] [] [] [] [] [] | + gtkorphan | [] | + gtkspell | [] [] [] [] [] [] [] [] [] [] | + gutenprint | [] [] | + hello | [] [] [] [] | + help2man | [] [] | + hylafax | [] | + idutils | [] [] [] [] [] | + indent | [] [] [] [] [] [] [] | + iso_15924 | [] [] [] [] | + iso_3166 | [] [] [] [] [] () [] [] [] [] [] [] [] [] | + iso_3166_2 | [] [] [] | + iso_4217 | [] [] [] [] [] [] [] [] | + iso_639 | [] [] [] [] [] [] [] [] [] | + iso_639_3 | [] [] | + jwhois | [] [] [] [] | + kbd | [] [] [] | + keytouch | [] [] [] | + keytouch-editor | [] [] [] | + keytouch-keyboa... | [] [] [] | + klavaro | [] [] | + latrine | [] [] | + ld | | + leafpad | [] [] [] [] [] [] [] [] [] | + libc | [] [] [] [] | + libexif | [] [] () [] | + libextractor | | + libgnutls | [] [] | + libgpewidget | [] [] [] | + libgpg-error | [] [] | + libgphoto2 | [] [] | + libgphoto2_port | [] [] [] [] [] | + libgsasl | [] [] [] [] [] | + libiconv | [] [] [] [] [] | + libidn | [] [] | + lifelines | [] [] | + liferea | [] [] [] [] [] () () [] | + lilypond | [] | + linkdr | [] [] [] | + lordsawar | | + lprng | [] | + lynx | [] [] [] | + m4 | [] [] [] [] [] | + mailfromd | [] | + mailutils | [] | + make | [] [] [] [] | + man-db | [] [] [] | + man-db-manpages | [] [] [] | + minicom | [] [] [] [] | + mkisofs | [] [] [] | + myserver | | + nano | [] [] [] [] | + opcodes | [] [] | + parted | [] [] [] [] | + pies | [] | + popt | [] [] [] [] | + psmisc | [] [] [] | + pspp | [] [] | + pwdutils | [] | + radius | [] [] [] | + recode | [] [] [] [] [] [] [] [] | + rosegarden | () () | + rpm | [] [] [] | + rush | [] [] | + sarg | | + screem | | + scrollkeeper | [] [] [] [] [] [] [] [] | + sed | [] [] [] [] [] [] [] [] [] | + sharutils | [] [] [] [] | + shishi | [] | + skencil | [] [] | + solfege | [] [] [] [] | + solfege-manual | [] [] [] | + soundtracker | [] | + sp | | + sysstat | [] [] [] [] | + tar | [] [] [] [] | + texinfo | [] [] [] [] | + tin | [] | + unicode-han-tra... | | + unicode-transla... | | + util-linux-ng | [] [] [] [] [] | + vice | [] | + vmm | [] | + vorbis-tools | [] [] | + wastesedge | [] | + wdiff | [] [] | + wget | [] [] [] [] [] [] [] | + wyslij-po | [] [] [] | + xchat | [] [] [] [] [] [] [] [] [] | + xdg-user-dirs | [] [] [] [] [] [] [] [] [] [] [] [] [] [] | + xkeyboard-config | [] [] [] | + +---------------------------------------------------+ + nl nn or pa pl ps pt pt_BR ro ru rw sk sl sq sr + 135 10 4 7 105 1 29 62 47 91 3 54 46 9 37 + + sv sw ta te tg th tr uk vi wa zh_CN zh_HK zh_TW + +---------------------------------------------------+ + a2ps | [] [] [] [] [] | 27 + aegis | [] | 9 + ant-phone | [] [] [] [] | 9 + anubis | [] [] [] [] | 15 + aspell | [] [] [] | 20 + bash | [] [] [] | 12 + bfd | [] | 6 + bibshelf | [] [] [] | 16 + binutils | [] [] | 8 + bison | [] [] | 12 + bison-runtime | [] [] [] [] [] [] | 29 + bluez-pin | [] [] [] [] [] [] [] [] | 37 + bombono-dvd | [] | 4 + buzztard | [] | 7 + cflow | [] [] [] | 9 + clisp | | 10 + coreutils | [] [] [] [] | 22 + cpio | [] [] [] [] [] [] | 13 + cppi | [] [] | 5 + cpplib | [] [] [] [] [] [] | 14 + cryptsetup | [] [] | 7 + dfarc | [] | 9 + dialog | [] [] [] [] [] [] [] | 30 + dico | [] | 2 + diffutils | [] [] [] [] [] [] | 30 + dink | | 4 + doodle | [] [] | 7 + e2fsprogs | [] [] [] | 11 + enscript | [] [] [] [] | 17 + exif | [] [] [] | 16 + fetchmail | [] [] [] | 17 + findutils | [] [] [] [] [] | 20 + flex | [] [] [] [] | 15 + freedink | [] | 10 + gas | [] | 4 + gawk | [] [] [] [] | 18 + gcal | [] [] | 5 + gcc | [] [] [] | 7 + gettext-examples | [] [] [] [] [] [] [] | 34 + gettext-runtime | [] [] [] [] [] [] [] | 29 + gettext-tools | [] [] [] [] [] [] | 22 + gip | [] [] [] [] | 22 + gjay | [] | 3 + gliv | [] [] [] | 14 + glunarclock | [] [] [] [] [] | 19 + gnubiff | [] [] | 4 + gnucash | () [] () [] () | 10 + gnuedu | [] [] | 7 + gnulib | [] [] [] [] | 16 + gnunet | [] | 1 + gnunet-gtk | [] [] [] | 5 + gnutls | [] [] [] | 10 + gold | [] | 4 + gpe-aerial | [] [] [] | 18 + gpe-beam | [] [] [] | 19 + gpe-bluetooth | [] [] [] | 13 + gpe-calendar | [] [] [] [] | 12 + gpe-clock | [] [] [] [] [] | 28 + gpe-conf | [] [] [] [] | 20 + gpe-contacts | [] [] [] | 17 + gpe-edit | [] [] [] | 12 + gpe-filemanager | [] [] [] [] | 16 + gpe-go | [] [] [] [] [] | 25 + gpe-login | [] [] [] | 11 + gpe-ownerinfo | [] [] [] [] [] | 25 + gpe-package | [] [] [] | 13 + gpe-sketchbook | [] [] [] | 20 + gpe-su | [] [] [] [] [] | 30 + gpe-taskmanager | [] [] [] [] [] | 29 + gpe-timesheet | [] [] [] [] [] | 25 + gpe-today | [] [] [] [] [] [] | 30 + gpe-todo | [] [] [] [] | 17 + gphoto2 | [] [] [] [] [] | 24 + gprof | [] [] [] | 15 + gpsdrive | [] [] [] | 11 + gramadoir | [] [] [] | 11 + grep | [] [] [] | 10 + grub | [] [] [] | 14 + gsasl | [] [] [] [] | 14 + gss | [] [] [] | 11 + gst-plugins-bad | [] [] [] [] | 26 + gst-plugins-base | [] [] [] [] [] | 24 + gst-plugins-good | [] [] [] [] | 24 + gst-plugins-ugly | [] [] [] [] [] | 29 + gstreamer | [] [] [] [] | 22 + gtick | [] [] [] | 13 + gtkam | [] [] [] | 20 + gtkorphan | [] [] [] | 14 + gtkspell | [] [] [] [] [] [] [] [] [] | 45 + gutenprint | [] | 10 + hello | [] [] [] [] [] [] | 21 + help2man | [] [] | 7 + hylafax | [] | 5 + idutils | [] [] [] [] | 17 + indent | [] [] [] [] [] [] | 30 + iso_15924 | () [] () [] [] | 16 + iso_3166 | [] [] () [] [] () [] [] [] () | 53 + iso_3166_2 | () [] () [] | 9 + iso_4217 | [] () [] [] () [] [] | 26 + iso_639 | [] [] [] () [] () [] [] [] [] | 38 + iso_639_3 | [] () | 8 + jwhois | [] [] [] [] [] | 16 + kbd | [] [] [] [] [] | 15 + keytouch | [] [] [] | 16 + keytouch-editor | [] [] [] | 14 + keytouch-keyboa... | [] [] [] | 14 + klavaro | [] | 11 + latrine | [] [] [] | 10 + ld | [] [] [] [] | 11 + leafpad | [] [] [] [] [] [] | 33 + libc | [] [] [] [] [] | 21 + libexif | [] () | 7 + libextractor | [] | 1 + libgnutls | [] [] [] | 9 + libgpewidget | [] [] [] | 14 + libgpg-error | [] [] [] | 9 + libgphoto2 | [] [] | 8 + libgphoto2_port | [] [] [] [] | 14 + libgsasl | [] [] [] | 13 + libiconv | [] [] [] [] | 21 + libidn | () [] [] | 11 + lifelines | [] | 4 + liferea | [] [] [] | 21 + lilypond | [] | 7 + linkdr | [] [] [] [] [] | 17 + lordsawar | | 1 + lprng | [] | 3 + lynx | [] [] [] [] | 17 + m4 | [] [] [] [] | 19 + mailfromd | [] [] | 3 + mailutils | [] | 5 + make | [] [] [] [] | 21 + man-db | [] [] [] | 8 + man-db-manpages | | 4 + minicom | [] [] | 16 + mkisofs | [] [] | 9 + myserver | | 0 + nano | [] [] [] [] | 21 + opcodes | [] [] [] | 11 + parted | [] [] [] [] [] | 15 + pies | [] [] | 3 + popt | [] [] [] [] [] [] | 27 + psmisc | [] [] | 11 + pspp | | 4 + pwdutils | [] [] | 6 + radius | [] [] | 9 + recode | [] [] [] [] | 28 + rosegarden | () | 0 + rpm | [] [] [] | 11 + rush | [] [] | 4 + sarg | | 1 + screem | [] | 3 + scrollkeeper | [] [] [] [] [] | 27 + sed | [] [] [] [] [] | 30 + sharutils | [] [] [] [] [] | 22 + shishi | [] | 3 + skencil | [] [] | 7 + solfege | [] [] [] [] | 16 + solfege-manual | [] | 8 + soundtracker | [] [] [] | 9 + sp | [] | 3 + sysstat | [] [] | 15 + tar | [] [] [] [] [] [] | 23 + texinfo | [] [] [] [] [] | 17 + tin | | 4 + unicode-han-tra... | | 0 + unicode-transla... | | 2 + util-linux-ng | [] [] [] [] | 20 + vice | () () | 1 + vmm | [] | 4 + vorbis-tools | [] | 6 + wastesedge | | 2 + wdiff | [] [] | 7 + wget | [] [] [] [] [] | 26 + wyslij-po | [] [] | 8 + xchat | [] [] [] [] [] [] | 36 + xdg-user-dirs | [] [] [] [] [] [] [] [] [] [] | 63 + xkeyboard-config | [] [] [] | 22 + +---------------------------------------------------+ + 85 teams sv sw ta te tg th tr uk vi wa zh_CN zh_HK zh_TW + 178 domains 119 1 3 3 0 10 65 51 155 17 98 7 41 2618 + + Some counters in the preceding matrix are higher than the number of +visible blocks let us expect. This is because a few extra PO files are +used for implementing regional variants of languages, or language +dialects. + + For a PO file in the matrix above to be effective, the package to +which it applies should also have been internationalized and +distributed as such by its maintainer. There might be an observable +lag between the mere existence a PO file and its wide availability in a +distribution. + + If June 2010 seems to be old, you may fetch a more recent copy of +this `ABOUT-NLS' file on most GNU archive sites. The most up-to-date +matrix with full percentage details can be found at +`http://translationproject.org/extra/matrix.html'. + +1.5 Using `gettext' in new packages +=================================== + +If you are writing a freely available program and want to +internationalize it you are welcome to use GNU `gettext' in your +package. Of course you have to respect the GNU Library General Public +License which covers the use of the GNU `gettext' library. This means +in particular that even non-free programs can use `libintl' as a shared +library, whereas only free software can use `libintl' as a static +library or use modified versions of `libintl'. + + Once the sources are changed appropriately and the setup can handle +the use of `gettext' the only thing missing are the translations. The +Free Translation Project is also available for packages which are not +developed inside the GNU project. Therefore the information given above +applies also for every other Free Software Project. Contact +`coordinator@translationproject.org' to make the `.pot' files available +to the translation teams. + diff --git a/ChangeLog b/ChangeLog index 938ace1..714fcb1 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,2 +1,17 @@ +2014-03-20 gettextize + + * m4/gettext.m4: New file, from gettext-0.18.3. + * m4/iconv.m4: New file, from gettext-0.18.3. + * m4/lib-ld.m4: New file, from gettext-0.18.3. + * m4/lib-link.m4: New file, from gettext-0.18.3. + * m4/lib-prefix.m4: New file, from gettext-0.18.3. + * m4/nls.m4: New file, from gettext-0.18.3. + * m4/po.m4: New file, from gettext-0.18.3. + * m4/progtest.m4: New file, from gettext-0.18.3. + * Makefile.am (SUBDIRS): Add po. + (ACLOCAL_AMFLAGS): Add -I m4. + (EXTRA_DIST): New variable. + * configure.ac (AC_CONFIG_FILES): Add po/Makefile.in. + 2014-02-20 Tim Ruehsen * inital setup diff --git a/Makefile.am b/Makefile.am index 6064291..2b7d6a4 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,7 +1,7 @@ # got some hints from https://gitorious.org/openismus-playground/examplelib/source -SUBDIRS = po include libpsl src examples tests -ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS} +SUBDIRS = po include src tests +ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS} # Enable GTK-Doc during make distcheck #DISTCHECK_CONFIGURE_FLAGS = --enable-gtk-doc --enable-man @@ -12,3 +12,5 @@ ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS} ## in which case the correct install location would be $(datadir)/pkgconfig. pkgconfigdir = $(libdir)/pkgconfig pkgconfig_DATA = libpsl-$(LIBPSL_API_VERSION).pc + +EXTRA_DIST = config.rpath diff --git a/README.md b/README.md index 2572a4f..3e701b3 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ Building from git Download project and prepare sources with - git clone http://github.com/rockdaboot/mget + git clone http://github.com/rockdaboot/libpsl ./autogen.sh ./configure make diff --git a/configure.ac b/configure.ac index 972bbc4..016fc71 100644 --- a/configure.ac +++ b/configure.ac @@ -9,9 +9,15 @@ AM_INIT_AUTOMAKE([1.10 -Wall no-define]) AC_CONFIG_HEADERS([config.h]) AC_PROG_CXX LT_INIT([disable-static]) -C_CONFIG_MACRO_DIR([m4]) +AC_CONFIG_MACRO_DIR([m4]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) +# +# Gettext +# +AM_GNU_GETTEXT([external],[need-ngettext]) +AM_GNU_GETTEXT_VERSION([0.18.1]) + # Define these substitions here to keep all version information in one place. # For information on how to properly maintain the library version information, # refer to the libtool manual, section "Updating library version information": @@ -44,6 +50,7 @@ AC_CONFIG_FILES([Makefile include/Makefile src/Makefile po/Makefile.in + data/Makefile tests/Makefile libpsl-${LIBPSL_API_VERSION}.pc:libpsl.pc.in]) AC_OUTPUT diff --git a/data/Makefile.am b/data/Makefile.am new file mode 100644 index 0000000..5b766f9 --- /dev/null +++ b/data/Makefile.am @@ -0,0 +1,3 @@ +filesdir = $(datadir)/@PACKAGE@ +files_DATA = effective_tld_names.dat +EXTRA_DIST = $(files_DATA) diff --git a/data/effective_tld_names.dat b/data/effective_tld_names.dat new file mode 100644 index 0000000..9edd20d --- /dev/null +++ b/data/effective_tld_names.dat @@ -0,0 +1,8038 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at http://mozilla.org/MPL/2.0/. + +// ===BEGIN ICANN DOMAINS=== + +// ac : http://en.wikipedia.org/wiki/.ac +ac +com.ac +edu.ac +gov.ac +net.ac +mil.ac +org.ac + +// ad : http://en.wikipedia.org/wiki/.ad +ad +nom.ad + +// ae : http://en.wikipedia.org/wiki/.ae +// see also: "Domain Name Eligibility Policy" at http://www.aeda.ae/eng/aepolicy.php +ae +co.ae +net.ae +org.ae +sch.ae +ac.ae +gov.ae +mil.ae + +// aero : see http://www.information.aero/index.php?id=66 +aero +accident-investigation.aero +accident-prevention.aero +aerobatic.aero +aeroclub.aero +aerodrome.aero +agents.aero +aircraft.aero +airline.aero +airport.aero +air-surveillance.aero +airtraffic.aero +air-traffic-control.aero +ambulance.aero +amusement.aero +association.aero +author.aero +ballooning.aero +broker.aero +caa.aero +cargo.aero +catering.aero +certification.aero +championship.aero +charter.aero +civilaviation.aero +club.aero +conference.aero +consultant.aero +consulting.aero +control.aero +council.aero +crew.aero +design.aero +dgca.aero +educator.aero +emergency.aero +engine.aero +engineer.aero +entertainment.aero +equipment.aero +exchange.aero +express.aero +federation.aero +flight.aero +freight.aero +fuel.aero +gliding.aero +government.aero +groundhandling.aero +group.aero +hanggliding.aero +homebuilt.aero +insurance.aero +journal.aero +journalist.aero +leasing.aero +logistics.aero +magazine.aero +maintenance.aero +marketplace.aero +media.aero +microlight.aero +modelling.aero +navigation.aero +parachuting.aero +paragliding.aero +passenger-association.aero +pilot.aero +press.aero +production.aero +recreation.aero +repbody.aero +res.aero +research.aero +rotorcraft.aero +safety.aero +scientist.aero +services.aero +show.aero +skydiving.aero +software.aero +student.aero +taxi.aero +trader.aero +trading.aero +trainer.aero +union.aero +workinggroup.aero +works.aero + +// af : http://www.nic.af/help.jsp +af +gov.af +com.af +org.af +net.af +edu.af + +// ag : http://www.nic.ag/prices.htm +ag +com.ag +org.ag +net.ag +co.ag +nom.ag + +// ai : http://nic.com.ai/ +ai +off.ai +com.ai +net.ai +org.ai + +// al : http://www.ert.gov.al/ert_alb/faq_det.html?Id=31 +al +com.al +edu.al +gov.al +mil.al +net.al +org.al + +// am : http://en.wikipedia.org/wiki/.am +am + +// an : http://www.una.an/an_domreg/default.asp +an +com.an +net.an +org.an +edu.an + +// ao : http://en.wikipedia.org/wiki/.ao +// http://www.dns.ao/REGISTR.DOC +ao +ed.ao +gv.ao +og.ao +co.ao +pb.ao +it.ao + +// aq : http://en.wikipedia.org/wiki/.aq +aq + +// ar : https://nic.ar/normativa-vigente.xhtml +ar +com.ar +edu.ar +gob.ar +int.ar +mil.ar +net.ar +org.ar +tur.ar + +// arpa : http://en.wikipedia.org/wiki/.arpa +// Confirmed by registry 2008-06-18 +arpa +e164.arpa +in-addr.arpa +ip6.arpa +iris.arpa +uri.arpa +urn.arpa + +// as : http://en.wikipedia.org/wiki/.as +as +gov.as + +// asia : http://en.wikipedia.org/wiki/.asia +asia + +// at : http://en.wikipedia.org/wiki/.at +// Confirmed by registry 2008-06-17 +at +ac.at +co.at +gv.at +or.at + +// au : http://en.wikipedia.org/wiki/.au +// http://www.auda.org.au/ +au +// 2LDs +com.au +net.au +org.au +edu.au +gov.au +asn.au +id.au +csiro.au +// Historic 2LDs (closed to new registration, but sites still exist) +info.au +conf.au +oz.au +// CGDNs - http://www.cgdn.org.au/ +act.au +nsw.au +nt.au +qld.au +sa.au +tas.au +vic.au +wa.au +// 3LDs +act.edu.au +nsw.edu.au +nt.edu.au +qld.edu.au +sa.edu.au +tas.edu.au +vic.edu.au +wa.edu.au +act.gov.au +// nsw.gov.au Bug 547985 - Removed at request of +// nt.gov.au Bug 940478 - Removed at request of Greg Connors +qld.gov.au +sa.gov.au +tas.gov.au +vic.gov.au +wa.gov.au + +// aw : http://en.wikipedia.org/wiki/.aw +aw +com.aw + +// ax : http://en.wikipedia.org/wiki/.ax +ax + +// az : http://en.wikipedia.org/wiki/.az +az +com.az +net.az +int.az +gov.az +org.az +edu.az +info.az +pp.az +mil.az +name.az +pro.az +biz.az + +// ba : http://en.wikipedia.org/wiki/.ba +ba +org.ba +net.ba +edu.ba +gov.ba +mil.ba +unsa.ba +unbi.ba +co.ba +com.ba +rs.ba + +// bb : http://en.wikipedia.org/wiki/.bb +bb +biz.bb +com.bb +edu.bb +gov.bb +info.bb +net.bb +org.bb +store.bb + +// bd : http://en.wikipedia.org/wiki/.bd +*.bd + +// be : http://en.wikipedia.org/wiki/.be +// Confirmed by registry 2008-06-08 +be +ac.be + +// bf : http://en.wikipedia.org/wiki/.bf +bf +gov.bf + +// bg : http://en.wikipedia.org/wiki/.bg +// https://www.register.bg/user/static/rules/en/index.html +bg +a.bg +b.bg +c.bg +d.bg +e.bg +f.bg +g.bg +h.bg +i.bg +j.bg +k.bg +l.bg +m.bg +n.bg +o.bg +p.bg +q.bg +r.bg +s.bg +t.bg +u.bg +v.bg +w.bg +x.bg +y.bg +z.bg +0.bg +1.bg +2.bg +3.bg +4.bg +5.bg +6.bg +7.bg +8.bg +9.bg + +// bh : http://en.wikipedia.org/wiki/.bh +bh +com.bh +edu.bh +net.bh +org.bh +gov.bh + +// bi : http://en.wikipedia.org/wiki/.bi +// http://whois.nic.bi/ +bi +co.bi +com.bi +edu.bi +or.bi +org.bi + +// biz : http://en.wikipedia.org/wiki/.biz +biz + +// bj : http://en.wikipedia.org/wiki/.bj +bj +asso.bj +barreau.bj +gouv.bj + +// bm : http://www.bermudanic.bm/dnr-text.txt +bm +com.bm +edu.bm +gov.bm +net.bm +org.bm + +// bn : http://en.wikipedia.org/wiki/.bn +*.bn + +// bo : http://www.nic.bo/ +bo +com.bo +edu.bo +gov.bo +gob.bo +int.bo +org.bo +net.bo +mil.bo +tv.bo + +// br : http://registro.br/dominio/categoria.html +// Submitted by registry 2014-03-04 +br +adm.br +adv.br +agr.br +am.br +arq.br +art.br +ato.br +b.br +bio.br +blog.br +bmd.br +cim.br +cng.br +cnt.br +com.br +coop.br +ecn.br +eco.br +edu.br +emp.br +eng.br +esp.br +etc.br +eti.br +far.br +flog.br +fm.br +fnd.br +fot.br +fst.br +g12.br +ggf.br +gov.br +imb.br +ind.br +inf.br +jor.br +jus.br +leg.br +lel.br +mat.br +med.br +mil.br +mp.br +mus.br +net.br +nom.br +not.br +ntr.br +odo.br +org.br +ppg.br +pro.br +psc.br +psi.br +qsl.br +radio.br +rec.br +slg.br +srv.br +taxi.br +teo.br +tmp.br +trd.br +tur.br +tv.br +vet.br +vlog.br +wiki.br +zlg.br + +// bs : http://www.nic.bs/rules.html +bs +com.bs +net.bs +org.bs +edu.bs +gov.bs + +// bt : http://en.wikipedia.org/wiki/.bt +bt +com.bt +edu.bt +gov.bt +net.bt +org.bt + +// bv : No registrations at this time. +// Submitted by registry 2006-06-16 +bv + +// bw : http://en.wikipedia.org/wiki/.bw +// http://www.gobin.info/domainname/bw.doc +// list of other 2nd level tlds ? +bw +co.bw +org.bw + +// by : http://en.wikipedia.org/wiki/.by +// http://tld.by/rules_2006_en.html +// list of other 2nd level tlds ? +by +gov.by +mil.by +// Official information does not indicate that com.by is a reserved +// second-level domain, but it's being used as one (see www.google.com.by and +// www.yahoo.com.by, for example), so we list it here for safety's sake. +com.by + +// http://hoster.by/ +of.by + +// bz : http://en.wikipedia.org/wiki/.bz +// http://www.belizenic.bz/ +bz +com.bz +net.bz +org.bz +edu.bz +gov.bz + +// ca : http://en.wikipedia.org/wiki/.ca +ca +// ca geographical names +ab.ca +bc.ca +mb.ca +nb.ca +nf.ca +nl.ca +ns.ca +nt.ca +nu.ca +on.ca +pe.ca +qc.ca +sk.ca +yk.ca +// gc.ca: http://en.wikipedia.org/wiki/.gc.ca +// see also: http://registry.gc.ca/en/SubdomainFAQ +gc.ca + +// cat : http://en.wikipedia.org/wiki/.cat +cat + +// cc : http://en.wikipedia.org/wiki/.cc +cc + +// cd : http://en.wikipedia.org/wiki/.cd +// see also: https://www.nic.cd/domain/insertDomain_2.jsp?act=1 +cd +gov.cd + +// cf : http://en.wikipedia.org/wiki/.cf +cf + +// cg : http://en.wikipedia.org/wiki/.cg +cg + +// ch : http://en.wikipedia.org/wiki/.ch +ch + +// ci : http://en.wikipedia.org/wiki/.ci +// http://www.nic.ci/index.php?page=charte +ci +org.ci +or.ci +com.ci +co.ci +edu.ci +ed.ci +ac.ci +net.ci +go.ci +asso.ci +aéroport.ci +int.ci +presse.ci +md.ci +gouv.ci + +// ck : http://en.wikipedia.org/wiki/.ck +*.ck +!www.ck + +// cl : http://en.wikipedia.org/wiki/.cl +cl +gov.cl +gob.cl +co.cl +mil.cl + +// cm : http://en.wikipedia.org/wiki/.cm +cm +gov.cm + +// cn : http://en.wikipedia.org/wiki/.cn +// Submitted by registry 2008-06-11 +cn +ac.cn +com.cn +edu.cn +gov.cn +net.cn +org.cn +mil.cn +公司.cn +网络.cn +網絡.cn +// cn geographic names +ah.cn +bj.cn +cq.cn +fj.cn +gd.cn +gs.cn +gz.cn +gx.cn +ha.cn +hb.cn +he.cn +hi.cn +hl.cn +hn.cn +jl.cn +js.cn +jx.cn +ln.cn +nm.cn +nx.cn +qh.cn +sc.cn +sd.cn +sh.cn +sn.cn +sx.cn +tj.cn +xj.cn +xz.cn +yn.cn +zj.cn +hk.cn +mo.cn +tw.cn + +// co : http://en.wikipedia.org/wiki/.co +// Submitted by registry 2008-06-11 +co +arts.co +com.co +edu.co +firm.co +gov.co +info.co +int.co +mil.co +net.co +nom.co +org.co +rec.co +web.co + +// com : http://en.wikipedia.org/wiki/.com +com + +// coop : http://en.wikipedia.org/wiki/.coop +coop + +// cr : http://www.nic.cr/niccr_publico/showRegistroDominiosScreen.do +cr +ac.cr +co.cr +ed.cr +fi.cr +go.cr +or.cr +sa.cr + +// cu : http://en.wikipedia.org/wiki/.cu +cu +com.cu +edu.cu +org.cu +net.cu +gov.cu +inf.cu + +// cv : http://en.wikipedia.org/wiki/.cv +cv + +// cw : http://www.una.cw/cw_registry/ +// Confirmed by registry 2013-03-26 +cw +com.cw +edu.cw +net.cw +org.cw + +// cx : http://en.wikipedia.org/wiki/.cx +// list of other 2nd level tlds ? +cx +gov.cx + +// cy : http://en.wikipedia.org/wiki/.cy +*.cy + +// cz : http://en.wikipedia.org/wiki/.cz +cz + +// de : http://en.wikipedia.org/wiki/.de +// Confirmed by registry (with technical +// reservations) 2008-07-01 +de + +// dj : http://en.wikipedia.org/wiki/.dj +dj + +// dk : http://en.wikipedia.org/wiki/.dk +// Confirmed by registry 2008-06-17 +dk + +// dm : http://en.wikipedia.org/wiki/.dm +dm +com.dm +net.dm +org.dm +edu.dm +gov.dm + +// do : http://en.wikipedia.org/wiki/.do +do +art.do +com.do +edu.do +gob.do +gov.do +mil.do +net.do +org.do +sld.do +web.do + +// dz : http://en.wikipedia.org/wiki/.dz +dz +com.dz +org.dz +net.dz +gov.dz +edu.dz +asso.dz +pol.dz +art.dz + +// ec : http://www.nic.ec/reg/paso1.asp +// Submitted by registry 2008-07-04 +ec +com.ec +info.ec +net.ec +fin.ec +k12.ec +med.ec +pro.ec +org.ec +edu.ec +gov.ec +gob.ec +mil.ec + +// edu : http://en.wikipedia.org/wiki/.edu +edu + +// ee : http://www.eenet.ee/EENet/dom_reeglid.html#lisa_B +ee +edu.ee +gov.ee +riik.ee +lib.ee +med.ee +com.ee +pri.ee +aip.ee +org.ee +fie.ee + +// eg : http://en.wikipedia.org/wiki/.eg +eg +com.eg +edu.eg +eun.eg +gov.eg +mil.eg +name.eg +net.eg +org.eg +sci.eg + +// er : http://en.wikipedia.org/wiki/.er +*.er + +// es : https://www.nic.es/site_ingles/ingles/dominios/index.html +es +com.es +nom.es +org.es +gob.es +edu.es + +// et : http://en.wikipedia.org/wiki/.et +*.et + +// eu : http://en.wikipedia.org/wiki/.eu +eu + +// fi : http://en.wikipedia.org/wiki/.fi +fi +// aland.fi : http://en.wikipedia.org/wiki/.ax +// This domain is being phased out in favor of .ax. As there are still many +// domains under aland.fi, we still keep it on the list until aland.fi is +// completely removed. +// TODO: Check for updates (expected to be phased out around Q1/2009) +aland.fi + +// fj : http://en.wikipedia.org/wiki/.fj +*.fj + +// fk : http://en.wikipedia.org/wiki/.fk +*.fk + +// fm : http://en.wikipedia.org/wiki/.fm +fm + +// fo : http://en.wikipedia.org/wiki/.fo +fo + +// fr : http://www.afnic.fr/ +// domaines descriptifs : http://www.afnic.fr/obtenir/chartes/nommage-fr/annexe-descriptifs +fr +com.fr +asso.fr +nom.fr +prd.fr +presse.fr +tm.fr +// domaines sectoriels : http://www.afnic.fr/obtenir/chartes/nommage-fr/annexe-sectoriels +aeroport.fr +assedic.fr +avocat.fr +avoues.fr +cci.fr +chambagri.fr +chirurgiens-dentistes.fr +experts-comptables.fr +geometre-expert.fr +gouv.fr +greta.fr +huissier-justice.fr +medecin.fr +notaires.fr +pharmacien.fr +port.fr +veterinaire.fr + +// ga : http://en.wikipedia.org/wiki/.ga +ga + +// gb : This registry is effectively dormant +// Submitted by registry 2008-06-12 +gb + +// gd : http://en.wikipedia.org/wiki/.gd +gd + +// ge : http://www.nic.net.ge/policy_en.pdf +ge +com.ge +edu.ge +gov.ge +org.ge +mil.ge +net.ge +pvt.ge + +// gf : http://en.wikipedia.org/wiki/.gf +gf + +// gg : http://www.channelisles.net/register-domains/ +// Confirmed by registry 2013-11-28 +gg +co.gg +net.gg +org.gg + +// gh : http://en.wikipedia.org/wiki/.gh +// see also: http://www.nic.gh/reg_now.php +// Although domains directly at second level are not possible at the moment, +// they have been possible for some time and may come back. +gh +com.gh +edu.gh +gov.gh +org.gh +mil.gh + +// gi : http://www.nic.gi/rules.html +gi +com.gi +ltd.gi +gov.gi +mod.gi +edu.gi +org.gi + +// gl : http://en.wikipedia.org/wiki/.gl +// http://nic.gl +gl + +// gm : http://www.nic.gm/htmlpages%5Cgm-policy.htm +gm + +// gn : http://psg.com/dns/gn/gn.txt +// Submitted by registry 2008-06-17 +gn +ac.gn +com.gn +edu.gn +gov.gn +org.gn +net.gn + +// gov : http://en.wikipedia.org/wiki/.gov +gov + +// gp : http://www.nic.gp/index.php?lang=en +gp +com.gp +net.gp +mobi.gp +edu.gp +org.gp +asso.gp + +// gq : http://en.wikipedia.org/wiki/.gq +gq + +// gr : https://grweb.ics.forth.gr/english/1617-B-2005.html +// Submitted by registry 2008-06-09 +gr +com.gr +edu.gr +net.gr +org.gr +gov.gr + +// gs : http://en.wikipedia.org/wiki/.gs +gs + +// gt : http://www.gt/politicas_de_registro.html +gt +com.gt +edu.gt +gob.gt +ind.gt +mil.gt +net.gt +org.gt + +// gu : http://gadao.gov.gu/registration.txt +*.gu + +// gw : http://en.wikipedia.org/wiki/.gw +gw + +// gy : http://en.wikipedia.org/wiki/.gy +// http://registry.gy/ +gy +co.gy +com.gy +net.gy + +// hk : https://www.hkdnr.hk +// Submitted by registry 2008-06-11 +hk +com.hk +edu.hk +gov.hk +idv.hk +net.hk +org.hk +公司.hk +教育.hk +敎育.hk +政府.hk +個人.hk +个人.hk +箇人.hk +網络.hk +网络.hk +组織.hk +網絡.hk +网絡.hk +组织.hk +組織.hk +組织.hk + +// hm : http://en.wikipedia.org/wiki/.hm +hm + +// hn : http://www.nic.hn/politicas/ps02,,05.html +hn +com.hn +edu.hn +org.hn +net.hn +mil.hn +gob.hn + +// hr : http://www.dns.hr/documents/pdf/HRTLD-regulations.pdf +hr +iz.hr +from.hr +name.hr +com.hr + +// ht : http://www.nic.ht/info/charte.cfm +ht +com.ht +shop.ht +firm.ht +info.ht +adult.ht +net.ht +pro.ht +org.ht +med.ht +art.ht +coop.ht +pol.ht +asso.ht +edu.ht +rel.ht +gouv.ht +perso.ht + +// hu : http://www.domain.hu/domain/English/sld.html +// Confirmed by registry 2008-06-12 +hu +co.hu +info.hu +org.hu +priv.hu +sport.hu +tm.hu +2000.hu +agrar.hu +bolt.hu +casino.hu +city.hu +erotica.hu +erotika.hu +film.hu +forum.hu +games.hu +hotel.hu +ingatlan.hu +jogasz.hu +konyvelo.hu +lakas.hu +media.hu +news.hu +reklam.hu +sex.hu +shop.hu +suli.hu +szex.hu +tozsde.hu +utazas.hu +video.hu + +// id : https://register.pandi.or.id/ +id +ac.id +biz.id +co.id +desa.id +go.id +mil.id +my.id +net.id +or.id +sch.id +web.id + +// ie : http://en.wikipedia.org/wiki/.ie +ie +gov.ie + +// il : http://en.wikipedia.org/wiki/.il +*.il + +// im : https://www.nic.im/ +// Submitted by registry 2013-11-15 +im +ac.im +co.im +com.im +ltd.co.im +net.im +org.im +plc.co.im +tt.im +tv.im + +// in : http://en.wikipedia.org/wiki/.in +// see also: http://www.inregistry.in/policies/ +// Please note, that nic.in is not an offical eTLD, but used by most +// government institutions. +in +co.in +firm.in +net.in +org.in +gen.in +ind.in +nic.in +ac.in +edu.in +res.in +gov.in +mil.in + +// info : http://en.wikipedia.org/wiki/.info +info + +// int : http://en.wikipedia.org/wiki/.int +// Confirmed by registry 2008-06-18 +int +eu.int + +// io : http://www.nic.io/rules.html +// list of other 2nd level tlds ? +io +com.io + +// iq : http://www.cmc.iq/english/iq/iqregister1.htm +iq +gov.iq +edu.iq +mil.iq +com.iq +org.iq +net.iq + +// ir : http://www.nic.ir/Terms_and_Conditions_ir,_Appendix_1_Domain_Rules +// Also see http://www.nic.ir/Internationalized_Domain_Names +// Two .ir entries added at request of , 2010-04-16 +ir +ac.ir +co.ir +gov.ir +id.ir +net.ir +org.ir +sch.ir +// xn--mgba3a4f16a.ir (.ir, Persian YEH) +ایران.ir +// xn--mgba3a4fra.ir (.ir, Arabic YEH) +ايران.ir + +// is : http://www.isnic.is/domain/rules.php +// Confirmed by registry 2008-12-06 +is +net.is +com.is +edu.is +gov.is +org.is +int.is + +// it : http://en.wikipedia.org/wiki/.it +it +gov.it +edu.it +// Reserved geo-names: +// http://www.nic.it/documenti/regolamenti-e-linee-guida/regolamento-assegnazione-versione-6.0.pdf +// There is also a list of reserved geo-names corresponding to Italian municipalities +// http://www.nic.it/documenti/appendice-c.pdf, but it is not included here. +// Regions +abr.it +abruzzo.it +aosta-valley.it +aostavalley.it +bas.it +basilicata.it +cal.it +calabria.it +cam.it +campania.it +emilia-romagna.it +emiliaromagna.it +emr.it +friuli-v-giulia.it +friuli-ve-giulia.it +friuli-vegiulia.it +friuli-venezia-giulia.it +friuli-veneziagiulia.it +friuli-vgiulia.it +friuliv-giulia.it +friulive-giulia.it +friulivegiulia.it +friulivenezia-giulia.it +friuliveneziagiulia.it +friulivgiulia.it +fvg.it +laz.it +lazio.it +lig.it +liguria.it +lom.it +lombardia.it +lombardy.it +lucania.it +mar.it +marche.it +mol.it +molise.it +piedmont.it +piemonte.it +pmn.it +pug.it +puglia.it +sar.it +sardegna.it +sardinia.it +sic.it +sicilia.it +sicily.it +taa.it +tos.it +toscana.it +trentino-a-adige.it +trentino-aadige.it +trentino-alto-adige.it +trentino-altoadige.it +trentino-s-tirol.it +trentino-stirol.it +trentino-sud-tirol.it +trentino-sudtirol.it +trentino-sued-tirol.it +trentino-suedtirol.it +trentinoa-adige.it +trentinoaadige.it +trentinoalto-adige.it +trentinoaltoadige.it +trentinos-tirol.it +trentinostirol.it +trentinosud-tirol.it +trentinosudtirol.it +trentinosued-tirol.it +trentinosuedtirol.it +tuscany.it +umb.it +umbria.it +val-d-aosta.it +val-daosta.it +vald-aosta.it +valdaosta.it +valle-aosta.it +valle-d-aosta.it +valle-daosta.it +valleaosta.it +valled-aosta.it +valledaosta.it +vallee-aoste.it +valleeaoste.it +vao.it +vda.it +ven.it +veneto.it +// Provinces +ag.it +agrigento.it +al.it +alessandria.it +alto-adige.it +altoadige.it +an.it +ancona.it +andria-barletta-trani.it +andria-trani-barletta.it +andriabarlettatrani.it +andriatranibarletta.it +ao.it +aosta.it +aoste.it +ap.it +aq.it +aquila.it +ar.it +arezzo.it +ascoli-piceno.it +ascolipiceno.it +asti.it +at.it +av.it +avellino.it +ba.it +balsan.it +bari.it +barletta-trani-andria.it +barlettatraniandria.it +belluno.it +benevento.it +bergamo .it +bg.it +bi.it +biella.it +bl.it +bn.it +bo.it +bologna.it +bolzano.it +bozen.it +br.it +brescia.it +brindisi.it +bs.it +bt.it +bz.it +ca.it +cagliari.it +caltanissetta.it +campidano-medio.it +campidanomedio.it +campobasso.it +carbonia-iglesias.it +carboniaiglesias.it +carrara-massa.it +carraramassa.it +caserta.it +catania.it +catanzaro.it +cb.it +ce.it +cesena-forli.it +cesenaforli.it +ch.it +chieti.it +ci.it +cl.it +cn.it +co.it +como.it +cosenza.it +cr.it +cremona.it +crotone.it +cs.it +ct.it +cuneo.it +cz.it +dell-ogliastra.it +dellogliastra.it +en.it +enna.it +fc.it +fe.it +fermo.it +ferrara.it +fg.it +fi.it +firenze.it +florence.it +fm.it +foggia.it +forli-cesena.it +forlicesena.it +fr.it +frosinone.it +ge.it +genoa.it +genova.it +go.it +gorizia.it +gr.it +grosseto.it +iglesias-carbonia.it +iglesiascarbonia.it +im.it +imperia.it +is.it +isernia.it +kr.it +la-spezia.it +laquila.it +laspezia.it +latina.it +lc.it +le.it +lecce.it +lecco.it +li.it +livorno.it +lo.it +lodi.it +lt.it +lu.it +lucca.it +macerata.it +mantova.it +massa-carrara.it +massacarrara.it +matera.it +mb.it +mc.it +me.it +medio-campidano.it +mediocampidano.it +messina.it +mi.it +milan.it +milano.it +mn.it +mo.it +modena.it +monza-brianza.it +monza-e-della-brianza.it +monza.it +monzabrianza.it +monzaebrianza.it +monzaedellabrianza.it +ms.it +mt.it +na.it +naples.it +napoli.it +no.it +novara.it +nu.it +nuoro.it +og.it +ogliastra.it +olbia-tempio.it +olbiatempio.it +or.it +oristano.it +ot.it +pa.it +padova.it +padua.it +palermo.it +parma.it +pavia.it +pc.it +pd.it +pe.it +perugia.it +pesaro-urbino.it +pesarourbino.it +pescara.it +pg.it +pi.it +piacenza.it +pisa.it +pistoia.it +pn.it +po.it +pordenone.it +potenza .it +pr.it +prato.it +pt.it +pu.it +pv.it +pz.it +ra.it +ragusa.it +ravenna.it +rc.it +re.it +reggio-calabria.it +reggio-emilia.it +reggiocalabria.it +reggioemilia.it +rg.it +ri.it +rieti.it +rimini.it +rm.it +rn.it +ro.it +roma.it +rome.it +rovigo.it +sa.it +salerno.it +sassari.it +savona.it +si.it +siena.it +siracusa.it +so.it +sondrio.it +sp.it +sr.it +ss.it +suedtirol.it +sv.it +ta.it +taranto.it +te.it +tempio-olbia.it +tempioolbia.it +teramo.it +terni.it +tn.it +to.it +torino.it +tp.it +tr.it +trani-andria-barletta.it +trani-barletta-andria.it +traniandriabarletta.it +tranibarlettaandria.it +trapani.it +trentino.it +trento.it +treviso.it +trieste.it +ts.it +turin.it +tv.it +ud.it +udine.it +urbino-pesaro.it +urbinopesaro.it +va.it +varese.it +vb.it +vc.it +ve.it +venezia.it +venice.it +verbania.it +vercelli.it +verona.it +vi.it +vibo-valentia.it +vibovalentia.it +vicenza.it +viterbo.it +vr.it +vs.it +vt.it +vv.it + +// je : http://www.channelisles.net/register-domains/ +// Confirmed by registry 2013-11-28 +je +co.je +net.je +org.je + +// jm : http://www.com.jm/register.html +*.jm + +// jo : http://www.dns.jo/Registration_policy.aspx +jo +com.jo +org.jo +net.jo +edu.jo +sch.jo +gov.jo +mil.jo +name.jo + +// jobs : http://en.wikipedia.org/wiki/.jobs +jobs + +// jp : http://en.wikipedia.org/wiki/.jp +// http://jprs.co.jp/en/jpdomain.html +// Submitted by registry 2014-02-28 +jp +// jp organizational type names +ac.jp +ad.jp +co.jp +ed.jp +go.jp +gr.jp +lg.jp +ne.jp +or.jp +// jp preficture type names +aichi.jp +akita.jp +aomori.jp +chiba.jp +ehime.jp +fukui.jp +fukuoka.jp +fukushima.jp +gifu.jp +gunma.jp +hiroshima.jp +hokkaido.jp +hyogo.jp +ibaraki.jp +ishikawa.jp +iwate.jp +kagawa.jp +kagoshima.jp +kanagawa.jp +kochi.jp +kumamoto.jp +kyoto.jp +mie.jp +miyagi.jp +miyazaki.jp +nagano.jp +nagasaki.jp +nara.jp +niigata.jp +oita.jp +okayama.jp +okinawa.jp +osaka.jp +saga.jp +saitama.jp +shiga.jp +shimane.jp +shizuoka.jp +tochigi.jp +tokushima.jp +tokyo.jp +tottori.jp +toyama.jp +wakayama.jp +yamagata.jp +yamaguchi.jp +yamanashi.jp +// jp geographic type names +// http://jprs.jp/doc/rule/saisoku-1.html +*.kawasaki.jp +*.kitakyushu.jp +*.kobe.jp +*.nagoya.jp +*.sapporo.jp +*.sendai.jp +*.yokohama.jp +!city.kawasaki.jp +!city.kitakyushu.jp +!city.kobe.jp +!city.nagoya.jp +!city.sapporo.jp +!city.sendai.jp +!city.yokohama.jp +// 4th level registration +aisai.aichi.jp +ama.aichi.jp +anjo.aichi.jp +asuke.aichi.jp +chiryu.aichi.jp +chita.aichi.jp +fuso.aichi.jp +gamagori.aichi.jp +handa.aichi.jp +hazu.aichi.jp +hekinan.aichi.jp +higashiura.aichi.jp +ichinomiya.aichi.jp +inazawa.aichi.jp +inuyama.aichi.jp +isshiki.aichi.jp +iwakura.aichi.jp +kanie.aichi.jp +kariya.aichi.jp +kasugai.aichi.jp +kira.aichi.jp +kiyosu.aichi.jp +komaki.aichi.jp +konan.aichi.jp +kota.aichi.jp +mihama.aichi.jp +miyoshi.aichi.jp +nishio.aichi.jp +nisshin.aichi.jp +obu.aichi.jp +oguchi.aichi.jp +oharu.aichi.jp +okazaki.aichi.jp +owariasahi.aichi.jp +seto.aichi.jp +shikatsu.aichi.jp +shinshiro.aichi.jp +shitara.aichi.jp +tahara.aichi.jp +takahama.aichi.jp +tobishima.aichi.jp +toei.aichi.jp +togo.aichi.jp +tokai.aichi.jp +tokoname.aichi.jp +toyoake.aichi.jp +toyohashi.aichi.jp +toyokawa.aichi.jp +toyone.aichi.jp +toyota.aichi.jp +tsushima.aichi.jp +yatomi.aichi.jp +akita.akita.jp +daisen.akita.jp +fujisato.akita.jp +gojome.akita.jp +hachirogata.akita.jp +happou.akita.jp +higashinaruse.akita.jp +honjo.akita.jp +honjyo.akita.jp +ikawa.akita.jp +kamikoani.akita.jp +kamioka.akita.jp +katagami.akita.jp +kazuno.akita.jp +kitaakita.akita.jp +kosaka.akita.jp +kyowa.akita.jp +misato.akita.jp +mitane.akita.jp +moriyoshi.akita.jp +nikaho.akita.jp +noshiro.akita.jp +odate.akita.jp +oga.akita.jp +ogata.akita.jp +semboku.akita.jp +yokote.akita.jp +yurihonjo.akita.jp +aomori.aomori.jp +gonohe.aomori.jp +hachinohe.aomori.jp +hashikami.aomori.jp +hiranai.aomori.jp +hirosaki.aomori.jp +itayanagi.aomori.jp +kuroishi.aomori.jp +misawa.aomori.jp +mutsu.aomori.jp +nakadomari.aomori.jp +noheji.aomori.jp +oirase.aomori.jp +owani.aomori.jp +rokunohe.aomori.jp +sannohe.aomori.jp +shichinohe.aomori.jp +shingo.aomori.jp +takko.aomori.jp +towada.aomori.jp +tsugaru.aomori.jp +tsuruta.aomori.jp +abiko.chiba.jp +asahi.chiba.jp +chonan.chiba.jp +chosei.chiba.jp +choshi.chiba.jp +chuo.chiba.jp +funabashi.chiba.jp +futtsu.chiba.jp +hanamigawa.chiba.jp +ichihara.chiba.jp +ichikawa.chiba.jp +ichinomiya.chiba.jp +inzai.chiba.jp +isumi.chiba.jp +kamagaya.chiba.jp +kamogawa.chiba.jp +kashiwa.chiba.jp +katori.chiba.jp +katsuura.chiba.jp +kimitsu.chiba.jp +kisarazu.chiba.jp +kozaki.chiba.jp +kujukuri.chiba.jp +kyonan.chiba.jp +matsudo.chiba.jp +midori.chiba.jp +mihama.chiba.jp +minamiboso.chiba.jp +mobara.chiba.jp +mutsuzawa.chiba.jp +nagara.chiba.jp +nagareyama.chiba.jp +narashino.chiba.jp +narita.chiba.jp +noda.chiba.jp +oamishirasato.chiba.jp +omigawa.chiba.jp +onjuku.chiba.jp +otaki.chiba.jp +sakae.chiba.jp +sakura.chiba.jp +shimofusa.chiba.jp +shirako.chiba.jp +shiroi.chiba.jp +shisui.chiba.jp +sodegaura.chiba.jp +sosa.chiba.jp +tako.chiba.jp +tateyama.chiba.jp +togane.chiba.jp +tohnosho.chiba.jp +tomisato.chiba.jp +urayasu.chiba.jp +yachimata.chiba.jp +yachiyo.chiba.jp +yokaichiba.chiba.jp +yokoshibahikari.chiba.jp +yotsukaido.chiba.jp +ainan.ehime.jp +honai.ehime.jp +ikata.ehime.jp +imabari.ehime.jp +iyo.ehime.jp +kamijima.ehime.jp +kihoku.ehime.jp +kumakogen.ehime.jp +masaki.ehime.jp +matsuno.ehime.jp +matsuyama.ehime.jp +namikata.ehime.jp +niihama.ehime.jp +ozu.ehime.jp +saijo.ehime.jp +seiyo.ehime.jp +shikokuchuo.ehime.jp +tobe.ehime.jp +toon.ehime.jp +uchiko.ehime.jp +uwajima.ehime.jp +yawatahama.ehime.jp +echizen.fukui.jp +eiheiji.fukui.jp +fukui.fukui.jp +ikeda.fukui.jp +katsuyama.fukui.jp +mihama.fukui.jp +minamiechizen.fukui.jp +obama.fukui.jp +ohi.fukui.jp +ono.fukui.jp +sabae.fukui.jp +sakai.fukui.jp +takahama.fukui.jp +tsuruga.fukui.jp +wakasa.fukui.jp +ashiya.fukuoka.jp +buzen.fukuoka.jp +chikugo.fukuoka.jp +chikuho.fukuoka.jp +chikujo.fukuoka.jp +chikushino.fukuoka.jp +chikuzen.fukuoka.jp +chuo.fukuoka.jp +dazaifu.fukuoka.jp +fukuchi.fukuoka.jp +hakata.fukuoka.jp +higashi.fukuoka.jp +hirokawa.fukuoka.jp +hisayama.fukuoka.jp +iizuka.fukuoka.jp +inatsuki.fukuoka.jp +kaho.fukuoka.jp +kasuga.fukuoka.jp +kasuya.fukuoka.jp +kawara.fukuoka.jp +keisen.fukuoka.jp +koga.fukuoka.jp +kurate.fukuoka.jp +kurogi.fukuoka.jp +kurume.fukuoka.jp +minami.fukuoka.jp +miyako.fukuoka.jp +miyama.fukuoka.jp +miyawaka.fukuoka.jp +mizumaki.fukuoka.jp +munakata.fukuoka.jp +nakagawa.fukuoka.jp +nakama.fukuoka.jp +nishi.fukuoka.jp +nogata.fukuoka.jp +ogori.fukuoka.jp +okagaki.fukuoka.jp +okawa.fukuoka.jp +oki.fukuoka.jp +omuta.fukuoka.jp +onga.fukuoka.jp +onojo.fukuoka.jp +oto.fukuoka.jp +saigawa.fukuoka.jp +sasaguri.fukuoka.jp +shingu.fukuoka.jp +shinyoshitomi.fukuoka.jp +shonai.fukuoka.jp +soeda.fukuoka.jp +sue.fukuoka.jp +tachiarai.fukuoka.jp +tagawa.fukuoka.jp +takata.fukuoka.jp +toho.fukuoka.jp +toyotsu.fukuoka.jp +tsuiki.fukuoka.jp +ukiha.fukuoka.jp +umi.fukuoka.jp +usui.fukuoka.jp +yamada.fukuoka.jp +yame.fukuoka.jp +yanagawa.fukuoka.jp +yukuhashi.fukuoka.jp +aizubange.fukushima.jp +aizumisato.fukushima.jp +aizuwakamatsu.fukushima.jp +asakawa.fukushima.jp +bandai.fukushima.jp +date.fukushima.jp +fukushima.fukushima.jp +furudono.fukushima.jp +futaba.fukushima.jp +hanawa.fukushima.jp +higashi.fukushima.jp +hirata.fukushima.jp +hirono.fukushima.jp +iitate.fukushima.jp +inawashiro.fukushima.jp +ishikawa.fukushima.jp +iwaki.fukushima.jp +izumizaki.fukushima.jp +kagamiishi.fukushima.jp +kaneyama.fukushima.jp +kawamata.fukushima.jp +kitakata.fukushima.jp +kitashiobara.fukushima.jp +koori.fukushima.jp +koriyama.fukushima.jp +kunimi.fukushima.jp +miharu.fukushima.jp +mishima.fukushima.jp +namie.fukushima.jp +nango.fukushima.jp +nishiaizu.fukushima.jp +nishigo.fukushima.jp +okuma.fukushima.jp +omotego.fukushima.jp +ono.fukushima.jp +otama.fukushima.jp +samegawa.fukushima.jp +shimogo.fukushima.jp +shirakawa.fukushima.jp +showa.fukushima.jp +soma.fukushima.jp +sukagawa.fukushima.jp +taishin.fukushima.jp +tamakawa.fukushima.jp +tanagura.fukushima.jp +tenei.fukushima.jp +yabuki.fukushima.jp +yamato.fukushima.jp +yamatsuri.fukushima.jp +yanaizu.fukushima.jp +yugawa.fukushima.jp +anpachi.gifu.jp +ena.gifu.jp +gifu.gifu.jp +ginan.gifu.jp +godo.gifu.jp +gujo.gifu.jp +hashima.gifu.jp +hichiso.gifu.jp +hida.gifu.jp +higashishirakawa.gifu.jp +ibigawa.gifu.jp +ikeda.gifu.jp +kakamigahara.gifu.jp +kani.gifu.jp +kasahara.gifu.jp +kasamatsu.gifu.jp +kawaue.gifu.jp +kitagata.gifu.jp +mino.gifu.jp +minokamo.gifu.jp +mitake.gifu.jp +mizunami.gifu.jp +motosu.gifu.jp +nakatsugawa.gifu.jp +ogaki.gifu.jp +sakahogi.gifu.jp +seki.gifu.jp +sekigahara.gifu.jp +shirakawa.gifu.jp +tajimi.gifu.jp +takayama.gifu.jp +tarui.gifu.jp +toki.gifu.jp +tomika.gifu.jp +wanouchi.gifu.jp +yamagata.gifu.jp +yaotsu.gifu.jp +yoro.gifu.jp +annaka.gunma.jp +chiyoda.gunma.jp +fujioka.gunma.jp +higashiagatsuma.gunma.jp +isesaki.gunma.jp +itakura.gunma.jp +kanna.gunma.jp +kanra.gunma.jp +katashina.gunma.jp +kawaba.gunma.jp +kiryu.gunma.jp +kusatsu.gunma.jp +maebashi.gunma.jp +meiwa.gunma.jp +midori.gunma.jp +minakami.gunma.jp +naganohara.gunma.jp +nakanojo.gunma.jp +nanmoku.gunma.jp +numata.gunma.jp +oizumi.gunma.jp +ora.gunma.jp +ota.gunma.jp +shibukawa.gunma.jp +shimonita.gunma.jp +shinto.gunma.jp +showa.gunma.jp +takasaki.gunma.jp +takayama.gunma.jp +tamamura.gunma.jp +tatebayashi.gunma.jp +tomioka.gunma.jp +tsukiyono.gunma.jp +tsumagoi.gunma.jp +ueno.gunma.jp +yoshioka.gunma.jp +asaminami.hiroshima.jp +daiwa.hiroshima.jp +etajima.hiroshima.jp +fuchu.hiroshima.jp +fukuyama.hiroshima.jp +hatsukaichi.hiroshima.jp +higashihiroshima.hiroshima.jp +hongo.hiroshima.jp +jinsekikogen.hiroshima.jp +kaita.hiroshima.jp +kui.hiroshima.jp +kumano.hiroshima.jp +kure.hiroshima.jp +mihara.hiroshima.jp +miyoshi.hiroshima.jp +naka.hiroshima.jp +onomichi.hiroshima.jp +osakikamijima.hiroshima.jp +otake.hiroshima.jp +saka.hiroshima.jp +sera.hiroshima.jp +seranishi.hiroshima.jp +shinichi.hiroshima.jp +shobara.hiroshima.jp +takehara.hiroshima.jp +abashiri.hokkaido.jp +abira.hokkaido.jp +aibetsu.hokkaido.jp +akabira.hokkaido.jp +akkeshi.hokkaido.jp +asahikawa.hokkaido.jp +ashibetsu.hokkaido.jp +ashoro.hokkaido.jp +assabu.hokkaido.jp +atsuma.hokkaido.jp +bibai.hokkaido.jp +biei.hokkaido.jp +bifuka.hokkaido.jp +bihoro.hokkaido.jp +biratori.hokkaido.jp +chippubetsu.hokkaido.jp +chitose.hokkaido.jp +date.hokkaido.jp +ebetsu.hokkaido.jp +embetsu.hokkaido.jp +eniwa.hokkaido.jp +erimo.hokkaido.jp +esan.hokkaido.jp +esashi.hokkaido.jp +fukagawa.hokkaido.jp +fukushima.hokkaido.jp +furano.hokkaido.jp +furubira.hokkaido.jp +haboro.hokkaido.jp +hakodate.hokkaido.jp +hamatonbetsu.hokkaido.jp +hidaka.hokkaido.jp +higashikagura.hokkaido.jp +higashikawa.hokkaido.jp +hiroo.hokkaido.jp +hokuryu.hokkaido.jp +hokuto.hokkaido.jp +honbetsu.hokkaido.jp +horokanai.hokkaido.jp +horonobe.hokkaido.jp +ikeda.hokkaido.jp +imakane.hokkaido.jp +ishikari.hokkaido.jp +iwamizawa.hokkaido.jp +iwanai.hokkaido.jp +kamifurano.hokkaido.jp +kamikawa.hokkaido.jp +kamishihoro.hokkaido.jp +kamisunagawa.hokkaido.jp +kamoenai.hokkaido.jp +kayabe.hokkaido.jp +kembuchi.hokkaido.jp +kikonai.hokkaido.jp +kimobetsu.hokkaido.jp +kitahiroshima.hokkaido.jp +kitami.hokkaido.jp +kiyosato.hokkaido.jp +koshimizu.hokkaido.jp +kunneppu.hokkaido.jp +kuriyama.hokkaido.jp +kuromatsunai.hokkaido.jp +kushiro.hokkaido.jp +kutchan.hokkaido.jp +kyowa.hokkaido.jp +mashike.hokkaido.jp +matsumae.hokkaido.jp +mikasa.hokkaido.jp +minamifurano.hokkaido.jp +mombetsu.hokkaido.jp +moseushi.hokkaido.jp +mukawa.hokkaido.jp +muroran.hokkaido.jp +naie.hokkaido.jp +nakagawa.hokkaido.jp +nakasatsunai.hokkaido.jp +nakatombetsu.hokkaido.jp +nanae.hokkaido.jp +nanporo.hokkaido.jp +nayoro.hokkaido.jp +nemuro.hokkaido.jp +niikappu.hokkaido.jp +niki.hokkaido.jp +nishiokoppe.hokkaido.jp +noboribetsu.hokkaido.jp +numata.hokkaido.jp +obihiro.hokkaido.jp +obira.hokkaido.jp +oketo.hokkaido.jp +okoppe.hokkaido.jp +otaru.hokkaido.jp +otobe.hokkaido.jp +otofuke.hokkaido.jp +otoineppu.hokkaido.jp +oumu.hokkaido.jp +ozora.hokkaido.jp +pippu.hokkaido.jp +rankoshi.hokkaido.jp +rebun.hokkaido.jp +rikubetsu.hokkaido.jp +rishiri.hokkaido.jp +rishirifuji.hokkaido.jp +saroma.hokkaido.jp +sarufutsu.hokkaido.jp +shakotan.hokkaido.jp +shari.hokkaido.jp +shibecha.hokkaido.jp +shibetsu.hokkaido.jp +shikabe.hokkaido.jp +shikaoi.hokkaido.jp +shimamaki.hokkaido.jp +shimizu.hokkaido.jp +shimokawa.hokkaido.jp +shinshinotsu.hokkaido.jp +shintoku.hokkaido.jp +shiranuka.hokkaido.jp +shiraoi.hokkaido.jp +shiriuchi.hokkaido.jp +sobetsu.hokkaido.jp +sunagawa.hokkaido.jp +taiki.hokkaido.jp +takasu.hokkaido.jp +takikawa.hokkaido.jp +takinoue.hokkaido.jp +teshikaga.hokkaido.jp +tobetsu.hokkaido.jp +tohma.hokkaido.jp +tomakomai.hokkaido.jp +tomari.hokkaido.jp +toya.hokkaido.jp +toyako.hokkaido.jp +toyotomi.hokkaido.jp +toyoura.hokkaido.jp +tsubetsu.hokkaido.jp +tsukigata.hokkaido.jp +urakawa.hokkaido.jp +urausu.hokkaido.jp +uryu.hokkaido.jp +utashinai.hokkaido.jp +wakkanai.hokkaido.jp +wassamu.hokkaido.jp +yakumo.hokkaido.jp +yoichi.hokkaido.jp +aioi.hyogo.jp +akashi.hyogo.jp +ako.hyogo.jp +amagasaki.hyogo.jp +aogaki.hyogo.jp +asago.hyogo.jp +ashiya.hyogo.jp +awaji.hyogo.jp +fukusaki.hyogo.jp +goshiki.hyogo.jp +harima.hyogo.jp +himeji.hyogo.jp +ichikawa.hyogo.jp +inagawa.hyogo.jp +itami.hyogo.jp +kakogawa.hyogo.jp +kamigori.hyogo.jp +kamikawa.hyogo.jp +kasai.hyogo.jp +kasuga.hyogo.jp +kawanishi.hyogo.jp +miki.hyogo.jp +minamiawaji.hyogo.jp +nishinomiya.hyogo.jp +nishiwaki.hyogo.jp +ono.hyogo.jp +sanda.hyogo.jp +sannan.hyogo.jp +sasayama.hyogo.jp +sayo.hyogo.jp +shingu.hyogo.jp +shinonsen.hyogo.jp +shiso.hyogo.jp +sumoto.hyogo.jp +taishi.hyogo.jp +taka.hyogo.jp +takarazuka.hyogo.jp +takasago.hyogo.jp +takino.hyogo.jp +tamba.hyogo.jp +tatsuno.hyogo.jp +toyooka.hyogo.jp +yabu.hyogo.jp +yashiro.hyogo.jp +yoka.hyogo.jp +yokawa.hyogo.jp +ami.ibaraki.jp +asahi.ibaraki.jp +bando.ibaraki.jp +chikusei.ibaraki.jp +daigo.ibaraki.jp +fujishiro.ibaraki.jp +hitachi.ibaraki.jp +hitachinaka.ibaraki.jp +hitachiomiya.ibaraki.jp +hitachiota.ibaraki.jp +ibaraki.ibaraki.jp +ina.ibaraki.jp +inashiki.ibaraki.jp +itako.ibaraki.jp +iwama.ibaraki.jp +joso.ibaraki.jp +kamisu.ibaraki.jp +kasama.ibaraki.jp +kashima.ibaraki.jp +kasumigaura.ibaraki.jp +koga.ibaraki.jp +miho.ibaraki.jp +mito.ibaraki.jp +moriya.ibaraki.jp +naka.ibaraki.jp +namegata.ibaraki.jp +oarai.ibaraki.jp +ogawa.ibaraki.jp +omitama.ibaraki.jp +ryugasaki.ibaraki.jp +sakai.ibaraki.jp +sakuragawa.ibaraki.jp +shimodate.ibaraki.jp +shimotsuma.ibaraki.jp +shirosato.ibaraki.jp +sowa.ibaraki.jp +suifu.ibaraki.jp +takahagi.ibaraki.jp +tamatsukuri.ibaraki.jp +tokai.ibaraki.jp +tomobe.ibaraki.jp +tone.ibaraki.jp +toride.ibaraki.jp +tsuchiura.ibaraki.jp +tsukuba.ibaraki.jp +uchihara.ibaraki.jp +ushiku.ibaraki.jp +yachiyo.ibaraki.jp +yamagata.ibaraki.jp +yawara.ibaraki.jp +yuki.ibaraki.jp +anamizu.ishikawa.jp +hakui.ishikawa.jp +hakusan.ishikawa.jp +kaga.ishikawa.jp +kahoku.ishikawa.jp +kanazawa.ishikawa.jp +kawakita.ishikawa.jp +komatsu.ishikawa.jp +nakanoto.ishikawa.jp +nanao.ishikawa.jp +nomi.ishikawa.jp +nonoichi.ishikawa.jp +noto.ishikawa.jp +shika.ishikawa.jp +suzu.ishikawa.jp +tsubata.ishikawa.jp +tsurugi.ishikawa.jp +uchinada.ishikawa.jp +wajima.ishikawa.jp +fudai.iwate.jp +fujisawa.iwate.jp +hanamaki.iwate.jp +hiraizumi.iwate.jp +hirono.iwate.jp +ichinohe.iwate.jp +ichinoseki.iwate.jp +iwaizumi.iwate.jp +iwate.iwate.jp +joboji.iwate.jp +kamaishi.iwate.jp +kanegasaki.iwate.jp +karumai.iwate.jp +kawai.iwate.jp +kitakami.iwate.jp +kuji.iwate.jp +kunohe.iwate.jp +kuzumaki.iwate.jp +miyako.iwate.jp +mizusawa.iwate.jp +morioka.iwate.jp +ninohe.iwate.jp +noda.iwate.jp +ofunato.iwate.jp +oshu.iwate.jp +otsuchi.iwate.jp +rikuzentakata.iwate.jp +shiwa.iwate.jp +shizukuishi.iwate.jp +sumita.iwate.jp +tanohata.iwate.jp +tono.iwate.jp +yahaba.iwate.jp +yamada.iwate.jp +ayagawa.kagawa.jp +higashikagawa.kagawa.jp +kanonji.kagawa.jp +kotohira.kagawa.jp +manno.kagawa.jp +marugame.kagawa.jp +mitoyo.kagawa.jp +naoshima.kagawa.jp +sanuki.kagawa.jp +tadotsu.kagawa.jp +takamatsu.kagawa.jp +tonosho.kagawa.jp +uchinomi.kagawa.jp +utazu.kagawa.jp +zentsuji.kagawa.jp +akune.kagoshima.jp +amami.kagoshima.jp +hioki.kagoshima.jp +isa.kagoshima.jp +isen.kagoshima.jp +izumi.kagoshima.jp +kagoshima.kagoshima.jp +kanoya.kagoshima.jp +kawanabe.kagoshima.jp +kinko.kagoshima.jp +kouyama.kagoshima.jp +makurazaki.kagoshima.jp +matsumoto.kagoshima.jp +minamitane.kagoshima.jp +nakatane.kagoshima.jp +nishinoomote.kagoshima.jp +satsumasendai.kagoshima.jp +soo.kagoshima.jp +tarumizu.kagoshima.jp +yusui.kagoshima.jp +aikawa.kanagawa.jp +atsugi.kanagawa.jp +ayase.kanagawa.jp +chigasaki.kanagawa.jp +ebina.kanagawa.jp +fujisawa.kanagawa.jp +hadano.kanagawa.jp +hakone.kanagawa.jp +hiratsuka.kanagawa.jp +isehara.kanagawa.jp +kaisei.kanagawa.jp +kamakura.kanagawa.jp +kiyokawa.kanagawa.jp +matsuda.kanagawa.jp +minamiashigara.kanagawa.jp +miura.kanagawa.jp +nakai.kanagawa.jp +ninomiya.kanagawa.jp +odawara.kanagawa.jp +oi.kanagawa.jp +oiso.kanagawa.jp +sagamihara.kanagawa.jp +samukawa.kanagawa.jp +tsukui.kanagawa.jp +yamakita.kanagawa.jp +yamato.kanagawa.jp +yokosuka.kanagawa.jp +yugawara.kanagawa.jp +zama.kanagawa.jp +zushi.kanagawa.jp +aki.kochi.jp +geisei.kochi.jp +hidaka.kochi.jp +higashitsuno.kochi.jp +ino.kochi.jp +kagami.kochi.jp +kami.kochi.jp +kitagawa.kochi.jp +kochi.kochi.jp +mihara.kochi.jp +motoyama.kochi.jp +muroto.kochi.jp +nahari.kochi.jp +nakamura.kochi.jp +nankoku.kochi.jp +nishitosa.kochi.jp +niyodogawa.kochi.jp +ochi.kochi.jp +okawa.kochi.jp +otoyo.kochi.jp +otsuki.kochi.jp +sakawa.kochi.jp +sukumo.kochi.jp +susaki.kochi.jp +tosa.kochi.jp +tosashimizu.kochi.jp +toyo.kochi.jp +tsuno.kochi.jp +umaji.kochi.jp +yasuda.kochi.jp +yusuhara.kochi.jp +amakusa.kumamoto.jp +arao.kumamoto.jp +aso.kumamoto.jp +choyo.kumamoto.jp +gyokuto.kumamoto.jp +hitoyoshi.kumamoto.jp +kamiamakusa.kumamoto.jp +kashima.kumamoto.jp +kikuchi.kumamoto.jp +kosa.kumamoto.jp +kumamoto.kumamoto.jp +mashiki.kumamoto.jp +mifune.kumamoto.jp +minamata.kumamoto.jp +minamioguni.kumamoto.jp +nagasu.kumamoto.jp +nishihara.kumamoto.jp +oguni.kumamoto.jp +ozu.kumamoto.jp +sumoto.kumamoto.jp +takamori.kumamoto.jp +uki.kumamoto.jp +uto.kumamoto.jp +yamaga.kumamoto.jp +yamato.kumamoto.jp +yatsushiro.kumamoto.jp +ayabe.kyoto.jp +fukuchiyama.kyoto.jp +higashiyama.kyoto.jp +ide.kyoto.jp +ine.kyoto.jp +joyo.kyoto.jp +kameoka.kyoto.jp +kamo.kyoto.jp +kita.kyoto.jp +kizu.kyoto.jp +kumiyama.kyoto.jp +kyotamba.kyoto.jp +kyotanabe.kyoto.jp +kyotango.kyoto.jp +maizuru.kyoto.jp +minami.kyoto.jp +minamiyamashiro.kyoto.jp +miyazu.kyoto.jp +muko.kyoto.jp +nagaokakyo.kyoto.jp +nakagyo.kyoto.jp +nantan.kyoto.jp +oyamazaki.kyoto.jp +sakyo.kyoto.jp +seika.kyoto.jp +tanabe.kyoto.jp +uji.kyoto.jp +ujitawara.kyoto.jp +wazuka.kyoto.jp +yamashina.kyoto.jp +yawata.kyoto.jp +asahi.mie.jp +inabe.mie.jp +ise.mie.jp +kameyama.mie.jp +kawagoe.mie.jp +kiho.mie.jp +kisosaki.mie.jp +kiwa.mie.jp +komono.mie.jp +kumano.mie.jp +kuwana.mie.jp +matsusaka.mie.jp +meiwa.mie.jp +mihama.mie.jp +minamiise.mie.jp +misugi.mie.jp +miyama.mie.jp +nabari.mie.jp +shima.mie.jp +suzuka.mie.jp +tado.mie.jp +taiki.mie.jp +taki.mie.jp +tamaki.mie.jp +toba.mie.jp +tsu.mie.jp +udono.mie.jp +ureshino.mie.jp +watarai.mie.jp +yokkaichi.mie.jp +furukawa.miyagi.jp +higashimatsushima.miyagi.jp +ishinomaki.miyagi.jp +iwanuma.miyagi.jp +kakuda.miyagi.jp +kami.miyagi.jp +kawasaki.miyagi.jp +kesennuma.miyagi.jp +marumori.miyagi.jp +matsushima.miyagi.jp +minamisanriku.miyagi.jp +misato.miyagi.jp +murata.miyagi.jp +natori.miyagi.jp +ogawara.miyagi.jp +ohira.miyagi.jp +onagawa.miyagi.jp +osaki.miyagi.jp +rifu.miyagi.jp +semine.miyagi.jp +shibata.miyagi.jp +shichikashuku.miyagi.jp +shikama.miyagi.jp +shiogama.miyagi.jp +shiroishi.miyagi.jp +tagajo.miyagi.jp +taiwa.miyagi.jp +tome.miyagi.jp +tomiya.miyagi.jp +wakuya.miyagi.jp +watari.miyagi.jp +yamamoto.miyagi.jp +zao.miyagi.jp +aya.miyazaki.jp +ebino.miyazaki.jp +gokase.miyazaki.jp +hyuga.miyazaki.jp +kadogawa.miyazaki.jp +kawaminami.miyazaki.jp +kijo.miyazaki.jp +kitagawa.miyazaki.jp +kitakata.miyazaki.jp +kitaura.miyazaki.jp +kobayashi.miyazaki.jp +kunitomi.miyazaki.jp +kushima.miyazaki.jp +mimata.miyazaki.jp +miyakonojo.miyazaki.jp +miyazaki.miyazaki.jp +morotsuka.miyazaki.jp +nichinan.miyazaki.jp +nishimera.miyazaki.jp +nobeoka.miyazaki.jp +saito.miyazaki.jp +shiiba.miyazaki.jp +shintomi.miyazaki.jp +takaharu.miyazaki.jp +takanabe.miyazaki.jp +takazaki.miyazaki.jp +tsuno.miyazaki.jp +achi.nagano.jp +agematsu.nagano.jp +anan.nagano.jp +aoki.nagano.jp +asahi.nagano.jp +azumino.nagano.jp +chikuhoku.nagano.jp +chikuma.nagano.jp +chino.nagano.jp +fujimi.nagano.jp +hakuba.nagano.jp +hara.nagano.jp +hiraya.nagano.jp +iida.nagano.jp +iijima.nagano.jp +iiyama.nagano.jp +iizuna.nagano.jp +ikeda.nagano.jp +ikusaka.nagano.jp +ina.nagano.jp +karuizawa.nagano.jp +kawakami.nagano.jp +kiso.nagano.jp +kisofukushima.nagano.jp +kitaaiki.nagano.jp +komagane.nagano.jp +komoro.nagano.jp +matsukawa.nagano.jp +matsumoto.nagano.jp +miasa.nagano.jp +minamiaiki.nagano.jp +minamimaki.nagano.jp +minamiminowa.nagano.jp +minowa.nagano.jp +miyada.nagano.jp +miyota.nagano.jp +mochizuki.nagano.jp +nagano.nagano.jp +nagawa.nagano.jp +nagiso.nagano.jp +nakagawa.nagano.jp +nakano.nagano.jp +nozawaonsen.nagano.jp +obuse.nagano.jp +ogawa.nagano.jp +okaya.nagano.jp +omachi.nagano.jp +omi.nagano.jp +ookuwa.nagano.jp +ooshika.nagano.jp +otaki.nagano.jp +otari.nagano.jp +sakae.nagano.jp +sakaki.nagano.jp +saku.nagano.jp +sakuho.nagano.jp +shimosuwa.nagano.jp +shinanomachi.nagano.jp +shiojiri.nagano.jp +suwa.nagano.jp +suzaka.nagano.jp +takagi.nagano.jp +takamori.nagano.jp +takayama.nagano.jp +tateshina.nagano.jp +tatsuno.nagano.jp +togakushi.nagano.jp +togura.nagano.jp +tomi.nagano.jp +ueda.nagano.jp +wada.nagano.jp +yamagata.nagano.jp +yamanouchi.nagano.jp +yasaka.nagano.jp +yasuoka.nagano.jp +chijiwa.nagasaki.jp +futsu.nagasaki.jp +goto.nagasaki.jp +hasami.nagasaki.jp +hirado.nagasaki.jp +iki.nagasaki.jp +isahaya.nagasaki.jp +kawatana.nagasaki.jp +kuchinotsu.nagasaki.jp +matsuura.nagasaki.jp +nagasaki.nagasaki.jp +obama.nagasaki.jp +omura.nagasaki.jp +oseto.nagasaki.jp +saikai.nagasaki.jp +sasebo.nagasaki.jp +seihi.nagasaki.jp +shimabara.nagasaki.jp +shinkamigoto.nagasaki.jp +togitsu.nagasaki.jp +tsushima.nagasaki.jp +unzen.nagasaki.jp +ando.nara.jp +gose.nara.jp +heguri.nara.jp +higashiyoshino.nara.jp +ikaruga.nara.jp +ikoma.nara.jp +kamikitayama.nara.jp +kanmaki.nara.jp +kashiba.nara.jp +kashihara.nara.jp +katsuragi.nara.jp +kawai.nara.jp +kawakami.nara.jp +kawanishi.nara.jp +koryo.nara.jp +kurotaki.nara.jp +mitsue.nara.jp +miyake.nara.jp +nara.nara.jp +nosegawa.nara.jp +oji.nara.jp +ouda.nara.jp +oyodo.nara.jp +sakurai.nara.jp +sango.nara.jp +shimoichi.nara.jp +shimokitayama.nara.jp +shinjo.nara.jp +soni.nara.jp +takatori.nara.jp +tawaramoto.nara.jp +tenkawa.nara.jp +tenri.nara.jp +uda.nara.jp +yamatokoriyama.nara.jp +yamatotakada.nara.jp +yamazoe.nara.jp +yoshino.nara.jp +aga.niigata.jp +agano.niigata.jp +gosen.niigata.jp +itoigawa.niigata.jp +izumozaki.niigata.jp +joetsu.niigata.jp +kamo.niigata.jp +kariwa.niigata.jp +kashiwazaki.niigata.jp +minamiuonuma.niigata.jp +mitsuke.niigata.jp +muika.niigata.jp +murakami.niigata.jp +myoko.niigata.jp +nagaoka.niigata.jp +niigata.niigata.jp +ojiya.niigata.jp +omi.niigata.jp +sado.niigata.jp +sanjo.niigata.jp +seiro.niigata.jp +seirou.niigata.jp +sekikawa.niigata.jp +shibata.niigata.jp +tagami.niigata.jp +tainai.niigata.jp +tochio.niigata.jp +tokamachi.niigata.jp +tsubame.niigata.jp +tsunan.niigata.jp +uonuma.niigata.jp +yahiko.niigata.jp +yoita.niigata.jp +yuzawa.niigata.jp +beppu.oita.jp +bungoono.oita.jp +bungotakada.oita.jp +hasama.oita.jp +hiji.oita.jp +himeshima.oita.jp +hita.oita.jp +kamitsue.oita.jp +kokonoe.oita.jp +kuju.oita.jp +kunisaki.oita.jp +kusu.oita.jp +oita.oita.jp +saiki.oita.jp +taketa.oita.jp +tsukumi.oita.jp +usa.oita.jp +usuki.oita.jp +yufu.oita.jp +akaiwa.okayama.jp +asakuchi.okayama.jp +bizen.okayama.jp +hayashima.okayama.jp +ibara.okayama.jp +kagamino.okayama.jp +kasaoka.okayama.jp +kibichuo.okayama.jp +kumenan.okayama.jp +kurashiki.okayama.jp +maniwa.okayama.jp +misaki.okayama.jp +nagi.okayama.jp +niimi.okayama.jp +nishiawakura.okayama.jp +okayama.okayama.jp +satosho.okayama.jp +setouchi.okayama.jp +shinjo.okayama.jp +shoo.okayama.jp +soja.okayama.jp +takahashi.okayama.jp +tamano.okayama.jp +tsuyama.okayama.jp +wake.okayama.jp +yakage.okayama.jp +aguni.okinawa.jp +ginowan.okinawa.jp +ginoza.okinawa.jp +gushikami.okinawa.jp +haebaru.okinawa.jp +higashi.okinawa.jp +hirara.okinawa.jp +iheya.okinawa.jp +ishigaki.okinawa.jp +ishikawa.okinawa.jp +itoman.okinawa.jp +izena.okinawa.jp +kadena.okinawa.jp +kin.okinawa.jp +kitadaito.okinawa.jp +kitanakagusuku.okinawa.jp +kumejima.okinawa.jp +kunigami.okinawa.jp +minamidaito.okinawa.jp +motobu.okinawa.jp +nago.okinawa.jp +naha.okinawa.jp +nakagusuku.okinawa.jp +nakijin.okinawa.jp +nanjo.okinawa.jp +nishihara.okinawa.jp +ogimi.okinawa.jp +okinawa.okinawa.jp +onna.okinawa.jp +shimoji.okinawa.jp +taketomi.okinawa.jp +tarama.okinawa.jp +tokashiki.okinawa.jp +tomigusuku.okinawa.jp +tonaki.okinawa.jp +urasoe.okinawa.jp +uruma.okinawa.jp +yaese.okinawa.jp +yomitan.okinawa.jp +yonabaru.okinawa.jp +yonaguni.okinawa.jp +zamami.okinawa.jp +abeno.osaka.jp +chihayaakasaka.osaka.jp +chuo.osaka.jp +daito.osaka.jp +fujiidera.osaka.jp +habikino.osaka.jp +hannan.osaka.jp +higashiosaka.osaka.jp +higashisumiyoshi.osaka.jp +higashiyodogawa.osaka.jp +hirakata.osaka.jp +ibaraki.osaka.jp +ikeda.osaka.jp +izumi.osaka.jp +izumiotsu.osaka.jp +izumisano.osaka.jp +kadoma.osaka.jp +kaizuka.osaka.jp +kanan.osaka.jp +kashiwara.osaka.jp +katano.osaka.jp +kawachinagano.osaka.jp +kishiwada.osaka.jp +kita.osaka.jp +kumatori.osaka.jp +matsubara.osaka.jp +minato.osaka.jp +minoh.osaka.jp +misaki.osaka.jp +moriguchi.osaka.jp +neyagawa.osaka.jp +nishi.osaka.jp +nose.osaka.jp +osakasayama.osaka.jp +sakai.osaka.jp +sayama.osaka.jp +sennan.osaka.jp +settsu.osaka.jp +shijonawate.osaka.jp +shimamoto.osaka.jp +suita.osaka.jp +tadaoka.osaka.jp +taishi.osaka.jp +tajiri.osaka.jp +takaishi.osaka.jp +takatsuki.osaka.jp +tondabayashi.osaka.jp +toyonaka.osaka.jp +toyono.osaka.jp +yao.osaka.jp +ariake.saga.jp +arita.saga.jp +fukudomi.saga.jp +genkai.saga.jp +hamatama.saga.jp +hizen.saga.jp +imari.saga.jp +kamimine.saga.jp +kanzaki.saga.jp +karatsu.saga.jp +kashima.saga.jp +kitagata.saga.jp +kitahata.saga.jp +kiyama.saga.jp +kouhoku.saga.jp +kyuragi.saga.jp +nishiarita.saga.jp +ogi.saga.jp +omachi.saga.jp +ouchi.saga.jp +saga.saga.jp +shiroishi.saga.jp +taku.saga.jp +tara.saga.jp +tosu.saga.jp +yoshinogari.saga.jp +arakawa.saitama.jp +asaka.saitama.jp +chichibu.saitama.jp +fujimi.saitama.jp +fujimino.saitama.jp +fukaya.saitama.jp +hanno.saitama.jp +hanyu.saitama.jp +hasuda.saitama.jp +hatogaya.saitama.jp +hatoyama.saitama.jp +hidaka.saitama.jp +higashichichibu.saitama.jp +higashimatsuyama.saitama.jp +honjo.saitama.jp +ina.saitama.jp +iruma.saitama.jp +iwatsuki.saitama.jp +kamiizumi.saitama.jp +kamikawa.saitama.jp +kamisato.saitama.jp +kasukabe.saitama.jp +kawagoe.saitama.jp +kawaguchi.saitama.jp +kawajima.saitama.jp +kazo.saitama.jp +kitamoto.saitama.jp +koshigaya.saitama.jp +kounosu.saitama.jp +kuki.saitama.jp +kumagaya.saitama.jp +matsubushi.saitama.jp +minano.saitama.jp +misato.saitama.jp +miyashiro.saitama.jp +miyoshi.saitama.jp +moroyama.saitama.jp +nagatoro.saitama.jp +namegawa.saitama.jp +niiza.saitama.jp +ogano.saitama.jp +ogawa.saitama.jp +ogose.saitama.jp +okegawa.saitama.jp +omiya.saitama.jp +otaki.saitama.jp +ranzan.saitama.jp +ryokami.saitama.jp +saitama.saitama.jp +sakado.saitama.jp +satte.saitama.jp +sayama.saitama.jp +shiki.saitama.jp +shiraoka.saitama.jp +soka.saitama.jp +sugito.saitama.jp +toda.saitama.jp +tokigawa.saitama.jp +tokorozawa.saitama.jp +tsurugashima.saitama.jp +urawa.saitama.jp +warabi.saitama.jp +yashio.saitama.jp +yokoze.saitama.jp +yono.saitama.jp +yorii.saitama.jp +yoshida.saitama.jp +yoshikawa.saitama.jp +yoshimi.saitama.jp +aisho.shiga.jp +gamo.shiga.jp +higashiomi.shiga.jp +hikone.shiga.jp +koka.shiga.jp +konan.shiga.jp +kosei.shiga.jp +koto.shiga.jp +kusatsu.shiga.jp +maibara.shiga.jp +moriyama.shiga.jp +nagahama.shiga.jp +nishiazai.shiga.jp +notogawa.shiga.jp +omihachiman.shiga.jp +otsu.shiga.jp +ritto.shiga.jp +ryuoh.shiga.jp +takashima.shiga.jp +takatsuki.shiga.jp +torahime.shiga.jp +toyosato.shiga.jp +yasu.shiga.jp +akagi.shimane.jp +ama.shimane.jp +gotsu.shimane.jp +hamada.shimane.jp +higashiizumo.shimane.jp +hikawa.shimane.jp +hikimi.shimane.jp +izumo.shimane.jp +kakinoki.shimane.jp +masuda.shimane.jp +matsue.shimane.jp +misato.shimane.jp +nishinoshima.shimane.jp +ohda.shimane.jp +okinoshima.shimane.jp +okuizumo.shimane.jp +shimane.shimane.jp +tamayu.shimane.jp +tsuwano.shimane.jp +unnan.shimane.jp +yakumo.shimane.jp +yasugi.shimane.jp +yatsuka.shimane.jp +arai.shizuoka.jp +atami.shizuoka.jp +fuji.shizuoka.jp +fujieda.shizuoka.jp +fujikawa.shizuoka.jp +fujinomiya.shizuoka.jp +fukuroi.shizuoka.jp +gotemba.shizuoka.jp +haibara.shizuoka.jp +hamamatsu.shizuoka.jp +higashiizu.shizuoka.jp +ito.shizuoka.jp +iwata.shizuoka.jp +izu.shizuoka.jp +izunokuni.shizuoka.jp +kakegawa.shizuoka.jp +kannami.shizuoka.jp +kawanehon.shizuoka.jp +kawazu.shizuoka.jp +kikugawa.shizuoka.jp +kosai.shizuoka.jp +makinohara.shizuoka.jp +matsuzaki.shizuoka.jp +minamiizu.shizuoka.jp +mishima.shizuoka.jp +morimachi.shizuoka.jp +nishiizu.shizuoka.jp +numazu.shizuoka.jp +omaezaki.shizuoka.jp +shimada.shizuoka.jp +shimizu.shizuoka.jp +shimoda.shizuoka.jp +shizuoka.shizuoka.jp +susono.shizuoka.jp +yaizu.shizuoka.jp +yoshida.shizuoka.jp +ashikaga.tochigi.jp +bato.tochigi.jp +haga.tochigi.jp +ichikai.tochigi.jp +iwafune.tochigi.jp +kaminokawa.tochigi.jp +kanuma.tochigi.jp +karasuyama.tochigi.jp +kuroiso.tochigi.jp +mashiko.tochigi.jp +mibu.tochigi.jp +moka.tochigi.jp +motegi.tochigi.jp +nasu.tochigi.jp +nasushiobara.tochigi.jp +nikko.tochigi.jp +nishikata.tochigi.jp +nogi.tochigi.jp +ohira.tochigi.jp +ohtawara.tochigi.jp +oyama.tochigi.jp +sakura.tochigi.jp +sano.tochigi.jp +shimotsuke.tochigi.jp +shioya.tochigi.jp +takanezawa.tochigi.jp +tochigi.tochigi.jp +tsuga.tochigi.jp +ujiie.tochigi.jp +utsunomiya.tochigi.jp +yaita.tochigi.jp +aizumi.tokushima.jp +anan.tokushima.jp +ichiba.tokushima.jp +itano.tokushima.jp +kainan.tokushima.jp +komatsushima.tokushima.jp +matsushige.tokushima.jp +mima.tokushima.jp +minami.tokushima.jp +miyoshi.tokushima.jp +mugi.tokushima.jp +nakagawa.tokushima.jp +naruto.tokushima.jp +sanagochi.tokushima.jp +shishikui.tokushima.jp +tokushima.tokushima.jp +wajiki.tokushima.jp +adachi.tokyo.jp +akiruno.tokyo.jp +akishima.tokyo.jp +aogashima.tokyo.jp +arakawa.tokyo.jp +bunkyo.tokyo.jp +chiyoda.tokyo.jp +chofu.tokyo.jp +chuo.tokyo.jp +edogawa.tokyo.jp +fuchu.tokyo.jp +fussa.tokyo.jp +hachijo.tokyo.jp +hachioji.tokyo.jp +hamura.tokyo.jp +higashikurume.tokyo.jp +higashimurayama.tokyo.jp +higashiyamato.tokyo.jp +hino.tokyo.jp +hinode.tokyo.jp +hinohara.tokyo.jp +inagi.tokyo.jp +itabashi.tokyo.jp +katsushika.tokyo.jp +kita.tokyo.jp +kiyose.tokyo.jp +kodaira.tokyo.jp +koganei.tokyo.jp +kokubunji.tokyo.jp +komae.tokyo.jp +koto.tokyo.jp +kouzushima.tokyo.jp +kunitachi.tokyo.jp +machida.tokyo.jp +meguro.tokyo.jp +minato.tokyo.jp +mitaka.tokyo.jp +mizuho.tokyo.jp +musashimurayama.tokyo.jp +musashino.tokyo.jp +nakano.tokyo.jp +nerima.tokyo.jp +ogasawara.tokyo.jp +okutama.tokyo.jp +ome.tokyo.jp +oshima.tokyo.jp +ota.tokyo.jp +setagaya.tokyo.jp +shibuya.tokyo.jp +shinagawa.tokyo.jp +shinjuku.tokyo.jp +suginami.tokyo.jp +sumida.tokyo.jp +tachikawa.tokyo.jp +taito.tokyo.jp +tama.tokyo.jp +toshima.tokyo.jp +chizu.tottori.jp +hino.tottori.jp +kawahara.tottori.jp +koge.tottori.jp +kotoura.tottori.jp +misasa.tottori.jp +nanbu.tottori.jp +nichinan.tottori.jp +sakaiminato.tottori.jp +tottori.tottori.jp +wakasa.tottori.jp +yazu.tottori.jp +yonago.tottori.jp +asahi.toyama.jp +fuchu.toyama.jp +fukumitsu.toyama.jp +funahashi.toyama.jp +himi.toyama.jp +imizu.toyama.jp +inami.toyama.jp +johana.toyama.jp +kamiichi.toyama.jp +kurobe.toyama.jp +nakaniikawa.toyama.jp +namerikawa.toyama.jp +nanto.toyama.jp +nyuzen.toyama.jp +oyabe.toyama.jp +taira.toyama.jp +takaoka.toyama.jp +tateyama.toyama.jp +toga.toyama.jp +tonami.toyama.jp +toyama.toyama.jp +unazuki.toyama.jp +uozu.toyama.jp +yamada.toyama.jp +arida.wakayama.jp +aridagawa.wakayama.jp +gobo.wakayama.jp +hashimoto.wakayama.jp +hidaka.wakayama.jp +hirogawa.wakayama.jp +inami.wakayama.jp +iwade.wakayama.jp +kainan.wakayama.jp +kamitonda.wakayama.jp +katsuragi.wakayama.jp +kimino.wakayama.jp +kinokawa.wakayama.jp +kitayama.wakayama.jp +koya.wakayama.jp +koza.wakayama.jp +kozagawa.wakayama.jp +kudoyama.wakayama.jp +kushimoto.wakayama.jp +mihama.wakayama.jp +misato.wakayama.jp +nachikatsuura.wakayama.jp +shingu.wakayama.jp +shirahama.wakayama.jp +taiji.wakayama.jp +tanabe.wakayama.jp +wakayama.wakayama.jp +yuasa.wakayama.jp +yura.wakayama.jp +asahi.yamagata.jp +funagata.yamagata.jp +higashine.yamagata.jp +iide.yamagata.jp +kahoku.yamagata.jp +kaminoyama.yamagata.jp +kaneyama.yamagata.jp +kawanishi.yamagata.jp +mamurogawa.yamagata.jp +mikawa.yamagata.jp +murayama.yamagata.jp +nagai.yamagata.jp +nakayama.yamagata.jp +nanyo.yamagata.jp +nishikawa.yamagata.jp +obanazawa.yamagata.jp +oe.yamagata.jp +oguni.yamagata.jp +ohkura.yamagata.jp +oishida.yamagata.jp +sagae.yamagata.jp +sakata.yamagata.jp +sakegawa.yamagata.jp +shinjo.yamagata.jp +shirataka.yamagata.jp +shonai.yamagata.jp +takahata.yamagata.jp +tendo.yamagata.jp +tozawa.yamagata.jp +tsuruoka.yamagata.jp +yamagata.yamagata.jp +yamanobe.yamagata.jp +yonezawa.yamagata.jp +yuza.yamagata.jp +abu.yamaguchi.jp +hagi.yamaguchi.jp +hikari.yamaguchi.jp +hofu.yamaguchi.jp +iwakuni.yamaguchi.jp +kudamatsu.yamaguchi.jp +mitou.yamaguchi.jp +nagato.yamaguchi.jp +oshima.yamaguchi.jp +shimonoseki.yamaguchi.jp +shunan.yamaguchi.jp +tabuse.yamaguchi.jp +tokuyama.yamaguchi.jp +toyota.yamaguchi.jp +ube.yamaguchi.jp +yuu.yamaguchi.jp +chuo.yamanashi.jp +doshi.yamanashi.jp +fuefuki.yamanashi.jp +fujikawa.yamanashi.jp +fujikawaguchiko.yamanashi.jp +fujiyoshida.yamanashi.jp +hayakawa.yamanashi.jp +hokuto.yamanashi.jp +ichikawamisato.yamanashi.jp +kai.yamanashi.jp +kofu.yamanashi.jp +koshu.yamanashi.jp +kosuge.yamanashi.jp +minami-alps.yamanashi.jp +minobu.yamanashi.jp +nakamichi.yamanashi.jp +nanbu.yamanashi.jp +narusawa.yamanashi.jp +nirasaki.yamanashi.jp +nishikatsura.yamanashi.jp +oshino.yamanashi.jp +otsuki.yamanashi.jp +showa.yamanashi.jp +tabayama.yamanashi.jp +tsuru.yamanashi.jp +uenohara.yamanashi.jp +yamanakako.yamanashi.jp +yamanashi.yamanashi.jp + +// ke : http://www.kenic.or.ke/index.php?option=com_content&task=view&id=117&Itemid=145 +*.ke + +// kg : http://www.domain.kg/dmn_n.html +kg +org.kg +net.kg +com.kg +edu.kg +gov.kg +mil.kg + +// kh : http://www.mptc.gov.kh/dns_registration.htm +*.kh + +// ki : http://www.ki/dns/index.html +ki +edu.ki +biz.ki +net.ki +org.ki +gov.ki +info.ki +com.ki + +// km : http://en.wikipedia.org/wiki/.km +// http://www.domaine.km/documents/charte.doc +km +org.km +nom.km +gov.km +prd.km +tm.km +edu.km +mil.km +ass.km +com.km +// These are only mentioned as proposed suggestions at domaine.km, but +// http://en.wikipedia.org/wiki/.km says they're available for registration: +coop.km +asso.km +presse.km +medecin.km +notaires.km +pharmaciens.km +veterinaire.km +gouv.km + +// kn : http://en.wikipedia.org/wiki/.kn +// http://www.dot.kn/domainRules.html +kn +net.kn +org.kn +edu.kn +gov.kn + +// kp : http://www.kcce.kp/en_index.php +kp +com.kp +edu.kp +gov.kp +org.kp +rep.kp +tra.kp + +// kr : http://en.wikipedia.org/wiki/.kr +// see also: http://domain.nida.or.kr/eng/registration.jsp +kr +ac.kr +co.kr +es.kr +go.kr +hs.kr +kg.kr +mil.kr +ms.kr +ne.kr +or.kr +pe.kr +re.kr +sc.kr +// kr geographical names +busan.kr +chungbuk.kr +chungnam.kr +daegu.kr +daejeon.kr +gangwon.kr +gwangju.kr +gyeongbuk.kr +gyeonggi.kr +gyeongnam.kr +incheon.kr +jeju.kr +jeonbuk.kr +jeonnam.kr +seoul.kr +ulsan.kr + +// kw : http://en.wikipedia.org/wiki/.kw +*.kw + +// ky : http://www.icta.ky/da_ky_reg_dom.php +// Confirmed by registry 2008-06-17 +ky +edu.ky +gov.ky +com.ky +org.ky +net.ky + +// kz : http://en.wikipedia.org/wiki/.kz +// see also: http://www.nic.kz/rules/index.jsp +kz +org.kz +edu.kz +net.kz +gov.kz +mil.kz +com.kz + +// la : http://en.wikipedia.org/wiki/.la +// Submitted by registry 2008-06-10 +la +int.la +net.la +info.la +edu.la +gov.la +per.la +com.la +org.la + +// lb : http://en.wikipedia.org/wiki/.lb +// Submitted by registry 2008-06-17 +lb +com.lb +edu.lb +gov.lb +net.lb +org.lb + +// lc : http://en.wikipedia.org/wiki/.lc +// see also: http://www.nic.lc/rules.htm +lc +com.lc +net.lc +co.lc +org.lc +edu.lc +gov.lc + +// li : http://en.wikipedia.org/wiki/.li +li + +// lk : http://www.nic.lk/seclevpr.html +lk +gov.lk +sch.lk +net.lk +int.lk +com.lk +org.lk +edu.lk +ngo.lk +soc.lk +web.lk +ltd.lk +assn.lk +grp.lk +hotel.lk + +// lr : http://psg.com/dns/lr/lr.txt +// Submitted by registry 2008-06-17 +lr +com.lr +edu.lr +gov.lr +org.lr +net.lr + +// ls : http://en.wikipedia.org/wiki/.ls +ls +co.ls +org.ls + +// lt : http://en.wikipedia.org/wiki/.lt +lt +// gov.lt : http://www.gov.lt/index_en.php +gov.lt + +// lu : http://www.dns.lu/en/ +lu + +// lv : http://www.nic.lv/DNS/En/generic.php +lv +com.lv +edu.lv +gov.lv +org.lv +mil.lv +id.lv +net.lv +asn.lv +conf.lv + +// ly : http://www.nic.ly/regulations.php +ly +com.ly +net.ly +gov.ly +plc.ly +edu.ly +sch.ly +med.ly +org.ly +id.ly + +// ma : http://en.wikipedia.org/wiki/.ma +// http://www.anrt.ma/fr/admin/download/upload/file_fr782.pdf +ma +co.ma +net.ma +gov.ma +org.ma +ac.ma +press.ma + +// mc : http://www.nic.mc/ +mc +tm.mc +asso.mc + +// md : http://en.wikipedia.org/wiki/.md +md + +// me : http://en.wikipedia.org/wiki/.me +me +co.me +net.me +org.me +edu.me +ac.me +gov.me +its.me +priv.me + +// mg : http://www.nic.mg/tarif.htm +mg +org.mg +nom.mg +gov.mg +prd.mg +tm.mg +edu.mg +mil.mg +com.mg + +// mh : http://en.wikipedia.org/wiki/.mh +mh + +// mil : http://en.wikipedia.org/wiki/.mil +mil + +// mk : http://en.wikipedia.org/wiki/.mk +// see also: http://dns.marnet.net.mk/postapka.php +mk +com.mk +org.mk +net.mk +edu.mk +gov.mk +inf.mk +name.mk + +// ml : http://www.gobin.info/domainname/ml-template.doc +// see also: http://en.wikipedia.org/wiki/.ml +ml +com.ml +edu.ml +gouv.ml +gov.ml +net.ml +org.ml +presse.ml + +// mm : http://en.wikipedia.org/wiki/.mm +*.mm + +// mn : http://en.wikipedia.org/wiki/.mn +mn +gov.mn +edu.mn +org.mn + +// mo : http://www.monic.net.mo/ +mo +com.mo +net.mo +org.mo +edu.mo +gov.mo + +// mobi : http://en.wikipedia.org/wiki/.mobi +mobi + +// mp : http://www.dot.mp/ +// Confirmed by registry 2008-06-17 +mp + +// mq : http://en.wikipedia.org/wiki/.mq +mq + +// mr : http://en.wikipedia.org/wiki/.mr +mr +gov.mr + +// ms : http://www.nic.ms/pdf/MS_Domain_Name_Rules.pdf +ms +com.ms +edu.ms +gov.ms +net.ms +org.ms + +// mt : https://www.nic.org.mt/go/policy +// Submitted by registry 2013-11-19 +mt +com.mt +edu.mt +net.mt +org.mt + +// mu : http://en.wikipedia.org/wiki/.mu +mu +com.mu +net.mu +org.mu +gov.mu +ac.mu +co.mu +or.mu + +// museum : http://about.museum/naming/ +// http://index.museum/ +museum +academy.museum +agriculture.museum +air.museum +airguard.museum +alabama.museum +alaska.museum +amber.museum +ambulance.museum +american.museum +americana.museum +americanantiques.museum +americanart.museum +amsterdam.museum +and.museum +annefrank.museum +anthro.museum +anthropology.museum +antiques.museum +aquarium.museum +arboretum.museum +archaeological.museum +archaeology.museum +architecture.museum +art.museum +artanddesign.museum +artcenter.museum +artdeco.museum +arteducation.museum +artgallery.museum +arts.museum +artsandcrafts.museum +asmatart.museum +assassination.museum +assisi.museum +association.museum +astronomy.museum +atlanta.museum +austin.museum +australia.museum +automotive.museum +aviation.museum +axis.museum +badajoz.museum +baghdad.museum +bahn.museum +bale.museum +baltimore.museum +barcelona.museum +baseball.museum +basel.museum +baths.museum +bauern.museum +beauxarts.museum +beeldengeluid.museum +bellevue.museum +bergbau.museum +berkeley.museum +berlin.museum +bern.museum +bible.museum +bilbao.museum +bill.museum +birdart.museum +birthplace.museum +bonn.museum +boston.museum +botanical.museum +botanicalgarden.museum +botanicgarden.museum +botany.museum +brandywinevalley.museum +brasil.museum +bristol.museum +british.museum +britishcolumbia.museum +broadcast.museum +brunel.museum +brussel.museum +brussels.museum +bruxelles.museum +building.museum +burghof.museum +bus.museum +bushey.museum +cadaques.museum +california.museum +cambridge.museum +can.museum +canada.museum +capebreton.museum +carrier.museum +cartoonart.museum +casadelamoneda.museum +castle.museum +castres.museum +celtic.museum +center.museum +chattanooga.museum +cheltenham.museum +chesapeakebay.museum +chicago.museum +children.museum +childrens.museum +childrensgarden.museum +chiropractic.museum +chocolate.museum +christiansburg.museum +cincinnati.museum +cinema.museum +circus.museum +civilisation.museum +civilization.museum +civilwar.museum +clinton.museum +clock.museum +coal.museum +coastaldefence.museum +cody.museum +coldwar.museum +collection.museum +colonialwilliamsburg.museum +coloradoplateau.museum +columbia.museum +columbus.museum +communication.museum +communications.museum +community.museum +computer.museum +computerhistory.museum +comunicações.museum +contemporary.museum +contemporaryart.museum +convent.museum +copenhagen.museum +corporation.museum +correios-e-telecomunicações.museum +corvette.museum +costume.museum +countryestate.museum +county.museum +crafts.museum +cranbrook.museum +creation.museum +cultural.museum +culturalcenter.museum +culture.museum +cyber.museum +cymru.museum +dali.museum +dallas.museum +database.museum +ddr.museum +decorativearts.museum +delaware.museum +delmenhorst.museum +denmark.museum +depot.museum +design.museum +detroit.museum +dinosaur.museum +discovery.museum +dolls.museum +donostia.museum +durham.museum +eastafrica.museum +eastcoast.museum +education.museum +educational.museum +egyptian.museum +eisenbahn.museum +elburg.museum +elvendrell.museum +embroidery.museum +encyclopedic.museum +england.museum +entomology.museum +environment.museum +environmentalconservation.museum +epilepsy.museum +essex.museum +estate.museum +ethnology.museum +exeter.museum +exhibition.museum +family.museum +farm.museum +farmequipment.museum +farmers.museum +farmstead.museum +field.museum +figueres.museum +filatelia.museum +film.museum +fineart.museum +finearts.museum +finland.museum +flanders.museum +florida.museum +force.museum +fortmissoula.museum +fortworth.museum +foundation.museum +francaise.museum +frankfurt.museum +franziskaner.museum +freemasonry.museum +freiburg.museum +fribourg.museum +frog.museum +fundacio.museum +furniture.museum +gallery.museum +garden.museum +gateway.museum +geelvinck.museum +gemological.museum +geology.museum +georgia.museum +giessen.museum +glas.museum +glass.museum +gorge.museum +grandrapids.museum +graz.museum +guernsey.museum +halloffame.museum +hamburg.museum +handson.museum +harvestcelebration.museum +hawaii.museum +health.museum +heimatunduhren.museum +hellas.museum +helsinki.museum +hembygdsforbund.museum +heritage.museum +histoire.museum +historical.museum +historicalsociety.museum +historichouses.museum +historisch.museum +historisches.museum +history.museum +historyofscience.museum +horology.museum +house.museum +humanities.museum +illustration.museum +imageandsound.museum +indian.museum +indiana.museum +indianapolis.museum +indianmarket.museum +intelligence.museum +interactive.museum +iraq.museum +iron.museum +isleofman.museum +jamison.museum +jefferson.museum +jerusalem.museum +jewelry.museum +jewish.museum +jewishart.museum +jfk.museum +journalism.museum +judaica.museum +judygarland.museum +juedisches.museum +juif.museum +karate.museum +karikatur.museum +kids.museum +koebenhavn.museum +koeln.museum +kunst.museum +kunstsammlung.museum +kunstunddesign.museum +labor.museum +labour.museum +lajolla.museum +lancashire.museum +landes.museum +lans.museum +läns.museum +larsson.museum +lewismiller.museum +lincoln.museum +linz.museum +living.museum +livinghistory.museum +localhistory.museum +london.museum +losangeles.museum +louvre.museum +loyalist.museum +lucerne.museum +luxembourg.museum +luzern.museum +mad.museum +madrid.museum +mallorca.museum +manchester.museum +mansion.museum +mansions.museum +manx.museum +marburg.museum +maritime.museum +maritimo.museum +maryland.museum +marylhurst.museum +media.museum +medical.museum +medizinhistorisches.museum +meeres.museum +memorial.museum +mesaverde.museum +michigan.museum +midatlantic.museum +military.museum +mill.museum +miners.museum +mining.museum +minnesota.museum +missile.museum +missoula.museum +modern.museum +moma.museum +money.museum +monmouth.museum +monticello.museum +montreal.museum +moscow.museum +motorcycle.museum +muenchen.museum +muenster.museum +mulhouse.museum +muncie.museum +museet.museum +museumcenter.museum +museumvereniging.museum +music.museum +national.museum +nationalfirearms.museum +nationalheritage.museum +nativeamerican.museum +naturalhistory.museum +naturalhistorymuseum.museum +naturalsciences.museum +nature.museum +naturhistorisches.museum +natuurwetenschappen.museum +naumburg.museum +naval.museum +nebraska.museum +neues.museum +newhampshire.museum +newjersey.museum +newmexico.museum +newport.museum +newspaper.museum +newyork.museum +niepce.museum +norfolk.museum +north.museum +nrw.museum +nuernberg.museum +nuremberg.museum +nyc.museum +nyny.museum +oceanographic.museum +oceanographique.museum +omaha.museum +online.museum +ontario.museum +openair.museum +oregon.museum +oregontrail.museum +otago.museum +oxford.museum +pacific.museum +paderborn.museum +palace.museum +paleo.museum +palmsprings.museum +panama.museum +paris.museum +pasadena.museum +pharmacy.museum +philadelphia.museum +philadelphiaarea.museum +philately.museum +phoenix.museum +photography.museum +pilots.museum +pittsburgh.museum +planetarium.museum +plantation.museum +plants.museum +plaza.museum +portal.museum +portland.museum +portlligat.museum +posts-and-telecommunications.museum +preservation.museum +presidio.museum +press.museum +project.museum +public.museum +pubol.museum +quebec.museum +railroad.museum +railway.museum +research.museum +resistance.museum +riodejaneiro.museum +rochester.museum +rockart.museum +roma.museum +russia.museum +saintlouis.museum +salem.museum +salvadordali.museum +salzburg.museum +sandiego.museum +sanfrancisco.museum +santabarbara.museum +santacruz.museum +santafe.museum +saskatchewan.museum +satx.museum +savannahga.museum +schlesisches.museum +schoenbrunn.museum +schokoladen.museum +school.museum +schweiz.museum +science.museum +scienceandhistory.museum +scienceandindustry.museum +sciencecenter.museum +sciencecenters.museum +science-fiction.museum +sciencehistory.museum +sciences.museum +sciencesnaturelles.museum +scotland.museum +seaport.museum +settlement.museum +settlers.museum +shell.museum +sherbrooke.museum +sibenik.museum +silk.museum +ski.museum +skole.museum +society.museum +sologne.museum +soundandvision.museum +southcarolina.museum +southwest.museum +space.museum +spy.museum +square.museum +stadt.museum +stalbans.museum +starnberg.museum +state.museum +stateofdelaware.museum +station.museum +steam.museum +steiermark.museum +stjohn.museum +stockholm.museum +stpetersburg.museum +stuttgart.museum +suisse.museum +surgeonshall.museum +surrey.museum +svizzera.museum +sweden.museum +sydney.museum +tank.museum +tcm.museum +technology.museum +telekommunikation.museum +television.museum +texas.museum +textile.museum +theater.museum +time.museum +timekeeping.museum +topology.museum +torino.museum +touch.museum +town.museum +transport.museum +tree.museum +trolley.museum +trust.museum +trustee.museum +uhren.museum +ulm.museum +undersea.museum +university.museum +usa.museum +usantiques.museum +usarts.museum +uscountryestate.museum +usculture.museum +usdecorativearts.museum +usgarden.museum +ushistory.museum +ushuaia.museum +uslivinghistory.museum +utah.museum +uvic.museum +valley.museum +vantaa.museum +versailles.museum +viking.museum +village.museum +virginia.museum +virtual.museum +virtuel.museum +vlaanderen.museum +volkenkunde.museum +wales.museum +wallonie.museum +war.museum +washingtondc.museum +watchandclock.museum +watch-and-clock.museum +western.museum +westfalen.museum +whaling.museum +wildlife.museum +williamsburg.museum +windmill.museum +workshop.museum +york.museum +yorkshire.museum +yosemite.museum +youth.museum +zoological.museum +zoology.museum +ירושלים.museum +иком.museum + +// mv : http://en.wikipedia.org/wiki/.mv +// "mv" included because, contra Wikipedia, google.mv exists. +mv +aero.mv +biz.mv +com.mv +coop.mv +edu.mv +gov.mv +info.mv +int.mv +mil.mv +museum.mv +name.mv +net.mv +org.mv +pro.mv + +// mw : http://www.registrar.mw/ +mw +ac.mw +biz.mw +co.mw +com.mw +coop.mw +edu.mw +gov.mw +int.mw +museum.mw +net.mw +org.mw + +// mx : http://www.nic.mx/ +// Submitted by registry 2008-06-19 +mx +com.mx +org.mx +gob.mx +edu.mx +net.mx + +// my : http://www.mynic.net.my/ +my +com.my +net.my +org.my +gov.my +edu.my +mil.my +name.my + +// mz : http://www.gobin.info/domainname/mz-template.doc +*.mz +!teledata.mz + +// na : http://www.na-nic.com.na/ +// http://www.info.na/domain/ +na +info.na +pro.na +name.na +school.na +or.na +dr.na +us.na +mx.na +ca.na +in.na +cc.na +tv.na +ws.na +mobi.na +co.na +com.na +org.na + +// name : has 2nd-level tlds, but there's no list of them +name + +// nc : http://www.cctld.nc/ +nc +asso.nc + +// ne : http://en.wikipedia.org/wiki/.ne +ne + +// net : http://en.wikipedia.org/wiki/.net +net + +// nf : http://en.wikipedia.org/wiki/.nf +nf +com.nf +net.nf +per.nf +rec.nf +web.nf +arts.nf +firm.nf +info.nf +other.nf +store.nf + +// ng : http://psg.com/dns/ng/ +ng +com.ng +edu.ng +name.ng +net.ng +org.ng +sch.ng +gov.ng +mil.ng +mobi.ng + +// ni : http://www.nic.ni/dominios.htm +*.ni + +// nl : http://www.domain-registry.nl/ace.php/c,728,122,,,,Home.html +// Confirmed by registry (with technical +// reservations) 2008-06-08 +nl + +// BV.nl will be a registry for dutch BV's (besloten vennootschap) +bv.nl + +// no : http://www.norid.no/regelverk/index.en.html +// The Norwegian registry has declined to notify us of updates. The web pages +// referenced below are the official source of the data. There is also an +// announce mailing list: +// https://postlister.uninett.no/sympa/info/norid-diskusjon +no +// Norid generic domains : http://www.norid.no/regelverk/vedlegg-c.en.html +fhs.no +vgs.no +fylkesbibl.no +folkebibl.no +museum.no +idrett.no +priv.no +// Non-Norid generic domains : http://www.norid.no/regelverk/vedlegg-d.en.html +mil.no +stat.no +dep.no +kommune.no +herad.no +// no geographical names : http://www.norid.no/regelverk/vedlegg-b.en.html +// counties +aa.no +ah.no +bu.no +fm.no +hl.no +hm.no +jan-mayen.no +mr.no +nl.no +nt.no +of.no +ol.no +oslo.no +rl.no +sf.no +st.no +svalbard.no +tm.no +tr.no +va.no +vf.no +// primary and lower secondary schools per county +gs.aa.no +gs.ah.no +gs.bu.no +gs.fm.no +gs.hl.no +gs.hm.no +gs.jan-mayen.no +gs.mr.no +gs.nl.no +gs.nt.no +gs.of.no +gs.ol.no +gs.oslo.no +gs.rl.no +gs.sf.no +gs.st.no +gs.svalbard.no +gs.tm.no +gs.tr.no +gs.va.no +gs.vf.no +// cities +akrehamn.no +åkrehamn.no +algard.no +ålgård.no +arna.no +brumunddal.no +bryne.no +bronnoysund.no +brønnøysund.no +drobak.no +drøbak.no +egersund.no +fetsund.no +floro.no +florø.no +fredrikstad.no +hokksund.no +honefoss.no +hønefoss.no +jessheim.no +jorpeland.no +jørpeland.no +kirkenes.no +kopervik.no +krokstadelva.no +langevag.no +langevåg.no +leirvik.no +mjondalen.no +mjøndalen.no +mo-i-rana.no +mosjoen.no +mosjøen.no +nesoddtangen.no +orkanger.no +osoyro.no +osøyro.no +raholt.no +råholt.no +sandnessjoen.no +sandnessjøen.no +skedsmokorset.no +slattum.no +spjelkavik.no +stathelle.no +stavern.no +stjordalshalsen.no +stjørdalshalsen.no +tananger.no +tranby.no +vossevangen.no +// communities +afjord.no +åfjord.no +agdenes.no +al.no +ål.no +alesund.no +ålesund.no +alstahaug.no +alta.no +áltá.no +alaheadju.no +álaheadju.no +alvdal.no +amli.no +åmli.no +amot.no +åmot.no +andebu.no +andoy.no +andøy.no +andasuolo.no +ardal.no +årdal.no +aremark.no +arendal.no +ås.no +aseral.no +åseral.no +asker.no +askim.no +askvoll.no +askoy.no +askøy.no +asnes.no +åsnes.no +audnedaln.no +aukra.no +aure.no +aurland.no +aurskog-holand.no +aurskog-høland.no +austevoll.no +austrheim.no +averoy.no +averøy.no +balestrand.no +ballangen.no +balat.no +bálát.no +balsfjord.no +bahccavuotna.no +báhccavuotna.no +bamble.no +bardu.no +beardu.no +beiarn.no +bajddar.no +bájddar.no +baidar.no +báidár.no +berg.no +bergen.no +berlevag.no +berlevåg.no +bearalvahki.no +bearalváhki.no +bindal.no +birkenes.no +bjarkoy.no +bjarkøy.no +bjerkreim.no +bjugn.no +bodo.no +bodø.no +badaddja.no +bådåddjå.no +budejju.no +bokn.no +bremanger.no +bronnoy.no +brønnøy.no +bygland.no +bykle.no +barum.no +bærum.no +bo.telemark.no +bø.telemark.no +bo.nordland.no +bø.nordland.no +bievat.no +bievát.no +bomlo.no +bømlo.no +batsfjord.no +båtsfjord.no +bahcavuotna.no +báhcavuotna.no +dovre.no +drammen.no +drangedal.no +dyroy.no +dyrøy.no +donna.no +dønna.no +eid.no +eidfjord.no +eidsberg.no +eidskog.no +eidsvoll.no +eigersund.no +elverum.no +enebakk.no +engerdal.no +etne.no +etnedal.no +evenes.no +evenassi.no +evenášši.no +evje-og-hornnes.no +farsund.no +fauske.no +fuossko.no +fuoisku.no +fedje.no +fet.no +finnoy.no +finnøy.no +fitjar.no +fjaler.no +fjell.no +flakstad.no +flatanger.no +flekkefjord.no +flesberg.no +flora.no +fla.no +flå.no +folldal.no +forsand.no +fosnes.no +frei.no +frogn.no +froland.no +frosta.no +frana.no +fræna.no +froya.no +frøya.no +fusa.no +fyresdal.no +forde.no +førde.no +gamvik.no +gangaviika.no +gáŋgaviika.no +gaular.no +gausdal.no +gildeskal.no +gildeskål.no +giske.no +gjemnes.no +gjerdrum.no +gjerstad.no +gjesdal.no +gjovik.no +gjøvik.no +gloppen.no +gol.no +gran.no +grane.no +granvin.no +gratangen.no +grimstad.no +grong.no +kraanghke.no +kråanghke.no +grue.no +gulen.no +hadsel.no +halden.no +halsa.no +hamar.no +hamaroy.no +habmer.no +hábmer.no +hapmir.no +hápmir.no +hammerfest.no +hammarfeasta.no +hámmárfeasta.no +haram.no +hareid.no +harstad.no +hasvik.no +aknoluokta.no +ákŋoluokta.no +hattfjelldal.no +aarborte.no +haugesund.no +hemne.no +hemnes.no +hemsedal.no +heroy.more-og-romsdal.no +herøy.møre-og-romsdal.no +heroy.nordland.no +herøy.nordland.no +hitra.no +hjartdal.no +hjelmeland.no +hobol.no +hobøl.no +hof.no +hol.no +hole.no +holmestrand.no +holtalen.no +holtålen.no +hornindal.no +horten.no +hurdal.no +hurum.no +hvaler.no +hyllestad.no +hagebostad.no +hægebostad.no +hoyanger.no +høyanger.no +hoylandet.no +høylandet.no +ha.no +hå.no +ibestad.no +inderoy.no +inderøy.no +iveland.no +jevnaker.no +jondal.no +jolster.no +jølster.no +karasjok.no +karasjohka.no +kárášjohka.no +karlsoy.no +galsa.no +gálsá.no +karmoy.no +karmøy.no +kautokeino.no +guovdageaidnu.no +klepp.no +klabu.no +klæbu.no +kongsberg.no +kongsvinger.no +kragero.no +kragerø.no +kristiansand.no +kristiansund.no +krodsherad.no +krødsherad.no +kvalsund.no +rahkkeravju.no +ráhkkerávju.no +kvam.no +kvinesdal.no +kvinnherad.no +kviteseid.no +kvitsoy.no +kvitsøy.no +kvafjord.no +kvæfjord.no +giehtavuoatna.no +kvanangen.no +kvænangen.no +navuotna.no +návuotna.no +kafjord.no +kåfjord.no +gaivuotna.no +gáivuotna.no +larvik.no +lavangen.no +lavagis.no +loabat.no +loabát.no +lebesby.no +davvesiida.no +leikanger.no +leirfjord.no +leka.no +leksvik.no +lenvik.no +leangaviika.no +leaŋgaviika.no +lesja.no +levanger.no +lier.no +lierne.no +lillehammer.no +lillesand.no +lindesnes.no +lindas.no +lindås.no +lom.no +loppa.no +lahppi.no +láhppi.no +lund.no +lunner.no +luroy.no +lurøy.no +luster.no +lyngdal.no +lyngen.no +ivgu.no +lardal.no +lerdal.no +lærdal.no +lodingen.no +lødingen.no +lorenskog.no +lørenskog.no +loten.no +løten.no +malvik.no +masoy.no +måsøy.no +muosat.no +muosát.no +mandal.no +marker.no +marnardal.no +masfjorden.no +meland.no +meldal.no +melhus.no +meloy.no +meløy.no +meraker.no +meråker.no +moareke.no +moåreke.no +midsund.no +midtre-gauldal.no +modalen.no +modum.no +molde.no +moskenes.no +moss.no +mosvik.no +malselv.no +målselv.no +malatvuopmi.no +málatvuopmi.no +namdalseid.no +aejrie.no +namsos.no +namsskogan.no +naamesjevuemie.no +nååmesjevuemie.no +laakesvuemie.no +nannestad.no +narvik.no +narviika.no +naustdal.no +nedre-eiker.no +nes.akershus.no +nes.buskerud.no +nesna.no +nesodden.no +nesseby.no +unjarga.no +unjárga.no +nesset.no +nissedal.no +nittedal.no +nord-aurdal.no +nord-fron.no +nord-odal.no +norddal.no +nordkapp.no +davvenjarga.no +davvenjárga.no +nordre-land.no +nordreisa.no +raisa.no +ráisa.no +nore-og-uvdal.no +notodden.no +naroy.no +nærøy.no +notteroy.no +nøtterøy.no +odda.no +oksnes.no +øksnes.no +oppdal.no +oppegard.no +oppegård.no +orkdal.no +orland.no +ørland.no +orskog.no +ørskog.no +orsta.no +ørsta.no +os.hedmark.no +os.hordaland.no +osen.no +osteroy.no +osterøy.no +ostre-toten.no +østre-toten.no +overhalla.no +ovre-eiker.no +øvre-eiker.no +oyer.no +øyer.no +oygarden.no +øygarden.no +oystre-slidre.no +øystre-slidre.no +porsanger.no +porsangu.no +porsáŋgu.no +porsgrunn.no +radoy.no +radøy.no +rakkestad.no +rana.no +ruovat.no +randaberg.no +rauma.no +rendalen.no +rennebu.no +rennesoy.no +rennesøy.no +rindal.no +ringebu.no +ringerike.no +ringsaker.no +rissa.no +risor.no +risør.no +roan.no +rollag.no +rygge.no +ralingen.no +rælingen.no +rodoy.no +rødøy.no +romskog.no +rømskog.no +roros.no +røros.no +rost.no +røst.no +royken.no +røyken.no +royrvik.no +røyrvik.no +rade.no +råde.no +salangen.no +siellak.no +saltdal.no +salat.no +sálát.no +sálat.no +samnanger.no +sande.more-og-romsdal.no +sande.møre-og-romsdal.no +sande.vestfold.no +sandefjord.no +sandnes.no +sandoy.no +sandøy.no +sarpsborg.no +sauda.no +sauherad.no +sel.no +selbu.no +selje.no +seljord.no +sigdal.no +siljan.no +sirdal.no +skaun.no +skedsmo.no +ski.no +skien.no +skiptvet.no +skjervoy.no +skjervøy.no +skierva.no +skiervá.no +skjak.no +skjåk.no +skodje.no +skanland.no +skånland.no +skanit.no +skánit.no +smola.no +smøla.no +snillfjord.no +snasa.no +snåsa.no +snoasa.no +snaase.no +snåase.no +sogndal.no +sokndal.no +sola.no +solund.no +songdalen.no +sortland.no +spydeberg.no +stange.no +stavanger.no +steigen.no +steinkjer.no +stjordal.no +stjørdal.no +stokke.no +stor-elvdal.no +stord.no +stordal.no +storfjord.no +omasvuotna.no +strand.no +stranda.no +stryn.no +sula.no +suldal.no +sund.no +sunndal.no +surnadal.no +sveio.no +svelvik.no +sykkylven.no +sogne.no +søgne.no +somna.no +sømna.no +sondre-land.no +søndre-land.no +sor-aurdal.no +sør-aurdal.no +sor-fron.no +sør-fron.no +sor-odal.no +sør-odal.no +sor-varanger.no +sør-varanger.no +matta-varjjat.no +mátta-várjjat.no +sorfold.no +sørfold.no +sorreisa.no +sørreisa.no +sorum.no +sørum.no +tana.no +deatnu.no +time.no +tingvoll.no +tinn.no +tjeldsund.no +dielddanuorri.no +tjome.no +tjøme.no +tokke.no +tolga.no +torsken.no +tranoy.no +tranøy.no +tromso.no +tromsø.no +tromsa.no +romsa.no +trondheim.no +troandin.no +trysil.no +trana.no +træna.no +trogstad.no +trøgstad.no +tvedestrand.no +tydal.no +tynset.no +tysfjord.no +divtasvuodna.no +divttasvuotna.no +tysnes.no +tysvar.no +tysvær.no +tonsberg.no +tønsberg.no +ullensaker.no +ullensvang.no +ulvik.no +utsira.no +vadso.no +vadsø.no +cahcesuolo.no +čáhcesuolo.no +vaksdal.no +valle.no +vang.no +vanylven.no +vardo.no +vardø.no +varggat.no +várggát.no +vefsn.no +vaapste.no +vega.no +vegarshei.no +vegårshei.no +vennesla.no +verdal.no +verran.no +vestby.no +vestnes.no +vestre-slidre.no +vestre-toten.no +vestvagoy.no +vestvågøy.no +vevelstad.no +vik.no +vikna.no +vindafjord.no +volda.no +voss.no +varoy.no +værøy.no +vagan.no +vågan.no +voagat.no +vagsoy.no +vågsøy.no +vaga.no +vågå.no +valer.ostfold.no +våler.østfold.no +valer.hedmark.no +våler.hedmark.no + +// np : http://www.mos.com.np/register.html +*.np + +// nr : http://cenpac.net.nr/dns/index.html +// Confirmed by registry 2008-06-17 +nr +biz.nr +info.nr +gov.nr +edu.nr +org.nr +net.nr +com.nr + +// nu : http://en.wikipedia.org/wiki/.nu +nu + +// nz : http://en.wikipedia.org/wiki/.nz +*.nz + +// om : http://en.wikipedia.org/wiki/.om +om +co.om +com.om +edu.om +gov.om +med.om +museum.om +net.om +org.om +pro.om + +// org : http://en.wikipedia.org/wiki/.org +org + +// pa : http://www.nic.pa/ +// Some additional second level "domains" resolve directly as hostnames, such as +// pannet.pa, so we add a rule for "pa". +pa +ac.pa +gob.pa +com.pa +org.pa +sld.pa +edu.pa +net.pa +ing.pa +abo.pa +med.pa +nom.pa + +// pe : https://www.nic.pe/InformeFinalComision.pdf +pe +edu.pe +gob.pe +nom.pe +mil.pe +org.pe +com.pe +net.pe + +// pf : http://www.gobin.info/domainname/formulaire-pf.pdf +pf +com.pf +org.pf +edu.pf + +// pg : http://en.wikipedia.org/wiki/.pg +*.pg + +// ph : http://www.domains.ph/FAQ2.asp +// Submitted by registry 2008-06-13 +ph +com.ph +net.ph +org.ph +gov.ph +edu.ph +ngo.ph +mil.ph +i.ph + +// pk : http://pk5.pknic.net.pk/pk5/msgNamepk.PK +pk +com.pk +net.pk +edu.pk +org.pk +fam.pk +biz.pk +web.pk +gov.pk +gob.pk +gok.pk +gon.pk +gop.pk +gos.pk +info.pk + +// pl : http://www.dns.pl/english/ +pl +// NASK functional domains (nask.pl / dns.pl) : http://www.dns.pl/english/dns-funk.html +aid.pl +agro.pl +atm.pl +auto.pl +biz.pl +com.pl +edu.pl +gmina.pl +gsm.pl +info.pl +mail.pl +miasta.pl +media.pl +mil.pl +net.pl +nieruchomosci.pl +nom.pl +org.pl +pc.pl +powiat.pl +priv.pl +realestate.pl +rel.pl +sex.pl +shop.pl +sklep.pl +sos.pl +szkola.pl +targi.pl +tm.pl +tourism.pl +travel.pl +turystyka.pl +// ICM functional domains (icm.edu.pl) +6bone.pl +art.pl +mbone.pl +// Government domains (administred by ippt.gov.pl) +gov.pl +uw.gov.pl +um.gov.pl +ug.gov.pl +upow.gov.pl +starostwo.gov.pl +so.gov.pl +sr.gov.pl +po.gov.pl +pa.gov.pl +// other functional domains +ngo.pl +irc.pl +usenet.pl +// NASK geographical domains : http://www.dns.pl/english/dns-regiony.html +augustow.pl +babia-gora.pl +bedzin.pl +beskidy.pl +bialowieza.pl +bialystok.pl +bielawa.pl +bieszczady.pl +boleslawiec.pl +bydgoszcz.pl +bytom.pl +cieszyn.pl +czeladz.pl +czest.pl +dlugoleka.pl +elblag.pl +elk.pl +glogow.pl +gniezno.pl +gorlice.pl +grajewo.pl +ilawa.pl +jaworzno.pl +jelenia-gora.pl +jgora.pl +kalisz.pl +kazimierz-dolny.pl +karpacz.pl +kartuzy.pl +kaszuby.pl +katowice.pl +kepno.pl +ketrzyn.pl +klodzko.pl +kobierzyce.pl +kolobrzeg.pl +konin.pl +konskowola.pl +kutno.pl +lapy.pl +lebork.pl +legnica.pl +lezajsk.pl +limanowa.pl +lomza.pl +lowicz.pl +lubin.pl +lukow.pl +malbork.pl +malopolska.pl +mazowsze.pl +mazury.pl +mielec.pl +mielno.pl +mragowo.pl +naklo.pl +nowaruda.pl +nysa.pl +olawa.pl +olecko.pl +olkusz.pl +olsztyn.pl +opoczno.pl +opole.pl +ostroda.pl +ostroleka.pl +ostrowiec.pl +ostrowwlkp.pl +pila.pl +pisz.pl +podhale.pl +podlasie.pl +polkowice.pl +pomorze.pl +pomorskie.pl +prochowice.pl +pruszkow.pl +przeworsk.pl +pulawy.pl +radom.pl +rawa-maz.pl +rybnik.pl +rzeszow.pl +sanok.pl +sejny.pl +siedlce.pl +slask.pl +slupsk.pl +sosnowiec.pl +stalowa-wola.pl +skoczow.pl +starachowice.pl +stargard.pl +suwalki.pl +swidnica.pl +swiebodzin.pl +swinoujscie.pl +szczecin.pl +szczytno.pl +tarnobrzeg.pl +tgory.pl +turek.pl +tychy.pl +ustka.pl +walbrzych.pl +warmia.pl +warszawa.pl +waw.pl +wegrow.pl +wielun.pl +wlocl.pl +wloclawek.pl +wodzislaw.pl +wolomin.pl +wroclaw.pl +zachpomor.pl +zagan.pl +zarow.pl +zgora.pl +zgorzelec.pl +// TASK geographical domains (www.task.gda.pl/uslugi/dns) +gda.pl +gdansk.pl +gdynia.pl +med.pl +sopot.pl +// other geographical domains +gliwice.pl +krakow.pl +poznan.pl +wroc.pl +zakopane.pl + +// pm : http://www.afnic.fr/medias/documents/AFNIC-naming-policy2012.pdf +pm + +// pn : http://www.government.pn/PnRegistry/policies.htm +pn +gov.pn +co.pn +org.pn +edu.pn +net.pn + +// post : http://en.wikipedia.org/wiki/.post +post + +// pr : http://www.nic.pr/index.asp?f=1 +pr +com.pr +net.pr +org.pr +gov.pr +edu.pr +isla.pr +pro.pr +biz.pr +info.pr +name.pr +// these aren't mentioned on nic.pr, but on http://en.wikipedia.org/wiki/.pr +est.pr +prof.pr +ac.pr + +// pro : http://www.nic.pro/support_faq.htm +pro +aca.pro +bar.pro +cpa.pro +jur.pro +law.pro +med.pro +eng.pro + +// ps : http://en.wikipedia.org/wiki/.ps +// http://www.nic.ps/registration/policy.html#reg +ps +edu.ps +gov.ps +sec.ps +plo.ps +com.ps +org.ps +net.ps + +// pt : http://online.dns.pt/dns/start_dns +pt +net.pt +gov.pt +org.pt +edu.pt +int.pt +publ.pt +com.pt +nome.pt + +// pw : http://en.wikipedia.org/wiki/.pw +pw +co.pw +ne.pw +or.pw +ed.pw +go.pw +belau.pw + +// py : http://www.nic.py/pautas.html#seccion_9 +// Confirmed by registry 2012-10-03 +py +com.py +coop.py +edu.py +gov.py +mil.py +net.py +org.py + +// qa : http://domains.qa/en/ +qa +com.qa +edu.qa +gov.qa +mil.qa +name.qa +net.qa +org.qa +sch.qa + +// re : http://www.afnic.re/obtenir/chartes/nommage-re/annexe-descriptifs +re +com.re +asso.re +nom.re + +// ro : http://www.rotld.ro/ +ro +com.ro +org.ro +tm.ro +nt.ro +nom.ro +info.ro +rec.ro +arts.ro +firm.ro +store.ro +www.ro + +// rs : http://en.wikipedia.org/wiki/.rs +rs +co.rs +org.rs +edu.rs +ac.rs +gov.rs +in.rs + +// ru : http://www.cctld.ru/ru/docs/aktiv_8.php +// Industry domains +ru +ac.ru +com.ru +edu.ru +int.ru +net.ru +org.ru +pp.ru +// Geographical domains +adygeya.ru +altai.ru +amur.ru +arkhangelsk.ru +astrakhan.ru +bashkiria.ru +belgorod.ru +bir.ru +bryansk.ru +buryatia.ru +cbg.ru +chel.ru +chelyabinsk.ru +chita.ru +chukotka.ru +chuvashia.ru +dagestan.ru +dudinka.ru +e-burg.ru +grozny.ru +irkutsk.ru +ivanovo.ru +izhevsk.ru +jar.ru +joshkar-ola.ru +kalmykia.ru +kaluga.ru +kamchatka.ru +karelia.ru +kazan.ru +kchr.ru +kemerovo.ru +khabarovsk.ru +khakassia.ru +khv.ru +kirov.ru +koenig.ru +komi.ru +kostroma.ru +krasnoyarsk.ru +kuban.ru +kurgan.ru +kursk.ru +lipetsk.ru +magadan.ru +mari.ru +mari-el.ru +marine.ru +mordovia.ru +mosreg.ru +msk.ru +murmansk.ru +nalchik.ru +nnov.ru +nov.ru +novosibirsk.ru +nsk.ru +omsk.ru +orenburg.ru +oryol.ru +palana.ru +penza.ru +perm.ru +pskov.ru +ptz.ru +rnd.ru +ryazan.ru +sakhalin.ru +samara.ru +saratov.ru +simbirsk.ru +smolensk.ru +spb.ru +stavropol.ru +stv.ru +surgut.ru +tambov.ru +tatarstan.ru +tom.ru +tomsk.ru +tsaritsyn.ru +tsk.ru +tula.ru +tuva.ru +tver.ru +tyumen.ru +udm.ru +udmurtia.ru +ulan-ude.ru +vladikavkaz.ru +vladimir.ru +vladivostok.ru +volgograd.ru +vologda.ru +voronezh.ru +vrn.ru +vyatka.ru +yakutia.ru +yamal.ru +yaroslavl.ru +yekaterinburg.ru +yuzhno-sakhalinsk.ru +// More geographical domains +amursk.ru +baikal.ru +cmw.ru +fareast.ru +jamal.ru +kms.ru +k-uralsk.ru +kustanai.ru +kuzbass.ru +magnitka.ru +mytis.ru +nakhodka.ru +nkz.ru +norilsk.ru +oskol.ru +pyatigorsk.ru +rubtsovsk.ru +snz.ru +syzran.ru +vdonsk.ru +zgrad.ru +// State domains +gov.ru +mil.ru +// Technical domains +test.ru + +// rw : http://www.nic.rw/cgi-bin/policy.pl +rw +gov.rw +net.rw +edu.rw +ac.rw +com.rw +co.rw +int.rw +mil.rw +gouv.rw + +// sa : http://www.nic.net.sa/ +sa +com.sa +net.sa +org.sa +gov.sa +med.sa +pub.sa +edu.sa +sch.sa + +// sb : http://www.sbnic.net.sb/ +// Submitted by registry 2008-06-08 +sb +com.sb +edu.sb +gov.sb +net.sb +org.sb + +// sc : http://www.nic.sc/ +sc +com.sc +gov.sc +net.sc +org.sc +edu.sc + +// sd : http://www.isoc.sd/sudanic.isoc.sd/billing_pricing.htm +// Submitted by registry 2008-06-17 +sd +com.sd +net.sd +org.sd +edu.sd +med.sd +tv.sd +gov.sd +info.sd + +// se : http://en.wikipedia.org/wiki/.se +// Submitted by registry 2008-06-24 +se +a.se +ac.se +b.se +bd.se +brand.se +c.se +d.se +e.se +f.se +fh.se +fhsk.se +fhv.se +g.se +h.se +i.se +k.se +komforb.se +kommunalforbund.se +komvux.se +l.se +lanbib.se +m.se +n.se +naturbruksgymn.se +o.se +org.se +p.se +parti.se +pp.se +press.se +r.se +s.se +sshn.se +t.se +tm.se +u.se +w.se +x.se +y.se +z.se + +// sg : http://www.nic.net.sg/page/registration-policies-procedures-and-guidelines +sg +com.sg +net.sg +org.sg +gov.sg +edu.sg +per.sg + +// sh : http://www.nic.sh/registrar.html +sh +com.sh +net.sh +gov.sh +org.sh +mil.sh + +// si : http://en.wikipedia.org/wiki/.si +si + +// sj : No registrations at this time. +// Submitted by registry 2008-06-16 +sj + +// sk : http://en.wikipedia.org/wiki/.sk +// list of 2nd level domains ? +sk + +// sl : http://www.nic.sl +// Submitted by registry 2008-06-12 +sl +com.sl +net.sl +edu.sl +gov.sl +org.sl + +// sm : http://en.wikipedia.org/wiki/.sm +sm + +// sn : http://en.wikipedia.org/wiki/.sn +sn +art.sn +com.sn +edu.sn +gouv.sn +org.sn +perso.sn +univ.sn + +// so : http://www.soregistry.com/ +so +com.so +net.so +org.so + +// sr : http://en.wikipedia.org/wiki/.sr +sr + +// st : http://www.nic.st/html/policyrules/ +st +co.st +com.st +consulado.st +edu.st +embaixada.st +gov.st +mil.st +net.st +org.st +principe.st +saotome.st +store.st + +// su : http://en.wikipedia.org/wiki/.su +su + +// sv : http://www.svnet.org.sv/niveldos.pdf +sv +com.sv +edu.sv +gob.sv +org.sv +red.sv + +// sx : http://en.wikipedia.org/wiki/.sx +// Confirmed by registry 2012-05-31 +sx +gov.sx + +// sy : http://en.wikipedia.org/wiki/.sy +// see also: http://www.gobin.info/domainname/sy.doc +sy +edu.sy +gov.sy +net.sy +mil.sy +com.sy +org.sy + +// sz : http://en.wikipedia.org/wiki/.sz +// http://www.sispa.org.sz/ +sz +co.sz +ac.sz +org.sz + +// tc : http://en.wikipedia.org/wiki/.tc +tc + +// td : http://en.wikipedia.org/wiki/.td +td + +// tel: http://en.wikipedia.org/wiki/.tel +// http://www.telnic.org/ +tel + +// tf : http://en.wikipedia.org/wiki/.tf +tf + +// tg : http://en.wikipedia.org/wiki/.tg +// http://www.nic.tg/ +tg + +// th : http://en.wikipedia.org/wiki/.th +// Submitted by registry 2008-06-17 +th +ac.th +co.th +go.th +in.th +mi.th +net.th +or.th + +// tj : http://www.nic.tj/policy.html +tj +ac.tj +biz.tj +co.tj +com.tj +edu.tj +go.tj +gov.tj +int.tj +mil.tj +name.tj +net.tj +nic.tj +org.tj +test.tj +web.tj + +// tk : http://en.wikipedia.org/wiki/.tk +tk + +// tl : http://en.wikipedia.org/wiki/.tl +tl +gov.tl + +// tm : http://www.nic.tm/local.html +tm +com.tm +co.tm +org.tm +net.tm +nom.tm +gov.tm +mil.tm +edu.tm + +// tn : http://en.wikipedia.org/wiki/.tn +// http://whois.ati.tn/ +tn +com.tn +ens.tn +fin.tn +gov.tn +ind.tn +intl.tn +nat.tn +net.tn +org.tn +info.tn +perso.tn +tourism.tn +edunet.tn +rnrt.tn +rns.tn +rnu.tn +mincom.tn +agrinet.tn +defense.tn +turen.tn + +// to : http://en.wikipedia.org/wiki/.to +// Submitted by registry 2008-06-17 +to +com.to +gov.to +net.to +org.to +edu.to +mil.to + +// tp : No registrations at this time. +// Submitted by Ryan Sleevi 2014-01-03 +tp + +// tr : http://en.wikipedia.org/wiki/.tr +*.tr +!nic.tr +// Used by government in the TRNC +// http://en.wikipedia.org/wiki/.nc.tr +gov.nc.tr + +// travel : http://en.wikipedia.org/wiki/.travel +travel + +// tt : http://www.nic.tt/ +tt +co.tt +com.tt +org.tt +net.tt +biz.tt +info.tt +pro.tt +int.tt +coop.tt +jobs.tt +mobi.tt +travel.tt +museum.tt +aero.tt +name.tt +gov.tt +edu.tt + +// tv : http://en.wikipedia.org/wiki/.tv +// Not listing any 2LDs as reserved since none seem to exist in practice, +// Wikipedia notwithstanding. +tv + +// tw : http://en.wikipedia.org/wiki/.tw +tw +edu.tw +gov.tw +mil.tw +com.tw +net.tw +org.tw +idv.tw +game.tw +ebiz.tw +club.tw +網路.tw +組織.tw +商業.tw + +// tz : http://www.tznic.or.tz/index.php/domains +// Confirmed by registry 2013-01-22 +tz +ac.tz +co.tz +go.tz +hotel.tz +info.tz +me.tz +mil.tz +mobi.tz +ne.tz +or.tz +sc.tz +tv.tz + +// ua : https://hostmaster.ua/policy/?ua +// Submitted by registry 2012-04-27 +ua +// ua 2LD +com.ua +edu.ua +gov.ua +in.ua +net.ua +org.ua +// ua geographic names +// https://hostmaster.ua/2ld/ +cherkassy.ua +cherkasy.ua +chernigov.ua +chernihiv.ua +chernivtsi.ua +chernovtsy.ua +ck.ua +cn.ua +cr.ua +crimea.ua +cv.ua +dn.ua +dnepropetrovsk.ua +dnipropetrovsk.ua +dominic.ua +donetsk.ua +dp.ua +if.ua +ivano-frankivsk.ua +kh.ua +kharkiv.ua +kharkov.ua +kherson.ua +khmelnitskiy.ua +khmelnytskyi.ua +kiev.ua +kirovograd.ua +km.ua +kr.ua +krym.ua +ks.ua +kv.ua +kyiv.ua +lg.ua +lt.ua +lugansk.ua +lutsk.ua +lv.ua +lviv.ua +mk.ua +mykolaiv.ua +nikolaev.ua +od.ua +odesa.ua +odessa.ua +pl.ua +poltava.ua +rivne.ua +rovno.ua +rv.ua +sb.ua +sebastopol.ua +sevastopol.ua +sm.ua +sumy.ua +te.ua +ternopil.ua +uz.ua +uzhgorod.ua +vinnica.ua +vinnytsia.ua +vn.ua +volyn.ua +yalta.ua +zaporizhzhe.ua +zaporizhzhia.ua +zhitomir.ua +zhytomyr.ua +zp.ua +zt.ua + +// Private registries in .ua +co.ua +pp.ua + +// ug : https://www.registry.co.ug/ +ug +co.ug +or.ug +ac.ug +sc.ug +go.ug +ne.ug +com.ug +org.ug + +// uk : http://en.wikipedia.org/wiki/.uk +// Submitted by registry 2012-10-02 +// and tweaked by us pending further consultation. +*.uk +*.sch.uk +!bl.uk +!british-library.uk +!jet.uk +!mod.uk +!national-library-scotland.uk +!nel.uk +!nic.uk +!nls.uk +!parliament.uk + +// us : http://en.wikipedia.org/wiki/.us +us +dni.us +fed.us +isa.us +kids.us +nsn.us +// us geographic names +ak.us +al.us +ar.us +as.us +az.us +ca.us +co.us +ct.us +dc.us +de.us +fl.us +ga.us +gu.us +hi.us +ia.us +id.us +il.us +in.us +ks.us +ky.us +la.us +ma.us +md.us +me.us +mi.us +mn.us +mo.us +ms.us +mt.us +nc.us +nd.us +ne.us +nh.us +nj.us +nm.us +nv.us +ny.us +oh.us +ok.us +or.us +pa.us +pr.us +ri.us +sc.us +sd.us +tn.us +tx.us +ut.us +vi.us +vt.us +va.us +wa.us +wi.us +wv.us +wy.us +// The registrar notes several more specific domains available in each state, +// such as state.*.us, dst.*.us, etc., but resolution of these is somewhat +// haphazard; in some states these domains resolve as addresses, while in others +// only subdomains are available, or even nothing at all. We include the +// most common ones where it's clear that different sites are different +// entities. +k12.ak.us +k12.al.us +k12.ar.us +k12.as.us +k12.az.us +k12.ca.us +k12.co.us +k12.ct.us +k12.dc.us +k12.de.us +k12.fl.us +k12.ga.us +k12.gu.us +// k12.hi.us Bug 614565 - Hawaii has a state-wide DOE login +k12.ia.us +k12.id.us +k12.il.us +k12.in.us +k12.ks.us +k12.ky.us +k12.la.us +k12.ma.us +k12.md.us +k12.me.us +k12.mi.us +k12.mn.us +k12.mo.us +k12.ms.us +k12.mt.us +k12.nc.us +k12.nd.us +k12.ne.us +k12.nh.us +k12.nj.us +k12.nm.us +k12.nv.us +k12.ny.us +k12.oh.us +k12.ok.us +k12.or.us +k12.pa.us +k12.pr.us +k12.ri.us +k12.sc.us +// k12.sd.us Bug 934131 - Removed at request of James Booze +k12.tn.us +k12.tx.us +k12.ut.us +k12.vi.us +k12.vt.us +k12.va.us +k12.wa.us +k12.wi.us +// k12.wv.us Bug 947705 - Removed at request of Verne Britton +k12.wy.us + +cc.ak.us +cc.al.us +cc.ar.us +cc.as.us +cc.az.us +cc.ca.us +cc.co.us +cc.ct.us +cc.dc.us +cc.de.us +cc.fl.us +cc.ga.us +cc.gu.us +cc.hi.us +cc.ia.us +cc.id.us +cc.il.us +cc.in.us +cc.ks.us +cc.ky.us +cc.la.us +cc.ma.us +cc.md.us +cc.me.us +cc.mi.us +cc.mn.us +cc.mo.us +cc.ms.us +cc.mt.us +cc.nc.us +cc.nd.us +cc.ne.us +cc.nh.us +cc.nj.us +cc.nm.us +cc.nv.us +cc.ny.us +cc.oh.us +cc.ok.us +cc.or.us +cc.pa.us +cc.pr.us +cc.ri.us +cc.sc.us +cc.sd.us +cc.tn.us +cc.tx.us +cc.ut.us +cc.vi.us +cc.vt.us +cc.va.us +cc.wa.us +cc.wi.us +cc.wv.us +cc.wy.us + +lib.ak.us +lib.al.us +lib.ar.us +lib.as.us +lib.az.us +lib.ca.us +lib.co.us +lib.ct.us +lib.dc.us +lib.de.us +lib.fl.us +lib.ga.us +lib.gu.us +lib.hi.us +lib.ia.us +lib.id.us +lib.il.us +lib.in.us +lib.ks.us +lib.ky.us +lib.la.us +lib.ma.us +lib.md.us +lib.me.us +lib.mi.us +lib.mn.us +lib.mo.us +lib.ms.us +lib.mt.us +lib.nc.us +lib.nd.us +lib.ne.us +lib.nh.us +lib.nj.us +lib.nm.us +lib.nv.us +lib.ny.us +lib.oh.us +lib.ok.us +lib.or.us +lib.pa.us +lib.pr.us +lib.ri.us +lib.sc.us +lib.sd.us +lib.tn.us +lib.tx.us +lib.ut.us +lib.vi.us +lib.vt.us +lib.va.us +lib.wa.us +lib.wi.us +// lib.wv.us Bug 941670 - Removed at request of Larry W Arnold +lib.wy.us + +// k12.ma.us contains school districts in Massachusetts. The 4LDs are +// managed indepedently except for private (PVT), charter (CHTR) and +// parochial (PAROCH) schools. Those are delegated dorectly to the +// 5LD operators. +pvt.k12.ma.us +chtr.k12.ma.us +paroch.k12.ma.us + +// uy : http://www.nic.org.uy/ +uy +com.uy +edu.uy +gub.uy +mil.uy +net.uy +org.uy + +// uz : http://www.reg.uz/ +uz +co.uz +com.uz +net.uz +org.uz + +// va : http://en.wikipedia.org/wiki/.va +va + +// vc : http://en.wikipedia.org/wiki/.vc +// Submitted by registry 2008-06-13 +vc +com.vc +net.vc +org.vc +gov.vc +mil.vc +edu.vc + +// ve : https://registro.nic.ve/ +// Confirmed by registry 2012-10-04 +ve +co.ve +com.ve +e12.ve +edu.ve +gov.ve +info.ve +mil.ve +net.ve +org.ve +web.ve + +// vg : http://en.wikipedia.org/wiki/.vg +vg + +// vi : http://www.nic.vi/newdomainform.htm +// http://www.nic.vi/Domain_Rules/body_domain_rules.html indicates some other +// TLDs are "reserved", such as edu.vi and gov.vi, but doesn't actually say they +// are available for registration (which they do not seem to be). +vi +co.vi +com.vi +k12.vi +net.vi +org.vi + +// vn : https://www.dot.vn/vnnic/vnnic/domainregistration.jsp +vn +com.vn +net.vn +org.vn +edu.vn +gov.vn +int.vn +ac.vn +biz.vn +info.vn +name.vn +pro.vn +health.vn + +// vu : http://en.wikipedia.org/wiki/.vu +// list of 2nd level tlds ? +vu + +// wf : http://www.afnic.fr/medias/documents/AFNIC-naming-policy2012.pdf +wf + +// ws : http://en.wikipedia.org/wiki/.ws +// http://samoanic.ws/index.dhtml +ws +com.ws +net.ws +org.ws +gov.ws +edu.ws + +// yt : http://www.afnic.fr/medias/documents/AFNIC-naming-policy2012.pdf +yt + +// IDN ccTLDs +// Please sort by ISO 3166 ccTLD, then punicode string +// when submitting patches and follow this format: +// ("" ) : +// [optional sponsoring org] +// + +// xn--mgbaam7a8h ("Emerat" Arabic) : AE +// http://nic.ae/english/arabicdomain/rules.jsp +امارات + +// xn--54b7fta0cc ("Bangla" Bangla) : BD +বাংলা + +// xn--fiqs8s ("China" Chinese-Han-Simplified <.Zhongguo>) : CN +// CNNIC +// http://cnnic.cn/html/Dir/2005/10/11/3218.htm +中国 + +// xn--fiqz9s ("China" Chinese-Han-Traditional <.Zhongguo>) : CN +// CNNIC +// http://cnnic.cn/html/Dir/2005/10/11/3218.htm +中國 + +// xn--lgbbat1ad8j ("Algeria / Al Jazair" Arabic) : DZ +الجزائر + +// xn--wgbh1c ("Egypt" Arabic .masr) : EG +// http://www.dotmasr.eg/ +مصر + +// xn--node ("ge" Georgian (Mkhedruli)) : GE +გე + +// xn--j6w193g ("Hong Kong" Chinese-Han) : HK +// https://www2.hkirc.hk/register/rules.jsp +香港 + +// xn--h2brj9c ("Bharat" Devanagari) : IN +// India +भारत + +// xn--mgbbh1a71e ("Bharat" Arabic) : IN +// India +بھارت + +// xn--fpcrj9c3d ("Bharat" Telugu) : IN +// India +భారత్ + +// xn--gecrj9c ("Bharat" Gujarati) : IN +// India +ભારત + +// xn--s9brj9c ("Bharat" Gurmukhi) : IN +// India +ਭਾਰਤ + +// xn--45brj9c ("Bharat" Bengali) : IN +// India +ভারত + +// xn--xkc2dl3a5ee0h ("India" Tamil) : IN +// India +இந்தியா + +// xn--mgba3a4f16a ("Iran" Persian) : IR +ایران + +// xn--mgba3a4fra ("Iran" Arabic) : IR +ايران + +// xn--mgbayh7gpa ("al-Ordon" Arabic) : JO +// National Information Technology Center (NITC) +// Royal Scientific Society, Al-Jubeiha +الاردن + +// xn--3e0b707e ("Republic of Korea" Hangul) : KR +한국 + +// xn--80ao21a ("Kaz" Kazakh) : KZ +қаз + +// xn--fzc2c9e2c ("Lanka" Sinhalese-Sinhala) : LK +// http://nic.lk +ලංකා + +// xn--xkc2al3hye2a ("Ilangai" Tamil) : LK +// http://nic.lk +இலங்கை + +// xn--mgbc0a9azcg ("Morocco / al-Maghrib" Arabic) : MA +المغرب + +// xn--l1acc ("mon" Mongolian) : MN +мон + +// xn--mgbx4cd0ab ("Malaysia" Malay) : MY +مليسيا + +// xn--mgb9awbf ("Oman" Arabic) : OM +عمان + +// xn--ygbi2ammx ("Falasteen" Arabic) : PS +// The Palestinian National Internet Naming Authority (PNINA) +// http://www.pnina.ps +فلسطين + +// xn--90a3ac ("srb" Cyrillic) : RS +срб + +// xn--p1ai ("rf" Russian-Cyrillic) : RU +// http://www.cctld.ru/en/docs/rulesrf.php +рф + +// xn--wgbl6a ("Qatar" Arabic) : QA +// http://www.ict.gov.qa/ +قطر + +// xn--mgberp4a5d4ar ("AlSaudiah" Arabic) : SA +// http://www.nic.net.sa/ +السعودية + +// xn--mgberp4a5d4a87g ("AlSaudiah" Arabic) variant : SA +السعودیة + +// xn--mgbqly7c0a67fbc ("AlSaudiah" Arabic) variant : SA +السعودیۃ + +// xn--mgbqly7cvafr ("AlSaudiah" Arabic) variant : SA +السعوديه + +// xn--ogbpf8fl ("Syria" Arabic) : SY +سورية + +// xn--mgbtf8fl ("Syria" Arabic) variant : SY +سوريا + +// xn--yfro4i67o Singapore ("Singapore" Chinese-Han) : SG +新加坡 + +// xn--clchc0ea0b2g2a9gcd ("Singapore" Tamil) : SG +சிங்கப்பூர் + +// xn--o3cw4h ("Thai" Thai) : TH +// http://www.thnic.co.th +ไทย + +// xn--pgbs0dh ("Tunis") : TN +// http://nic.tn +تونس + +// xn--kpry57d ("Taiwan" Chinese-Han-Traditional) : TW +// http://www.twnic.net/english/dn/dn_07a.htm +台灣 + +// xn--kprw13d ("Taiwan" Chinese-Han-Simplified) : TW +// http://www.twnic.net/english/dn/dn_07a.htm +台湾 + +// xn--nnx388a ("Taiwan") variant : TW +臺灣 + +// xn--j1amh ("ukr" Cyrillic) : UA +укр + +// xn--mgb2ddes ("AlYemen" Arabic) : YE +اليمن + +// xxx : http://icmregistry.com +xxx + +// ye : http://www.y.net.ye/services/domain_name.htm +*.ye + +// za : http://www.zadna.org.za/slds.html +*.za + +// zm : http://en.wikipedia.org/wiki/.zm +*.zm + +// zw : http://en.wikipedia.org/wiki/.zw +*.zw + + +// xn--80asehdb : 2013-07-14 CORE Association +онлайн + +// xn--80aswg : 2013-07-14 CORE Association +сайт + +// xn--ngbc5azd : 2013-07-14 International Domain Registry Pty. Ltd. +شبكة + +// xn--unup4y : 2013-07-14 Spring Fields, LLC +游戏 + +// xn--vhquv : 2013-08-28 Dash McCook, LLC +企业 + +// camera : 2013-08-28 Atomic Maple, LLC +camera + +// clothing : 2013-08-28 Steel Lake, LLC +clothing + +// lighting : 2013-08-28 John McCook, LLC +lighting + +// singles : 2013-08-28 Fern Madison, LLC +singles + +// ventures : 2013-08-28 Binky Lake, LLC +ventures + +// voyage : 2013-08-28 Ruby House, LLC +voyage + +// guru : 2013-08-28 Pioneer Cypress, LLC +guru + +// holdings : 2013-08-28 John Madison, LLC +holdings + +// equipment : 2013-08-28 Corn Station, LLC +equipment + +// bike : 2013-08-28 Grand Hollow, LLC +bike + +// estate : 2013-08-28 Trixy Park, LLC +estate + +// tattoo : 2013-08-30 Uniregistry,Corp. +tattoo + +// xn--3ds443g : 2013-09-09 TLD Registry Limited +在线 + +// xn--fiq228c5hs : 2013-09-09 TLD Registry Limited +中文网 + +// land : 2013-09-10 Pine Moon, LLC +land + +// plumbing : 2013-09-10 Spring Tigers, LLC +plumbing + +// contractors : 2013-09-10 Magic Woods, LLC +contractors + +// sexy : 2013-09-11 Uniregistry,Corp. +sexy + +// menu : 2013-09-11 Wedding TLD2, LLC +menu + +// xn--rhqv96g : 2013-09-11 Stable Tone Limited +世界 + +// uno : 2013-09-11 Dot Latin, LLC +uno + +// gallery : 2013-09-13 Sugar House, LLC +gallery + +// technology : 2013-09-13 Auburn Falls +technology + +// xn--3bst00m : 2013-09-13 Eagle Horizon Limited +集团 + +// reviews : 2013-09-13 Extra Cover, LLC +reviews + +// guide : 2013-09-13 Snow Moon, LLC +guide + +// xn--6qq986b3x1 : 2013-09-13 Tycoon Treasure Limited +我爱你 + +// graphics : 2013-09-13 Over Madison, LLC +graphics + +// construction : 2013-09-13 Fox Dynamite, LLC +construction + +// onl : 2013-09-16 I-Registry Ltd. +onl + +// xn--q9jyb4c : 2013-09-17 Charleston Road Registry +みんな + +// diamonds : 2013-09-23 John Edge, LLC +diamonds + +// kiwi : 2013-09-23 Dot Kiwi Limited +kiwi + +// enterprises : 2013-09-23 Snow Oaks LLC +enterprises + +// today : 2013-09-23 Pearl Woods, LLC +today + +// futbol : 2013-09-23 Atomic Falls, LLC +futbol + +// photography : 2013-09-23 Sugar Glen, LLC +photography + +// tips : 2013-09-23 Corn Willow, LLC +tips + +// directory : 2013-09-23 Extra Madison, LLC +directory + +// kitchen : 2013-09-23 Just Goodbye, LLC +kitchen + +// xn--6frz82g : 2013-09-24 Afilias Limited +移动 + +// kim : 2013-09-24 Afilias Limited +kim + +// xn--cg4bki : 2013-09-27 Samsung SDS Co., LTD +삼성 + +// monash : 2013-10-01 Monash University +monash + +// wed : 2013-10-02 Atgron, Inc. +wed + +// pink : 2013-10-02 Afilias Limited +pink + +// ruhr : 2013-10-02 regiodot GmbH & Co. KG +ruhr + +// buzz : 2013-10-03 DOTSTRATEGY CO. +buzz + +// careers : 2013-10-03 Wild Corner, LLC +careers + +// shoes : 2013-10-03 Binky Galley, LLC +shoes + +// xn--4gbrim : 2013-10-07 Suhub Electronic Establishment +موقع + +// career : 2013-10-09 dotCareer, LLC +career + +// otsuka : 2013-10-11 Otsuka Holdings Co. Ltd. +otsuka + +// xn--fiQ64b : 2013-10-14 CITIC Group Corporation +中信 + +// gift : 2013-10-18 Uniregistry Corp. +gift + +// recipes : 2013-10-18 Grand Island, LLC +recipes + +// coffee : 2013-10-18 Trixy Cover, LLC +coffee + +// luxury : 2013-10-18 Luxury Partners, LLC +luxury + +// domains : 2013-10-18 Sugar Cross, LLC +domains + +// photos : 2013-10-18 Sea Corner, LLC +photos + +// limo : 2013-10-18 Hidden Frostbite, LLC +limo + +// viajes : 2013-10-18 Black Madison, LLC +viajes + +// wang : 2013-10-24 Zodiac Leo Limited +wang + +// democrat : 2013-10-24 United TLD Holdco Ltd. +democrat + +// mango : 2013-10-25 PUNTO FA S.L. +mango + +// cab : 2013-10-25 Half Sunset, LLC +cab + +// support : 2013-10-25 Grand Orchard, LLC +support + +// dance : 2013-10-25 United TLD Holdco Ltd. +dance + +// nagoya : 2013-10-25 GMO Registry, Inc. +nagoya + +// computer : 2013-10-25 Pine Mill, LLC +computer + +// wien : 2013-10-28 punkt.wien GmbH +wien + +// berlin : 2013-10-31 dotBERLIN GmbH & Co. KG +berlin + +// codes : 2013-10-31 Puff Willow, LLC +codes + +// email : 2013-10-31 Spring Madison, LLC +email + +// xn--mgbab2bd : 2013-10-31 CORE Association +بازار + +// repair : 2013-11-07 Lone Sunset, LLC +repair + +// holiday : 2013-11-07 Goose Woods, LLC +holiday + +// center : 2013-11-07 Tin Mill, LLC +center + +// systems : 2013-11-07 Dash Cypress, LLC +systems + +// wiki : 2013-11-07 Top Level Design, LLC +wiki + +// ceo : 2013-11-07 CEOTLD Pty Ltd +ceo + +// international : 2013-11-07 Wild Way, LLC +international + +// solar : 2013-11-07 Ruby Town, LLC +solar + +// company : 2013-11-07 Silver Avenue, LLC +company + +// education : 2013-11-07 Brice Way, LLC +education + +// training : 2013-11-07 Wild Willow, LLC +training + +// academy : 2013-11-07 Half Oaks, LLC +academy + +// marketing : 2013-11-07 Fern Pass, LLC +marketing + +// florist : 2013-11-08 Half Cypress, LLC +florist + +// solutions : 2013-11-07 Silver Cover, LLC +solutions + +// build : 2013-11-07 Plan Bee LLC +build + +// institute : 2013-11-07 Outer Maple, LLC +institute + +// builders : 2013-11-07 Atomic Madison, LLC +builders + +// red : 2013-11-07 Afilias Limited +red + +// blue : 2013-11-07 Afilias Limited +blue + +// ninja : 2013-11-07 United TLD Holdco Ltd. +ninja + +// business : 2013-11-07 Spring Cross, LLC +business + +// gal : 2013-11-07 Asociación puntoGAL +gal + +// social : 2013-11-07 United TLD Holdco Ltd. +social + +// house : 2013-11-07 Sugar Park, LLC +house + +// camp : 2013-11-07 Delta Dynamite, LLC +camp + +// immobilien : 2013-11-07 United TLD Holdco Ltd. +immobilien + +// moda : 2013-11-07 United TLD Holdco Ltd. +moda + +// glass : 2013-11-07 Black Cover, LLC +glass + +// management : 2013-11-07 John Goodbye, LLC +management + +// kaufen : 2013-11-07 United TLD Holdco Ltd. +kaufen + +// farm : 2013-11-07 Just Maple, LLC +farm + +// xn--55qw42g : 2013-11-08 China Organizational Name Administration Center +公益 + +// xn--zfr164b : 2013-11-08 China Organizational Name Administration Center +政务 + +// club : 2013-11-08 .CLUB DOMAINS, LLC +club + +// voting : 2013-11-13 Valuetainment Corp. +voting + +// TOKYO : 2013-11-13 GMO Registry, Inc. +tokyo + +// moe : 2013-11-13 Interlink Co., Ltd. +moe + +// guitars : 2013-11-14 Uniregistry, Corp. +guitars + +// bargains : 2013-11-14 Half Hallow, LLC +bargains + +// xn--nqv7fs00ema : 2013-11-14 Public Interest Registry +组织机构 + +// desi : 2013-11-14 Desi Networks LLC +desi + +// cool : 2013-11-14 Koko Lake, LLC +cool + +// boutique : 2013-11-14 Over Galley, LLC +boutique + +// pics : 2013-11-14 Uniregistry, Corp. +pics + +// xn--c1avg : 2013-11-14 Public Interest Registry +орг + +// xn--55qx5d : 2013-11-14 Computer Network Information Center of Chinese Academy of Sciences （China Internet Network Information Center） +公司 + +// xn--io0a7i : 2013-11-14 Computer Network Information Center of Chinese Academy of Sciences （China Internet Network Information Center） +网络 + +// cheap : 2013-11-14 Sand Cover, LLC +cheap + +// xn--xhq521b : 2013-11-14 Guangzhou YU Wei Information Technology Co., Ltd. +广东 + +// photo : 2013-11-14 Uniregistry, Corp. +photo + +// network : 2013-11-14 Trixy Manor, LLC +network + +// zone : 2013-11-14 Outer Falls, LLC +zone + +// xn--nqv7f : 2013-11-14 Public Interest Registry +机构 + +// link : 2013-11-14 Uniregistry, Corp. +link + +// QPON : 2013-11-14 dotCOOL, Inc. +qpon + +// xn--i1b6b1a6a2e : 2013-11-14 Public Interest Registry +संगठन + +// agency : 2013-11-14 Steel Falls, LLC +agency + +// tienda : 2013-11-14 Victor Manor, LLC +tienda + +// works : 2013-11-14 Little Dynamite, LLC +works + +// london : 2013-11-14 Dot London Domains Limited +london + +// watch : 2013-11-14 Sand Shadow, LLC +watch + +// rocks : 2013-11-14 Ruby Moon, LLC +rocks + +// SHIKSHA : 2013-11-14 Afilias Limited +shiksha + +// xn--d1acj3b : 2013-11-21 The Foundation for Network Initiatives “The Smart Internet” +дети + +// budapest : 2013-11-21 Top Level Domain Holdings Limited +budapest + +// nrw : 2013-11-21 Minds + Machines GmbH +nrw + +// VOTE : 2013-11-21 Monolith Registry LLC +vote + +// fishing : 2013-11-21 Top Level Domain Holdings Limited +fishing + +// expert : 2013-11-21 Magic Pass, LLC +expert + +// horse : 2013-11-21 Top Level Domain Holdings Limited +horse + +// christmas : 2013-11-21 Uniregistry, Corp. +christmas + +// cooking : 2013-11-21 Top Level Domain Holdings Limited +cooking + +// xn--czru2d : 2013-11-21 Zodiac Capricorn Limited +商城 + +// casa : 2013-11-21 Top Level Domain Holdings Limited +casa + +// rich : 2013-11-21 I-REGISTRY Ltd., Niederlassung Deutschland +rich + +// VOTO : 2013-11-21 Monolith Registry LLC +voto + +// tools : 2013-11-21 Pioneer North, LLC +tools + +// xn--45q11c : 2013-11-21 Zodiac Scorpio Limited +八卦 + +// praxi : 2013-12-05 Praxi S.p.A. +praxi + +// events : 2013-12-05 Pioneer Maple, LLC +events + +// flights : 2013-12-05 Fox Station, LLC +flights + +// report : 2013-12-05 Binky Glen, LLC +report + +// partners : 2013-12-05 Magic Glen, LLC +partners + +// neustar : 2013-12-05 NeuStar, Inc. +neustar + +// rentals : 2013-12-05 Big Hollow,LLC +rentals + +// catering : 2013-12-05 New Falls. LLC +catering + +// community : 2013-12-05 Fox Orchard, LLC +community + +// maison : 2013-12-05 Victor Frostbite, LLC +maison + +// parts : 2013-12-05 Sea Goodbye, LLC +parts + +// cleaning : 2013-12-05 Fox Shadow, LLC +cleaning + +// okinawa : 2013-12-05 BusinessRalliart inc. +okinawa + +// foundation : 2013-12-05 John Dale, LLC +foundation + +// properties : 2013-12-05 Big Pass, LLC +properties + +// vacations : 2013-12-05 Atomic Tigers, LLC +vacations + +// productions : 2013-12-05 Magic Birch, LLC +productions + +// industries : 2013-12-05 Outer House, LLC +industries + +// haus : 2013-12-05 Pixie Edge, LLC +haus + +// vision : 2013-12-05 Koko Station, LLC +vision + +// mormon : 2013-12-05 IRI Domain Management, LLC (""Applicant"") +mormon + +// cards : 2013-12-05 Foggy Hollow, LLC +cards + +// ink : 2013-12-05 Top Level Design, LLC +ink + +// villas : 2013-12-05 New Sky, LLC +villas + +// consulting : 2013-12-05 Pixie Station, LLC +consulting + +// cruises : 2013-12-05 Spring Way, LLC +cruises + +// krd : 2013-12-05 KRG Department of Information Technology +krd + +// xyz : 2013-12-05 XYZ.COM LLC +xyz + +// dating : 2013-12-05 Pine Fest, LLC +dating + +// exposed : 2013-12-05 Victor Beach, LLC +exposed + +// condos : 2013-12-05 Pine House, LLC +condos + +// eus : 2013-12-12 Puntueus Fundazioa +eus + +// Caravan : 2013-12-12 Caravan International, Inc. +caravan + +// actor : 2013-12-12 United TLD Holdco Ltd. +actor + +// saarland : 2013-12-12 dotSaarland GmbH +saarland + +// yokohama : 2013-12-12 GMO Registry, Inc. +yokohama + +// pub : 2013-12-12 United TLD Holdco Ltd. +pub + +// xn--p1acf : 2013-12-12 Rusnames Limited +рус + +// ren : 2013-12-12 Beijing Qianxiang Wangjing Technology Development Co., Ltd. +ren + +// fish : 2013-12-12 Fox Woods, LLC +fish + +// BAR : 2013-12-12 Punto 2012 Sociedad Anonima Promotora de Inversion de Capital Variable +bar + +// DNP : 2013-12-13 Dai Nippon Printing Co., Ltd. +dnp + +// bid : 2013-12-19 dot Bid Limited +bid + +// supply : 2013-12-19 Half Falls, LLC +supply + +// Miami : 2013-12-19 Top Level Domain Holdings Limited +miami + +// supplies : 2013-12-19 Atomic Fields, LLC +supplies + +// quebec : 2013-12-19 PointQuébec Inc +quebec + +// MOSCOW : 2013-12-19 Foundation for Assistance for Internet Technologies and Infrastructure Development (FAITID) +moscow + +// globo : 2013-12-19 Globo Comunicação e Participações S.A +globo + +// AXA : 2013-12-19 AXA SA +axa + +// xn--80adxhks : 2013-12-19 Foundation for Assistance for Internet Technologies and Infrastructure Development (FAITID) +москва + +// xn--czrs0t : 2013-12-19 Wild Island, LLC +商店 + +// vodka : 2013-12-19 Top Level Domain Holdings Limited +vodka + +// REST : 2013-12-19 Punto 2012 Sociedad Anonima Promotora de Inversion de Capital Variable +rest + +// frogans : 2013-12-19 OP3FT +frogans + +// WTC : 2013-12-19 World Trade Centers Association, Inc. +wtc + +// rodeo : 2013-12-19 Top Level Domain Holdings Limited +rodeo + +// sohu : 2013-12-19 Sohu.com Limited +sohu + +// BEST : 2013-12-19 BestTLD Pty Ltd +best + +// country : 2013-12-19 Top Level Domain Holdings Limited +country + +// KRED : 2013-12-19 KredTLD Pty Ltd +kred + +// feedback : 2013-12-19 Top Level Spectrum, Inc. +feedback + +// work : 2013-12-19 Top Level Domain Holdings Limited +work + +// luxe : 2014-01-09 Top Level Domain Holdings Limited +luxe + +// ryukyu : 2014-01-09 BusinessRalliart inc. +ryukyu + +// autos : 2014-01-09 DERAutos, LLC +autos + +// homes : 2014-01-09 DERHomes, LLC +homes + +// jetzt : 2014-01-09 New TLD Company AB +jetzt + +// yachts : 2014-01-09 DERYachts, LLC +yachts + +// motorcycles : 2014-01-09 DERMotorcycles, LLC +motorcycles + +// mini : 2014-01-09 Bayerische Motoren Werke Aktiengesellschaft +mini + +// ggee : 2014-01-09 GMO Internet, Inc. +ggee + +// beer : 2014-01-09 Top Level Domain Holdings Limited +beer + +// xn--1qqw23a : 2014-01-13 Guangzhou YU Wei Information Technology Co., Ltd. +佛山 + +// college : 2014-01-16 XYZ.COM LLC +college + +// ovh : 2014-01-16 OVH SAS +ovh + +// meet : 2014-01-16 Afilias Limited +meet + +// xn--ses554g : 2014-01-16 HU YI GLOBAL INFORMATION RESOURCES (HOLDING) COMPANY. HONGKONG LIMITED +网址 + +// gop : 2014-01-16 Republican State Leadership Committee, Inc. +gop + +// blackfriday : 2014-01-16 Uniregistry, Corp. +blackfriday + +// lacaixa : 2014-01-16 CAIXA D'ESTALVIS I PENSIONS DE BARCELONA +lacaixa + +// xn--czr694b : 2014-01-16 HU YI GLOBAL INFORMATION RESOURCES(HOLDING) COMPANY.HONGKONG LIMITED +商标 + +// vegas : 2014-01-16 Dot Vegas, Inc. +vegas + +// black : 2014-01-16 Afilias Limited +black + +// soy : 2014-01-23 Charleston Road Registry Inc. +soy + +// trade : 2014-01-23 Elite Registry Limited +trade + +// gent : 2014-01-23 COMBELL GROUP NV/SA +gent + +// ing : 2014-01-23 Charleston Road Registry Inc. +ing + +// dad : 2014-01-23 Charleston Road Registry Inc. +dad + +// shriram : 2014-01-23 Shriram Capital Ltd. +shriram + +// bayern : 2014-01-23 Bayern Connect GmbH +bayern + +// scot : 2014-01-23 Dot Scot Registry Limited +scot + +// webcam : 2014-01-23 dot Webcam Limited +webcam + +// foo : 2014-01-23 Charleston Road Registry Inc. +foo + +// eat : 2014-01-23 Charleston Road Registry Inc. +eat + +// nyc : 2014-01-23 The City of New York +nyc + +// prod : 2014-01-23 Charleston Road Registry Inc. +prod + +// how : 2014-01-23 Charleston Road Registry Inc. +how + +// day : 2014-01-30 Charleston Road Registry Inc. +day + +// meme : 2014-01-30 Charleston Road Registry Inc. +meme + +// mov : 2014-01-30 Charleston Road Registry Inc. +mov + +// paris : 2014-01-30 City of Paris +paris + +// boo : 2014-01-30 Charleston Road Registry Inc. +boo + +// new : 2014-01-30 Charleston Road Registry Inc. +new + +// ifm : 2014-01-30 ifm electronic gmbh +ifm + +// life : 2014-02-06 Trixy Oaks, LLC +life + +// archi : 2014-02-06 STARTING DOT LIMITED +archi + +// spiegel : 2014-02-06 SPIEGEL-Verlag Rudolf Augstein GmbH & Co. KG +spiegel + +// brussels : 2014-02-06 DNS.be vzw +brussels + +// church : 2014-02-06 Holly Fileds, LLC +church + +// here : 2014-02-06 Charleston Road Registry Inc. +here + +// dabur : 2014-02-06 Dabur India Limited +dabur + +// vlaanderen : 2014-02-06 DNS.be vzw +vlaanderen + +// cologne : 2014-02-06 NetCologne Gesellschaft für Telekommunikation mbH +cologne + +// xn--kput3i : 2014-02-13 Beijing RITT-Net Technology Development Co., Ltd +手机 + +// wme : 2014-02-13 William Morris Endeavor Entertainment, LLC +wme + +// nhk : 2014-02-13 Japan Broadcasting Corporation (NHK) +nhk + +// suzuki : 2014-02-20 SUZUKI MOTOR CORPORATION +suzuki + +// whoswho : 2014-02-20 Who's Who Registry +whoswho + +// scb : 2014-02-20 The Siam Commercial Bank Public Company Limited ("SCB""\) +scb + +// hamburg : 2014-02-20 Hamburg Top-Level-Domain GmbH +hamburg + +// services : 2014-02-27 Fox Castle, LLC +services + +// bzh : 2014-02-27 Association www.bzh +bzh + +// rio : 2014-02-27 Empresa Municipal de Informática SA - IPLANRIO +rio + + +// ===END ICANN DOMAINS=== +// ===BEGIN PRIVATE DOMAINS=== + +// Amazon CloudFront : https://aws.amazon.com/cloudfront/ +// Submitted by Donavan Miller 2013-03-22 +cloudfront.net + +// Amazon Elastic Compute Cloud: https://aws.amazon.com/ec2/ +// Submitted by Osman Surkatty 2013-04-02 +compute.amazonaws.com +us-east-1.amazonaws.com +compute-1.amazonaws.com +z-1.compute-1.amazonaws.com +z-2.compute-1.amazonaws.com +ap-northeast-1.compute.amazonaws.com +ap-southeast-1.compute.amazonaws.com +ap-southeast-2.compute.amazonaws.com +eu-west-1.compute.amazonaws.com +sa-east-1.compute.amazonaws.com +us-gov-west-1.compute.amazonaws.com +us-west-1.compute.amazonaws.com +us-west-2.compute.amazonaws.com + +// Amazon Elastic Beanstalk : https://aws.amazon.com/elasticbeanstalk/ +// Submitted by Adam Stein 2013-04-02 +elasticbeanstalk.com + +// Amazon Elastic Load Balancing : https://aws.amazon.com/elasticloadbalancing/ +// Submitted by Scott Vidmar 2013-03-27 +elb.amazonaws.com + +// Amazon S3 : https://aws.amazon.com/s3/ +// Submitted by Courtney Eckhardt 2013-03-22 +s3.amazonaws.com +s3-us-west-2.amazonaws.com +s3-us-west-1.amazonaws.com +s3-eu-west-1.amazonaws.com +s3-ap-southeast-1.amazonaws.com +s3-ap-southeast-2.amazonaws.com +s3-ap-northeast-1.amazonaws.com +s3-sa-east-1.amazonaws.com +s3-us-gov-west-1.amazonaws.com +s3-fips-us-gov-west-1.amazonaws.com +s3-website-us-east-1.amazonaws.com +s3-website-us-west-2.amazonaws.com +s3-website-us-west-1.amazonaws.com +s3-website-eu-west-1.amazonaws.com +s3-website-ap-southeast-1.amazonaws.com +s3-website-ap-southeast-2.amazonaws.com +s3-website-ap-northeast-1.amazonaws.com +s3-website-sa-east-1.amazonaws.com +s3-website-us-gov-west-1.amazonaws.com + +// BetaInABox +// Submitted by adrian@betainabox.com 2012-09-13 +betainabox.com + +// CentralNic : http://www.centralnic.com/names/domains +// Submitted by registry 2012-09-27 +ae.org +ar.com +br.com +cn.com +com.de +de.com +eu.com +gb.com +gb.net +gr.com +hu.com +hu.net +jp.net +jpn.com +kr.com +no.com +qc.com +ru.com +sa.com +se.com +se.net +uk.com +uk.net +us.com +us.org +uy.com +za.com + +// c.la : http://www.c.la/ +c.la + +// cloudControl : https://www.cloudcontrol.com/ +// Submitted by Tobias Wilken 2013-07-23 +cloudcontrolled.com +cloudcontrolapp.com + +// co.ca : http://registry.co.ca/ +co.ca + +// CoDNS B.V. +co.nl +co.no + +// Cupcake : https://cupcake.io/ +// Submitted by Jonathan Rudenberg 2013-10-08 +cupcake.is + +// DreamHost : http://www.dreamhost.com/ +// Submitted by Andrew Farmer 2012-10-02 +dreamhosters.com + +// DynDNS.com : http://www.dyndns.com/services/dns/dyndns/ +dyndns-at-home.com +dyndns-at-work.com +dyndns-blog.com +dyndns-free.com +dyndns-home.com +dyndns-ip.com +dyndns-mail.com +dyndns-office.com +dyndns-pics.com +dyndns-remote.com +dyndns-server.com +dyndns-web.com +dyndns-wiki.com +dyndns-work.com +dyndns.biz +dyndns.info +dyndns.org +dyndns.tv +at-band-camp.net +ath.cx +barrel-of-knowledge.info +barrell-of-knowledge.info +better-than.tv +blogdns.com +blogdns.net +blogdns.org +blogsite.org +boldlygoingnowhere.org +broke-it.net +buyshouses.net +cechire.com +dnsalias.com +dnsalias.net +dnsalias.org +dnsdojo.com +dnsdojo.net +dnsdojo.org +does-it.net +doesntexist.com +doesntexist.org +dontexist.com +dontexist.net +dontexist.org +doomdns.com +doomdns.org +dvrdns.org +dyn-o-saur.com +dynalias.com +dynalias.net +dynalias.org +dynathome.net +dyndns.ws +endofinternet.net +endofinternet.org +endoftheinternet.org +est-a-la-maison.com +est-a-la-masion.com +est-le-patron.com +est-mon-blogueur.com +for-better.biz +for-more.biz +for-our.info +for-some.biz +for-the.biz +forgot.her.name +forgot.his.name +from-ak.com +from-al.com +from-ar.com +from-az.net +from-ca.com +from-co.net +from-ct.com +from-dc.com +from-de.com +from-fl.com +from-ga.com +from-hi.com +from-ia.com +from-id.com +from-il.com +from-in.com +from-ks.com +from-ky.com +from-la.net +from-ma.com +from-md.com +from-me.org +from-mi.com +from-mn.com +from-mo.com +from-ms.com +from-mt.com +from-nc.com +from-nd.com +from-ne.com +from-nh.com +from-nj.com +from-nm.com +from-nv.com +from-ny.net +from-oh.com +from-ok.com +from-or.com +from-pa.com +from-pr.com +from-ri.com +from-sc.com +from-sd.com +from-tn.com +from-tx.com +from-ut.com +from-va.com +from-vt.com +from-wa.com +from-wi.com +from-wv.com +from-wy.com +ftpaccess.cc +fuettertdasnetz.de +game-host.org +game-server.cc +getmyip.com +gets-it.net +go.dyndns.org +gotdns.com +gotdns.org +groks-the.info +groks-this.info +ham-radio-op.net +here-for-more.info +hobby-site.com +hobby-site.org +home.dyndns.org +homedns.org +homeftp.net +homeftp.org +homeip.net +homelinux.com +homelinux.net +homelinux.org +homeunix.com +homeunix.net +homeunix.org +iamallama.com +in-the-band.net +is-a-anarchist.com +is-a-blogger.com +is-a-bookkeeper.com +is-a-bruinsfan.org +is-a-bulls-fan.com +is-a-candidate.org +is-a-caterer.com +is-a-celticsfan.org +is-a-chef.com +is-a-chef.net +is-a-chef.org +is-a-conservative.com +is-a-cpa.com +is-a-cubicle-slave.com +is-a-democrat.com +is-a-designer.com +is-a-doctor.com +is-a-financialadvisor.com +is-a-geek.com +is-a-geek.net +is-a-geek.org +is-a-green.com +is-a-guru.com +is-a-hard-worker.com +is-a-hunter.com +is-a-knight.org +is-a-landscaper.com +is-a-lawyer.com +is-a-liberal.com +is-a-libertarian.com +is-a-linux-user.org +is-a-llama.com +is-a-musician.com +is-a-nascarfan.com +is-a-nurse.com +is-a-painter.com +is-a-patsfan.org +is-a-personaltrainer.com +is-a-photographer.com +is-a-player.com +is-a-republican.com +is-a-rockstar.com +is-a-socialist.com +is-a-soxfan.org +is-a-student.com +is-a-teacher.com +is-a-techie.com +is-a-therapist.com +is-an-accountant.com +is-an-actor.com +is-an-actress.com +is-an-anarchist.com +is-an-artist.com +is-an-engineer.com +is-an-entertainer.com +is-by.us +is-certified.com +is-found.org +is-gone.com +is-into-anime.com +is-into-cars.com +is-into-cartoons.com +is-into-games.com +is-leet.com +is-lost.org +is-not-certified.com +is-saved.org +is-slick.com +is-uberleet.com +is-very-bad.org +is-very-evil.org +is-very-good.org +is-very-nice.org +is-very-sweet.org +is-with-theband.com +isa-geek.com +isa-geek.net +isa-geek.org +isa-hockeynut.com +issmarterthanyou.com +isteingeek.de +istmein.de +kicks-ass.net +kicks-ass.org +knowsitall.info +land-4-sale.us +lebtimnetz.de +leitungsen.de +likes-pie.com +likescandy.com +merseine.nu +mine.nu +misconfused.org +mypets.ws +myphotos.cc +neat-url.com +office-on-the.net +on-the-web.tv +podzone.net +podzone.org +readmyblog.org +saves-the-whales.com +scrapper-site.net +scrapping.cc +selfip.biz +selfip.com +selfip.info +selfip.net +selfip.org +sells-for-less.com +sells-for-u.com +sells-it.net +sellsyourhome.org +servebbs.com +servebbs.net +servebbs.org +serveftp.net +serveftp.org +servegame.org +shacknet.nu +simple-url.com +space-to-rent.com +stuff-4-sale.org +stuff-4-sale.us +teaches-yoga.com +thruhere.net +traeumtgerade.de +webhop.biz +webhop.info +webhop.net +webhop.org +worse-than.tv +writesthisblog.com + +// Fastly Inc. http://www.fastly.com/ +// Submitted by Vladimir Vuksan 2013-05-31 +a.ssl.fastly.net +b.ssl.fastly.net +global.ssl.fastly.net +a.prod.fastly.net +global.prod.fastly.net + +// GitHub, Inc. +// Submitted by Ben Toews 2014-02-06 +github.io +githubusercontent.com + +// GlobeHosting, Inc. +// Submitted by Zoltan Egresi 2013-07-12 +ro.com + +// Google, Inc. +// Submitted by Eduardo Vela 2012-10-24 +appspot.com +blogspot.be +blogspot.bj +blogspot.ca +blogspot.cf +blogspot.ch +blogspot.co.at +blogspot.co.il +blogspot.co.nz +blogspot.co.uk +blogspot.com +blogspot.com.ar +blogspot.com.au +blogspot.com.br +blogspot.com.es +blogspot.cv +blogspot.cz +blogspot.de +blogspot.dk +blogspot.fi +blogspot.fr +blogspot.gr +blogspot.hk +blogspot.hu +blogspot.ie +blogspot.in +blogspot.it +blogspot.jp +blogspot.kr +blogspot.mr +blogspot.mx +blogspot.nl +blogspot.no +blogspot.pt +blogspot.re +blogspot.ro +blogspot.se +blogspot.sg +blogspot.sk +blogspot.td +blogspot.tw +codespot.com +googleapis.com +googlecode.com +withgoogle.com + +// Heroku : https://www.heroku.com/ +// Submitted by Tom Maher 2013-05-02 +herokuapp.com +herokussl.com + +// iki.fi +// Submitted by Hannu Aronsson 2009-11-05 +iki.fi + +// info.at : http://www.info.at/ +biz.at +info.at + +// Michau Enterprises Limited : http://www.co.pl/ +co.pl + +// Microsoft : http://microsoft.com +// Submitted by Barry Dorrans 2014-01-24 +azurewebsites.net +azure-mobile.net +cloudapp.net + +// NYC.mn : http://www.information.nyc.mn +// Submitted by Matthew Brown 2013-03-11 +nyc.mn + +// Opera Software, A.S.A. +// Submitted by Yngve Pettersen 2009-11-26 +operaunite.com + +// Red Hat, Inc. OpenShift : https://openshift.redhat.com/ +// Submitted by Tim Kramer 2012-10-24 +rhcloud.com + +// priv.at : http://www.nic.priv.at/ +// Submitted by registry 2008-06-09 +priv.at + +// ZaNiC : http://www.za.net/ +// Submitted by registry 2009-10-03 +za.net +za.org + +// ===END PRIVATE DOMAINS=== diff --git a/include/libpsl.h b/include/libpsl.h index e69de29..fd2a404 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -0,0 +1,60 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * This file is part of libpsl. + * + * Libpsl is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Libpsl is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with libpsl. If not, see . + * + * + * Header file for libpsl library routines + * + * Changelog + * 20.03.2014 Tim Ruehsen created + * + */ + +#ifndef _LIBPSL_LIBPSL_H +#define _LIBPSL_LIBPSL_H + +// Let C++ include C headers +#ifdef __cplusplus +# define PSL_BEGIN_DECLS extern "C" { +# define PSL_END_DECLS } +#else +# define PSL_BEGIN_DECLS +# define PSL_END_DECLS +#endif + +#if ENABLE_NLS != 0 +# include +# define _(STRING) gettext(STRING) +#else +# define _(STRING) STRING +# define ngettext(STRING1,STRING2,N) STRING2 +#endif + +PSL_BEGIN_DECLS + +typedef struct _psl_ctx_st psl_ctx_t; + +void + psl_free(psl_ctx_t **psl); +psl_ctx_t * + psl_load_file(const char *fname); +int + psl_is_tld(const psl_ctx_t *psl, const char *domain); + +PSL_END_DECLS + +#endif /* _LIBPSL_LIBPSL_H */ diff --git a/libpsl.pc.in b/libpsl.pc.in new file mode 100644 index 0000000..13ffa3c --- /dev/null +++ b/libpsl.pc.in @@ -0,0 +1,11 @@ +prefix=@prefix@ +exec_prefix=@exec_prefix@ +libdir=@libdir@ +includedir=@includedir@ + +Name: @PACKAGE_NAME@ +Description: Publix Suffix List C library. +Version: @PACKAGE_VERSION@ +URL: @PACKAGE_URL@ +Libs: -L${libdir} -llibpsl-@LIBPSL_API_VERSION@ +Cflags: -I${includedir}/libpsl-@LIBPSL_API_VERSION@ -I${libdir}/libpsl-@LIBPSL_API_VERSION@/include diff --git a/po/Makevars b/po/Makevars new file mode 100644 index 0000000..1da13e7 --- /dev/null +++ b/po/Makevars @@ -0,0 +1,53 @@ +# Makefile variables for PO directory in any package using GNU gettext. + +# Usually the message domain is the same as the package name. +DOMAIN = $(PACKAGE) + +# These two variables depend on the location of this directory. +subdir = po +top_builddir = .. + +# These options get passed to xgettext. +XGETTEXT_OPTIONS = --keyword=_ --keyword=N_ + +# This is the copyright holder that gets inserted into the header of the +# $(DOMAIN).pot file. Set this to the copyright holder of the surrounding +# package. (Note that the msgstr strings, extracted from the package's +# sources, belong to the copyright holder of the package.) Translators are +# expected to transfer the copyright for their translations to this person +# or entity, or to disclaim their copyright. The empty string stands for +# the public domain; in this case the translators are expected to disclaim +# their copyright. +COPYRIGHT_HOLDER = Tim Ruehsen + +# This is the email address or URL to which the translators shall report +# bugs in the untranslated strings: +# - Strings which are not entire sentences, see the maintainer guidelines +# in the GNU gettext documentation, section 'Preparing Strings'. +# - Strings which use unclear terms or require additional context to be +# understood. +# - Strings which make invalid assumptions about notation of date, time or +# money. +# - Pluralisation problems. +# - Incorrect English spelling. +# - Incorrect formatting. +# It can be your email address, or a mailing list address where translators +# can write to without being subscribed, or the URL of a web page through +# which the translators can contact you. +MSGID_BUGS_ADDRESS = tim.ruehsen@gmx.de + +# This is the list of locale categories, beyond LC_MESSAGES, for which the +# message catalogs shall be used. It is usually empty. +EXTRA_LOCALE_CATEGORIES = + +# This tells whether the $(DOMAIN).pot file contains messages with an 'msgctxt' +# context. Possible values are "yes" and "no". Set this to yes if the +# package uses functions taking also a message context, like pgettext(), or +# if in $(XGETTEXT_OPTIONS) you define keywords with a context argument. +USE_MSGCTXT = no + +# These options get passed to msgmerge. +# Useful options are in particular: +# --previous to keep previous msgids of translated messages, +# --quiet to reduce the verbosity. +MSGMERGE_OPTIONS = diff --git a/po/POTFILES.in b/po/POTFILES.in new file mode 100644 index 0000000..eb6164a --- /dev/null +++ b/po/POTFILES.in @@ -0,0 +1,2 @@ +# List of source files which contain translatable strings. +src/psl.c diff --git a/src/Makefile.am b/src/Makefile.am new file mode 100644 index 0000000..f8339d5 --- /dev/null +++ b/src/Makefile.am @@ -0,0 +1,14 @@ +lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la +libpsl_@LIBPSL_API_VERSION@_la_SOURCES = psl.c + +libpsl_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include + +# include ABI version information +libpsl_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) + +#bin_PROGRAMS = test_linking +#noinst_PROGRAMS = test_linking +#test_linking_SOURCES = test_linking.c +#test_linking_CPPFLAGS = -I$(top_srcdir)/include +#test_linking_LDADD = libpsl-@LIBPSL_API_VERSION@.la +#test_linking_LDFLAGS = -static diff --git a/src/psl.c b/src/psl.c new file mode 100644 index 0000000..85e9859 --- /dev/null +++ b/src/psl.c @@ -0,0 +1,333 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * This file is part of MGet. + * + * Mget is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Mget is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Mget. If not, see . + * + * + * Public Suffix List routines (right now experimental) + * + * Changelog + * 19.03.2014 Tim Ruehsen created from libmget/cookie.c + * + */ + +#define _GNU_SOURCE + +#if HAVE_CONFIG_H +# include +#endif + +#include +#include +#include +#include + +#include + +#define countof(a) (sizeof(a)/sizeof(*(a))) + +typedef struct { + char + label_buf[42]; + const char * + label; + unsigned short + length; + unsigned char + nlabels, // number of labels + wildcard; // this is a wildcard rule (e.g. *.sapporo.jp) +} _psl_entry_t; + +// stripped down version libmget vector routines +typedef struct { + int + (*cmp)(const _psl_entry_t *, const _psl_entry_t *); // comparison function + _psl_entry_t + **entry; // pointer to array of pointers to elements + int + max, // allocated elements + cur; // number of elements in use +} _psl_vector_t; + +struct _psl_ctx_st { + _psl_vector_t + *suffixes, + *suffix_exceptions; +}; + +static _psl_vector_t *_vector_alloc(int max, int (*cmp)(const _psl_entry_t *, const _psl_entry_t *)) +{ + _psl_vector_t *v; + + if (!(v = calloc(1, sizeof(_psl_vector_t)))) + return NULL; + + if (!(v->entry = malloc(max * sizeof(_psl_entry_t *)))) { + free(v); + return NULL; + } + + v->max = max; + v->cmp = cmp; + return v; +} + +static void _vector_free(_psl_vector_t **v) +{ + if (v && *v) { + if ((*v)->entry) { + int it; + + for (it = 0; it < (*v)->cur; it++) + free((*v)->entry[it]); + + free((*v)->entry); + } + free(*v); + } +} + +static _psl_entry_t *_vector_get(const _psl_vector_t *v, int pos) +{ + if (pos < 0 || !v || pos >= v->cur) return NULL; + + return v->entry[pos]; +} + +// the entries must be sorted by +static int _vector_find(const _psl_vector_t *v, const _psl_entry_t *elem) +{ + if (v) { + int l, r, m; + int res; + + // binary search for element (exact match) + for (l = 0, r = v->cur - 1; l <= r;) { + m = (l + r) / 2; + if ((res = v->cmp(elem, v->entry[m])) > 0) l = m + 1; + else if (res < 0) r = m - 1; + else return m; + } + } + + return -1; // not found +} + +static int _vector_add(_psl_vector_t *v, const _psl_entry_t *elem) +{ + if (v) { + void *elemp; + + elemp = malloc(sizeof(_psl_entry_t)); + memcpy(elemp, elem, sizeof(_psl_entry_t)); + + if (v->max == v->cur) + v->entry = realloc(v->entry, (v->max *= 2) * sizeof(_psl_entry_t *)); + + v->entry[v->cur++] = elemp; + return v->cur - 1; + } + + return -1; +} + +static int _compare(const void *p1, const void *p2, void *v) +{ + return ((_psl_vector_t *)v)->cmp(*((_psl_entry_t **)p1), *((_psl_entry_t **)p2)); +} + +static void _vector_sort(_psl_vector_t *v) +{ + if (v && v->cmp) + qsort_r(v->entry, v->cur, sizeof(_psl_vector_t *), _compare, v); +} + +// by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) + +static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) +{ + int n; + + if ((n = s2->nlabels - s1->nlabels)) + return n; // most labels first + + if ((n=s1->length - s2->length)) + return n; // shorter rules first + + return strcmp(s1->label, s2->label); +} + +static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) +{ + const char *src; + char *dst; + + suffix->label = suffix->label_buf; + + if (length >= sizeof(suffix->label_buf) - 1) { + suffix->nlabels = 0; + fprintf(stderr, _("Suffix rule too long (ignored): %s\n"), rule); + return; + } + + if (*rule == '*') { + if (*++rule != '.') { + suffix->nlabels = 0; + fprintf(stderr, _("Unsupported kind of rule (ignored): %s\n"), rule); + return; + } + rule++; + suffix->wildcard = 1; + suffix->length = (unsigned char)length - 2; + } else { + suffix->wildcard = 0; + suffix->length = (unsigned char)length; + } + + suffix->nlabels = 1; + + for (dst = suffix->label_buf, src = rule; *src;) { + if (*src == '.') + suffix->nlabels++; + *dst++ = tolower(*src++); + } + *dst = 0; +} + +int psl_is_tld(const psl_ctx_t *psl, const char *domain) +{ + _psl_entry_t suffix, *rule; + const char *p, *label_bak; + unsigned short length_bak; + + // this function should be called without leading dots, just make shure + suffix.label = domain + (*domain == '.'); + suffix.length = strlen(suffix.label); + suffix.wildcard = 0; + suffix.nlabels = 1; + + for (p = suffix.label; *p; p++) + if (*p == '.') + suffix.nlabels++; + + // if domain has enough labels, it won't match + rule = _vector_get(psl->suffixes, 0); + if (!rule || rule->nlabels < suffix.nlabels - 1) + return 0; + + rule = _vector_get(psl->suffixes, _vector_find(psl->suffixes, &suffix)); + if (rule) { + // definitely a match, no matter if the found rule is a wildcard or not + return 1; + } + + label_bak = suffix.label; + length_bak = suffix.length; + + if ((suffix.label = strchr(suffix.label, '.'))) { + suffix.label++; + suffix.length = strlen(suffix.label); + suffix.nlabels--; + + rule = _vector_get(psl->suffixes, _vector_find(psl->suffixes, &suffix)); + if (rule) { + if (rule->wildcard) { + // now that we matched a wildcard, we have to check for an exception + suffix.label = label_bak; + suffix.length = length_bak; + suffix.nlabels++; + + if (_vector_get(psl->suffix_exceptions, _vector_find(psl->suffix_exceptions, &suffix)) != 0) + return 0; + + return 1; + } + } + } + + return 0; +} + +psl_ctx_t *psl_load_file(const char *fname) +{ + psl_ctx_t *psl; + _psl_entry_t suffix, *suffixp; + FILE *fp; + int nsuffixes = 0; + char *buf = NULL, *linep, *p; + size_t bufsize = 0; + ssize_t buflen; + + if (!(psl = calloc(1, sizeof(psl_ctx_t)))) + return NULL; + + // as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. + // as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. + psl->suffixes = _vector_alloc(8*1024, _suffix_compare); + psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); + + if ((fp = fopen(fname, "r"))) { + while ((buflen = getline(&buf, &bufsize, fp)) >= 0) { + linep = buf; + + while (isspace(*linep)) linep++; // ignore leading whitespace + if (!*linep) continue; // skip empty lines + + if (*linep == '/' && linep[1] == '/') + continue; // skip comments + + // parse suffix rule + for (p = linep; *linep && !isspace(*linep);) linep++; + *linep = 0; + + if (*p == '!') { + // add to exceptions + _suffix_init(&suffix, p + 1, linep - p - 1); + suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); + } else { + _suffix_init(&suffix, p, linep - p); + suffixp = _vector_get(psl->suffixes, _vector_add(psl->suffixes, &suffix)); + } + + if (suffixp) + suffixp->label = suffixp->label_buf; // set label to changed address + + nsuffixes++;; + } + + free(buf); + fclose(fp); + + _vector_sort(psl->suffix_exceptions); + _vector_sort(psl->suffixes); + + printf("loaded %d (%d/%d) suffixes\n", nsuffixes, psl->suffixes->cur, psl->suffix_exceptions->cur); + + } else + fprintf(stderr, _("Failed to open PSL file '%s'\n"), fname); + + return psl; +} + +void psl_free(psl_ctx_t **psl) +{ + if (psl && *psl) { + _vector_free(&(*psl)->suffixes); + _vector_free(&(*psl)->suffix_exceptions); + free(*psl); + *psl = NULL; + } +} diff --git a/tests/Makefile.am b/tests/Makefile.am index 0055c82..8f5c810 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -2,31 +2,14 @@ DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" AM_CPPFLAGS = -Wno-missing-field-initializers -I$(top_srcdir)/include AM_LDFLAGS = -static -LDADD = libtest.la ../src/libpsl-@LIBPSL_API_VERSION@.la +LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la -PSL_TESTS = test +PSL_TESTS = test-is-tld check_PROGRAMS = $(PSL_TESTS) -test_SOURCES = test.c -test_LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la -test_parse_html_LDADD = ../libmget/libmget-@LIBPSL_API_VERSION@.la - -noinst_LTLIBRARIES = libtest.la -libtest_la_SOURCES = libtest.c -libtest_la_CPPFLAGS = -I$(top_srcdir)/tests -I$(top_srcdir)/include -#libtest_LDADD = libtest.o - -EXTRA_DIST = libtest.h -dist-hook: - rm -f $(distdir)/files/elb_bibel.txt -# cp $(top_srcdir)/data/public_suffixes.txt $(distdir)/files/ -# rm -rf `find $(distdir)/files -name CVS` - -#dist-hook: -# mkdir $(distdir)/random -# cp -p $(srcdir)/random/a1 $(srcdir)/random/a2 $(distdir)/random +test_is_tld_SOURCES = test-is-tld.c +test_is_tld_LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la TESTS_ENVIRONMENT = TESTS_VALGRIND="@VALGRIND_ENVIRONMENT@" -#TESTS = test $(PSL_TESTS) TESTS = $(PSL_TESTS) diff --git a/tests/test-is-tld.c b/tests/test-is-tld.c new file mode 100644 index 0000000..0f63b9b --- /dev/null +++ b/tests/test-is-tld.c @@ -0,0 +1,105 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * This file is part of MGet. + * + * Mget is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Mget is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Mget. If not, see . + * + * + * Public Suffix List routines (right now experimental) + * + * Changelog + * 19.03.2014 Tim Ruehsen created from libmget/cookie.c + * + */ + +#if HAVE_CONFIG_H +# include +#endif + +#include +#include +#include + +#include + +#define countof(a) (sizeof(a)/sizeof(*(a))) + +static int + ok, + failed; + +static void test_psl(void) +{ + static const struct test_data { + const char + *domain; + int + result; + } test_data[] = { + { "www.example.com", 0 }, + { "com.ar", 1 }, + { "www.com.ar", 0 }, + { "cc.ar.us", 1 }, + { ".cc.ar.us", 1 }, + { "www.cc.ar.us", 0 }, + { "www.ck", 0 }, // exception from *.ck + { "abc.www.ck", 0 }, + { "xxx.ck", 1 }, + { "www.xxx.ck", 0 }, + }; + unsigned it; + psl_ctx_t *psl; + + psl = psl_load_file(DATADIR "/effective_tld_names.dat"); + + for (it = 0; it < countof(test_data); it++) { + const struct test_data *t = &test_data[it]; + int result = psl_is_tld(psl, t->domain); + + if (result == t->result) { + ok++; + } else { + failed++; + printf("psl_is_tld(%s)=%d (expected %d)\n", t->domain, result, t->result); + } + } + + psl_free(&psl); +} + +int main(int argc, const char * const *argv) +{ + // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + if (argc == 1) { + const char *valgrind = getenv("TESTS_VALGRIND"); + + if (valgrind && *valgrind) { + char cmd[strlen(valgrind)+strlen(argv[0])+32]; + + snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + return system(cmd) != 0; + } + } + + test_psl(); + + if (failed) { + printf("Summary: %d out of %d tests failed\n", failed, ok + failed); + return 1; + } + + printf("Summary: All %d tests passed\n", ok + failed); + return 0; +} From 2327025f3af6fcb19616bb414800eeaf2ffab067 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 21 Mar 2014 11:04:01 +0100 Subject: [PATCH 004/104] fix 'make distcheck' --- Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.am b/Makefile.am index 2b7d6a4..be23856 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,6 +1,6 @@ # got some hints from https://gitorious.org/openismus-playground/examplelib/source -SUBDIRS = po include src tests +SUBDIRS = po include src tests data ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS} # Enable GTK-Doc during make distcheck From f22a3fcb710f0281767ec83d38a8464cb2f03aa0 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 21 Mar 2014 11:05:09 +0100 Subject: [PATCH 005/104] do not redefine _GNU_SOURCE --- src/psl.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/psl.c b/src/psl.c index 85e9859..15fbb6d 100644 --- a/src/psl.c +++ b/src/psl.c @@ -24,7 +24,10 @@ * */ -#define _GNU_SOURCE +// need _GNU_SOURCE for qsort_r() +#ifndef _GNU_SOURCE +# define _GNU_SOURCE +#endif #if HAVE_CONFIG_H # include From cf46f5d84b32df2586f937f34c1718e2a352eb79 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 21 Mar 2014 15:41:27 +0100 Subject: [PATCH 006/104] replaced getline() by fgets() for compatibility reasons --- src/psl.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/src/psl.c b/src/psl.c index 15fbb6d..c011137 100644 --- a/src/psl.c +++ b/src/psl.c @@ -270,9 +270,7 @@ psl_ctx_t *psl_load_file(const char *fname) _psl_entry_t suffix, *suffixp; FILE *fp; int nsuffixes = 0; - char *buf = NULL, *linep, *p; - size_t bufsize = 0; - ssize_t buflen; + char buf[256], *linep, *p; if (!(psl = calloc(1, sizeof(psl_ctx_t)))) return NULL; @@ -283,9 +281,7 @@ psl_ctx_t *psl_load_file(const char *fname) psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); if ((fp = fopen(fname, "r"))) { - while ((buflen = getline(&buf, &bufsize, fp)) >= 0) { - linep = buf; - + while ((linep = fgets(&buf, sizeof(buf), fp))) { while (isspace(*linep)) linep++; // ignore leading whitespace if (!*linep) continue; // skip empty lines @@ -311,7 +307,6 @@ psl_ctx_t *psl_load_file(const char *fname) nsuffixes++;; } - free(buf); fclose(fp); _vector_sort(psl->suffix_exceptions); From e3c28f8a86de471cf3e9b0822e28735d42a525d7 Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Fri, 21 Mar 2014 14:18:36 -0400 Subject: [PATCH 007/104] fgets into buf directly, rather than the stack --- src/psl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/psl.c b/src/psl.c index c011137..5a2f819 100644 --- a/src/psl.c +++ b/src/psl.c @@ -281,7 +281,7 @@ psl_ctx_t *psl_load_file(const char *fname) psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); if ((fp = fopen(fname, "r"))) { - while ((linep = fgets(&buf, sizeof(buf), fp))) { + while ((linep = fgets(buf, sizeof(buf), fp))) { while (isspace(*linep)) linep++; // ignore leading whitespace if (!*linep) continue; // skip empty lines From cdeea860f722fff43d1e62232d21684ab58379c6 Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Fri, 21 Mar 2014 14:19:25 -0400 Subject: [PATCH 008/104] git should ignore ephemeral files. --- .gitignore | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 .gitignore diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..de162c9 --- /dev/null +++ b/.gitignore @@ -0,0 +1,57 @@ +*~ +Makefile +Makefile.in +aclocal.m4 +autom4te.cache/ +compile +config.guess +config.h +config.h.in +config.log +config.rpath +config.status +config.sub +configure +data/Makefile +data/Makefile.in +depcomp +include/Makefile +include/Makefile.in +install-sh +libpsl-*.pc +libtool +ltmain.sh +m4/ +missing +po/Makefile +po/Makefile.in +po/Makefile.in.in +po/Makevars.template +po/POTFILES +po/Rules-quot +po/boldquot.sed +po/en@boldquot.header +po/en@quot.header +po/insert-header.sin +po/psl.pot +po/quot.sed +po/remove-potcdate.sin +po/remove-potcdate.sed +po/stamp-po +src/.deps/ +src/.libs/ +src/Makefile +src/Makefile.in +src/libpsl-*.la +src/libpsl_*_la-psl.lo +stamp-h1 +test-driver +tests/.deps/ +tests/Makefile +tests/Makefile.in +tests/test-is-tld +tests/test-is-tld.log +tests/test-is-tld.o +tests/test-is-tld.trs +tests/test-suite.log +psl-*.tar.gz From 2d99b964ff6eb373ddcbcaa7047b5ae6fe7bc6fe Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Fri, 21 Mar 2014 14:26:55 -0400 Subject: [PATCH 009/104] avoid a printf in the library In general, we don't want libraries to send data to the standard file descriptors. There are more that need fixing. Note: this introduces a new API (psl_suffix_count() and psl_suffix_exception_count) to enable the same sort of output from the test. But this new API seems to imply the internal structure of the public suffix list. Do we want to expose this API? There could be some other PSL mechanism (e.g. DBOUND) that doesn't have these counts, and a drop-in replacement would not know what to return here. --- include/libpsl.h | 8 ++++++++ src/psl.c | 15 +++++++++++++-- tests/test-is-tld.c | 3 +++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index fd2a404..9a7c934 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -55,6 +55,14 @@ psl_ctx_t * int psl_is_tld(const psl_ctx_t *psl, const char *domain); +/* does not include exceptions */ +int + psl_suffix_count(const psl_ctx_t *psl); +/* just counts exceptions */ +int + psl_suffix_exception_count(const psl_ctx_t *psl); + + PSL_END_DECLS #endif /* _LIBPSL_LIBPSL_H */ diff --git a/src/psl.c b/src/psl.c index 5a2f819..cd3e5fa 100644 --- a/src/psl.c +++ b/src/psl.c @@ -312,14 +312,25 @@ psl_ctx_t *psl_load_file(const char *fname) _vector_sort(psl->suffix_exceptions); _vector_sort(psl->suffixes); - printf("loaded %d (%d/%d) suffixes\n", nsuffixes, psl->suffixes->cur, psl->suffix_exceptions->cur); - } else fprintf(stderr, _("Failed to open PSL file '%s'\n"), fname); return psl; } + +/* does not include exceptions */ +int psl_suffix_count(const psl_ctx_t *psl) +{ + return psl->suffixes->cur; +} +/* just counts exceptions */ +int psl_suffix_exception_count(const psl_ctx_t *psl) +{ + return psl->suffix_exceptions->cur; +} + + void psl_free(psl_ctx_t **psl) { if (psl && *psl) { diff --git a/tests/test-is-tld.c b/tests/test-is-tld.c index 0f63b9b..1868af0 100644 --- a/tests/test-is-tld.c +++ b/tests/test-is-tld.c @@ -64,6 +64,9 @@ static void test_psl(void) psl = psl_load_file(DATADIR "/effective_tld_names.dat"); + printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + + for (it = 0; it < countof(test_data); it++) { const struct test_data *t = &test_data[it]; int result = psl_is_tld(psl, t->domain); From 4e674ccbae09672bda9adec75debc9d434226340 Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Fri, 21 Mar 2014 14:39:17 -0400 Subject: [PATCH 010/104] document indentation conventions for emacs users --- .dir-locals.el | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 .dir-locals.el diff --git a/.dir-locals.el b/.dir-locals.el new file mode 100644 index 0000000..9b28057 --- /dev/null +++ b/.dir-locals.el @@ -0,0 +1,9 @@ +;; emacs local configuration settings for libpsl source +;; surmised by dkg on 2014-03-21 14:35:49-0400 + +((c-mode + (indent-tabs-mode . t) + (tab-width . 4) + (c-basic-offset . 4) + (c-file-style . "linux")) + ) From c07ea9d0a05ec09568ffa2b804b90b1551669517 Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Fri, 21 Mar 2014 14:43:27 -0400 Subject: [PATCH 011/104] return NULL from psl_load_file() if the file could not be read. --- src/psl.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/src/psl.c b/src/psl.c index cd3e5fa..88ccc9e 100644 --- a/src/psl.c +++ b/src/psl.c @@ -275,12 +275,12 @@ psl_ctx_t *psl_load_file(const char *fname) if (!(psl = calloc(1, sizeof(psl_ctx_t)))) return NULL; - // as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. - // as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. - psl->suffixes = _vector_alloc(8*1024, _suffix_compare); - psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); - if ((fp = fopen(fname, "r"))) { + // as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. + // as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. + psl->suffixes = _vector_alloc(8*1024, _suffix_compare); + psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); + while ((linep = fgets(buf, sizeof(buf), fp))) { while (isspace(*linep)) linep++; // ignore leading whitespace if (!*linep) continue; // skip empty lines @@ -312,8 +312,10 @@ psl_ctx_t *psl_load_file(const char *fname) _vector_sort(psl->suffix_exceptions); _vector_sort(psl->suffixes); - } else - fprintf(stderr, _("Failed to open PSL file '%s'\n"), fname); + } else { + free(psl); + return NULL; + } return psl; } From 59835ed6964f058ae08c1bbe36be0e1544361eff Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 21 Mar 2014 21:25:44 +0100 Subject: [PATCH 012/104] added Travis-CI .travis.yml file --- .travis.yml | 9 +++++++++ README.md | 2 ++ 2 files changed, 11 insertions(+) create mode 100644 .travis.yml diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..1310697 --- /dev/null +++ b/.travis.yml @@ -0,0 +1,9 @@ +language: c +compiler: + - gcc + - clang +# Change this to your needs +script: ./autogen.sh && ./configure && make && make check && make distcheck +before_install: + - sudo apt-get -qq update + - sudo apt-get -q install autoconf automake autopoint libtool gettext diff --git a/README.md b/README.md index 3e701b3..0f6ef75 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,5 @@ +[![Build Status](https://travis-ci.org/rockdaboot/libpsl.png?branch=master)](https://travis-ci.org/rockdaboot/libpsl) + libpsl - C library to handle the Public Suffix List =================================================== From 0bead9b000d0aa51c5b1867775643a93a08c81da Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 08:28:52 +0100 Subject: [PATCH 013/104] added Mailing List info --- README.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/README.md b/README.md index 0f6ef75..a5c314d 100644 --- a/README.md +++ b/README.md @@ -16,3 +16,16 @@ Download project and prepare sources with ./configure make make check + +Mailing List +------------ + +[Mailing List](https://groups.google.com/forum/#!forum/libpsl-bugs) + +To join the mailing list send an email to + + + +and follow the instructions provided by the answer mail. + +Or click [join](https://groups.google.com/forum/#!forum/libpsl-bugs/join). From 577fd14bb4392505ac85443a115ebd58cc5734fb Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 10:26:59 +0100 Subject: [PATCH 014/104] renamed psl_is_tld() to psl_is_public() --- include/libpsl.h | 2 +- src/psl.c | 2 +- tests/Makefile.am | 4 ++-- tests/{test-is-tld.c => test-is-public.c} | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) rename tests/{test-is-tld.c => test-is-public.c} (98%) diff --git a/include/libpsl.h b/include/libpsl.h index 9a7c934..349e88c 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -53,7 +53,7 @@ void psl_ctx_t * psl_load_file(const char *fname); int - psl_is_tld(const psl_ctx_t *psl, const char *domain); + psl_is_public(const psl_ctx_t *psl, const char *domain); /* does not include exceptions */ int diff --git a/src/psl.c b/src/psl.c index 88ccc9e..2b31f81 100644 --- a/src/psl.c +++ b/src/psl.c @@ -210,7 +210,7 @@ static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) *dst = 0; } -int psl_is_tld(const psl_ctx_t *psl, const char *domain) +int psl_is_public(const psl_ctx_t *psl, const char *domain) { _psl_entry_t suffix, *rule; const char *p, *label_bak; diff --git a/tests/Makefile.am b/tests/Makefile.am index 8f5c810..f6d6f11 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -4,11 +4,11 @@ AM_CPPFLAGS = -Wno-missing-field-initializers -I$(top_srcdir)/include AM_LDFLAGS = -static LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la -PSL_TESTS = test-is-tld +PSL_TESTS = test-is-public check_PROGRAMS = $(PSL_TESTS) -test_is_tld_SOURCES = test-is-tld.c +test_is_tld_SOURCES = test-is-public.c test_is_tld_LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la TESTS_ENVIRONMENT = TESTS_VALGRIND="@VALGRIND_ENVIRONMENT@" diff --git a/tests/test-is-tld.c b/tests/test-is-public.c similarity index 98% rename from tests/test-is-tld.c rename to tests/test-is-public.c index 1868af0..1bfdc6b 100644 --- a/tests/test-is-tld.c +++ b/tests/test-is-public.c @@ -69,7 +69,7 @@ static void test_psl(void) for (it = 0; it < countof(test_data); it++) { const struct test_data *t = &test_data[it]; - int result = psl_is_tld(psl, t->domain); + int result = psl_is_public(psl, t->domain); if (result == t->result) { ok++; From de7d394223bccf2a53e4dd4d8e432cea92bad374 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 14:28:19 +0100 Subject: [PATCH 015/104] whitespace glitch --- tests/test-is-public.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 1bfdc6b..09fb057 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -64,8 +64,7 @@ static void test_psl(void) psl = psl_load_file(DATADIR "/effective_tld_names.dat"); - printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); - + printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); for (it = 0; it < countof(test_data); it++) { const struct test_data *t = &test_data[it]; From a707b267c94245996d17e853d95be1d7d15154ae Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 14:28:55 +0100 Subject: [PATCH 016/104] new function psl_load_fp() --- include/libpsl.h | 4 +++ src/psl.c | 75 ++++++++++++++++++++++++++---------------------- 2 files changed, 45 insertions(+), 34 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index 349e88c..aff9292 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -27,6 +27,8 @@ #ifndef _LIBPSL_LIBPSL_H #define _LIBPSL_LIBPSL_H +#include + // Let C++ include C headers #ifdef __cplusplus # define PSL_BEGIN_DECLS extern "C" { @@ -52,6 +54,8 @@ void psl_free(psl_ctx_t **psl); psl_ctx_t * psl_load_file(const char *fname); +psl_ctx_t * + psl_load_fp(FILE *fp); int psl_is_public(const psl_ctx_t *psl, const char *domain); diff --git a/src/psl.c b/src/psl.c index 2b31f81..5fa03a9 100644 --- a/src/psl.c +++ b/src/psl.c @@ -265,58 +265,65 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) } psl_ctx_t *psl_load_file(const char *fname) +{ + FILE *fp; + psl_ctx_t *psl = NULL; + + if ((fp = fopen(fname, "r"))) { + psl = psl_load_fp(fp); + fclose(fp); + } + + return psl; +} + +psl_ctx_t *psl_load_fp(FILE *fp) { psl_ctx_t *psl; _psl_entry_t suffix, *suffixp; - FILE *fp; int nsuffixes = 0; char buf[256], *linep, *p; + if (!fp) + return NULL; + if (!(psl = calloc(1, sizeof(psl_ctx_t)))) return NULL; - if ((fp = fopen(fname, "r"))) { - // as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. - // as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. - psl->suffixes = _vector_alloc(8*1024, _suffix_compare); - psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); + // as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. + // as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. + psl->suffixes = _vector_alloc(8*1024, _suffix_compare); + psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); - while ((linep = fgets(buf, sizeof(buf), fp))) { - while (isspace(*linep)) linep++; // ignore leading whitespace - if (!*linep) continue; // skip empty lines + while ((linep = fgets(buf, sizeof(buf), fp))) { + while (isspace(*linep)) linep++; // ignore leading whitespace + if (!*linep) continue; // skip empty lines - if (*linep == '/' && linep[1] == '/') - continue; // skip comments + if (*linep == '/' && linep[1] == '/') + continue; // skip comments - // parse suffix rule - for (p = linep; *linep && !isspace(*linep);) linep++; - *linep = 0; + // parse suffix rule + for (p = linep; *linep && !isspace(*linep);) linep++; + *linep = 0; - if (*p == '!') { - // add to exceptions - _suffix_init(&suffix, p + 1, linep - p - 1); - suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); - } else { - _suffix_init(&suffix, p, linep - p); - suffixp = _vector_get(psl->suffixes, _vector_add(psl->suffixes, &suffix)); - } - - if (suffixp) - suffixp->label = suffixp->label_buf; // set label to changed address - - nsuffixes++;; + if (*p == '!') { + // add to exceptions + _suffix_init(&suffix, p + 1, linep - p - 1); + suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); + } else { + _suffix_init(&suffix, p, linep - p); + suffixp = _vector_get(psl->suffixes, _vector_add(psl->suffixes, &suffix)); } - fclose(fp); + if (suffixp) + suffixp->label = suffixp->label_buf; // set label to changed address - _vector_sort(psl->suffix_exceptions); - _vector_sort(psl->suffixes); - - } else { - free(psl); - return NULL; + nsuffixes++;; } + _vector_sort(psl->suffix_exceptions); + _vector_sort(psl->suffixes); + return psl; } From 3998137fd03fa6c34fedfd420515e8ea99edbfd7 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 20:35:56 +0100 Subject: [PATCH 017/104] added PSL inline library --- include/Makefile.am | 2 +- include/libpsl-inline.h | 67 +++++++++++++++ src/Makefile.am | 34 ++++++-- src/psl-inline.c | 153 ++++++++++++++++++++++++++++++++++ src/psl.c | 16 ++-- src/psl2c.c | 63 ++++++++++++++ tests/Makefile.am | 6 +- tests/test-is-public-inline.c | 106 +++++++++++++++++++++++ 8 files changed, 428 insertions(+), 19 deletions(-) create mode 100644 include/libpsl-inline.h create mode 100644 src/psl-inline.c create mode 100644 src/psl2c.c create mode 100644 tests/test-is-public-inline.c diff --git a/include/Makefile.am b/include/Makefile.am index a45de59..dd315d8 100644 --- a/include/Makefile.am +++ b/include/Makefile.am @@ -1 +1 @@ -include_HEADERS = libpsl.h +include_HEADERS = libpsl.h libpsl-inline.h diff --git a/include/libpsl-inline.h b/include/libpsl-inline.h new file mode 100644 index 0000000..d9997a6 --- /dev/null +++ b/include/libpsl-inline.h @@ -0,0 +1,67 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * This file is part of libpsl. + * + * Libpsl is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Libpsl is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with libpsl. If not, see . + * + * + * Header file for libpsl library routines + * + * Changelog + * 22.03.2014 Tim Ruehsen created + * + */ + +#ifndef _LIBPSL_LIBPSL_INLINE_H +#define _LIBPSL_LIBPSL_INLINE_H + +#include + +// Let C++ include C headers +#ifdef __cplusplus +# define PSL_BEGIN_DECLS extern "C" { +# define PSL_END_DECLS } +#else +# define PSL_BEGIN_DECLS +# define PSL_END_DECLS +#endif + +#if ENABLE_NLS != 0 +# include +# define _(STRING) gettext(STRING) +#else +# define _(STRING) STRING +# define ngettext(STRING1,STRING2,N) STRING2 +#endif + +PSL_BEGIN_DECLS + +void + psl_inline_init(void); +void + psl_inline_deinit(void); +int + psl_inline_is_public(const char *domain); + +/* does not include exceptions */ +int + psl_inline_suffix_count(void); +/* just counts exceptions */ +int + psl_inline_suffix_exception_count(void); + +PSL_END_DECLS + +#endif /* _LIBPSL_LIBPSL_INLINE_H */ diff --git a/src/Makefile.am b/src/Makefile.am index f8339d5..f86bb4b 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1,14 +1,30 @@ -lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la +#EXTRA_DIST = $(top_srcdir)/data/effective_tld_names.dat + +# suffixes.c must be created before psl.c is compiled +BUILT_SOURCES = suffixes.c + +# suffixes.c is a built source that must be cleaned +CLEANFILES = suffixes.c + +# build two libraries, 'inline' version with PSL entries compiled in +lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la libpsl-inline-@LIBPSL_API_VERSION@.la + libpsl_@LIBPSL_API_VERSION@_la_SOURCES = psl.c - libpsl_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include - # include ABI version information libpsl_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) -#bin_PROGRAMS = test_linking -#noinst_PROGRAMS = test_linking -#test_linking_SOURCES = test_linking.c -#test_linking_CPPFLAGS = -I$(top_srcdir)/include -#test_linking_LDADD = libpsl-@LIBPSL_API_VERSION@.la -#test_linking_LDFLAGS = -static +libpsl_inline_@LIBPSL_API_VERSION@_la_SOURCES = psl-inline.c +libpsl_inline_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include +# include ABI version information +libpsl_inline_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) + +noinst_PROGRAMS = psl2c +psl2c_SOURCES = psl2c.c +psl2c_CPPFLAGS = -I$(top_srcdir)/include +#psl2c_LDADD = libpsl-@LIBPSL_API_VERSION@.la +#psl2c_LDFLAGS = -static + +# Build rule for suffix.c +suffixes.c: $(top_srcdir)/data/effective_tld_names.dat psl2c$(EXEEXT) + ./psl2c$(EXEEXT) <$(top_srcdir)/data/effective_tld_names.dat >suffixes.c diff --git a/src/psl-inline.c b/src/psl-inline.c new file mode 100644 index 0000000..b441182 --- /dev/null +++ b/src/psl-inline.c @@ -0,0 +1,153 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * This file is part of MGet. + * + * Mget is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Mget is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Mget. If not, see . + * + * + * Public Suffix List routines (right now experimental) + * + * Changelog + * 22.03.2014 Tim Ruehsen created + * + */ + +// need _GNU_SOURCE for qsort_r() +#ifndef _GNU_SOURCE +# define _GNU_SOURCE +#endif + +#if HAVE_CONFIG_H +# include +#endif + +#include +#include +#include +#include + +#include + +#define countof(a) (sizeof(a)/sizeof(*(a))) + +typedef struct { + char + label_buf[42]; + const char * + label; + unsigned short + length; + unsigned char + nlabels, // number of labels + wildcard; // this is a wildcard rule (e.g. *.sapporo.jp) +} _psl_entry_t; + +#include "suffixes.c" + +// by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) + +static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) +{ + int n; + + if ((n = s2->nlabels - s1->nlabels)) + return n; // most labels first + + if ((n = s1->length - s2->length)) + return n; // shorter rules first + + return strcmp(s1->label, s2->label); +} + +void psl_inline_init(void) +{ + size_t it; + + for (it = 0; it < countof(suffixes); it++) + suffixes[it].label = suffixes[it].label_buf; + + for (it = 0; it < countof(suffix_exceptions); it++) + suffix_exceptions[it].label = suffix_exceptions[it].label_buf; +} + +void psl_inline_deinit(void) +{ +} + +int psl_inline_is_public(const char *domain) +{ + _psl_entry_t suffix, *rule; + const char *p, *label_bak; + unsigned short length_bak; + + // this function should be called without leading dots, just make sure + suffix.label = domain + (*domain == '.'); + suffix.length = strlen(suffix.label); + suffix.wildcard = 0; + suffix.nlabels = 1; + + for (p = suffix.label; *p; p++) + if (*p == '.') + suffix.nlabels++; + + // if domain has enough labels, it won't match + rule = &suffixes[0]; + if (!rule || rule->nlabels < suffix.nlabels - 1) + return 0; + + rule = bsearch(&suffix, suffixes, countof(suffixes), sizeof(suffixes[0]), (int(*)(const void *, const void *))_suffix_compare); + if (rule) { + // definitely a match, no matter if the found rule is a wildcard or not + return 1; + } + + label_bak = suffix.label; + length_bak = suffix.length; + + if ((suffix.label = strchr(suffix.label, '.'))) { + suffix.label++; + suffix.length = strlen(suffix.label); + suffix.nlabels--; + + rule = bsearch(&suffix, suffixes, countof(suffixes), sizeof(suffixes[0]), (int(*)(const void *, const void *))_suffix_compare); + if (rule) { + if (rule->wildcard) { + // now that we matched a wildcard, we have to check for an exception + suffix.label = label_bak; + suffix.length = length_bak; + suffix.nlabels++; + + if (bsearch(&suffix, suffix_exceptions, countof(suffix_exceptions), sizeof(suffix_exceptions[0]), (int(*)(const void *, const void *))_suffix_compare)) + return 0; // found an exception, so 'domain' is not a public suffix + + return 1; + } + } + } + + return 0; +} + +/* does not include exceptions */ +int psl_inline_suffix_count(void) +{ + return countof(suffixes); +} + +/* just counts exceptions */ +int psl_inline_suffix_exception_count(void) +{ + return countof(suffix_exceptions); +} diff --git a/src/psl.c b/src/psl.c index 5fa03a9..aba1594 100644 --- a/src/psl.c +++ b/src/psl.c @@ -158,6 +158,11 @@ static void _vector_sort(_psl_vector_t *v) qsort_r(v->entry, v->cur, sizeof(_psl_vector_t *), _compare, v); } +static inline int _vector_size(_psl_vector_t *v) +{ + return v ? v->cur : 0; +} + // by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) @@ -167,7 +172,7 @@ static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) if ((n = s2->nlabels - s1->nlabels)) return n; // most labels first - if ((n=s1->length - s2->length)) + if ((n = s1->length - s2->length)) return n; // shorter rules first return strcmp(s1->label, s2->label); @@ -216,7 +221,7 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) const char *p, *label_bak; unsigned short length_bak; - // this function should be called without leading dots, just make shure + // this function should be called without leading dots, just make sure suffix.label = domain + (*domain == '.'); suffix.length = strlen(suffix.label); suffix.wildcard = 0; @@ -327,19 +332,18 @@ psl_ctx_t *psl_load_fp(FILE *fp) return psl; } - /* does not include exceptions */ int psl_suffix_count(const psl_ctx_t *psl) { - return psl->suffixes->cur; + return _vector_size(psl->suffixes); } + /* just counts exceptions */ int psl_suffix_exception_count(const psl_ctx_t *psl) { - return psl->suffix_exceptions->cur; + return _vector_size(psl->suffix_exceptions); } - void psl_free(psl_ctx_t **psl) { if (psl && *psl) { diff --git a/src/psl2c.c b/src/psl2c.c new file mode 100644 index 0000000..ffe1487 --- /dev/null +++ b/src/psl2c.c @@ -0,0 +1,63 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * This file is part of libpsl. + * + * Libpsl is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Libpsl is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with libpsl. If not, see . + * + * + * Precompile Public Suffix List into + * + * Changelog + * 22.03.2014 Tim Ruehsen created + * + */ + +#if HAVE_CONFIG_H +# include +#endif + +#include "psl.c" + +static void _print_psl_entries(_psl_vector_t *v, const char *varname) +{ + int it; + + printf("// automatically generated by psl2c\n"); + printf("static _psl_entry_t %s[] = {\n", varname); + + for (it = 0; it < v->cur; it++) { + _psl_entry_t *e = _vector_get(v, it); + + printf("\t{ \"%s\", NULL, %hd, %hhd, %hhd },\n", + e->label_buf, e->length, e->nlabels, e->wildcard); + } + + printf("};\n"); +} + +// int main(int argc, const char **argv) +int main(void) +{ + psl_ctx_t *psl; + + if (!(psl = psl_load_fp(stdin))) + return 1; + + _print_psl_entries(psl->suffixes, "suffixes"); + _print_psl_entries(psl->suffix_exceptions, "suffix_exceptions"); + + psl_free(&psl); + return 0; +} diff --git a/tests/Makefile.am b/tests/Makefile.am index f6d6f11..6595619 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -4,12 +4,12 @@ AM_CPPFLAGS = -Wno-missing-field-initializers -I$(top_srcdir)/include AM_LDFLAGS = -static LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la -PSL_TESTS = test-is-public +PSL_TESTS = test-is-public test-is-public-inline check_PROGRAMS = $(PSL_TESTS) -test_is_tld_SOURCES = test-is-public.c -test_is_tld_LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la +#test_is_public_inline_SOURCES = test-is-public-inline.c +test_is_public_inline_LDADD = ../src/libpsl-inline-@LIBPSL_API_VERSION@.la TESTS_ENVIRONMENT = TESTS_VALGRIND="@VALGRIND_ENVIRONMENT@" TESTS = $(PSL_TESTS) diff --git a/tests/test-is-public-inline.c b/tests/test-is-public-inline.c new file mode 100644 index 0000000..62d0c42 --- /dev/null +++ b/tests/test-is-public-inline.c @@ -0,0 +1,106 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * This file is part of MGet. + * + * Mget is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Mget is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Mget. If not, see . + * + * + * Public Suffix List routines (right now experimental) + * + * Changelog + * 19.03.2014 Tim Ruehsen created from libmget/cookie.c + * + */ + +#if HAVE_CONFIG_H +# include +#endif + +#include +#include +#include + +#include + +#define countof(a) (sizeof(a)/sizeof(*(a))) + +static int + ok, + failed; + +static void test_psl(void) +{ + static const struct test_data { + const char + *domain; + int + result; + } test_data[] = { + { "www.example.com", 0 }, + { "com.ar", 1 }, + { "www.com.ar", 0 }, + { "cc.ar.us", 1 }, + { ".cc.ar.us", 1 }, + { "www.cc.ar.us", 0 }, + { "www.ck", 0 }, // exception from *.ck + { "abc.www.ck", 0 }, + { "xxx.ck", 1 }, + { "www.xxx.ck", 0 }, + }; + unsigned it; + + psl_inline_init(); + + printf("have %d suffixes and %d exceptions\n", psl_inline_suffix_count(), psl_inline_suffix_exception_count()); + + for (it = 0; it < countof(test_data); it++) { + const struct test_data *t = &test_data[it]; + int result = psl_inline_is_public(t->domain); + + if (result == t->result) { + ok++; + } else { + failed++; + printf("psl_is_tld(%s)=%d (expected %d)\n", t->domain, result, t->result); + } + } + + psl_inline_deinit(); +} + +int main(int argc, const char * const *argv) +{ + // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + if (argc == 1) { + const char *valgrind = getenv("TESTS_VALGRIND"); + + if (valgrind && *valgrind) { + char cmd[strlen(valgrind)+strlen(argv[0])+32]; + + snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + return system(cmd) != 0; + } + } + + test_psl(); + + if (failed) { + printf("Summary: %d out of %d tests failed\n", failed, ok + failed); + return 1; + } + + printf("Summary: All %d tests passed\n", ok + failed); + return 0; +} From 1c90fac381d6703ac3f0d28caace88d9d92080d6 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 21:36:02 +0100 Subject: [PATCH 018/104] added test with utf-8 domain --- tests/test-is-public-inline.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/test-is-public-inline.c b/tests/test-is-public-inline.c index 62d0c42..e3430da 100644 --- a/tests/test-is-public-inline.c +++ b/tests/test-is-public-inline.c @@ -42,6 +42,8 @@ static int static void test_psl(void) { + // punycode generation: idn 商标 + // octal code generation: echo -n "商标" | od -b static const struct test_data { const char *domain; @@ -58,6 +60,10 @@ static void test_psl(void) { "abc.www.ck", 0 }, { "xxx.ck", 1 }, { "www.xxx.ck", 0 }, + { "\345\225\206\346\240\207", 1 }, // xn--czr694b oder 商标 + { "www.\345\225\206\346\240\207", 0 }, +// { "xn--czr694b", 1 }, +// { "www.xn--czr694b", 1 }, }; unsigned it; From 99d057d51404aa6c18bdf12b69d3b60ffcce5132 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 22:19:20 +0100 Subject: [PATCH 019/104] revert logic for psl_is_public() --- src/psl.c | 12 ++++++------ src/psl2c.c | 12 ++++++++++++ tests/test-is-public-inline.c | 24 ++++++++++++------------ tests/test-is-public.c | 20 ++++++++++---------- 4 files changed, 40 insertions(+), 28 deletions(-) diff --git a/src/psl.c b/src/psl.c index aba1594..fed72b7 100644 --- a/src/psl.c +++ b/src/psl.c @@ -231,15 +231,15 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) if (*p == '.') suffix.nlabels++; - // if domain has enough labels, it won't match + // if domain has enough labels, it is public rule = _vector_get(psl->suffixes, 0); if (!rule || rule->nlabels < suffix.nlabels - 1) - return 0; + return 1; rule = _vector_get(psl->suffixes, _vector_find(psl->suffixes, &suffix)); if (rule) { // definitely a match, no matter if the found rule is a wildcard or not - return 1; + return 0; } label_bak = suffix.label; @@ -259,14 +259,14 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) suffix.nlabels++; if (_vector_get(psl->suffix_exceptions, _vector_find(psl->suffix_exceptions, &suffix)) != 0) - return 0; + return 1; // found an exception, so 'domain' is public - return 1; + return 0; } } } - return 0; + return 1; } psl_ctx_t *psl_load_file(const char *fname) diff --git a/src/psl2c.c b/src/psl2c.c index ffe1487..511e19b 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -28,6 +28,7 @@ # include #endif +// # include #include "psl.c" static void _print_psl_entries(_psl_vector_t *v, const char *varname) @@ -42,6 +43,17 @@ static void _print_psl_entries(_psl_vector_t *v, const char *varname) printf("\t{ \"%s\", NULL, %hd, %hhd, %hhd },\n", e->label_buf, e->length, e->nlabels, e->wildcard); +/* + if (str_needs_encoding(e->label_buf)) { + char *asc = NULL; + int rc; + + if ((rc = idn2_lookup_u8((uint8_t *)e->label_buf, (uint8_t **)&asc, 0)) == IDN2_OK) { + fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, asc); + } else + fprintf(stderr, "toASCII(%s) failed (%d): %s\n", e->label_buf, rc, idn2_strerror(rc)); + } +*/ } printf("};\n"); diff --git a/tests/test-is-public-inline.c b/tests/test-is-public-inline.c index e3430da..1777317 100644 --- a/tests/test-is-public-inline.c +++ b/tests/test-is-public-inline.c @@ -50,18 +50,18 @@ static void test_psl(void) int result; } test_data[] = { - { "www.example.com", 0 }, - { "com.ar", 1 }, - { "www.com.ar", 0 }, - { "cc.ar.us", 1 }, - { ".cc.ar.us", 1 }, - { "www.cc.ar.us", 0 }, - { "www.ck", 0 }, // exception from *.ck - { "abc.www.ck", 0 }, - { "xxx.ck", 1 }, - { "www.xxx.ck", 0 }, - { "\345\225\206\346\240\207", 1 }, // xn--czr694b oder 商标 - { "www.\345\225\206\346\240\207", 0 }, + { "www.example.com", 1 }, + { "com.ar", 0 }, + { "www.com.ar", 1 }, + { "cc.ar.us", 0 }, + { ".cc.ar.us", 0 }, + { "www.cc.ar.us", 1 }, + { "www.ck", 1 }, // exception from *.ck + { "abc.www.ck", 1 }, + { "xxx.ck", 0 }, + { "www.xxx.ck", 1 }, + { "\345\225\206\346\240\207", 0 }, // xn--czr694b oder 商标 + { "www.\345\225\206\346\240\207", 1 }, // { "xn--czr694b", 1 }, // { "www.xn--czr694b", 1 }, }; diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 09fb057..00db2d0 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -48,16 +48,16 @@ static void test_psl(void) int result; } test_data[] = { - { "www.example.com", 0 }, - { "com.ar", 1 }, - { "www.com.ar", 0 }, - { "cc.ar.us", 1 }, - { ".cc.ar.us", 1 }, - { "www.cc.ar.us", 0 }, - { "www.ck", 0 }, // exception from *.ck - { "abc.www.ck", 0 }, - { "xxx.ck", 1 }, - { "www.xxx.ck", 0 }, + { "www.example.com", 1 }, + { "com.ar", 0 }, + { "www.com.ar", 1 }, + { "cc.ar.us", 0 }, + { ".cc.ar.us", 0 }, + { "www.cc.ar.us", 1 }, + { "www.ck", 1 }, // exception from *.ck + { "abc.www.ck", 1 }, + { "xxx.ck", 0 }, + { "www.xxx.ck", 1 }, }; unsigned it; psl_ctx_t *psl; From a906062b854e32764fb3efcfe42e6f672dec9058 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 22:55:34 +0100 Subject: [PATCH 020/104] added IDNA2008 punycode support for psl_inline_is_public() --- src/Makefile.am | 4 +-- src/psl-inline.c | 14 +++++----- src/psl.c | 4 +-- src/psl2c.c | 52 +++++++++++++++++++++++++++-------- tests/test-is-public-inline.c | 6 ++-- tests/test-is-public.c | 6 +++- 6 files changed, 59 insertions(+), 27 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index f86bb4b..bbd54e0 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -21,8 +21,8 @@ libpsl_inline_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSIO noinst_PROGRAMS = psl2c psl2c_SOURCES = psl2c.c -psl2c_CPPFLAGS = -I$(top_srcdir)/include -#psl2c_LDADD = libpsl-@LIBPSL_API_VERSION@.la +psl2c_CPPFLAGS = -I$(top_srcdir)/include -D _GNU_SOURCE +psl2c_LDADD = -lidn2 #psl2c_LDFLAGS = -static # Build rule for suffix.c diff --git a/src/psl-inline.c b/src/psl-inline.c index b441182..901c57d 100644 --- a/src/psl-inline.c +++ b/src/psl-inline.c @@ -44,7 +44,7 @@ typedef struct { char - label_buf[42]; + label_buf[48]; const char * label; unsigned short @@ -102,15 +102,15 @@ int psl_inline_is_public(const char *domain) if (*p == '.') suffix.nlabels++; - // if domain has enough labels, it won't match + // if domain has enough labels, it is public rule = &suffixes[0]; if (!rule || rule->nlabels < suffix.nlabels - 1) - return 0; + return 1; rule = bsearch(&suffix, suffixes, countof(suffixes), sizeof(suffixes[0]), (int(*)(const void *, const void *))_suffix_compare); if (rule) { // definitely a match, no matter if the found rule is a wildcard or not - return 1; + return 0; } label_bak = suffix.label; @@ -130,14 +130,14 @@ int psl_inline_is_public(const char *domain) suffix.nlabels++; if (bsearch(&suffix, suffix_exceptions, countof(suffix_exceptions), sizeof(suffix_exceptions[0]), (int(*)(const void *, const void *))_suffix_compare)) - return 0; // found an exception, so 'domain' is not a public suffix + return 1; // found an exception, so 'domain' is public - return 1; + return 0; } } } - return 0; + return 1; } /* does not include exceptions */ diff --git a/src/psl.c b/src/psl.c index fed72b7..3a1cd1c 100644 --- a/src/psl.c +++ b/src/psl.c @@ -44,7 +44,7 @@ typedef struct { char - label_buf[42]; + label_buf[48]; const char * label; unsigned short @@ -187,7 +187,7 @@ static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) if (length >= sizeof(suffix->label_buf) - 1) { suffix->nlabels = 0; - fprintf(stderr, _("Suffix rule too long (ignored): %s\n"), rule); + fprintf(stderr, _("Suffix rule too long (%zd, ignored): %s\n"), length, rule); return; } diff --git a/src/psl2c.c b/src/psl2c.c index 511e19b..08108f4 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -28,7 +28,10 @@ # include #endif -// # include +//#ifdef WITH_LIBIDN2 +# include +//#endif + #include "psl.c" static void _print_psl_entries(_psl_vector_t *v, const char *varname) @@ -43,22 +46,44 @@ static void _print_psl_entries(_psl_vector_t *v, const char *varname) printf("\t{ \"%s\", NULL, %hd, %hhd, %hhd },\n", e->label_buf, e->length, e->nlabels, e->wildcard); -/* - if (str_needs_encoding(e->label_buf)) { - char *asc = NULL; - int rc; - - if ((rc = idn2_lookup_u8((uint8_t *)e->label_buf, (uint8_t **)&asc, 0)) == IDN2_OK) { - fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, asc); - } else - fprintf(stderr, "toASCII(%s) failed (%d): %s\n", e->label_buf, rc, idn2_strerror(rc)); - } -*/ } printf("};\n"); } +static int _str_needs_encoding(const char *s) +{ + while (*s > 0) s++; + + return !!*s; +} + +static void _add_punycode_if_needed(_psl_vector_t *v) +{ + int it; + + for (it = 0; it < v->cur; it++) { + _psl_entry_t *e = _vector_get(v, it); + + if (_str_needs_encoding(e->label_buf)) { + _psl_entry_t suffix; + char *asc = NULL; + int rc; + + + if ((rc = idn2_lookup_u8((uint8_t *)e->label_buf, (uint8_t **)&asc, 0)) == IDN2_OK) { + fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, asc); + _suffix_init(&suffix, asc, strlen(asc)); + suffix.wildcard = e->wildcard; + _vector_add(v, &suffix); + } else + fprintf(stderr, "toASCII(%s) failed (%d): %s\n", e->label_buf, rc, idn2_strerror(rc)); + } + } + + _vector_sort(v); +} + // int main(int argc, const char **argv) int main(void) { @@ -67,6 +92,9 @@ int main(void) if (!(psl = psl_load_fp(stdin))) return 1; + _add_punycode_if_needed(psl->suffixes); + _add_punycode_if_needed(psl->suffix_exceptions); + _print_psl_entries(psl->suffixes, "suffixes"); _print_psl_entries(psl->suffix_exceptions, "suffix_exceptions"); diff --git a/tests/test-is-public-inline.c b/tests/test-is-public-inline.c index 1777317..e946617 100644 --- a/tests/test-is-public-inline.c +++ b/tests/test-is-public-inline.c @@ -62,8 +62,8 @@ static void test_psl(void) { "www.xxx.ck", 1 }, { "\345\225\206\346\240\207", 0 }, // xn--czr694b oder 商标 { "www.\345\225\206\346\240\207", 1 }, -// { "xn--czr694b", 1 }, -// { "www.xn--czr694b", 1 }, + { "xn--czr694b", 0 }, + { "www.xn--czr694b", 1 }, }; unsigned it; @@ -79,7 +79,7 @@ static void test_psl(void) ok++; } else { failed++; - printf("psl_is_tld(%s)=%d (expected %d)\n", t->domain, result, t->result); + printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); } } diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 00db2d0..d535f29 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -42,6 +42,8 @@ static int static void test_psl(void) { + // punycode generation: idn 商标 + // octal code generation: echo -n "商标" | od -b static const struct test_data { const char *domain; @@ -58,6 +60,8 @@ static void test_psl(void) { "abc.www.ck", 1 }, { "xxx.ck", 0 }, { "www.xxx.ck", 1 }, + { "\345\225\206\346\240\207", 0 }, // xn--czr694b oder 商标 + { "www.\345\225\206\346\240\207", 1 }, }; unsigned it; psl_ctx_t *psl; @@ -74,7 +78,7 @@ static void test_psl(void) ok++; } else { failed++; - printf("psl_is_tld(%s)=%d (expected %d)\n", t->domain, result, t->result); + printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); } } From 4a54c3cf2a8c0c823415ea69580a61466c3749c6 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 22:57:10 +0100 Subject: [PATCH 021/104] removed punycode debug output --- src/psl2c.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/psl2c.c b/src/psl2c.c index 08108f4..b3ad38a 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -72,7 +72,7 @@ static void _add_punycode_if_needed(_psl_vector_t *v) if ((rc = idn2_lookup_u8((uint8_t *)e->label_buf, (uint8_t **)&asc, 0)) == IDN2_OK) { - fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, asc); + // fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, asc); _suffix_init(&suffix, asc, strlen(asc)); suffix.wildcard = e->wildcard; _vector_add(v, &suffix); From 3b33ff25756066be50e29cc739c229f1a5c068a9 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 22 Mar 2014 23:05:55 +0100 Subject: [PATCH 022/104] added libidn2 to .travis.yml --- .travis.yml | 2 +- src/psl2c.c | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 1310697..447231f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,4 +6,4 @@ compiler: script: ./autogen.sh && ./configure && make && make check && make distcheck before_install: - sudo apt-get -qq update - - sudo apt-get -q install autoconf automake autopoint libtool gettext + - sudo apt-get -q install autoconf automake autopoint libtool gettext libidn2-0 libidn2-0-dev diff --git a/src/psl2c.c b/src/psl2c.c index b3ad38a..6fbb785 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -70,7 +70,6 @@ static void _add_punycode_if_needed(_psl_vector_t *v) char *asc = NULL; int rc; - if ((rc = idn2_lookup_u8((uint8_t *)e->label_buf, (uint8_t **)&asc, 0)) == IDN2_OK) { // fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, asc); _suffix_init(&suffix, asc, strlen(asc)); From 63fc945fff69fe682f3e7e0a11d4dd83259271b9 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 23 Mar 2014 19:33:36 +0100 Subject: [PATCH 023/104] fill in AUTHORS --- AUTHORS | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/AUTHORS b/AUTHORS index e69de29..ba6bf51 100644 --- a/AUTHORS +++ b/AUTHORS @@ -0,0 +1,11 @@ +Authors of and contributors to libpsl. +Thank you very much for spending your time ! + +Also many thanks for anyone who contributed ideas, +took part in discussions or 'just' asked questions. + +Please drop me a note if you feel you should have +been mentioned here. + +Tim Ruehsen (Original implementation of libpsl) +Daniel Kahn Gillmor From 07712557427a5f38b9481142f7715bfab5ed59e3 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 23 Mar 2014 21:49:19 +0100 Subject: [PATCH 024/104] added info functions --- include/libpsl-inline.h | 10 ++++++ src/Makefile.am | 2 +- src/psl-inline.c | 23 ++++++++++--- src/psl2c.c | 63 +++++++++++++++++++++++++++-------- tests/test-is-public-inline.c | 9 +++++ 5 files changed, 88 insertions(+), 19 deletions(-) diff --git a/include/libpsl-inline.h b/include/libpsl-inline.h index d9997a6..6012395 100644 --- a/include/libpsl-inline.h +++ b/include/libpsl-inline.h @@ -62,6 +62,16 @@ int int psl_inline_suffix_exception_count(void); +// returns compilation time +time_t + psl_inline_builtin_compile_time(void); +// returns mtime of PSL source file +time_t + psl_inline_builtin_file_time(void); +// returns MD5 checksum of PSL source file +const char * + psl_inline_builtin_md5sum(void); + PSL_END_DECLS #endif /* _LIBPSL_LIBPSL_INLINE_H */ diff --git a/src/Makefile.am b/src/Makefile.am index bbd54e0..db0a7db 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -27,4 +27,4 @@ psl2c_LDADD = -lidn2 # Build rule for suffix.c suffixes.c: $(top_srcdir)/data/effective_tld_names.dat psl2c$(EXEEXT) - ./psl2c$(EXEEXT) <$(top_srcdir)/data/effective_tld_names.dat >suffixes.c + ./psl2c$(EXEEXT) $(top_srcdir)/data/effective_tld_names.dat suffixes.c diff --git a/src/psl-inline.c b/src/psl-inline.c index 901c57d..3002e08 100644 --- a/src/psl-inline.c +++ b/src/psl-inline.c @@ -24,11 +24,6 @@ * */ -// need _GNU_SOURCE for qsort_r() -#ifndef _GNU_SOURCE -# define _GNU_SOURCE -#endif - #if HAVE_CONFIG_H # include #endif @@ -151,3 +146,21 @@ int psl_inline_suffix_exception_count(void) { return countof(suffix_exceptions); } + +// returns compilation time +time_t psl_inline_builtin_compile_time(void) +{ + return _psl_compile_time; +} + +// returns mtime of PSL source file +time_t psl_inline_builtin_file_time(void) +{ + return _psl_file_time; +} + +// returns MD5 checksum of PSL source file +const char *psl_inline_builtin_md5sum(void) +{ + return _psl_md5_checksum; +} diff --git a/src/psl2c.c b/src/psl2c.c index 6fbb785..8c9ce0d 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -28,27 +28,31 @@ # include #endif +#include +#include +#include + //#ifdef WITH_LIBIDN2 # include //#endif #include "psl.c" -static void _print_psl_entries(_psl_vector_t *v, const char *varname) +static void _print_psl_entries(FILE *fpout, const _psl_vector_t *v, const char *varname) { int it; - printf("// automatically generated by psl2c\n"); - printf("static _psl_entry_t %s[] = {\n", varname); + fprintf(fpout, "// automatically generated by psl2c\n"); + fprintf(fpout, "static _psl_entry_t %s[] = {\n", varname); for (it = 0; it < v->cur; it++) { _psl_entry_t *e = _vector_get(v, it); - printf("\t{ \"%s\", NULL, %hd, %hhd, %hhd },\n", + fprintf(fpout, "\t{ \"%s\", NULL, %hd, %hhd, %hhd },\n", e->label_buf, e->length, e->nlabels, e->wildcard); } - printf("};\n"); + fprintf(fpout, "};\n"); } static int _str_needs_encoding(const char *s) @@ -83,20 +87,53 @@ static void _add_punycode_if_needed(_psl_vector_t *v) _vector_sort(v); } -// int main(int argc, const char **argv) -int main(void) +int main(int argc, const char **argv) { + FILE *fpout; psl_ctx_t *psl; + int ret = 0; - if (!(psl = psl_load_fp(stdin))) + if (argc != 3) { + fprintf(stderr, "Usage: psl2c \n"); + fprintf(stderr, " is the 'effective_tld_names.dat' (aka Public Suffix List)\n"); + fprintf(stderr, " is the the C filename to be generated from \n"); return 1; + } - _add_punycode_if_needed(psl->suffixes); - _add_punycode_if_needed(psl->suffix_exceptions); + if (!(psl = psl_load_file(argv[1]))) + return 2; - _print_psl_entries(psl->suffixes, "suffixes"); - _print_psl_entries(psl->suffix_exceptions, "suffix_exceptions"); + if ((fpout = fopen(argv[2], "w"))) { + FILE *pp; + struct stat st; + char cmd[16 + strlen(argv[1])], checksum[64] = ""; + + _add_punycode_if_needed(psl->suffixes); + _add_punycode_if_needed(psl->suffix_exceptions); + + _print_psl_entries(fpout, psl->suffixes, "suffixes"); + _print_psl_entries(fpout, psl->suffix_exceptions, "suffix_exceptions"); + + snprintf(cmd, sizeof(cmd), "md5sum %s", argv[1]); + if ((pp = popen(cmd, "r"))) { + if (fscanf(pp, "%63[0-9a-zA-Z]", checksum) < 1) + *checksum = 0; + pclose(pp); + } + + if (stat(argv[1], &st) != 0) + st.st_mtime = 0; + fprintf(fpout, "static time_t _psl_file_time = %lu;\n", st.st_mtime); + fprintf(fpout, "static time_t _psl_compile_time = %lu;\n", time(NULL)); + fprintf(fpout, "static char _psl_md5_checksum[] = \"%s\";\n", checksum); + + if (fclose(fpout) != 0) + ret = 4; + } else { + fprintf(stderr, "Failed to write open '%s'\n", argv[2]); + ret = 3; + } psl_free(&psl); - return 0; + return ret; } diff --git a/tests/test-is-public-inline.c b/tests/test-is-public-inline.c index e946617..0af7915 100644 --- a/tests/test-is-public-inline.c +++ b/tests/test-is-public-inline.c @@ -83,6 +83,15 @@ static void test_psl(void) } } + printf("psl_builtin_compile_time()=%ld\n", psl_inline_builtin_compile_time()); + psl_inline_builtin_compile_time() == 0 ? failed++ : ok++; + + printf("psl_builtin_file_time()=%ld\n", psl_inline_builtin_file_time()); + psl_inline_builtin_file_time() == 0 ? failed++ : ok++; + + printf("psl_builtin_md5sum()=%s\n", psl_inline_builtin_md5sum()); + *psl_inline_builtin_md5sum() == 0 ? failed++ : ok++; + psl_inline_deinit(); } From e379ba90cf8e94ce176c8b49be5e36f9ec444795 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 24 Mar 2014 09:48:01 +0100 Subject: [PATCH 025/104] replaced md5 by sha1 checksumming --- include/libpsl-inline.h | 5 +++-- src/psl-inline.c | 6 +++--- src/psl2c.c | 4 ++-- tests/test-is-public-inline.c | 4 ++-- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/include/libpsl-inline.h b/include/libpsl-inline.h index 6012395..d4ad50f 100644 --- a/include/libpsl-inline.h +++ b/include/libpsl-inline.h @@ -68,9 +68,10 @@ time_t // returns mtime of PSL source file time_t psl_inline_builtin_file_time(void); -// returns MD5 checksum of PSL source file + +// returns MD5 checksum (hex-encoded, lowercase) of PSL source file const char * - psl_inline_builtin_md5sum(void); + psl_inline_builtin_sha1sum(void); PSL_END_DECLS diff --git a/src/psl-inline.c b/src/psl-inline.c index 3002e08..51600d9 100644 --- a/src/psl-inline.c +++ b/src/psl-inline.c @@ -159,8 +159,8 @@ time_t psl_inline_builtin_file_time(void) return _psl_file_time; } -// returns MD5 checksum of PSL source file -const char *psl_inline_builtin_md5sum(void) +// returns MD5 checksum (hex-encoded, lowercase) of PSL source file +const char *psl_inline_builtin_sha1sum(void) { - return _psl_md5_checksum; + return _psl_sha1_checksum; } diff --git a/src/psl2c.c b/src/psl2c.c index 8c9ce0d..5672fa7 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -114,7 +114,7 @@ int main(int argc, const char **argv) _print_psl_entries(fpout, psl->suffixes, "suffixes"); _print_psl_entries(fpout, psl->suffix_exceptions, "suffix_exceptions"); - snprintf(cmd, sizeof(cmd), "md5sum %s", argv[1]); + snprintf(cmd, sizeof(cmd), "sha1sum %s", argv[1]); if ((pp = popen(cmd, "r"))) { if (fscanf(pp, "%63[0-9a-zA-Z]", checksum) < 1) *checksum = 0; @@ -125,7 +125,7 @@ int main(int argc, const char **argv) st.st_mtime = 0; fprintf(fpout, "static time_t _psl_file_time = %lu;\n", st.st_mtime); fprintf(fpout, "static time_t _psl_compile_time = %lu;\n", time(NULL)); - fprintf(fpout, "static char _psl_md5_checksum[] = \"%s\";\n", checksum); + fprintf(fpout, "static char _psl_sha1_checksum[] = \"%s\";\n", checksum); if (fclose(fpout) != 0) ret = 4; diff --git a/tests/test-is-public-inline.c b/tests/test-is-public-inline.c index 0af7915..56afe41 100644 --- a/tests/test-is-public-inline.c +++ b/tests/test-is-public-inline.c @@ -89,8 +89,8 @@ static void test_psl(void) printf("psl_builtin_file_time()=%ld\n", psl_inline_builtin_file_time()); psl_inline_builtin_file_time() == 0 ? failed++ : ok++; - printf("psl_builtin_md5sum()=%s\n", psl_inline_builtin_md5sum()); - *psl_inline_builtin_md5sum() == 0 ? failed++ : ok++; + printf("psl_builtin_sha1sum()=%s\n", psl_inline_builtin_sha1sum()); + *psl_inline_builtin_sha1sum() == 0 ? failed++ : ok++; psl_inline_deinit(); } From eaf4f01b78ebff8eabca3a12219f4fe1c2b9c7ed Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 24 Mar 2014 15:53:33 +0100 Subject: [PATCH 026/104] added check for idn2 --- autogen.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/autogen.sh b/autogen.sh index 3006b3d..b279297 100755 --- a/autogen.sh +++ b/autogen.sh @@ -1,7 +1,12 @@ # !/bin/sh -e if test -z `which autoreconf`; then - echo "No autoreconf found. You must install the autoconf package." + echo "No 'autoreconf' found. You must install the autoconf package." + exit 1 +fi + +if test -z `which idn2`; then + echo "No 'idn2' found. You must install the idn2 package." exit 1 fi From 3b94a03638f13c71a756ed6fe4c98d7b0ef3ec71 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 24 Mar 2014 15:54:43 +0100 Subject: [PATCH 027/104] use idn2 instead of libidn2 to avoid the need for GPL3+ license --- src/Makefile.am | 2 +- src/psl2c.c | 19 ++++++++++++++++++- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index db0a7db..0b3b9dc 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -22,7 +22,7 @@ libpsl_inline_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSIO noinst_PROGRAMS = psl2c psl2c_SOURCES = psl2c.c psl2c_CPPFLAGS = -I$(top_srcdir)/include -D _GNU_SOURCE -psl2c_LDADD = -lidn2 +#psl2c_LDADD = -lidn2 #psl2c_LDFLAGS = -static # Build rule for suffix.c diff --git a/src/psl2c.c b/src/psl2c.c index 5672fa7..ea785aa 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -71,7 +71,9 @@ static void _add_punycode_if_needed(_psl_vector_t *v) if (_str_needs_encoding(e->label_buf)) { _psl_entry_t suffix; - char *asc = NULL; + + // the following lines will have GPL3+ license issues +/* char *asc = NULL; int rc; if ((rc = idn2_lookup_u8((uint8_t *)e->label_buf, (uint8_t **)&asc, 0)) == IDN2_OK) { @@ -81,6 +83,21 @@ static void _add_punycode_if_needed(_psl_vector_t *v) _vector_add(v, &suffix); } else fprintf(stderr, "toASCII(%s) failed (%d): %s\n", e->label_buf, rc, idn2_strerror(rc)); +*/ + + // this is much slower than the libidn2 API but should have no license issues + FILE *pp; + char cmd[16 + strlen(e->label_buf)], lookupname[64] = ""; + snprintf(cmd, sizeof(cmd), "idn2 '%s'", e->label_buf); + if ((pp = popen(cmd, "r"))) { + if (fscanf(pp, "%63s", lookupname) >= 1) { + _suffix_init(&suffix, lookupname, strlen(lookupname)); + suffix.wildcard = e->wildcard; + _vector_add(v, &suffix); + } + pclose(pp); + } else + fprintf(stderr, "Failed to call popen(%s, \"r\")\n", cmd); } } From 9d1c62eb075abc66dee30fdba6ffb692afbfd93e Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 24 Mar 2014 17:29:56 +0100 Subject: [PATCH 028/104] merged libpsl and libpsl-inline --- include/libpsl.h | 52 ++++--- src/Makefile.am | 9 +- src/psl.c | 143 ++++++++++++++----- src/psl2c.c | 256 +++++++++++++++++++++++++++++++++- tests/Makefile.am | 2 +- tests/test-is-public-inline.c | 47 ++++--- tests/test-is-public.c | 27 ++-- 7 files changed, 445 insertions(+), 91 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index aff9292..894e125 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -1,22 +1,26 @@ /* * Copyright(c) 2014 Tim Ruehsen * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + * * This file is part of libpsl. * - * Libpsl is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Libpsl is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with libpsl. If not, see . - * - * * Header file for libpsl library routines * * Changelog @@ -50,21 +54,35 @@ PSL_BEGIN_DECLS typedef struct _psl_ctx_st psl_ctx_t; +int + psl_global_init(void); +void + psl_global_deinit(void); void psl_free(psl_ctx_t **psl); psl_ctx_t * psl_load_file(const char *fname); psl_ctx_t * psl_load_fp(FILE *fp); +psl_ctx_t * + psl_builtin(void); int psl_is_public(const psl_ctx_t *psl, const char *domain); - -/* does not include exceptions */ +// does not include exceptions int psl_suffix_count(const psl_ctx_t *psl); -/* just counts exceptions */ +// just counts exceptions int psl_suffix_exception_count(const psl_ctx_t *psl); +// returns compilation time +time_t + psl_builtin_compile_time(void); +// returns mtime of PSL source file +time_t + psl_builtin_file_time(void); +// returns MD5 checksum (hex-encoded, lowercase) of PSL source file +const char * + psl_builtin_sha1sum(void); PSL_END_DECLS diff --git a/src/Makefile.am b/src/Makefile.am index 0b3b9dc..bc0dcf4 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -7,17 +7,18 @@ BUILT_SOURCES = suffixes.c CLEANFILES = suffixes.c # build two libraries, 'inline' version with PSL entries compiled in -lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la libpsl-inline-@LIBPSL_API_VERSION@.la +#lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la libpsl-inline-@LIBPSL_API_VERSION@.la +lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la libpsl_@LIBPSL_API_VERSION@_la_SOURCES = psl.c libpsl_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include # include ABI version information libpsl_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) -libpsl_inline_@LIBPSL_API_VERSION@_la_SOURCES = psl-inline.c -libpsl_inline_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include +#libpsl_inline_@LIBPSL_API_VERSION@_la_SOURCES = psl-inline.c +#libpsl_inline_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include # include ABI version information -libpsl_inline_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) +#libpsl_inline_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) noinst_PROGRAMS = psl2c psl2c_SOURCES = psl2c.c diff --git a/src/psl.c b/src/psl.c index 3a1cd1c..7e04326 100644 --- a/src/psl.c +++ b/src/psl.c @@ -1,23 +1,27 @@ /* * Copyright(c) 2014 Tim Ruehsen * - * This file is part of MGet. + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + * + * This file is part of libpsl. * - * Mget is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Mget is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Mget. If not, see . - * - * - * Public Suffix List routines (right now experimental) + * Public Suffix List routines * * Changelog * 19.03.2014 Tim Ruehsen created from libmget/cookie.c @@ -42,6 +46,9 @@ #define countof(a) (sizeof(a)/sizeof(*(a))) +// an invalid pointer +#define _PSL_INTERNAL 1 + typedef struct { char label_buf[48]; @@ -71,6 +78,12 @@ struct _psl_ctx_st { *suffix_exceptions; }; +#include "suffixes.c" + +// references to this PSL will result in lookups to built-in data +static psl_ctx_t + _builtin_psl; + static _psl_vector_t *_vector_alloc(int max, int (*cmp)(const _psl_entry_t *, const _psl_entry_t *)) { _psl_vector_t *v; @@ -232,11 +245,19 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) suffix.nlabels++; // if domain has enough labels, it is public - rule = _vector_get(psl->suffixes, 0); + if (psl == &_builtin_psl) + rule = &suffixes[0]; + else + rule = _vector_get(psl->suffixes, 0); + if (!rule || rule->nlabels < suffix.nlabels - 1) return 1; - rule = _vector_get(psl->suffixes, _vector_find(psl->suffixes, &suffix)); + if (psl == &_builtin_psl) + rule = bsearch(&suffix, suffixes, countof(suffixes), sizeof(suffixes[0]), (int(*)(const void *, const void *))_suffix_compare); + else + rule = _vector_get(psl->suffixes, _vector_find(psl->suffixes, &suffix)); + if (rule) { // definitely a match, no matter if the found rule is a wildcard or not return 0; @@ -250,7 +271,11 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) suffix.length = strlen(suffix.label); suffix.nlabels--; - rule = _vector_get(psl->suffixes, _vector_find(psl->suffixes, &suffix)); + if (psl == &_builtin_psl) + rule = bsearch(&suffix, suffixes, countof(suffixes), sizeof(suffixes[0]), (int(*)(const void *, const void *))_suffix_compare); + else + rule = _vector_get(psl->suffixes, _vector_find(psl->suffixes, &suffix)); + if (rule) { if (rule->wildcard) { // now that we matched a wildcard, we have to check for an exception @@ -258,8 +283,13 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) suffix.length = length_bak; suffix.nlabels++; - if (_vector_get(psl->suffix_exceptions, _vector_find(psl->suffix_exceptions, &suffix)) != 0) - return 1; // found an exception, so 'domain' is public + if (psl == &_builtin_psl) { + if (bsearch(&suffix, suffix_exceptions, countof(suffix_exceptions), sizeof(suffix_exceptions[0]), (int(*)(const void *, const void *))_suffix_compare)) + return 1; // found an exception, so 'domain' is public + } else { + if (_vector_get(psl->suffix_exceptions, _vector_find(psl->suffix_exceptions, &suffix)) != 0) + return 1; // found an exception, so 'domain' is public + } return 0; } @@ -269,6 +299,23 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) return 1; } +int psl_global_init(void) +{ + size_t it; + + for (it = 0; it < countof(suffixes); it++) + suffixes[it].label = suffixes[it].label_buf; + + for (it = 0; it < countof(suffix_exceptions); it++) + suffix_exceptions[it].label = suffix_exceptions[it].label_buf; + + return 0; // 0 = OK +} + +void psl_global_deinit(void) +{ +} + psl_ctx_t *psl_load_file(const char *fname) { FILE *fp; @@ -332,24 +379,56 @@ psl_ctx_t *psl_load_fp(FILE *fp) return psl; } -/* does not include exceptions */ -int psl_suffix_count(const psl_ctx_t *psl) +// return built-in PSL structure +psl_ctx_t *psl_builtin(void) { - return _vector_size(psl->suffixes); -} - -/* just counts exceptions */ -int psl_suffix_exception_count(const psl_ctx_t *psl) -{ - return _vector_size(psl->suffix_exceptions); + return &_builtin_psl; } void psl_free(psl_ctx_t **psl) { if (psl && *psl) { - _vector_free(&(*psl)->suffixes); - _vector_free(&(*psl)->suffix_exceptions); + if (*psl != &_builtin_psl) { + _vector_free(&(*psl)->suffixes); + _vector_free(&(*psl)->suffix_exceptions); + } free(*psl); *psl = NULL; } } + +/* does not include exceptions */ +int psl_suffix_count(const psl_ctx_t *psl) +{ + if (psl == &_builtin_psl) + return countof(suffixes); + else + return _vector_size(psl->suffixes); +} + +/* just counts exceptions */ +int psl_suffix_exception_count(const psl_ctx_t *psl) +{ + if (psl == &_builtin_psl) + return countof(suffix_exceptions); + else + return _vector_size(psl->suffix_exceptions); +} + +// returns compilation time +time_t psl_builtin_compile_time(void) +{ + return _psl_compile_time; +} + +// returns mtime of PSL source file +time_t psl_builtin_file_time(void) +{ + return _psl_file_time; +} + +// returns MD5 checksum (hex-encoded, lowercase) of PSL source file +const char *psl_builtin_sha1sum(void) +{ + return _psl_sha1_checksum; +} diff --git a/src/psl2c.c b/src/psl2c.c index ea785aa..d18f280 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -29,14 +29,253 @@ #endif #include +#include +#include #include +#include #include //#ifdef WITH_LIBIDN2 -# include +//# include //#endif -#include "psl.c" +#include + +typedef struct { + char + label_buf[48]; + const char * + label; + unsigned short + length; + unsigned char + nlabels, // number of labels + wildcard; // this is a wildcard rule (e.g. *.sapporo.jp) +} _psl_entry_t; + +// stripped down version libmget vector routines +typedef struct { + int + (*cmp)(const _psl_entry_t *, const _psl_entry_t *); // comparison function + _psl_entry_t + **entry; // pointer to array of pointers to elements + int + max, // allocated elements + cur; // number of elements in use +} _psl_vector_t; + +struct _psl_ctx_st { + _psl_vector_t + *suffixes, + *suffix_exceptions; +}; + +static _psl_vector_t *_vector_alloc(int max, int (*cmp)(const _psl_entry_t *, const _psl_entry_t *)) +{ + _psl_vector_t *v; + + if (!(v = calloc(1, sizeof(_psl_vector_t)))) + return NULL; + + if (!(v->entry = malloc(max * sizeof(_psl_entry_t *)))) { + free(v); + return NULL; + } + + v->max = max; + v->cmp = cmp; + return v; +} + +static void _vector_free(_psl_vector_t **v) +{ + if (v && *v) { + if ((*v)->entry) { + int it; + + for (it = 0; it < (*v)->cur; it++) + free((*v)->entry[it]); + + free((*v)->entry); + } + free(*v); + } +} + +static _psl_entry_t *_vector_get(const _psl_vector_t *v, int pos) +{ + if (pos < 0 || !v || pos >= v->cur) return NULL; + + return v->entry[pos]; +} + +// the entries must be sorted by +static int _vector_find(const _psl_vector_t *v, const _psl_entry_t *elem) +{ + if (v) { + int l, r, m; + int res; + + // binary search for element (exact match) + for (l = 0, r = v->cur - 1; l <= r;) { + m = (l + r) / 2; + if ((res = v->cmp(elem, v->entry[m])) > 0) l = m + 1; + else if (res < 0) r = m - 1; + else return m; + } + } + + return -1; // not found +} + +static int _vector_add(_psl_vector_t *v, const _psl_entry_t *elem) +{ + if (v) { + void *elemp; + + elemp = malloc(sizeof(_psl_entry_t)); + memcpy(elemp, elem, sizeof(_psl_entry_t)); + + if (v->max == v->cur) + v->entry = realloc(v->entry, (v->max *= 2) * sizeof(_psl_entry_t *)); + + v->entry[v->cur++] = elemp; + return v->cur - 1; + } + + return -1; +} + +static int _compare(const void *p1, const void *p2, void *v) +{ + return ((_psl_vector_t *)v)->cmp(*((_psl_entry_t **)p1), *((_psl_entry_t **)p2)); +} + +static void _vector_sort(_psl_vector_t *v) +{ + if (v && v->cmp) + qsort_r(v->entry, v->cur, sizeof(_psl_vector_t *), _compare, v); +} + +static inline int _vector_size(_psl_vector_t *v) +{ + return v ? v->cur : 0; +} + +// by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) + +static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) +{ + int n; + + if ((n = s2->nlabels - s1->nlabels)) + return n; // most labels first + + if ((n = s1->length - s2->length)) + return n; // shorter rules first + + return strcmp(s1->label, s2->label); +} + +static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) +{ + const char *src; + char *dst; + + suffix->label = suffix->label_buf; + + if (length >= sizeof(suffix->label_buf) - 1) { + suffix->nlabels = 0; + fprintf(stderr, "Suffix rule too long (%zd, ignored): %s\n", length, rule); + return; + } + + if (*rule == '*') { + if (*++rule != '.') { + suffix->nlabels = 0; + fprintf(stderr, "Unsupported kind of rule (ignored): %s\n", rule); + return; + } + rule++; + suffix->wildcard = 1; + suffix->length = (unsigned char)length - 2; + } else { + suffix->wildcard = 0; + suffix->length = (unsigned char)length; + } + + suffix->nlabels = 1; + + for (dst = suffix->label_buf, src = rule; *src;) { + if (*src == '.') + suffix->nlabels++; + *dst++ = tolower(*src++); + } + *dst = 0; +} + +psl_ctx_t *psl_load_file(const char *fname) +{ + FILE *fp; + psl_ctx_t *psl = NULL; + + if ((fp = fopen(fname, "r"))) { + psl = psl_load_fp(fp); + fclose(fp); + } + + return psl; +} + +psl_ctx_t *psl_load_fp(FILE *fp) +{ + psl_ctx_t *psl; + _psl_entry_t suffix, *suffixp; + int nsuffixes = 0; + char buf[256], *linep, *p; + + if (!fp) + return NULL; + + if (!(psl = calloc(1, sizeof(psl_ctx_t)))) + return NULL; + + // as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. + // as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. + psl->suffixes = _vector_alloc(8*1024, _suffix_compare); + psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); + + while ((linep = fgets(buf, sizeof(buf), fp))) { + while (isspace(*linep)) linep++; // ignore leading whitespace + if (!*linep) continue; // skip empty lines + + if (*linep == '/' && linep[1] == '/') + continue; // skip comments + + // parse suffix rule + for (p = linep; *linep && !isspace(*linep);) linep++; + *linep = 0; + + if (*p == '!') { + // add to exceptions + _suffix_init(&suffix, p + 1, linep - p - 1); + suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); + } else { + _suffix_init(&suffix, p, linep - p); + suffixp = _vector_get(psl->suffixes, _vector_add(psl->suffixes, &suffix)); + } + + if (suffixp) + suffixp->label = suffixp->label_buf; // set label to changed address + + nsuffixes++;; + } + + _vector_sort(psl->suffix_exceptions); + _vector_sort(psl->suffixes); + + return psl; +} static void _print_psl_entries(FILE *fpout, const _psl_vector_t *v, const char *varname) { @@ -55,6 +294,16 @@ static void _print_psl_entries(FILE *fpout, const _psl_vector_t *v, const char * fprintf(fpout, "};\n"); } +void psl_free(psl_ctx_t **psl) +{ + if (psl && *psl) { + _vector_free(&(*psl)->suffixes); + _vector_free(&(*psl)->suffix_exceptions); + free(*psl); + *psl = NULL; + } +} + static int _str_needs_encoding(const char *s) { while (*s > 0) s++; @@ -90,7 +339,8 @@ static void _add_punycode_if_needed(_psl_vector_t *v) char cmd[16 + strlen(e->label_buf)], lookupname[64] = ""; snprintf(cmd, sizeof(cmd), "idn2 '%s'", e->label_buf); if ((pp = popen(cmd, "r"))) { - if (fscanf(pp, "%63s", lookupname) >= 1) { + if (fscanf(pp, "%63s", lookupname) >= 1 && strcmp(e->label_buf, lookupname)) { + // fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, lookupname); _suffix_init(&suffix, lookupname, strlen(lookupname)); suffix.wildcard = e->wildcard; _vector_add(v, &suffix); diff --git a/tests/Makefile.am b/tests/Makefile.am index 6595619..b644ffb 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -9,7 +9,7 @@ PSL_TESTS = test-is-public test-is-public-inline check_PROGRAMS = $(PSL_TESTS) #test_is_public_inline_SOURCES = test-is-public-inline.c -test_is_public_inline_LDADD = ../src/libpsl-inline-@LIBPSL_API_VERSION@.la +#test_is_public_inline_LDADD = ../src/libpsl-inline-@LIBPSL_API_VERSION@.la TESTS_ENVIRONMENT = TESTS_VALGRIND="@VALGRIND_ENVIRONMENT@" TESTS = $(PSL_TESTS) diff --git a/tests/test-is-public-inline.c b/tests/test-is-public-inline.c index 56afe41..8ab4ad2 100644 --- a/tests/test-is-public-inline.c +++ b/tests/test-is-public-inline.c @@ -32,7 +32,7 @@ #include #include -#include +#include #define countof(a) (sizeof(a)/sizeof(*(a))) @@ -66,33 +66,36 @@ static void test_psl(void) { "www.xn--czr694b", 1 }, }; unsigned it; + psl_ctx_t *psl; - psl_inline_init(); + if (psl_global_init() == 0) { + psl = psl_builtin(); - printf("have %d suffixes and %d exceptions\n", psl_inline_suffix_count(), psl_inline_suffix_exception_count()); + printf("have %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); - for (it = 0; it < countof(test_data); it++) { - const struct test_data *t = &test_data[it]; - int result = psl_inline_is_public(t->domain); + for (it = 0; it < countof(test_data); it++) { + const struct test_data *t = &test_data[it]; + int result = psl_is_public(psl, t->domain); - if (result == t->result) { - ok++; - } else { - failed++; - printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); + if (result == t->result) { + ok++; + } else { + failed++; + printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); + } } + + printf("psl_builtin_compile_time()=%ld\n", psl_builtin_compile_time()); + psl_builtin_compile_time() == 0 ? failed++ : ok++; + + printf("psl_builtin_file_time()=%ld\n", psl_builtin_file_time()); + psl_builtin_file_time() == 0 ? failed++ : ok++; + + printf("psl_builtin_sha1sum()=%s\n", psl_builtin_sha1sum()); + *psl_builtin_sha1sum() == 0 ? failed++ : ok++; + + psl_global_deinit(); } - - printf("psl_builtin_compile_time()=%ld\n", psl_inline_builtin_compile_time()); - psl_inline_builtin_compile_time() == 0 ? failed++ : ok++; - - printf("psl_builtin_file_time()=%ld\n", psl_inline_builtin_file_time()); - psl_inline_builtin_file_time() == 0 ? failed++ : ok++; - - printf("psl_builtin_sha1sum()=%s\n", psl_inline_builtin_sha1sum()); - *psl_inline_builtin_sha1sum() == 0 ? failed++ : ok++; - - psl_inline_deinit(); } int main(int argc, const char * const *argv) diff --git a/tests/test-is-public.c b/tests/test-is-public.c index d535f29..1057fa8 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -66,23 +66,26 @@ static void test_psl(void) unsigned it; psl_ctx_t *psl; - psl = psl_load_file(DATADIR "/effective_tld_names.dat"); + if (psl_global_init() == 0) { + psl = psl_load_file(DATADIR "/effective_tld_names.dat"); - printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); - for (it = 0; it < countof(test_data); it++) { - const struct test_data *t = &test_data[it]; - int result = psl_is_public(psl, t->domain); + for (it = 0; it < countof(test_data); it++) { + const struct test_data *t = &test_data[it]; + int result = psl_is_public(psl, t->domain); - if (result == t->result) { - ok++; - } else { - failed++; - printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); + if (result == t->result) { + ok++; + } else { + failed++; + printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); + } } - } - psl_free(&psl); + psl_free(&psl); + psl_global_deinit(); + } } int main(int argc, const char * const *argv) From e4950f35d984e753d21e9adc8944f643a0256859 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 24 Mar 2014 17:30:46 +0100 Subject: [PATCH 029/104] added package idn2 in .travis.yml --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 447231f..2690a66 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,4 +6,4 @@ compiler: script: ./autogen.sh && ./configure && make && make check && make distcheck before_install: - sudo apt-get -qq update - - sudo apt-get -q install autoconf automake autopoint libtool gettext libidn2-0 libidn2-0-dev + - sudo apt-get -q install autoconf automake autopoint libtool gettext idn2 libidn2-0 libidn2-0-dev From 8b096a5edadc07715ed99fdedecdf72647cec215 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 24 Mar 2014 20:41:46 +0100 Subject: [PATCH 030/104] LGPL->MIT license, some cleanups --- COPYING | 18 +- LICENSE | 182 ++---------------- include/Makefile.am | 2 +- include/libpsl-inline.h | 78 -------- src/psl-inline.c | 166 ---------------- src/psl2c.c | 51 ++--- tests/Makefile.am | 9 +- ...blic-inline.c => test-is-public-builtin.c} | 30 +-- tests/test-is-public.c | 30 +-- 9 files changed, 90 insertions(+), 476 deletions(-) mode change 120000 => 100644 COPYING delete mode 100644 include/libpsl-inline.h delete mode 100644 src/psl-inline.c rename tests/{test-is-public-inline.c => test-is-public-builtin.c} (67%) diff --git a/COPYING b/COPYING deleted file mode 120000 index caeca07..0000000 --- a/COPYING +++ /dev/null @@ -1 +0,0 @@ -/usr/share/automake-1.14/COPYING \ No newline at end of file diff --git a/COPYING b/COPYING new file mode 100644 index 0000000..1c94b98 --- /dev/null +++ b/COPYING @@ -0,0 +1,17 @@ + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. diff --git a/LICENSE b/LICENSE index bde60ce..1c94b98 100644 --- a/LICENSE +++ b/LICENSE @@ -1,165 +1,17 @@ -GNU LESSER GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 - - Copyright (C) 2007 Free Software Foundation, Inc. - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - - This version of the GNU Lesser General Public License incorporates -the terms and conditions of version 3 of the GNU General Public -License, supplemented by the additional permissions listed below. - - 0. Additional Definitions. - - As used herein, "this License" refers to version 3 of the GNU Lesser -General Public License, and the "GNU GPL" refers to version 3 of the GNU -General Public License. - - "The Library" refers to a covered work governed by this License, -other than an Application or a Combined Work as defined below. - - An "Application" is any work that makes use of an interface provided -by the Library, but which is not otherwise based on the Library. -Defining a subclass of a class defined by the Library is deemed a mode -of using an interface provided by the Library. - - A "Combined Work" is a work produced by combining or linking an -Application with the Library. The particular version of the Library -with which the Combined Work was made is also called the "Linked -Version". - - The "Minimal Corresponding Source" for a Combined Work means the -Corresponding Source for the Combined Work, excluding any source code -for portions of the Combined Work that, considered in isolation, are -based on the Application, and not on the Linked Version. - - The "Corresponding Application Code" for a Combined Work means the -object code and/or source code for the Application, including any data -and utility programs needed for reproducing the Combined Work from the -Application, but excluding the System Libraries of the Combined Work. - - 1. Exception to Section 3 of the GNU GPL. - - You may convey a covered work under sections 3 and 4 of this License -without being bound by section 3 of the GNU GPL. - - 2. Conveying Modified Versions. - - If you modify a copy of the Library, and, in your modifications, a -facility refers to a function or data to be supplied by an Application -that uses the facility (other than as an argument passed when the -facility is invoked), then you may convey a copy of the modified -version: - - a) under this License, provided that you make a good faith effort to - ensure that, in the event an Application does not supply the - function or data, the facility still operates, and performs - whatever part of its purpose remains meaningful, or - - b) under the GNU GPL, with none of the additional permissions of - this License applicable to that copy. - - 3. Object Code Incorporating Material from Library Header Files. - - The object code form of an Application may incorporate material from -a header file that is part of the Library. You may convey such object -code under terms of your choice, provided that, if the incorporated -material is not limited to numerical parameters, data structure -layouts and accessors, or small macros, inline functions and templates -(ten or fewer lines in length), you do both of the following: - - a) Give prominent notice with each copy of the object code that the - Library is used in it and that the Library and its use are - covered by this License. - - b) Accompany the object code with a copy of the GNU GPL and this license - document. - - 4. Combined Works. - - You may convey a Combined Work under terms of your choice that, -taken together, effectively do not restrict modification of the -portions of the Library contained in the Combined Work and reverse -engineering for debugging such modifications, if you also do each of -the following: - - a) Give prominent notice with each copy of the Combined Work that - the Library is used in it and that the Library and its use are - covered by this License. - - b) Accompany the Combined Work with a copy of the GNU GPL and this license - document. - - c) For a Combined Work that displays copyright notices during - execution, include the copyright notice for the Library among - these notices, as well as a reference directing the user to the - copies of the GNU GPL and this license document. - - d) Do one of the following: - - 0) Convey the Minimal Corresponding Source under the terms of this - License, and the Corresponding Application Code in a form - suitable for, and under terms that permit, the user to - recombine or relink the Application with a modified version of - the Linked Version to produce a modified Combined Work, in the - manner specified by section 6 of the GNU GPL for conveying - Corresponding Source. - - 1) Use a suitable shared library mechanism for linking with the - Library. A suitable mechanism is one that (a) uses at run time - a copy of the Library already present on the user's computer - system, and (b) will operate properly with a modified version - of the Library that is interface-compatible with the Linked - Version. - - e) Provide Installation Information, but only if you would otherwise - be required to provide such information under section 6 of the - GNU GPL, and only to the extent that such information is - necessary to install and execute a modified version of the - Combined Work produced by recombining or relinking the - Application with a modified version of the Linked Version. (If - you use option 4d0, the Installation Information must accompany - the Minimal Corresponding Source and Corresponding Application - Code. If you use option 4d1, you must provide the Installation - Information in the manner specified by section 6 of the GNU GPL - for conveying Corresponding Source.) - - 5. Combined Libraries. - - You may place library facilities that are a work based on the -Library side by side in a single library together with other library -facilities that are not Applications and are not covered by this -License, and convey such a combined library under terms of your -choice, if you do both of the following: - - a) Accompany the combined library with a copy of the same work based - on the Library, uncombined with any other library facilities, - conveyed under the terms of this License. - - b) Give prominent notice with the combined library that part of it - is a work based on the Library, and explaining where to find the - accompanying uncombined form of the same work. - - 6. Revised Versions of the GNU Lesser General Public License. - - The Free Software Foundation may publish revised and/or new versions -of the GNU Lesser General Public License from time to time. Such new -versions will be similar in spirit to the present version, but may -differ in detail to address new problems or concerns. - - Each version is given a distinguishing version number. If the -Library as you received it specifies that a certain numbered version -of the GNU Lesser General Public License "or any later version" -applies to it, you have the option of following the terms and -conditions either of that published version or of any later version -published by the Free Software Foundation. If the Library as you -received it does not specify a version number of the GNU Lesser -General Public License, you may choose any version of the GNU Lesser -General Public License ever published by the Free Software Foundation. - - If the Library as you received it specifies that a proxy can decide -whether future versions of the GNU Lesser General Public License shall -apply, that proxy's public statement of acceptance of any version is -permanent authorization for you to choose that version for the -Library. \ No newline at end of file + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. diff --git a/include/Makefile.am b/include/Makefile.am index dd315d8..a45de59 100644 --- a/include/Makefile.am +++ b/include/Makefile.am @@ -1 +1 @@ -include_HEADERS = libpsl.h libpsl-inline.h +include_HEADERS = libpsl.h diff --git a/include/libpsl-inline.h b/include/libpsl-inline.h deleted file mode 100644 index d4ad50f..0000000 --- a/include/libpsl-inline.h +++ /dev/null @@ -1,78 +0,0 @@ -/* - * Copyright(c) 2014 Tim Ruehsen - * - * This file is part of libpsl. - * - * Libpsl is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Libpsl is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with libpsl. If not, see . - * - * - * Header file for libpsl library routines - * - * Changelog - * 22.03.2014 Tim Ruehsen created - * - */ - -#ifndef _LIBPSL_LIBPSL_INLINE_H -#define _LIBPSL_LIBPSL_INLINE_H - -#include - -// Let C++ include C headers -#ifdef __cplusplus -# define PSL_BEGIN_DECLS extern "C" { -# define PSL_END_DECLS } -#else -# define PSL_BEGIN_DECLS -# define PSL_END_DECLS -#endif - -#if ENABLE_NLS != 0 -# include -# define _(STRING) gettext(STRING) -#else -# define _(STRING) STRING -# define ngettext(STRING1,STRING2,N) STRING2 -#endif - -PSL_BEGIN_DECLS - -void - psl_inline_init(void); -void - psl_inline_deinit(void); -int - psl_inline_is_public(const char *domain); - -/* does not include exceptions */ -int - psl_inline_suffix_count(void); -/* just counts exceptions */ -int - psl_inline_suffix_exception_count(void); - -// returns compilation time -time_t - psl_inline_builtin_compile_time(void); -// returns mtime of PSL source file -time_t - psl_inline_builtin_file_time(void); - -// returns MD5 checksum (hex-encoded, lowercase) of PSL source file -const char * - psl_inline_builtin_sha1sum(void); - -PSL_END_DECLS - -#endif /* _LIBPSL_LIBPSL_INLINE_H */ diff --git a/src/psl-inline.c b/src/psl-inline.c deleted file mode 100644 index 51600d9..0000000 --- a/src/psl-inline.c +++ /dev/null @@ -1,166 +0,0 @@ -/* - * Copyright(c) 2014 Tim Ruehsen - * - * This file is part of MGet. - * - * Mget is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Mget is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Mget. If not, see . - * - * - * Public Suffix List routines (right now experimental) - * - * Changelog - * 22.03.2014 Tim Ruehsen created - * - */ - -#if HAVE_CONFIG_H -# include -#endif - -#include -#include -#include -#include - -#include - -#define countof(a) (sizeof(a)/sizeof(*(a))) - -typedef struct { - char - label_buf[48]; - const char * - label; - unsigned short - length; - unsigned char - nlabels, // number of labels - wildcard; // this is a wildcard rule (e.g. *.sapporo.jp) -} _psl_entry_t; - -#include "suffixes.c" - -// by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) - -static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) -{ - int n; - - if ((n = s2->nlabels - s1->nlabels)) - return n; // most labels first - - if ((n = s1->length - s2->length)) - return n; // shorter rules first - - return strcmp(s1->label, s2->label); -} - -void psl_inline_init(void) -{ - size_t it; - - for (it = 0; it < countof(suffixes); it++) - suffixes[it].label = suffixes[it].label_buf; - - for (it = 0; it < countof(suffix_exceptions); it++) - suffix_exceptions[it].label = suffix_exceptions[it].label_buf; -} - -void psl_inline_deinit(void) -{ -} - -int psl_inline_is_public(const char *domain) -{ - _psl_entry_t suffix, *rule; - const char *p, *label_bak; - unsigned short length_bak; - - // this function should be called without leading dots, just make sure - suffix.label = domain + (*domain == '.'); - suffix.length = strlen(suffix.label); - suffix.wildcard = 0; - suffix.nlabels = 1; - - for (p = suffix.label; *p; p++) - if (*p == '.') - suffix.nlabels++; - - // if domain has enough labels, it is public - rule = &suffixes[0]; - if (!rule || rule->nlabels < suffix.nlabels - 1) - return 1; - - rule = bsearch(&suffix, suffixes, countof(suffixes), sizeof(suffixes[0]), (int(*)(const void *, const void *))_suffix_compare); - if (rule) { - // definitely a match, no matter if the found rule is a wildcard or not - return 0; - } - - label_bak = suffix.label; - length_bak = suffix.length; - - if ((suffix.label = strchr(suffix.label, '.'))) { - suffix.label++; - suffix.length = strlen(suffix.label); - suffix.nlabels--; - - rule = bsearch(&suffix, suffixes, countof(suffixes), sizeof(suffixes[0]), (int(*)(const void *, const void *))_suffix_compare); - if (rule) { - if (rule->wildcard) { - // now that we matched a wildcard, we have to check for an exception - suffix.label = label_bak; - suffix.length = length_bak; - suffix.nlabels++; - - if (bsearch(&suffix, suffix_exceptions, countof(suffix_exceptions), sizeof(suffix_exceptions[0]), (int(*)(const void *, const void *))_suffix_compare)) - return 1; // found an exception, so 'domain' is public - - return 0; - } - } - } - - return 1; -} - -/* does not include exceptions */ -int psl_inline_suffix_count(void) -{ - return countof(suffixes); -} - -/* just counts exceptions */ -int psl_inline_suffix_exception_count(void) -{ - return countof(suffix_exceptions); -} - -// returns compilation time -time_t psl_inline_builtin_compile_time(void) -{ - return _psl_compile_time; -} - -// returns mtime of PSL source file -time_t psl_inline_builtin_file_time(void) -{ - return _psl_file_time; -} - -// returns MD5 checksum (hex-encoded, lowercase) of PSL source file -const char *psl_inline_builtin_sha1sum(void) -{ - return _psl_sha1_checksum; -} diff --git a/src/psl2c.c b/src/psl2c.c index d18f280..25bba97 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -1,22 +1,26 @@ /* * Copyright(c) 2014 Tim Ruehsen * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + * * This file is part of libpsl. * - * Libpsl is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Libpsl is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with libpsl. If not, see . - * - * * Precompile Public Suffix List into * * Changelog @@ -109,25 +113,6 @@ static _psl_entry_t *_vector_get(const _psl_vector_t *v, int pos) return v->entry[pos]; } -// the entries must be sorted by -static int _vector_find(const _psl_vector_t *v, const _psl_entry_t *elem) -{ - if (v) { - int l, r, m; - int res; - - // binary search for element (exact match) - for (l = 0, r = v->cur - 1; l <= r;) { - m = (l + r) / 2; - if ((res = v->cmp(elem, v->entry[m])) > 0) l = m + 1; - else if (res < 0) r = m - 1; - else return m; - } - } - - return -1; // not found -} - static int _vector_add(_psl_vector_t *v, const _psl_entry_t *elem) { if (v) { diff --git a/tests/Makefile.am b/tests/Makefile.am index b644ffb..f7e21cf 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,15 +1,12 @@ #DEFS = @DEFS@ -DDATADIR=\"$(datadir)/@PACKAGE@\" -DSRCDIR=\"$(srcdir)\" DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" -AM_CPPFLAGS = -Wno-missing-field-initializers -I$(top_srcdir)/include -AM_LDFLAGS = -static +AM_CPPFLAGS = -I$(top_srcdir)/include +#AM_LDFLAGS = -static LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la -PSL_TESTS = test-is-public test-is-public-inline +PSL_TESTS = test-is-public test-is-public-builtin check_PROGRAMS = $(PSL_TESTS) -#test_is_public_inline_SOURCES = test-is-public-inline.c -#test_is_public_inline_LDADD = ../src/libpsl-inline-@LIBPSL_API_VERSION@.la - TESTS_ENVIRONMENT = TESTS_VALGRIND="@VALGRIND_ENVIRONMENT@" TESTS = $(PSL_TESTS) diff --git a/tests/test-is-public-inline.c b/tests/test-is-public-builtin.c similarity index 67% rename from tests/test-is-public-inline.c rename to tests/test-is-public-builtin.c index 8ab4ad2..adc28d4 100644 --- a/tests/test-is-public-inline.c +++ b/tests/test-is-public-builtin.c @@ -1,23 +1,27 @@ /* * Copyright(c) 2014 Tim Ruehsen * - * This file is part of MGet. + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: * - * Mget is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. * - * Mget is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. * - * You should have received a copy of the GNU General Public License - * along with Mget. If not, see . + * This file is part of the test suite of libpsl. * - * - * Public Suffix List routines (right now experimental) + * Test case for psl built-in functions * * Changelog * 19.03.2014 Tim Ruehsen created from libmget/cookie.c diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 1057fa8..34ad543 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -1,23 +1,27 @@ /* * Copyright(c) 2014 Tim Ruehsen * - * This file is part of MGet. + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: * - * Mget is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. * - * Mget is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. * - * You should have received a copy of the GNU General Public License - * along with Mget. If not, see . + * This file is part of the test suite of libpsl. * - * - * Public Suffix List routines (right now experimental) + * Test case for psl_load_file(), psl_is_public(), psl_free() * * Changelog * 19.03.2014 Tim Ruehsen created from libmget/cookie.c From 2ec5dc7567a4278f40499024c1bc0e56ffac7875 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 24 Mar 2014 20:54:34 +0100 Subject: [PATCH 031/104] small cleanups in Makefile.am --- src/Makefile.am | 8 -------- tests/Makefile.am | 2 -- 2 files changed, 10 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index bc0dcf4..05d4e13 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -6,8 +6,6 @@ BUILT_SOURCES = suffixes.c # suffixes.c is a built source that must be cleaned CLEANFILES = suffixes.c -# build two libraries, 'inline' version with PSL entries compiled in -#lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la libpsl-inline-@LIBPSL_API_VERSION@.la lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la libpsl_@LIBPSL_API_VERSION@_la_SOURCES = psl.c @@ -15,16 +13,10 @@ libpsl_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include # include ABI version information libpsl_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) -#libpsl_inline_@LIBPSL_API_VERSION@_la_SOURCES = psl-inline.c -#libpsl_inline_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include -# include ABI version information -#libpsl_inline_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) - noinst_PROGRAMS = psl2c psl2c_SOURCES = psl2c.c psl2c_CPPFLAGS = -I$(top_srcdir)/include -D _GNU_SOURCE #psl2c_LDADD = -lidn2 -#psl2c_LDFLAGS = -static # Build rule for suffix.c suffixes.c: $(top_srcdir)/data/effective_tld_names.dat psl2c$(EXEEXT) diff --git a/tests/Makefile.am b/tests/Makefile.am index f7e21cf..811d7bd 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,7 +1,5 @@ -#DEFS = @DEFS@ -DDATADIR=\"$(datadir)/@PACKAGE@\" -DSRCDIR=\"$(srcdir)\" DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" AM_CPPFLAGS = -I$(top_srcdir)/include -#AM_LDFLAGS = -static LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la PSL_TESTS = test-is-public test-is-public-builtin From 8f7c266b26587aa51aba12c026ca30242f1d9e83 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 24 Mar 2014 21:18:48 +0100 Subject: [PATCH 032/104] test all entries of effective_tld_names.dat plus some variations --- tests/Makefile.am | 2 +- tests/test-is-public-all.c | 144 +++++++++++++++++++++++++++++++++++++ 2 files changed, 145 insertions(+), 1 deletion(-) create mode 100644 tests/test-is-public-all.c diff --git a/tests/Makefile.am b/tests/Makefile.am index 811d7bd..bf2514c 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -2,7 +2,7 @@ DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" AM_CPPFLAGS = -I$(top_srcdir)/include LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la -PSL_TESTS = test-is-public test-is-public-builtin +PSL_TESTS = test-is-public test-is-public-builtin test-is-public-all check_PROGRAMS = $(PSL_TESTS) diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c new file mode 100644 index 0000000..fb5ed6e --- /dev/null +++ b/tests/test-is-public-all.c @@ -0,0 +1,144 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + * + * This file is part of the test suite of libpsl. + * + * Test psl_is_public() for all entries in effective_tld_names.dat + * + * Changelog + * 19.03.2014 Tim Ruehsen created from libmget/cookie.c + * + */ + +#if HAVE_CONFIG_H +# include +#endif + +#include +#include +#include +#include + +#include + +#define countof(a) (sizeof(a)/sizeof(*(a))) + +static int + ok, + failed; + +static void test_psl(void) +{ + FILE *fp; + psl_ctx_t *psl; + unsigned it, result; + char buf[256], domain[64], *linep, *p; + + if (psl_global_init() == 0) { + psl = psl_load_file(DATADIR "/effective_tld_names.dat"); + + printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + + if ((fp = fopen(DATADIR "/effective_tld_names.dat", "r"))) { + while ((linep = fgets(buf, sizeof(buf), fp))) { + while (isspace(*linep)) linep++; // ignore leading whitespace + if (!*linep) continue; // skip empty lines + + if (*linep == '/' && linep[1] == '/') + continue; // skip comments + + // parse suffix rule + for (p = linep; *linep && !isspace(*linep);) linep++; + *linep = 0; + + if (*p == '!') { // an exception to a wildcard, e.g. !www.ck (wildcard is *.ck) + if (!(result = psl_is_public(psl, p + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 1)\n", p, result); + } else ok++; + + if ((result = psl_is_public(psl, strchr(p, '.') + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", strchr(p, '.') + 1, result); + } else ok++; + } + else if (*p == '*') { // a wildcard, e.g. *.ck + if ((result = psl_is_public(psl, p + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", p + 1, result); + } else ok++; + + *p = 'x'; + if ((result = psl_is_public(psl, p))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", p, result); + } else ok++; + } + else { + if ((result = psl_is_public(psl, p))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", p, result); + } else ok++; + + snprintf(domain, sizeof(domain), "xxxx.%s", p); + if (!(result = psl_is_public(psl, domain))) { + failed++; + printf("psl_is_public(%s)=%d (expected 1)\n", domain, result); + } else ok++; + } + } + + fclose(fp); + } else { + printf("Failed to open %s\n", DATADIR "/effective_tld_names.dat"); + failed++; + } + + psl_free(&psl); + psl_global_deinit(); + } else + failed++; +} + +int main(int argc, const char * const *argv) +{ + // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + if (argc == 1) { + const char *valgrind = getenv("TESTS_VALGRIND"); + + if (valgrind && *valgrind) { + char cmd[strlen(valgrind)+strlen(argv[0])+32]; + + snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + return system(cmd) != 0; + } + } + + test_psl(); + + if (failed) { + printf("Summary: %d out of %d tests failed\n", failed, ok + failed); + return 1; + } + + printf("Summary: All %d tests passed\n", ok + failed); + return 0; +} From 6e7e58aa38ad478aeac7c8e0537c81fda459199e Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Mon, 24 Mar 2014 18:18:45 -0400 Subject: [PATCH 033/104] builtin should be returned const callers should never try to do any non-const operation with the builtin public suffix list. --- include/libpsl.h | 2 +- src/psl.c | 6 +++--- tests/test-is-public-builtin.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index 894e125..c458226 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -64,7 +64,7 @@ psl_ctx_t * psl_load_file(const char *fname); psl_ctx_t * psl_load_fp(FILE *fp); -psl_ctx_t * +const psl_ctx_t * psl_builtin(void); int psl_is_public(const psl_ctx_t *psl, const char *domain); diff --git a/src/psl.c b/src/psl.c index 7e04326..01e654c 100644 --- a/src/psl.c +++ b/src/psl.c @@ -81,8 +81,8 @@ struct _psl_ctx_st { #include "suffixes.c" // references to this PSL will result in lookups to built-in data -static psl_ctx_t - _builtin_psl; +static const psl_ctx_t + _builtin_psl = { .suffixes = NULL, .suffix_exceptions = NULL, }; static _psl_vector_t *_vector_alloc(int max, int (*cmp)(const _psl_entry_t *, const _psl_entry_t *)) { @@ -380,7 +380,7 @@ psl_ctx_t *psl_load_fp(FILE *fp) } // return built-in PSL structure -psl_ctx_t *psl_builtin(void) +const psl_ctx_t *psl_builtin(void) { return &_builtin_psl; } diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index adc28d4..4a86036 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -70,7 +70,7 @@ static void test_psl(void) { "www.xn--czr694b", 1 }, }; unsigned it; - psl_ctx_t *psl; + const psl_ctx_t *psl; if (psl_global_init() == 0) { psl = psl_builtin(); From 73acfc570b0552c873f026682d7b050689bf2e04 Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Mon, 24 Mar 2014 18:33:27 -0400 Subject: [PATCH 034/104] avoid the need for psl_global_init() and psl_global_deinit() There is no need to for an initialization function if the builtin structs if the comparison function will look at label_buf directly when label == NULL. This simplifies the API for users, who now don't have to worry about library initialization and deinitialization functions (these sort of functions can cause headaches in chained library loads, esp. in plugin architectures like PAM). --- src/psl.c | 19 +------------------ tests/test-is-public-all.c | 4 ---- tests/test-is-public-builtin.c | 4 ---- tests/test-is-public.c | 3 --- 4 files changed, 1 insertion(+), 29 deletions(-) diff --git a/src/psl.c b/src/psl.c index 01e654c..484100e 100644 --- a/src/psl.c +++ b/src/psl.c @@ -188,7 +188,7 @@ static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) if ((n = s1->length - s2->length)) return n; // shorter rules first - return strcmp(s1->label, s2->label); + return strcmp(s1->label, s2->label ? s2->label : s2->label_buf); } static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) @@ -299,23 +299,6 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) return 1; } -int psl_global_init(void) -{ - size_t it; - - for (it = 0; it < countof(suffixes); it++) - suffixes[it].label = suffixes[it].label_buf; - - for (it = 0; it < countof(suffix_exceptions); it++) - suffix_exceptions[it].label = suffix_exceptions[it].label_buf; - - return 0; // 0 = OK -} - -void psl_global_deinit(void) -{ -} - psl_ctx_t *psl_load_file(const char *fname) { FILE *fp; diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index fb5ed6e..5c4d93a 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -52,7 +52,6 @@ static void test_psl(void) unsigned it, result; char buf[256], domain[64], *linep, *p; - if (psl_global_init() == 0) { psl = psl_load_file(DATADIR "/effective_tld_names.dat"); printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); @@ -113,9 +112,6 @@ static void test_psl(void) } psl_free(&psl); - psl_global_deinit(); - } else - failed++; } int main(int argc, const char * const *argv) diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index 4a86036..ff1dad4 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -72,7 +72,6 @@ static void test_psl(void) unsigned it; const psl_ctx_t *psl; - if (psl_global_init() == 0) { psl = psl_builtin(); printf("have %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); @@ -97,9 +96,6 @@ static void test_psl(void) printf("psl_builtin_sha1sum()=%s\n", psl_builtin_sha1sum()); *psl_builtin_sha1sum() == 0 ? failed++ : ok++; - - psl_global_deinit(); - } } int main(int argc, const char * const *argv) diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 34ad543..07c3df0 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -70,7 +70,6 @@ static void test_psl(void) unsigned it; psl_ctx_t *psl; - if (psl_global_init() == 0) { psl = psl_load_file(DATADIR "/effective_tld_names.dat"); printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); @@ -88,8 +87,6 @@ static void test_psl(void) } psl_free(&psl); - psl_global_deinit(); - } } int main(int argc, const char * const *argv) From fd6320db9e264aed7328f689c0fce1644e7548f7 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 26 Mar 2014 09:09:22 +0100 Subject: [PATCH 035/104] removed init/deinit function declarations --- include/libpsl.h | 4 ---- 1 file changed, 4 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index c458226..52f44ca 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -54,10 +54,6 @@ PSL_BEGIN_DECLS typedef struct _psl_ctx_st psl_ctx_t; -int - psl_global_init(void); -void - psl_global_deinit(void); void psl_free(psl_ctx_t **psl); psl_ctx_t * From 1d079fceeb1f91fc880ff7b31805beb5c626bb8d Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 26 Mar 2014 09:11:54 +0100 Subject: [PATCH 036/104] fixed indentation --- tests/test-is-public-all.c | 100 ++++++++++++++++----------------- tests/test-is-public-builtin.c | 34 +++++------ tests/test-is-public.c | 24 ++++---- 3 files changed, 79 insertions(+), 79 deletions(-) diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index 5c4d93a..8edf920 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -52,66 +52,66 @@ static void test_psl(void) unsigned it, result; char buf[256], domain[64], *linep, *p; - psl = psl_load_file(DATADIR "/effective_tld_names.dat"); + psl = psl_load_file(DATADIR "/effective_tld_names.dat"); - printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); - if ((fp = fopen(DATADIR "/effective_tld_names.dat", "r"))) { - while ((linep = fgets(buf, sizeof(buf), fp))) { - while (isspace(*linep)) linep++; // ignore leading whitespace - if (!*linep) continue; // skip empty lines + if ((fp = fopen(DATADIR "/effective_tld_names.dat", "r"))) { + while ((linep = fgets(buf, sizeof(buf), fp))) { + while (isspace(*linep)) linep++; // ignore leading whitespace + if (!*linep) continue; // skip empty lines - if (*linep == '/' && linep[1] == '/') - continue; // skip comments + if (*linep == '/' && linep[1] == '/') + continue; // skip comments - // parse suffix rule - for (p = linep; *linep && !isspace(*linep);) linep++; - *linep = 0; + // parse suffix rule + for (p = linep; *linep && !isspace(*linep);) linep++; + *linep = 0; - if (*p == '!') { // an exception to a wildcard, e.g. !www.ck (wildcard is *.ck) - if (!(result = psl_is_public(psl, p + 1))) { - failed++; - printf("psl_is_public(%s)=%d (expected 1)\n", p, result); - } else ok++; + if (*p == '!') { // an exception to a wildcard, e.g. !www.ck (wildcard is *.ck) + if (!(result = psl_is_public(psl, p + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 1)\n", p, result); + } else ok++; - if ((result = psl_is_public(psl, strchr(p, '.') + 1))) { - failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", strchr(p, '.') + 1, result); - } else ok++; - } - else if (*p == '*') { // a wildcard, e.g. *.ck - if ((result = psl_is_public(psl, p + 1))) { - failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", p + 1, result); - } else ok++; - - *p = 'x'; - if ((result = psl_is_public(psl, p))) { - failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", p, result); - } else ok++; - } - else { - if ((result = psl_is_public(psl, p))) { - failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", p, result); - } else ok++; - - snprintf(domain, sizeof(domain), "xxxx.%s", p); - if (!(result = psl_is_public(psl, domain))) { - failed++; - printf("psl_is_public(%s)=%d (expected 1)\n", domain, result); - } else ok++; - } + if ((result = psl_is_public(psl, strchr(p, '.') + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", strchr(p, '.') + 1, result); + } else ok++; } + else if (*p == '*') { // a wildcard, e.g. *.ck + if ((result = psl_is_public(psl, p + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", p + 1, result); + } else ok++; - fclose(fp); - } else { - printf("Failed to open %s\n", DATADIR "/effective_tld_names.dat"); - failed++; + *p = 'x'; + if ((result = psl_is_public(psl, p))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", p, result); + } else ok++; + } + else { + if ((result = psl_is_public(psl, p))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", p, result); + } else ok++; + + snprintf(domain, sizeof(domain), "xxxx.%s", p); + if (!(result = psl_is_public(psl, domain))) { + failed++; + printf("psl_is_public(%s)=%d (expected 1)\n", domain, result); + } else ok++; + } } - psl_free(&psl); + fclose(fp); + } else { + printf("Failed to open %s\n", DATADIR "/effective_tld_names.dat"); + failed++; + } + + psl_free(&psl); } int main(int argc, const char * const *argv) diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index ff1dad4..f563038 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -72,30 +72,30 @@ static void test_psl(void) unsigned it; const psl_ctx_t *psl; - psl = psl_builtin(); + psl = psl_builtin(); - printf("have %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + printf("have %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); - for (it = 0; it < countof(test_data); it++) { - const struct test_data *t = &test_data[it]; - int result = psl_is_public(psl, t->domain); + for (it = 0; it < countof(test_data); it++) { + const struct test_data *t = &test_data[it]; + int result = psl_is_public(psl, t->domain); - if (result == t->result) { - ok++; - } else { - failed++; - printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); - } + if (result == t->result) { + ok++; + } else { + failed++; + printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); } + } - printf("psl_builtin_compile_time()=%ld\n", psl_builtin_compile_time()); - psl_builtin_compile_time() == 0 ? failed++ : ok++; + printf("psl_builtin_compile_time()=%ld\n", psl_builtin_compile_time()); + psl_builtin_compile_time() == 0 ? failed++ : ok++; - printf("psl_builtin_file_time()=%ld\n", psl_builtin_file_time()); - psl_builtin_file_time() == 0 ? failed++ : ok++; + printf("psl_builtin_file_time()=%ld\n", psl_builtin_file_time()); + psl_builtin_file_time() == 0 ? failed++ : ok++; - printf("psl_builtin_sha1sum()=%s\n", psl_builtin_sha1sum()); - *psl_builtin_sha1sum() == 0 ? failed++ : ok++; + printf("psl_builtin_sha1sum()=%s\n", psl_builtin_sha1sum()); + *psl_builtin_sha1sum() == 0 ? failed++ : ok++; } int main(int argc, const char * const *argv) diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 07c3df0..75c6f2a 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -70,23 +70,23 @@ static void test_psl(void) unsigned it; psl_ctx_t *psl; - psl = psl_load_file(DATADIR "/effective_tld_names.dat"); + psl = psl_load_file(DATADIR "/effective_tld_names.dat"); - printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); - for (it = 0; it < countof(test_data); it++) { - const struct test_data *t = &test_data[it]; - int result = psl_is_public(psl, t->domain); + for (it = 0; it < countof(test_data); it++) { + const struct test_data *t = &test_data[it]; + int result = psl_is_public(psl, t->domain); - if (result == t->result) { - ok++; - } else { - failed++; - printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); - } + if (result == t->result) { + ok++; + } else { + failed++; + printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); } + } - psl_free(&psl); + psl_free(&psl); } int main(int argc, const char * const *argv) From 8791ad0b9a3624f59e45784df7ebdf488b6cf217 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 26 Mar 2014 09:12:41 +0100 Subject: [PATCH 037/104] removed unused variable 'it' --- tests/test-is-public-all.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index 8edf920..e207625 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -49,7 +49,7 @@ static void test_psl(void) { FILE *fp; psl_ctx_t *psl; - unsigned it, result; + int result; char buf[256], domain[64], *linep, *p; psl = psl_load_file(DATADIR "/effective_tld_names.dat"); From a18777c2e3066bc489438a5bcb0b15c0a6a7542e Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 26 Mar 2014 17:14:25 +0100 Subject: [PATCH 038/104] new function psl_registered_domain() --- include/libpsl.h | 3 +++ src/psl.c | 26 ++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/include/libpsl.h b/include/libpsl.h index 52f44ca..b500fcc 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -64,6 +64,9 @@ const psl_ctx_t * psl_builtin(void); int psl_is_public(const psl_ctx_t *psl, const char *domain); +// return pointer to longest registered domain within 'domain' or NULL if none found +const char * + psl_registered_domain(const psl_ctx_t *psl, const char *domain); // does not include exceptions int psl_suffix_count(const psl_ctx_t *psl); diff --git a/src/psl.c b/src/psl.c index 484100e..7ddee6a 100644 --- a/src/psl.c +++ b/src/psl.c @@ -299,6 +299,32 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) return 1; } +// return NULL, if string domain does not contain a registered domain +// else return a pointer to the longest registered domain within 'domain' +const char *psl_registered_domain(const psl_ctx_t *psl, const char *domain) +{ + const char *p, *ret_domain; + + // We check from right to left, e.g. in www.xxx.org we check org, xxx.org, www.xxx.org in this order + // for being a registered domain. + + if (!(p = strrchr(domain, '.'))) + return psl_is_public(psl, domain) ? NULL : domain; + + for (ret_domain = NULL; ;) { + if (psl_is_public(psl, p)) + return ret_domain; + else if (p == domain) + return domain; + + ret_domain = p + 1; + + // go left to next dot + while (p > domain && *--p != '.') + ; + } +} + psl_ctx_t *psl_load_file(const char *fname) { FILE *fp; From 96574a795ce7660d1d55ed079d73ccb9571e1c35 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 26 Mar 2014 21:15:25 +0100 Subject: [PATCH 039/104] added reference to Gmane mailing list archive --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index a5c314d..3ca4d3f 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,7 @@ libpsl - C library to handle the Public Suffix List =================================================== Find more information [here](http://publicsuffix.org/). + Download the Public Suffix List [here](https://hg.mozilla.org/mozilla-central/raw-file/tip/netwerk/dns/effective_tld_names.dat). Building from git @@ -20,6 +21,7 @@ Download project and prepare sources with Mailing List ------------ +[Mailing List Archive](http://news.gmane.org/gmane.network.dns.libpsl.bugs) [Mailing List](https://groups.google.com/forum/#!forum/libpsl-bugs) To join the mailing list send an email to From fd0ff2023bd14988873539e9bcb452631e405449 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 26 Mar 2014 22:27:31 +0100 Subject: [PATCH 040/104] added psl_registrable_domain(), renamed psl_registered_domain -> psl_unregistrable_domain --- data/test_psl.txt | 98 +++++++++++++++++++++++++++ include/libpsl.h | 7 +- src/psl.c | 29 +++++++- tests/Makefile.am | 2 +- tests/test-is-public-all.c | 2 +- tests/test-registrable-domain.c | 116 ++++++++++++++++++++++++++++++++ 6 files changed, 249 insertions(+), 5 deletions(-) create mode 100644 data/test_psl.txt create mode 100644 tests/test-registrable-domain.c diff --git a/data/test_psl.txt b/data/test_psl.txt new file mode 100644 index 0000000..35c8ccf --- /dev/null +++ b/data/test_psl.txt @@ -0,0 +1,98 @@ +// Any copyright is dedicated to the Public Domain. +// http://creativecommons.org/publicdomain/zero/1.0/ + +// null input. +checkPublicSuffix(null, null); +// Mixed case. +checkPublicSuffix('COM', null); +checkPublicSuffix('example.COM', 'example.com'); +checkPublicSuffix('WwW.example.COM', 'example.com'); +// Leading dot. +checkPublicSuffix('.com', null); +checkPublicSuffix('.example', null); +checkPublicSuffix('.example.com', null); +checkPublicSuffix('.example.example', null); +// Unlisted TLD. +checkPublicSuffix('example', null); +checkPublicSuffix('example.example', 'example.example'); +checkPublicSuffix('b.example.example', 'example.example'); +checkPublicSuffix('a.b.example.example', 'example.example'); +// Listed, but non-Internet, TLD. +//checkPublicSuffix('local', null); +//checkPublicSuffix('example.local', null); +//checkPublicSuffix('b.example.local', null); +//checkPublicSuffix('a.b.example.local', null); +// TLD with only 1 rule. +checkPublicSuffix('biz', null); +checkPublicSuffix('domain.biz', 'domain.biz'); +checkPublicSuffix('b.domain.biz', 'domain.biz'); +checkPublicSuffix('a.b.domain.biz', 'domain.biz'); +// TLD with some 2-level rules. +checkPublicSuffix('com', null); +checkPublicSuffix('example.com', 'example.com'); +checkPublicSuffix('b.example.com', 'example.com'); +checkPublicSuffix('a.b.example.com', 'example.com'); +checkPublicSuffix('uk.com', null); +checkPublicSuffix('example.uk.com', 'example.uk.com'); +checkPublicSuffix('b.example.uk.com', 'example.uk.com'); +checkPublicSuffix('a.b.example.uk.com', 'example.uk.com'); +checkPublicSuffix('test.ac', 'test.ac'); +// TLD with only 1 (wildcard) rule. +checkPublicSuffix('cy', null); +checkPublicSuffix('c.cy', null); +checkPublicSuffix('b.c.cy', 'b.c.cy'); +checkPublicSuffix('a.b.c.cy', 'b.c.cy'); +// More complex TLD. +checkPublicSuffix('jp', null); +checkPublicSuffix('test.jp', 'test.jp'); +checkPublicSuffix('www.test.jp', 'test.jp'); +checkPublicSuffix('ac.jp', null); +checkPublicSuffix('test.ac.jp', 'test.ac.jp'); +checkPublicSuffix('www.test.ac.jp', 'test.ac.jp'); +checkPublicSuffix('kyoto.jp', null); +checkPublicSuffix('test.kyoto.jp', 'test.kyoto.jp'); +checkPublicSuffix('ide.kyoto.jp', null); +checkPublicSuffix('b.ide.kyoto.jp', 'b.ide.kyoto.jp'); +checkPublicSuffix('a.b.ide.kyoto.jp', 'b.ide.kyoto.jp'); +checkPublicSuffix('c.kobe.jp', null); +checkPublicSuffix('b.c.kobe.jp', 'b.c.kobe.jp'); +checkPublicSuffix('a.b.c.kobe.jp', 'b.c.kobe.jp'); +checkPublicSuffix('city.kobe.jp', 'city.kobe.jp'); +checkPublicSuffix('www.city.kobe.jp', 'city.kobe.jp'); +// TLD with a wildcard rule and exceptions. +checkPublicSuffix('ck', null); +checkPublicSuffix('test.ck', null); +checkPublicSuffix('b.test.ck', 'b.test.ck'); +checkPublicSuffix('a.b.test.ck', 'b.test.ck'); +checkPublicSuffix('www.ck', 'www.ck'); +checkPublicSuffix('www.www.ck', 'www.ck'); +// US K12. +checkPublicSuffix('us', null); +checkPublicSuffix('test.us', 'test.us'); +checkPublicSuffix('www.test.us', 'test.us'); +checkPublicSuffix('ak.us', null); +checkPublicSuffix('test.ak.us', 'test.ak.us'); +checkPublicSuffix('www.test.ak.us', 'test.ak.us'); +checkPublicSuffix('k12.ak.us', null); +checkPublicSuffix('test.k12.ak.us', 'test.k12.ak.us'); +checkPublicSuffix('www.test.k12.ak.us', 'test.k12.ak.us'); +// IDN labels. +checkPublicSuffix('食狮.com.cn', '食狮.com.cn'); +checkPublicSuffix('食狮.公司.cn', '食狮.公司.cn'); +checkPublicSuffix('www.食狮.公司.cn', '食狮.公司.cn'); +checkPublicSuffix('shishi.公司.cn', 'shishi.公司.cn'); +checkPublicSuffix('公司.cn', null); +checkPublicSuffix('食狮.中国', '食狮.中国'); +checkPublicSuffix('www.食狮.中国', '食狮.中国'); +checkPublicSuffix('shishi.中国', 'shishi.中国'); +checkPublicSuffix('中国', null); +// Same as above, but punycoded. +checkPublicSuffix('xn--85x722f.com.cn', 'xn--85x722f.com.cn'); +checkPublicSuffix('xn--85x722f.xn--55qx5d.cn', 'xn--85x722f.xn--55qx5d.cn'); +checkPublicSuffix('www.xn--85x722f.xn--55qx5d.cn', 'xn--85x722f.xn--55qx5d.cn'); +checkPublicSuffix('shishi.xn--55qx5d.cn', 'shishi.xn--55qx5d.cn'); +checkPublicSuffix('xn--55qx5d.cn', null); +checkPublicSuffix('xn--85x722f.xn--fiqs8s', 'xn--85x722f.xn--fiqs8s'); +checkPublicSuffix('www.xn--85x722f.xn--fiqs8s', 'xn--85x722f.xn--fiqs8s'); +checkPublicSuffix('shishi.xn--fiqs8s', 'shishi.xn--fiqs8s'); +checkPublicSuffix('xn--fiqs8s', null); diff --git a/include/libpsl.h b/include/libpsl.h index b500fcc..eeb4de6 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -64,9 +64,12 @@ const psl_ctx_t * psl_builtin(void); int psl_is_public(const psl_ctx_t *psl, const char *domain); -// return pointer to longest registered domain within 'domain' or NULL if none found +// returns the longest unregistrable domain within 'domain' or NULL if none found const char * - psl_registered_domain(const psl_ctx_t *psl, const char *domain); + psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain); +// returns the shortest possible registrable domain part or NULL if domain is not registrable at all +const char * + psl_registrable_domain(const psl_ctx_t *psl, const char *domain); // does not include exceptions int psl_suffix_count(const psl_ctx_t *psl); diff --git a/src/psl.c b/src/psl.c index 7ddee6a..34450d4 100644 --- a/src/psl.c +++ b/src/psl.c @@ -301,10 +301,13 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) // return NULL, if string domain does not contain a registered domain // else return a pointer to the longest registered domain within 'domain' -const char *psl_registered_domain(const psl_ctx_t *psl, const char *domain) +const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) { const char *p, *ret_domain; + if (!psl || !domain) + return NULL; + // We check from right to left, e.g. in www.xxx.org we check org, xxx.org, www.xxx.org in this order // for being a registered domain. @@ -325,6 +328,30 @@ const char *psl_registered_domain(const psl_ctx_t *psl, const char *domain) } } +// returns the shortest possible registrable domain part or NULL if domain is not registrable at all +const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) +{ + const char *p; + int ispublic; + + if (!psl || !domain || *domain == '.') + return NULL; + + // We check from right to left, e.g. in www.xxx.org we check org, xxx.org, www.xxx.org in this order + // for being a registrable domain. + + if (!(p = strrchr(domain, '.'))) + p = domain; + + while (!(ispublic = psl_is_public(psl, p)) && p > domain) { + // go left to next dot + while (p > domain && *--p != '.') + ; + } + + return ispublic ? (*p == '.' ? p + 1 : p) : NULL; +} + psl_ctx_t *psl_load_file(const char *fname) { FILE *fp; diff --git a/tests/Makefile.am b/tests/Makefile.am index bf2514c..d5b9e06 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -2,7 +2,7 @@ DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" AM_CPPFLAGS = -I$(top_srcdir)/include LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la -PSL_TESTS = test-is-public test-is-public-builtin test-is-public-all +PSL_TESTS = test-is-public test-is-public-builtin test-is-public-all test-registrable-domain check_PROGRAMS = $(PSL_TESTS) diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index e207625..b8b6f53 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -24,7 +24,7 @@ * Test psl_is_public() for all entries in effective_tld_names.dat * * Changelog - * 19.03.2014 Tim Ruehsen created from libmget/cookie.c + * 19.03.2014 Tim Ruehsen created * */ diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c new file mode 100644 index 0000000..e6ed954 --- /dev/null +++ b/tests/test-registrable-domain.c @@ -0,0 +1,116 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + * + * This file is part of the test suite of libpsl. + * + * Test psl_registered_domain() for all entries in test_psl.dat + * + * Changelog + * 26.03.2014 Tim Ruehsen created + * + */ + +#if HAVE_CONFIG_H +# include +#endif + +#include +#include +#include +#include + +#include + +#define countof(a) (sizeof(a)/sizeof(*(a))) +#define TESTDATA DATADIR"/test_psl.txt" +static int + ok, + failed; + +static void test_psl(void) +{ + FILE *fp; + const psl_ctx_t *psl; + const char *result; + char buf[256], domain[128], expected_regdom[128], *p; + + psl = psl_builtin(); + + printf("have %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + + if ((fp = fopen(TESTDATA, "r"))) { + while ((fgets(buf, sizeof(buf), fp))) { + if (sscanf(buf, " checkPublicSuffix('%127[^']' , '%127[^']", domain, expected_regdom) != 2) { + if (sscanf(buf, " checkPublicSuffix('%127[^']' , %127[nul]", domain, expected_regdom) != 2) + continue; + } + + // we have to lowercase the domain - the PSL API just takes lowercase + for (p = domain; *p; p++) + if (isupper(*p)) + *p = tolower(*p); + + result = psl_registrable_domain(psl, domain); + + if (result == NULL) { + if (strcmp(expected_regdom, "null")) { + failed++; + printf("psl_registrable_domain(%s)=NULL (expected %s)\n", domain, expected_regdom); + } else ok++; + } else { + if (strcmp(expected_regdom, result)) { + failed++; + printf("psl_registrable_domain(%s)=%s (expected %s)\n", domain, result, expected_regdom); + } else ok++; + } + } + + fclose(fp); + } else { + printf("Failed to open %s\n", TESTDATA); + failed++; + } +} + +int main(int argc, const char * const *argv) +{ + // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + if (argc == 1) { + const char *valgrind = getenv("TESTS_VALGRIND"); + + if (valgrind && *valgrind) { + char cmd[strlen(valgrind)+strlen(argv[0])+32]; + + snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + return system(cmd) != 0; + } + } + + test_psl(); + + if (failed) { + printf("Summary: %d out of %d tests failed\n", failed, ok + failed); + return 1; + } + + printf("Summary: All %d tests passed\n", ok + failed); + return 0; +} From e7599d2ca0ccc99f758c633d5c2c2f6245a967a0 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 27 Mar 2014 10:02:58 +0100 Subject: [PATCH 041/104] fixed punycode sorting --- src/psl2c.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/psl2c.c b/src/psl2c.c index 25bba97..b70e230 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -298,13 +298,14 @@ static int _str_needs_encoding(const char *s) static void _add_punycode_if_needed(_psl_vector_t *v) { - int it; + int it, n; - for (it = 0; it < v->cur; it++) { + // do not use 'it < v->cur' since v->cur is changed by _vector_add() ! + for (it = 0, n = v->cur; it < n; it++) { _psl_entry_t *e = _vector_get(v, it); if (_str_needs_encoding(e->label_buf)) { - _psl_entry_t suffix; + _psl_entry_t suffix, *suffixp; // the following lines will have GPL3+ license issues /* char *asc = NULL; @@ -314,7 +315,8 @@ static void _add_punycode_if_needed(_psl_vector_t *v) // fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, asc); _suffix_init(&suffix, asc, strlen(asc)); suffix.wildcard = e->wildcard; - _vector_add(v, &suffix); + suffixp = _vector_get(v, _vector_add(v, &suffix)); + suffixp->label = suffixp->e_label_buf; // set label to changed address } else fprintf(stderr, "toASCII(%s) failed (%d): %s\n", e->label_buf, rc, idn2_strerror(rc)); */ @@ -328,7 +330,8 @@ static void _add_punycode_if_needed(_psl_vector_t *v) // fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, lookupname); _suffix_init(&suffix, lookupname, strlen(lookupname)); suffix.wildcard = e->wildcard; - _vector_add(v, &suffix); + suffixp = _vector_get(v, _vector_add(v, &suffix)); + suffixp->label = suffixp->label_buf; // set label to changed address } pclose(pp); } else From 87f269b6fb76e410ff9d143ed6e540e658b127fe Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 27 Mar 2014 12:32:35 +0100 Subject: [PATCH 042/104] skip 'example' unregistered TLD tests, added some special tests --- tests/test-registrable-domain.c | 53 +++++++++++++++++++++++++-------- 1 file changed, 40 insertions(+), 13 deletions(-) diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index e6ed954..07cc18b 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -41,21 +41,49 @@ #define countof(a) (sizeof(a)/sizeof(*(a))) #define TESTDATA DATADIR"/test_psl.txt" + static int ok, failed; +static void test(const psl_ctx_t *psl, const char *domain, const char *expected_result) +{ + const char *result = psl_registrable_domain(psl, domain); + + if ((result && expected_result && !strcmp(result, expected_result)) || (!result && !expected_result)) { + ok++; + } else { + failed++; + printf("psl_registrable_domain(%s)=%s (expected %s)\n", + domain, result ? result : "NULL", expected_result ? expected_result : "NULL"); + } +} + static void test_psl(void) { FILE *fp; const psl_ctx_t *psl; - const char *result; char buf[256], domain[128], expected_regdom[128], *p; psl = psl_builtin(); printf("have %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + // special check with NULL values + test(NULL, NULL, NULL); + + // special check with NULL psl context + test(NULL, "www.example.com", NULL); + + // special check with NULL psl context and TLD + test(NULL, "com", NULL); + + // Norwegian with uppercase oe + // test(psl, "www.\303\230yer.no", "www.\303\270yer.no"); + + // Norwegian with lowercase oe + test(psl, "www.\303\270yer.no", "www.\303\270yer.no"); + if ((fp = fopen(TESTDATA, "r"))) { while ((fgets(buf, sizeof(buf), fp))) { if (sscanf(buf, " checkPublicSuffix('%127[^']' , '%127[^']", domain, expected_regdom) != 2) { @@ -68,19 +96,18 @@ static void test_psl(void) if (isupper(*p)) *p = tolower(*p); - result = psl_registrable_domain(psl, domain); - - if (result == NULL) { - if (strcmp(expected_regdom, "null")) { - failed++; - printf("psl_registrable_domain(%s)=NULL (expected %s)\n", domain, expected_regdom); - } else ok++; - } else { - if (strcmp(expected_regdom, result)) { - failed++; - printf("psl_registrable_domain(%s)=%s (expected %s)\n", domain, result, expected_regdom); - } else ok++; + // there are test cases with 'example' unlisted TLD, unsure how to handle these, so skip for the moment + { + size_t len; + + if ((len = strlen(domain)) >= 7 && !strcmp(domain + len - 7, "example")) + continue; } + + if (!strcmp(expected_regdom, "null")) + test(psl, domain, NULL); + else + test(psl, domain, expected_regdom); } fclose(fp); From d894c07bb82708b9e983e54e42a5cff6d047ebb3 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 27 Mar 2014 12:49:46 +0100 Subject: [PATCH 043/104] added test_psl.txt so satisfy distcheck --- data/Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/Makefile.am b/data/Makefile.am index 5b766f9..fa2cffb 100644 --- a/data/Makefile.am +++ b/data/Makefile.am @@ -1,3 +1,3 @@ filesdir = $(datadir)/@PACKAGE@ -files_DATA = effective_tld_names.dat +files_DATA = effective_tld_names.dat test_psl.txt EXTRA_DIST = $(files_DATA) From fb9ccbf88ebe39724c72269f28d0a5103d480e8e Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Thu, 27 Mar 2014 13:16:54 -0400 Subject: [PATCH 044/104] change psl_free() to take a psl_ctx_t *psl This uses the more common convention where the variable freed is the thing returned from the constructor directly, rather than having the deallocator also zero out the pointer itself. --- include/libpsl.h | 2 +- src/psl.c | 13 +++++-------- src/psl2c.c | 13 ++++++------- tests/test-is-public-all.c | 2 +- tests/test-is-public.c | 2 +- 5 files changed, 14 insertions(+), 18 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index eeb4de6..947360e 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -55,7 +55,7 @@ PSL_BEGIN_DECLS typedef struct _psl_ctx_st psl_ctx_t; void - psl_free(psl_ctx_t **psl); + psl_free(psl_ctx_t *psl); psl_ctx_t * psl_load_file(const char *fname); psl_ctx_t * diff --git a/src/psl.c b/src/psl.c index 34450d4..c05c795 100644 --- a/src/psl.c +++ b/src/psl.c @@ -421,15 +421,12 @@ const psl_ctx_t *psl_builtin(void) return &_builtin_psl; } -void psl_free(psl_ctx_t **psl) +void psl_free(psl_ctx_t *psl) { - if (psl && *psl) { - if (*psl != &_builtin_psl) { - _vector_free(&(*psl)->suffixes); - _vector_free(&(*psl)->suffix_exceptions); - } - free(*psl); - *psl = NULL; + if (psl && psl != &_builtin_psl) { + _vector_free(&psl->suffixes); + _vector_free(&psl->suffix_exceptions); + free(psl); } } diff --git a/src/psl2c.c b/src/psl2c.c index b70e230..7817e18 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -279,13 +279,12 @@ static void _print_psl_entries(FILE *fpout, const _psl_vector_t *v, const char * fprintf(fpout, "};\n"); } -void psl_free(psl_ctx_t **psl) +void psl_free(psl_ctx_t *psl) { - if (psl && *psl) { - _vector_free(&(*psl)->suffixes); - _vector_free(&(*psl)->suffix_exceptions); - free(*psl); - *psl = NULL; + if (psl) { + _vector_free(&psl->suffixes); + _vector_free(&psl->suffix_exceptions); + free(psl); } } @@ -389,6 +388,6 @@ int main(int argc, const char **argv) ret = 3; } - psl_free(&psl); + psl_free(psl); return ret; } diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index b8b6f53..ed3d638 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -111,7 +111,7 @@ static void test_psl(void) failed++; } - psl_free(&psl); + psl_free(psl); } int main(int argc, const char * const *argv) diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 75c6f2a..12f3ba6 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -86,7 +86,7 @@ static void test_psl(void) } } - psl_free(&psl); + psl_free(psl); } int main(int argc, const char * const *argv) From 619959db7a2dc33367ee6875b7f67520dd99dfa7 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 27 Mar 2014 21:29:17 +0100 Subject: [PATCH 045/104] added utf-8 to lowercase code using sed --- tests/test-registrable-domain.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index 07cc18b..b57ee6a 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -48,7 +48,34 @@ static int static void test(const psl_ctx_t *psl, const char *domain, const char *expected_result) { - const char *result = psl_registrable_domain(psl, domain); + const char *result; + char lookupname[128]; + + // check if there might be some utf-8 characters + if (domain) { + int utf8; + const char *p; + + for (p = domain, utf8 = 0; *p && !utf8; p++) + if (*p < 0) + utf8 = 1; + + // if we found utf-8, make sure to convert domain correctly to lowercase + // does it work, if we are not in a utf-8 env ? + if (utf8) { + FILE *pp; + char cmd[48 + strlen(domain)]; + + snprintf(cmd, sizeof(cmd), "echo -n '%s' | sed -e 's/./\\L\\0/g'", domain); + if ((pp = popen(cmd, "r"))) { + if (fscanf(pp, "%127s", lookupname) >= 1) + domain = lookupname; + pclose(pp); + } + } + } + + result = psl_registrable_domain(psl, domain); if ((result && expected_result && !strcmp(result, expected_result)) || (!result && !expected_result)) { ok++; @@ -79,7 +106,7 @@ static void test_psl(void) test(NULL, "com", NULL); // Norwegian with uppercase oe - // test(psl, "www.\303\230yer.no", "www.\303\270yer.no"); + test(psl, "www.\303\230yer.no", "www.\303\270yer.no"); // Norwegian with lowercase oe test(psl, "www.\303\270yer.no", "www.\303\270yer.no"); @@ -93,7 +120,7 @@ static void test_psl(void) // we have to lowercase the domain - the PSL API just takes lowercase for (p = domain; *p; p++) - if (isupper(*p)) + if (*p > 0 && isupper(*p)) *p = tolower(*p); // there are test cases with 'example' unlisted TLD, unsure how to handle these, so skip for the moment From 535751b6cc1511a996662365c18568e3a9361b8b Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 28 Mar 2014 22:02:42 +0100 Subject: [PATCH 046/104] prevailing rule is * --- src/psl.c | 3 +++ tests/test-registrable-domain.c | 8 -------- 2 files changed, 3 insertions(+), 8 deletions(-) diff --git a/src/psl.c b/src/psl.c index c05c795..18a71c5 100644 --- a/src/psl.c +++ b/src/psl.c @@ -261,6 +261,9 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) if (rule) { // definitely a match, no matter if the found rule is a wildcard or not return 0; + } else if (suffix.nlabels == 1) { + // unknown TLD, this is the prevailing '*' match + return 0; } label_bak = suffix.label; diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index b57ee6a..8534ea9 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -123,14 +123,6 @@ static void test_psl(void) if (*p > 0 && isupper(*p)) *p = tolower(*p); - // there are test cases with 'example' unlisted TLD, unsure how to handle these, so skip for the moment - { - size_t len; - - if ((len = strlen(domain)) >= 7 && !strcmp(domain + len - 7, "example")) - continue; - } - if (!strcmp(expected_regdom, "null")) test(psl, domain, NULL); else From acb06542c967e2d0a6d83dfb43dc2f646381468a Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 29 Mar 2014 17:42:54 +0100 Subject: [PATCH 047/104] finished comment --- src/psl2c.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/psl2c.c b/src/psl2c.c index 7817e18..7c9f826 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -21,7 +21,7 @@ * * This file is part of libpsl. * - * Precompile Public Suffix List into + * Precompile Public Suffix List into a C source file * * Changelog * 22.03.2014 Tim Ruehsen created From 9d2fde5fce18d629af2e5f108f640064036a3598 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 29 Mar 2014 18:01:03 +0100 Subject: [PATCH 048/104] check input params in exported functions --- src/psl.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/psl.c b/src/psl.c index 18a71c5..5523dfb 100644 --- a/src/psl.c +++ b/src/psl.c @@ -234,6 +234,9 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) const char *p, *label_bak; unsigned short length_bak; + if (!psl || !domain) + return 0; + // this function should be called without leading dots, just make sure suffix.label = domain + (*domain == '.'); suffix.length = strlen(suffix.label); @@ -360,6 +363,9 @@ psl_ctx_t *psl_load_file(const char *fname) FILE *fp; psl_ctx_t *psl = NULL; + if (!fname) + return NULL; + if ((fp = fopen(fname, "r"))) { psl = psl_load_fp(fp); fclose(fp); @@ -438,7 +444,7 @@ int psl_suffix_count(const psl_ctx_t *psl) { if (psl == &_builtin_psl) return countof(suffixes); - else + else if (psl) return _vector_size(psl->suffixes); } @@ -447,7 +453,7 @@ int psl_suffix_exception_count(const psl_ctx_t *psl) { if (psl == &_builtin_psl) return countof(suffix_exceptions); - else + else if (psl) return _vector_size(psl->suffix_exceptions); } From 834f28010023a7704f503831f10c56b8c1182e53 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 29 Mar 2014 18:12:45 +0100 Subject: [PATCH 049/104] removed fprintf from library code --- src/psl.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/src/psl.c b/src/psl.c index 5523dfb..108e549 100644 --- a/src/psl.c +++ b/src/psl.c @@ -191,7 +191,7 @@ static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) return strcmp(s1->label, s2->label ? s2->label : s2->label_buf); } -static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) +static int _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) { const char *src; char *dst; @@ -200,15 +200,15 @@ static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) if (length >= sizeof(suffix->label_buf) - 1) { suffix->nlabels = 0; - fprintf(stderr, _("Suffix rule too long (%zd, ignored): %s\n"), length, rule); - return; + // fprintf(stderr, _("Suffix rule too long (%zd, ignored): %s\n"), length, rule); + return -1; } if (*rule == '*') { if (*++rule != '.') { suffix->nlabels = 0; - fprintf(stderr, _("Unsupported kind of rule (ignored): %s\n"), rule); - return; + // fprintf(stderr, _("Unsupported kind of rule (ignored): %s\n"), rule); + return -2; } rule++; suffix->wildcard = 1; @@ -226,6 +226,8 @@ static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) *dst++ = tolower(*src++); } *dst = 0; + + return 0; } int psl_is_public(const psl_ctx_t *psl, const char *domain) @@ -405,11 +407,11 @@ psl_ctx_t *psl_load_fp(FILE *fp) if (*p == '!') { // add to exceptions - _suffix_init(&suffix, p + 1, linep - p - 1); - suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); + if (_suffix_init(&suffix, p + 1, linep - p - 1) == 0) + suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); } else { - _suffix_init(&suffix, p, linep - p); - suffixp = _vector_get(psl->suffixes, _vector_add(psl->suffixes, &suffix)); + if (_suffix_init(&suffix, p, linep - p) == 0) + suffixp = _vector_get(psl->suffixes, _vector_add(psl->suffixes, &suffix)); } if (suffixp) From a65b7fdad4cc2a02ca14aeffa6abf33760fdc22e Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 29 Mar 2014 18:41:49 +0100 Subject: [PATCH 050/104] lowercase libpsl in AC_INIT --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 016fc71..9fb937d 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([LibPSL], [0.1], [tim.ruehsen@gmx.de], [psl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.1], [tim.ruehsen@gmx.de], [psl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define]) From cf266fc49f0346f6f32a35f72b2b93db6e5f408c Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 29 Mar 2014 18:42:24 +0100 Subject: [PATCH 051/104] own line for mailing list reference --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 3ca4d3f..a554b4d 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,7 @@ Mailing List ------------ [Mailing List Archive](http://news.gmane.org/gmane.network.dns.libpsl.bugs) + [Mailing List](https://groups.google.com/forum/#!forum/libpsl-bugs) To join the mailing list send an email to From 3d3884fc6fa57ec5038cd459fde58595401573ef Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 29 Mar 2014 18:58:24 +0100 Subject: [PATCH 052/104] fixed errors introduced in last change --- src/psl.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/psl.c b/src/psl.c index 108e549..5b10d23 100644 --- a/src/psl.c +++ b/src/psl.c @@ -409,9 +409,13 @@ psl_ctx_t *psl_load_fp(FILE *fp) // add to exceptions if (_suffix_init(&suffix, p + 1, linep - p - 1) == 0) suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); + else + suffixp = NULL; } else { if (_suffix_init(&suffix, p, linep - p) == 0) suffixp = _vector_get(psl->suffixes, _vector_add(psl->suffixes, &suffix)); + else + suffixp = NULL; } if (suffixp) @@ -448,6 +452,8 @@ int psl_suffix_count(const psl_ctx_t *psl) return countof(suffixes); else if (psl) return _vector_size(psl->suffixes); + else + return 0; } /* just counts exceptions */ @@ -457,6 +463,8 @@ int psl_suffix_exception_count(const psl_ctx_t *psl) return countof(suffix_exceptions); else if (psl) return _vector_size(psl->suffix_exceptions); + else + return 0; } // returns compilation time From 2cc4be71c43ac43764bc3d4cd1ad1dd3b62b94f7 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 30 Mar 2014 12:06:16 +0200 Subject: [PATCH 053/104] removed incomplete and unneeded initialization of _builtin_psl --- src/psl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/psl.c b/src/psl.c index 5b10d23..0aaf374 100644 --- a/src/psl.c +++ b/src/psl.c @@ -82,7 +82,7 @@ struct _psl_ctx_st { // references to this PSL will result in lookups to built-in data static const psl_ctx_t - _builtin_psl = { .suffixes = NULL, .suffix_exceptions = NULL, }; + _builtin_psl; static _psl_vector_t *_vector_alloc(int max, int (*cmp)(const _psl_entry_t *, const _psl_entry_t *)) { From 51daebd29557ab9ada92c7db89e743013273ab17 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 30 Mar 2014 12:10:45 +0200 Subject: [PATCH 054/104] include time.h to have time_t --- include/libpsl.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/libpsl.h b/include/libpsl.h index 947360e..8bb4a74 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -32,6 +32,7 @@ #define _LIBPSL_LIBPSL_H #include +#include // Let C++ include C headers #ifdef __cplusplus From 31319a3dc7a15bf4ef8bd3bd82cb2cfbc94dbe51 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 30 Mar 2014 17:02:56 +0200 Subject: [PATCH 055/104] added ./configure --disable-builtin to build without PSL data --- configure.ac | 15 ++++++++++++++- src/psl2c.c | 23 +++++++++++++++++++++++ tests/test-is-public-builtin.c | 2 ++ tests/test-registrable-domain.c | 4 +++- 4 files changed, 42 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 9fb937d..9807e79 100644 --- a/configure.ac +++ b/configure.ac @@ -25,6 +25,19 @@ AM_GNU_GETTEXT_VERSION([0.18.1]) AC_SUBST([LIBPSL_SO_VERSION], [0:0:0]) AC_SUBST([LIBPSL_API_VERSION], [0.1]) +# Check for idn2 +AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) + +# Check for enable/disable builtin PSL data +AC_ARG_ENABLE(builtin, + AS_HELP_STRING([--disable-builtin], [do not compile PSL data into library]), + [ + enable_builtin=no + ], [ + enable_builtin=yes + AC_DEFINE([WITH_BUILTIN], [1], [compile PSL data into library]) + ]) + # Check for valgrind ac_enable_valgrind=no AC_ARG_ENABLE(valgrind-tests, @@ -63,6 +76,6 @@ AC_MSG_NOTICE([Summary of build options: Compiler: ${CC} CFlags: ${CFLAGS} ${CPPFLAGS} LDFlags: ${LDFLAGS} + Builtin PSL: ${enable_builtin} Tests: ${TESTS_INFO} ]) - diff --git a/src/psl2c.c b/src/psl2c.c index 7c9f826..1c29d84 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -43,6 +43,8 @@ //# include //#endif +#ifdef WITH_BUILTIN + #include typedef struct { @@ -340,11 +342,14 @@ static void _add_punycode_if_needed(_psl_vector_t *v) _vector_sort(v); } +#endif // WITH_BUILTIN int main(int argc, const char **argv) { FILE *fpout; +#ifdef WITH_BUILTIN psl_ctx_t *psl; +#endif int ret = 0; if (argc != 3) { @@ -354,6 +359,7 @@ int main(int argc, const char **argv) return 1; } +#ifdef WITH_BUILTIN if (!(psl = psl_load_file(argv[1]))) return 2; @@ -389,5 +395,22 @@ int main(int argc, const char **argv) } psl_free(psl); +#else + if ((fpout = fopen(argv[2], "w"))) { + fprintf(fpout, "static _psl_entry_t suffixes[0];\n"); + fprintf(fpout, "static _psl_entry_t suffix_exceptions[0];\n"); + fprintf(fpout, "static time_t _psl_file_time;\n"); + fprintf(fpout, "static time_t _psl_compile_time;\n"); + fprintf(fpout, "static char _psl_sha1_checksum[]= \"\";\n"); + + if (fclose(fpout) != 0) + ret = 4; + } else { + fprintf(stderr, "Failed to write open '%s'\n", argv[2]); + ret = 3; + } + +#endif // WITH_BUILTIN + return ret; } diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index f563038..13f7e5b 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -112,7 +112,9 @@ int main(int argc, const char * const *argv) } } +#ifdef WITH_BUILTIN test_psl(); +#endif if (failed) { printf("Summary: %d out of %d tests failed\n", failed, ok + failed); diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index 8534ea9..7afecf8 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -150,8 +150,10 @@ int main(int argc, const char * const *argv) } } +#ifdef WITH_BUILTIN test_psl(); - +#endif + if (failed) { printf("Summary: %d out of %d tests failed\n", failed, ok + failed); return 1; From 65a1f039392b18114a0cea844f689480356536f1 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 30 Mar 2014 21:52:00 +0200 Subject: [PATCH 056/104] small code fixes --- src/psl.c | 3 --- src/psl2c.c | 5 ----- tests/test-is-public-all.c | 4 +--- tests/test-registrable-domain.c | 3 +-- 4 files changed, 2 insertions(+), 13 deletions(-) diff --git a/src/psl.c b/src/psl.c index 0aaf374..9374273 100644 --- a/src/psl.c +++ b/src/psl.c @@ -46,9 +46,6 @@ #define countof(a) (sizeof(a)/sizeof(*(a))) -// an invalid pointer -#define _PSL_INTERNAL 1 - typedef struct { char label_buf[48]; diff --git a/src/psl2c.c b/src/psl2c.c index 1c29d84..c5bf12f 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -144,11 +144,6 @@ static void _vector_sort(_psl_vector_t *v) qsort_r(v->entry, v->cur, sizeof(_psl_vector_t *), _compare, v); } -static inline int _vector_size(_psl_vector_t *v) -{ - return v ? v->cur : 0; -} - // by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index ed3d638..2d7f2b9 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -39,8 +39,6 @@ #include -#define countof(a) (sizeof(a)/sizeof(*(a))) - static int ok, failed; @@ -121,7 +119,7 @@ int main(int argc, const char * const *argv) const char *valgrind = getenv("TESTS_VALGRIND"); if (valgrind && *valgrind) { - char cmd[strlen(valgrind)+strlen(argv[0])+32]; + char cmd[strlen(valgrind) + strlen(argv[0]) + 32]; snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); return system(cmd) != 0; diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index 7afecf8..ebd1da6 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -39,7 +39,6 @@ #include -#define countof(a) (sizeof(a)/sizeof(*(a))) #define TESTDATA DATADIR"/test_psl.txt" static int @@ -143,7 +142,7 @@ int main(int argc, const char * const *argv) const char *valgrind = getenv("TESTS_VALGRIND"); if (valgrind && *valgrind) { - char cmd[strlen(valgrind)+strlen(argv[0])+32]; + char cmd[strlen(valgrind) + strlen(argv[0]) + 32]; snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); return system(cmd) != 0; From 9d18b46ed91187d07403a962317eaedde0fc1ae0 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 2 Apr 2014 10:26:40 +0200 Subject: [PATCH 057/104] moved gettext defines away from header file --- include/libpsl.h | 20 ++++---------------- src/psl.c | 20 ++++++++++++++++++++ 2 files changed, 24 insertions(+), 16 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index 8bb4a74..706b1e8 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -34,24 +34,10 @@ #include #include -// Let C++ include C headers #ifdef __cplusplus -# define PSL_BEGIN_DECLS extern "C" { -# define PSL_END_DECLS } -#else -# define PSL_BEGIN_DECLS -# define PSL_END_DECLS +extern "C" { #endif -#if ENABLE_NLS != 0 -# include -# define _(STRING) gettext(STRING) -#else -# define _(STRING) STRING -# define ngettext(STRING1,STRING2,N) STRING2 -#endif - -PSL_BEGIN_DECLS typedef struct _psl_ctx_st psl_ctx_t; @@ -88,6 +74,8 @@ const char * psl_builtin_sha1sum(void); -PSL_END_DECLS +#ifdef __cplusplus +} +#endif #endif /* _LIBPSL_LIBPSL_H */ diff --git a/src/psl.c b/src/psl.c index 9374273..d1761eb 100644 --- a/src/psl.c +++ b/src/psl.c @@ -37,6 +37,14 @@ # include #endif +#if ENABLE_NLS != 0 +# include +# define _(STRING) gettext(STRING) +#else +# define _(STRING) STRING +# define ngettext(STRING1,STRING2,N) STRING2 +#endif + #include #include #include @@ -44,6 +52,17 @@ #include +/** + * SECTION:libpsl + * @short_description: Public Suffix List library functions + * @title: libpsl + * @stability: unstable + * @include: libpsl.h + * + * Public Suffix List library functions. + * + */ + #define countof(a) (sizeof(a)/sizeof(*(a))) typedef struct { @@ -75,6 +94,7 @@ struct _psl_ctx_st { *suffix_exceptions; }; +// include the PSL data compiled by 'psl2c' #include "suffixes.c" // references to this PSL will result in lookups to built-in data From 1d72e45d85550dd00f99c3e50b762651a70f223f Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 2 Apr 2014 10:27:13 +0200 Subject: [PATCH 058/104] added GTKDOC support --- Makefile.am | 4 +- autogen.sh | 15 +++- configure.ac | 33 ++++++++ docs/libpsl/Makefile.am | 135 ++++++++++++++++++++++++++++++++ docs/libpsl/libpsl-docs.sgml | 37 +++++++++ docs/libpsl/libpsl-sections.txt | 17 ++++ 6 files changed, 238 insertions(+), 3 deletions(-) create mode 100644 docs/libpsl/Makefile.am create mode 100644 docs/libpsl/libpsl-docs.sgml create mode 100644 docs/libpsl/libpsl-sections.txt diff --git a/Makefile.am b/Makefile.am index be23856..073e4b3 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,10 +1,10 @@ # got some hints from https://gitorious.org/openismus-playground/examplelib/source -SUBDIRS = po include src tests data +SUBDIRS = po include src data docs/libpsl tests ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS} # Enable GTK-Doc during make distcheck -#DISTCHECK_CONFIGURE_FLAGS = --enable-gtk-doc --enable-man +DISTCHECK_CONFIGURE_FLAGS = --enable-gtk-doc --enable-man ## Install the generated pkg-config file (.pc) into the expected location for ## architecture-dependent package configuration information. Occasionally, diff --git a/autogen.sh b/autogen.sh index b279297..159a3c0 100755 --- a/autogen.sh +++ b/autogen.sh @@ -10,6 +10,15 @@ if test -z `which idn2`; then exit 1 fi +GTKDOCIZE=`which gtkdocize 2>/dev/null` +if test -z $GTKDOCIZE; then + echo "No gtk-doc support found. You can't build the docs." + echo "EXTRA_DIST =" >gtk-doc.make + echo "CLEANFILES =" >>gtk-doc.make +else + gtkdocize || exit $? +fi + mkdir m4 2>/dev/null autoreconf --install --force --symlink || exit $? @@ -19,5 +28,9 @@ echo "----------------------------------------------------------------" echo "Initialized build system. For a common configuration please run:" echo "----------------------------------------------------------------" echo -echo "./configure" +if test -z $GTKDOCIZE; then + echo "./configure" +else + echo "./configure --enable-gtk-doc" +fi echo diff --git a/configure.ac b/configure.ac index 9807e79..f10d2d8 100644 --- a/configure.ac +++ b/configure.ac @@ -18,6 +18,38 @@ m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) AM_GNU_GETTEXT([external],[need-ngettext]) AM_GNU_GETTEXT_VERSION([0.18.1]) +# +# check for gtk-doc +# +m4_ifdef([GTK_DOC_CHECK], [ +GTK_DOC_CHECK([1.15],[--flavour no-tmpl]) +],[ +AM_CONDITIONAL([ENABLE_GTK_DOC], false) +]) + +# +# enable creation of man pages +# +AC_ARG_ENABLE(man,[AC_HELP_STRING([--enable-man], + [generate man pages [default=auto]])],enable_man=yes,enable_man=no) +AS_IF([test "$enable_man" != no], [ + AC_PATH_PROG([XSLTPROC], [xsltproc]) + AS_IF([test -z "$XSLTPROC"], [ + AS_IF([test "$enable_man" = yes], [ + AC_MSG_ERROR([xsltproc is required for --enable-man]) + ]) + enable_man=no + ]) +]) +AM_CONDITIONAL(ENABLE_MAN, test x$enable_man != xno) +AC_MSG_CHECKING([whether to generate man pages]) +AS_IF([ test "$enable_man" != no ], [ + AC_MSG_RESULT([yes]) +], [ + AC_MSG_RESULT([no]) +]) + + # Define these substitions here to keep all version information in one place. # For information on how to properly maintain the library version information, # refer to the libtool manual, section "Updating library version information": @@ -63,6 +95,7 @@ AC_CONFIG_FILES([Makefile include/Makefile src/Makefile po/Makefile.in + docs/libpsl/Makefile data/Makefile tests/Makefile libpsl-${LIBPSL_API_VERSION}.pc:libpsl.pc.in]) diff --git a/docs/libpsl/Makefile.am b/docs/libpsl/Makefile.am new file mode 100644 index 0000000..97279b5 --- /dev/null +++ b/docs/libpsl/Makefile.am @@ -0,0 +1,135 @@ +## Process this file with automake to produce Makefile.in + +# We require automake 1.6 at least. +AUTOMAKE_OPTIONS = 1.6 + +# This is a blank Makefile.am for using gtk-doc. +# Copy this to your project's API docs directory and modify the variables to +# suit your project. See the GTK+ Makefiles in gtk+/docs/reference for examples +# of using the various options. + +# The name of the module, e.g. 'glib'. +DOC_MODULE=libpsl + +# Uncomment for versioned docs and specify the version of the module, e.g. '2'. +#DOC_MODULE_VERSION=2 + + +# The top-level SGML file. You can change this if you want to. +DOC_MAIN_SGML_FILE=$(DOC_MODULE)-docs.sgml + +# Directories containing the source code. +# gtk-doc will search all .c and .h files beneath these paths +# for inline comments documenting functions and macros. +# e.g. DOC_SOURCE_DIR=$(top_srcdir)/gtk $(top_srcdir)/gdk +DOC_SOURCE_DIR=$(top_srcdir)/src $(top_srcdir)/include + +# Extra options to pass to gtkdoc-scangobj. Not normally needed. +#SCANGOBJ_OPTIONS=--ignore-decorators="UNUSED_RESULT|CONST|PURE" + +# Extra options to supply to gtkdoc-scan. +# e.g. SCAN_OPTIONS=--deprecated-guards="GTK_DISABLE_DEPRECATED" +SCAN_OPTIONS=--ignore-decorators="G_GNUC_PSL_UNUSED" + +# Extra options to supply to gtkdoc-mkdb. +# e.g. MKDB_OPTIONS=--xml-mode --output-format=xml +MKDB_OPTIONS=--xml-mode --output-format=xml + +# Extra options to supply to gtkdoc-mktmpl +# e.g. MKTMPL_OPTIONS=--only-section-tmpl +MKTMPL_OPTIONS= + +# Extra options to supply to gtkdoc-mkhtml +MKHTML_OPTIONS= + +# Extra options to supply to gtkdoc-fixref. Not normally needed. +# e.g. FIXXREF_OPTIONS=--extra-dir=../gdk-pixbuf/html --extra-dir=../gdk/html +FIXXREF_OPTIONS= + +# Used for dependencies. The docs will be rebuilt if any of these change. +# e.g. HFILE_GLOB=$(top_srcdir)/gtk/*.h +# e.g. CFILE_GLOB=$(top_srcdir)/gtk/*.c +HFILE_GLOB=$(top_srcdir)/include/*.h +CFILE_GLOB=$(top_srcdir)/src/*.c + +# Extra header to include when scanning, which are not under DOC_SOURCE_DIR +# e.g. EXTRA_HFILES=$(top_srcdir}/contrib/extra.h +EXTRA_HFILES= + +# Header files or dirs to ignore when scanning. Use base file/dir names +# e.g. IGNORE_HFILES=gtkdebug.h gtkintl.h private_code +IGNORE_HFILES=private.h + +# Images to copy into HTML directory. +# e.g. HTML_IMAGES=$(top_srcdir)/gtk/stock-icons/stock_about_24.png +HTML_IMAGES= + +# Extra SGML files that are included by $(DOC_MAIN_SGML_FILE). +# e.g. content_files=running.sgml building.sgml changes-2.0.sgml +content_files= + +# SGML files where gtk-doc abbrevations (#GtkWidget) are expanded +# These files must be listed here *and* in content_files +# e.g. expand_content_files=running.sgml +expand_content_files= + +# CFLAGS and LDFLAGS for compiling gtkdoc-scangobj with your library. +# Only needed if you are using gtkdoc-scangobj to dynamically query widget +# signals and properties. +# e.g. GTKDOC_CFLAGS=-I$(top_srcdir) -I$(top_builddir) $(GTK_DEBUG_FLAGS) +# e.g. GTKDOC_LIBS=$(top_builddir)/gtk/$(gtktargetlib) +GTKDOC_CFLAGS= +GTKDOC_LIBS= + +# This includes the standard gtk-doc make rules, copied by gtkdocize. +include $(top_srcdir)/gtk-doc.make + +# Other files to distribute +# e.g. EXTRA_DIST += version.xml.in +EXTRA_DIST += + +# Files not to distribute +# for --rebuild-types in $(SCAN_OPTIONS), e.g. $(DOC_MODULE).types +# for --rebuild-sections in $(SCAN_OPTIONS) e.g. $(DOC_MODULE)-sections.txt +#DISTCLEANFILES += + +# Comment this out if you want 'make check' to test you doc status +# and run some sanity checks +if ENABLE_GTK_DOC +TESTS_ENVIRONMENT = cd $(srcdir) && \ + DOC_MODULE=$(DOC_MODULE) DOC_MAIN_SGML_FILE=$(DOC_MAIN_SGML_FILE) \ + SRCDIR=$(abs_srcdir) BUILDDIR=$(abs_builddir) +#TESTS = $(GTKDOC_CHECK) +endif + +-include $(top_srcdir)/git.mk + +theMANS = +man_MANS = + +if ENABLE_GTK_DOC +if ENABLE_MAN + +theMANS += libpsl.3 +man_MANS += docs $(theMANS) + +%.3: +#.xml.3: + @file=xml/`echo $@|cut -d'.' -f1`.xml; \ + @XSLTPROC@ -nonet http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl $$file + +endif +endif + +BUILT_EXTRA_DIST = $(theMANS) +EXTRA_DIST += $(theMANS) +CLEANFILES ?= +CLEANFILES += $(theMANS) libpsl-overrides.txt libpsl-decl.txt libpsl-decl-list.txt + +clean-local: + rm -rf xml html + +dist-hook-local: all-local + +libpsl-docs-clean: clean + cd $(srcdir) && rm -rf xml html diff --git a/docs/libpsl/libpsl-docs.sgml b/docs/libpsl/libpsl-docs.sgml new file mode 100644 index 0000000..944cb03 --- /dev/null +++ b/docs/libpsl/libpsl-docs.sgml @@ -0,0 +1,37 @@ + + +]> + + + libpsl Reference Manual + + for libpsl [VERSION]. + The latest version of this documentation can be found on-line at + GitHub. + + + + + libpsl functions + + + + Object Hierarchy + + + + API Index + + + + Index of deprecated API + + + + + diff --git a/docs/libpsl/libpsl-sections.txt b/docs/libpsl/libpsl-sections.txt new file mode 100644 index 0000000..a795a1b --- /dev/null +++ b/docs/libpsl/libpsl-sections.txt @@ -0,0 +1,17 @@ +
+libpsl +Public Suffix List functions +psl_ctx_t +psl_free +psl_load_file +psl_load_fp +psl_builtin +psl_is_public +psl_unregistrable_domain +psl_registrable_domain +psl_suffix_count +psl_suffix_exception_count +psl_builtin_compile_time +psl_builtin_file_time +psl_builtin_sha1sum +
From a0efde2ef30ee6cf47c59f01586e22a3340d9707 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 2 Apr 2014 10:46:31 +0200 Subject: [PATCH 059/104] added installation of gtk-doc-tools --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 2690a66..45afd5b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,4 +6,4 @@ compiler: script: ./autogen.sh && ./configure && make && make check && make distcheck before_install: - sudo apt-get -qq update - - sudo apt-get -q install autoconf automake autopoint libtool gettext idn2 libidn2-0 libidn2-0-dev + - sudo apt-get -q install autoconf automake autopoint libtool gtk-doc-tools gettext idn2 libidn2-0 libidn2-0-dev From 3900084f9c421e3db80cc9b01a5fb3d99655f7eb Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 2 Apr 2014 11:23:38 +0200 Subject: [PATCH 060/104] create m4 directory before gtkdocize --- autogen.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/autogen.sh b/autogen.sh index 159a3c0..d86dce8 100755 --- a/autogen.sh +++ b/autogen.sh @@ -10,6 +10,9 @@ if test -z `which idn2`; then exit 1 fi +# create m4 before gtkdocize +mkdir m4 2>/dev/null + GTKDOCIZE=`which gtkdocize 2>/dev/null` if test -z $GTKDOCIZE; then echo "No gtk-doc support found. You can't build the docs." @@ -19,8 +22,6 @@ else gtkdocize || exit $? fi -mkdir m4 2>/dev/null - autoreconf --install --force --symlink || exit $? echo From 4a33b309a3b3891e9d0de7bd2ee59c985719a375 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 2 Apr 2014 11:34:02 +0200 Subject: [PATCH 061/104] ./configure --enable-gtk-doc in .travis.yml --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 45afd5b..e3e280d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,7 +3,7 @@ compiler: - gcc - clang # Change this to your needs -script: ./autogen.sh && ./configure && make && make check && make distcheck +script: ./autogen.sh && ./configure --enable-gtk-doc && make -j4 && make check -j4 && make distcheck -j4 before_install: - sudo apt-get -qq update - sudo apt-get -q install autoconf automake autopoint libtool gtk-doc-tools gettext idn2 libidn2-0 libidn2-0-dev From c37830f6fc5bbf64f14c302f04796d512cadb0ae Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 6 Apr 2014 22:30:50 +0200 Subject: [PATCH 062/104] reverted meaning of psl_is_public (again), started function docs --- src/psl.c | 101 ++++++++++++++++++++++++++------- tests/test-is-public-all.c | 50 ++++++++-------- tests/test-is-public-builtin.c | 28 ++++----- tests/test-is-public.c | 24 ++++---- 4 files changed, 131 insertions(+), 72 deletions(-) diff --git a/src/psl.c b/src/psl.c index d1761eb..f1818ac 100644 --- a/src/psl.c +++ b/src/psl.c @@ -247,6 +247,22 @@ static int _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) return 0; } +/** + * psl_is_public: + * @psl: PSL context + * @domain: Domain string + * + * This function checks if @domain is a public suffix by the means of the + * [Mozilla Public Suffix List](http://publicsuffix.org). + * + * This can be used for e.g. cookie domain verification. + * You should never accept a cookie who's domain is a public suffix. + * + * @psl is a context returned by either psl_load_file(), psl_load_fp() or + * psl_builtin(). + * + * Returns: 1 if domain is a public suffix, 0 if not. + */ int psl_is_public(const psl_ctx_t *psl, const char *domain) { _psl_entry_t suffix, *rule; @@ -254,7 +270,7 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) unsigned short length_bak; if (!psl || !domain) - return 0; + return 1; // this function should be called without leading dots, just make sure suffix.label = domain + (*domain == '.'); @@ -273,7 +289,7 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) rule = _vector_get(psl->suffixes, 0); if (!rule || rule->nlabels < suffix.nlabels - 1) - return 1; + return 0; if (psl == &_builtin_psl) rule = bsearch(&suffix, suffixes, countof(suffixes), sizeof(suffixes[0]), (int(*)(const void *, const void *))_suffix_compare); @@ -282,10 +298,10 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) if (rule) { // definitely a match, no matter if the found rule is a wildcard or not - return 0; + return 1; } else if (suffix.nlabels == 1) { // unknown TLD, this is the prevailing '*' match - return 0; + return 1; } label_bak = suffix.label; @@ -310,22 +326,34 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) if (psl == &_builtin_psl) { if (bsearch(&suffix, suffix_exceptions, countof(suffix_exceptions), sizeof(suffix_exceptions[0]), (int(*)(const void *, const void *))_suffix_compare)) - return 1; // found an exception, so 'domain' is public + return 0; // found an exception, so 'domain' is not a public suffix } else { if (_vector_get(psl->suffix_exceptions, _vector_find(psl->suffix_exceptions, &suffix)) != 0) - return 1; // found an exception, so 'domain' is public + return 0; // found an exception, so 'domain' is not a public suffix } - return 0; + return 1; } } } - return 1; + return 0; } -// return NULL, if string domain does not contain a registered domain -// else return a pointer to the longest registered domain within 'domain' +/** + * psl_unregistrable_domain: + * @psl: PSL context + * @domain: Domain string + * + * This function finds the longest publix suffix part of @domain by the means + * of the [Mozilla Public Suffix List](http://publicsuffix.org). + * + * @psl is a context returned by either psl_load_file(), psl_load_fp() or + * psl_builtin(). + * + * Returns: Pointer to longest public suffix part of @domain or NULL if @domain + * does not contain a public suffix (or if @psl is NULL). + */ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) { const char *p, *ret_domain; @@ -337,10 +365,10 @@ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) // for being a registered domain. if (!(p = strrchr(domain, '.'))) - return psl_is_public(psl, domain) ? NULL : domain; + return psl_is_public(psl, domain) ? domain : NULL; for (ret_domain = NULL; ;) { - if (psl_is_public(psl, p)) + if (!psl_is_public(psl, p)) return ret_domain; else if (p == domain) return domain; @@ -353,7 +381,20 @@ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) } } -// returns the shortest possible registrable domain part or NULL if domain is not registrable at all +/** + * psl_registrable_domain: + * @psl: PSL context + * @domain: Domain string + * + * This function finds the shortest private suffix part of @domain by the means + * of the [Mozilla Public Suffix List](http://publicsuffix.org). + * + * @psl is a context returned by either psl_load_file(), psl_load_fp() or + * psl_builtin(). + * + * Returns: Pointer to shortest private suffix part of @domain or NULL if @domain + * does not contain a private suffix (or if @psl is NULL). + */ const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) { const char *p; @@ -368,15 +409,24 @@ const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) if (!(p = strrchr(domain, '.'))) p = domain; - while (!(ispublic = psl_is_public(psl, p)) && p > domain) { + while ((ispublic = psl_is_public(psl, p)) && p > domain) { // go left to next dot while (p > domain && *--p != '.') ; } - return ispublic ? (*p == '.' ? p + 1 : p) : NULL; + return ispublic ? NULL : (*p == '.' ? p + 1 : p); } +/** + * psl_load_file: + * @fname: Name of PSL file + * + * This function loads the public suffixes file named @fname. + * To free the allocated resources, call psl_free(). + * + * Returns: Pointer to a PSL context private or NULL on failure. + */ psl_ctx_t *psl_load_file(const char *fname) { FILE *fp; @@ -393,6 +443,15 @@ psl_ctx_t *psl_load_file(const char *fname) return psl; } +/** + * psl_load_fp: + * @fp: FILE pointer + * + * This function loads the public suffixes from a FILE pointer. + * To free the allocated resources, call psl_free(). + * + * Returns: Pointer to a PSL context private or NULL on failure. + */ psl_ctx_t *psl_load_fp(FILE *fp) { psl_ctx_t *psl; @@ -447,12 +506,6 @@ psl_ctx_t *psl_load_fp(FILE *fp) return psl; } -// return built-in PSL structure -const psl_ctx_t *psl_builtin(void) -{ - return &_builtin_psl; -} - void psl_free(psl_ctx_t *psl) { if (psl && psl != &_builtin_psl) { @@ -462,6 +515,12 @@ void psl_free(psl_ctx_t *psl) } } +// return built-in PSL structure +const psl_ctx_t *psl_builtin(void) +{ + return &_builtin_psl; +} + /* does not include exceptions */ int psl_suffix_count(const psl_ctx_t *psl) { diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index 2d7f2b9..016a85c 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -67,38 +67,38 @@ static void test_psl(void) *linep = 0; if (*p == '!') { // an exception to a wildcard, e.g. !www.ck (wildcard is *.ck) + if ((result = psl_is_public(psl, p + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 0)\n", p, result); + } else ok++; + + if (!(result = psl_is_public(psl, strchr(p, '.') + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 1)\n", strchr(p, '.') + 1, result); + } else ok++; + } + else if (*p == '*') { // a wildcard, e.g. *.ck if (!(result = psl_is_public(psl, p + 1))) { + failed++; + printf("psl_is_public(%s)=%d (expected 1)\n", p + 1, result); + } else ok++; + + *p = 'x'; + if (!(result = psl_is_public(psl, p))) { + failed++; + printf("psl_is_public(%s)=%d (expected 1)\n", p, result); + } else ok++; + } + else { + if (!(result = psl_is_public(psl, p))) { failed++; printf("psl_is_public(%s)=%d (expected 1)\n", p, result); } else ok++; - if ((result = psl_is_public(psl, strchr(p, '.') + 1))) { - failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", strchr(p, '.') + 1, result); - } else ok++; - } - else if (*p == '*') { // a wildcard, e.g. *.ck - if ((result = psl_is_public(psl, p + 1))) { - failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", p + 1, result); - } else ok++; - - *p = 'x'; - if ((result = psl_is_public(psl, p))) { - failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", p, result); - } else ok++; - } - else { - if ((result = psl_is_public(psl, p))) { - failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", p, result); - } else ok++; - snprintf(domain, sizeof(domain), "xxxx.%s", p); - if (!(result = psl_is_public(psl, domain))) { + if ((result = psl_is_public(psl, domain))) { failed++; - printf("psl_is_public(%s)=%d (expected 1)\n", domain, result); + printf("psl_is_public(%s)=%d (expected 0)\n", domain, result); } else ok++; } } diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index 13f7e5b..0a94915 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -54,20 +54,20 @@ static void test_psl(void) int result; } test_data[] = { - { "www.example.com", 1 }, - { "com.ar", 0 }, - { "www.com.ar", 1 }, - { "cc.ar.us", 0 }, - { ".cc.ar.us", 0 }, - { "www.cc.ar.us", 1 }, - { "www.ck", 1 }, // exception from *.ck - { "abc.www.ck", 1 }, - { "xxx.ck", 0 }, - { "www.xxx.ck", 1 }, - { "\345\225\206\346\240\207", 0 }, // xn--czr694b oder 商标 - { "www.\345\225\206\346\240\207", 1 }, - { "xn--czr694b", 0 }, - { "www.xn--czr694b", 1 }, + { "www.example.com", 0 }, + { "com.ar", 1 }, + { "www.com.ar", 0 }, + { "cc.ar.us", 1 }, + { ".cc.ar.us", 1 }, + { "www.cc.ar.us", 0 }, + { "www.ck", 0 }, // exception from *.ck + { "abc.www.ck", 0 }, + { "xxx.ck", 1 }, + { "www.xxx.ck", 0 }, + { "\345\225\206\346\240\207", 1 }, // xn--czr694b oder 商标 + { "www.\345\225\206\346\240\207", 0 }, + { "xn--czr694b", 1 }, + { "www.xn--czr694b", 0 }, }; unsigned it; const psl_ctx_t *psl; diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 12f3ba6..0155804 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -54,18 +54,18 @@ static void test_psl(void) int result; } test_data[] = { - { "www.example.com", 1 }, - { "com.ar", 0 }, - { "www.com.ar", 1 }, - { "cc.ar.us", 0 }, - { ".cc.ar.us", 0 }, - { "www.cc.ar.us", 1 }, - { "www.ck", 1 }, // exception from *.ck - { "abc.www.ck", 1 }, - { "xxx.ck", 0 }, - { "www.xxx.ck", 1 }, - { "\345\225\206\346\240\207", 0 }, // xn--czr694b oder 商标 - { "www.\345\225\206\346\240\207", 1 }, + { "www.example.com", 0 }, + { "com.ar", 1 }, + { "www.com.ar", 0 }, + { "cc.ar.us", 1 }, + { ".cc.ar.us", 1 }, + { "www.cc.ar.us", 0 }, + { "www.ck", 0 }, // exception from *.ck + { "abc.www.ck", 0 }, + { "xxx.ck", 1 }, + { "www.xxx.ck", 0 }, + { "\345\225\206\346\240\207", 1 }, // xn--czr694b oder 商标 + { "www.\345\225\206\346\240\207", 0 }, }; unsigned it; psl_ctx_t *psl; From 0dad83966de9c7e740e2dc9f3dc1ea0fb7088ee5 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 11 Apr 2014 16:30:20 +0200 Subject: [PATCH 063/104] added the missing function documentation --- src/psl.c | 111 ++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 99 insertions(+), 12 deletions(-) diff --git a/src/psl.c b/src/psl.c index f1818ac..bce1e6a 100644 --- a/src/psl.c +++ b/src/psl.c @@ -59,7 +59,7 @@ * @stability: unstable * @include: libpsl.h * - * Public Suffix List library functions. + * [Public Suffix List](http://publicsuffix.org/) library functions. * */ @@ -262,6 +262,8 @@ static int _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) * psl_builtin(). * * Returns: 1 if domain is a public suffix, 0 if not. + * + * Since: 0.1 */ int psl_is_public(const psl_ctx_t *psl, const char *domain) { @@ -351,8 +353,10 @@ int psl_is_public(const psl_ctx_t *psl, const char *domain) * @psl is a context returned by either psl_load_file(), psl_load_fp() or * psl_builtin(). * - * Returns: Pointer to longest public suffix part of @domain or NULL if @domain - * does not contain a public suffix (or if @psl is NULL). + * Returns: Pointer to longest public suffix part of @domain or %NULL if @domain + * does not contain a public suffix (or if @psl is %NULL). + * + * Since: 0.1 */ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) { @@ -392,8 +396,10 @@ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) * @psl is a context returned by either psl_load_file(), psl_load_fp() or * psl_builtin(). * - * Returns: Pointer to shortest private suffix part of @domain or NULL if @domain - * does not contain a private suffix (or if @psl is NULL). + * Returns: Pointer to shortest private suffix part of @domain or %NULL if @domain + * does not contain a private suffix (or if @psl is %NULL). + * + * Since: 0.1 */ const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) { @@ -425,7 +431,9 @@ const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) * This function loads the public suffixes file named @fname. * To free the allocated resources, call psl_free(). * - * Returns: Pointer to a PSL context private or NULL on failure. + * Returns: Pointer to a PSL context or %NULL on failure. + * + * Since: 0.1 */ psl_ctx_t *psl_load_file(const char *fname) { @@ -450,7 +458,9 @@ psl_ctx_t *psl_load_file(const char *fname) * This function loads the public suffixes from a FILE pointer. * To free the allocated resources, call psl_free(). * - * Returns: Pointer to a PSL context private or NULL on failure. + * Returns: Pointer to a PSL context or %NULL on failure. + * + * Since: 0.1 */ psl_ctx_t *psl_load_fp(FILE *fp) { @@ -506,6 +516,17 @@ psl_ctx_t *psl_load_fp(FILE *fp) return psl; } +/** + * psl_load_free: + * @psl: PSL context pointer + * + * This function frees the the PSL context that has been retrieved via + * psl_load_fp() or psl_load_file(). + * + * Returns: Pointer to a PSL context private or %NULL on failure. + * + * Since: 0.1 + */ void psl_free(psl_ctx_t *psl) { if (psl && psl != &_builtin_psl) { @@ -515,13 +536,36 @@ void psl_free(psl_ctx_t *psl) } } -// return built-in PSL structure +/** + * psl_builtin: + * + * This function returns the PSL context that has been generated and built in at compile-time. + * You don't have to free the returned context explicitely. + * + * If the generation of built-in data has been disabled during compilation, %NULL will be returned. + * + * Returns: Pointer to the built in PSL data or NULL if this data is not available. + * + * Since: 0.1 + */ const psl_ctx_t *psl_builtin(void) { return &_builtin_psl; } -/* does not include exceptions */ +/** + * psl_suffix_count: + * @psl: PSL context pointer + * + * This function returns number of public suffixes maintained by @psl. + * The number of exceptions within the Public Suffix List are not included. + * + * If the generation of built-in data has been disabled during compilation, 0 will be returned. + * + * Returns: Number of public suffixes entries in PSL context. + * + * Since: 0.1 + */ int psl_suffix_count(const psl_ctx_t *psl) { if (psl == &_builtin_psl) @@ -532,7 +576,18 @@ int psl_suffix_count(const psl_ctx_t *psl) return 0; } -/* just counts exceptions */ +/** + * psl_suffix_exception_count: + * @psl: PSL context pointer + * + * This function returns number of public suffix exceptions maintained by @psl. + * + * If the generation of built-in data has been disabled during compilation, 0 will be returned. + * + * Returns: Number of public suffix exceptions in PSL context. + * + * Since: 0.1 + */ int psl_suffix_exception_count(const psl_ctx_t *psl) { if (psl == &_builtin_psl) @@ -543,19 +598,51 @@ int psl_suffix_exception_count(const psl_ctx_t *psl) return 0; } -// returns compilation time +/** + * psl_builtin_compile_time: + * + * This function returns the time when the Publix Suffix List has been compiled into C code (by psl2c). + * + * If the generation of built-in data has been disabled during compilation, 0 will be returned. + * + * Returns: time_t value or 0. + * + * Since: 0.1 + */ time_t psl_builtin_compile_time(void) { return _psl_compile_time; } -// returns mtime of PSL source file +/** + * psl_builtin_file_time: + * + * This function returns the mtime of the Publix Suffix List file that has been built in. + * + * If the generation of built-in data has been disabled during compilation, 0 will be returned. + * + * Returns: time_t value or 0. + * + * Since: 0.1 + */ time_t psl_builtin_file_time(void) { return _psl_file_time; } // returns MD5 checksum (hex-encoded, lowercase) of PSL source file +/** + * psl_builtin_sha1sum: + * + * This function returns the SHA1 checksum of the Publix Suffix List file that has been built in. + * The returned string is in lowercase hex encoding, e.g. "2af1e9e3044eda0678bb05949d7cca2f769901d8". + * + * If the generation of built-in data has been disabled during compilation, an empty string will be returned. + * + * Returns: String containing SHA1 checksum or an empty string. + * + * Since: 0.1 + */ const char *psl_builtin_sha1sum(void) { return _psl_sha1_checksum; From 2a404722f686dd79ecd25f579300898378e6377f Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 11 Apr 2014 17:16:23 +0200 Subject: [PATCH 064/104] added a 'psl' tool to acces functions from command line --- Makefile.am | 2 +- configure.ac | 1 + tools/.deps/psl.Po | 126 +++++++++++++++++++++++++++++++++++++++++++++ tools/Makefile.am | 5 ++ tools/psl | Bin 0 -> 16343 bytes tools/psl.c | 98 +++++++++++++++++++++++++++++++++++ tools/psl.o | Bin 0 -> 18736 bytes 7 files changed, 231 insertions(+), 1 deletion(-) create mode 100644 tools/.deps/psl.Po create mode 100644 tools/Makefile.am create mode 100755 tools/psl create mode 100644 tools/psl.c create mode 100644 tools/psl.o diff --git a/Makefile.am b/Makefile.am index 073e4b3..1523ed7 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,6 +1,6 @@ # got some hints from https://gitorious.org/openismus-playground/examplelib/source -SUBDIRS = po include src data docs/libpsl tests +SUBDIRS = po include src tools data docs/libpsl tests ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS} # Enable GTK-Doc during make distcheck diff --git a/configure.ac b/configure.ac index f10d2d8..ffc5bbd 100644 --- a/configure.ac +++ b/configure.ac @@ -94,6 +94,7 @@ fi AC_CONFIG_FILES([Makefile include/Makefile src/Makefile + tools/Makefile po/Makefile.in docs/libpsl/Makefile data/Makefile diff --git a/tools/.deps/psl.Po b/tools/.deps/psl.Po new file mode 100644 index 0000000..367480e --- /dev/null +++ b/tools/.deps/psl.Po @@ -0,0 +1,126 @@ +psl.o: psl.c /usr/include/stdc-predef.h ../config.h /usr/include/stdlib.h \ + /usr/include/features.h /usr/include/x86_64-linux-gnu/sys/cdefs.h \ + /usr/include/x86_64-linux-gnu/bits/wordsize.h \ + /usr/include/x86_64-linux-gnu/gnu/stubs.h \ + /usr/include/x86_64-linux-gnu/gnu/stubs-64.h \ + /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h \ + /usr/include/x86_64-linux-gnu/bits/waitflags.h \ + /usr/include/x86_64-linux-gnu/bits/waitstatus.h /usr/include/endian.h \ + /usr/include/x86_64-linux-gnu/bits/endian.h \ + /usr/include/x86_64-linux-gnu/bits/byteswap.h \ + /usr/include/x86_64-linux-gnu/bits/types.h \ + /usr/include/x86_64-linux-gnu/bits/typesizes.h \ + /usr/include/x86_64-linux-gnu/bits/byteswap-16.h /usr/include/xlocale.h \ + /usr/include/x86_64-linux-gnu/sys/types.h /usr/include/time.h \ + /usr/include/x86_64-linux-gnu/sys/select.h \ + /usr/include/x86_64-linux-gnu/bits/select.h \ + /usr/include/x86_64-linux-gnu/bits/sigset.h \ + /usr/include/x86_64-linux-gnu/bits/time.h \ + /usr/include/x86_64-linux-gnu/bits/select2.h \ + /usr/include/x86_64-linux-gnu/sys/sysmacros.h \ + /usr/include/x86_64-linux-gnu/bits/pthreadtypes.h /usr/include/alloca.h \ + /usr/include/x86_64-linux-gnu/bits/stdlib-bsearch.h \ + /usr/include/x86_64-linux-gnu/bits/stdlib-float.h \ + /usr/include/x86_64-linux-gnu/bits/stdlib.h /usr/include/string.h \ + /usr/include/x86_64-linux-gnu/bits/string.h \ + /usr/include/x86_64-linux-gnu/bits/string2.h \ + /usr/include/x86_64-linux-gnu/bits/string3.h ../include/libpsl.h \ + /usr/include/stdio.h /usr/include/libio.h /usr/include/_G_config.h \ + /usr/include/wchar.h /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stdarg.h \ + /usr/include/x86_64-linux-gnu/bits/stdio_lim.h \ + /usr/include/x86_64-linux-gnu/bits/sys_errlist.h \ + /usr/include/x86_64-linux-gnu/bits/stdio.h \ + /usr/include/x86_64-linux-gnu/bits/stdio2.h \ + /usr/include/x86_64-linux-gnu/bits/timex.h + +/usr/include/stdc-predef.h: + +../config.h: + +/usr/include/stdlib.h: + +/usr/include/features.h: + +/usr/include/x86_64-linux-gnu/sys/cdefs.h: + +/usr/include/x86_64-linux-gnu/bits/wordsize.h: + +/usr/include/x86_64-linux-gnu/gnu/stubs.h: + +/usr/include/x86_64-linux-gnu/gnu/stubs-64.h: + +/usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h: + +/usr/include/x86_64-linux-gnu/bits/waitflags.h: + +/usr/include/x86_64-linux-gnu/bits/waitstatus.h: + +/usr/include/endian.h: + +/usr/include/x86_64-linux-gnu/bits/endian.h: + +/usr/include/x86_64-linux-gnu/bits/byteswap.h: + +/usr/include/x86_64-linux-gnu/bits/types.h: + +/usr/include/x86_64-linux-gnu/bits/typesizes.h: + +/usr/include/x86_64-linux-gnu/bits/byteswap-16.h: + +/usr/include/xlocale.h: + +/usr/include/x86_64-linux-gnu/sys/types.h: + +/usr/include/time.h: + +/usr/include/x86_64-linux-gnu/sys/select.h: + +/usr/include/x86_64-linux-gnu/bits/select.h: + +/usr/include/x86_64-linux-gnu/bits/sigset.h: + +/usr/include/x86_64-linux-gnu/bits/time.h: + +/usr/include/x86_64-linux-gnu/bits/select2.h: + +/usr/include/x86_64-linux-gnu/sys/sysmacros.h: + +/usr/include/x86_64-linux-gnu/bits/pthreadtypes.h: + +/usr/include/alloca.h: + +/usr/include/x86_64-linux-gnu/bits/stdlib-bsearch.h: + +/usr/include/x86_64-linux-gnu/bits/stdlib-float.h: + +/usr/include/x86_64-linux-gnu/bits/stdlib.h: + +/usr/include/string.h: + +/usr/include/x86_64-linux-gnu/bits/string.h: + +/usr/include/x86_64-linux-gnu/bits/string2.h: + +/usr/include/x86_64-linux-gnu/bits/string3.h: + +../include/libpsl.h: + +/usr/include/stdio.h: + +/usr/include/libio.h: + +/usr/include/_G_config.h: + +/usr/include/wchar.h: + +/usr/lib/gcc/x86_64-linux-gnu/4.8/include/stdarg.h: + +/usr/include/x86_64-linux-gnu/bits/stdio_lim.h: + +/usr/include/x86_64-linux-gnu/bits/sys_errlist.h: + +/usr/include/x86_64-linux-gnu/bits/stdio.h: + +/usr/include/x86_64-linux-gnu/bits/stdio2.h: + +/usr/include/x86_64-linux-gnu/bits/timex.h: diff --git a/tools/Makefile.am b/tools/Makefile.am new file mode 100644 index 0000000..397a6de --- /dev/null +++ b/tools/Makefile.am @@ -0,0 +1,5 @@ +bin_PROGRAMS = psl + +AM_CPPFLAGS = -I$(top_srcdir)/include +AM_LDFLAGS = -static +LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la diff --git a/tools/psl b/tools/psl new file mode 100755 index 0000000000000000000000000000000000000000..18f12f2784b273108f31c411671509ef31d55735 GIT binary patch literal 16343 zcmc&*eRxz?nLl?XnM^V;NdQR#q1*x?K$}TI1M=NT2z1g0N+D@cfy*SBBvX=^bUtWk z{XharL@Kr2ZJ(-z_1W##-D<1Pvn}FN2-0oU?XJ~Ytyb5r7EM5HD=nhcW`FOw=e=|1 zX5z;`yJw!c@ArG&^WJmMIrpA>KeC~9qf65S7nfKkh^qDlCCfYtQF5&$6>o*`ilC?x z^MnVSeEfq_LeRt*jwsg*y-Lpwm`AuvF?)a~}|GD^cjUPHO zciC4-MyG`HfB3AfiM*hXc%}H0eba`Qe)YAVfBJ(j-?6>&5s>ZWGb9U8HO-!+-q!Adh7eb$z&vHn&&p|j>e(_bs6cHsgNM2MpwK)9E}MB(v%cO zlg2>0qc7Sisg88CF9kUn<}gF}ofPn8H6*#=(e%C2hyodkxZd& z3DJFLBATKGk-ch=8AhZloC-tGkxZI=VMJnGqBfmO)W-XhwaG*$UCRhZq5(v(Ai54Zn1i5>Ak=RrBYP&?QE|qj7*rsv=rQSNukK{)I@-BHAUk zf~MkC{e;FM`*_6tgvRdaq^QawrC$JAxsZ}N8{dq|G+1Hd^Vmm{pp9>zA3`?1J)SK# zKGn@-yN%C%lw`YY{32Ec?6vWIHvWK(KgGr$wDIi&TxjT5-q4Z!bdfGXhelHQqn$7c z4L##MXEO#mzN`Z(<5gWa7S9a=Bf6KA#$Fvqsd_)rl*+LeB)yer8n(wqBz+^%l+v*$ zB;77_(d>c;jrHMU#Qvxr8Y5Mx^;J&kBe z*;o*CfL4<-?0=;%)GjYs=_EkY|V77x7$tCQ4} z##19xgQD@P)QrZjQ9c`=eh(c_Q(wBct~*dX_aHDK&U_n+q2ceh4v$@Wr1&5WcV}K> z)|+4*a>LAmXreORd;?R!=wqhn?E;8eERl|Q zM(;I6KO)ghmS})Pd!=aE8Hmh4^T9CFbrZ1tcWE`nhsvnccezCIp{bN^oT@Rifm1(D zc8{X^Q&+Ly(fQ)3w})zI{0j|T@rQ=b-|5uYVq9t#5+k8>z}>i zhJI-H>!Gt3t`BK1guZhn4dYX!7&`Rb)LT14!_S)bKU4Pm*Piw07?w+MeVrfuFdDk^ z+59horCmBl=Q}jCwmc6CsjI9IX0)Fv!KxPe<1g%#eWQ4;7)!zwu<^FWVO+E=!GQ^$oX<5RcXn+}67IeEln-vww2a2sXOx9c1>_CjGklEC z_%U*>q0go-(7g}?SUpZOV$kjoOX_{Sulj&mPb8T#oet`zQ+)u-5_XvJhebn9Z@QlGm+Xxv zQqo}}x;LDPO!Qb&BWw)^J~3E}nW<5<$M(eHcgFPiKq?xK>D5)q>LSse4EID<=~zPO zA20>Uokikisghh(B&w3DbWA)ZOcn`kwtICQvP6?b>i6U09|Uap!}$0=0$#+m&qM4LUOzIMKUZK^!yB03OUU zX^-Rzal&QNe7)^b@zJ@YQf>l4vzNWlf`Cd1hF>atj=ajA`ysCt%cPbN`#nL4F@KAq zIc~Z>s1%yOsvnnwY>%$=G1^L+^ZkaRFXso9<^K)f=Niu1o-Hur_`ZU_Q1G&X9zt=!VZvm}W^78U=M0knLDqbom4bWi)7}v6Y%EufGOk38;THmABenzgu+`shfYKviW()Sd(ZjiB!`2L3^%QZ^mGhzxo z-Sa$DxC$EOo)gr_qNBw62SP>B0kZr&HP>6TlK78MYfF%j8;O$lc2nsLQ`r6kK!{6c zQ*yt^-08x#1Fi~|(A`2Y`75~zw96oO?(?8aw@F%}q~tP=Lo^NEXqi{iO44qr?Zs&? zl`7q)%JgvvwQ5a=Up7id zBB?kAXT|xPx=^OcCJJYBNu_Kh4qE608nha`GMd@75S^YwAd!}*(RNQ!kZ>_u9-*-5 zeehiQGDH`krT-_8Iae#CU&`FHQEqN2bki-PS!PogW2I-<)8_xc6j?a~qM2|hYh#^K z)>-@{L^{=G*0faBR8VN1g;mOs`Bs=qJedDwn5)Qx40Dq*t}=}a-E^Bu1*cNR?D`N0 z83?Odfn0Mv{MiE8HAx`Mw*tBL9uB123S^%%5V{AZ!hvWwH8Kv?wA|JFH@o|^IxXTt2fJE4VdRd(jW6u|;mWxu>G2G zuGKeKZk>|NLd+XdD+mKM&F2y0b`Trk7^8yggW&TZa1i`mHa=h14}-rq!=F>_Gb7>f zm(q}b3|>X*=R-L@J> zx+)hEvd|pftLU+stDQKM#f3CQ*yYuqhr}x$!;y3-Bc^y5l&BuS!7J{r9BAMbO|`tOfQe6$`7P6loWgZshSs!}>15%*8qR7VES?V2 zHJHju0^0QzW=z!;Ez@;hLj@_8=3A~YHPg&9J%GBY=IRPv);U<%7(=rsoapX!l$*DQ3SIxYPe|TX!X;{>2}}yWaX9lqJBE*&lx_T zA<^})xat#F!??=tZ9=$n$qg<~Qp1HMwZWbTZDDH9XG5xNL8h|nD<~~h+3i^)8?rE8 zo{H*>Eas=AnbJiTJHHw}$nD~QuqT%sFR`?cNHRrgPoZPE#H+wG?H&JA9OWiAxSHHD ziN-qn(p`~S>~|W=8`S1!Pb^*A5ltn{T@~!0)b@0CPE^5dNv*OsHJJzmEI#cE1v*hZ z?d(M3O9Q= zE?%FxmuWwI|EhI++;@6)chO@#y}jP6ymj7M_rC4-Ui#UlcK3|!JA^CGg^rJ7gbp|5 zVc#du};cb)>Sxl0}M zsG#B-z;1E9g!4hXnGbu{A_#XGl-)bL54-SONi&RicejC^vX+~%xfbp+22u%u4K&)2 zOGfXCK!ns7ya#8<=VTq}ZtRaGmr5%TMn^aq5%fq-K9|$ev1GI-7U|OI-Yvr>3fpSt zcAlX&;K+OK?ZED6UnCZ{Tgd%8D^l8&GrIf2J=k+ZFb1@(fTXN5oiIA%ed&Ji?OTIJ zyswLzos3XCiAXZtXSOoO?qaT8zf?TlNBf*N8~x$Lo=8G=yqHj1jTB4@sqB}D>r4%* z?8^N{nOSFjby>V{b}s@v3a+*pxLW8+yKvIah&IAo6SSHZ9(#c5IQc4!xbK&l2uh~t) zoqN&)W^xe&h7gE4Z7GnI@&$iDtxz=4&gf~2wUbd^a^#6iT070CS!JTHHum+PMdtYg+ZyVi7g*d=%Ebj6AJxQhV`^U zzn)?JUZIsmmM>m5a&&OdCjiKjH4ZlR{Da{ z5wgfAXHgS7oYnDZ&Qrc!V)gME3a_{ExI#Y3q{pX{YbM@b_TPMbqOay0zM3ca`ngKU zI|{l)Qx3bP33hycz$ds!SdR-P=pI*X=AH{(ekw3Qm+vDu4}Z=t>tu4Zbb>$DnPpuX zw=k+JmP;hX#LMLtXV%#hKQ!GRe$p^Om!BuFZm#Q@A04m`{PA;v3I6!*-log9?h|zR*4?Jd_vI6G`L^7q%lF(9 zborLsrpx!!6Li0#y6i~>`4XF}x|{O^vL4S9FDScL6m;r4_4rcx%_REH4#I8V)9>`p zvjUGKPPqsq_w{4u223SP*vlAJagoG;2w?9v5@>8{sY!)<6ooZKfR=Y%HPJ2+AgVW< z>=l8oeKGhmA(od?vNw`QVnZ~GFd$1r`od(OumLR10`kSMK#CRz0r?JDAQ6`@X$2y^ zMt36IA2E8nV9S(gVT2Qj@IKRm@!LD02Jhj1yj&LOiNhfd0a=Ub5pR8oKxe$aKZ4b2 zpexdm?tyWbcAJ@u#=7H-hdVkFk-bde&CUqp$QPqnL@}y7O-T6P&X^4o-008^j#*qm z1x>*FdM0J+GEPuFH2`G$N>>mR9$G1eE~(5xKD*420#Abko*`NGYg|E4xPemk3nY~} zIPIGux{SRyu79^H2oCK#N+9=>3B%(qS-c7|z;h$v!=LR>xPqW)Us3{bg_47U=Xx?v zY|js(X{m(UK+B%@Hy>1i@_r^%WKr&z6uW@YULx1e`-hiRz)O@S$Ip5U_drg2lT7pe zq)q{&)R|yH{E97~6j7*b&-$dI z)_bUjsZ*fLGJ$oZy6j~bgx&s$f}j*Xq3lN_l{u6sl;QoL?e;vT4X8lpIVxm3_V|#) zp7+o1r^ce-DOpx=xDzKJrTRJlC#D2Jq5moS&yvtqIDgD~6o}oP-#h2`(doILy*TTC z5<SMWy7gwEqcosqe8p@2B(k1N@x;_fM{$`@t_@YpOYgNidF0^_%KqOR7oby@-j{ds$o_FP@h#>Hlb9$@ z-TF!FU#2$`QRJ(VCMu6k`?bp6+0MV$O(UI8jPk+u3`5FZi7oKtd=s6?9oJ#cinI^S zzL-bw`NbynstK&4Q@0cnM~G7QF*R|89R?0k*6m2KwBNGKtnxuMjxPX7V;T2*o`31| z=Bj^mjcNb7!@0A5Q!Dq;P0_i66NGMz*1XQK3*Cr>*Du-pJnKEEY<|A={!=#JBX}K= z%`XtVKFH>K1<(80{6fL=c{abudJig_Kk+*dlDjc;^Sqj^Kh=6Nl+DlaeTei^jCn0+ zSCzOMuUPQBWf!{f1q#nc*?e5wJkQt!ZhWc0^M{R%_-9}~*llM?+>N)&c)Yg@-D0No zT|_p&%zED}n_n(?+|A}^zMskFJ6AQP)NSqK+qEhGYB0)XgFnZ?VLxi{aY)@_u7kt+ z`Xv0TC-CX|$rEqVu%A`vGR?)OVOYzNe~QEEX{lc-_7?}GJs)U2ldHWX&y^oq zztD?6&VD-vu9l;J{#NNbj$ZZ1HBY9jq<6kJ01?Nk$l$5z$WnZ3~6C> zAEV>-8)rX{f}iUIjFCoK3W@v#Esaf#Q*zX6}}rmMU~l)<}_&(;}OhQyYjAD(ddr~6>?<9tRkU+MFD zi#^pUeZCLjw;>xOpS3b@Gx&Ojw6Kmz^iz}Y4}(wnamI81B>InopR3+)PNM&kivK$b8^~5BE}ueZqW1)Rpe< z-v<*LB|kaLQZ@y8`_Mv#-`Q{Dp5YM%DBqxDatP$ zgF*f%kv>G6@S&qqZ1?Q=%G2md#*N-^tc$(?rCmvEw8sqER%5r+B>9wX^6gx?Yp1ef z`?k{j=F+x3n~96QrOfq(DeWR=w5W0BUUD|iF!1H7bslZsfo9ZX1@yHm_rI)fU8yP7 PzPD_gxrLaqr>g%KIgv;l literal 0 HcmV?d00001 diff --git a/tools/psl.c b/tools/psl.c new file mode 100644 index 0000000..ff57435 --- /dev/null +++ b/tools/psl.c @@ -0,0 +1,98 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + * + * This file is part of libpsl. + * + * Using the libpsl functions via command line + * + * Changelog + * 11.04.2014 Tim Ruehsen created + * + */ + +#if HAVE_CONFIG_H +# include +#endif + +#include +#include +#include + +static void usage(int err) +{ + fprintf(stderr, "\n"); + fprintf(stderr, "Usage: psl [options]\n"); + fprintf(stderr, "\n"); + fprintf(stderr, "Options:\n"); + fprintf(stderr, " --is-public check if domains are public suffixes or not\n"); + fprintf(stderr, " --print-unreg-domain print the longest publix suffix part\n"); + fprintf(stderr, " --print-reg-domain print the shortest private suffix part\n"); + fprintf(stderr, "\n"); + + exit(err); +} + +int main(int argc, const char *const *argv) +{ + int mode = 1; + const char *const *arg; + const psl_ctx_t *psl = psl_builtin(); + + if (!psl) { + fprintf(stderr, "No PSL builtin data available\n"); + exit(2); + } + + for (arg = argv + 1; arg < argv + argc; arg++) { + if (!strncmp(*arg, "--", 2)) { + if (!strcmp(*arg, "--is-public")) + mode = 1; + else if (!strcmp(*arg, "--print-unreg-domain")) + mode = 2; + else if (!strcmp(*arg, "--print-reg-domain")) + mode = 3; + else if (!strcmp(*arg, "--")) { + arg++; + break; + } + else { + fprintf(stderr, "Unknown option '%s'\n", *arg); + usage(1); + } + } else + break; + } + + if (mode == 1) { + for (; arg < argv + argc; arg++) + printf("%s: %d\n", *arg, psl_is_public(psl, *arg)); + } + else if (mode == 2) { + for (; arg < argv + argc; arg++) + printf("%s: %s\n", *arg, psl_unregistrable_domain(psl, *arg)); + } + else if (mode == 3) { + for (; arg < argv + argc; arg++) + printf("%s: %s\n", *arg, psl_registrable_domain(psl, *arg)); + } + + return 0; +} diff --git a/tools/psl.o b/tools/psl.o new file mode 100644 index 0000000000000000000000000000000000000000..c17f25cdd9873062f36c84f908e1200e8329f1c8 GIT binary patch literal 18736 zcmd^G3vgV;mF+j9&sef08G$Y1-y>M^2WDj1#`teZ7?41ik1b<9_AnaFNQ0#rF&|qt zBoKd=2;;0FBw=AOBnu7@oWN$YcAS7sfDI{@@QF#Fk|h+WEFXtWz{^7M^WNKi?@Z6t zOai7-wN=|ydav*4)2DCW?$_`2d!uX5>^RHgIEs^_mZ+S?s8Y?7th~-PtMRH%El;=? ztmtsNGO@mNEbexP(jhmrJ{0QAQ_9N{h4?&(&IN(^Bd6RZn`(( z_9fyykyP4<4!TCv9S9|}f;qinYOb5pOZ6s_X)TzHtq-Lm`HFtOvN{d<;f`j(Tx+T` zzAm2F5O)&;=~yD}POVQ(tx%n*P)}rmJCN#gFEa(HH5KZ7O-U`NQ1z(=ZhbeMQx&RZ zRZC|__)je>JKKidUGX{wt8LrC_MykCx28Oh-?7Gb&W^sfRNMT+)mwfErTcKSTlOQd zgjvb%cg9eL2(0B!Sv;65+1 zwmjy8W>WsP?X?|4zwQ|NIn+C{J2Nq-@e-JOkf*%>a`Km3-gnX~x4iGk%&DHd4BEar zs?TlPXLk&}hE&t1?JM4I8+!B+^yUEepy{B&LE_M5Yg#UES<`f|W$j}-_hFabxa#Q9 zqv*;Xb{5bfnWlqUC@b^A;k{VppZ{w6vf`mPu7>8WTi~yr+^nj%{tg^y+rAi%5Vz+Z zL1%Te+2ie}VbKAc+0?aB2<*rBYqky-6&2x*D8w}~7ZfjQFJ5$Rp(4?EE?n1ba{SfX z#$vBGc~tebQ5bF*sy7-xfy{R8i0;LHTt#%Jbbk)u;ON{#(1Q{%l=5MK3NAs2E3Bcs zhvwkXSB^$WNhvG?%Io;h@_IDrEiosj%z&l(@}sbFKytuP{c1 z^JOr4jWN+dn@U-#pC^i0JD=g!&1|;XaOF((Ti&W%34Vi3gUemEziLtpbFE%*EI6Y1oXZcgG zJWe>Cit<|1(-C;i^nZocTx>1BrScJG6c!Y7b4D2~x<%ZqH}*ALLf)A_v5NNDS+3LI&_Asq3lY`{!|S>8=iCn9(mHQ+76E}seo3hF>Y4o1Wj44=oi#uZ+>P`(iT!6-E#roE#QeG9=jStT=c^?6vwLX|Vx zsjsT7^j3|p>Kr@Tsex6^Ge&u<7E~^*Y==9UcZ#a4Tmt3rkVaW(th5?Xh$xD2& zT4Qor9*Du%Iaz5m>%b%#G{;psonz616LU1UXeK&MwJ5r}@)F(ngi_U1i~Na0n;i&r z6V#Y=!)8Z$>TYTUcPFC^T%JgS3rn*B{sOR-VV}$uv9T$+#+Hu5u+-=GXOZqmL#ZB$ zskyRv6$Z_ePRqijQ$Yh|rw5cBToj&>WkDdZD$X5|9Mvfh<)}3hDp5zAPMmXpF*$0b zEO*o@33*j&aF`S}uyPn$pte-=sHOtdaHo}*9Q8Ma3VD&&J{XQ#A)$hy_#C{?QT5Zh zBD+v9u$K>K6@Uti$S{@^@+iGr=E0u_%M${_qm~VuPM;PVuGLyVYd9wBRjAx;S(Zur zZ^iYfS;KsnZ9D2h33*gXLNM=#(Ilr*`TSR@TO8lf6DJI7Ru?)&3P-4`Ji2J|uzIqQ z&l0Lnb3K0+z5nanQz2OJ`~wM|3arE%TMPPsk=+k(zC7v`=&R#=STzcc`n^1$PGcsO zY>dUjeVOh^={O@V%k-{ zG}63@UC_X|Cn0yC=LHxK#K!FMEe0pNV}b0w+_%e90b`}#@kdoKn21J$X%%cgKbVY! zx`PAhqzZ<6;j4#A#WqEt2qF}u?r^n&*Nw4AML>Yh5Rz1;&x8QJ zd`P$S98UD78dJ$|BTN^FRAV}k=!3Q)oc*EXx=2z_&bZ18Ni-XL=w~F+q zK^hqy9z8UO$5KJ_wF!m;heD4@e$cV~RA_y~j7+dE5r!V>u)v`8C%Pl(Rw9#DXD{z` z&v4;#3(Ru;{pn0RvbZM_k0jyammfM1?p+L*uYc|w*Wcs%&j$|FTMQTR{P`+_pL+Bp z!Rt(S+4zZZj%LRP+oo=u)!uO?4oj06mVpEW%0wxZ3y_`~rb~J-*QfPNpdrx`k--=Y zQjo%^hfmki9Sd^kQoxBv29XJs=gzTbxt_4uAnAz_jP@gSAk^JtP7o=yW}DL-b2?20 zGx1C+(me|!h;KaM-Q;>R#X{kAnE?}V@Bx-4n%2MwEBuaV#Yj7#&OYOe1@5$Ek*-)M zZU({MG=I9u;VJ|#Rn18`n=Vq$V2v|=WVvrAEOUzhpFm~$2Gm+r^Hoo4RoxEnnN{wM zB`sCc{;qUc)yyqrp7$!MrnOYLPzIc3RlZi(ZYmW>ZS050!a1EZ=U&dGF=zq&G^zq! zsgw#N6KEi_DKIbJYyvaV7Ybk~oyPLtO|z zeB{Va&G}~5Lav=o0t2z^(<>D=(^;NmeSD{aj~~ZcA0J{+6W$!Bw`sY(##_w4o$cL#Kp?410aI+qk+NK%EM}@eJ$2Qo1enZ{`82e&s znB(%y4P#60mjRX- zG9whIF(R;%i{n$s|66fjA(`)A#(!KVr2jVoCN;6%)%pt#eDy=aN|Jjcloe{D8b_8$ifH91a@)u|mTK-wx625z6v&j6=T{io?>V*PAJ z>v26kzex3Up>;USHzNzfEZ1ZBEyBb!SAVAZtC>h=?mxq?L#9yt>I%dUrv%sMxMEC2 zh_rMYy<2}zTe*YuA8qNoX6yIV@Wq!3@q^lN@F&~H<)u*j`*2~0bb~o%kFx!2fyf}Q zM7@1fz}oZ<8wJDi7VZi7SI9&3yHMm2FD=w8;9@Z%mNy%f{AynF(K{am%d7MgO95Q} zQlkJ)i?As1ns3(vMSXZRTp)SPXY64mUi0<30M1WqxuSB&!k6?c!1uS2fY6eiEyE1X z(?WP-5qwb*9Pg8bbizgOD~sTRMer{b!M7E`X|t}Mg7E>JHMmn)9`NAMFMO!Kf1QU(0wMop=ZMsqmVPYZks*?6DA z#5#OM{ocU(%EQ^h?f!{y=CkgH79W39!TeKbVL<K+7CNVsKYJ`b-ny9oFBWd=@O>5gi)$ZbOw!#T4?cYomLCCbuJV-Pm<0mi*M_!v2a_b$HMJ+US;9_>lGpD`hhkLKl6AD6mt9|@fAzxW)DiTxQM{bvoF z#eYNgZw1cl|LX#Og6#hm_`9Ufk7C?zhUU*m8Xv~*BfC!Ed>z&coaezTfgdJ6mJ0l3 zvO5I6ocy^&;9kO`0>>XvVoC`duQ5#537p4?=P&p79pc?4_%&pIN8s-ndByh&*14Db zc~tP{68{$h|0CJI5%>hMUl;foio?4CpGdfe+Gl^J5MC|tQwX0R@L7cOdjQt?fZDUy zb6g+h6Mvb|iI9D+z}J$_#RA7)_FxJNobRs*fo~@Jdjh|c^z~2kL67}^iFl6){x!t^ zg}}EG&h=QQhWh(Q!Pm~i;Uj@FpMSx}I((lyiS!sBAe+Yr-QYD0+h`U1b!2x~IBIda z(86)N+R47m!jaGG^%WM5{6o~AD=i%P2jLIXMuA^Rar=tEn@MMfh4VcQb-rQY*zQK+ ze@o!6A{o+mEgb#%KJg#2aMYiPWJvoh+}3|W;H>|Qg`<8o*)Lc)>R$nWm|nJUTmKD# zv;GkaM|~cTj|t~-e@&~yzKpiM$i=^}5pN>l%;ydGYzxO{41RuYwQ!tQjWk|sg#H22 z?-Km8h(BoIsNV;FnD{*=`_n`GZwvjyr2idD2mQI0_zzpS?f;XO4#s~cHVWxE3%BF{ zTY+=@|I@-To@2>AY~i;4pDi5qo8b@B2NsSx3&}oe;mDst{TfkyBJ;TSi5Zd+^N7-!~h6#DSnJe@Wd;s2Ase@gBCxCs6$!m+RT zw}9+d1phxs=WPo||2wod@LNVSAR!I?Xa1=I|1Z*6Rs;_c&i+KHy_n$h7xzsTj@d5w z!}K)^M}L_AJ;GW4X42m$_)|PuM*ZBvk?I;){ocY+pZT8%{YOaO{FhBGK1)e|99D*e zVL>WC2hX-}%rbvD;q3pLq`zA5FD3n$g(H>sWm_y9v&_F+;Oj_dpTIMOKP&K?34hDN z$wg=d?!Oj}S^P0Frcpj~<~SG&egJCW$mch4jRJp-Dx58FUO$5Z=XGy`!1?+A0m3oP zxIV;a-tHB+-mpXYPX*4;0Z$7Y|CjHn2;^3oACb#=TLswM-4-&sY;9H1)s=&Kx zJzp;HpODTK0-r%T>jds6|F;QzFX49zocC!Ymt7y&rZ8l2d?${H#~EdD+!@C`J0`}L z8&fsVS>geK^Z4?*#eAY=e|*b0w;L2XjQ0q98rjJr_$Gm`BK!Ix_}>ehudh3b;P(ie z`+HE}WM3FvZz%Jdtn@otZzG>`?+g zk8IXs{zZhlf_t#jA z{&U=Ue=YHkg-#dQ=096=>jvXBwEnRUKi^I)g4YY2$DQ|2ti$j1_<{nHxLwBaA9^q`{s7rG7Qt^3IL~L^&#?}#4_wYTum6t;ox6=2 z>M4OgO8Bn@&i(zJzC9*S`vw16n)e5a;P`*Qm{^DX;r%P) z-0nMq&+UF};W)m$pXGfj>+pVd5^V+;XC2;mGS2sv)*}4#EF9bA=Q-XlvJUSz;}#$1 zC%4P{YviL&7yM!3eIe`c{_(aVI`>#OW?dA4#QQ(i;eF!2S$x}{=PVqvydUI!9_#Ra z@Ewb9>wIkCn0*qRfrNd;g!>DA_C}k33&8q_*@^L10=X{ZZG=mmy9vi{bC{U_ePd?d z|1r)w^QfG0e$H Date: Fri, 11 Apr 2014 18:50:01 +0200 Subject: [PATCH 065/104] removed accidentially added blobs --- tools/.deps/psl.Po | 126 --------------------------------------------- tools/psl | Bin 16343 -> 0 bytes tools/psl.o | Bin 18736 -> 0 bytes 3 files changed, 126 deletions(-) delete mode 100644 tools/.deps/psl.Po delete mode 100755 tools/psl delete mode 100644 tools/psl.o diff --git a/tools/.deps/psl.Po b/tools/.deps/psl.Po deleted file mode 100644 index 367480e..0000000 --- a/tools/.deps/psl.Po +++ /dev/null @@ -1,126 +0,0 @@ -psl.o: psl.c /usr/include/stdc-predef.h ../config.h /usr/include/stdlib.h \ - /usr/include/features.h /usr/include/x86_64-linux-gnu/sys/cdefs.h \ - /usr/include/x86_64-linux-gnu/bits/wordsize.h \ - /usr/include/x86_64-linux-gnu/gnu/stubs.h \ - /usr/include/x86_64-linux-gnu/gnu/stubs-64.h \ - /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h \ - /usr/include/x86_64-linux-gnu/bits/waitflags.h \ - /usr/include/x86_64-linux-gnu/bits/waitstatus.h /usr/include/endian.h \ - /usr/include/x86_64-linux-gnu/bits/endian.h \ - /usr/include/x86_64-linux-gnu/bits/byteswap.h \ - /usr/include/x86_64-linux-gnu/bits/types.h \ - /usr/include/x86_64-linux-gnu/bits/typesizes.h \ - /usr/include/x86_64-linux-gnu/bits/byteswap-16.h /usr/include/xlocale.h \ - /usr/include/x86_64-linux-gnu/sys/types.h /usr/include/time.h \ - /usr/include/x86_64-linux-gnu/sys/select.h \ - /usr/include/x86_64-linux-gnu/bits/select.h \ - /usr/include/x86_64-linux-gnu/bits/sigset.h \ - /usr/include/x86_64-linux-gnu/bits/time.h \ - /usr/include/x86_64-linux-gnu/bits/select2.h \ - /usr/include/x86_64-linux-gnu/sys/sysmacros.h \ - /usr/include/x86_64-linux-gnu/bits/pthreadtypes.h /usr/include/alloca.h \ - /usr/include/x86_64-linux-gnu/bits/stdlib-bsearch.h \ - /usr/include/x86_64-linux-gnu/bits/stdlib-float.h \ - /usr/include/x86_64-linux-gnu/bits/stdlib.h /usr/include/string.h \ - /usr/include/x86_64-linux-gnu/bits/string.h \ - /usr/include/x86_64-linux-gnu/bits/string2.h \ - /usr/include/x86_64-linux-gnu/bits/string3.h ../include/libpsl.h \ - /usr/include/stdio.h /usr/include/libio.h /usr/include/_G_config.h \ - /usr/include/wchar.h /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stdarg.h \ - /usr/include/x86_64-linux-gnu/bits/stdio_lim.h \ - /usr/include/x86_64-linux-gnu/bits/sys_errlist.h \ - /usr/include/x86_64-linux-gnu/bits/stdio.h \ - /usr/include/x86_64-linux-gnu/bits/stdio2.h \ - /usr/include/x86_64-linux-gnu/bits/timex.h - -/usr/include/stdc-predef.h: - -../config.h: - -/usr/include/stdlib.h: - -/usr/include/features.h: - -/usr/include/x86_64-linux-gnu/sys/cdefs.h: - -/usr/include/x86_64-linux-gnu/bits/wordsize.h: - -/usr/include/x86_64-linux-gnu/gnu/stubs.h: - -/usr/include/x86_64-linux-gnu/gnu/stubs-64.h: - -/usr/lib/gcc/x86_64-linux-gnu/4.8/include/stddef.h: - -/usr/include/x86_64-linux-gnu/bits/waitflags.h: - -/usr/include/x86_64-linux-gnu/bits/waitstatus.h: - -/usr/include/endian.h: - -/usr/include/x86_64-linux-gnu/bits/endian.h: - -/usr/include/x86_64-linux-gnu/bits/byteswap.h: - -/usr/include/x86_64-linux-gnu/bits/types.h: - -/usr/include/x86_64-linux-gnu/bits/typesizes.h: - -/usr/include/x86_64-linux-gnu/bits/byteswap-16.h: - -/usr/include/xlocale.h: - -/usr/include/x86_64-linux-gnu/sys/types.h: - -/usr/include/time.h: - -/usr/include/x86_64-linux-gnu/sys/select.h: - -/usr/include/x86_64-linux-gnu/bits/select.h: - -/usr/include/x86_64-linux-gnu/bits/sigset.h: - -/usr/include/x86_64-linux-gnu/bits/time.h: - -/usr/include/x86_64-linux-gnu/bits/select2.h: - -/usr/include/x86_64-linux-gnu/sys/sysmacros.h: - -/usr/include/x86_64-linux-gnu/bits/pthreadtypes.h: - -/usr/include/alloca.h: - -/usr/include/x86_64-linux-gnu/bits/stdlib-bsearch.h: - -/usr/include/x86_64-linux-gnu/bits/stdlib-float.h: - -/usr/include/x86_64-linux-gnu/bits/stdlib.h: - -/usr/include/string.h: - -/usr/include/x86_64-linux-gnu/bits/string.h: - -/usr/include/x86_64-linux-gnu/bits/string2.h: - -/usr/include/x86_64-linux-gnu/bits/string3.h: - -../include/libpsl.h: - -/usr/include/stdio.h: - -/usr/include/libio.h: - -/usr/include/_G_config.h: - -/usr/include/wchar.h: - -/usr/lib/gcc/x86_64-linux-gnu/4.8/include/stdarg.h: - -/usr/include/x86_64-linux-gnu/bits/stdio_lim.h: - -/usr/include/x86_64-linux-gnu/bits/sys_errlist.h: - -/usr/include/x86_64-linux-gnu/bits/stdio.h: - -/usr/include/x86_64-linux-gnu/bits/stdio2.h: - -/usr/include/x86_64-linux-gnu/bits/timex.h: diff --git a/tools/psl b/tools/psl deleted file mode 100755 index 18f12f2784b273108f31c411671509ef31d55735..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 16343 zcmc&*eRxz?nLl?XnM^V;NdQR#q1*x?K$}TI1M=NT2z1g0N+D@cfy*SBBvX=^bUtWk z{XharL@Kr2ZJ(-z_1W##-D<1Pvn}FN2-0oU?XJ~Ytyb5r7EM5HD=nhcW`FOw=e=|1 zX5z;`yJw!c@ArG&^WJmMIrpA>KeC~9qf65S7nfKkh^qDlCCfYtQF5&$6>o*`ilC?x z^MnVSeEfq_LeRt*jwsg*y-Lpwm`AuvF?)a~}|GD^cjUPHO zciC4-MyG`HfB3AfiM*hXc%}H0eba`Qe)YAVfBJ(j-?6>&5s>ZWGb9U8HO-!+-q!Adh7eb$z&vHn&&p|j>e(_bs6cHsgNM2MpwK)9E}MB(v%cO zlg2>0qc7Sisg88CF9kUn<}gF}ofPn8H6*#=(e%C2hyodkxZd& z3DJFLBATKGk-ch=8AhZloC-tGkxZI=VMJnGqBfmO)W-XhwaG*$UCRhZq5(v(Ai54Zn1i5>Ak=RrBYP&?QE|qj7*rsv=rQSNukK{)I@-BHAUk zf~MkC{e;FM`*_6tgvRdaq^QawrC$JAxsZ}N8{dq|G+1Hd^Vmm{pp9>zA3`?1J)SK# zKGn@-yN%C%lw`YY{32Ec?6vWIHvWK(KgGr$wDIi&TxjT5-q4Z!bdfGXhelHQqn$7c z4L##MXEO#mzN`Z(<5gWa7S9a=Bf6KA#$Fvqsd_)rl*+LeB)yer8n(wqBz+^%l+v*$ zB;77_(d>c;jrHMU#Qvxr8Y5Mx^;J&kBe z*;o*CfL4<-?0=;%)GjYs=_EkY|V77x7$tCQ4} z##19xgQD@P)QrZjQ9c`=eh(c_Q(wBct~*dX_aHDK&U_n+q2ceh4v$@Wr1&5WcV}K> z)|+4*a>LAmXreORd;?R!=wqhn?E;8eERl|Q zM(;I6KO)ghmS})Pd!=aE8Hmh4^T9CFbrZ1tcWE`nhsvnccezCIp{bN^oT@Rifm1(D zc8{X^Q&+Ly(fQ)3w})zI{0j|T@rQ=b-|5uYVq9t#5+k8>z}>i zhJI-H>!Gt3t`BK1guZhn4dYX!7&`Rb)LT14!_S)bKU4Pm*Piw07?w+MeVrfuFdDk^ z+59horCmBl=Q}jCwmc6CsjI9IX0)Fv!KxPe<1g%#eWQ4;7)!zwu<^FWVO+E=!GQ^$oX<5RcXn+}67IeEln-vww2a2sXOx9c1>_CjGklEC z_%U*>q0go-(7g}?SUpZOV$kjoOX_{Sulj&mPb8T#oet`zQ+)u-5_XvJhebn9Z@QlGm+Xxv zQqo}}x;LDPO!Qb&BWw)^J~3E}nW<5<$M(eHcgFPiKq?xK>D5)q>LSse4EID<=~zPO zA20>Uokikisghh(B&w3DbWA)ZOcn`kwtICQvP6?b>i6U09|Uap!}$0=0$#+m&qM4LUOzIMKUZK^!yB03OUU zX^-Rzal&QNe7)^b@zJ@YQf>l4vzNWlf`Cd1hF>atj=ajA`ysCt%cPbN`#nL4F@KAq zIc~Z>s1%yOsvnnwY>%$=G1^L+^ZkaRFXso9<^K)f=Niu1o-Hur_`ZU_Q1G&X9zt=!VZvm}W^78U=M0knLDqbom4bWi)7}v6Y%EufGOk38;THmABenzgu+`shfYKviW()Sd(ZjiB!`2L3^%QZ^mGhzxo z-Sa$DxC$EOo)gr_qNBw62SP>B0kZr&HP>6TlK78MYfF%j8;O$lc2nsLQ`r6kK!{6c zQ*yt^-08x#1Fi~|(A`2Y`75~zw96oO?(?8aw@F%}q~tP=Lo^NEXqi{iO44qr?Zs&? zl`7q)%JgvvwQ5a=Up7id zBB?kAXT|xPx=^OcCJJYBNu_Kh4qE608nha`GMd@75S^YwAd!}*(RNQ!kZ>_u9-*-5 zeehiQGDH`krT-_8Iae#CU&`FHQEqN2bki-PS!PogW2I-<)8_xc6j?a~qM2|hYh#^K z)>-@{L^{=G*0faBR8VN1g;mOs`Bs=qJedDwn5)Qx40Dq*t}=}a-E^Bu1*cNR?D`N0 z83?Odfn0Mv{MiE8HAx`Mw*tBL9uB123S^%%5V{AZ!hvWwH8Kv?wA|JFH@o|^IxXTt2fJE4VdRd(jW6u|;mWxu>G2G zuGKeKZk>|NLd+XdD+mKM&F2y0b`Trk7^8yggW&TZa1i`mHa=h14}-rq!=F>_Gb7>f zm(q}b3|>X*=R-L@J> zx+)hEvd|pftLU+stDQKM#f3CQ*yYuqhr}x$!;y3-Bc^y5l&BuS!7J{r9BAMbO|`tOfQe6$`7P6loWgZshSs!}>15%*8qR7VES?V2 zHJHju0^0QzW=z!;Ez@;hLj@_8=3A~YHPg&9J%GBY=IRPv);U<%7(=rsoapX!l$*DQ3SIxYPe|TX!X;{>2}}yWaX9lqJBE*&lx_T zA<^})xat#F!??=tZ9=$n$qg<~Qp1HMwZWbTZDDH9XG5xNL8h|nD<~~h+3i^)8?rE8 zo{H*>Eas=AnbJiTJHHw}$nD~QuqT%sFR`?cNHRrgPoZPE#H+wG?H&JA9OWiAxSHHD ziN-qn(p`~S>~|W=8`S1!Pb^*A5ltn{T@~!0)b@0CPE^5dNv*OsHJJzmEI#cE1v*hZ z?d(M3O9Q= zE?%FxmuWwI|EhI++;@6)chO@#y}jP6ymj7M_rC4-Ui#UlcK3|!JA^CGg^rJ7gbp|5 zVc#du};cb)>Sxl0}M zsG#B-z;1E9g!4hXnGbu{A_#XGl-)bL54-SONi&RicejC^vX+~%xfbp+22u%u4K&)2 zOGfXCK!ns7ya#8<=VTq}ZtRaGmr5%TMn^aq5%fq-K9|$ev1GI-7U|OI-Yvr>3fpSt zcAlX&;K+OK?ZED6UnCZ{Tgd%8D^l8&GrIf2J=k+ZFb1@(fTXN5oiIA%ed&Ji?OTIJ zyswLzos3XCiAXZtXSOoO?qaT8zf?TlNBf*N8~x$Lo=8G=yqHj1jTB4@sqB}D>r4%* z?8^N{nOSFjby>V{b}s@v3a+*pxLW8+yKvIah&IAo6SSHZ9(#c5IQc4!xbK&l2uh~t) zoqN&)W^xe&h7gE4Z7GnI@&$iDtxz=4&gf~2wUbd^a^#6iT070CS!JTHHum+PMdtYg+ZyVi7g*d=%Ebj6AJxQhV`^U zzn)?JUZIsmmM>m5a&&OdCjiKjH4ZlR{Da{ z5wgfAXHgS7oYnDZ&Qrc!V)gME3a_{ExI#Y3q{pX{YbM@b_TPMbqOay0zM3ca`ngKU zI|{l)Qx3bP33hycz$ds!SdR-P=pI*X=AH{(ekw3Qm+vDu4}Z=t>tu4Zbb>$DnPpuX zw=k+JmP;hX#LMLtXV%#hKQ!GRe$p^Om!BuFZm#Q@A04m`{PA;v3I6!*-log9?h|zR*4?Jd_vI6G`L^7q%lF(9 zborLsrpx!!6Li0#y6i~>`4XF}x|{O^vL4S9FDScL6m;r4_4rcx%_REH4#I8V)9>`p zvjUGKPPqsq_w{4u223SP*vlAJagoG;2w?9v5@>8{sY!)<6ooZKfR=Y%HPJ2+AgVW< z>=l8oeKGhmA(od?vNw`QVnZ~GFd$1r`od(OumLR10`kSMK#CRz0r?JDAQ6`@X$2y^ zMt36IA2E8nV9S(gVT2Qj@IKRm@!LD02Jhj1yj&LOiNhfd0a=Ub5pR8oKxe$aKZ4b2 zpexdm?tyWbcAJ@u#=7H-hdVkFk-bde&CUqp$QPqnL@}y7O-T6P&X^4o-008^j#*qm z1x>*FdM0J+GEPuFH2`G$N>>mR9$G1eE~(5xKD*420#Abko*`NGYg|E4xPemk3nY~} zIPIGux{SRyu79^H2oCK#N+9=>3B%(qS-c7|z;h$v!=LR>xPqW)Us3{bg_47U=Xx?v zY|js(X{m(UK+B%@Hy>1i@_r^%WKr&z6uW@YULx1e`-hiRz)O@S$Ip5U_drg2lT7pe zq)q{&)R|yH{E97~6j7*b&-$dI z)_bUjsZ*fLGJ$oZy6j~bgx&s$f}j*Xq3lN_l{u6sl;QoL?e;vT4X8lpIVxm3_V|#) zp7+o1r^ce-DOpx=xDzKJrTRJlC#D2Jq5moS&yvtqIDgD~6o}oP-#h2`(doILy*TTC z5<SMWy7gwEqcosqe8p@2B(k1N@x;_fM{$`@t_@YpOYgNidF0^_%KqOR7oby@-j{ds$o_FP@h#>Hlb9$@ z-TF!FU#2$`QRJ(VCMu6k`?bp6+0MV$O(UI8jPk+u3`5FZi7oKtd=s6?9oJ#cinI^S zzL-bw`NbynstK&4Q@0cnM~G7QF*R|89R?0k*6m2KwBNGKtnxuMjxPX7V;T2*o`31| z=Bj^mjcNb7!@0A5Q!Dq;P0_i66NGMz*1XQK3*Cr>*Du-pJnKEEY<|A={!=#JBX}K= z%`XtVKFH>K1<(80{6fL=c{abudJig_Kk+*dlDjc;^Sqj^Kh=6Nl+DlaeTei^jCn0+ zSCzOMuUPQBWf!{f1q#nc*?e5wJkQt!ZhWc0^M{R%_-9}~*llM?+>N)&c)Yg@-D0No zT|_p&%zED}n_n(?+|A}^zMskFJ6AQP)NSqK+qEhGYB0)XgFnZ?VLxi{aY)@_u7kt+ z`Xv0TC-CX|$rEqVu%A`vGR?)OVOYzNe~QEEX{lc-_7?}GJs)U2ldHWX&y^oq zztD?6&VD-vu9l;J{#NNbj$ZZ1HBY9jq<6kJ01?Nk$l$5z$WnZ3~6C> zAEV>-8)rX{f}iUIjFCoK3W@v#Esaf#Q*zX6}}rmMU~l)<}_&(;}OhQyYjAD(ddr~6>?<9tRkU+MFD zi#^pUeZCLjw;>xOpS3b@Gx&Ojw6Kmz^iz}Y4}(wnamI81B>InopR3+)PNM&kivK$b8^~5BE}ueZqW1)Rpe< z-v<*LB|kaLQZ@y8`_Mv#-`Q{Dp5YM%DBqxDatP$ zgF*f%kv>G6@S&qqZ1?Q=%G2md#*N-^tc$(?rCmvEw8sqER%5r+B>9wX^6gx?Yp1ef z`?k{j=F+x3n~96QrOfq(DeWR=w5W0BUUD|iF!1H7bslZsfo9ZX1@yHm_rI)fU8yP7 PzPD_gxrLaqr>g%KIgv;l diff --git a/tools/psl.o b/tools/psl.o deleted file mode 100644 index c17f25cdd9873062f36c84f908e1200e8329f1c8..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 18736 zcmd^G3vgV;mF+j9&sef08G$Y1-y>M^2WDj1#`teZ7?41ik1b<9_AnaFNQ0#rF&|qt zBoKd=2;;0FBw=AOBnu7@oWN$YcAS7sfDI{@@QF#Fk|h+WEFXtWz{^7M^WNKi?@Z6t zOai7-wN=|ydav*4)2DCW?$_`2d!uX5>^RHgIEs^_mZ+S?s8Y?7th~-PtMRH%El;=? ztmtsNGO@mNEbexP(jhmrJ{0QAQ_9N{h4?&(&IN(^Bd6RZn`(( z_9fyykyP4<4!TCv9S9|}f;qinYOb5pOZ6s_X)TzHtq-Lm`HFtOvN{d<;f`j(Tx+T` zzAm2F5O)&;=~yD}POVQ(tx%n*P)}rmJCN#gFEa(HH5KZ7O-U`NQ1z(=ZhbeMQx&RZ zRZC|__)je>JKKidUGX{wt8LrC_MykCx28Oh-?7Gb&W^sfRNMT+)mwfErTcKSTlOQd zgjvb%cg9eL2(0B!Sv;65+1 zwmjy8W>WsP?X?|4zwQ|NIn+C{J2Nq-@e-JOkf*%>a`Km3-gnX~x4iGk%&DHd4BEar zs?TlPXLk&}hE&t1?JM4I8+!B+^yUEepy{B&LE_M5Yg#UES<`f|W$j}-_hFabxa#Q9 zqv*;Xb{5bfnWlqUC@b^A;k{VppZ{w6vf`mPu7>8WTi~yr+^nj%{tg^y+rAi%5Vz+Z zL1%Te+2ie}VbKAc+0?aB2<*rBYqky-6&2x*D8w}~7ZfjQFJ5$Rp(4?EE?n1ba{SfX z#$vBGc~tebQ5bF*sy7-xfy{R8i0;LHTt#%Jbbk)u;ON{#(1Q{%l=5MK3NAs2E3Bcs zhvwkXSB^$WNhvG?%Io;h@_IDrEiosj%z&l(@}sbFKytuP{c1 z^JOr4jWN+dn@U-#pC^i0JD=g!&1|;XaOF((Ti&W%34Vi3gUemEziLtpbFE%*EI6Y1oXZcgG zJWe>Cit<|1(-C;i^nZocTx>1BrScJG6c!Y7b4D2~x<%ZqH}*ALLf)A_v5NNDS+3LI&_Asq3lY`{!|S>8=iCn9(mHQ+76E}seo3hF>Y4o1Wj44=oi#uZ+>P`(iT!6-E#roE#QeG9=jStT=c^?6vwLX|Vx zsjsT7^j3|p>Kr@Tsex6^Ge&u<7E~^*Y==9UcZ#a4Tmt3rkVaW(th5?Xh$xD2& zT4Qor9*Du%Iaz5m>%b%#G{;psonz616LU1UXeK&MwJ5r}@)F(ngi_U1i~Na0n;i&r z6V#Y=!)8Z$>TYTUcPFC^T%JgS3rn*B{sOR-VV}$uv9T$+#+Hu5u+-=GXOZqmL#ZB$ zskyRv6$Z_ePRqijQ$Yh|rw5cBToj&>WkDdZD$X5|9Mvfh<)}3hDp5zAPMmXpF*$0b zEO*o@33*j&aF`S}uyPn$pte-=sHOtdaHo}*9Q8Ma3VD&&J{XQ#A)$hy_#C{?QT5Zh zBD+v9u$K>K6@Uti$S{@^@+iGr=E0u_%M${_qm~VuPM;PVuGLyVYd9wBRjAx;S(Zur zZ^iYfS;KsnZ9D2h33*gXLNM=#(Ilr*`TSR@TO8lf6DJI7Ru?)&3P-4`Ji2J|uzIqQ z&l0Lnb3K0+z5nanQz2OJ`~wM|3arE%TMPPsk=+k(zC7v`=&R#=STzcc`n^1$PGcsO zY>dUjeVOh^={O@V%k-{ zG}63@UC_X|Cn0yC=LHxK#K!FMEe0pNV}b0w+_%e90b`}#@kdoKn21J$X%%cgKbVY! zx`PAhqzZ<6;j4#A#WqEt2qF}u?r^n&*Nw4AML>Yh5Rz1;&x8QJ zd`P$S98UD78dJ$|BTN^FRAV}k=!3Q)oc*EXx=2z_&bZ18Ni-XL=w~F+q zK^hqy9z8UO$5KJ_wF!m;heD4@e$cV~RA_y~j7+dE5r!V>u)v`8C%Pl(Rw9#DXD{z` z&v4;#3(Ru;{pn0RvbZM_k0jyammfM1?p+L*uYc|w*Wcs%&j$|FTMQTR{P`+_pL+Bp z!Rt(S+4zZZj%LRP+oo=u)!uO?4oj06mVpEW%0wxZ3y_`~rb~J-*QfPNpdrx`k--=Y zQjo%^hfmki9Sd^kQoxBv29XJs=gzTbxt_4uAnAz_jP@gSAk^JtP7o=yW}DL-b2?20 zGx1C+(me|!h;KaM-Q;>R#X{kAnE?}V@Bx-4n%2MwEBuaV#Yj7#&OYOe1@5$Ek*-)M zZU({MG=I9u;VJ|#Rn18`n=Vq$V2v|=WVvrAEOUzhpFm~$2Gm+r^Hoo4RoxEnnN{wM zB`sCc{;qUc)yyqrp7$!MrnOYLPzIc3RlZi(ZYmW>ZS050!a1EZ=U&dGF=zq&G^zq! zsgw#N6KEi_DKIbJYyvaV7Ybk~oyPLtO|z zeB{Va&G}~5Lav=o0t2z^(<>D=(^;NmeSD{aj~~ZcA0J{+6W$!Bw`sY(##_w4o$cL#Kp?410aI+qk+NK%EM}@eJ$2Qo1enZ{`82e&s znB(%y4P#60mjRX- zG9whIF(R;%i{n$s|66fjA(`)A#(!KVr2jVoCN;6%)%pt#eDy=aN|Jjcloe{D8b_8$ifH91a@)u|mTK-wx625z6v&j6=T{io?>V*PAJ z>v26kzex3Up>;USHzNzfEZ1ZBEyBb!SAVAZtC>h=?mxq?L#9yt>I%dUrv%sMxMEC2 zh_rMYy<2}zTe*YuA8qNoX6yIV@Wq!3@q^lN@F&~H<)u*j`*2~0bb~o%kFx!2fyf}Q zM7@1fz}oZ<8wJDi7VZi7SI9&3yHMm2FD=w8;9@Z%mNy%f{AynF(K{am%d7MgO95Q} zQlkJ)i?As1ns3(vMSXZRTp)SPXY64mUi0<30M1WqxuSB&!k6?c!1uS2fY6eiEyE1X z(?WP-5qwb*9Pg8bbizgOD~sTRMer{b!M7E`X|t}Mg7E>JHMmn)9`NAMFMO!Kf1QU(0wMop=ZMsqmVPYZks*?6DA z#5#OM{ocU(%EQ^h?f!{y=CkgH79W39!TeKbVL<K+7CNVsKYJ`b-ny9oFBWd=@O>5gi)$ZbOw!#T4?cYomLCCbuJV-Pm<0mi*M_!v2a_b$HMJ+US;9_>lGpD`hhkLKl6AD6mt9|@fAzxW)DiTxQM{bvoF z#eYNgZw1cl|LX#Og6#hm_`9Ufk7C?zhUU*m8Xv~*BfC!Ed>z&coaezTfgdJ6mJ0l3 zvO5I6ocy^&;9kO`0>>XvVoC`duQ5#537p4?=P&p79pc?4_%&pIN8s-ndByh&*14Db zc~tP{68{$h|0CJI5%>hMUl;foio?4CpGdfe+Gl^J5MC|tQwX0R@L7cOdjQt?fZDUy zb6g+h6Mvb|iI9D+z}J$_#RA7)_FxJNobRs*fo~@Jdjh|c^z~2kL67}^iFl6){x!t^ zg}}EG&h=QQhWh(Q!Pm~i;Uj@FpMSx}I((lyiS!sBAe+Yr-QYD0+h`U1b!2x~IBIda z(86)N+R47m!jaGG^%WM5{6o~AD=i%P2jLIXMuA^Rar=tEn@MMfh4VcQb-rQY*zQK+ ze@o!6A{o+mEgb#%KJg#2aMYiPWJvoh+}3|W;H>|Qg`<8o*)Lc)>R$nWm|nJUTmKD# zv;GkaM|~cTj|t~-e@&~yzKpiM$i=^}5pN>l%;ydGYzxO{41RuYwQ!tQjWk|sg#H22 z?-Km8h(BoIsNV;FnD{*=`_n`GZwvjyr2idD2mQI0_zzpS?f;XO4#s~cHVWxE3%BF{ zTY+=@|I@-To@2>AY~i;4pDi5qo8b@B2NsSx3&}oe;mDst{TfkyBJ;TSi5Zd+^N7-!~h6#DSnJe@Wd;s2Ase@gBCxCs6$!m+RT zw}9+d1phxs=WPo||2wod@LNVSAR!I?Xa1=I|1Z*6Rs;_c&i+KHy_n$h7xzsTj@d5w z!}K)^M}L_AJ;GW4X42m$_)|PuM*ZBvk?I;){ocY+pZT8%{YOaO{FhBGK1)e|99D*e zVL>WC2hX-}%rbvD;q3pLq`zA5FD3n$g(H>sWm_y9v&_F+;Oj_dpTIMOKP&K?34hDN z$wg=d?!Oj}S^P0Frcpj~<~SG&egJCW$mch4jRJp-Dx58FUO$5Z=XGy`!1?+A0m3oP zxIV;a-tHB+-mpXYPX*4;0Z$7Y|CjHn2;^3oACb#=TLswM-4-&sY;9H1)s=&Kx zJzp;HpODTK0-r%T>jds6|F;QzFX49zocC!Ymt7y&rZ8l2d?${H#~EdD+!@C`J0`}L z8&fsVS>geK^Z4?*#eAY=e|*b0w;L2XjQ0q98rjJr_$Gm`BK!Ix_}>ehudh3b;P(ie z`+HE}WM3FvZz%Jdtn@otZzG>`?+g zk8IXs{zZhlf_t#jA z{&U=Ue=YHkg-#dQ=096=>jvXBwEnRUKi^I)g4YY2$DQ|2ti$j1_<{nHxLwBaA9^q`{s7rG7Qt^3IL~L^&#?}#4_wYTum6t;ox6=2 z>M4OgO8Bn@&i(zJzC9*S`vw16n)e5a;P`*Qm{^DX;r%P) z-0nMq&+UF};W)m$pXGfj>+pVd5^V+;XC2;mGS2sv)*}4#EF9bA=Q-XlvJUSz;}#$1 zC%4P{YviL&7yM!3eIe`c{_(aVI`>#OW?dA4#QQ(i;eF!2S$x}{=PVqvydUI!9_#Ra z@Ewb9>wIkCn0*qRfrNd;g!>DA_C}k33&8q_*@^L10=X{ZZG=mmy9vi{bC{U_ePd?d z|1r)w^QfG0e$H Date: Fri, 11 Apr 2014 22:25:43 +0200 Subject: [PATCH 066/104] generate tar ball libpsl-... instead of psl-... --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index ffc5bbd..3c75d10 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([libpsl], [0.1], [tim.ruehsen@gmx.de], [psl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.1], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define]) From af717a0d80a0d3ba19e00f1f4473c66c2face8b3 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 11 Apr 2014 22:29:34 +0200 Subject: [PATCH 067/104] removed autogenerated file 'INSTALL' from repo --- INSTALL | 1 - 1 file changed, 1 deletion(-) delete mode 120000 INSTALL diff --git a/INSTALL b/INSTALL deleted file mode 120000 index f812f5a..0000000 --- a/INSTALL +++ /dev/null @@ -1 +0,0 @@ -/usr/share/automake-1.14/INSTALL \ No newline at end of file From b49e681d8f0e498372c211cf302db1f57ad877fb Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 12 Apr 2014 16:04:42 +0200 Subject: [PATCH 068/104] renamed psl_is_public() to psl_is_public_suffix() --- docs/libpsl/libpsl-sections.txt | 2 +- include/libpsl.h | 2 +- src/psl.c | 10 +++++----- tests/test-is-public-all.c | 26 +++++++++++++------------- tests/test-is-public-builtin.c | 4 ++-- tests/test-is-public.c | 6 +++--- tools/psl.c | 2 +- 7 files changed, 26 insertions(+), 26 deletions(-) diff --git a/docs/libpsl/libpsl-sections.txt b/docs/libpsl/libpsl-sections.txt index a795a1b..7ee9e7f 100644 --- a/docs/libpsl/libpsl-sections.txt +++ b/docs/libpsl/libpsl-sections.txt @@ -6,7 +6,7 @@ psl_free psl_load_file psl_load_fp psl_builtin -psl_is_public +psl_is_public_suffix psl_unregistrable_domain psl_registrable_domain psl_suffix_count diff --git a/include/libpsl.h b/include/libpsl.h index 706b1e8..f356583 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -50,7 +50,7 @@ psl_ctx_t * const psl_ctx_t * psl_builtin(void); int - psl_is_public(const psl_ctx_t *psl, const char *domain); + psl_is_public_suffix(const psl_ctx_t *psl, const char *domain); // returns the longest unregistrable domain within 'domain' or NULL if none found const char * psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain); diff --git a/src/psl.c b/src/psl.c index bce1e6a..8cfff91 100644 --- a/src/psl.c +++ b/src/psl.c @@ -248,7 +248,7 @@ static int _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) } /** - * psl_is_public: + * psl_is_public_suffix: * @psl: PSL context * @domain: Domain string * @@ -265,7 +265,7 @@ static int _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) * * Since: 0.1 */ -int psl_is_public(const psl_ctx_t *psl, const char *domain) +int psl_is_public_suffix(const psl_ctx_t *psl, const char *domain) { _psl_entry_t suffix, *rule; const char *p, *label_bak; @@ -369,10 +369,10 @@ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) // for being a registered domain. if (!(p = strrchr(domain, '.'))) - return psl_is_public(psl, domain) ? domain : NULL; + return psl_is_public_suffix(psl, domain) ? domain : NULL; for (ret_domain = NULL; ;) { - if (!psl_is_public(psl, p)) + if (!psl_is_public_suffix(psl, p)) return ret_domain; else if (p == domain) return domain; @@ -415,7 +415,7 @@ const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) if (!(p = strrchr(domain, '.'))) p = domain; - while ((ispublic = psl_is_public(psl, p)) && p > domain) { + while ((ispublic = psl_is_public_suffix(psl, p)) && p > domain) { // go left to next dot while (p > domain && *--p != '.') ; diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index 016a85c..a784dec 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -21,7 +21,7 @@ * * This file is part of the test suite of libpsl. * - * Test psl_is_public() for all entries in effective_tld_names.dat + * Test psl_is_public_suffix() for all entries in effective_tld_names.dat * * Changelog * 19.03.2014 Tim Ruehsen created @@ -67,38 +67,38 @@ static void test_psl(void) *linep = 0; if (*p == '!') { // an exception to a wildcard, e.g. !www.ck (wildcard is *.ck) - if ((result = psl_is_public(psl, p + 1))) { + if ((result = psl_is_public_suffix(psl, p + 1))) { failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", p, result); + printf("psl_is_public_suffix(%s)=%d (expected 0)\n", p, result); } else ok++; - if (!(result = psl_is_public(psl, strchr(p, '.') + 1))) { + if (!(result = psl_is_public_suffix(psl, strchr(p, '.') + 1))) { failed++; - printf("psl_is_public(%s)=%d (expected 1)\n", strchr(p, '.') + 1, result); + printf("psl_is_public_suffix(%s)=%d (expected 1)\n", strchr(p, '.') + 1, result); } else ok++; } else if (*p == '*') { // a wildcard, e.g. *.ck - if (!(result = psl_is_public(psl, p + 1))) { + if (!(result = psl_is_public_suffix(psl, p + 1))) { failed++; - printf("psl_is_public(%s)=%d (expected 1)\n", p + 1, result); + printf("psl_is_public_suffix(%s)=%d (expected 1)\n", p + 1, result); } else ok++; *p = 'x'; - if (!(result = psl_is_public(psl, p))) { + if (!(result = psl_is_public_suffix(psl, p))) { failed++; - printf("psl_is_public(%s)=%d (expected 1)\n", p, result); + printf("psl_is_public_suffix(%s)=%d (expected 1)\n", p, result); } else ok++; } else { - if (!(result = psl_is_public(psl, p))) { + if (!(result = psl_is_public_suffix(psl, p))) { failed++; - printf("psl_is_public(%s)=%d (expected 1)\n", p, result); + printf("psl_is_public_suffix(%s)=%d (expected 1)\n", p, result); } else ok++; snprintf(domain, sizeof(domain), "xxxx.%s", p); - if ((result = psl_is_public(psl, domain))) { + if ((result = psl_is_public_suffix(psl, domain))) { failed++; - printf("psl_is_public(%s)=%d (expected 0)\n", domain, result); + printf("psl_is_public_suffix(%s)=%d (expected 0)\n", domain, result); } else ok++; } } diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index 0a94915..7f53b63 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -78,13 +78,13 @@ static void test_psl(void) for (it = 0; it < countof(test_data); it++) { const struct test_data *t = &test_data[it]; - int result = psl_is_public(psl, t->domain); + int result = psl_is_public_suffix(psl, t->domain); if (result == t->result) { ok++; } else { failed++; - printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); + printf("psl_is_public_suffix(%s)=%d (expected %d)\n", t->domain, result, t->result); } } diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 0155804..c6dc149 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -21,7 +21,7 @@ * * This file is part of the test suite of libpsl. * - * Test case for psl_load_file(), psl_is_public(), psl_free() + * Test case for psl_load_file(), psl_is_public_suffix(), psl_free() * * Changelog * 19.03.2014 Tim Ruehsen created from libmget/cookie.c @@ -76,13 +76,13 @@ static void test_psl(void) for (it = 0; it < countof(test_data); it++) { const struct test_data *t = &test_data[it]; - int result = psl_is_public(psl, t->domain); + int result = psl_is_public_suffix(psl, t->domain); if (result == t->result) { ok++; } else { failed++; - printf("psl_is_public(%s)=%d (expected %d)\n", t->domain, result, t->result); + printf("psl_is_public_suffix(%s)=%d (expected %d)\n", t->domain, result, t->result); } } diff --git a/tools/psl.c b/tools/psl.c index ff57435..6b5d0fb 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -83,7 +83,7 @@ int main(int argc, const char *const *argv) if (mode == 1) { for (; arg < argv + argc; arg++) - printf("%s: %d\n", *arg, psl_is_public(psl, *arg)); + printf("%s: %d\n", *arg, psl_is_public_suffix(psl, *arg)); } else if (mode == 2) { for (; arg < argv + argc; arg++) From 3fd48b932881c70605b6f2bea6cdaa6a15840eb8 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 12 Apr 2014 16:12:29 +0200 Subject: [PATCH 069/104] renamed --is-public to --is-public-suffix --- tools/psl.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/psl.c b/tools/psl.c index 6b5d0fb..d97156c 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -42,7 +42,7 @@ static void usage(int err) fprintf(stderr, "Usage: psl [options]\n"); fprintf(stderr, "\n"); fprintf(stderr, "Options:\n"); - fprintf(stderr, " --is-public check if domains are public suffixes or not\n"); + fprintf(stderr, " --is-public-suffix check if domains are public suffixes or not. [default]\n"); fprintf(stderr, " --print-unreg-domain print the longest publix suffix part\n"); fprintf(stderr, " --print-reg-domain print the shortest private suffix part\n"); fprintf(stderr, "\n"); @@ -63,7 +63,7 @@ int main(int argc, const char *const *argv) for (arg = argv + 1; arg < argv + argc; arg++) { if (!strncmp(*arg, "--", 2)) { - if (!strcmp(*arg, "--is-public")) + if (!strcmp(*arg, "--is-public-suffix")) mode = 1; else if (!strcmp(*arg, "--print-unreg-domain")) mode = 2; From c93268ce25ea3bd79aad936e275ffb6f95e89b46 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 13 Apr 2014 16:35:28 +0200 Subject: [PATCH 070/104] added ./configure --with-psl-file to set PSL file path --- configure.ac | 9 +++++++++ src/Makefile.am | 5 +++-- tests/Makefile.am | 2 +- tests/test-is-public-all.c | 6 +++--- tests/test-is-public.c | 2 +- 5 files changed, 17 insertions(+), 7 deletions(-) diff --git a/configure.ac b/configure.ac index 3c75d10..d6d31f8 100644 --- a/configure.ac +++ b/configure.ac @@ -89,6 +89,14 @@ else TESTS_INFO="Valgrind testing not enabled" fi +# Check for custom PSL file +AC_ARG_WITH(psl-file, + AC_HELP_STRING([--with-psl-file=[PATH]], + [path to PSL file]), + PSL_FILE=$withval, + PSL_FILE="\$(top_srcdir)/data/effective_tld_names.dat") +AC_SUBST(PSL_FILE) + # Override the template file name of the generated .pc file, so that there # is no need to rename the template file when the API version changes. AC_CONFIG_FILES([Makefile @@ -111,5 +119,6 @@ AC_MSG_NOTICE([Summary of build options: CFlags: ${CFLAGS} ${CPPFLAGS} LDFlags: ${LDFLAGS} Builtin PSL: ${enable_builtin} + PSL File: ${PSL_FILE} Tests: ${TESTS_INFO} ]) diff --git a/src/Makefile.am b/src/Makefile.am index 05d4e13..0f70961 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -19,5 +19,6 @@ psl2c_CPPFLAGS = -I$(top_srcdir)/include -D _GNU_SOURCE #psl2c_LDADD = -lidn2 # Build rule for suffix.c -suffixes.c: $(top_srcdir)/data/effective_tld_names.dat psl2c$(EXEEXT) - ./psl2c$(EXEEXT) $(top_srcdir)/data/effective_tld_names.dat suffixes.c +# PSL_FILE can be set by ./configure --with-psl-file=[PATH] +suffixes.c: $(PSL_FILE) psl2c$(EXEEXT) + ./psl2c$(EXEEXT) "$(PSL_FILE)" suffixes.c diff --git a/tests/Makefile.am b/tests/Makefile.am index d5b9e06..e867f17 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,4 +1,4 @@ -DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" +DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" -DPSL_FILE=\"$(PSL_FILE)\" AM_CPPFLAGS = -I$(top_srcdir)/include LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index a784dec..91f6098 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -50,11 +50,11 @@ static void test_psl(void) int result; char buf[256], domain[64], *linep, *p; - psl = psl_load_file(DATADIR "/effective_tld_names.dat"); + psl = psl_load_file(PSL_FILE); // PSL_FILE can be set by ./configure --with-psl-file=[PATH] printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); - if ((fp = fopen(DATADIR "/effective_tld_names.dat", "r"))) { + if ((fp = fopen(PSL_FILE, "r"))) { while ((linep = fgets(buf, sizeof(buf), fp))) { while (isspace(*linep)) linep++; // ignore leading whitespace if (!*linep) continue; // skip empty lines @@ -105,7 +105,7 @@ static void test_psl(void) fclose(fp); } else { - printf("Failed to open %s\n", DATADIR "/effective_tld_names.dat"); + printf("Failed to open %s\n", PSL_FILE); failed++; } diff --git a/tests/test-is-public.c b/tests/test-is-public.c index c6dc149..24cb32e 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -70,7 +70,7 @@ static void test_psl(void) unsigned it; psl_ctx_t *psl; - psl = psl_load_file(DATADIR "/effective_tld_names.dat"); + psl = psl_load_file(PSL_FILE); printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); From c481e8019d7577daf5ae8e7d7e00dd80d29fae7c Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 13 Apr 2014 16:43:03 +0200 Subject: [PATCH 071/104] added ./configure --with-psl-testfile to set PSL test file path --- configure.ac | 9 +++++++++ tests/Makefile.am | 2 +- tests/test-registrable-domain.c | 6 ++---- 3 files changed, 12 insertions(+), 5 deletions(-) diff --git a/configure.ac b/configure.ac index d6d31f8..1daf685 100644 --- a/configure.ac +++ b/configure.ac @@ -97,6 +97,14 @@ AC_ARG_WITH(psl-file, PSL_FILE="\$(top_srcdir)/data/effective_tld_names.dat") AC_SUBST(PSL_FILE) +# Check for custom PSL test file +AC_ARG_WITH(psl-testfile, + AC_HELP_STRING([--with-psl-testfile=[PATH]], + [path to PSL test file]), + PSL_TESTFILE=$withval, + PSL_TESTFILE="\$(top_srcdir)/data/test_psl.txt") +AC_SUBST(PSL_TESTFILE) + # Override the template file name of the generated .pc file, so that there # is no need to rename the template file when the API version changes. AC_CONFIG_FILES([Makefile @@ -120,5 +128,6 @@ AC_MSG_NOTICE([Summary of build options: LDFlags: ${LDFLAGS} Builtin PSL: ${enable_builtin} PSL File: ${PSL_FILE} + PSL Test File: ${PSL_TESTFILE} Tests: ${TESTS_INFO} ]) diff --git a/tests/Makefile.am b/tests/Makefile.am index e867f17..637d995 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,4 +1,4 @@ -DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" -DPSL_FILE=\"$(PSL_FILE)\" +DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" -DPSL_FILE=\"$(PSL_FILE)\" -DPSL_TESTFILE=\"$(PSL_TESTFILE)\" AM_CPPFLAGS = -I$(top_srcdir)/include LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index ebd1da6..57f1721 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -39,8 +39,6 @@ #include -#define TESTDATA DATADIR"/test_psl.txt" - static int ok, failed; @@ -110,7 +108,7 @@ static void test_psl(void) // Norwegian with lowercase oe test(psl, "www.\303\270yer.no", "www.\303\270yer.no"); - if ((fp = fopen(TESTDATA, "r"))) { + if ((fp = fopen(PSL_TESTFILE, "r"))) { while ((fgets(buf, sizeof(buf), fp))) { if (sscanf(buf, " checkPublicSuffix('%127[^']' , '%127[^']", domain, expected_regdom) != 2) { if (sscanf(buf, " checkPublicSuffix('%127[^']' , %127[nul]", domain, expected_regdom) != 2) @@ -130,7 +128,7 @@ static void test_psl(void) fclose(fp); } else { - printf("Failed to open %s\n", TESTDATA); + printf("Failed to open %s\n", PSL_TESTFILE); failed++; } } From 41171a949b29809f9874fdecae1e0e99cb9a015d Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 13 Apr 2014 21:49:03 +0200 Subject: [PATCH 072/104] added --print-info, --use-builtin-data, --load-psl-file to tools/psl.c --- tools/psl.c | 69 ++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 58 insertions(+), 11 deletions(-) diff --git a/tools/psl.c b/tools/psl.c index d97156c..f8b7b61 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -39,27 +39,33 @@ static void usage(int err) { fprintf(stderr, "\n"); - fprintf(stderr, "Usage: psl [options]\n"); + fprintf(stderr, "Usage: psl [options] \n"); fprintf(stderr, "\n"); fprintf(stderr, "Options:\n"); - fprintf(stderr, " --is-public-suffix check if domains are public suffixes or not. [default]\n"); - fprintf(stderr, " --print-unreg-domain print the longest publix suffix part\n"); - fprintf(stderr, " --print-reg-domain print the shortest private suffix part\n"); + fprintf(stderr, " --use-builtin-data use the builtin PSL data. [default]\n"); + fprintf(stderr, " --load-psl-file load PSL data from file.\n"); + fprintf(stderr, " --is-public-suffix check if domains are public suffixes or not. [default]\n"); + fprintf(stderr, " --print-unreg-domain print the longest publix suffix part\n"); + fprintf(stderr, " --print-reg-domain print the shortest private suffix part\n"); fprintf(stderr, "\n"); exit(err); } +static const char *time2str(time_t t) +{ + static char buf[64]; + struct tm *tp = localtime(&t); + + strftime(buf, sizeof(buf), "%a, %d %b %Y %T %z", tp); + return buf; +} + int main(int argc, const char *const *argv) { int mode = 1; - const char *const *arg; - const psl_ctx_t *psl = psl_builtin(); - - if (!psl) { - fprintf(stderr, "No PSL builtin data available\n"); - exit(2); - } + const char *const *arg, *psl_file = NULL; + psl_ctx_t *psl = (psl_ctx_t *) psl_builtin(); for (arg = argv + 1; arg < argv + argc; arg++) { if (!strncmp(*arg, "--", 2)) { @@ -69,6 +75,27 @@ int main(int argc, const char *const *argv) mode = 2; else if (!strcmp(*arg, "--print-reg-domain")) mode = 3; + else if (!strcmp(*arg, "--print-info")) + mode = 4; + else if (!strcmp(*arg, "--use-builtin-data")) { + psl_free(psl); + if (psl_file) { + fprintf(stderr, "Dropped data from %s\n", psl_file); + psl_file = NULL; + } + psl = (psl_ctx_t *) psl_builtin(); + } + else if (!strcmp(*arg, "--load-psl-file") && arg < argv + argc - 1) { + psl_free(psl); + if (psl_file) { + fprintf(stderr, "Dropped data from %s\n", psl_file); + psl_file = NULL; + } + if (!(psl = psl_load_file(psl_file = *(++arg)))) { + fprintf(stderr, "Failed to load PSL data from %s\n", psl_file); + psl_file = NULL; + } + } else if (!strcmp(*arg, "--")) { arg++; break; @@ -93,6 +120,26 @@ int main(int argc, const char *const *argv) for (; arg < argv + argc; arg++) printf("%s: %s\n", *arg, psl_registrable_domain(psl, *arg)); } + else if (mode == 4) { + if (psl != psl_builtin()) { + printf("suffixes: %d\n", psl_suffix_count(psl)); + printf("exceptions: %d\n", psl_suffix_exception_count(psl)); + } + + psl_free(psl); + psl = (psl_ctx_t *) psl_builtin(); + + if (psl) { + printf("builtin suffixes: %d\n", psl_suffix_count(psl)); + printf("builtin exceptions: %d\n", psl_suffix_exception_count(psl)); + printf("builtin compile time: %ld (%s)\n", psl_builtin_compile_time(), time2str(psl_builtin_compile_time())); + printf("builtin file time: %ld (%s)\n", psl_builtin_file_time(), time2str(psl_builtin_file_time())); + printf("builtin SHA1 file hash: %s\n", psl_builtin_sha1sum()); + } else + printf("No builtin PSL data available\n"); + } + + psl_free(psl); return 0; } From b8f7a7994721aa0844fd53f2c73877c0500c5850 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Tue, 15 Apr 2014 15:02:50 +0200 Subject: [PATCH 073/104] added checks for special .name domain --- tests/test-is-public-builtin.c | 9 +++++++++ tests/test-is-public.c | 9 +++++++++ 2 files changed, 18 insertions(+) diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index 7f53b63..fdc2701 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -68,6 +68,15 @@ static void test_psl(void) { "www.\345\225\206\346\240\207", 0 }, { "xn--czr694b", 1 }, { "www.xn--czr694b", 0 }, + // some special test follow ('name' and 'forgot.his.name' are public, but e.g. his.name is not) + { "name", 1 }, + { ".name", 1 }, + { "his.name", 0 }, + { ".his.name", 0 }, + { "forgot.his.name", 1 }, + { ".forgot.his.name", 1 }, + { "whoever.his.name", 0 }, + { "whoever.forgot.his.name", 0 }, }; unsigned it; const psl_ctx_t *psl; diff --git a/tests/test-is-public.c b/tests/test-is-public.c index 24cb32e..bf11d7e 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -66,6 +66,15 @@ static void test_psl(void) { "www.xxx.ck", 0 }, { "\345\225\206\346\240\207", 1 }, // xn--czr694b oder 商标 { "www.\345\225\206\346\240\207", 0 }, + // some special test follow ('name' and 'forgot.his.name' are public, but e.g. his.name is not) + { "name", 1 }, + { ".name", 1 }, + { "his.name", 0 }, + { ".his.name", 0 }, + { "forgot.his.name", 1 }, + { ".forgot.his.name", 1 }, + { "whoever.his.name", 0 }, + { "whoever.forgot.his.name", 0 }, }; unsigned it; psl_ctx_t *psl; From 2c7c11d8a6c4c739daa25b9e883b84f09c3a86d7 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 16 Apr 2014 10:52:35 +0200 Subject: [PATCH 074/104] added psl_is_cookie_domain_acceptable() --- include/libpsl.h | 4 ++ src/psl.c | 96 +++++++++++++++++++++++++++++++++--------------- 2 files changed, 70 insertions(+), 30 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index f356583..9d4f8db 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -49,8 +49,12 @@ psl_ctx_t * psl_load_fp(FILE *fp); const psl_ctx_t * psl_builtin(void); +// checks wether domain is a public suffix or not int psl_is_public_suffix(const psl_ctx_t *psl, const char *domain); +// checks wether cookie_domain is acceptable for domain or not +int + psl_is_cookie_domain_acceptable(const psl_ctx_t *psl, const char *hostname, const char *cookie_domain); // returns the longest unregistrable domain within 'domain' or NULL if none found const char * psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain); diff --git a/src/psl.c b/src/psl.c index 8cfff91..dce42ef 100644 --- a/src/psl.c +++ b/src/psl.c @@ -360,29 +360,20 @@ int psl_is_public_suffix(const psl_ctx_t *psl, const char *domain) */ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) { - const char *p, *ret_domain; - if (!psl || !domain) return NULL; - // We check from right to left, e.g. in www.xxx.org we check org, xxx.org, www.xxx.org in this order - // for being a registered domain. + // We check from left to right to catch special PSL entries like 'forgot.his.name': + // 'forgot.his.name' and 'name' are in the PSL while 'his.name' is not. - if (!(p = strrchr(domain, '.'))) - return psl_is_public_suffix(psl, domain) ? domain : NULL; - - for (ret_domain = NULL; ;) { - if (!psl_is_public_suffix(psl, p)) - return ret_domain; - else if (p == domain) - return domain; - - ret_domain = p + 1; - - // go left to next dot - while (p > domain && *--p != '.') - ; + while (!psl_is_public_suffix(psl, domain)) { + if ((domain = strchr(domain, '.'))) + domain++; + else + break; // prevent endless loop if psl_is_public_suffix() is broken. } + + return domain; } /** @@ -403,25 +394,23 @@ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) */ const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) { - const char *p; - int ispublic; + const char *p, *regdom = NULL; if (!psl || !domain || *domain == '.') return NULL; - // We check from right to left, e.g. in www.xxx.org we check org, xxx.org, www.xxx.org in this order - // for being a registrable domain. + // We check from left to right to catch special PSL entries like 'forgot.his.name': + // 'forgot.his.name' and 'name' are in the PSL while 'his.name' is not. - if (!(p = strrchr(domain, '.'))) - p = domain; - - while ((ispublic = psl_is_public_suffix(psl, p)) && p > domain) { - // go left to next dot - while (p > domain && *--p != '.') - ; + while (!psl_is_public_suffix(psl, domain)) { + if ((p = strchr(domain, '.'))) { + regdom = domain; + domain = p + 1; + } else + break; // prevent endless loop if psl_is_public_suffix() is broken. } - return ispublic ? NULL : (*p == '.' ? p + 1 : p); + return regdom; } /** @@ -647,3 +636,50 @@ const char *psl_builtin_sha1sum(void) { return _psl_sha1_checksum; } + +/** + * psl_is_cookie_domain_acceptable: + * @psl: PSL context pointer + * @hostname: The request hostname. + * @cookie_domain: The domain value from a cookie + * + * This helper function checks whether @cookie_domain is an acceptable cookie domain value for the request + * @hostname. + * + * Returns: 1 if acceptable, 0 if not acceptable. + * + * Since: 0.1 + */ +int psl_is_cookie_domain_acceptable(const psl_ctx_t *psl, const char *hostname, const char *cookie_domain) +{ + const char *registrable_domain, *p; + size_t hostname_length, cookie_domain_length; + + if (!psl || !hostname || !cookie_domain) + return 0; + + while (*cookie_domain == '.') + cookie_domain++; + + if (!strcmp(hostname, cookie_domain)) + return 1; // an exact match is acceptable (and pretty common) + + cookie_domain_length = strlen(cookie_domain); + hostname_length = strlen(hostname); + + if (cookie_domain_length >= hostname_length) + return 0; // cookie_domain is too long + + p = hostname + hostname_length - cookie_domain_length; + if (!strcmp(p, cookie_domain) && p[-1] == '.') { + // OK, cookie_domain matches, but it must be longer than the longest public suffix in 'hostname' + + if (!(p = psl_unregistrable_domain(psl, hostname))) + return 1; + + if (cookie_domain_length > strlen(p)) + return 1; + } + + return 0; +} From 5fa3b170bdb245b7a83a4002f1971dfd7d2ff48e Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 16 Apr 2014 10:53:59 +0200 Subject: [PATCH 075/104] added checks in test-registrable-domain.c --- tests/test-registrable-domain.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index 57f1721..c1756a9 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -108,6 +108,15 @@ static void test_psl(void) // Norwegian with lowercase oe test(psl, "www.\303\270yer.no", "www.\303\270yer.no"); + // special check with NULL psl context and TLD + test(psl, "whoever.forgot.his.name", "whoever.forgot.his.name"); + + // special check with NULL psl context and TLD + test(psl, "forgot.his.name", NULL); + + // special check with NULL psl context and TLD + test(psl, "his.name", "his.name"); + if ((fp = fopen(PSL_TESTFILE, "r"))) { while ((fgets(buf, sizeof(buf), fp))) { if (sscanf(buf, " checkPublicSuffix('%127[^']' , '%127[^']", domain, expected_regdom) != 2) { From d04e6f5c09c28f2a33d154a98b18da1dea9ba1a7 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 16 Apr 2014 10:55:57 +0200 Subject: [PATCH 076/104] new test test-is-cookie-domain-acceptable.c --- tests/Makefile.am | 3 +- tests/test-is-cookie-domain-acceptable.c | 119 +++++++++++++++++++++++ 2 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 tests/test-is-cookie-domain-acceptable.c diff --git a/tests/Makefile.am b/tests/Makefile.am index 637d995..b358ada 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -2,7 +2,8 @@ DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" -DPSL_FILE AM_CPPFLAGS = -I$(top_srcdir)/include LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la -PSL_TESTS = test-is-public test-is-public-builtin test-is-public-all test-registrable-domain +PSL_TESTS = test-is-public test-is-public-builtin test-is-public-all test-registrable-domain \ + test-is-cookie-domain-acceptable check_PROGRAMS = $(PSL_TESTS) diff --git a/tests/test-is-cookie-domain-acceptable.c b/tests/test-is-cookie-domain-acceptable.c new file mode 100644 index 0000000..6abd273 --- /dev/null +++ b/tests/test-is-cookie-domain-acceptable.c @@ -0,0 +1,119 @@ +/* + * Copyright(c) 2014 Tim Ruehsen + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + * + * This file is part of the test suite of libpsl. + * + * Test case for psl_is_cookie_doamin_acceptable() + * + * Changelog + * 15.04.2014 Tim Ruehsen created from libmget/cookie.c + * + */ + +#if HAVE_CONFIG_H +# include +#endif + +#include +#include +#include + +#include + +#define countof(a) (sizeof(a)/sizeof(*(a))) + +static int + ok, + failed; + +static void test_psl(void) +{ + // punycode generation: idn 商标 + // octal code generation: echo -n "商标" | od -b + static const struct test_data { + const char + *request_domain, + *cookie_domain; + int + result; + } test_data[] = { + { "www.dkg.forgot.his.name", "www.dkg.forgot.his.name", 1 }, + { "www.dkg.forgot.his.name", "dkg.forgot.his.name", 1 }, + { "www.dkg.forgot.his.name", "forgot.his.name", 0 }, + { "www.dkg.forgot.his.name", "his.name", 0 }, + { "www.dkg.forgot.his.name", "name", 0 }, + { "www.his.name", "www.his.name", 1 }, + { "www.his.name", "his.name", 1 }, + { "www.his.name", "name", 0 }, + { "www.example.com", "www.example.com", 1 }, + { "www.example.com", "example.com", 1 }, + { "www.example.com", "com", 0 }, // not accepted by normalization (PSL rule 'com') + { "www.example.com", "example.org", 0 }, + { "www.sa.gov.au", "sa.gov.au", 0 }, // not accepted by normalization (PSL rule '*.ar') + { "www.educ.ar", "educ.ar", 1 }, // PSL exception rule '!educ.ar' + }; + unsigned it; + psl_ctx_t *psl; + + psl = psl_load_file(PSL_FILE); + + printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); + + for (it = 0; it < countof(test_data); it++) { + const struct test_data *t = &test_data[it]; + int result = psl_is_cookie_domain_acceptable(psl, t->request_domain, t->cookie_domain); + + if (result == t->result) { + ok++; + } else { + failed++; + printf("psl_is_cookie_domain_acceptable(%s, %s)=%d (expected %d)\n", + t->request_domain, t->cookie_domain, result, t->result); + } + } + + psl_free(psl); +} + +int main(int argc, const char * const *argv) +{ + // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + if (argc == 1) { + const char *valgrind = getenv("TESTS_VALGRIND"); + + if (valgrind && *valgrind) { + char cmd[strlen(valgrind)+strlen(argv[0])+32]; + + snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + return system(cmd) != 0; + } + } + + test_psl(); + + if (failed) { + printf("Summary: %d out of %d tests failed\n", failed, ok + failed); + return 1; + } + + printf("Summary: All %d tests passed\n", ok + failed); + return 0; +} From 5e6bad036b99a28c980fb2c95f0f762d459321b5 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 16 Apr 2014 10:56:42 +0200 Subject: [PATCH 077/104] added --is-cookie-domain-acceptable to tools/psl.c --- tools/psl.c | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/tools/psl.c b/tools/psl.c index f8b7b61..50ddb56 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -42,11 +42,13 @@ static void usage(int err) fprintf(stderr, "Usage: psl [options] \n"); fprintf(stderr, "\n"); fprintf(stderr, "Options:\n"); - fprintf(stderr, " --use-builtin-data use the builtin PSL data. [default]\n"); - fprintf(stderr, " --load-psl-file load PSL data from file.\n"); - fprintf(stderr, " --is-public-suffix check if domains are public suffixes or not. [default]\n"); - fprintf(stderr, " --print-unreg-domain print the longest publix suffix part\n"); - fprintf(stderr, " --print-reg-domain print the shortest private suffix part\n"); + fprintf(stderr, " --use-builtin-data use the builtin PSL data. [default]\n"); + fprintf(stderr, " --load-psl-file load PSL data from file.\n"); + fprintf(stderr, " --is-public-suffix check if domains are public suffixes or not. [default]\n"); + fprintf(stderr, " --is-cookie-domain-acceptable \n"); + fprintf(stderr, " check if cookie-domain is acceptable for domains.\n"); + fprintf(stderr, " --print-unreg-domain print the longest publix suffix part\n"); + fprintf(stderr, " --print-reg-domain print the shortest private suffix part\n"); fprintf(stderr, "\n"); exit(err); @@ -64,7 +66,7 @@ static const char *time2str(time_t t) int main(int argc, const char *const *argv) { int mode = 1; - const char *const *arg, *psl_file = NULL; + const char *const *arg, *psl_file = NULL, *cookie_domain = NULL; psl_ctx_t *psl = (psl_ctx_t *) psl_builtin(); for (arg = argv + 1; arg < argv + argc; arg++) { @@ -76,7 +78,11 @@ int main(int argc, const char *const *argv) else if (!strcmp(*arg, "--print-reg-domain")) mode = 3; else if (!strcmp(*arg, "--print-info")) + mode = 99; + else if (!strcmp(*arg, "--is-cookie-domain-acceptable") && arg < argv + argc - 1) { mode = 4; + cookie_domain = *(++arg); + } else if (!strcmp(*arg, "--use-builtin-data")) { psl_free(psl); if (psl_file) { @@ -121,6 +127,10 @@ int main(int argc, const char *const *argv) printf("%s: %s\n", *arg, psl_registrable_domain(psl, *arg)); } else if (mode == 4) { + for (; arg < argv + argc; arg++) + printf("%s: %d\n", *arg, psl_is_cookie_domain_acceptable(psl, *arg, cookie_domain)); + } + else if (mode == 99) { if (psl != psl_builtin()) { printf("suffixes: %d\n", psl_suffix_count(psl)); printf("exceptions: %d\n", psl_suffix_exception_count(psl)); From c18f6fdc6fc07dd38e9e4d04708a9b5b2763db4a Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 16 Apr 2014 11:36:37 +0200 Subject: [PATCH 078/104] fixed docs --- docs/libpsl/libpsl-sections.txt | 3 ++- src/psl.c | 14 ++++++++++---- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/docs/libpsl/libpsl-sections.txt b/docs/libpsl/libpsl-sections.txt index 7ee9e7f..19f6599 100644 --- a/docs/libpsl/libpsl-sections.txt +++ b/docs/libpsl/libpsl-sections.txt @@ -2,10 +2,10 @@ libpsl Public Suffix List functions psl_ctx_t -psl_free psl_load_file psl_load_fp psl_builtin +psl_free psl_is_public_suffix psl_unregistrable_domain psl_registrable_domain @@ -14,4 +14,5 @@ psl_suffix_exception_count psl_builtin_compile_time psl_builtin_file_time psl_builtin_sha1sum +psl_is_cookie_domain_acceptable diff --git a/src/psl.c b/src/psl.c index dce42ef..fd9f7fb 100644 --- a/src/psl.c +++ b/src/psl.c @@ -193,7 +193,7 @@ static inline int _vector_size(_psl_vector_t *v) return v ? v->cur : 0; } -// by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) +// by this kind of sorting, we can easily see if a domain matches or not static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) { @@ -255,8 +255,7 @@ static int _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) * This function checks if @domain is a public suffix by the means of the * [Mozilla Public Suffix List](http://publicsuffix.org). * - * This can be used for e.g. cookie domain verification. - * You should never accept a cookie who's domain is a public suffix. + * For cookie domain checking see psl_is_cookie_domain_acceptable(). * * @psl is a context returned by either psl_load_file(), psl_load_fp() or * psl_builtin(). @@ -646,13 +645,20 @@ const char *psl_builtin_sha1sum(void) * This helper function checks whether @cookie_domain is an acceptable cookie domain value for the request * @hostname. * + * Examples: + * 1. Cookie domain 'example.com' would be acceptable for hostname 'www.example.com', + * but '.com' or 'com' would NOT be acceptable since 'com' is a public suffix. + * + * 2. Cookie domain 'his.name' would be acceptable for hostname 'remember.his.name', + * but NOT for 'forgot.his.name' since 'forgot.his.name' is a public suffix. + * * Returns: 1 if acceptable, 0 if not acceptable. * * Since: 0.1 */ int psl_is_cookie_domain_acceptable(const psl_ctx_t *psl, const char *hostname, const char *cookie_domain) { - const char *registrable_domain, *p; + const char *p; size_t hostname_length, cookie_domain_length; if (!psl || !hostname || !cookie_domain) From 9a31e7375151e77ec86959429fd94f16ad0637c9 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 16 Apr 2014 13:00:43 +0200 Subject: [PATCH 079/104] make distcheck non-parallel, -j4 fails --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index e3e280d..54965ca 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,7 +3,7 @@ compiler: - gcc - clang # Change this to your needs -script: ./autogen.sh && ./configure --enable-gtk-doc && make -j4 && make check -j4 && make distcheck -j4 +script: ./autogen.sh && ./configure --enable-gtk-doc && make -j4 && make check -j4 && make distcheck before_install: - sudo apt-get -qq update - sudo apt-get -q install autoconf automake autopoint libtool gtk-doc-tools gettext idn2 libidn2-0 libidn2-0-dev From ee0064532cba22668bca2e1d4b42c1aa258e3552 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 17 Apr 2014 12:31:06 +0200 Subject: [PATCH 080/104] added psl_builtin_filename() --- docs/libpsl/libpsl-sections.txt | 1 + include/libpsl.h | 5 ++++- src/psl.c | 17 ++++++++++++++++- src/psl2c.c | 6 ++++-- tools/psl.c | 1 + 5 files changed, 26 insertions(+), 4 deletions(-) diff --git a/docs/libpsl/libpsl-sections.txt b/docs/libpsl/libpsl-sections.txt index 19f6599..c6eaa16 100644 --- a/docs/libpsl/libpsl-sections.txt +++ b/docs/libpsl/libpsl-sections.txt @@ -14,5 +14,6 @@ psl_suffix_exception_count psl_builtin_compile_time psl_builtin_file_time psl_builtin_sha1sum +psl_builtin_filename psl_is_cookie_domain_acceptable diff --git a/include/libpsl.h b/include/libpsl.h index 9d4f8db..4ce408c 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -73,9 +73,12 @@ time_t // returns mtime of PSL source file time_t psl_builtin_file_time(void); -// returns MD5 checksum (hex-encoded, lowercase) of PSL source file +// returns SHA1 checksum (hex-encoded, lowercase) of PSL source file const char * psl_builtin_sha1sum(void); +// returns file name of PSL source file +const char * + psl_builtin_filename(void); #ifdef __cplusplus diff --git a/src/psl.c b/src/psl.c index fd9f7fb..d96fd7f 100644 --- a/src/psl.c +++ b/src/psl.c @@ -618,7 +618,6 @@ time_t psl_builtin_file_time(void) return _psl_file_time; } -// returns MD5 checksum (hex-encoded, lowercase) of PSL source file /** * psl_builtin_sha1sum: * @@ -636,6 +635,22 @@ const char *psl_builtin_sha1sum(void) return _psl_sha1_checksum; } +/** + * psl_builtin_filename: + * + * This function returns the file name of the Publix Suffix List file that has been built in. + * + * If the generation of built-in data has been disabled during compilation, an empty string will be returned. + * + * Returns: String containing the PSL file name or an empty string. + * + * Since: 0.1 + */ +const char *psl_builtin_filename(void) +{ + return _psl_filename; +} + /** * psl_is_cookie_domain_acceptable: * @psl: PSL context pointer diff --git a/src/psl2c.c b/src/psl2c.c index c5bf12f..e55301c 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -380,7 +380,8 @@ int main(int argc, const char **argv) st.st_mtime = 0; fprintf(fpout, "static time_t _psl_file_time = %lu;\n", st.st_mtime); fprintf(fpout, "static time_t _psl_compile_time = %lu;\n", time(NULL)); - fprintf(fpout, "static char _psl_sha1_checksum[] = \"%s\";\n", checksum); + fprintf(fpout, "static const char _psl_sha1_checksum[] = \"%s\";\n", checksum); + fprintf(fpout, "static const char _psl_filename[] = \"%s\";\n", checksum); if (fclose(fpout) != 0) ret = 4; @@ -396,7 +397,8 @@ int main(int argc, const char **argv) fprintf(fpout, "static _psl_entry_t suffix_exceptions[0];\n"); fprintf(fpout, "static time_t _psl_file_time;\n"); fprintf(fpout, "static time_t _psl_compile_time;\n"); - fprintf(fpout, "static char _psl_sha1_checksum[]= \"\";\n"); + fprintf(fpout, "static const char _psl_sha1_checksum[] = \"\";\n"); + fprintf(fpout, "static const char _psl_filename[] = \"\";\n"); if (fclose(fpout) != 0) ret = 4; diff --git a/tools/psl.c b/tools/psl.c index 50ddb56..0e8c926 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -142,6 +142,7 @@ int main(int argc, const char *const *argv) if (psl) { printf("builtin suffixes: %d\n", psl_suffix_count(psl)); printf("builtin exceptions: %d\n", psl_suffix_exception_count(psl)); + printf("builtin filename: %s\n", psl_builtin_filename()); printf("builtin compile time: %ld (%s)\n", psl_builtin_compile_time(), time2str(psl_builtin_compile_time())); printf("builtin file time: %ld (%s)\n", psl_builtin_file_time(), time2str(psl_builtin_file_time())); printf("builtin SHA1 file hash: %s\n", psl_builtin_sha1sum()); From ed9562848c25776857d65f61580c2fd355361dd8 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 17 Apr 2014 13:09:29 +0200 Subject: [PATCH 081/104] small --disable-builtin cleanups --- configure.ac | 1 + src/Makefile.am | 2 -- tests/Makefile.am | 11 +++++++++++ tests/test-is-public-builtin.c | 2 -- tests/test-registrable-domain.c | 2 -- 5 files changed, 12 insertions(+), 6 deletions(-) diff --git a/configure.ac b/configure.ac index 1daf685..05a4386 100644 --- a/configure.ac +++ b/configure.ac @@ -69,6 +69,7 @@ AC_ARG_ENABLE(builtin, enable_builtin=yes AC_DEFINE([WITH_BUILTIN], [1], [compile PSL data into library]) ]) +AM_CONDITIONAL([WITH_BUILTIN], [test $enable_builtin = yes]) # Check for valgrind ac_enable_valgrind=no diff --git a/src/Makefile.am b/src/Makefile.am index 0f70961..7a584bb 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1,5 +1,3 @@ -#EXTRA_DIST = $(top_srcdir)/data/effective_tld_names.dat - # suffixes.c must be created before psl.c is compiled BUILT_SOURCES = suffixes.c diff --git a/tests/Makefile.am b/tests/Makefile.am index b358ada..e200a22 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -2,9 +2,20 @@ DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" -DPSL_FILE AM_CPPFLAGS = -I$(top_srcdir)/include LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la +if WITH_BUILTIN + PSL_TESTS = test-is-public test-is-public-builtin test-is-public-all test-registrable-domain \ test-is-cookie-domain-acceptable +else + +# ./configure'd with '--disable-builtin' +# Do not call test-is-public-builtin here: it does not make sense. +# Do not call test-registrable-domain here: it would fail due to missing punycode entries in PSL file. +PSL_TESTS = test-is-public test-is-public-all test-is-cookie-domain-acceptable + +endif + check_PROGRAMS = $(PSL_TESTS) TESTS_ENVIRONMENT = TESTS_VALGRIND="@VALGRIND_ENVIRONMENT@" diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index fdc2701..e4ff3fc 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -121,9 +121,7 @@ int main(int argc, const char * const *argv) } } -#ifdef WITH_BUILTIN test_psl(); -#endif if (failed) { printf("Summary: %d out of %d tests failed\n", failed, ok + failed); diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index c1756a9..17bed2f 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -156,9 +156,7 @@ int main(int argc, const char * const *argv) } } -#ifdef WITH_BUILTIN test_psl(); -#endif if (failed) { printf("Summary: %d out of %d tests failed\n", failed, ok + failed); From 608d9d951faf1578c9bca2fcaa8e10681b4f8a5d Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Tue, 22 Apr 2014 16:49:00 +0200 Subject: [PATCH 082/104] added punycode considerations to the API docs --- src/psl.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/psl.c b/src/psl.c index d96fd7f..c5166f3 100644 --- a/src/psl.c +++ b/src/psl.c @@ -419,6 +419,14 @@ const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) * This function loads the public suffixes file named @fname. * To free the allocated resources, call psl_free(). * + * If you want to use punycode representations for functions like psl_is_public_suffix(), + * these have to exist as entries within @fname. This is a design decision to not pull in + * dependencies for UTF-8 case-handling and IDNA libraries. + * + * On the contrary, the builtin data already contains punycode entries. + * + * Have a look into psl2c.c for example code on how to convert UTF-8 to lowercase and to punycode. + * * Returns: Pointer to a PSL context or %NULL on failure. * * Since: 0.1 @@ -446,6 +454,8 @@ psl_ctx_t *psl_load_file(const char *fname) * This function loads the public suffixes from a FILE pointer. * To free the allocated resources, call psl_free(). * + * Have a look at psl_load_fp() for punycode considerations. + * * Returns: Pointer to a PSL context or %NULL on failure. * * Since: 0.1 @@ -530,7 +540,11 @@ void psl_free(psl_ctx_t *psl) * This function returns the PSL context that has been generated and built in at compile-time. * You don't have to free the returned context explicitely. * + * The builtin data also contains punycode entries, one for each international domain name. + * * If the generation of built-in data has been disabled during compilation, %NULL will be returned. + * So if using the builtin psl context, you can provide UTF-8 or punycode representations of domains to + * functions like psl_is_public_suffix(). * * Returns: Pointer to the built in PSL data or NULL if this data is not available. * From 7d3be7c5fe2cc38b4318815c6c36e20116c4422f Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 23 Apr 2014 16:12:50 +0200 Subject: [PATCH 083/104] migrated docs/libpsl/html to gh-pages branch and replaced with submodule link --- .gitmodules | 4 ++++ docs/libpsl/html | 1 + 2 files changed, 5 insertions(+) create mode 100644 .gitmodules create mode 160000 docs/libpsl/html diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..59e0bf1 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,4 @@ +[submodule "docs/libpsl/html"] + path = docs/libpsl/html + url = https://github.com/rockdaboot/libpsl + branch = gh-pages diff --git a/docs/libpsl/html b/docs/libpsl/html new file mode 160000 index 0000000..f219eb9 --- /dev/null +++ b/docs/libpsl/html @@ -0,0 +1 @@ +Subproject commit f219eb93eb24f488bbe0fe1da98d11505498b798 From 987b94fe948f768c4eb091c21dbaeb144db6238a Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 23 Apr 2014 16:43:44 +0200 Subject: [PATCH 084/104] removed the submodule again, caused nothing but headaches --- .gitmodules | 4 ---- docs/libpsl/html | 1 - 2 files changed, 5 deletions(-) delete mode 100644 .gitmodules delete mode 160000 docs/libpsl/html diff --git a/.gitmodules b/.gitmodules deleted file mode 100644 index 59e0bf1..0000000 --- a/.gitmodules +++ /dev/null @@ -1,4 +0,0 @@ -[submodule "docs/libpsl/html"] - path = docs/libpsl/html - url = https://github.com/rockdaboot/libpsl - branch = gh-pages diff --git a/docs/libpsl/html b/docs/libpsl/html deleted file mode 160000 index f219eb9..0000000 --- a/docs/libpsl/html +++ /dev/null @@ -1 +0,0 @@ -Subproject commit f219eb93eb24f488bbe0fe1da98d11505498b798 From 316e48d48d07adda961c206fd446b1bbe37bd98d Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 23 Apr 2014 17:13:15 +0200 Subject: [PATCH 085/104] added version to HTML docs --- configure.ac | 1 + docs/libpsl/libpsl-docs.sgml | 12 ++++++++---- docs/libpsl/version.xml.in | 1 + 3 files changed, 10 insertions(+), 4 deletions(-) create mode 100644 docs/libpsl/version.xml.in diff --git a/configure.ac b/configure.ac index 05a4386..cb45300 100644 --- a/configure.ac +++ b/configure.ac @@ -114,6 +114,7 @@ AC_CONFIG_FILES([Makefile tools/Makefile po/Makefile.in docs/libpsl/Makefile + docs/libpsl/version.xml data/Makefile tests/Makefile libpsl-${LIBPSL_API_VERSION}.pc:libpsl.pc.in]) diff --git a/docs/libpsl/libpsl-docs.sgml b/docs/libpsl/libpsl-docs.sgml index 944cb03..2aa080c 100644 --- a/docs/libpsl/libpsl-docs.sgml +++ b/docs/libpsl/libpsl-docs.sgml @@ -3,19 +3,23 @@ "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd" [ + ]> - libpsl Reference Manual + Libpsl Reference Manual - for libpsl [VERSION]. + for Libpsl &version;. The latest version of this documentation can be found on-line at GitHub. - - libpsl functions + + Libpsl Overview + + Libpsl provides functions to work with the Mozilla Public Suffix List. + diff --git a/docs/libpsl/version.xml.in b/docs/libpsl/version.xml.in new file mode 100644 index 0000000..3f2b373 --- /dev/null +++ b/docs/libpsl/version.xml.in @@ -0,0 +1 @@ +@LIBPSL_API_VERSION@ From f85acef96ed594fd190209d24e4c752a90f5081d Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 24 Apr 2014 12:39:31 +0200 Subject: [PATCH 086/104] added --help to tools/psl.c --- tools/psl.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/psl.c b/tools/psl.c index 0e8c926..7c53dce 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -102,6 +102,9 @@ int main(int argc, const char *const *argv) psl_file = NULL; } } + else if (!strcmp(*arg, "--help")) { + usage(0); + } else if (!strcmp(*arg, "--")) { arg++; break; From f4d44f202de705a9cd093a6c9dabdb4e56ff3e2f Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 24 Apr 2014 12:41:21 +0200 Subject: [PATCH 087/104] updated README.md --- README.md | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 69 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a554b4d..d71497e 100644 --- a/README.md +++ b/README.md @@ -3,10 +3,77 @@ libpsl - C library to handle the Public Suffix List =================================================== -Find more information [here](http://publicsuffix.org/). +A "public suffix" is a domain name under which Internet users can directly register own names. + +Browsers and other web clients can use it to + +- avoid privacy-leaking "supercookies" +- avoid privacy-leaking "super domain" certificates ([see post from Jeffry Walton](http://lists.gnu.org/archive/html/bug-wget/2014-03/msg00093.html)) +- domain highlighting parts of the domain in a user interface +- sorting domain lists by site + +Libpsl... + +- has built-in PSL data for fast access +- allows to load PSL data from files +- checks if a given domain is a "public suffix" +- provides immediate cookie domain verification +- finds the longest public part of a given domain +- finds the shortest private part of a given domain +- works with international domains (UTF-8 and IDNA2008 Punycode) +- is thread-safe + +Find more information about the Publix Suffix List [here](http://publicsuffix.org/). Download the Public Suffix List [here](https://hg.mozilla.org/mozilla-central/raw-file/tip/netwerk/dns/effective_tld_names.dat). + +API Documentation +----------------- + +You find the current API documentation [here](https://rockdaboot.github.io/libpsl). + + +Quick API example +----------------- + + #include + #include + + int main(int argc, char **argv) + { + const char *domain = "www.example.com"; + const char *cookie_domain = ".com"; + const psl_ctx_t *psl = psl_builtin(); + int is_public, is_acceptable; + + is_public = psl_is_public_suffix(psl, domain); + printf("%s %s a public suffix.\n", domain, is_public ? "is" : "is not"); + + is_acceptable = psl_is_cookie_domain_acceptable(psl, domain, cookie_domain); + printf("cookie domain '%s' %s acceptable for domain '%s'.\n", + cookie_domain, is_acceptable ? "is" : "is not", domain); + + return 0; + } + +Command Line Tool +----------------- + +Libpsl comes with a tool 'psl' that gives you access to most of the +library API via command line. + + $ psl --help + +prints the usage. + +License +------- + +Libpsl is made available under the terms of the MIT license.
+See the LICENSE file that accompanies this distribution for the full text of the license. + + Building from git ----------------- @@ -18,6 +85,7 @@ Download project and prepare sources with make make check + Mailing List ------------ From d0b89debdca95c2162f0b1a36a5c7b7a8d294e91 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 25 Apr 2014 12:36:59 +0200 Subject: [PATCH 088/104] prepared for initial release --- AUTHORS | 4 ++-- NEWS | 4 ++++ configure.ac | 13 ++++++++++--- 3 files changed, 16 insertions(+), 5 deletions(-) diff --git a/AUTHORS b/AUTHORS index ba6bf51..b1bf772 100644 --- a/AUTHORS +++ b/AUTHORS @@ -7,5 +7,5 @@ took part in discussions or 'just' asked questions. Please drop me a note if you feel you should have been mentioned here. -Tim Ruehsen (Original implementation of libpsl) -Daniel Kahn Gillmor +Tim Ruehsen (Implementation of libpsl) +Daniel Kahn Gillmor (Discussion, Ideas, Organization) diff --git a/NEWS b/NEWS index e69de29..05c0239 100644 --- a/NEWS +++ b/NEWS @@ -0,0 +1,4 @@ +Copyright (C) 2014 Tim Ruehsen + +25.04.2014 + Initial release V0.2 of libpsl diff --git a/configure.ac b/configure.ac index cb45300..a810230 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([libpsl], [0.1], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.2], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define]) @@ -54,8 +54,15 @@ AS_IF([ test "$enable_man" != no ], [ # For information on how to properly maintain the library version information, # refer to the libtool manual, section "Updating library version information": # http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html -AC_SUBST([LIBPSL_SO_VERSION], [0:0:0]) -AC_SUBST([LIBPSL_API_VERSION], [0.1]) +# +# 1. Start with version information of ‘0:0:0’ for each libtool library. +# 2. Update the version information only immediately before a public release of your software. More frequent updates are unnecessary, and only guarantee that the current interface number gets larger faster. +# 3. If the library source code has changed at all since the last update, then increment revision (‘c:r:a’ becomes ‘c:r+1:a’). +# 4. If any interfaces have been added, removed, or changed since the last update, increment current, and set revision to 0. +# 5. If any interfaces have been added since the last public release, then increment age. +# 6. If any interfaces have been removed or changed since the last public release, then set age to 0. +AC_SUBST([LIBPSL_SO_VERSION], [0:1:0]) +AC_SUBST([LIBPSL_API_VERSION], [0.2]) # Check for idn2 AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) From 2919a702be04b20e0bd72f97d31f3db7e8ab67a6 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 25 Apr 2014 16:03:32 +0200 Subject: [PATCH 089/104] Updated to the latest PSL --- NEWS | 6 +- configure.ac | 6 +- data/effective_tld_names.dat | 877 ++++++++++++++++++++--------------- 3 files changed, 517 insertions(+), 372 deletions(-) diff --git a/NEWS b/NEWS index 05c0239..1b9ebc9 100644 --- a/NEWS +++ b/NEWS @@ -1,4 +1,6 @@ Copyright (C) 2014 Tim Ruehsen -25.04.2014 - Initial release V0.2 of libpsl +25.04.2014 Hotfix release V0.2.1 + * Updated to the latest Publix Suffix List + +25.04.2014 Initial release V0.2 diff --git a/configure.ac b/configure.ac index a810230..9ca46bd 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([libpsl], [0.2], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.2.1], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define]) @@ -61,8 +61,8 @@ AS_IF([ test "$enable_man" != no ], [ # 4. If any interfaces have been added, removed, or changed since the last update, increment current, and set revision to 0. # 5. If any interfaces have been added since the last public release, then increment age. # 6. If any interfaces have been removed or changed since the last public release, then set age to 0. -AC_SUBST([LIBPSL_SO_VERSION], [0:1:0]) -AC_SUBST([LIBPSL_API_VERSION], [0.2]) +AC_SUBST([LIBPSL_SO_VERSION], [0:2:0]) +AC_SUBST([LIBPSL_API_VERSION], [0.2.1]) # Check for idn2 AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) diff --git a/data/effective_tld_names.dat b/data/effective_tld_names.dat index 9edd20d..989ec21 100644 --- a/data/effective_tld_names.dat +++ b/data/effective_tld_names.dat @@ -1192,373 +1192,373 @@ edu.it // There is also a list of reserved geo-names corresponding to Italian municipalities // http://www.nic.it/documenti/appendice-c.pdf, but it is not included here. // Regions -abr.it -abruzzo.it -aosta-valley.it -aostavalley.it -bas.it -basilicata.it -cal.it -calabria.it -cam.it -campania.it -emilia-romagna.it -emiliaromagna.it -emr.it -friuli-v-giulia.it -friuli-ve-giulia.it -friuli-vegiulia.it -friuli-venezia-giulia.it -friuli-veneziagiulia.it -friuli-vgiulia.it -friuliv-giulia.it -friulive-giulia.it -friulivegiulia.it -friulivenezia-giulia.it -friuliveneziagiulia.it -friulivgiulia.it -fvg.it -laz.it -lazio.it -lig.it -liguria.it -lom.it -lombardia.it -lombardy.it -lucania.it -mar.it -marche.it -mol.it -molise.it -piedmont.it -piemonte.it -pmn.it -pug.it -puglia.it +abr.it +abruzzo.it +aosta-valley.it +aostavalley.it +bas.it +basilicata.it +cal.it +calabria.it +cam.it +campania.it +emilia-romagna.it +emiliaromagna.it +emr.it +friuli-v-giulia.it +friuli-ve-giulia.it +friuli-vegiulia.it +friuli-venezia-giulia.it +friuli-veneziagiulia.it +friuli-vgiulia.it +friuliv-giulia.it +friulive-giulia.it +friulivegiulia.it +friulivenezia-giulia.it +friuliveneziagiulia.it +friulivgiulia.it +fvg.it +laz.it +lazio.it +lig.it +liguria.it +lom.it +lombardia.it +lombardy.it +lucania.it +mar.it +marche.it +mol.it +molise.it +piedmont.it +piemonte.it +pmn.it +pug.it +puglia.it sar.it -sardegna.it -sardinia.it -sic.it -sicilia.it -sicily.it -taa.it -tos.it -toscana.it -trentino-a-adige.it -trentino-aadige.it -trentino-alto-adige.it -trentino-altoadige.it -trentino-s-tirol.it -trentino-stirol.it -trentino-sud-tirol.it -trentino-sudtirol.it -trentino-sued-tirol.it -trentino-suedtirol.it -trentinoa-adige.it -trentinoaadige.it -trentinoalto-adige.it -trentinoaltoadige.it -trentinos-tirol.it -trentinostirol.it -trentinosud-tirol.it -trentinosudtirol.it -trentinosued-tirol.it -trentinosuedtirol.it -tuscany.it -umb.it -umbria.it -val-d-aosta.it -val-daosta.it -vald-aosta.it -valdaosta.it -valle-aosta.it -valle-d-aosta.it -valle-daosta.it -valleaosta.it -valled-aosta.it -valledaosta.it -vallee-aoste.it -valleeaoste.it -vao.it -vda.it -ven.it -veneto.it +sardegna.it +sardinia.it +sic.it +sicilia.it +sicily.it +taa.it +tos.it +toscana.it +trentino-a-adige.it +trentino-aadige.it +trentino-alto-adige.it +trentino-altoadige.it +trentino-s-tirol.it +trentino-stirol.it +trentino-sud-tirol.it +trentino-sudtirol.it +trentino-sued-tirol.it +trentino-suedtirol.it +trentinoa-adige.it +trentinoaadige.it +trentinoalto-adige.it +trentinoaltoadige.it +trentinos-tirol.it +trentinostirol.it +trentinosud-tirol.it +trentinosudtirol.it +trentinosued-tirol.it +trentinosuedtirol.it +tuscany.it +umb.it +umbria.it +val-d-aosta.it +val-daosta.it +vald-aosta.it +valdaosta.it +valle-aosta.it +valle-d-aosta.it +valle-daosta.it +valleaosta.it +valled-aosta.it +valledaosta.it +vallee-aoste.it +valleeaoste.it +vao.it +vda.it +ven.it +veneto.it // Provinces -ag.it -agrigento.it -al.it -alessandria.it -alto-adige.it -altoadige.it -an.it -ancona.it -andria-barletta-trani.it -andria-trani-barletta.it -andriabarlettatrani.it -andriatranibarletta.it -ao.it -aosta.it -aoste.it -ap.it -aq.it -aquila.it -ar.it -arezzo.it -ascoli-piceno.it -ascolipiceno.it -asti.it -at.it -av.it -avellino.it -ba.it -balsan.it -bari.it -barletta-trani-andria.it -barlettatraniandria.it -belluno.it -benevento.it -bergamo .it -bg.it -bi.it -biella.it -bl.it -bn.it -bo.it -bologna.it -bolzano.it -bozen.it -br.it -brescia.it -brindisi.it -bs.it -bt.it -bz.it -ca.it -cagliari.it -caltanissetta.it -campidano-medio.it -campidanomedio.it -campobasso.it -carbonia-iglesias.it -carboniaiglesias.it -carrara-massa.it -carraramassa.it -caserta.it -catania.it -catanzaro.it -cb.it -ce.it -cesena-forli.it -cesenaforli.it -ch.it -chieti.it -ci.it -cl.it -cn.it -co.it -como.it -cosenza.it -cr.it -cremona.it -crotone.it -cs.it -ct.it -cuneo.it -cz.it -dell-ogliastra.it -dellogliastra.it -en.it -enna.it -fc.it -fe.it -fermo.it -ferrara.it -fg.it -fi.it -firenze.it -florence.it -fm.it -foggia.it -forli-cesena.it +ag.it +agrigento.it +al.it +alessandria.it +alto-adige.it +altoadige.it +an.it +ancona.it +andria-barletta-trani.it +andria-trani-barletta.it +andriabarlettatrani.it +andriatranibarletta.it +ao.it +aosta.it +aoste.it +ap.it +aq.it +aquila.it +ar.it +arezzo.it +ascoli-piceno.it +ascolipiceno.it +asti.it +at.it +av.it +avellino.it +ba.it +balsan.it +bari.it +barletta-trani-andria.it +barlettatraniandria.it +belluno.it +benevento.it +bergamo.it +bg.it +bi.it +biella.it +bl.it +bn.it +bo.it +bologna.it +bolzano.it +bozen.it +br.it +brescia.it +brindisi.it +bs.it +bt.it +bz.it +ca.it +cagliari.it +caltanissetta.it +campidano-medio.it +campidanomedio.it +campobasso.it +carbonia-iglesias.it +carboniaiglesias.it +carrara-massa.it +carraramassa.it +caserta.it +catania.it +catanzaro.it +cb.it +ce.it +cesena-forli.it +cesenaforli.it +ch.it +chieti.it +ci.it +cl.it +cn.it +co.it +como.it +cosenza.it +cr.it +cremona.it +crotone.it +cs.it +ct.it +cuneo.it +cz.it +dell-ogliastra.it +dellogliastra.it +en.it +enna.it +fc.it +fe.it +fermo.it +ferrara.it +fg.it +fi.it +firenze.it +florence.it +fm.it +foggia.it +forli-cesena.it forlicesena.it -fr.it -frosinone.it -ge.it -genoa.it -genova.it -go.it -gorizia.it -gr.it -grosseto.it -iglesias-carbonia.it -iglesiascarbonia.it -im.it -imperia.it -is.it -isernia.it -kr.it -la-spezia.it -laquila.it -laspezia.it -latina.it -lc.it -le.it -lecce.it -lecco.it -li.it -livorno.it -lo.it -lodi.it -lt.it -lu.it -lucca.it -macerata.it -mantova.it -massa-carrara.it -massacarrara.it -matera.it -mb.it -mc.it -me.it -medio-campidano.it -mediocampidano.it -messina.it -mi.it -milan.it -milano.it -mn.it -mo.it -modena.it -monza-brianza.it -monza-e-della-brianza.it -monza.it -monzabrianza.it -monzaebrianza.it -monzaedellabrianza.it -ms.it -mt.it -na.it -naples.it -napoli.it -no.it -novara.it -nu.it -nuoro.it -og.it -ogliastra.it -olbia-tempio.it -olbiatempio.it -or.it -oristano.it -ot.it -pa.it -padova.it -padua.it -palermo.it -parma.it -pavia.it -pc.it -pd.it -pe.it -perugia.it -pesaro-urbino.it -pesarourbino.it -pescara.it -pg.it -pi.it -piacenza.it -pisa.it -pistoia.it -pn.it -po.it -pordenone.it -potenza .it -pr.it -prato.it -pt.it -pu.it -pv.it -pz.it -ra.it -ragusa.it -ravenna.it -rc.it -re.it +fr.it +frosinone.it +ge.it +genoa.it +genova.it +go.it +gorizia.it +gr.it +grosseto.it +iglesias-carbonia.it +iglesiascarbonia.it +im.it +imperia.it +is.it +isernia.it +kr.it +la-spezia.it +laquila.it +laspezia.it +latina.it +lc.it +le.it +lecce.it +lecco.it +li.it +livorno.it +lo.it +lodi.it +lt.it +lu.it +lucca.it +macerata.it +mantova.it +massa-carrara.it +massacarrara.it +matera.it +mb.it +mc.it +me.it +medio-campidano.it +mediocampidano.it +messina.it +mi.it +milan.it +milano.it +mn.it +mo.it +modena.it +monza-brianza.it +monza-e-della-brianza.it +monza.it +monzabrianza.it +monzaebrianza.it +monzaedellabrianza.it +ms.it +mt.it +na.it +naples.it +napoli.it +no.it +novara.it +nu.it +nuoro.it +og.it +ogliastra.it +olbia-tempio.it +olbiatempio.it +or.it +oristano.it +ot.it +pa.it +padova.it +padua.it +palermo.it +parma.it +pavia.it +pc.it +pd.it +pe.it +perugia.it +pesaro-urbino.it +pesarourbino.it +pescara.it +pg.it +pi.it +piacenza.it +pisa.it +pistoia.it +pn.it +po.it +pordenone.it +potenza.it +pr.it +prato.it +pt.it +pu.it +pv.it +pz.it +ra.it +ragusa.it +ravenna.it +rc.it +re.it reggio-calabria.it -reggio-emilia.it -reggiocalabria.it -reggioemilia.it -rg.it -ri.it -rieti.it -rimini.it -rm.it -rn.it -ro.it -roma.it -rome.it -rovigo.it -sa.it -salerno.it -sassari.it -savona.it -si.it -siena.it -siracusa.it -so.it -sondrio.it -sp.it -sr.it -ss.it -suedtirol.it -sv.it -ta.it -taranto.it -te.it -tempio-olbia.it -tempioolbia.it -teramo.it -terni.it -tn.it -to.it -torino.it -tp.it -tr.it -trani-andria-barletta.it -trani-barletta-andria.it -traniandriabarletta.it -tranibarlettaandria.it -trapani.it -trentino.it -trento.it -treviso.it -trieste.it -ts.it -turin.it -tv.it -ud.it -udine.it -urbino-pesaro.it -urbinopesaro.it -va.it -varese.it -vb.it -vc.it -ve.it -venezia.it -venice.it -verbania.it -vercelli.it -verona.it -vi.it -vibo-valentia.it -vibovalentia.it -vicenza.it -viterbo.it -vr.it -vs.it -vt.it -vv.it +reggio-emilia.it +reggiocalabria.it +reggioemilia.it +rg.it +ri.it +rieti.it +rimini.it +rm.it +rn.it +ro.it +roma.it +rome.it +rovigo.it +sa.it +salerno.it +sassari.it +savona.it +si.it +siena.it +siracusa.it +so.it +sondrio.it +sp.it +sr.it +ss.it +suedtirol.it +sv.it +ta.it +taranto.it +te.it +tempio-olbia.it +tempioolbia.it +teramo.it +terni.it +tn.it +to.it +torino.it +tp.it +tr.it +trani-andria-barletta.it +trani-barletta-andria.it +traniandriabarletta.it +tranibarlettaandria.it +trapani.it +trentino.it +trento.it +treviso.it +trieste.it +ts.it +turin.it +tv.it +ud.it +udine.it +urbino-pesaro.it +urbinopesaro.it +va.it +varese.it +vb.it +vc.it +ve.it +venezia.it +venice.it +verbania.it +vercelli.it +verona.it +vi.it +vibo-valentia.it +vibovalentia.it +vicenza.it +viterbo.it +vr.it +vs.it +vt.it +vv.it // je : http://www.channelisles.net/register-domains/ // Confirmed by registry 2013-11-28 @@ -5733,7 +5733,7 @@ gov.sd info.sd // se : http://en.wikipedia.org/wiki/.se -// Submitted by registry 2008-06-24 +// Submitted by registry 2014-03-18 se a.se ac.se @@ -5767,7 +5767,6 @@ pp.se press.se r.se s.se -sshn.se t.se tm.se u.se @@ -7535,6 +7534,126 @@ bzh // rio : 2014-02-27 Empresa Municipal de Informática SA - IPLANRIO rio +// cash : 2014-03-07 Delta Lake, LLC +cash + +// gives : 2014-03-07 United TLD Holdco Ltd. +gives + +// hiphop : 2014-03-07 Uniregistry, Corp. +hiphop + +// degree : 2014-03-07 Puff House, LLC +degree + +// digital : 2014-03-07 Dash Park, LLC +digital + +// rehab : 2014-03-07 United TLD Holdco Ltd. +rehab + +// wtf : 2014-03-07 Hidden Way, LLC +wtf + +// financial : 2014-03-07 Just Cover, LLC +financial + +// limited : 2014-03-07 Big Fest, LLC +limited + +// discount : 2014-03-07 Holly Hill, LLC +discount + +// fail : 2014-03-07 Atomic Pipe, LLC +fail + +// vet : 2014-03-07 Wild Dale, LLC +vet + +// ngo : 2014-03-07 Public Interest Registry +ngo + +// fitness : 2014-03-07 Brice Orchard, LLC +fitness + +// schule : 2014-03-07 Outer Moon, LLC +schule + +// navy : 2014-03-07 United TLD Holdco Ltd. +navy + +// bio : 2014-03-07 STARTING DOT LIMITED +bio + +// ong : 2014-03-07 Public Interest Registry +ong + +// town : 2014-03-07 Koko Moon, LLC +town + +// toys : 2014-03-07 Pioneer Orchard, LLC +toys + +// army : 2014-03-07 United TLD Holdco Ltd. +army + +// engineering : 2014-03-07 Romeo Canyon +engineering + +// capital : 2014-03-07 Delta Mill, LLC +capital + +// exchange : 2014-03-07 Spring Falls, LLC +exchange + +// fan : 2014-03-07 Goose Glen, LLC +fan + +// market : 2014-03-07 Victor Way, LLC +market + +// media : 2014-03-07 Grand Glen, LLC +media + +// lease : 2014-03-07 Victor Trail, LLC +lease + +// university : 2014-03-07 Little Station, LLC +university + +// reisen : 2014-03-07 New Cypress, LLC +reisen + +// airforce : 2014-03-07 United TLD Holdco Ltd. +airforce + +// pictures : 2014-03-07 Foggy Sky, LLC +pictures + +// gripe : 2014-03-07 Corn Sunset, LLC +gripe + +// engineering : 2014-03-07 United TLD Holdco Ltd. +engineering + +// associates : 2014-03-07 Baxter Hill, LLC +associates + +// xn--mxtq1m : 2014-03-07 Net-Chinese Co., Ltd. +政府 + +// williamhill : 2014-03-13 William Hill Organization Limited +williamhill + +// hiv : 2014-03-13 dotHIV gemeinnuetziger e.V. +hiv + +// sca : 2014-03-13 SVENSKA CELLULOSA AKTIEBOLAGET SCA (publ) +sca + +// reise : 2014-03-13 dotreise GmbH +reise + // ===END ICANN DOMAINS=== // ===BEGIN PRIVATE DOMAINS=== @@ -7604,12 +7723,12 @@ de.com eu.com gb.com gb.net -gr.com hu.com hu.net jp.net jpn.com kr.com +mex.com no.com qc.com ru.com @@ -7619,10 +7738,30 @@ se.net uk.com uk.net us.com -us.org uy.com +za.bz za.com +// Africa.com Web Solutions Ltd : https://registry.africa.com +// Submitted by Gavin Brown 2014-02-04 +africa.com + +// iDOT Services Limited : http://www.domain.gr.com +// Submitted by Gavin Brown 2014-02-04 +gr.com + +// Radix FZC : http://domains.in.net +// Submitted by Gavin Brown 2014-02-04 +in.net + +// US REGISTRY LLC : http://us.org +// Submitted by Gavin Brown 2014-02-04 +us.org + +// co.com Registry, LLC : https://registry.co.com +// Submitted by Gavin Brown 2014-02-04 +co.com + // c.la : http://www.c.la/ c.la @@ -8022,6 +8161,10 @@ nyc.mn // Submitted by Yngve Pettersen 2009-11-26 operaunite.com +// OutSystems +// Submitted by Duarte Santos 2014-03-11 +outsystemscloud.com + // Red Hat, Inc. OpenShift : https://openshift.redhat.com/ // Submitted by Tim Kramer 2012-10-24 rhcloud.com From 1efb6b75f4de712a883abf41ea238e13ff08e726 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 12 May 2014 12:20:59 +0200 Subject: [PATCH 090/104] changed code to C89 standard --- include/libpsl.h | 24 +++--- src/psl.c | 93 +++++++++++++----------- src/psl2c.c | 85 ++++++++++++---------- tests/test-is-cookie-domain-acceptable.c | 16 ++-- tests/test-is-public-all.c | 22 +++--- tests/test-is-public-builtin.c | 18 +++-- tests/test-is-public.c | 18 +++-- tests/test-registrable-domain.c | 37 +++++----- tools/psl.c | 3 +- 9 files changed, 171 insertions(+), 145 deletions(-) diff --git a/include/libpsl.h b/include/libpsl.h index 4ce408c..a0a1931 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -41,42 +41,46 @@ extern "C" { typedef struct _psl_ctx_st psl_ctx_t; +/* frees PSL context */ void psl_free(psl_ctx_t *psl); +/* loads PSL data from file */ psl_ctx_t * psl_load_file(const char *fname); +/* loads PSL data from FILE pointer */ psl_ctx_t * psl_load_fp(FILE *fp); +/* retrieves builtin PSL data */ const psl_ctx_t * psl_builtin(void); -// checks wether domain is a public suffix or not +/* checks wether domain is a public suffix or not */ int psl_is_public_suffix(const psl_ctx_t *psl, const char *domain); -// checks wether cookie_domain is acceptable for domain or not +/* checks wether cookie_domain is acceptable for domain or not */ int psl_is_cookie_domain_acceptable(const psl_ctx_t *psl, const char *hostname, const char *cookie_domain); -// returns the longest unregistrable domain within 'domain' or NULL if none found +/* returns the longest unregistrable domain within 'domain' or NULL if none found */ const char * psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain); -// returns the shortest possible registrable domain part or NULL if domain is not registrable at all +/* returns the shortest possible registrable domain part or NULL if domain is not registrable at all */ const char * psl_registrable_domain(const psl_ctx_t *psl, const char *domain); -// does not include exceptions +/* does not include exceptions */ int psl_suffix_count(const psl_ctx_t *psl); -// just counts exceptions +/* just counts exceptions */ int psl_suffix_exception_count(const psl_ctx_t *psl); -// returns compilation time +/* returns compilation time */ time_t psl_builtin_compile_time(void); -// returns mtime of PSL source file +/* returns mtime of PSL source file */ time_t psl_builtin_file_time(void); -// returns SHA1 checksum (hex-encoded, lowercase) of PSL source file +/* returns SHA1 checksum (hex-encoded, lowercase) of PSL source file */ const char * psl_builtin_sha1sum(void); -// returns file name of PSL source file +/* returns file name of PSL source file */ const char * psl_builtin_filename(void); diff --git a/src/psl.c b/src/psl.c index c5166f3..5333020 100644 --- a/src/psl.c +++ b/src/psl.c @@ -28,7 +28,7 @@ * */ -// need _GNU_SOURCE for qsort_r() +/* need _GNU_SOURCE for qsort_r() */ #ifndef _GNU_SOURCE # define _GNU_SOURCE #endif @@ -73,19 +73,19 @@ typedef struct { unsigned short length; unsigned char - nlabels, // number of labels - wildcard; // this is a wildcard rule (e.g. *.sapporo.jp) + nlabels, /* number of labels */ + wildcard; /* this is a wildcard rule (e.g. *.sapporo.jp) */ } _psl_entry_t; -// stripped down version libmget vector routines +/* stripped down version libmget vector routines */ typedef struct { int - (*cmp)(const _psl_entry_t *, const _psl_entry_t *); // comparison function + (*cmp)(const _psl_entry_t *, const _psl_entry_t *); /* comparison function */ _psl_entry_t - **entry; // pointer to array of pointers to elements + **entry; /* pointer to array of pointers to elements */ int - max, // allocated elements - cur; // number of elements in use + max, /* allocated elements */ + cur; /* number of elements in use */ } _psl_vector_t; struct _psl_ctx_st { @@ -94,10 +94,10 @@ struct _psl_ctx_st { *suffix_exceptions; }; -// include the PSL data compiled by 'psl2c' +/* include the PSL data compiled by 'psl2c' */ #include "suffixes.c" -// references to this PSL will result in lookups to built-in data +/* references to this PSL will result in lookups to built-in data */ static const psl_ctx_t _builtin_psl; @@ -140,14 +140,14 @@ static _psl_entry_t *_vector_get(const _psl_vector_t *v, int pos) return v->entry[pos]; } -// the entries must be sorted by +/* the entries must be sorted by */ static int _vector_find(const _psl_vector_t *v, const _psl_entry_t *elem) { if (v) { int l, r, m; int res; - // binary search for element (exact match) + /* binary search for element (exact match) */ for (l = 0, r = v->cur - 1; l <= r;) { m = (l + r) / 2; if ((res = v->cmp(elem, v->entry[m])) > 0) l = m + 1; @@ -156,7 +156,7 @@ static int _vector_find(const _psl_vector_t *v, const _psl_entry_t *elem) } } - return -1; // not found + return -1; /* not found */ } static int _vector_add(_psl_vector_t *v, const _psl_entry_t *elem) @@ -188,22 +188,21 @@ static void _vector_sort(_psl_vector_t *v) qsort_r(v->entry, v->cur, sizeof(_psl_vector_t *), _compare, v); } -static inline int _vector_size(_psl_vector_t *v) +static int _vector_size(_psl_vector_t *v) { return v ? v->cur : 0; } -// by this kind of sorting, we can easily see if a domain matches or not - +/* by this kind of sorting, we can easily see if a domain matches or not */ static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) { int n; if ((n = s2->nlabels - s1->nlabels)) - return n; // most labels first + return n; /* most labels first */ if ((n = s1->length - s2->length)) - return n; // shorter rules first + return n; /* shorter rules first */ return strcmp(s1->label, s2->label ? s2->label : s2->label_buf); } @@ -217,14 +216,14 @@ static int _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) if (length >= sizeof(suffix->label_buf) - 1) { suffix->nlabels = 0; - // fprintf(stderr, _("Suffix rule too long (%zd, ignored): %s\n"), length, rule); + /* fprintf(stderr, _("Suffix rule too long (%zd, ignored): %s\n"), length, rule); */ return -1; } if (*rule == '*') { if (*++rule != '.') { suffix->nlabels = 0; - // fprintf(stderr, _("Unsupported kind of rule (ignored): %s\n"), rule); + /* fprintf(stderr, _("Unsupported kind of rule (ignored): %s\n"), rule); */ return -2; } rule++; @@ -273,7 +272,7 @@ int psl_is_public_suffix(const psl_ctx_t *psl, const char *domain) if (!psl || !domain) return 1; - // this function should be called without leading dots, just make sure + /* this function should be called without leading dots, just make sure */ suffix.label = domain + (*domain == '.'); suffix.length = strlen(suffix.label); suffix.wildcard = 0; @@ -283,7 +282,7 @@ int psl_is_public_suffix(const psl_ctx_t *psl, const char *domain) if (*p == '.') suffix.nlabels++; - // if domain has enough labels, it is public + /* if domain has enough labels, it is public */ if (psl == &_builtin_psl) rule = &suffixes[0]; else @@ -298,10 +297,10 @@ int psl_is_public_suffix(const psl_ctx_t *psl, const char *domain) rule = _vector_get(psl->suffixes, _vector_find(psl->suffixes, &suffix)); if (rule) { - // definitely a match, no matter if the found rule is a wildcard or not + /* definitely a match, no matter if the found rule is a wildcard or not */ return 1; } else if (suffix.nlabels == 1) { - // unknown TLD, this is the prevailing '*' match + /* unknown TLD, this is the prevailing '*' match */ return 1; } @@ -320,17 +319,17 @@ int psl_is_public_suffix(const psl_ctx_t *psl, const char *domain) if (rule) { if (rule->wildcard) { - // now that we matched a wildcard, we have to check for an exception + /* now that we matched a wildcard, we have to check for an exception */ suffix.label = label_bak; suffix.length = length_bak; suffix.nlabels++; if (psl == &_builtin_psl) { if (bsearch(&suffix, suffix_exceptions, countof(suffix_exceptions), sizeof(suffix_exceptions[0]), (int(*)(const void *, const void *))_suffix_compare)) - return 0; // found an exception, so 'domain' is not a public suffix + return 0; /* found an exception, so 'domain' is not a public suffix */ } else { if (_vector_get(psl->suffix_exceptions, _vector_find(psl->suffix_exceptions, &suffix)) != 0) - return 0; // found an exception, so 'domain' is not a public suffix + return 0; /* found an exception, so 'domain' is not a public suffix */ } return 1; @@ -362,14 +361,16 @@ const char *psl_unregistrable_domain(const psl_ctx_t *psl, const char *domain) if (!psl || !domain) return NULL; - // We check from left to right to catch special PSL entries like 'forgot.his.name': - // 'forgot.his.name' and 'name' are in the PSL while 'his.name' is not. + /* + * We check from left to right to catch special PSL entries like 'forgot.his.name': + * 'forgot.his.name' and 'name' are in the PSL while 'his.name' is not. + */ while (!psl_is_public_suffix(psl, domain)) { if ((domain = strchr(domain, '.'))) domain++; else - break; // prevent endless loop if psl_is_public_suffix() is broken. + break; /* prevent endless loop if psl_is_public_suffix() is broken. */ } return domain; @@ -398,15 +399,17 @@ const char *psl_registrable_domain(const psl_ctx_t *psl, const char *domain) if (!psl || !domain || *domain == '.') return NULL; - // We check from left to right to catch special PSL entries like 'forgot.his.name': - // 'forgot.his.name' and 'name' are in the PSL while 'his.name' is not. + /* + * We check from left to right to catch special PSL entries like 'forgot.his.name': + * 'forgot.his.name' and 'name' are in the PSL while 'his.name' is not. + */ while (!psl_is_public_suffix(psl, domain)) { if ((p = strchr(domain, '.'))) { regdom = domain; domain = p + 1; } else - break; // prevent endless loop if psl_is_public_suffix() is broken. + break; /* prevent endless loop if psl_is_public_suffix() is broken. */ } return regdom; @@ -473,24 +476,26 @@ psl_ctx_t *psl_load_fp(FILE *fp) if (!(psl = calloc(1, sizeof(psl_ctx_t)))) return NULL; - // as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. - // as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. + /* + * as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. + * as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. + */ psl->suffixes = _vector_alloc(8*1024, _suffix_compare); psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); while ((linep = fgets(buf, sizeof(buf), fp))) { - while (isspace(*linep)) linep++; // ignore leading whitespace - if (!*linep) continue; // skip empty lines + while (isspace(*linep)) linep++; /* ignore leading whitespace */ + if (!*linep) continue; /* skip empty lines */ if (*linep == '/' && linep[1] == '/') - continue; // skip comments + continue; /* skip comments */ - // parse suffix rule + /* parse suffix rule */ for (p = linep; *linep && !isspace(*linep);) linep++; *linep = 0; if (*p == '!') { - // add to exceptions + /* add to exceptions */ if (_suffix_init(&suffix, p + 1, linep - p - 1) == 0) suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); else @@ -503,7 +508,7 @@ psl_ctx_t *psl_load_fp(FILE *fp) } if (suffixp) - suffixp->label = suffixp->label_buf; // set label to changed address + suffixp->label = suffixp->label_buf; /* set label to changed address */ nsuffixes++;; } @@ -697,17 +702,17 @@ int psl_is_cookie_domain_acceptable(const psl_ctx_t *psl, const char *hostname, cookie_domain++; if (!strcmp(hostname, cookie_domain)) - return 1; // an exact match is acceptable (and pretty common) + return 1; /* an exact match is acceptable (and pretty common) */ cookie_domain_length = strlen(cookie_domain); hostname_length = strlen(hostname); if (cookie_domain_length >= hostname_length) - return 0; // cookie_domain is too long + return 0; /* cookie_domain is too long */ p = hostname + hostname_length - cookie_domain_length; if (!strcmp(p, cookie_domain) && p[-1] == '.') { - // OK, cookie_domain matches, but it must be longer than the longest public suffix in 'hostname' + /* OK, cookie_domain matches, but it must be longer than the longest public suffix in 'hostname' */ if (!(p = psl_unregistrable_domain(psl, hostname))) return 1; diff --git a/src/psl2c.c b/src/psl2c.c index e55301c..48643bc 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -39,9 +39,11 @@ #include #include -//#ifdef WITH_LIBIDN2 -//# include -//#endif +/* +#ifdef WITH_LIBIDN2 +# include +#endif +*/ #ifdef WITH_BUILTIN @@ -55,19 +57,19 @@ typedef struct { unsigned short length; unsigned char - nlabels, // number of labels - wildcard; // this is a wildcard rule (e.g. *.sapporo.jp) + nlabels, /* number of labels */ + wildcard; /* this is a wildcard rule (e.g. *.sapporo.jp) */ } _psl_entry_t; -// stripped down version libmget vector routines +/* stripped down version libmget vector routines */ typedef struct { int - (*cmp)(const _psl_entry_t *, const _psl_entry_t *); // comparison function + (*cmp)(const _psl_entry_t *, const _psl_entry_t *); /* comparison function */ _psl_entry_t - **entry; // pointer to array of pointers to elements + **entry; /* pointer to array of pointers to elements */ int - max, // allocated elements - cur; // number of elements in use + max, /* allocated elements */ + cur; /* number of elements in use */ } _psl_vector_t; struct _psl_ctx_st { @@ -144,17 +146,17 @@ static void _vector_sort(_psl_vector_t *v) qsort_r(v->entry, v->cur, sizeof(_psl_vector_t *), _compare, v); } -// by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) +/* by this kind of sorting, we can easily see if a domain matches or not (match = supercookie !) */ static int _suffix_compare(const _psl_entry_t *s1, const _psl_entry_t *s2) { int n; if ((n = s2->nlabels - s1->nlabels)) - return n; // most labels first + return n; /* most labels first */ if ((n = s1->length - s2->length)) - return n; // shorter rules first + return n; /* shorter rules first */ return strcmp(s1->label, s2->label); } @@ -168,7 +170,7 @@ static void _suffix_init(_psl_entry_t *suffix, const char *rule, size_t length) if (length >= sizeof(suffix->label_buf) - 1) { suffix->nlabels = 0; - fprintf(stderr, "Suffix rule too long (%zd, ignored): %s\n", length, rule); + fprintf(stderr, "Suffix rule too long (%d, ignored): %s\n", (int) length, rule); return; } @@ -222,24 +224,26 @@ psl_ctx_t *psl_load_fp(FILE *fp) if (!(psl = calloc(1, sizeof(psl_ctx_t)))) return NULL; - // as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. - // as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. + /* + * as of 02.11.2012, the list at http://publicsuffix.org/list/ contains ~6000 rules and 40 exceptions. + * as of 19.02.2014, the list at http://publicsuffix.org/list/ contains ~6500 rules and 19 exceptions. + */ psl->suffixes = _vector_alloc(8*1024, _suffix_compare); psl->suffix_exceptions = _vector_alloc(64, _suffix_compare); while ((linep = fgets(buf, sizeof(buf), fp))) { - while (isspace(*linep)) linep++; // ignore leading whitespace - if (!*linep) continue; // skip empty lines + while (isspace(*linep)) linep++; /* ignore leading whitespace */ + if (!*linep) continue; /* skip empty lines */ if (*linep == '/' && linep[1] == '/') - continue; // skip comments + continue; /* skip comments */ - // parse suffix rule + /* parse suffix rule */ for (p = linep; *linep && !isspace(*linep);) linep++; *linep = 0; if (*p == '!') { - // add to exceptions + /* add to exceptions */ _suffix_init(&suffix, p + 1, linep - p - 1); suffixp = _vector_get(psl->suffix_exceptions, _vector_add(psl->suffix_exceptions, &suffix)); } else { @@ -248,7 +252,7 @@ psl_ctx_t *psl_load_fp(FILE *fp) } if (suffixp) - suffixp->label = suffixp->label_buf; // set label to changed address + suffixp->label = suffixp->label_buf; /* set label to changed address */ nsuffixes++;; } @@ -263,14 +267,14 @@ static void _print_psl_entries(FILE *fpout, const _psl_vector_t *v, const char * { int it; - fprintf(fpout, "// automatically generated by psl2c\n"); + fprintf(fpout, "/* automatically generated by psl2c */\n"); fprintf(fpout, "static _psl_entry_t %s[] = {\n", varname); for (it = 0; it < v->cur; it++) { _psl_entry_t *e = _vector_get(v, it); - fprintf(fpout, "\t{ \"%s\", NULL, %hd, %hhd, %hhd },\n", - e->label_buf, e->length, e->nlabels, e->wildcard); + fprintf(fpout, "\t{ \"%s\", NULL, %hd, %d, %d },\n", + e->label_buf, e->length, (int) e->nlabels, (int) e->wildcard); } fprintf(fpout, "};\n"); @@ -296,14 +300,14 @@ static void _add_punycode_if_needed(_psl_vector_t *v) { int it, n; - // do not use 'it < v->cur' since v->cur is changed by _vector_add() ! + /* do not use 'it < v->cur' since v->cur is changed by _vector_add() ! */ for (it = 0, n = v->cur; it < n; it++) { _psl_entry_t *e = _vector_get(v, it); if (_str_needs_encoding(e->label_buf)) { _psl_entry_t suffix, *suffixp; - // the following lines will have GPL3+ license issues + /* the following lines will have GPL3+ license issues */ /* char *asc = NULL; int rc; @@ -317,17 +321,17 @@ static void _add_punycode_if_needed(_psl_vector_t *v) fprintf(stderr, "toASCII(%s) failed (%d): %s\n", e->label_buf, rc, idn2_strerror(rc)); */ - // this is much slower than the libidn2 API but should have no license issues + /* this is much slower than the libidn2 API but should have no license issues */ FILE *pp; - char cmd[16 + strlen(e->label_buf)], lookupname[64] = ""; + char cmd[16 + sizeof(e->label_buf)], lookupname[64] = ""; snprintf(cmd, sizeof(cmd), "idn2 '%s'", e->label_buf); if ((pp = popen(cmd, "r"))) { if (fscanf(pp, "%63s", lookupname) >= 1 && strcmp(e->label_buf, lookupname)) { - // fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, lookupname); + /* fprintf(stderr, "idn2 '%s' -> '%s'\n", e->label_buf, lookupname); */ _suffix_init(&suffix, lookupname, strlen(lookupname)); suffix.wildcard = e->wildcard; suffixp = _vector_get(v, _vector_add(v, &suffix)); - suffixp->label = suffixp->label_buf; // set label to changed address + suffixp->label = suffixp->label_buf; /* set label to changed address */ } pclose(pp); } else @@ -337,7 +341,7 @@ static void _add_punycode_if_needed(_psl_vector_t *v) _vector_sort(v); } -#endif // WITH_BUILTIN +#endif /* WITH_BUILTIN */ int main(int argc, const char **argv) { @@ -361,7 +365,7 @@ int main(int argc, const char **argv) if ((fpout = fopen(argv[2], "w"))) { FILE *pp; struct stat st; - char cmd[16 + strlen(argv[1])], checksum[64] = ""; + char *cmd, checksum[64] = ""; _add_punycode_if_needed(psl->suffixes); _add_punycode_if_needed(psl->suffix_exceptions); @@ -369,11 +373,14 @@ int main(int argc, const char **argv) _print_psl_entries(fpout, psl->suffixes, "suffixes"); _print_psl_entries(fpout, psl->suffix_exceptions, "suffix_exceptions"); - snprintf(cmd, sizeof(cmd), "sha1sum %s", argv[1]); - if ((pp = popen(cmd, "r"))) { - if (fscanf(pp, "%63[0-9a-zA-Z]", checksum) < 1) - *checksum = 0; - pclose(pp); + if ((cmd = malloc(16 + strlen(argv[1])))) { + snprintf(cmd, 16 + strlen(argv[1]), "sha1sum %s", argv[1]); + if ((pp = popen(cmd, "r"))) { + if (fscanf(pp, "%63[0-9a-zA-Z]", checksum) < 1) + *checksum = 0; + pclose(pp); + } + free(cmd); } if (stat(argv[1], &st) != 0) @@ -407,7 +414,7 @@ int main(int argc, const char **argv) ret = 3; } -#endif // WITH_BUILTIN +#endif /* WITH_BUILTIN */ return ret; } diff --git a/tests/test-is-cookie-domain-acceptable.c b/tests/test-is-cookie-domain-acceptable.c index 6abd273..79b8f34 100644 --- a/tests/test-is-cookie-domain-acceptable.c +++ b/tests/test-is-cookie-domain-acceptable.c @@ -35,6 +35,7 @@ #include #include #include +#include #include @@ -46,8 +47,6 @@ static int static void test_psl(void) { - // punycode generation: idn 商标 - // octal code generation: echo -n "商标" | od -b static const struct test_data { const char *request_domain, @@ -65,10 +64,10 @@ static void test_psl(void) { "www.his.name", "name", 0 }, { "www.example.com", "www.example.com", 1 }, { "www.example.com", "example.com", 1 }, - { "www.example.com", "com", 0 }, // not accepted by normalization (PSL rule 'com') + { "www.example.com", "com", 0 }, /* not accepted by normalization (PSL rule 'com') */ { "www.example.com", "example.org", 0 }, - { "www.sa.gov.au", "sa.gov.au", 0 }, // not accepted by normalization (PSL rule '*.ar') - { "www.educ.ar", "educ.ar", 1 }, // PSL exception rule '!educ.ar' + { "www.sa.gov.au", "sa.gov.au", 0 }, /* not accepted by normalization (PSL rule '*.ar') */ + { "www.educ.ar", "educ.ar", 1 }, /* PSL exception rule '!educ.ar' */ }; unsigned it; psl_ctx_t *psl; @@ -95,14 +94,15 @@ static void test_psl(void) int main(int argc, const char * const *argv) { - // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + /* if VALGRIND testing is enabled, we have to call ourselves with valgrind checking */ if (argc == 1) { const char *valgrind = getenv("TESTS_VALGRIND"); if (valgrind && *valgrind) { - char cmd[strlen(valgrind)+strlen(argv[0])+32]; + size_t cmdsize = strlen(valgrind) + strlen(argv[0]) + 32; + char *cmd = alloca(cmdsize); - snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + snprintf(cmd, cmdsize, "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); return system(cmd) != 0; } } diff --git a/tests/test-is-public-all.c b/tests/test-is-public-all.c index 91f6098..5889642 100644 --- a/tests/test-is-public-all.c +++ b/tests/test-is-public-all.c @@ -36,6 +36,7 @@ #include #include #include +#include #include @@ -50,23 +51,23 @@ static void test_psl(void) int result; char buf[256], domain[64], *linep, *p; - psl = psl_load_file(PSL_FILE); // PSL_FILE can be set by ./configure --with-psl-file=[PATH] + psl = psl_load_file(PSL_FILE); /* PSL_FILE can be set by ./configure --with-psl-file=[PATH] */ printf("loaded %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); if ((fp = fopen(PSL_FILE, "r"))) { while ((linep = fgets(buf, sizeof(buf), fp))) { - while (isspace(*linep)) linep++; // ignore leading whitespace - if (!*linep) continue; // skip empty lines + while (isspace(*linep)) linep++; /* ignore leading whitespace */ + if (!*linep) continue; /* skip empty lines */ if (*linep == '/' && linep[1] == '/') - continue; // skip comments + continue; /* skip comments */ - // parse suffix rule + /* parse suffix rule */ for (p = linep; *linep && !isspace(*linep);) linep++; *linep = 0; - if (*p == '!') { // an exception to a wildcard, e.g. !www.ck (wildcard is *.ck) + if (*p == '!') { /* an exception to a wildcard, e.g. !www.ck (wildcard is *.ck) */ if ((result = psl_is_public_suffix(psl, p + 1))) { failed++; printf("psl_is_public_suffix(%s)=%d (expected 0)\n", p, result); @@ -77,7 +78,7 @@ static void test_psl(void) printf("psl_is_public_suffix(%s)=%d (expected 1)\n", strchr(p, '.') + 1, result); } else ok++; } - else if (*p == '*') { // a wildcard, e.g. *.ck + else if (*p == '*') { /* a wildcard, e.g. *.ck */ if (!(result = psl_is_public_suffix(psl, p + 1))) { failed++; printf("psl_is_public_suffix(%s)=%d (expected 1)\n", p + 1, result); @@ -114,14 +115,15 @@ static void test_psl(void) int main(int argc, const char * const *argv) { - // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + /* if VALGRIND testing is enabled, we have to call ourselves with valgrind checking */ if (argc == 1) { const char *valgrind = getenv("TESTS_VALGRIND"); if (valgrind && *valgrind) { - char cmd[strlen(valgrind) + strlen(argv[0]) + 32]; + size_t cmdsize = strlen(valgrind) + strlen(argv[0]) + 32; + char *cmd = alloca(cmdsize); - snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + snprintf(cmd, cmdsize, "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); return system(cmd) != 0; } } diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index e4ff3fc..3c9e5f4 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -35,6 +35,7 @@ #include #include #include +#include #include @@ -46,8 +47,8 @@ static int static void test_psl(void) { - // punycode generation: idn 商标 - // octal code generation: echo -n "商标" | od -b + /* punycode generation: idn 商标 */ + /* octal code generation: echo -n "商标" | od -b */ static const struct test_data { const char *domain; @@ -60,15 +61,15 @@ static void test_psl(void) { "cc.ar.us", 1 }, { ".cc.ar.us", 1 }, { "www.cc.ar.us", 0 }, - { "www.ck", 0 }, // exception from *.ck + { "www.ck", 0 }, /* exception from *.ck */ { "abc.www.ck", 0 }, { "xxx.ck", 1 }, { "www.xxx.ck", 0 }, - { "\345\225\206\346\240\207", 1 }, // xn--czr694b oder 商标 + { "\345\225\206\346\240\207", 1 }, /* xn--czr694b oder 商标 */ { "www.\345\225\206\346\240\207", 0 }, { "xn--czr694b", 1 }, { "www.xn--czr694b", 0 }, - // some special test follow ('name' and 'forgot.his.name' are public, but e.g. his.name is not) + /* some special test follow ('name' and 'forgot.his.name' are public, but e.g. his.name is not) */ { "name", 1 }, { ".name", 1 }, { "his.name", 0 }, @@ -109,14 +110,15 @@ static void test_psl(void) int main(int argc, const char * const *argv) { - // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + /* if VALGRIND testing is enabled, we have to call ourselves with valgrind checking */ if (argc == 1) { const char *valgrind = getenv("TESTS_VALGRIND"); if (valgrind && *valgrind) { - char cmd[strlen(valgrind)+strlen(argv[0])+32]; + size_t cmdsize = strlen(valgrind) + strlen(argv[0]) + 32; + char *cmd = alloca(cmdsize); - snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + snprintf(cmd, cmdsize, "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); return system(cmd) != 0; } } diff --git a/tests/test-is-public.c b/tests/test-is-public.c index bf11d7e..d4a2c28 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -35,6 +35,7 @@ #include #include #include +#include #include @@ -46,8 +47,8 @@ static int static void test_psl(void) { - // punycode generation: idn 商标 - // octal code generation: echo -n "商标" | od -b + /* punycode generation: idn 商标 */ + /* octal code generation: echo -n "商标" | od -b */ static const struct test_data { const char *domain; @@ -60,13 +61,13 @@ static void test_psl(void) { "cc.ar.us", 1 }, { ".cc.ar.us", 1 }, { "www.cc.ar.us", 0 }, - { "www.ck", 0 }, // exception from *.ck + { "www.ck", 0 }, /* exception from *.ck */ { "abc.www.ck", 0 }, { "xxx.ck", 1 }, { "www.xxx.ck", 0 }, - { "\345\225\206\346\240\207", 1 }, // xn--czr694b oder 商标 + { "\345\225\206\346\240\207", 1 }, /* xn--czr694b oder 商标 */ { "www.\345\225\206\346\240\207", 0 }, - // some special test follow ('name' and 'forgot.his.name' are public, but e.g. his.name is not) + /* some special test follow ('name' and 'forgot.his.name' are public, but e.g. his.name is not) */ { "name", 1 }, { ".name", 1 }, { "his.name", 0 }, @@ -100,14 +101,15 @@ static void test_psl(void) int main(int argc, const char * const *argv) { - // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + /* if VALGRIND testing is enabled, we have to call ourselves with valgrind checking */ if (argc == 1) { const char *valgrind = getenv("TESTS_VALGRIND"); if (valgrind && *valgrind) { - char cmd[strlen(valgrind)+strlen(argv[0])+32]; + size_t cmdsize = strlen(valgrind) + strlen(argv[0]) + 32; + char *cmd = alloca(cmdsize); - snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + snprintf(cmd, cmdsize, "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); return system(cmd) != 0; } } diff --git a/tests/test-registrable-domain.c b/tests/test-registrable-domain.c index 17bed2f..c075500 100644 --- a/tests/test-registrable-domain.c +++ b/tests/test-registrable-domain.c @@ -36,6 +36,7 @@ #include #include #include +#include #include @@ -48,7 +49,7 @@ static void test(const psl_ctx_t *psl, const char *domain, const char *expected_ const char *result; char lookupname[128]; - // check if there might be some utf-8 characters + /* check if there might be some utf-8 characters */ if (domain) { int utf8; const char *p; @@ -57,13 +58,14 @@ static void test(const psl_ctx_t *psl, const char *domain, const char *expected_ if (*p < 0) utf8 = 1; - // if we found utf-8, make sure to convert domain correctly to lowercase - // does it work, if we are not in a utf-8 env ? + /* if we found utf-8, make sure to convert domain correctly to lowercase */ + /* does it work, if we are not in a utf-8 env ? */ if (utf8) { FILE *pp; - char cmd[48 + strlen(domain)]; + size_t cmdsize = 48 + strlen(domain); + char *cmd = alloca(cmdsize); - snprintf(cmd, sizeof(cmd), "echo -n '%s' | sed -e 's/./\\L\\0/g'", domain); + snprintf(cmd, cmdsize, "echo -n '%s' | sed -e 's/./\\L\\0/g'", domain); if ((pp = popen(cmd, "r"))) { if (fscanf(pp, "%127s", lookupname) >= 1) domain = lookupname; @@ -93,28 +95,28 @@ static void test_psl(void) printf("have %d suffixes and %d exceptions\n", psl_suffix_count(psl), psl_suffix_exception_count(psl)); - // special check with NULL values + /* special check with NULL values */ test(NULL, NULL, NULL); - // special check with NULL psl context + /* special check with NULL psl context */ test(NULL, "www.example.com", NULL); - // special check with NULL psl context and TLD + /* special check with NULL psl context and TLD */ test(NULL, "com", NULL); - // Norwegian with uppercase oe + /* Norwegian with uppercase oe */ test(psl, "www.\303\230yer.no", "www.\303\270yer.no"); - // Norwegian with lowercase oe + /* Norwegian with lowercase oe */ test(psl, "www.\303\270yer.no", "www.\303\270yer.no"); - // special check with NULL psl context and TLD + /* special check with NULL psl context and TLD */ test(psl, "whoever.forgot.his.name", "whoever.forgot.his.name"); - // special check with NULL psl context and TLD + /* special check with NULL psl context and TLD */ test(psl, "forgot.his.name", NULL); - // special check with NULL psl context and TLD + /* special check with NULL psl context and TLD */ test(psl, "his.name", "his.name"); if ((fp = fopen(PSL_TESTFILE, "r"))) { @@ -124,7 +126,7 @@ static void test_psl(void) continue; } - // we have to lowercase the domain - the PSL API just takes lowercase + /* we have to lowercase the domain - the PSL API just takes lowercase */ for (p = domain; *p; p++) if (*p > 0 && isupper(*p)) *p = tolower(*p); @@ -144,14 +146,15 @@ static void test_psl(void) int main(int argc, const char * const *argv) { - // if VALGRIND testing is enabled, we have to call ourselves with valgrind checking + /* if VALGRIND testing is enabled, we have to call ourselves with valgrind checking */ if (argc == 1) { const char *valgrind = getenv("TESTS_VALGRIND"); if (valgrind && *valgrind) { - char cmd[strlen(valgrind) + strlen(argv[0]) + 32]; + size_t cmdsize = strlen(valgrind) + strlen(argv[0]) + 32; + char *cmd = alloca(cmdsize); - snprintf(cmd, sizeof(cmd), "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); + snprintf(cmd, cmdsize, "TESTS_VALGRIND="" %s %s", valgrind, argv[0]); return system(cmd) != 0; } } diff --git a/tools/psl.c b/tools/psl.c index 7c53dce..718e97e 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -54,12 +54,13 @@ static void usage(int err) exit(err); } +/* RFC 2822-compliant date format */ static const char *time2str(time_t t) { static char buf[64]; struct tm *tp = localtime(&t); - strftime(buf, sizeof(buf), "%a, %d %b %Y %T %z", tp); + strftime(buf, sizeof(buf), "%a, %d %b %Y %H:%M:%S %Z", tp); return buf; } From 7d3e2eebb2f560424f7cae5ab84d8cdd5e12f245 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 12 May 2014 12:27:32 +0200 Subject: [PATCH 091/104] use alloca instead of malloc for command buffer allocation --- src/psl2c.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/src/psl2c.c b/src/psl2c.c index 48643bc..c3364fa 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -365,7 +365,8 @@ int main(int argc, const char **argv) if ((fpout = fopen(argv[2], "w"))) { FILE *pp; struct stat st; - char *cmd, checksum[64] = ""; + size_t cmdsize = 16 + strlen(argv[1]); + char *cmd = alloca(cmdsize), checksum[64] = ""; _add_punycode_if_needed(psl->suffixes); _add_punycode_if_needed(psl->suffix_exceptions); @@ -373,14 +374,11 @@ int main(int argc, const char **argv) _print_psl_entries(fpout, psl->suffixes, "suffixes"); _print_psl_entries(fpout, psl->suffix_exceptions, "suffix_exceptions"); - if ((cmd = malloc(16 + strlen(argv[1])))) { - snprintf(cmd, 16 + strlen(argv[1]), "sha1sum %s", argv[1]); - if ((pp = popen(cmd, "r"))) { - if (fscanf(pp, "%63[0-9a-zA-Z]", checksum) < 1) - *checksum = 0; - pclose(pp); - } - free(cmd); + snprintf(cmd, cmdsize, "sha1sum %s", argv[1]); + if ((pp = popen(cmd, "r"))) { + if (fscanf(pp, "%63[0-9a-zA-Z]", checksum) < 1) + *checksum = 0; + pclose(pp); } if (stat(argv[1], &st) != 0) From 61753f43ab82de44e6b9fccae4191d8278574556 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 12 May 2014 14:15:40 +0200 Subject: [PATCH 092/104] added special test cases '.', empty string, NULL, unknown TLD --- tests/test-is-public-builtin.c | 4 ++++ tests/test-is-public.c | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/tests/test-is-public-builtin.c b/tests/test-is-public-builtin.c index 3c9e5f4..ffcac12 100644 --- a/tests/test-is-public-builtin.c +++ b/tests/test-is-public-builtin.c @@ -78,6 +78,10 @@ static void test_psl(void) { ".forgot.his.name", 1 }, { "whoever.his.name", 0 }, { "whoever.forgot.his.name", 0 }, + { ".", 1 }, /* special case */ + { "", 1 }, /* special case */ + { NULL, 1 }, /* special case */ + { "adfhoweirh", 1 }, /* unknown TLD */ }; unsigned it; const psl_ctx_t *psl; diff --git a/tests/test-is-public.c b/tests/test-is-public.c index d4a2c28..1f3b8b3 100644 --- a/tests/test-is-public.c +++ b/tests/test-is-public.c @@ -76,6 +76,10 @@ static void test_psl(void) { ".forgot.his.name", 1 }, { "whoever.his.name", 0 }, { "whoever.forgot.his.name", 0 }, + { ".", 1 }, /* special case */ + { "", 1 }, /* special case */ + { NULL, 1 }, /* special case */ + { "adfhoweirh", 1 }, /* unknown TLD */ }; unsigned it; psl_ctx_t *psl; From b07c37b7c6057c21b5df7405e1ec1d63a3e09f44 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 26 May 2014 11:13:52 +0200 Subject: [PATCH 093/104] build static library by default --- configure.ac | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 9ca46bd..7b54d90 100644 --- a/configure.ac +++ b/configure.ac @@ -8,7 +8,9 @@ AM_INIT_AUTOMAKE([1.10 -Wall no-define]) # the library. AC_CONFIG_HEADERS([config.h]) AC_PROG_CXX -LT_INIT([disable-static]) +m4_ifdef([AM_PROG_AR], [AM_PROG_AR]) +#LT_INIT([disable-static]) +LT_INIT AC_CONFIG_MACRO_DIR([m4]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) From 2ce72b5e3b73fc55753b79b4597bfeb1c0df95bf Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Mon, 26 May 2014 11:32:46 +0200 Subject: [PATCH 094/104] Release V0.2.2 --- NEWS | 5 +++++ configure.ac | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/NEWS b/NEWS index 1b9ebc9..2e79e4f 100644 --- a/NEWS +++ b/NEWS @@ -1,5 +1,10 @@ Copyright (C) 2014 Tim Ruehsen +26.05.2014 Release V0.2.2 + * changed code to C89 + * added a few test cases + * build static library by default + 25.04.2014 Hotfix release V0.2.1 * Updated to the latest Publix Suffix List diff --git a/configure.ac b/configure.ac index 7b54d90..c304310 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([libpsl], [0.2.1], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.2.2], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define]) @@ -63,8 +63,8 @@ AS_IF([ test "$enable_man" != no ], [ # 4. If any interfaces have been added, removed, or changed since the last update, increment current, and set revision to 0. # 5. If any interfaces have been added since the last public release, then increment age. # 6. If any interfaces have been removed or changed since the last public release, then set age to 0. -AC_SUBST([LIBPSL_SO_VERSION], [0:2:0]) -AC_SUBST([LIBPSL_API_VERSION], [0.2.1]) +AC_SUBST([LIBPSL_SO_VERSION], [0:3:0]) +AC_SUBST([LIBPSL_API_VERSION], [0.2.2]) # Check for idn2 AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) From e9be57241d848c5914874e1de28702e489da3491 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Tue, 27 May 2014 11:57:45 +0200 Subject: [PATCH 095/104] API version 0.2 less verbose --- NEWS | 3 +++ configure.ac | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/NEWS b/NEWS index 2e79e4f..9d7a431 100644 --- a/NEWS +++ b/NEWS @@ -1,5 +1,8 @@ Copyright (C) 2014 Tim Ruehsen +27.05.2014 Release V0.2.3 + * changed API version to 0.2 + 26.05.2014 Release V0.2.2 * changed code to C89 * added a few test cases diff --git a/configure.ac b/configure.ac index c304310..8615f71 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([libpsl], [0.2.2], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.2.3], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define]) @@ -64,7 +64,7 @@ AS_IF([ test "$enable_man" != no ], [ # 5. If any interfaces have been added since the last public release, then increment age. # 6. If any interfaces have been removed or changed since the last public release, then set age to 0. AC_SUBST([LIBPSL_SO_VERSION], [0:3:0]) -AC_SUBST([LIBPSL_API_VERSION], [0.2.2]) +AC_SUBST([LIBPSL_API_VERSION], [0.2]) # Check for idn2 AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) From 0eeec32ee3f63dcd0d945b537e820ee1b56ebeb3 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 30 May 2014 16:08:47 +0200 Subject: [PATCH 096/104] fix return value of psl_builtin() to NULL if no builtin PSL data available --- src/psl.c | 4 ++++ tools/psl.c | 10 ++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/src/psl.c b/src/psl.c index 5333020..77ed564 100644 --- a/src/psl.c +++ b/src/psl.c @@ -557,7 +557,11 @@ void psl_free(psl_ctx_t *psl) */ const psl_ctx_t *psl_builtin(void) { +#ifdef WITH_BUILTIN return &_builtin_psl; +#else + return NULL; +#endif } /** diff --git a/tools/psl.c b/tools/psl.c index 718e97e..841340c 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -90,7 +90,8 @@ int main(int argc, const char *const *argv) fprintf(stderr, "Dropped data from %s\n", psl_file); psl_file = NULL; } - psl = (psl_ctx_t *) psl_builtin(); + if (!(psl = (psl_ctx_t *) psl_builtin())) + printf("No builtin PSL data available\n"); } else if (!strcmp(*arg, "--load-psl-file") && arg < argv + argc - 1) { psl_free(psl); @@ -118,6 +119,11 @@ int main(int argc, const char *const *argv) break; } + if (!psl && mode != 99) { + printf("No PSL data available - aborting\n"); + exit(2); + } + if (mode == 1) { for (; arg < argv + argc; arg++) printf("%s: %d\n", *arg, psl_is_public_suffix(psl, *arg)); @@ -135,7 +141,7 @@ int main(int argc, const char *const *argv) printf("%s: %d\n", *arg, psl_is_cookie_domain_acceptable(psl, *arg, cookie_domain)); } else if (mode == 99) { - if (psl != psl_builtin()) { + if (psl && psl != psl_builtin()) { printf("suffixes: %d\n", psl_suffix_count(psl)); printf("exceptions: %d\n", psl_suffix_exception_count(psl)); } From 57b53c990cb82a1a028fddffee7230081279ec23 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Fri, 30 May 2014 17:18:59 +0200 Subject: [PATCH 097/104] Release V0.2.4 --- NEWS | 3 +++ configure.ac | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/NEWS b/NEWS index 9d7a431..c54d3c8 100644 --- a/NEWS +++ b/NEWS @@ -1,5 +1,8 @@ Copyright (C) 2014 Tim Ruehsen +30.05.2014 + * Fixed psl_builtin() to return NULL if no built-in PSL data is available + 27.05.2014 Release V0.2.3 * changed API version to 0.2 diff --git a/configure.ac b/configure.ac index 8615f71..e87579b 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([libpsl], [0.2.3], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.2.4], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define]) @@ -63,7 +63,7 @@ AS_IF([ test "$enable_man" != no ], [ # 4. If any interfaces have been added, removed, or changed since the last update, increment current, and set revision to 0. # 5. If any interfaces have been added since the last public release, then increment age. # 6. If any interfaces have been removed or changed since the last public release, then set age to 0. -AC_SUBST([LIBPSL_SO_VERSION], [0:3:0]) +AC_SUBST([LIBPSL_SO_VERSION], [0:4:0]) AC_SUBST([LIBPSL_API_VERSION], [0.2]) # Check for idn2 From cadb9634758c823728cc2d026a8286358b5f90b9 Mon Sep 17 00:00:00 2001 From: Daniel Kahn Gillmor Date: Sat, 31 May 2014 18:56:07 +0200 Subject: [PATCH 098/104] remove LIBPSL_API_VERSION for simplicity --- Makefile.am | 2 +- configure.ac | 4 ++-- docs/libpsl/version.xml.in | 2 +- libpsl.pc.in | 4 ++-- src/Makefile.am | 8 ++++---- tests/Makefile.am | 2 +- tools/Makefile.am | 2 +- 7 files changed, 12 insertions(+), 12 deletions(-) diff --git a/Makefile.am b/Makefile.am index 1523ed7..bc329fa 100644 --- a/Makefile.am +++ b/Makefile.am @@ -11,6 +11,6 @@ DISTCHECK_CONFIGURE_FLAGS = --enable-gtk-doc --enable-man ## pkg-config files are also used for architecture-independent data packages, ## in which case the correct install location would be $(datadir)/pkgconfig. pkgconfigdir = $(libdir)/pkgconfig -pkgconfig_DATA = libpsl-$(LIBPSL_API_VERSION).pc +pkgconfig_DATA = libpsl.pc EXTRA_DIST = config.rpath diff --git a/configure.ac b/configure.ac index e87579b..7712005 100644 --- a/configure.ac +++ b/configure.ac @@ -64,7 +64,7 @@ AS_IF([ test "$enable_man" != no ], [ # 5. If any interfaces have been added since the last public release, then increment age. # 6. If any interfaces have been removed or changed since the last public release, then set age to 0. AC_SUBST([LIBPSL_SO_VERSION], [0:4:0]) -AC_SUBST([LIBPSL_API_VERSION], [0.2]) +AC_SUBST([LIBPSL_VERSION], $VERSION) # Check for idn2 AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) @@ -126,7 +126,7 @@ AC_CONFIG_FILES([Makefile docs/libpsl/version.xml data/Makefile tests/Makefile - libpsl-${LIBPSL_API_VERSION}.pc:libpsl.pc.in]) + libpsl.pc:libpsl.pc.in]) AC_OUTPUT AC_MSG_NOTICE([Summary of build options: diff --git a/docs/libpsl/version.xml.in b/docs/libpsl/version.xml.in index 3f2b373..72fdcab 100644 --- a/docs/libpsl/version.xml.in +++ b/docs/libpsl/version.xml.in @@ -1 +1 @@ -@LIBPSL_API_VERSION@ +@LIBPSL_VERSION@ diff --git a/libpsl.pc.in b/libpsl.pc.in index 13ffa3c..b04a358 100644 --- a/libpsl.pc.in +++ b/libpsl.pc.in @@ -7,5 +7,5 @@ Name: @PACKAGE_NAME@ Description: Publix Suffix List C library. Version: @PACKAGE_VERSION@ URL: @PACKAGE_URL@ -Libs: -L${libdir} -llibpsl-@LIBPSL_API_VERSION@ -Cflags: -I${includedir}/libpsl-@LIBPSL_API_VERSION@ -I${libdir}/libpsl-@LIBPSL_API_VERSION@/include +Libs: -L${libdir} -llibpsl +Cflags: -I${includedir}/libpsl -I${libdir}/libpsl/include diff --git a/src/Makefile.am b/src/Makefile.am index 7a584bb..93010e4 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -4,12 +4,12 @@ BUILT_SOURCES = suffixes.c # suffixes.c is a built source that must be cleaned CLEANFILES = suffixes.c -lib_LTLIBRARIES = libpsl-@LIBPSL_API_VERSION@.la +lib_LTLIBRARIES = libpsl.la -libpsl_@LIBPSL_API_VERSION@_la_SOURCES = psl.c -libpsl_@LIBPSL_API_VERSION@_la_CPPFLAGS = -I$(top_srcdir)/include +libpsl_la_SOURCES = psl.c +libpsl_la_CPPFLAGS = -I$(top_srcdir)/include # include ABI version information -libpsl_@LIBPSL_API_VERSION@_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) +libpsl_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) noinst_PROGRAMS = psl2c psl2c_SOURCES = psl2c.c diff --git a/tests/Makefile.am b/tests/Makefile.am index e200a22..9234320 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,6 +1,6 @@ DEFS = @DEFS@ -DDATADIR=\"$(top_srcdir)/data\" -DSRCDIR=\"$(srcdir)\" -DPSL_FILE=\"$(PSL_FILE)\" -DPSL_TESTFILE=\"$(PSL_TESTFILE)\" AM_CPPFLAGS = -I$(top_srcdir)/include -LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la +LDADD = ../src/libpsl.la if WITH_BUILTIN diff --git a/tools/Makefile.am b/tools/Makefile.am index 397a6de..38d72f5 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -2,4 +2,4 @@ bin_PROGRAMS = psl AM_CPPFLAGS = -I$(top_srcdir)/include AM_LDFLAGS = -static -LDADD = ../src/libpsl-@LIBPSL_API_VERSION@.la +LDADD = ../src/libpsl.la From ecdadf76e53b6b8cf262f58bd47c9549767e5b7e Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sat, 31 May 2014 19:33:30 +0200 Subject: [PATCH 099/104] added psl_get_version(), bumped version to 0.2.5 --- NEWS | 10 +++++++--- configure.ac | 4 ++-- docs/libpsl/libpsl-sections.txt | 1 + include/libpsl.h | 3 +++ src/psl.c | 14 ++++++++++++++ 5 files changed, 27 insertions(+), 5 deletions(-) diff --git a/NEWS b/NEWS index c54d3c8..8faee0e 100644 --- a/NEWS +++ b/NEWS @@ -1,7 +1,11 @@ Copyright (C) 2014 Tim Ruehsen -30.05.2014 - * Fixed psl_builtin() to return NULL if no built-in PSL data is available +31.05.2014 Release V0.2.5 + * added psl_get_version() + * removed version from library name + +30.05.2014 Release V0.2.4 + * fixed psl_builtin() to return NULL if no built-in PSL data is available 27.05.2014 Release V0.2.3 * changed API version to 0.2 @@ -12,6 +16,6 @@ Copyright (C) 2014 Tim Ruehsen * build static library by default 25.04.2014 Hotfix release V0.2.1 - * Updated to the latest Publix Suffix List + * updated to the latest Publix Suffix List 25.04.2014 Initial release V0.2 diff --git a/configure.ac b/configure.ac index 7712005..705c5d5 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([libpsl], [0.2.4], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.2.5], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define]) @@ -63,7 +63,7 @@ AS_IF([ test "$enable_man" != no ], [ # 4. If any interfaces have been added, removed, or changed since the last update, increment current, and set revision to 0. # 5. If any interfaces have been added since the last public release, then increment age. # 6. If any interfaces have been removed or changed since the last public release, then set age to 0. -AC_SUBST([LIBPSL_SO_VERSION], [0:4:0]) +AC_SUBST([LIBPSL_SO_VERSION], [1:0:1]) AC_SUBST([LIBPSL_VERSION], $VERSION) # Check for idn2 diff --git a/docs/libpsl/libpsl-sections.txt b/docs/libpsl/libpsl-sections.txt index c6eaa16..b2b1617 100644 --- a/docs/libpsl/libpsl-sections.txt +++ b/docs/libpsl/libpsl-sections.txt @@ -16,4 +16,5 @@ psl_builtin_file_time psl_builtin_sha1sum psl_builtin_filename psl_is_cookie_domain_acceptable +psl_get_version diff --git a/include/libpsl.h b/include/libpsl.h index a0a1931..265bdf6 100644 --- a/include/libpsl.h +++ b/include/libpsl.h @@ -83,6 +83,9 @@ const char * /* returns file name of PSL source file */ const char * psl_builtin_filename(void); +/* returns library version */ +const char * + psl_get_version(void); #ifdef __cplusplus diff --git a/src/psl.c b/src/psl.c index 77ed564..881052e 100644 --- a/src/psl.c +++ b/src/psl.c @@ -674,6 +674,20 @@ const char *psl_builtin_filename(void) return _psl_filename; } +/** + * psl_get_version: + * + * Get libpsl version. + * + * Returns: String containing version of libpsl. + * + * Since: 0.2.5 + **/ +const char *psl_get_version (void) +{ + return PACKAGE_VERSION; +} + /** * psl_is_cookie_domain_acceptable: * @psl: PSL context pointer From 8d9e899039a376b22c0bad58f13d49cfcb21c4f7 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Sun, 1 Jun 2014 12:01:47 +0200 Subject: [PATCH 100/104] added --version to psl utility --- tools/psl.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/tools/psl.c b/tools/psl.c index 841340c..9d684e6 100644 --- a/tools/psl.c +++ b/tools/psl.c @@ -42,6 +42,7 @@ static void usage(int err) fprintf(stderr, "Usage: psl [options] \n"); fprintf(stderr, "\n"); fprintf(stderr, "Options:\n"); + fprintf(stderr, " --version show library version information\n"); fprintf(stderr, " --use-builtin-data use the builtin PSL data. [default]\n"); fprintf(stderr, " --load-psl-file load PSL data from file.\n"); fprintf(stderr, " --is-public-suffix check if domains are public suffixes or not. [default]\n"); @@ -49,6 +50,7 @@ static void usage(int err) fprintf(stderr, " check if cookie-domain is acceptable for domains.\n"); fprintf(stderr, " --print-unreg-domain print the longest publix suffix part\n"); fprintf(stderr, " --print-reg-domain print the shortest private suffix part\n"); + fprintf(stderr, " --print-info print info about library builtin data\n"); fprintf(stderr, "\n"); exit(err); @@ -107,6 +109,14 @@ int main(int argc, const char *const *argv) else if (!strcmp(*arg, "--help")) { usage(0); } + else if (!strcmp(*arg, "--version")) { + printf("psl %s\n", PACKAGE_VERSION); + printf("libpsl %s\n", psl_get_version()); + printf("\n"); + printf("Copyright (C) 2014 Tim Ruehsen\n"); + printf("License: MIT\n"); + exit(0); + } else if (!strcmp(*arg, "--")) { arg++; break; From 58daea97ce271aeaa47d913d35d8904a6277a378 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Wed, 4 Jun 2014 13:20:34 +0200 Subject: [PATCH 101/104] added IDNA2008 UTS#46 via libicu --- .travis.yml | 3 ++- README.md | 1 + configure.ac | 13 ++++++++++--- src/psl2c.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 60 insertions(+), 6 deletions(-) diff --git a/.travis.yml b/.travis.yml index 54965ca..27bcd14 100644 --- a/.travis.yml +++ b/.travis.yml @@ -5,5 +5,6 @@ compiler: # Change this to your needs script: ./autogen.sh && ./configure --enable-gtk-doc && make -j4 && make check -j4 && make distcheck before_install: + - apt-cache search libicu | grep icu - sudo apt-get -qq update - - sudo apt-get -q install autoconf automake autopoint libtool gtk-doc-tools gettext idn2 libidn2-0 libidn2-0-dev + - sudo apt-get -q install autoconf automake autopoint libtool gtk-doc-tools gettext idn2 libidn2-0 libidn2-0-dev libicu-dev diff --git a/README.md b/README.md index d71497e..6bb6711 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,7 @@ Libpsl... - finds the shortest private part of a given domain - works with international domains (UTF-8 and IDNA2008 Punycode) - is thread-safe +- handles IDNA2008 UTS#46 (library has to be compiled with libicu) Find more information about the Publix Suffix List [here](http://publicsuffix.org/). diff --git a/configure.ac b/configure.ac index 705c5d5..ffd406a 100644 --- a/configure.ac +++ b/configure.ac @@ -66,9 +66,6 @@ AS_IF([ test "$enable_man" != no ], [ AC_SUBST([LIBPSL_SO_VERSION], [1:0:1]) AC_SUBST([LIBPSL_VERSION], $VERSION) -# Check for idn2 -AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) - # Check for enable/disable builtin PSL data AC_ARG_ENABLE(builtin, AS_HELP_STRING([--disable-builtin], [do not compile PSL data into library]), @@ -80,6 +77,16 @@ AC_ARG_ENABLE(builtin, ]) AM_CONDITIONAL([WITH_BUILTIN], [test $enable_builtin = yes]) +AC_ARG_WITH(icu, AS_HELP_STRING([--without-icu], [disable ICU punycode conversion]), with_icu=$withval, with_icu=yes) +if test $with_icu != "no" +then + AC_CHECK_LIB(icuuc, uidna_openUTS46, [with_icu=yes; AC_SUBST(ICU_LIBS, "-licu") AC_DEFINE([WITH_LIBICU], [1], [Use libicu])], [with_icu=no; AC_MSG_WARN(*** LIBICU was not found. Falling back to idn2.)]) +fi +AM_CONDITIONAL([WITH_LIBICU], [test $with_icu = "yes"]) + +# Check for idn2 +AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) + # Check for valgrind ac_enable_valgrind=no AC_ARG_ENABLE(valgrind-tests, diff --git a/src/psl2c.c b/src/psl2c.c index c3364fa..8bcad50 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -45,6 +45,11 @@ #endif */ +#ifdef WITH_LIBICU +# include +# include +#endif + #ifdef WITH_BUILTIN #include @@ -267,7 +272,19 @@ static void _print_psl_entries(FILE *fpout, const _psl_vector_t *v, const char * { int it; - fprintf(fpout, "/* automatically generated by psl2c */\n"); +#ifdef WITH_LIBICU + do { + UVersionInfo version_info; + char version[U_MAX_VERSION_STRING_LENGTH]; + + u_getVersion(version_info); + u_versionToString(version_info, version); + fprintf(fpout, "/* automatically generated by psl2c (punycode generated with libicu/%s) */\n", version); + } while (0); +#else + fprintf(fpout, "/* automatically generated by psl2c (punycode generated with idn2) */\n"); +#endif + fprintf(fpout, "static _psl_entry_t %s[] = {\n", varname); for (it = 0; it < v->cur; it++) { @@ -306,6 +323,7 @@ static void _add_punycode_if_needed(_psl_vector_t *v) if (_str_needs_encoding(e->label_buf)) { _psl_entry_t suffix, *suffixp; + char lookupname[64] = ""; /* the following lines will have GPL3+ license issues */ /* char *asc = NULL; @@ -321,9 +339,35 @@ static void _add_punycode_if_needed(_psl_vector_t *v) fprintf(stderr, "toASCII(%s) failed (%d): %s\n", e->label_buf, rc, idn2_strerror(rc)); */ +#ifdef WITH_LIBICU + UIDNA *idna; + UErrorCode status = 0; + + /* IDNA2003 punycode conversion */ + /* destLen = uidna_toASCII(e->label_buf, (int32_t) strlen(e->label_buf), lookupname, (int32_t) sizeof(lookupname), + UIDNA_DEFAULT, NULL, &status); + */ + + /* IDNA2008 UTS#46 punycode conversion */ + if ((idna = uidna_openUTS46(UIDNA_USE_STD3_RULES, &status))) { + uidna_nameToASCII(idna, (UChar *) e->label_buf, (int32_t) strlen(e->label_buf), + (UChar *) lookupname, (int32_t) sizeof(lookupname), NULL, &status); + uidna_close(idna); + } + + if (U_FAILURE(status)) { + fprintf(stderr, "Failed to convert '%s' to ASCII\n", e->label_buf); + } else if (strcmp(e->label_buf, lookupname)) { + /* fprintf(stderr, "libicu '%s' -> '%s'\n", e->label_buf, lookupname); */ + _suffix_init(&suffix, lookupname, strlen(lookupname)); + suffix.wildcard = e->wildcard; + suffixp = _vector_get(v, _vector_add(v, &suffix)); + suffixp->label = suffixp->label_buf; /* set label to changed address */ + } /* else ignore */ +#else /* this is much slower than the libidn2 API but should have no license issues */ FILE *pp; - char cmd[16 + sizeof(e->label_buf)], lookupname[64] = ""; + char cmd[16 + sizeof(e->label_buf)]; snprintf(cmd, sizeof(cmd), "idn2 '%s'", e->label_buf); if ((pp = popen(cmd, "r"))) { if (fscanf(pp, "%63s", lookupname) >= 1 && strcmp(e->label_buf, lookupname)) { @@ -336,6 +380,7 @@ static void _add_punycode_if_needed(_psl_vector_t *v) pclose(pp); } else fprintf(stderr, "Failed to call popen(%s, \"r\")\n", cmd); +#endif } } From 79cd551b17622e694b053a21c5d722263ee21370 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 5 Jun 2014 11:39:28 +0200 Subject: [PATCH 102/104] fixed libicu implementation, use pkg-config for libicu detection --- configure.ac | 14 ++++---------- src/Makefile.am | 4 ++-- src/psl2c.c | 44 ++++++++++++++++++++++++++++++-------------- 3 files changed, 36 insertions(+), 26 deletions(-) diff --git a/configure.ac b/configure.ac index ffd406a..52da77c 100644 --- a/configure.ac +++ b/configure.ac @@ -74,19 +74,13 @@ AC_ARG_ENABLE(builtin, ], [ enable_builtin=yes AC_DEFINE([WITH_BUILTIN], [1], [compile PSL data into library]) + + PKG_CHECK_MODULES(LIBICU, [icu-uc], + [AC_DEFINE([WITH_LIBICU], [1], [generate PSL data with IDNA2008 UTS#46 punycode])], + [AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.))]) ]) AM_CONDITIONAL([WITH_BUILTIN], [test $enable_builtin = yes]) -AC_ARG_WITH(icu, AS_HELP_STRING([--without-icu], [disable ICU punycode conversion]), with_icu=$withval, with_icu=yes) -if test $with_icu != "no" -then - AC_CHECK_LIB(icuuc, uidna_openUTS46, [with_icu=yes; AC_SUBST(ICU_LIBS, "-licu") AC_DEFINE([WITH_LIBICU], [1], [Use libicu])], [with_icu=no; AC_MSG_WARN(*** LIBICU was not found. Falling back to idn2.)]) -fi -AM_CONDITIONAL([WITH_LIBICU], [test $with_icu = "yes"]) - -# Check for idn2 -AC_CHECK_PROG(HAVE_IDN2, idn2, yes, AC_MSG_ERROR(Cannot find required tool 'idn2'.)) - # Check for valgrind ac_enable_valgrind=no AC_ARG_ENABLE(valgrind-tests, diff --git a/src/Makefile.am b/src/Makefile.am index 93010e4..0fe1ec7 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -13,8 +13,8 @@ libpsl_la_LDFLAGS = -version-info $(LIBPSL_SO_VERSION) noinst_PROGRAMS = psl2c psl2c_SOURCES = psl2c.c -psl2c_CPPFLAGS = -I$(top_srcdir)/include -D _GNU_SOURCE -#psl2c_LDADD = -lidn2 +psl2c_CPPFLAGS = -I$(top_srcdir)/include -D _GNU_SOURCE $(LIBICU_CFLAGS) +psl2c_LDADD = $(LIBICU_LIBS) # Build rule for suffix.c # PSL_FILE can be set by ./configure --with-psl-file=[PATH] diff --git a/src/psl2c.c b/src/psl2c.c index 8bcad50..d0d9f0d 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -47,6 +47,7 @@ #ifdef WITH_LIBICU # include +# include # include #endif @@ -349,21 +350,36 @@ static void _add_punycode_if_needed(_psl_vector_t *v) */ /* IDNA2008 UTS#46 punycode conversion */ - if ((idna = uidna_openUTS46(UIDNA_USE_STD3_RULES, &status))) { - uidna_nameToASCII(idna, (UChar *) e->label_buf, (int32_t) strlen(e->label_buf), - (UChar *) lookupname, (int32_t) sizeof(lookupname), NULL, &status); - uidna_close(idna); - } +// if ((idna = uidna_openUTS46(UIDNA_USE_STD3_RULES, &status))) { + if ((idna = uidna_openUTS46(UIDNA_DEFAULT, &status))) { + UChar utf16_dst[64], utf16_src[64]; + int32_t utf16_src_length; + UIDNAInfo info = UIDNA_INFO_INITIALIZER; + + u_strFromUTF8(utf16_src, sizeof(utf16_src)/sizeof(utf16_src[0]), &utf16_src_length, e->label_buf, (int32_t) strlen(e->label_buf), &status); + if (U_SUCCESS(status)) { + int32_t dst_length = uidna_nameToASCII(idna, utf16_src, utf16_src_length, utf16_dst, sizeof(utf16_dst)/sizeof(utf16_dst[0]), &info, &status); + if (U_SUCCESS(status)) { + u_strToUTF8(lookupname, (int32_t) sizeof(lookupname), NULL, utf16_dst, dst_length, &status); + if (U_SUCCESS(status)) { + if (strcmp(e->label_buf, lookupname)) { + /* fprintf(stderr, "libicu '%s' -> '%s'\n", e->label_buf, lookupname); */ + _suffix_init(&suffix, lookupname, strlen(lookupname)); + suffix.wildcard = e->wildcard; + suffixp = _vector_get(v, _vector_add(v, &suffix)); + suffixp->label = suffixp->label_buf; /* set label to changed address */ + } // else ignore + } else + fprintf(stderr, "Failed to convert UTF-16 to UTF-8 (status %d)\n", status); + } else + fprintf(stderr, "Failed to convert to ASCII (status %d)\n", status); + } else + fprintf(stderr, "Failed to convert UTF-8 to UTF-16 (status %d)\n", status); + + uidna_close(idna); + } else + fprintf(stderr, "Failed to get UTS46 IDNA handle\n"); - if (U_FAILURE(status)) { - fprintf(stderr, "Failed to convert '%s' to ASCII\n", e->label_buf); - } else if (strcmp(e->label_buf, lookupname)) { - /* fprintf(stderr, "libicu '%s' -> '%s'\n", e->label_buf, lookupname); */ - _suffix_init(&suffix, lookupname, strlen(lookupname)); - suffix.wildcard = e->wildcard; - suffixp = _vector_get(v, _vector_add(v, &suffix)); - suffixp->label = suffixp->label_buf; /* set label to changed address */ - } /* else ignore */ #else /* this is much slower than the libidn2 API but should have no license issues */ FILE *pp; From 7621dce71d4f4103771b4400661252ce6333ee05 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 5 Jun 2014 11:53:29 +0200 Subject: [PATCH 103/104] fixed C89 comment incompatibility --- src/psl2c.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/psl2c.c b/src/psl2c.c index d0d9f0d..a6b5b6c 100644 --- a/src/psl2c.c +++ b/src/psl2c.c @@ -350,8 +350,7 @@ static void _add_punycode_if_needed(_psl_vector_t *v) */ /* IDNA2008 UTS#46 punycode conversion */ -// if ((idna = uidna_openUTS46(UIDNA_USE_STD3_RULES, &status))) { - if ((idna = uidna_openUTS46(UIDNA_DEFAULT, &status))) { + if ((idna = uidna_openUTS46(UIDNA_USE_STD3_RULES, &status))) { UChar utf16_dst[64], utf16_src[64]; int32_t utf16_src_length; UIDNAInfo info = UIDNA_INFO_INITIALIZER; @@ -368,7 +367,7 @@ static void _add_punycode_if_needed(_psl_vector_t *v) suffix.wildcard = e->wildcard; suffixp = _vector_get(v, _vector_add(v, &suffix)); suffixp->label = suffixp->label_buf; /* set label to changed address */ - } // else ignore + } /* else ignore */ } else fprintf(stderr, "Failed to convert UTF-16 to UTF-8 (status %d)\n", status); } else From b67ef20c827187bd128a854b135c90352fe499e6 Mon Sep 17 00:00:00 2001 From: Tim Ruehsen Date: Thu, 5 Jun 2014 16:29:20 +0200 Subject: [PATCH 104/104] Release V0.3.0 --- NEWS | 5 +++++ README.md | 2 +- configure.ac | 2 +- 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/NEWS b/NEWS index 8faee0e..0e6c51e 100644 --- a/NEWS +++ b/NEWS @@ -1,5 +1,10 @@ Copyright (C) 2014 Tim Ruehsen +05.06.2014 Release V0.3.0 + * added support for libicu in psl2c (IDNA2008 UTS#46) + this needs pkg-config and libicu-dev installed + * added --version to psl utility + 31.05.2014 Release V0.2.5 * added psl_get_version() * removed version from library name diff --git a/README.md b/README.md index 6bb6711..7bc8fbc 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ Libpsl... - finds the shortest private part of a given domain - works with international domains (UTF-8 and IDNA2008 Punycode) - is thread-safe -- handles IDNA2008 UTS#46 (library has to be compiled with libicu) +- handles IDNA2008 UTS#46 (libicu is used by psl2c if installed) Find more information about the Publix Suffix List [here](http://publicsuffix.org/). diff --git a/configure.ac b/configure.ac index 52da77c..970a0b3 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ -AC_INIT([libpsl], [0.2.5], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) +AC_INIT([libpsl], [0.3.0], [tim.ruehsen@gmx.de], [libpsl], [http://github.com/rockdaboot/libpsl]) AC_PREREQ([2.59]) AM_INIT_AUTOMAKE([1.10 -Wall no-define])