From 810d9b6da5c8e598c12c35ad20122eddb42239c2 Mon Sep 17 00:00:00 2001 From: "Philip.Hazel" Date: Wed, 19 Jul 2017 16:04:15 +0000 Subject: [PATCH] Code tidies for 10.30-RC1 release candidate. --- CMakeLists.txt | 2 +- ChangeLog | 172 ++--- NEWS | 57 ++ NON-AUTOTOOLS-BUILD | 4 +- README | 63 +- RunTest | 6 +- configure.ac | 20 +- doc/html/NON-AUTOTOOLS-BUILD.txt | 4 +- doc/html/README.txt | 63 +- doc/html/pcre2.html | 2 +- doc/html/pcre2_code_free.html | 4 +- doc/html/pcre2_compile.html | 2 +- doc/html/pcre2_config.html | 4 +- doc/html/pcre2_converted_pattern_free.html | 4 +- doc/html/pcre2_maketables.html | 2 +- doc/html/pcre2_match.html | 10 +- doc/html/pcre2_pattern_info.html | 4 +- doc/html/pcre2_set_newline.html | 2 +- doc/html/pcre2_set_recursion_limit.html | 2 +- doc/html/pcre2_substitute.html | 2 +- doc/html/pcre2build.html | 20 +- doc/html/pcre2compat.html | 2 +- doc/html/pcre2grep.html | 22 +- doc/html/pcre2perform.html | 12 +- doc/html/pcre2syntax.html | 4 +- doc/pcre2.3 | 2 +- doc/pcre2.txt | 162 ++--- doc/pcre2_code_free.3 | 4 +- doc/pcre2_compile.3 | 2 +- doc/pcre2_config.3 | 4 +- doc/pcre2_converted_pattern_free.3 | 4 +- doc/pcre2_dfa_match.3 | 8 +- doc/pcre2_maketables.3 | 2 +- doc/pcre2_match.3 | 10 +- doc/pcre2_pattern_info.3 | 4 +- doc/pcre2_set_newline.3 | 2 +- doc/pcre2_set_recursion_limit.3 | 2 +- doc/pcre2_substitute.3 | 2 +- doc/pcre2build.3 | 21 +- doc/pcre2compat.3 | 2 +- doc/pcre2grep.1 | 22 +- doc/pcre2perform.3 | 12 +- doc/pcre2syntax.3 | 4 +- perltest.sh | 14 +- src/config.h.generic | 23 +- src/pcre2.h | 4 +- src/pcre2.h.generic | 65 +- src/pcre2_config.c | 6 +- src/pcre2_context.c | 4 +- src/pcre2_error.c | 2 +- src/pcre2_internal.h | 4 +- src/pcre2_jit_compile.c | 14 +- src/pcre2_pattern_info.c | 2 +- src/pcre2_tables.c | 12 +- src/pcre2_ucp.h | 4 +- src/pcre2posix.c | 8 +- src/pcre2posix.h | 4 +- src/pcre2test.c | 9 +- src/sljit/sljitLir.c | 7 +- testdata/testinput1 | 11 - testdata/testinput2 | 743 ++++++++++---------- testdata/testoutput1 | 21 - testdata/testoutput2 | 753 +++++++++++---------- testdata/testoutput8-32-3 | 2 - testdata/testoutput8-32-4 | 2 - 65 files changed, 1320 insertions(+), 1152 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 5ebe695..fbc37fe 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -429,7 +429,7 @@ SET(PCRE2_SOURCES src/pcre2_compile.c src/pcre2_config.c src/pcre2_context.c - src/pcre2_convert.c + src/pcre2_convert.c src/pcre2_dfa_match.c src/pcre2_error.c src/pcre2_find_bracket.c diff --git a/ChangeLog b/ChangeLog index 9d61911..b8b39e8 100644 --- a/ChangeLog +++ b/ChangeLog @@ -2,105 +2,105 @@ Change Log for PCRE2 -------------------- -Version 10.30-DEV 09-March-2017 -------------------------------- +Version 10.30-RC1 18-July-2017 +------------------------------ -1. The main interpreter, pcre2_match(), has been refactored into a new version -that does not use recursive function calls (and therefore the stack) for -remembering backtracking positions. This makes --disable-stack-for-recursion a +1. The main interpreter, pcre2_match(), has been refactored into a new version +that does not use recursive function calls (and therefore the stack) for +remembering backtracking positions. This makes --disable-stack-for-recursion a NOOP. The new implementation allows backtracking into recursive group calls in -patterns, making it more compatible with Perl, and also fixes some other -hard-to-do issues such as #1887 in Bugzilla. The code is also cleaner because -the old code had a number of fudges to try to reduce stack usage. It seems to +patterns, making it more compatible with Perl, and also fixes some other +hard-to-do issues such as #1887 in Bugzilla. The code is also cleaner because +the old code had a number of fudges to try to reduce stack usage. It seems to run no slower than the old code. A number of bugs in the refactored code were subsequently fixed during testing before release, but after the code was made available in the repository. These bugs were never in fully released code, but are noted here for the record. - (a) If a pattern had fewer capturing parentheses than the ovector supplied in + (a) If a pattern had fewer capturing parentheses than the ovector supplied in the match data block, a memory error (detectable by ASAN) occurred after a match, because the external block was being set from non-existent internal ovector fields. Fixes oss-fuzz issue 781. - (b) A pattern with very many capturing parentheses (when the internal frame - size was greater than the initial frame vector on the stack) caused a - crash. A vector on the heap is now set up at the start of matching if the - vector on the stack is not big enough to handle at least 10 frames. - Fixes oss-fuzz issue 783. - + (b) A pattern with very many capturing parentheses (when the internal frame + size was greater than the initial frame vector on the stack) caused a + crash. A vector on the heap is now set up at the start of matching if the + vector on the stack is not big enough to handle at least 10 frames. + Fixes oss-fuzz issue 783. + (c) Handling of (*VERB)s in recursions was wrong in some cases. - + (d) Captures in negative assertions that were used as conditions were not - happening if the assertion matched via (*ACCEPT). - - (e) Mark values were not being passed out of recursions. - - (f) Refactor some code in do_callout() to avoid picky compiler warnings about + happening if the assertion matched via (*ACCEPT). + + (e) Mark values were not being passed out of recursions. + + (f) Refactor some code in do_callout() to avoid picky compiler warnings about negative indices. Fixes oss-fuzz issue 1454. - + (g) Similarly refactor the way the variable length ovector is addressed for similar reasons. Fixes oss-fuzz issue 1465. - + 2. Now that pcre2_match() no longer uses recursive function calls (see above), -the "match limit recursion" value seems misnamed. It still exists, and limits -the depth of tree that is searched. To avoid future confusion, it has been -renamed as "depth limit" in all relevant places (--with-depth-limit, -(*LIMIT_DEPTH), pcre2_set_depth_limit(), etc) but the old names are still -available for backwards compatibility. - +the "match limit recursion" value seems misnamed. It still exists, and limits +the depth of tree that is searched. To avoid future confusion, it has been +renamed as "depth limit" in all relevant places (--with-depth-limit, +(*LIMIT_DEPTH), pcre2_set_depth_limit(), etc) but the old names are still +available for backwards compatibility. + 3. Hardened pcre2test so as to reduce the number of bugs reported by fuzzers: - (a) Check for malloc failures when getting memory for the ovector (POSIX) or - the match data block (non-POSIX). - + (a) Check for malloc failures when getting memory for the ovector (POSIX) or + the match data block (non-POSIX). + 4. In the 32-bit library in non-UTF mode, an attempt to find a Unicode property for a character with a code point greater than 0x10ffff (the Unicode maximum) caused a crash. -5. If a lookbehind assertion that contained a back reference to a group +5. If a lookbehind assertion that contained a back reference to a group appearing later in the pattern was compiled with the PCRE2_ANCHORED option, -undefined actions (often a segmentation fault) could occur, depending on what -other options were set. An example assertion is (?" should be ">=" in opcode check in pcre2_auto_possess.c. - (b) Added some casts to avoid "suspicious implicit sign extension". - (c) Resource leaks in pcre2test in rare error cases. + (b) Added some casts to avoid "suspicious implicit sign extension". + (c) Resource leaks in pcre2test in rare error cases. (d) Avoid warning for never-use case OP_TABLE_LENGTH which is just a fudge - for checking at compile time that tables are the right size. - (e) Add missing "fall through" comment. - -29. Implemented PCRE2_EXTENDED_MORE and related /xx and (?xx) features. + for checking at compile time that tables are the right size. + (e) Add missing "fall through" comment. + +29. Implemented PCRE2_EXTENDED_MORE and related /xx and (?xx) features. 30. Implement (?n: for PCRE2_NO_AUTO_CAPTURE, because Perl now has this. -31. If more than one of "push", "pushcopy", or "pushtablescopy" were set in +31. If more than one of "push", "pushcopy", or "pushtablescopy" were set in pcre2test, a crash could occur. -32. Make -bigstack in RunTest allocate a 64Mb stack (instead of 16 MB) so that +32. Make -bigstack in RunTest allocate a 64Mb stack (instead of 16 MB) so that all the tests can run with clang's sanitizing options. -33. Implement extra compile options in the compile context and add the first +33. Implement extra compile options in the compile context and add the first one: PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES. 34. Implement newline type PCRE2_NEWLINE_NUL. -35. A lookbehind assertion that had a zero-length branch caused undefined +35. A lookbehind assertion that had a zero-length branch caused undefined behaviour when processed by pcre2_dfa_match(). This is oss-fuzz issue 1859. -36. The match limit value now also applies to pcre2_dfa_match() as there are -patterns that can use up a lot of resources without necessarily recursing very +36. The match limit value now also applies to pcre2_dfa_match() as there are +patterns that can use up a lot of resources without necessarily recursing very deeply. (Compare item 10.23/36.) This should fix oss-fuzz #1761. 37. Implement PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL. -38. Fix returned offsets from regexec() when REG_STARTEND is used with a +38. Fix returned offsets from regexec() when REG_STARTEND is used with a starting offset greater than zero. 39. Implement REG_PEND (GNU extension) for the POSIX wrapper. -40. Implement the subject_literal modifier in pcre2test, and allow jitstack on +40. Implement the subject_literal modifier in pcre2test, and allow jitstack on pattern lines. 41. Implement PCRE2_LITERAL and use it to support REG_NOSPEC. -42. Implement PCRE2_EXTRA_MATCH_LINE and PCRE2_EXTRA_MATCH_WORD for the benefit +42. Implement PCRE2_EXTRA_MATCH_LINE and PCRE2_EXTRA_MATCH_WORD for the benefit of pcre2grep. -43. Re-implement pcre2grep's -F, -w, and -x options using PCRE2_LITERAL, +43. Re-implement pcre2grep's -F, -w, and -x options using PCRE2_LITERAL, PCRE2_EXTRA_MATCH_WORD, and PCRE2_EXTRA_MATCH_LINE. This fixes two bugs: (a) The -F option did not work for fixed strings containing \E. - (b) The -w option did not work for patterns with multiple branches. - + (b) The -w option did not work for patterns with multiple branches. + 44. Added configuration options for the SELinux compatible execmem allocator in JIT. -45. Increased the limit for searching for a "must be present" code unit in -subjects from 1000 to 2000 for 8-bit searches, since they use memchr() and are +45. Increased the limit for searching for a "must be present" code unit in +subjects from 1000 to 2000 for 8-bit searches, since they use memchr() and are much faster. 46. Arrange for anchored patterns to record and use "first code unit" data, -because this can give a fast "no match" without searching for a "required code +because this can give a fast "no match" without searching for a "required code unit". Previously only non-anchored patterns did this. 47. Upgraded the Unicode tables from Unicode 8.0.0 to Unicode 10.0.0. 48. Add the callout_no_where modifier to pcre2test. -49. Update extended grapheme breaking rules to the latest set that are in +49. Update extended grapheme breaking rules to the latest set that are in Unicode Standard Annex #29. -50. Added experimental foreign pattern conversion facilities +50. Added experimental foreign pattern conversion facilities (pcre2_pattern_convert() and friends). diff --git a/NEWS b/NEWS index 08798bb..cacbed5 100644 --- a/NEWS +++ b/NEWS @@ -1,6 +1,63 @@ News about PCRE2 releases ------------------------- +Version 10.30-RC1 18-July-2017 +------------------------------ + +The full list of changes that includes bugfixes and tidies is, as always, in +ChangeLog. These are the most important new features: + +1. The main interpreter, pcre2_match(), has been refactored into a new version +that does not use recursive function calls (and therefore the system stack) for +remembering backtracking positions. This makes --disable-stack-for-recursion a +NOOP. The new implementation allows backtracking into recursive group calls in +patterns, making it more compatible with Perl, and also fixes some other +previously hard-to-do issues. For patterns that have a lot of backtracking, the +heap is now used, and there is explicit limit on the amount, settable by +pcre2_set_heap_limit() or (*LIMIT_HEAP=xxx). The "recursion limit" is retained, +but is renamed as "depth limit" (though the old names remain for +compatibility). + +There is also a change in the way callouts from pcre2_match() are handled. The +offset_vector field in the callout block is no longer a pointer to the +actual ovector that was passed to the matching function in the match data +block. Instead it points to an internal ovector of a size large enough to hold +all possible captured substrings in the pattern. + +2. The new option PCRE2_ENDANCHORED insists that a pattern match must end at +the end of the subject. + +3. The new option PCRE2_EXTENDED_MORE implements Perl's /xx feature, and +pcre2test is upgraded to support it. Setting within the pattern by (?xx) is +also supported. + +4. (?n) can be used to set PCRE2_NO_AUTO_CAPTURE, because Perl now has this. + +5. Additional compile options in the compile context are now available, and the +first two are: PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES and +PCRE2_EXTRA_BAD_ESCAPE_IS LITERAL. + +6. The newline type PCRE2_NEWLINE_NUL is now available. + +7. The match limit value now also applies to pcre2_dfa_match() as there are +patterns that can use up a lot of resources without necessarily recursing very +deeply. + +8. The option REG_PEND (a GNU extension) is now available for the POSIX +wrapper. Also there is a new option PCRE2_LITERAL which is used to support +REG_NOSPEC. + +9. PCRE2_EXTRA_MATCH_LINE and PCRE2_EXTRA_MATCH_WORD are implemented for the +benefit of pcre2grep, and pcre2grep's -F, -w, and -x options are re-implemented +using PCRE2_LITERAL, PCRE2_EXTRA_MATCH_WORD, and PCRE2_EXTRA_MATCH_LINE. This +is tidier and also fixes some bugs. + +10. The Unicode tables are upgraded from Unicode 8.0.0 to Unicode 10.0.0. + +11. There are some experimental functions for converting foreign patterns +(globs and POSIX patterns) into PCRE2 patterns. + + Version 10.23 14-February-2017 ------------------------------ diff --git a/NON-AUTOTOOLS-BUILD b/NON-AUTOTOOLS-BUILD index fe20152..f8c6359 100644 --- a/NON-AUTOTOOLS-BUILD +++ b/NON-AUTOTOOLS-BUILD @@ -179,8 +179,8 @@ can skip ahead to the CMake section. STACK SIZE IN WINDOWS ENVIRONMENTS -Prior to release 10.30 the default system stack size of 1Mb in some Windows -environments caused issues with some tests. This should no longer be the case +Prior to release 10.30 the default system stack size of 1Mb in some Windows +environments caused issues with some tests. This should no longer be the case for 10.30 and later releases. diff --git a/README b/README index 6bf5b46..bed0513 100644 --- a/README +++ b/README @@ -173,7 +173,7 @@ library. They are also documented in the pcre2build man page. architectures. If you try to enable it on an unsupported architecture, there will be a compile time error. If you are running under SELinux you may also want to add --enable-jit-sealloc, which enables the use of an execmem - allocator in JIT that is compatible with SELinux. This has no effect if JIT + allocator in JIT that is compatible with SELinux. This has no effect if JIT is not enabled. . If you do not want to make use of the default support for UTF-8 Unicode @@ -198,13 +198,14 @@ library. They are also documented in the pcre2build man page. or starting a pattern with (*UCP). . You can build PCRE2 to recognize either CR or LF or the sequence CRLF, or any - of the preceding, or any of the Unicode newline sequences, as indicating the - end of a line. Whatever you specify at build time is the default; the caller - of PCRE2 can change the selection at run time. The default newline indicator - is a single LF character (the Unix standard). You can specify the default - newline indicator by adding --enable-newline-is-cr, --enable-newline-is-lf, - --enable-newline-is-crlf, --enable-newline-is-anycrlf, or - --enable-newline-is-any to the "configure" command, respectively. + of the preceding, or any of the Unicode newline sequences, or the NUL (zero) + character as indicating the end of a line. Whatever you specify at build time + is the default; the caller of PCRE2 can change the selection at run time. The + default newline indicator is a single LF character (the Unix standard). You + can specify the default newline indicator by adding --enable-newline-is-cr, + --enable-newline-is-lf, --enable-newline-is-crlf, + --enable-newline-is-anycrlf, --enable-newline-is-any, or + --enable-newline-is-nul to the "configure" command, respectively. . By default, the sequence \R in a pattern matches any Unicode line ending sequence. This is independent of the option specifying what PCRE2 considers @@ -227,15 +228,15 @@ library. They are also documented in the pcre2build man page. --with-parens-nest-limit=500 . PCRE2 has a counter that can be set to limit the amount of computing resource - it uses when matching a pattern with the Perl-compatible matching function. - If the limit is exceeded during a match, the match fails. The default is ten - million. You can change the default by setting, for example, + it uses when matching a pattern. If the limit is exceeded during a match, the + match fails. The default is ten million. You can change the default by + setting, for example, --with-match-limit=500000 on the "configure" command. This is just the default; individual calls to - pcre2_match() can supply their own value. There is more discussion in the - pcre2api man page (search for pcre2_set_match_limit). + pcre2_match() or pcre2_dfa_match() can supply their own value. There is more + discussion in the pcre2api man page (search for pcre2_set_match_limit). . There is a separate counter that limits the depth of nested backtracking during a matching process, which indirectly limits the amount of heap memory @@ -246,15 +247,15 @@ library. They are also documented in the pcre2build man page. There is more discussion in the pcre2api man page (search for pcre2_set_depth_limit). - -. You can also set an explicit limit on the amount of heap memory used by + +. You can also set an explicit limit on the amount of heap memory used by the pcre2_match() interpreter: - + --with-heap-limit=500 - - The units are kilobytes. This limit does not apply when the JIT optimization - (which has its own memory control features) is used. There is more discussion - on the pcre2api man page (search for pcre2_set_heap_limit). + + The units are kilobytes. This limit does not apply when the JIT optimization + (which has its own memory control features) is used. There is more discussion + on the pcre2api man page (search for pcre2_set_heap_limit). . In the 8-bit library, the default maximum compiled pattern size is around 64K bytes. You can increase this by adding --with-link-size=3 to the @@ -659,9 +660,10 @@ with the perltest.sh script, and test 5 checking PCRE2-specific things. Tests 6 and 7 check the pcre2_dfa_match() alternative matching function, in non-UTF mode and UTF-mode with Unicode property support, respectively. -Test 8 checks some internal offsets and code size features; it is run only when -the default "link size" of 2 is set (in other cases the sizes change) and when -Unicode support is enabled. +Test 8 checks some internal offsets and code size features, but it is run only +when Unicode support is enabled. The output is different in 8-bit, 16-bit, and +32-bit modes and for different link sizes, so there are different output files +for each mode and link size. Tests 9 and 10 are run only in 8-bit mode, and tests 11 and 12 are run only in 16-bit and 32-bit modes. These are tests that generate different output in @@ -671,7 +673,7 @@ Test 13 checks the handling of non-UTF characters greater than 255 by pcre2_dfa_match() in 16-bit and 32-bit modes. Test 14 contains some special UTF and UCP tests that give different output for -the different widths. +different code unit widths. Test 15 contains a number of tests that must not be run with JIT. They check, among other non-JIT things, the match-limiting features of the intepretive @@ -692,6 +694,9 @@ patterns to a file, and then reloading and checking them. Tests 21 and 22 test \C support when the use of \C is not locked out, without and with UTF support, respectively. Test 23 tests \C when it is locked out. +Tests 24 and 25 test the experimental pattern conversion functions, without and +with UTF support, respectively. + Character tables ---------------- @@ -710,7 +715,7 @@ specified for ./configure, a different version of pcre2_chartables.c is built by the program dftables (compiled from dftables.c), which uses the ANSI C character handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table sources. This means that the default C -locale which is set for your system will control the contents of these default +locale that is set for your system will control the contents of these default tables. You can change the default tables by editing pcre2_chartables.c and then re-building PCRE2. If you do this, you should take care to ensure that the file does not get automatically re-generated. The best way to do this is to @@ -765,6 +770,7 @@ The distribution should contain the files listed below. src/pcre2_compile.c ) src/pcre2_config.c ) src/pcre2_context.c ) + src/pcre2_convert.c ) src/pcre2_dfa_match.c ) src/pcre2_error.c ) src/pcre2_find_bracket.c ) @@ -804,7 +810,6 @@ The distribution should contain the files listed below. src/pcre2demo.c simple demonstration of coding calls to PCRE2 src/pcre2grep.c source of a grep utility that uses PCRE2 src/pcre2test.c comprehensive test program - src/pcre2_printint.c part of pcre2test src/pcre2_jit_test.c JIT test program (C) Auxiliary files: @@ -869,12 +874,12 @@ The distribution should contain the files listed below. (E) Auxiliary files for building PCRE2 "by hand" - pcre2.h.generic ) a version of the public PCRE2 header file + src/pcre2.h.generic ) a version of the public PCRE2 header file ) for use in non-"configure" environments - config.h.generic ) a version of config.h for use in non-"configure" + src/config.h.generic ) a version of config.h for use in non-"configure" ) environments Philip Hazel Email local part: ph10 Email domain: cam.ac.uk -Last updated: 17 June 2017 +Last updated: 18 July 2017 diff --git a/RunTest b/RunTest index c0e3b1f..bc912da 100755 --- a/RunTest +++ b/RunTest @@ -830,7 +830,7 @@ for bmode in "$test8" "$test16" "$test32"; do if [ $supportBSC -ne 0 ] ; then echo " Skipped because \C is not disabled" else - $sim $valgrind ${opt:+$vjs} ./pcre2test -q $setstack $bmode $opt $testdata/testinput23 testtry + $sim $valgrind ./pcre2test -q $setstack $bmode $testdata/testinput23 testtry checkresult $? 23 "" fi fi @@ -839,7 +839,7 @@ for bmode in "$test8" "$test16" "$test32"; do if [ "$do24" = yes ] ; then echo $title24 - $sim $valgrind ${opt:+$vjs} ./pcre2test -q $setstack $bmode $opt $testdata/testinput24 testtry + $sim $valgrind ./pcre2test -q $setstack $bmode $testdata/testinput24 testtry checkresult $? 24 "" fi @@ -850,7 +850,7 @@ for bmode in "$test8" "$test16" "$test32"; do if [ $utf -eq 0 ] ; then echo " Skipped because UTF-$bits support is not available" else - $sim $valgrind ${opt:+$vjs} ./pcre2test -q $setstack $bmode $opt $testdata/testinput25 testtry + $sim $valgrind ./pcre2test -q $setstack $bmode $testdata/testinput25 testtry checkresult $? 25 "" fi fi diff --git a/configure.ac b/configure.ac index d713dc2..8ebd341 100644 --- a/configure.ac +++ b/configure.ac @@ -10,17 +10,17 @@ dnl be defined as -RC2, for example. For real releases, it should be empty. m4_define(pcre2_major, [10]) m4_define(pcre2_minor, [30]) -m4_define(pcre2_prerelease, [-DEV]) -m4_define(pcre2_date, [2017-03-05]) +m4_define(pcre2_prerelease, [-RC1]) +m4_define(pcre2_date, [2017-07-18]) # NOTE: The CMakeLists.txt file searches for the above variables in the first # 50 lines of this file. Please update that if the variables above are moved. # Libtool shared library interface versions (current:revision:age) -m4_define(libpcre2_8_version, [5:0:5]) -m4_define(libpcre2_16_version, [5:0:5]) -m4_define(libpcre2_32_version, [5:0:5]) -m4_define(libpcre2_posix_version, [1:1:0]) +m4_define(libpcre2_8_version, [6:0:6]) +m4_define(libpcre2_16_version, [6:0:6]) +m4_define(libpcre2_32_version, [6:0:6]) +m4_define(libpcre2_posix_version, [2:0:0]) AC_PREREQ(2.57) AC_INIT(PCRE2, pcre2_major.pcre2_minor[]pcre2_prerelease, , pcre2) @@ -277,7 +277,7 @@ AC_ARG_WITH(parens-nest-limit, AC_ARG_WITH(heap-limit, AS_HELP_STRING([--with-heap-limit=N], [default limit on heap memory (kilobytes, default=20000000)]), - , with_heap_limit=20000000) + , with_heap_limit=20000000) # Handle --with-match-limit=N AC_ARG_WITH(match-limit, @@ -301,7 +301,7 @@ AC_ARG_WITH(match-limit-depth, AC_ARG_WITH(match-limit-recursion,, , with_match_limit_recursion=UNSET) - + # Handle --enable-valgrind AC_ARG_ENABLE(valgrind, AS_HELP_STRING([--enable-valgrind], @@ -370,7 +370,7 @@ case "$enable_newline" in crlf) ac_pcre2_newline_value=3 ;; any) ac_pcre2_newline_value=4 ;; anycrlf) ac_pcre2_newline_value=5 ;; - nul) ac_pcre2_newline_value=6 ;; + nul) ac_pcre2_newline_value=6 ;; *) AC_MSG_ERROR([invalid argument \"$enable_newline\" to --enable-newline option]) ;; @@ -737,7 +737,7 @@ AC_DEFINE_UNQUOTED([MATCH_LIMIT_DEPTH], [$with_match_limit_depth], [ AC_DEFINE_UNQUOTED([HEAP_LIMIT], [$with_heap_limit], [ This limits the amount of memory that pcre2_match() may use while matching - a pattern. The value is in kilobytes.]) + a pattern. The value is in kilobytes.]) AC_DEFINE([MAX_NAME_SIZE], [32], [ This limit is parameterized just in case anybody ever wants to diff --git a/doc/html/NON-AUTOTOOLS-BUILD.txt b/doc/html/NON-AUTOTOOLS-BUILD.txt index fe20152..f8c6359 100644 --- a/doc/html/NON-AUTOTOOLS-BUILD.txt +++ b/doc/html/NON-AUTOTOOLS-BUILD.txt @@ -179,8 +179,8 @@ can skip ahead to the CMake section. STACK SIZE IN WINDOWS ENVIRONMENTS -Prior to release 10.30 the default system stack size of 1Mb in some Windows -environments caused issues with some tests. This should no longer be the case +Prior to release 10.30 the default system stack size of 1Mb in some Windows +environments caused issues with some tests. This should no longer be the case for 10.30 and later releases. diff --git a/doc/html/README.txt b/doc/html/README.txt index 6bf5b46..bed0513 100644 --- a/doc/html/README.txt +++ b/doc/html/README.txt @@ -173,7 +173,7 @@ library. They are also documented in the pcre2build man page. architectures. If you try to enable it on an unsupported architecture, there will be a compile time error. If you are running under SELinux you may also want to add --enable-jit-sealloc, which enables the use of an execmem - allocator in JIT that is compatible with SELinux. This has no effect if JIT + allocator in JIT that is compatible with SELinux. This has no effect if JIT is not enabled. . If you do not want to make use of the default support for UTF-8 Unicode @@ -198,13 +198,14 @@ library. They are also documented in the pcre2build man page. or starting a pattern with (*UCP). . You can build PCRE2 to recognize either CR or LF or the sequence CRLF, or any - of the preceding, or any of the Unicode newline sequences, as indicating the - end of a line. Whatever you specify at build time is the default; the caller - of PCRE2 can change the selection at run time. The default newline indicator - is a single LF character (the Unix standard). You can specify the default - newline indicator by adding --enable-newline-is-cr, --enable-newline-is-lf, - --enable-newline-is-crlf, --enable-newline-is-anycrlf, or - --enable-newline-is-any to the "configure" command, respectively. + of the preceding, or any of the Unicode newline sequences, or the NUL (zero) + character as indicating the end of a line. Whatever you specify at build time + is the default; the caller of PCRE2 can change the selection at run time. The + default newline indicator is a single LF character (the Unix standard). You + can specify the default newline indicator by adding --enable-newline-is-cr, + --enable-newline-is-lf, --enable-newline-is-crlf, + --enable-newline-is-anycrlf, --enable-newline-is-any, or + --enable-newline-is-nul to the "configure" command, respectively. . By default, the sequence \R in a pattern matches any Unicode line ending sequence. This is independent of the option specifying what PCRE2 considers @@ -227,15 +228,15 @@ library. They are also documented in the pcre2build man page. --with-parens-nest-limit=500 . PCRE2 has a counter that can be set to limit the amount of computing resource - it uses when matching a pattern with the Perl-compatible matching function. - If the limit is exceeded during a match, the match fails. The default is ten - million. You can change the default by setting, for example, + it uses when matching a pattern. If the limit is exceeded during a match, the + match fails. The default is ten million. You can change the default by + setting, for example, --with-match-limit=500000 on the "configure" command. This is just the default; individual calls to - pcre2_match() can supply their own value. There is more discussion in the - pcre2api man page (search for pcre2_set_match_limit). + pcre2_match() or pcre2_dfa_match() can supply their own value. There is more + discussion in the pcre2api man page (search for pcre2_set_match_limit). . There is a separate counter that limits the depth of nested backtracking during a matching process, which indirectly limits the amount of heap memory @@ -246,15 +247,15 @@ library. They are also documented in the pcre2build man page. There is more discussion in the pcre2api man page (search for pcre2_set_depth_limit). - -. You can also set an explicit limit on the amount of heap memory used by + +. You can also set an explicit limit on the amount of heap memory used by the pcre2_match() interpreter: - + --with-heap-limit=500 - - The units are kilobytes. This limit does not apply when the JIT optimization - (which has its own memory control features) is used. There is more discussion - on the pcre2api man page (search for pcre2_set_heap_limit). + + The units are kilobytes. This limit does not apply when the JIT optimization + (which has its own memory control features) is used. There is more discussion + on the pcre2api man page (search for pcre2_set_heap_limit). . In the 8-bit library, the default maximum compiled pattern size is around 64K bytes. You can increase this by adding --with-link-size=3 to the @@ -659,9 +660,10 @@ with the perltest.sh script, and test 5 checking PCRE2-specific things. Tests 6 and 7 check the pcre2_dfa_match() alternative matching function, in non-UTF mode and UTF-mode with Unicode property support, respectively. -Test 8 checks some internal offsets and code size features; it is run only when -the default "link size" of 2 is set (in other cases the sizes change) and when -Unicode support is enabled. +Test 8 checks some internal offsets and code size features, but it is run only +when Unicode support is enabled. The output is different in 8-bit, 16-bit, and +32-bit modes and for different link sizes, so there are different output files +for each mode and link size. Tests 9 and 10 are run only in 8-bit mode, and tests 11 and 12 are run only in 16-bit and 32-bit modes. These are tests that generate different output in @@ -671,7 +673,7 @@ Test 13 checks the handling of non-UTF characters greater than 255 by pcre2_dfa_match() in 16-bit and 32-bit modes. Test 14 contains some special UTF and UCP tests that give different output for -the different widths. +different code unit widths. Test 15 contains a number of tests that must not be run with JIT. They check, among other non-JIT things, the match-limiting features of the intepretive @@ -692,6 +694,9 @@ patterns to a file, and then reloading and checking them. Tests 21 and 22 test \C support when the use of \C is not locked out, without and with UTF support, respectively. Test 23 tests \C when it is locked out. +Tests 24 and 25 test the experimental pattern conversion functions, without and +with UTF support, respectively. + Character tables ---------------- @@ -710,7 +715,7 @@ specified for ./configure, a different version of pcre2_chartables.c is built by the program dftables (compiled from dftables.c), which uses the ANSI C character handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table sources. This means that the default C -locale which is set for your system will control the contents of these default +locale that is set for your system will control the contents of these default tables. You can change the default tables by editing pcre2_chartables.c and then re-building PCRE2. If you do this, you should take care to ensure that the file does not get automatically re-generated. The best way to do this is to @@ -765,6 +770,7 @@ The distribution should contain the files listed below. src/pcre2_compile.c ) src/pcre2_config.c ) src/pcre2_context.c ) + src/pcre2_convert.c ) src/pcre2_dfa_match.c ) src/pcre2_error.c ) src/pcre2_find_bracket.c ) @@ -804,7 +810,6 @@ The distribution should contain the files listed below. src/pcre2demo.c simple demonstration of coding calls to PCRE2 src/pcre2grep.c source of a grep utility that uses PCRE2 src/pcre2test.c comprehensive test program - src/pcre2_printint.c part of pcre2test src/pcre2_jit_test.c JIT test program (C) Auxiliary files: @@ -869,12 +874,12 @@ The distribution should contain the files listed below. (E) Auxiliary files for building PCRE2 "by hand" - pcre2.h.generic ) a version of the public PCRE2 header file + src/pcre2.h.generic ) a version of the public PCRE2 header file ) for use in non-"configure" environments - config.h.generic ) a version of config.h for use in non-"configure" + src/config.h.generic ) a version of config.h for use in non-"configure" ) environments Philip Hazel Email local part: ph10 Email domain: cam.ac.uk -Last updated: 17 June 2017 +Last updated: 18 July 2017 diff --git a/doc/html/pcre2.html b/doc/html/pcre2.html index 7a60d28..b61c579 100644 --- a/doc/html/pcre2.html +++ b/doc/html/pcre2.html @@ -137,7 +137,7 @@ large search tree against a string that will never match. Nested unlimited repeats in a pattern are a common example. PCRE2 provides some protection against this: see the pcre2_set_match_limit() function in the pcre2api -page. There is a similar function called pcre2_set_depth_limit() that can +page. There is a similar function called pcre2_set_depth_limit() that can be used to restrict the amount of memory that is used.


USER DOCUMENTATION
diff --git a/doc/html/pcre2_code_free.html b/doc/html/pcre2_code_free.html index e464a04..5fce3c5 100644 --- a/doc/html/pcre2_code_free.html +++ b/doc/html/pcre2_code_free.html @@ -26,8 +26,8 @@ DESCRIPTION

This function frees the memory used for a compiled pattern, including any -memory used by the JIT compiler. If the compiled pattern was created by a call -to pcre2_code_copy_with_tables(), the memory for the character tables is +memory used by the JIT compiler. If the compiled pattern was created by a call +to pcre2_code_copy_with_tables(), the memory for the character tables is also freed.

diff --git a/doc/html/pcre2_compile.html b/doc/html/pcre2_compile.html index da103cd..0a9eafa 100644 --- a/doc/html/pcre2_compile.html +++ b/doc/html/pcre2_compile.html @@ -64,7 +64,7 @@ The option bits are: PCRE2_ENDANCHORED Pattern can match only at end of subject PCRE2_EXTENDED Ignore white space and # comments PCRE2_FIRSTLINE Force matching to be before newline - PCRE2_LITERAL Pattern characters are all literal + PCRE2_LITERAL Pattern characters are all literal PCRE2_MATCH_UNSET_BACKREF Match unset back references PCRE2_MULTILINE ^ and $ match newlines within data PCRE2_NEVER_BACKSLASH_C Lock out the use of \C in patterns diff --git a/doc/html/pcre2_config.html b/doc/html/pcre2_config.html index 7929d62..465f6a1 100644 --- a/doc/html/pcre2_config.html +++ b/doc/html/pcre2_config.html @@ -45,7 +45,7 @@ point to a uint32_t integer variable. The available codes are: PCRE2_CONFIG_BSR Indicates what \R matches by default: PCRE2_BSR_UNICODE PCRE2_BSR_ANYCRLF - PCRE2_CONFIG_HEAPLIMIT Default heap memory limit + PCRE2_CONFIG_HEAPLIMIT Default heap memory limit PCRE2_CONFIG_DEPTHLIMIT Default backtracking depth limit PCRE2_CONFIG_JIT Availability of just-in-time compiler support (1=yes 0=no) PCRE2_CONFIG_JITTARGET Information (a string) about the target architecture for the JIT compiler @@ -57,7 +57,7 @@ point to a uint32_t integer variable. The available codes are: PCRE2_NEWLINE_CRLF PCRE2_NEWLINE_ANY PCRE2_NEWLINE_ANYCRLF - PCRE2_NEWLINE_NUL + PCRE2_NEWLINE_NUL PCRE2_CONFIG_PARENSLIMIT Default parentheses nesting limit PCRE2_CONFIG_RECURSIONLIMIT Obsolete: use PCRE2_CONFIG_DEPTHLIMIT PCRE2_CONFIG_STACKRECURSE Obsolete: always returns 0 diff --git a/doc/html/pcre2_converted_pattern_free.html b/doc/html/pcre2_converted_pattern_free.html index 961f04f..11adefd 100644 --- a/doc/html/pcre2_converted_pattern_free.html +++ b/doc/html/pcre2_converted_pattern_free.html @@ -26,8 +26,8 @@ DESCRIPTION

This function is part of an experimental set of pattern conversion functions. -It frees the memory occupied by a converted pattern that was obtained by -calling pcre2_pattern_convert() with arguments that caused it to place +It frees the memory occupied by a converted pattern that was obtained by +calling pcre2_pattern_convert() with arguments that caused it to place the converted pattern into newly obtained heap memory.

diff --git a/doc/html/pcre2_maketables.html b/doc/html/pcre2_maketables.html index 995c23a..6d240e3 100644 --- a/doc/html/pcre2_maketables.html +++ b/doc/html/pcre2_maketables.html @@ -25,7 +25,7 @@ SYNOPSIS DESCRIPTION

-This function builds a set of character tables for character code points that +This function builds a set of character tables for character code points that are less than 256. These can be passed to pcre2_compile() in a compile context in order to override the internal, built-in tables (which were either defaulted or made by pcre2_maketables() when PCRE2 was compiled). See the diff --git a/doc/html/pcre2_match.html b/doc/html/pcre2_match.html index 724a39f..5f6f0b1 100644 --- a/doc/html/pcre2_match.html +++ b/doc/html/pcre2_match.html @@ -43,14 +43,14 @@ offsets to captured substrings. Its arguments are: A match context is needed only if you want to: 

   Set up a callout function
-  Set a matching offset limit 
-  Change the heap memory limit 
-  Change the backtracking match limit 
+  Set a matching offset limit
+  Change the heap memory limit
+  Change the backtracking match limit
   Change the backtracking depth limit
   Set custom memory management specifically for the match
The length and startoffset values are code -units, not characters. The length may be given as PCRE2_ZERO_TERMINATE for a +units, not characters. The length may be given as PCRE2_ZERO_TERMINATE for a subject that is terminated by a binary zero code unit. The options are: 
   PCRE2_ANCHORED          Match only at the first position
@@ -59,7 +59,7 @@ subject that is terminated by a binary zero code unit. The options are:
   PCRE2_NOTEOL            Subject string is not the end of a line
   PCRE2_NOTEMPTY          An empty string is not a valid match
   PCRE2_NOTEMPTY_ATSTART  An empty string at the start of the subject is not a valid match
-  PCRE2_NO_JIT            Do not use JIT matching                          
+  PCRE2_NO_JIT            Do not use JIT matching
   PCRE2_NO_UTF_CHECK      Do not check the subject for UTF validity (only relevant if PCRE2_UTF
                            was set at compile time)
   PCRE2_PARTIAL_HARD      Return PCRE2_ERROR_PARTIAL for a partial match even if there is a full match
diff --git a/doc/html/pcre2_pattern_info.html b/doc/html/pcre2_pattern_info.html
index d07f9ed..ae3e7ff 100644
--- a/doc/html/pcre2_pattern_info.html
+++ b/doc/html/pcre2_pattern_info.html
@@ -48,7 +48,7 @@ request are as follows:
                                1 first code unit is set
                                2 start of string or after newline
   PCRE2_INFO_FIRSTCODEUNIT   First code unit when type is 1
-  PCRE2_INFO_FRAMESIZE       Size of backtracking frame 
+  PCRE2_INFO_FRAMESIZE       Size of backtracking frame
   PCRE2_INFO_HASBACKSLASHC   Return 1 if pattern contains \C
   PCRE2_INFO_HASCRORLF       Return 1 if explicit CR or LF matches exist in the pattern
   PCRE2_INFO_HEAPLIMIT       Heap memory limit if set, otherwise PCRE2_ERROR_UNSET
@@ -71,7 +71,7 @@ request are as follows:
                                PCRE2_NEWLINE_CRLF
                                PCRE2_NEWLINE_ANY
                                PCRE2_NEWLINE_ANYCRLF
-                               PCRE2_NEWLINE_NUL 
+                               PCRE2_NEWLINE_NUL
   PCRE2_INFO_RECURSIONLIMIT  Obsolete synonym for PCRE2_INFO_DEPTHLIMIT
   PCRE2_INFO_SIZE            Size of compiled pattern
diff --git a/doc/html/pcre2_set_newline.html b/doc/html/pcre2_set_newline.html index a078f69..ba81300 100644 --- a/doc/html/pcre2_set_newline.html +++ b/doc/html/pcre2_set_newline.html @@ -35,7 +35,7 @@ matching patterns. The second argument must be one of: PCRE2_NEWLINE_CRLF CR followed by LF only PCRE2_NEWLINE_ANYCRLF Any of the above PCRE2_NEWLINE_ANY Any Unicode newline sequence - PCRE2_NEWLINE_NUL The NUL character (binary zero) + PCRE2_NEWLINE_NUL The NUL character (binary zero) The result is zero for success or PCRE2_ERROR_BADDATA if the second argument is invalid. diff --git a/doc/html/pcre2_set_recursion_limit.html b/doc/html/pcre2_set_recursion_limit.html index c415aa3..9ff68c2 100644 --- a/doc/html/pcre2_set_recursion_limit.html +++ b/doc/html/pcre2_set_recursion_limit.html @@ -26,7 +26,7 @@ SYNOPSIS DESCRIPTION

-This function is obsolete and should not be used in new code. Use +This function is obsolete and should not be used in new code. Use pcre2_set_depth_limit() instead.

diff --git a/doc/html/pcre2_substitute.html b/doc/html/pcre2_substitute.html index c937802..2215ce9 100644 --- a/doc/html/pcre2_substitute.html +++ b/doc/html/pcre2_substitute.html @@ -60,7 +60,7 @@ want to: The length, startoffset and rlength values are code units, not characters, as is the contents of the variable pointed at by outlengthptr, which is updated to the actual length of the new string. -The subject and replacement lengths can be given as PCRE2_ZERO_TERMINATED for +The subject and replacement lengths can be given as PCRE2_ZERO_TERMINATED for zero-terminated strings. The options are: 

   PCRE2_ANCHORED             Match only at the first position
diff --git a/doc/html/pcre2build.html b/doc/html/pcre2build.html
index 3dfe07f..823e605 100644
--- a/doc/html/pcre2build.html
+++ b/doc/html/pcre2build.html
@@ -87,10 +87,10 @@ Options that specify values have names that start with --with.
 
BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES

 

 By default, a library called libpcre2-8 is built, containing functions
-that take string arguments contained in vectors of bytes, interpreted either as
+that take string arguments contained in arrays of bytes, interpreted either as
 single-byte characters, or UTF-8 strings. You can also build two other
 libraries, called libpcre2-16 and libpcre2-32, which process
-strings that are contained in vectors of 16-bit and 32-bit code units,
+strings that are contained in arrays of 16-bit and 32-bit code units,
 respectively. These can be interpreted either as single-unit characters or
 UTF-16/UTF-32 strings. To build these additional libraries, add one or both of
 the following to the configure command:
@@ -208,19 +208,23 @@ to the configure command. There is a fourth option, specified by
   --enable-newline-is-anycrlf
which causes PCRE2 to recognize any of the three sequences CR, LF, or CRLF as -indicating a line ending. Finally, a fifth option, specified by +indicating a line ending. A fifth option, specified by 
   --enable-newline-is-any
causes PCRE2 to recognize any Unicode newline sequence. The Unicode newline sequences are the three just mentioned, plus the single characters VT (vertical tab, U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line -separator, U+2028), and PS (paragraph separator, U+2029). +separator, U+2028), and PS (paragraph separator, U+2029). The final option is +
+  --enable-newline-is-nul
+
+which causes NUL (binary zero) is set as the default line-ending character.

Whatever default line ending convention is selected when PCRE2 is built can be overridden by applications that use the library. At build time it is -conventional to use the standard for your operating system. +recommended to use the standard for your operating system.


WHAT \R MATCHES

@@ -301,7 +305,9 @@ because the size of each backtracking "frame" depends on the number of capturing parentheses in a pattern, the amount of heap that is used before the limit is reached varies from pattern to pattern. This limit was more useful in versions before 10.30, where function recursion was used for backtracking. -However, as well as applying to pcre2_match(), this limit also controls +

+

+As well as applying to pcre2_match(), the depth limit also controls the depth of recursive function calls in pcre2_dfa_match(). These are used for lookaround assertions, atomic groups, and recursion within patterns. The limit does not apply to JIT matching. @@ -559,7 +565,7 @@ Cambridge, England.


REVISION

-Last updated: 17 June 2017 +Last updated: 18 July 2017
Copyright © 1997-2017 University of Cambridge.
diff --git a/doc/html/pcre2compat.html b/doc/html/pcre2compat.html index 5c890b0..e6d2e7e 100644 --- a/doc/html/pcre2compat.html +++ b/doc/html/pcre2compat.html @@ -85,7 +85,7 @@ documentation for details.

8. Subroutine calls (whether recursive or not) were treated as atomic groups up to PCRE2 release 10.23, but from release 10.30 this changed, and backtracking -into subroutine calls is now supported, as in Perl. +into subroutine calls is now supported, as in Perl.

9. If any of the backtracking control verbs are used in a subpattern that is diff --git a/doc/html/pcre2grep.html b/doc/html/pcre2grep.html index 0a028a0..b5565a8 100644 --- a/doc/html/pcre2grep.html +++ b/doc/html/pcre2grep.html @@ -517,20 +517,20 @@ memory. There are three options that set resource limits for matching. The --match-limit option provides a means of limiting computing resource usage when processing patterns that are not going to match, but which have a very large number of possibilities in their search trees. The classic example -is a pattern that uses nested unlimited repeats. Internally, PCRE2 has a -counter that is incremented each time around its main processing loop. If the +is a pattern that uses nested unlimited repeats. Internally, PCRE2 has a +counter that is incremented each time around its main processing loop. If the value set by --match-limit is reached, an error occurs.

The --heap-limit option specifies, as a number of kilobytes, the amount of heap memory that may be used for matching. Heap memory is needed only if matching the pattern requires a significant number of nested backtracking -points to be remembered. This parameter can be set to zero to forbid the use of +points to be remembered. This parameter can be set to zero to forbid the use of heap memory altogether.

The --depth-limit option limits the depth of nested backtracking points, -which indirectly limits the amount of memory that is used. The amount of memory +which indirectly limits the amount of memory that is used. The amount of memory needed for each backtracking point depends on the number of capturing parentheses in the pattern, so the amount of memory that is used before this limit acts varies from pattern to pattern. This limit is of use only if it is @@ -538,7 +538,7 @@ set smaller than --match-limit.

There are no short forms for these options. The default settings are specified -when the PCRE2 library is compiled, with the default defaults being very large +when the PCRE2 library is compiled, with the default defaults being very large and so effectively unlimited.

@@ -841,7 +841,7 @@ patterns are ignored by pcre2grep. A callout in a PCRE2 pattern is of the form (?C<arg>) where the argument is either a number or a quoted string (see the pcre2callout -documentation for details). Numbered callouts are ignored by pcre2grep; +documentation for details). Numbered callouts are ignored by pcre2grep; only callouts with string arguments are useful.


@@ -892,10 +892,10 @@ Echoing a specific string If the callout string starts with a pipe (vertical bar) character, the rest of the string is written to the output, having been passed through the same escape processing as text from the --output option. This provides a simple echoing -facility that avoids calling an external program or script. No terminator is +facility that avoids calling an external program or script. No terminator is added to the string, so if you want a newline, you must include it explicitly. -Matching continues normally after the string is output. If you want to see only -the callout output but not any output from an actual match, you should end the +Matching continues normally after the string is output. If you want to see only +the callout output but not any output from an actual match, you should end the relevant pattern with (*FAIL).


MATCHING ERRORS
@@ -910,8 +910,8 @@ there are more than 20 such errors, pcre2grep gives up.

The --match-limit option of pcre2grep can be used to set the -overall resource limit. There are also other limits that affect the amount of -memory used during matching; see the discussion of --heap-limit and +overall resource limit. There are also other limits that affect the amount of +memory used during matching; see the discussion of --heap-limit and --depth-limit above.


DIAGNOSTICS
diff --git a/doc/html/pcre2perform.html b/doc/html/pcre2perform.html index 1a53493..28f4f73 100644 --- a/doc/html/pcre2perform.html +++ b/doc/html/pcre2perform.html @@ -29,7 +29,7 @@ of them.
COMPILED PATTERN MEMORY USAGE

Patterns are compiled by PCRE2 into a reasonably efficient interpretive code, -so that most simple patterns do not use much memory for storing the compiled +so that most simple patterns do not use much memory for storing the compiled version. However, there is one case where the memory usage of a compiled pattern can be unexpectedly large. If a parenthesized subpattern has a quantifier with a minimum greater than 1 and/or a limited maximum, the whole @@ -91,7 +91,7 @@ vector is used. Rewriting patterns to be time-efficient, as described below, may also reduce the memory requirements.

-In contrast to pcre2_match(), pcre2_dfa_match() does use recursive +In contrast to pcre2_match(), pcre2_dfa_match() does use recursive function calls, but only for processing atomic groups, lookaround assertions, and recursion within the pattern. Too much nested recursion may cause stack issues. The "match depth" parameter can be used to limit the depth of function @@ -184,7 +184,7 @@ appreciable time with strings longer than about 20 characters.

In many cases, the solution to this kind of performance issue is to use an -atomic group or a possessive quantifier. This can often reduce memory +atomic group or a possessive quantifier. This can often reduce memory requirements as well. As another example, consider this pattern: 

   ([^<]|<(?!inet))+
@@ -205,7 +205,7 @@ are "swallowed" in one item inside the parentheses, and a possessive quantifier
 is used to stop any backtracking into the runs of non-"<" characters. This
 version also uses a lot less memory because entry to a new set of parentheses
 happens only when a "<" character that is not followed by "inet" is encountered
-(and we assume this is relatively rare). 
+(and we assume this is relatively rare).
 

 

 This example shows that one way of optimizing performance when matching long
@@ -216,10 +216,10 @@ than one character whenever possible.
 SETTING RESOURCE LIMITS
 

 

-You can set limits on the amount of processing that takes place when matching, 
+You can set limits on the amount of processing that takes place when matching,
 and on the amount of heap memory that is used. The default values of the limits
 are very large, and unlikely ever to operate. They can be changed when PCRE2 is
-built, and they can also be set when pcre2_match() or 
+built, and they can also be set when pcre2_match() or
 pcre2_dfa_match() is called. For details of these interfaces, see the
 pcre2build
 documentation and the section entitled
diff --git a/doc/html/pcre2syntax.html b/doc/html/pcre2syntax.html
index ce7e7da..9098f47 100644
--- a/doc/html/pcre2syntax.html
+++ b/doc/html/pcre2syntax.html
@@ -430,11 +430,11 @@ but some of them use Unicode properties if PCRE2_UCP is set. You can use
   (?i)            caseless
   (?J)            allow duplicate names
   (?m)            multiline
-  (?n)            no auto capture 
+  (?n)            no auto capture
   (?s)            single line (dotall)
   (?U)            default ungreedy (lazy)
   (?x)            extended: ignore white space except in classes
-  (?xx)           as (?x) but also ignore space and tab in classes 
+  (?xx)           as (?x) but also ignore space and tab in classes
   (?-...)         unset option(s)
The following are recognized only at the very start of a pattern or after one diff --git a/doc/pcre2.3 b/doc/pcre2.3 index 02dddaf..83a7655 100644 --- a/doc/pcre2.3 +++ b/doc/pcre2.3 @@ -130,7 +130,7 @@ against this: see the \fBpcre2_set_match_limit()\fP function in the .\" HREF \fBpcre2api\fP .\" -page. There is a similar function called \fBpcre2_set_depth_limit()\fP that can +page. There is a similar function called \fBpcre2_set_depth_limit()\fP that can be used to restrict the amount of memory that is used. . . diff --git a/doc/pcre2.txt b/doc/pcre2.txt index 186cbc7..bb0314f 100644 --- a/doc/pcre2.txt +++ b/doc/pcre2.txt @@ -170,8 +170,8 @@ REVISION Last updated: 01 April 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2API(3) Library Functions Manual PCRE2API(3) @@ -3432,8 +3432,8 @@ REVISION Last updated: 10 July 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2BUILD(3) Library Functions Manual PCRE2BUILD(3) @@ -3487,10 +3487,10 @@ PCRE2 BUILD-TIME OPTIONS BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES By default, a library called libpcre2-8 is built, containing functions - that take string arguments contained in vectors of bytes, interpreted + that take string arguments contained in arrays of bytes, interpreted either as single-byte characters, or UTF-8 strings. You can also build two other libraries, called libpcre2-16 and libpcre2-32, which process - strings that are contained in vectors of 16-bit and 32-bit code units, + strings that are contained in arrays of 16-bit and 32-bit code units, respectively. These can be interpreted either as single-unit characters or UTF-16/UTF-32 strings. To build these additional libraries, add one or both of the following to the configure command: @@ -3609,7 +3609,7 @@ NEWLINE RECOGNITION --enable-newline-is-anycrlf which causes PCRE2 to recognize any of the three sequences CR, LF, or - CRLF as indicating a line ending. Finally, a fifth option, specified by + CRLF as indicating a line ending. A fifth option, specified by --enable-newline-is-any @@ -3617,97 +3617,103 @@ NEWLINE RECOGNITION newline sequences are the three just mentioned, plus the single charac- ters VT (vertical tab, U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS (paragraph separator, - U+2029). + U+2029). The final option is + + --enable-newline-is-nul + + which causes NUL (binary zero) is set as the default line-ending char- + acter. Whatever default line ending convention is selected when PCRE2 is built - can be overridden by applications that use the library. At build time - it is conventional to use the standard for your operating system. + can be overridden by applications that use the library. At build time + it is recommended to use the standard for your operating system. WHAT \R MATCHES - By default, the sequence \R in a pattern matches any Unicode newline - sequence, independently of what has been selected as the line ending + By default, the sequence \R in a pattern matches any Unicode newline + sequence, independently of what has been selected as the line ending sequence. If you specify --enable-bsr-anycrlf - the default is changed so that \R matches only CR, LF, or CRLF. What- - ever is selected when PCRE2 is built can be overridden by applications + the default is changed so that \R matches only CR, LF, or CRLF. What- + ever is selected when PCRE2 is built can be overridden by applications that use the library. HANDLING VERY LARGE PATTERNS - Within a compiled pattern, offset values are used to point from one - part to another (for example, from an opening parenthesis to an alter- - nation metacharacter). By default, in the 8-bit and 16-bit libraries, - two-byte values are used for these offsets, leading to a maximum size - for a compiled pattern of around 64K code units. This is sufficient to + Within a compiled pattern, offset values are used to point from one + part to another (for example, from an opening parenthesis to an alter- + nation metacharacter). By default, in the 8-bit and 16-bit libraries, + two-byte values are used for these offsets, leading to a maximum size + for a compiled pattern of around 64K code units. This is sufficient to handle all but the most gigantic patterns. Nevertheless, some people do - want to process truly enormous patterns, so it is possible to compile - PCRE2 to use three-byte or four-byte offsets by adding a setting such + want to process truly enormous patterns, so it is possible to compile + PCRE2 to use three-byte or four-byte offsets by adding a setting such as --with-link-size=3 - to the configure command. The value given must be 2, 3, or 4. For the - 16-bit library, a value of 3 is rounded up to 4. In these libraries, - using longer offsets slows down the operation of PCRE2 because it has - to load additional data when handling them. For the 32-bit library the - value is always 4 and cannot be overridden; the value of --with-link- + to the configure command. The value given must be 2, 3, or 4. For the + 16-bit library, a value of 3 is rounded up to 4. In these libraries, + using longer offsets slows down the operation of PCRE2 because it has + to load additional data when handling them. For the 32-bit library the + value is always 4 and cannot be overridden; the value of --with-link- size is ignored. LIMITING PCRE2 RESOURCE USAGE The pcre2_match() function increments a counter each time it goes round - its main loop. Putting a limit on this counter controls the amount of - computing resource used by a single call to pcre2_match(). The limit + its main loop. Putting a limit on this counter controls the amount of + computing resource used by a single call to pcre2_match(). The limit can be changed at run time, as described in the pcre2api documentation. - The default is 10 million, but this can be changed by adding a setting + The default is 10 million, but this can be changed by adding a setting such as --with-match-limit=500000 - to the configure command. This setting also applies to the - pcre2_dfa_match() matching function, and to JIT matching (though the + to the configure command. This setting also applies to the + pcre2_dfa_match() matching function, and to JIT matching (though the counting is done differently). - The pcre2_match() function starts out using a 20K vector on the system - stack to record backtracking points. The more nested backtracking + The pcre2_match() function starts out using a 20K vector on the system + stack to record backtracking points. The more nested backtracking points there are (that is, the deeper the search tree), the more memory - is needed. If the initial vector is not large enough, heap memory is + is needed. If the initial vector is not large enough, heap memory is used, up to a certain limit, which is specified in kilobytes. The limit can be changed at run time, as described in the pcre2api documentation. - The default limit (in effect unlimited) is 20 million. You can change + The default limit (in effect unlimited) is 20 million. You can change this by a setting such as --with-heap-limit=500 - which limits the amount of heap to 500 kilobytes. This limit applies - only to interpretive matching in pcre2_match(). It does not apply when - JIT (which has its own memory arrangements) is used, nor does it apply + which limits the amount of heap to 500 kilobytes. This limit applies + only to interpretive matching in pcre2_match(). It does not apply when + JIT (which has its own memory arrangements) is used, nor does it apply to pcre2_dfa_match(). - You can also explicitly limit the depth of nested backtracking in the + You can also explicitly limit the depth of nested backtracking in the pcre2_match() interpreter. This limit defaults to the value that is set - for --with-match-limit. You can set a lower default limit by adding, + for --with-match-limit. You can set a lower default limit by adding, for example, --with-match-limit_depth=10000 - to the configure command. This value can be overridden at run time. - This depth limit indirectly limits the amount of heap memory that is - used, but because the size of each backtracking "frame" depends on the - number of capturing parentheses in a pattern, the amount of heap that - is used before the limit is reached varies from pattern to pattern. - This limit was more useful in versions before 10.30, where function - recursion was used for backtracking. However, as well as applying to - pcre2_match(), this limit also controls the depth of recursive function - calls in pcre2_dfa_match(). These are used for lookaround assertions, - atomic groups, and recursion within patterns. The limit does not apply - to JIT matching. + to the configure command. This value can be overridden at run time. + This depth limit indirectly limits the amount of heap memory that is + used, but because the size of each backtracking "frame" depends on the + number of capturing parentheses in a pattern, the amount of heap that + is used before the limit is reached varies from pattern to pattern. + This limit was more useful in versions before 10.30, where function + recursion was used for backtracking. + + As well as applying to pcre2_match(), the depth limit also controls the + depth of recursive function calls in pcre2_dfa_match(). These are used + for lookaround assertions, atomic groups, and recursion within pat- + terns. The limit does not apply to JIT matching. CREATING CHARACTER TABLES AT BUILD TIME @@ -3969,11 +3975,11 @@ AUTHOR REVISION - Last updated: 17 June 2017 + Last updated: 18 July 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2CALLOUT(3) Library Functions Manual PCRE2CALLOUT(3) @@ -4366,8 +4372,8 @@ REVISION Last updated: 14 April 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2COMPAT(3) Library Functions Manual PCRE2COMPAT(3) @@ -4564,8 +4570,8 @@ REVISION Last updated: 18 April 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2JIT(3) Library Functions Manual PCRE2JIT(3) @@ -4958,8 +4964,8 @@ REVISION Last updated: 31 March 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2LIMITS(3) Library Functions Manual PCRE2LIMITS(3) @@ -5029,8 +5035,8 @@ REVISION Last updated: 30 March 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2MATCHING(3) Library Functions Manual PCRE2MATCHING(3) @@ -5248,8 +5254,8 @@ REVISION Last updated: 29 September 2014 Copyright (c) 1997-2014 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2PARTIAL(3) Library Functions Manual PCRE2PARTIAL(3) @@ -5688,8 +5694,8 @@ REVISION Last updated: 22 December 2014 Copyright (c) 1997-2014 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2PATTERN(3) Library Functions Manual PCRE2PATTERN(3) @@ -8790,8 +8796,8 @@ REVISION Last updated: 05 July 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2PERFORM(3) Library Functions Manual PCRE2PERFORM(3) @@ -9018,8 +9024,8 @@ REVISION Last updated: 08 April 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2POSIX(3) Library Functions Manual PCRE2POSIX(3) @@ -9326,8 +9332,8 @@ REVISION Last updated: 15 June 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2SAMPLE(3) Library Functions Manual PCRE2SAMPLE(3) @@ -9595,8 +9601,8 @@ REVISION Last updated: 21 March 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2SYNTAX(3) Library Functions Manual PCRE2SYNTAX(3) @@ -10043,8 +10049,8 @@ REVISION Last updated: 17 June 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRE2UNICODE(3) Library Functions Manual PCRE2UNICODE(3) @@ -10300,5 +10306,5 @@ REVISION Last updated: 17 May 2017 Copyright (c) 1997-2017 University of Cambridge. ------------------------------------------------------------------------------ - - + + diff --git a/doc/pcre2_code_free.3 b/doc/pcre2_code_free.3 index 58a8b1c..7376869 100644 --- a/doc/pcre2_code_free.3 +++ b/doc/pcre2_code_free.3 @@ -14,8 +14,8 @@ PCRE2 - Perl-compatible regular expressions (revised API) .rs .sp This function frees the memory used for a compiled pattern, including any -memory used by the JIT compiler. If the compiled pattern was created by a call -to \fBpcre2_code_copy_with_tables()\fP, the memory for the character tables is +memory used by the JIT compiler. If the compiled pattern was created by a call +to \fBpcre2_code_copy_with_tables()\fP, the memory for the character tables is also freed. .P There is a complete description of the PCRE2 native API in the diff --git a/doc/pcre2_compile.3 b/doc/pcre2_compile.3 index acc5736..19f35c3 100644 --- a/doc/pcre2_compile.3 +++ b/doc/pcre2_compile.3 @@ -52,7 +52,7 @@ The option bits are: PCRE2_ENDANCHORED Pattern can match only at end of subject PCRE2_EXTENDED Ignore white space and # comments PCRE2_FIRSTLINE Force matching to be before newline - PCRE2_LITERAL Pattern characters are all literal + PCRE2_LITERAL Pattern characters are all literal PCRE2_MATCH_UNSET_BACKREF Match unset back references PCRE2_MULTILINE ^ and $ match newlines within data PCRE2_NEVER_BACKSLASH_C Lock out the use of \eC in patterns diff --git a/doc/pcre2_config.3 b/doc/pcre2_config.3 index 6c48da6..1f99370 100644 --- a/doc/pcre2_config.3 +++ b/doc/pcre2_config.3 @@ -31,7 +31,7 @@ point to a uint32_t integer variable. The available codes are: PCRE2_CONFIG_BSR Indicates what \eR matches by default: PCRE2_BSR_UNICODE PCRE2_BSR_ANYCRLF - PCRE2_CONFIG_HEAPLIMIT Default heap memory limit + PCRE2_CONFIG_HEAPLIMIT Default heap memory limit PCRE2_CONFIG_DEPTHLIMIT Default backtracking depth limit .\" JOIN PCRE2_CONFIG_JIT Availability of just-in-time compiler @@ -47,7 +47,7 @@ point to a uint32_t integer variable. The available codes are: PCRE2_NEWLINE_CRLF PCRE2_NEWLINE_ANY PCRE2_NEWLINE_ANYCRLF - PCRE2_NEWLINE_NUL + PCRE2_NEWLINE_NUL PCRE2_CONFIG_PARENSLIMIT Default parentheses nesting limit PCRE2_CONFIG_RECURSIONLIMIT Obsolete: use PCRE2_CONFIG_DEPTHLIMIT PCRE2_CONFIG_STACKRECURSE Obsolete: always returns 0 diff --git a/doc/pcre2_converted_pattern_free.3 b/doc/pcre2_converted_pattern_free.3 index 1f4c8e6..687e078 100644 --- a/doc/pcre2_converted_pattern_free.3 +++ b/doc/pcre2_converted_pattern_free.3 @@ -14,8 +14,8 @@ PCRE2 - Perl-compatible regular expressions (revised API) .rs .sp This function is part of an experimental set of pattern conversion functions. -It frees the memory occupied by a converted pattern that was obtained by -calling \fBpcre2_pattern_convert()\fP with arguments that caused it to place +It frees the memory occupied by a converted pattern that was obtained by +calling \fBpcre2_pattern_convert()\fP with arguments that caused it to place the converted pattern into newly obtained heap memory. .P The pattern conversion functions are described in the diff --git a/doc/pcre2_dfa_match.3 b/doc/pcre2_dfa_match.3 index 32a22c8..7839145 100644 --- a/doc/pcre2_dfa_match.3 +++ b/doc/pcre2_dfa_match.3 @@ -43,17 +43,17 @@ The options are: PCRE2_NOTBOL Subject is not the beginning of a line PCRE2_NOTEOL Subject is not the end of a line PCRE2_NOTEMPTY An empty string is not a valid match -.\" JOIN +.\" JOIN PCRE2_NOTEMPTY_ATSTART An empty string at the start of the subject is not a valid match -.\" JOIN +.\" JOIN PCRE2_NO_UTF_CHECK Do not check the subject for UTF validity (only relevant if PCRE2_UTF was set at compile time) -.\" JOIN +.\" JOIN PCRE2_PARTIAL_HARD Return PCRE2_ERROR_PARTIAL for a partial match even if there is a full match -.\" JOIN +.\" JOIN PCRE2_PARTIAL_SOFT Return PCRE2_ERROR_PARTIAL for a partial match if no full matches are found PCRE2_DFA_RESTART Restart after a partial match diff --git a/doc/pcre2_maketables.3 b/doc/pcre2_maketables.3 index 1250315..740954b 100644 --- a/doc/pcre2_maketables.3 +++ b/doc/pcre2_maketables.3 @@ -12,7 +12,7 @@ PCRE2 - Perl-compatible regular expressions (revised API) .SH DESCRIPTION .rs .sp -This function builds a set of character tables for character code points that +This function builds a set of character tables for character code points that are less than 256. These can be passed to \fBpcre2_compile()\fP in a compile context in order to override the internal, built-in tables (which were either defaulted or made by \fBpcre2_maketables()\fP when PCRE2 was compiled). See the diff --git a/doc/pcre2_match.3 b/doc/pcre2_match.3 index feaa470..ee4e159 100644 --- a/doc/pcre2_match.3 +++ b/doc/pcre2_match.3 @@ -31,14 +31,14 @@ offsets to captured substrings. Its arguments are: A match context is needed only if you want to: .sp Set up a callout function - Set a matching offset limit - Change the heap memory limit - Change the backtracking match limit + Set a matching offset limit + Change the heap memory limit + Change the backtracking match limit Change the backtracking depth limit Set custom memory management specifically for the match .sp The \fIlength\fP and \fIstartoffset\fP values are code -units, not characters. The length may be given as PCRE2_ZERO_TERMINATE for a +units, not characters. The length may be given as PCRE2_ZERO_TERMINATE for a subject that is terminated by a binary zero code unit. The options are: .sp PCRE2_ANCHORED Match only at the first position @@ -49,7 +49,7 @@ subject that is terminated by a binary zero code unit. The options are: .\" JOIN PCRE2_NOTEMPTY_ATSTART An empty string at the start of the subject is not a valid match - PCRE2_NO_JIT Do not use JIT matching + PCRE2_NO_JIT Do not use JIT matching .\" JOIN PCRE2_NO_UTF_CHECK Do not check the subject for UTF validity (only relevant if PCRE2_UTF diff --git a/doc/pcre2_pattern_info.3 b/doc/pcre2_pattern_info.3 index 3c3bae5..256e386 100644 --- a/doc/pcre2_pattern_info.3 +++ b/doc/pcre2_pattern_info.3 @@ -38,7 +38,7 @@ request are as follows: 1 first code unit is set 2 start of string or after newline PCRE2_INFO_FIRSTCODEUNIT First code unit when type is 1 - PCRE2_INFO_FRAMESIZE Size of backtracking frame + PCRE2_INFO_FRAMESIZE Size of backtracking frame PCRE2_INFO_HASBACKSLASHC Return 1 if pattern contains \eC .\" JOIN PCRE2_INFO_HASCRORLF Return 1 if explicit CR or LF matches @@ -71,7 +71,7 @@ request are as follows: PCRE2_NEWLINE_CRLF PCRE2_NEWLINE_ANY PCRE2_NEWLINE_ANYCRLF - PCRE2_NEWLINE_NUL + PCRE2_NEWLINE_NUL PCRE2_INFO_RECURSIONLIMIT Obsolete synonym for PCRE2_INFO_DEPTHLIMIT PCRE2_INFO_SIZE Size of compiled pattern .sp diff --git a/doc/pcre2_set_newline.3 b/doc/pcre2_set_newline.3 index 5d58701..0bccfc7 100644 --- a/doc/pcre2_set_newline.3 +++ b/doc/pcre2_set_newline.3 @@ -23,7 +23,7 @@ matching patterns. The second argument must be one of: PCRE2_NEWLINE_CRLF CR followed by LF only PCRE2_NEWLINE_ANYCRLF Any of the above PCRE2_NEWLINE_ANY Any Unicode newline sequence - PCRE2_NEWLINE_NUL The NUL character (binary zero) + PCRE2_NEWLINE_NUL The NUL character (binary zero) .sp The result is zero for success or PCRE2_ERROR_BADDATA if the second argument is invalid. diff --git a/doc/pcre2_set_recursion_limit.3 b/doc/pcre2_set_recursion_limit.3 index 1b74456..26f4257 100644 --- a/doc/pcre2_set_recursion_limit.3 +++ b/doc/pcre2_set_recursion_limit.3 @@ -14,7 +14,7 @@ PCRE2 - Perl-compatible regular expressions (revised API) .SH DESCRIPTION .rs .sp -This function is obsolete and should not be used in new code. Use +This function is obsolete and should not be used in new code. Use \fBpcre2_set_depth_limit()\fP instead. .P There is a complete description of the PCRE2 native API in the diff --git a/doc/pcre2_substitute.3 b/doc/pcre2_substitute.3 index 17de3ec..7da668c 100644 --- a/doc/pcre2_substitute.3 +++ b/doc/pcre2_substitute.3 @@ -48,7 +48,7 @@ want to: The \fIlength\fP, \fIstartoffset\fP and \fIrlength\fP values are code units, not characters, as is the contents of the variable pointed at by \fIoutlengthptr\fP, which is updated to the actual length of the new string. -The subject and replacement lengths can be given as PCRE2_ZERO_TERMINATED for +The subject and replacement lengths can be given as PCRE2_ZERO_TERMINATED for zero-terminated strings. The options are: .sp PCRE2_ANCHORED Match only at the first position diff --git a/doc/pcre2build.3 b/doc/pcre2build.3 index 8b081d9..7586d22 100644 --- a/doc/pcre2build.3 +++ b/doc/pcre2build.3 @@ -1,4 +1,4 @@ -.TH PCRE2BUILD 3 "17 June 2017" "PCRE2 10.30" +.TH PCRE2BUILD 3 "18 July 2017" "PCRE2 10.30" .SH NAME PCRE2 - Perl-compatible regular expressions (revised API) . @@ -66,10 +66,10 @@ Options that specify values have names that start with --with. .rs .sp By default, a library called \fBlibpcre2-8\fP is built, containing functions -that take string arguments contained in vectors of bytes, interpreted either as +that take string arguments contained in arrays of bytes, interpreted either as single-byte characters, or UTF-8 strings. You can also build two other libraries, called \fBlibpcre2-16\fP and \fBlibpcre2-32\fP, which process -strings that are contained in vectors of 16-bit and 32-bit code units, +strings that are contained in arrays of 16-bit and 32-bit code units, respectively. These can be interpreted either as single-unit characters or UTF-16/UTF-32 strings. To build these additional libraries, add one or both of the following to the \fBconfigure\fP command: @@ -197,18 +197,22 @@ to the \fBconfigure\fP command. There is a fourth option, specified by --enable-newline-is-anycrlf .sp which causes PCRE2 to recognize any of the three sequences CR, LF, or CRLF as -indicating a line ending. Finally, a fifth option, specified by +indicating a line ending. A fifth option, specified by .sp --enable-newline-is-any .sp causes PCRE2 to recognize any Unicode newline sequence. The Unicode newline sequences are the three just mentioned, plus the single characters VT (vertical tab, U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line -separator, U+2028), and PS (paragraph separator, U+2029). +separator, U+2028), and PS (paragraph separator, U+2029). The final option is +.sp + --enable-newline-is-nul +.sp +which causes NUL (binary zero) is set as the default line-ending character. .P Whatever default line ending convention is selected when PCRE2 is built can be overridden by applications that use the library. At build time it is -conventional to use the standard for your operating system. +recommended to use the standard for your operating system. . . .SH "WHAT \eR MATCHES" @@ -297,7 +301,8 @@ because the size of each backtracking "frame" depends on the number of capturing parentheses in a pattern, the amount of heap that is used before the limit is reached varies from pattern to pattern. This limit was more useful in versions before 10.30, where function recursion was used for backtracking. -However, as well as applying to \fBpcre2_match()\fP, this limit also controls +.P +As well as applying to \fBpcre2_match()\fP, the depth limit also controls the depth of recursive function calls in \fBpcre2_dfa_match()\fP. These are used for lookaround assertions, atomic groups, and recursion within patterns. The limit does not apply to JIT matching. @@ -577,6 +582,6 @@ Cambridge, England. .rs .sp .nf -Last updated: 17 June 2017 +Last updated: 18 July 2017 Copyright (c) 1997-2017 University of Cambridge. .fi diff --git a/doc/pcre2compat.3 b/doc/pcre2compat.3 index b3722df..8094ebd 100644 --- a/doc/pcre2compat.3 +++ b/doc/pcre2compat.3 @@ -71,7 +71,7 @@ documentation for details. .P 8. Subroutine calls (whether recursive or not) were treated as atomic groups up to PCRE2 release 10.23, but from release 10.30 this changed, and backtracking -into subroutine calls is now supported, as in Perl. +into subroutine calls is now supported, as in Perl. .P 9. If any of the backtracking control verbs are used in a subpattern that is called as a subroutine (whether or not recursively), their effect is confined diff --git a/doc/pcre2grep.1 b/doc/pcre2grep.1 index f6f79b4..8d5f338 100644 --- a/doc/pcre2grep.1 +++ b/doc/pcre2grep.1 @@ -446,25 +446,25 @@ memory. There are three options that set resource limits for matching. The \fB--match-limit\fP option provides a means of limiting computing resource usage when processing patterns that are not going to match, but which have a very large number of possibilities in their search trees. The classic example -is a pattern that uses nested unlimited repeats. Internally, PCRE2 has a -counter that is incremented each time around its main processing loop. If the +is a pattern that uses nested unlimited repeats. Internally, PCRE2 has a +counter that is incremented each time around its main processing loop. If the value set by \fB--match-limit\fP is reached, an error occurs. .sp The \fB--heap-limit\fP option specifies, as a number of kilobytes, the amount of heap memory that may be used for matching. Heap memory is needed only if matching the pattern requires a significant number of nested backtracking -points to be remembered. This parameter can be set to zero to forbid the use of +points to be remembered. This parameter can be set to zero to forbid the use of heap memory altogether. .sp The \fB--depth-limit\fP option limits the depth of nested backtracking points, -which indirectly limits the amount of memory that is used. The amount of memory +which indirectly limits the amount of memory that is used. The amount of memory needed for each backtracking point depends on the number of capturing parentheses in the pattern, so the amount of memory that is used before this limit acts varies from pattern to pattern. This limit is of use only if it is set smaller than \fB--match-limit\fP. .sp There are no short forms for these options. The default settings are specified -when the PCRE2 library is compiled, with the default defaults being very large +when the PCRE2 library is compiled, with the default defaults being very large and so effectively unlimited. .TP \fB--max-buffer-size=\fInumber\fP @@ -747,7 +747,7 @@ either a number or a quoted string (see the .\" HREF \fBpcre2callout\fP .\" -documentation for details). Numbered callouts are ignored by \fBpcre2grep\fP; +documentation for details). Numbered callouts are ignored by \fBpcre2grep\fP; only callouts with string arguments are useful. . . @@ -797,10 +797,10 @@ matcher backtracks in the normal way. If the callout string starts with a pipe (vertical bar) character, the rest of the string is written to the output, having been passed through the same escape processing as text from the --output option. This provides a simple echoing -facility that avoids calling an external program or script. No terminator is +facility that avoids calling an external program or script. No terminator is added to the string, so if you want a newline, you must include it explicitly. -Matching continues normally after the string is output. If you want to see only -the callout output but not any output from an actual match, you should end the +Matching continues normally after the string is output. If you want to see only +the callout output but not any output from an actual match, you should end the relevant pattern with (*FAIL). . . @@ -816,8 +816,8 @@ message and the line that caused the problem to the standard error stream. If there are more than 20 such errors, \fBpcre2grep\fP gives up. .P The \fB--match-limit\fP option of \fBpcre2grep\fP can be used to set the -overall resource limit. There are also other limits that affect the amount of -memory used during matching; see the discussion of \fB--heap-limit\fP and +overall resource limit. There are also other limits that affect the amount of +memory used during matching; see the discussion of \fB--heap-limit\fP and \fB--depth-limit\fP above. . . diff --git a/doc/pcre2perform.3 b/doc/pcre2perform.3 index 0781102..8b49a2a 100644 --- a/doc/pcre2perform.3 +++ b/doc/pcre2perform.3 @@ -12,7 +12,7 @@ of them. .rs .sp Patterns are compiled by PCRE2 into a reasonably efficient interpretive code, -so that most simple patterns do not use much memory for storing the compiled +so that most simple patterns do not use much memory for storing the compiled version. However, there is one case where the memory usage of a compiled pattern can be unexpectedly large. If a parenthesized subpattern has a quantifier with a minimum greater than 1 and/or a limited maximum, the whole @@ -76,7 +76,7 @@ memory can be limited; if the limit is set to zero, only the initial stack vector is used. Rewriting patterns to be time-efficient, as described below, may also reduce the memory requirements. .P -In contrast to \fBpcre2_match()\fP, \fBpcre2_dfa_match()\fP does use recursive +In contrast to \fBpcre2_match()\fP, \fBpcre2_dfa_match()\fP does use recursive function calls, but only for processing atomic groups, lookaround assertions, and recursion within the pattern. Too much nested recursion may cause stack issues. The "match depth" parameter can be used to limit the depth of function @@ -163,7 +163,7 @@ applied to a whole line of "a" characters, whereas the latter takes an appreciable time with strings longer than about 20 characters. .P In many cases, the solution to this kind of performance issue is to use an -atomic group or a possessive quantifier. This can often reduce memory +atomic group or a possessive quantifier. This can often reduce memory requirements as well. As another example, consider this pattern: .sp ([^<]|<(?!inet))+ @@ -184,7 +184,7 @@ are "swallowed" in one item inside the parentheses, and a possessive quantifier is used to stop any backtracking into the runs of non-"<" characters. This version also uses a lot less memory because entry to a new set of parentheses happens only when a "<" character that is not followed by "inet" is encountered -(and we assume this is relatively rare). +(and we assume this is relatively rare). .P This example shows that one way of optimizing performance when matching long subject strings is to write repeated parenthesized subpatterns to match more @@ -194,10 +194,10 @@ than one character whenever possible. .SS "SETTING RESOURCE LIMITS" .rs .sp -You can set limits on the amount of processing that takes place when matching, +You can set limits on the amount of processing that takes place when matching, and on the amount of heap memory that is used. The default values of the limits are very large, and unlikely ever to operate. They can be changed when PCRE2 is -built, and they can also be set when \fBpcre2_match()\fP or +built, and they can also be set when \fBpcre2_match()\fP or \fBpcre2_dfa_match()\fP is called. For details of these interfaces, see the .\" HREF \fBpcre2build\fP diff --git a/doc/pcre2syntax.3 b/doc/pcre2syntax.3 index 27f0aab..6eb0235 100644 --- a/doc/pcre2syntax.3 +++ b/doc/pcre2syntax.3 @@ -407,11 +407,11 @@ but some of them use Unicode properties if PCRE2_UCP is set. You can use (?i) caseless (?J) allow duplicate names (?m) multiline - (?n) no auto capture + (?n) no auto capture (?s) single line (dotall) (?U) default ungreedy (lazy) (?x) extended: ignore white space except in classes - (?xx) as (?x) but also ignore space and tab in classes + (?xx) as (?x) but also ignore space and tab in classes (?-...) unset option(s) .sp The following are recognized only at the very start of a pattern or after one diff --git a/perltest.sh b/perltest.sh index 806a888..1a7679a 100755 --- a/perltest.sh +++ b/perltest.sh @@ -50,7 +50,7 @@ fi # ucp sets Perl's /u modifier # utf invoke UTF-8 functionality # -# The data lines must not have any pcre2test modifiers. Unless +# The data lines must not have any pcre2test modifiers. Unless # "subject_litersl" is on the pattern, data lines are processed as # Perl double-quoted strings, so if they contain " $ or @ characters, these # have to be escaped. For this reason, all such characters in the @@ -141,20 +141,20 @@ for (;;) chomp($pattern); $pattern =~ s/\s+$//; - + # Split the pattern from the modifiers and adjust them as necessary. $pattern =~ /^\s*((.).*\2)(.*)$/s; $pat = $1; $mod = $3; - + # The private "aftertext" modifier means "print $' afterwards". $showrest = ($mod =~ s/aftertext,?//); - + # The "subject_literal" modifer disables escapes in subjects. - - $subject_literal = ($mod =~ s/subject_literal,?//); + + $subject_literal = ($mod =~ s/subject_literal,?//); # "allaftertext" is used by pcre2test to print remainders after captures @@ -238,7 +238,7 @@ for (;;) $x = $_; } else - { + { $x = eval "\"$_\""; # To get escapes processed } diff --git a/src/config.h.generic b/src/config.h.generic index 98a55d5..f794576 100644 --- a/src/config.h.generic +++ b/src/config.h.generic @@ -132,6 +132,12 @@ sure both macros are undefined; an emulation function will then be used. */ /* Define to 1 if you have the header file. */ /* #undef HAVE_ZLIB_H */ +/* This limits the amount of memory that pcre2_match() may use while matching + a pattern. The value is in kilobytes. */ +#ifndef HEAP_LIMIT +#define HEAP_LIMIT 20000000 +#endif + /* The value of LINK_SIZE determines the number of bytes used to store links as offsets within the compiled regex. The default is 2, which allows for compiled patterns up to 64K long. This covers the vast majority of cases. @@ -148,7 +154,7 @@ sure both macros are undefined; an emulation function will then be used. */ #endif /* The value of MATCH_LIMIT determines the default number of times the - internal match() function can record a backtrack position during a single + pcre2_match() function can record a backtrack position during a single matching attempt. There is a runtime interface for setting a different limit. The limit exists in order to catch runaway regular expressions that take for ever to determine that they do not match. The default is set very @@ -188,8 +194,8 @@ sure both macros are undefined; an emulation function will then be used. */ /* The value of NEWLINE_DEFAULT determines the default newline character sequence. PCRE2 client programs can override this by selecting other values - at run time. The valid values are 1 (CR), 2 (LF), 3 (CRLF), 4 (ANY), and 5 - (ANYCRLF). */ + at run time. The valid values are 1 (CR), 2 (LF), 3 (CRLF), 4 (ANY), 5 + (ANYCRLF), and 6 (NUL). */ #ifndef NEWLINE_DEFAULT #define NEWLINE_DEFAULT 2 #endif @@ -204,7 +210,7 @@ sure both macros are undefined; an emulation function will then be used. */ #define PACKAGE_NAME "PCRE2" /* Define to the full name and version of this package. */ -#define PACKAGE_STRING "PCRE2 10.30-DEV" +#define PACKAGE_STRING "PCRE2 10.30-RC1" /* Define to the one symbol short name of this package. */ #define PACKAGE_TARNAME "pcre2" @@ -213,7 +219,7 @@ sure both macros are undefined; an emulation function will then be used. */ #define PACKAGE_URL "" /* Define to the version of this package. */ -#define PACKAGE_VERSION "10.30-DEV" +#define PACKAGE_VERSION "10.30-RC1" /* The value of PARENS_NEST_LIMIT specifies the maximum depth of nested parentheses (of any kind) in a pattern. This limits the amount of system @@ -261,6 +267,11 @@ sure both macros are undefined; an emulation function will then be used. */ your system. */ /* #undef PTHREAD_CREATE_JOINABLE */ +/* Define to any non-zero number to enable support for SELinux compatible + executable memory allocator in JIT. Note that this will have no effect + unless SUPPORT_JIT is also defined. */ +/* #undef SLJIT_PROT_EXECUTABLE_ALLOCATOR */ + /* Define to 1 if you have the ANSI C header files. */ /* #undef STDC_HEADERS */ @@ -328,7 +339,7 @@ sure both macros are undefined; an emulation function will then be used. */ #endif /* Version number of package */ -#define VERSION "10.30-DEV" +#define VERSION "10.30-RC1" /* Define to 1 if on MINIX. */ /* #undef _MINIX */ diff --git a/src/pcre2.h b/src/pcre2.h index 36d357a..7cafb4c 100644 --- a/src/pcre2.h +++ b/src/pcre2.h @@ -43,8 +43,8 @@ POSSIBILITY OF SUCH DAMAGE. #define PCRE2_MAJOR 10 #define PCRE2_MINOR 30 -#define PCRE2_PRERELEASE -DEV -#define PCRE2_DATE 2017-03-05 +#define PCRE2_PRERELEASE -RC1 +#define PCRE2_DATE 2017-07-18 /* When an application links to a PCRE DLL in Windows, the symbols that are imported have to be identified as such. When building PCRE2, the appropriate diff --git a/src/pcre2.h.generic b/src/pcre2.h.generic index 21f610d..7cafb4c 100644 --- a/src/pcre2.h.generic +++ b/src/pcre2.h.generic @@ -43,8 +43,8 @@ POSSIBILITY OF SUCH DAMAGE. #define PCRE2_MAJOR 10 #define PCRE2_MINOR 30 -#define PCRE2_PRERELEASE -DEV -#define PCRE2_DATE 2017-03-05 +#define PCRE2_PRERELEASE -RC1 +#define PCRE2_DATE 2017-07-18 /* When an application links to a PCRE DLL in Windows, the symbols that are imported have to be identified as such. When building PCRE2, the appropriate @@ -138,6 +138,14 @@ D is inspected during pcre2_dfa_match() execution #define PCRE2_ALT_VERBNAMES 0x00400000u /* C */ #define PCRE2_USE_OFFSET_LIMIT 0x00800000u /* J M D */ #define PCRE2_EXTENDED_MORE 0x01000000u /* C */ +#define PCRE2_LITERAL 0x02000000u /* C */ + +/* An additional compile options word is available in the compile context. */ + +#define PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES 0x00000001u /* C */ +#define PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL 0x00000002u /* C */ +#define PCRE2_EXTRA_MATCH_WORD 0x00000004u /* C */ +#define PCRE2_EXTRA_MATCH_LINE 0x00000008u /* C */ /* These are for pcre2_jit_compile(). */ @@ -176,6 +184,16 @@ ignored for pcre2_jit_match(). */ #define PCRE2_NO_JIT 0x00002000u +/* Options for pcre2_pattern_convert(). */ + +#define PCRE2_CONVERT_UTF 0x00000001u +#define PCRE2_CONVERT_NO_UTF_CHECK 0x00000002u +#define PCRE2_CONVERT_POSIX_BASIC 0x00000004u +#define PCRE2_CONVERT_POSIX_EXTENDED 0x00000008u +#define PCRE2_CONVERT_GLOB 0x00000010u +#define PCRE2_CONVERT_GLOB_NO_WILD_SEPARATOR 0x00000030u +#define PCRE2_CONVERT_GLOB_NO_STARSTAR 0x00000050u + /* Newline and \R settings, for use in compile contexts. The newline values must be kept in step with values set in config.h and both sets must all be greater than zero. */ @@ -185,6 +203,7 @@ greater than zero. */ #define PCRE2_NEWLINE_CRLF 3 #define PCRE2_NEWLINE_ANY 4 #define PCRE2_NEWLINE_ANYCRLF 5 +#define PCRE2_NEWLINE_NUL 6 #define PCRE2_BSR_UNICODE 1 #define PCRE2_BSR_ANYCRLF 2 @@ -270,6 +289,8 @@ numbers must not be changed. */ #define PCRE2_ERROR_TOOMANYREPLACE (-61) #define PCRE2_ERROR_BADSERIALIZEDDATA (-62) #define PCRE2_ERROR_HEAPLIMIT (-63) +#define PCRE2_ERROR_CONVERT_SYNTAX (-64) + /* Request types for pcre2_pattern_info() */ @@ -351,6 +372,9 @@ typedef struct pcre2_real_compile_context pcre2_compile_context; \ struct pcre2_real_match_context; \ typedef struct pcre2_real_match_context pcre2_match_context; \ \ +struct pcre2_real_convert_context; \ +typedef struct pcre2_real_convert_context pcre2_convert_context; \ +\ struct pcre2_real_code; \ typedef struct pcre2_real_code pcre2_code; \ \ @@ -434,6 +458,8 @@ PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ pcre2_set_bsr(pcre2_compile_context *, uint32_t); \ PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ pcre2_set_character_tables(pcre2_compile_context *, const unsigned char *); \ +PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ + pcre2_set_compile_extra_options(pcre2_compile_context *, uint32_t); \ PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ pcre2_set_max_pattern_length(pcre2_compile_context *, PCRE2_SIZE); \ PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ @@ -466,6 +492,18 @@ PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ pcre2_set_recursion_memory_management(pcre2_match_context *, \ void *(*)(PCRE2_SIZE, void *), void (*)(void *, void *), void *); +#define PCRE2_CONVERT_CONTEXT_FUNCTIONS \ +PCRE2_EXP_DECL pcre2_convert_context PCRE2_CALL_CONVENTION \ + *pcre2_convert_context_copy(pcre2_convert_context *); \ +PCRE2_EXP_DECL pcre2_convert_context PCRE2_CALL_CONVENTION \ + *pcre2_convert_context_create(pcre2_general_context *); \ +PCRE2_EXP_DECL void PCRE2_CALL_CONVENTION \ + pcre2_convert_context_free(pcre2_convert_context *); \ +PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ + pcre2_set_glob_escape(pcre2_convert_context *, uint32_t); \ +PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ + pcre2_set_glob_separator(pcre2_convert_context *, uint32_t); + /* Functions concerned with compiling a pattern to PCRE internal code. */ @@ -572,6 +610,16 @@ PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ PCRE2_SIZE, PCRE2_UCHAR *, PCRE2_SIZE *); +/* Functions for converting pattern source strings. */ + +#define PCRE2_CONVERT_FUNCTIONS \ +PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \ + pcre2_pattern_convert(PCRE2_SPTR, PCRE2_SIZE, uint32_t, PCRE2_UCHAR **, \ + PCRE2_SIZE *, pcre2_convert_context *); \ +PCRE2_EXP_DECL void PCRE2_CALL_CONVENTION \ + pcre2_converted_pattern_free(PCRE2_UCHAR *); + + /* Functions for JIT processing */ #define PCRE2_JIT_FUNCTIONS \ @@ -623,6 +671,7 @@ pcre2_compile are called by application code. */ #define pcre2_real_code PCRE2_SUFFIX(pcre2_real_code_) #define pcre2_real_general_context PCRE2_SUFFIX(pcre2_real_general_context_) #define pcre2_real_compile_context PCRE2_SUFFIX(pcre2_real_compile_context_) +#define pcre2_real_convert_context PCRE2_SUFFIX(pcre2_real_convert_context_) #define pcre2_real_match_context PCRE2_SUFFIX(pcre2_real_match_context_) #define pcre2_real_jit_stack PCRE2_SUFFIX(pcre2_real_jit_stack_) #define pcre2_real_match_data PCRE2_SUFFIX(pcre2_real_match_data_) @@ -634,6 +683,7 @@ pcre2_compile are called by application code. */ #define pcre2_callout_enumerate_block PCRE2_SUFFIX(pcre2_callout_enumerate_block_) #define pcre2_general_context PCRE2_SUFFIX(pcre2_general_context_) #define pcre2_compile_context PCRE2_SUFFIX(pcre2_compile_context_) +#define pcre2_convert_context PCRE2_SUFFIX(pcre2_convert_context_) #define pcre2_match_context PCRE2_SUFFIX(pcre2_match_context_) #define pcre2_match_data PCRE2_SUFFIX(pcre2_match_data_) @@ -649,6 +699,10 @@ pcre2_compile are called by application code. */ #define pcre2_compile_context_create PCRE2_SUFFIX(pcre2_compile_context_create_) #define pcre2_compile_context_free PCRE2_SUFFIX(pcre2_compile_context_free_) #define pcre2_config PCRE2_SUFFIX(pcre2_config_) +#define pcre2_convert_context_copy PCRE2_SUFFIX(pcre2_convert_context_copy_) +#define pcre2_convert_context_create PCRE2_SUFFIX(pcre2_convert_context_create_) +#define pcre2_convert_context_free PCRE2_SUFFIX(pcre2_convert_context_free_) +#define pcre2_converted_pattern_free PCRE2_SUFFIX(pcre2_converted_pattern_free_) #define pcre2_dfa_match PCRE2_SUFFIX(pcre2_dfa_match_) #define pcre2_general_context_copy PCRE2_SUFFIX(pcre2_general_context_copy_) #define pcre2_general_context_create PCRE2_SUFFIX(pcre2_general_context_create_) @@ -672,6 +726,7 @@ pcre2_compile are called by application code. */ #define pcre2_match_data_create PCRE2_SUFFIX(pcre2_match_data_create_) #define pcre2_match_data_create_from_pattern PCRE2_SUFFIX(pcre2_match_data_create_from_pattern_) #define pcre2_match_data_free PCRE2_SUFFIX(pcre2_match_data_free_) +#define pcre2_pattern_convert PCRE2_SUFFIX(pcre2_pattern_convert_) #define pcre2_pattern_info PCRE2_SUFFIX(pcre2_pattern_info_) #define pcre2_serialize_decode PCRE2_SUFFIX(pcre2_serialize_decode_) #define pcre2_serialize_encode PCRE2_SUFFIX(pcre2_serialize_encode_) @@ -680,8 +735,11 @@ pcre2_compile are called by application code. */ #define pcre2_set_bsr PCRE2_SUFFIX(pcre2_set_bsr_) #define pcre2_set_callout PCRE2_SUFFIX(pcre2_set_callout_) #define pcre2_set_character_tables PCRE2_SUFFIX(pcre2_set_character_tables_) +#define pcre2_set_compile_extra_options PCRE2_SUFFIX(pcre2_set_compile_extra_options_) #define pcre2_set_compile_recursion_guard PCRE2_SUFFIX(pcre2_set_compile_recursion_guard_) #define pcre2_set_depth_limit PCRE2_SUFFIX(pcre2_set_depth_limit_) +#define pcre2_set_glob_escape PCRE2_SUFFIX(pcre2_set_glob_escape_) +#define pcre2_set_glob_separator PCRE2_SUFFIX(pcre2_set_glob_separator_) #define pcre2_set_heap_limit PCRE2_SUFFIX(pcre2_set_heap_limit_) #define pcre2_set_match_limit PCRE2_SUFFIX(pcre2_set_match_limit_) #define pcre2_set_max_pattern_length PCRE2_SUFFIX(pcre2_set_max_pattern_length_) @@ -716,6 +774,8 @@ PCRE2_STRUCTURE_LIST \ PCRE2_GENERAL_INFO_FUNCTIONS \ PCRE2_GENERAL_CONTEXT_FUNCTIONS \ PCRE2_COMPILE_CONTEXT_FUNCTIONS \ +PCRE2_CONVERT_CONTEXT_FUNCTIONS \ +PCRE2_CONVERT_FUNCTIONS \ PCRE2_MATCH_CONTEXT_FUNCTIONS \ PCRE2_COMPILE_FUNCTIONS \ PCRE2_PATTERN_INFO_FUNCTIONS \ @@ -745,6 +805,7 @@ PCRE2_TYPES_STRUCTURES_AND_FUNCTIONS #undef PCRE2_GENERAL_INFO_FUNCTIONS #undef PCRE2_GENERAL_CONTEXT_FUNCTIONS #undef PCRE2_COMPILE_CONTEXT_FUNCTIONS +#undef PCRE2_CONVERT_CONTEXT_FUNCTIONS #undef PCRE2_MATCH_CONTEXT_FUNCTIONS #undef PCRE2_COMPILE_FUNCTIONS #undef PCRE2_PATTERN_INFO_FUNCTIONS diff --git a/src/pcre2_config.c b/src/pcre2_config.c index a60ff69..d009c0a 100644 --- a/src/pcre2_config.c +++ b/src/pcre2_config.c @@ -84,7 +84,7 @@ if (where == NULL) /* Requests a length */ return PCRE2_ERROR_BADOPTION; case PCRE2_CONFIG_BSR: - case PCRE2_CONFIG_HEAPLIMIT: + case PCRE2_CONFIG_HEAPLIMIT: case PCRE2_CONFIG_JIT: case PCRE2_CONFIG_LINKSIZE: case PCRE2_CONFIG_MATCHLIMIT: @@ -151,7 +151,7 @@ switch (what) case PCRE2_CONFIG_DEPTHLIMIT: *((uint32_t *)where) = MATCH_LIMIT_DEPTH; break; - + case PCRE2_CONFIG_NEWLINE: *((uint32_t *)where) = NEWLINE_DEFAULT; break; @@ -160,7 +160,7 @@ switch (what) *((uint32_t *)where) = PARENS_NEST_LIMIT; break; - /* This is now obsolete. The stack is no longer used via recursion for + /* This is now obsolete. The stack is no longer used via recursion for handling backtracking in pcre2_match(). */ case PCRE2_CONFIG_STACKRECURSE: diff --git a/src/pcre2_context.c b/src/pcre2_context.c index 700dd0f..9e557da 100644 --- a/src/pcre2_context.c +++ b/src/pcre2_context.c @@ -198,7 +198,7 @@ const pcre2_convert_context PRIV(default_convert_context) = { CHAR_BACKSLASH, /* Default path separator */ CHAR_GRAVE_ACCENT /* Default escape character */ #else /* Not Windows */ - CHAR_SLASH, /* Default path separator */ + CHAR_SLASH, /* Default path separator */ CHAR_BACKSLASH /* Default escape character */ #endif }; @@ -359,7 +359,7 @@ switch(newline) case PCRE2_NEWLINE_CRLF: case PCRE2_NEWLINE_ANY: case PCRE2_NEWLINE_ANYCRLF: - case PCRE2_NEWLINE_NUL: + case PCRE2_NEWLINE_NUL: ccontext->newline_convention = newline; return 0; diff --git a/src/pcre2_error.c b/src/pcre2_error.c index daeb2a2..d98cae9 100644 --- a/src/pcre2_error.c +++ b/src/pcre2_error.c @@ -177,7 +177,7 @@ static const unsigned char compile_error_texts[] = /* 90 */ "internal error: bad code value in parsed_skip()\0" "PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES is not allowed in UTF-16 mode\0" - "invalid option bits with PCRE2_LITERAL\0" + "invalid option bits with PCRE2_LITERAL\0" ; /* Match-time and UTF error texts are in the same format. */ diff --git a/src/pcre2_internal.h b/src/pcre2_internal.h index bc52b9a..9ccce25 100644 --- a/src/pcre2_internal.h +++ b/src/pcre2_internal.h @@ -240,7 +240,7 @@ not rely on this. */ #define COMPILE_ERROR_BASE 100 -/* The initial frames vector for remembering backtracking points in +/* The initial frames vector for remembering backtracking points in pcre2_match() is allocated on the system stack, of this size (bytes). The size must be a multiple of sizeof(PCRE2_SPTR) in all environments, so making it a multiple of 8 is best. Typical frame sizes are a few hundred bytes (it depends @@ -557,7 +557,7 @@ enum { PCRE2_MATCHEDBY_INTERPRETER, /* pcre2_match() */ #define MAGIC_NUMBER 0x50435245UL /* 'PCRE' */ /* The maximum remaining length of subject we are prepared to search for a -req_unit match. In 8-bit mode, memchr() is used and is much faster than the +req_unit match. In 8-bit mode, memchr() is used and is much faster than the search loop that has to be used in 16-bit and 32-bit modes. */ #if PCRE2_CODE_UNIT_WIDTH == 8 diff --git a/src/pcre2_jit_compile.c b/src/pcre2_jit_compile.c index 31817ae..c7bf0b2 100644 --- a/src/pcre2_jit_compile.c +++ b/src/pcre2_jit_compile.c @@ -3829,7 +3829,7 @@ while (TRUE) { case OP_CHARI: caseless = TRUE; - /* Fall through */ + /* Fall through */ case OP_CHAR: last = FALSE; cc++; @@ -3861,7 +3861,7 @@ while (TRUE) case OP_MINPLUSI: case OP_POSPLUSI: caseless = TRUE; - /* Fall through */ + /* Fall through */ case OP_PLUS: case OP_MINPLUS: case OP_POSPLUS: @@ -3870,7 +3870,7 @@ while (TRUE) case OP_EXACTI: caseless = TRUE; - /* Fall through */ + /* Fall through */ case OP_EXACT: repeat = GET2(cc, 1); last = FALSE; @@ -3881,7 +3881,7 @@ while (TRUE) case OP_MINQUERYI: case OP_POSQUERYI: caseless = TRUE; - /* Fall through */ + /* Fall through */ case OP_QUERY: case OP_MINQUERY: case OP_POSQUERY: @@ -4351,7 +4351,7 @@ struct sljit_jump *quit; struct sljit_jump *partial_quit[2]; sljit_u8 instruction[8]; sljit_s32 tmp1_ind = sljit_get_register_index(TMP1); -sljit_s32 tmp2_ind = sljit_get_register_index(TMP2); +// sljit_s32 tmp2_ind = sljit_get_register_index(TMP2); sljit_s32 str_ptr_ind = sljit_get_register_index(STR_PTR); sljit_s32 data_ind = 0; sljit_s32 tmp_ind = 1; @@ -4376,7 +4376,9 @@ if (common->mode == PCRE2_JIT_COMPLETE) OP1(SLJIT_MOV, TMP1, 0, SLJIT_IMM, character_to_int32(char1 | bit)); -SLJIT_ASSERT(tmp1_ind < 8 && tmp2_ind == 1); +// SLJIT_ASSERT(tmp1_ind < 8 && tmp2_ind == 1); + +SLJIT_ASSERT(tmp1_ind < 8); /* MOVD xmm, r/m32 */ instruction[0] = 0x66; diff --git a/src/pcre2_pattern_info.c b/src/pcre2_pattern_info.c index de21310..540707b 100644 --- a/src/pcre2_pattern_info.c +++ b/src/pcre2_pattern_info.c @@ -80,7 +80,7 @@ if (where == NULL) /* Requests field length */ case PCRE2_INFO_FIRSTCODEUNIT: case PCRE2_INFO_HASBACKSLASHC: case PCRE2_INFO_HASCRORLF: - case PCRE2_INFO_HEAPLIMIT: + case PCRE2_INFO_HEAPLIMIT: case PCRE2_INFO_JCHANGED: case PCRE2_INFO_LASTCODETYPE: case PCRE2_INFO_LASTCODEUNIT: diff --git a/src/pcre2_tables.c b/src/pcre2_tables.c index 4398895..9f8dc29 100644 --- a/src/pcre2_tables.c +++ b/src/pcre2_tables.c @@ -167,15 +167,15 @@ are implementing). 6. Do not break after Prepend characters. 7. Do not break within emoji modifier sequences (E_Base or E_Base_GAZ followed - by E_Modifier). Extend characters are allowed before the modifier; this + by E_Modifier). Extend characters are allowed before the modifier; this cannot be represented in this table, the code has to deal with it. - + 8. Do not break within emoji zwj sequences (ZWJ followed by Glue_After_Zwj or E_Base_GAZ). - -9. Do not break within emoji flag sequences. That is, do not break between - regional indicator (RI) symbols if there are an odd number of RI characters - before the break point. This table encodes "join RI characters"; the code + +9. Do not break within emoji flag sequences. That is, do not break between + regional indicator (RI) symbols if there are an odd number of RI characters + before the break point. This table encodes "join RI characters"; the code has to deal with checking for previous adjoining RIs. 10. Otherwise, break everywhere. diff --git a/src/pcre2_ucp.h b/src/pcre2_ucp.h index d79d206..defba4c 100644 --- a/src/pcre2_ucp.h +++ b/src/pcre2_ucp.h @@ -264,7 +264,7 @@ enum { ucp_Multani, ucp_Old_Hungarian, ucp_SignWriting, - /* New for Unicode 10.0.0 (no update since 8.0.0) */ + /* New for Unicode 10.0.0 (no update since 8.0.0) */ ucp_Adlam, ucp_Bhaiksuki, ucp_Marchen, @@ -272,7 +272,7 @@ enum { ucp_Osage, ucp_Tangut, ucp_Masaram_Gondi, - ucp_Nushu, + ucp_Nushu, ucp_Soyombo, ucp_Zanabazar_Square }; diff --git a/src/pcre2posix.c b/src/pcre2posix.c index ebe187d..026943e 100644 --- a/src/pcre2posix.c +++ b/src/pcre2posix.c @@ -142,7 +142,7 @@ static const int eint2[] = { 32, REG_INVARG, /* this version of PCRE2 does not have Unicode support */ 37, REG_EESCAPE, /* PCRE2 does not support \L, \l, \N{name}, \U, or \u */ 56, REG_INVARG, /* internal error: unknown newline setting */ - 92, REG_INVARG, /* invalid option bits with PCRE2_LITERAL */ + 92, REG_INVARG, /* invalid option bits with PCRE2_LITERAL */ }; /* Table of texts corresponding to POSIX error codes */ @@ -239,7 +239,7 @@ int re_nsub = 0; patlen = ((cflags & REG_PEND) != 0)? (PCRE2_SIZE)(preg->re_endp - pattern) : PCRE2_ZERO_TERMINATED; - + if ((cflags & REG_ICASE) != 0) options |= PCRE2_CASELESS; if ((cflags & REG_NEWLINE) != 0) options |= PCRE2_MULTILINE; if ((cflags & REG_DOTALL) != 0) options |= PCRE2_DOTALL; @@ -249,7 +249,7 @@ if ((cflags & REG_UCP) != 0) options |= PCRE2_UCP; if ((cflags & REG_UNGREEDY) != 0) options |= PCRE2_UNGREEDY; preg->re_cflags = cflags; -preg->re_pcre2_code = pcre2_compile((PCRE2_SPTR)pattern, patlen, options, +preg->re_pcre2_code = pcre2_compile((PCRE2_SPTR)pattern, patlen, options, &errorcode, &erroffset, NULL); preg->re_erroffset = erroffset; @@ -262,7 +262,7 @@ if (preg->re_pcre2_code == NULL) if (errorcode < COMPILE_ERROR_BASE) return REG_BADPAT; errorcode -= COMPILE_ERROR_BASE; - + if (errorcode < (int)(sizeof(eint1)/sizeof(const int))) return eint1[errorcode]; for (i = 0; i < sizeof(eint2)/sizeof(const int); i += 2) diff --git a/src/pcre2posix.h b/src/pcre2posix.h index 651eedd..4ae1d3c 100644 --- a/src/pcre2posix.h +++ b/src/pcre2posix.h @@ -93,13 +93,13 @@ enum { }; -/* The structure representing a compiled regular expression. It is also used +/* The structure representing a compiled regular expression. It is also used for passing the pattern end pointer when REG_PEND is set. */ typedef struct { void *re_pcre2_code; void *re_match_data; - const char *re_endp; + const char *re_endp; size_t re_nsub; size_t re_erroffset; int re_cflags; diff --git a/src/pcre2test.c b/src/pcre2test.c index 3ce261c..5d1cbdc 100644 --- a/src/pcre2test.c +++ b/src/pcre2test.c @@ -4073,7 +4073,8 @@ else fprintf(outfile, "%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s% * Show compile extra options * *************************************************/ -/* Called for unsupported POSIX options. +/* Called only for unsupported POSIX options at present, and therefore needed +only when the 8-bit library is being compiled. Arguments: options an options word @@ -4083,17 +4084,21 @@ Arguments: Returns: nothing */ +#ifdef SUPPORT_PCRE2_8 static void show_compile_extra_options(uint32_t options, const char *before, const char *after) { if (options == 0) fprintf(outfile, "%s %s", before, after); -else fprintf(outfile, "%s%s%s%s", +else fprintf(outfile, "%s%s%s%s%s%s", before, ((options & PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES) != 0)? " allow_surrogate_escapes" : "", ((options & PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL) != 0)? " bad_escape_is_literal" : "", + ((options & PCRE2_EXTRA_MATCH_WORD) != 0)? " match_word" : "", + ((options & PCRE2_EXTRA_MATCH_LINE) != 0)? " match_line" : "", after); } +#endif diff --git a/src/sljit/sljitLir.c b/src/sljit/sljitLir.c index 66cdda3..d64b62e 100644 --- a/src/sljit/sljitLir.c +++ b/src/sljit/sljitLir.c @@ -124,10 +124,10 @@ /* SLJIT_REWRITABLE_JUMP is 0x1000. */ #if (defined SLJIT_CONFIG_X86 && SLJIT_CONFIG_X86) -# define PATCH_MB 0x4 -# define PATCH_MW 0x8 +# define PATCH_MB 0x4 +# define PATCH_MW 0x8 #if (defined SLJIT_CONFIG_X86_64 && SLJIT_CONFIG_X86_64) -# define PATCH_MD 0x10 +# define PATCH_MD 0x10 #endif #endif @@ -1555,6 +1555,7 @@ static SLJIT_INLINE CHECK_RETURN_TYPE check_sljit_emit_cmov(struct sljit_compile sljit_s32 dst_reg, sljit_s32 src, sljit_sw srcw) { +(void)srcw; /* To stop compiler warning */ #if (defined SLJIT_ARGUMENT_CHECKS && SLJIT_ARGUMENT_CHECKS) CHECK_ARGUMENT(!(type & ~(0xff | SLJIT_I32_OP))); CHECK_ARGUMENT((type & 0xff) >= SLJIT_EQUAL && (type & 0xff) <= SLJIT_ORDERED_F64); diff --git a/testdata/testinput1 b/testdata/testinput1 index 8ef81e1..8c76e21 100644 --- a/testdata/testinput1 +++ b/testdata/testinput1 @@ -95,17 +95,6 @@ aaac abbbbbbbbbbbac -/^(b+|a){1,2}?bc/ - bbc - -/^(b*|ba){1,2}?bc/ - babc - bbabc - bababc -\= Expect no match - bababbc - babababc - /^(ba|b*){1,2}?bc/ babc bbabc diff --git a/testdata/testinput2 b/testdata/testinput2 index a22a6ca..f3b7674 100644 --- a/testdata/testinput2 +++ b/testdata/testinput2 @@ -1,12 +1,12 @@ # This set of tests is not Perl-compatible. It checks on special features # of PCRE2's API, error diagnostics, and the compiled code of some patterns. -# It also checks the non-Perl syntax that PCRE2 supports (Python, .NET, -# Oniguruma). There are also some tests where PCRE2 and Perl differ, -# either because PCRE2 can't be compatible, or there is a possible Perl +# It also checks the non-Perl syntax that PCRE2 supports (Python, .NET, +# Oniguruma). There are also some tests where PCRE2 and Perl differ, +# either because PCRE2 can't be compatible, or there is a possible Perl # bug. # NOTE: This is a non-UTF set of tests. When UTF support is needed, use -# test 5. +# test 5. #forbid_utf #newline_default lf any anycrlf @@ -330,7 +330,7 @@ \= Expect no match aaaa aaaaaa - + # Perl does not fail these two for the final subjects. Neither did PCRE until # release 8.01. The problem is in backtracking into a subpattern that contains # a recursive reference to itself. PCRE has now made these into atomic patterns. @@ -1219,7 +1219,7 @@ aaabbb aaabbb\=callout_data=0 aaabbb\=callout_data=1 -\= Expect no match +\= Expect no match aaabbb\=callout_data=-1 /ab(?Pcd)ef(?Pgh)/I @@ -1291,19 +1291,19 @@ /a+b/IB,auto_callout ab aaaab -\= Expect no match +\= Expect no match aaaacb /(abc|def)x/IB,auto_callout abcx defx -\= Expect no match +\= Expect no match abcdefzx /(abc|def)x/IB,auto_callout abcx defx -\= Expect no match +\= Expect no match abcdefzx /(ab|cd){3,4}/I,auto_callout @@ -1312,11 +1312,11 @@ abcdcdcdcdcd /([ab]{,4}c|xy)/IB,auto_callout -\= Expect no match +\= Expect no match Note: that { does NOT introduce a quantifier /([ab]{,4}c|xy)/IB,auto_callout -\= Expect no match +\= Expect no match Note: that { does NOT introduce a quantifier /([ab]{1,4}c|xy){4,5}?123/IB,auto_callout @@ -1465,7 +1465,7 @@ a1b\=copy=A a2b\=copy=A a1b\=copy=Z,copy=A - + /(?|(?)(?)(?)|(?)(?)(?))/I,dupnames /^(?Pa)(?Pb)/I,dupnames @@ -1553,7 +1553,7 @@ \= Expect no match xyz\nabclf xyz\rabclf - + /^abc/Im,newline=cr xyz\rabc \= Expect no match @@ -2029,11 +2029,11 @@ afoo \r\nfoo \nfoo - + /^$/gm,newline=any abc\r\rxyz - abc\n\rxyz -\= Expect no match + abc\n\rxyz +\= Expect no match abc\r\nxyz /(?m)^$/g,newline=any,aftertext @@ -2041,7 +2041,7 @@ /(?m)^$|^\r\n/g,newline=any,aftertext abc\r\n\r\n - + /(?m)$/g,newline=any,aftertext abc\r\n\r\n @@ -2050,7 +2050,7 @@ /^X/m XABC -\= Expect no match +\= Expect no match XABC\=notbol /(ab|c)(?-1)/B @@ -2059,8 +2059,8 @@ /xy(?+1)(abc)/B xyabcabc \= Expect no match - xyabc - + xyabc + /x(?-0)y/ /x(?-1)y/ @@ -2073,13 +2073,13 @@ abcX Y \= Expect no match - abcY - + abcY + /^((?(+1)X|Y)(abc))+/B YabcXabc YabcXabcXabc \= Expect no match - XabcXabc + XabcXabc /(?(-1)a)/B @@ -2092,30 +2092,30 @@ /(?tom|bon)-\k{A}/ tom-tom - bon-bon + bon-bon \= Expect no match - tom-bon + tom-bon /\g{A/ /(?|(abc)|(xyz))/B >abc< - >xyz< + >xyz< /(x)(?|(abc)|(xyz))(x)/B xabcx - xxyzx + xxyzx /(x)(?|(abc)(pqr)|(xyz))(x)/B xabcpqrx - xxyzx + xxyzx /\H++X/B \= Expect no match XXXX - + /\H+\hY/B - XXXX Y + XXXX Y /\H+ Y/B @@ -2169,7 +2169,7 @@ /^a+(*FAIL)/auto_callout \= Expect no match aaaaaa - + /a+b?c+(*FAIL)/auto_callout \= Expect no match aaabccc @@ -2181,7 +2181,7 @@ /a+b?(*COMMIT)c+(*FAIL)/auto_callout \= Expect no match aaabccc - + /a+b?(*SKIP)c+(*FAIL)/auto_callout \= Expect no match aaabcccaaabccc @@ -2189,7 +2189,7 @@ /a+b?(*THEN)c+(*FAIL)/auto_callout \= Expect no match aaabccc - + /a(*MARK)b/ /(?i:A{1,}\6666666666)/ @@ -2203,52 +2203,52 @@ /.+A/newline=crlf \= Expect no match \r\nA - + /\nA/newline=crlf - \r\nA + \r\nA /[\r\n]A/newline=crlf - \r\nA + \r\nA /(\r|\n)A/newline=crlf - \r\nA + \r\nA /a(*CR)b/ /(*CR)a.b/ a\nb \= Expect no match - a\rb + a\rb /(*CR)a.b/newline=lf a\nb \= Expect no match - a\rb + a\rb /(*LF)a.b/newline=CRLF a\rb \= Expect no match - a\nb + a\nb /(*CRLF)a.b/ a\rb - a\nb + a\nb \= Expect no match - a\r\nb + a\r\nb /(*ANYCRLF)a.b/newline=CR \= Expect no match a\rb - a\nb - a\r\nb + a\nb + a\r\nb /(*ANY)a.b/newline=cr \= Expect no match a\rb - a\nb - a\r\nb - a\x85b - + a\nb + a\r\nb + a\x85b + /(*ANY).*/g abc\r\ndef @@ -2257,24 +2257,24 @@ /(*CRLF).*/g abc\r\ndef - + /(*NUL)^.*/ a\nb\x00ccc - + /(*NUL)^.*/s a\nb\x00ccc - + /^x/m,newline=NUL ab\x00xy - + /'#comment' 0d 0a 00 '^x\' 0a 'y'/x,newline=nul,hex - x\nyz + x\nyz /(*NUL)^X\NY/ X\nY X\rY \= Expect no match - X\x00Y + X\x00Y /a\Rb/I,bsr=anycrlf a\rb @@ -2282,57 +2282,57 @@ a\r\nb \= Expect no match a\x85b - a\x0bb + a\x0bb /a\Rb/I,bsr=unicode a\rb a\nb a\r\nb a\x85b - a\x0bb - + a\x0bb + /a\R?b/I,bsr=anycrlf a\rb a\nb a\r\nb \= Expect no match a\x85b - a\x0bb + a\x0bb /a\R?b/I,bsr=unicode a\rb a\nb a\r\nb a\x85b - a\x0bb - + a\x0bb + /a\R{2,4}b/I,bsr=anycrlf a\r\n\nb a\n\r\rb a\r\n\r\n\r\n\r\nb \= Expect no match a\x85\x85b - a\x0b\x0bb + a\x0b\x0bb /a\R{2,4}b/I,bsr=unicode a\r\rb a\n\n\nb a\r\n\n\r\rb a\x85\x85b - a\x0b\x0bb -\= Expect no match - a\r\r\r\r\rb - + a\x0b\x0bb +\= Expect no match + a\r\r\r\r\rb + /(*BSR_ANYCRLF)a\Rb/I a\nb - a\rb + a\rb /(*BSR_UNICODE)a\Rb/I a\x85b /(*BSR_ANYCRLF)(*CRLF)a\Rb/I a\nb - a\rb + a\rb /(*CRLF)(*BSR_UNICODE)a\Rb/I a\x85b @@ -2395,9 +2395,9 @@ /^(?+1)(?x|y){0}z/ xzxx - yzyy + yzyy \= Expect no match - xxz + xxz /(\3)(\1)(a)/ \= Expect no match @@ -2405,12 +2405,12 @@ /(\3)(\1)(a)/alt_bsux,allow_empty_class,match_unset_backref,dupnames cat - + /TA]/ - The ACTA] comes + The ACTA] comes /TA]/alt_bsux,allow_empty_class,match_unset_backref,dupnames - The ACTA] comes + The ACTA] comes /(?2)[]a()b](abc)/ abcbabc @@ -2420,12 +2420,12 @@ /(?1)[]a()b](abc)/ abcbabc -\= Expect no match +\= Expect no match abcXabc /(?1)[^]a()b](abc)/ abcXabc -\= Expect no match +\= Expect no match abcbabc /(?2)[]a()b](abc)(xyz)/ @@ -2447,23 +2447,23 @@ /a[]+b/alt_bsux,allow_empty_class,match_unset_backref,dupnames \= Expect no match - ab + ab /a[]*+b/alt_bsux,allow_empty_class,match_unset_backref,dupnames \= Expect no match - ab + ab /a[^]b/alt_bsux,allow_empty_class,match_unset_backref,dupnames aXb - a\nb + a\nb \= Expect no match - ab - + ab + /a[^]+b/alt_bsux,allow_empty_class,match_unset_backref,dupnames aXb - a\nX\nXb + a\nX\nXb \= Expect no match - ab + ab /a(?!)b/B @@ -2476,32 +2476,32 @@ /(?(?=.*b)b|^)/I,auto_callout adc - abc - + abc + /(?(?=b).*b|^d)/I /(?(?=.*b).*b|^d)/I /xyz/auto_callout - xyz - abcxyz -\= Expect no match + xyz + abcxyz +\= Expect no match abc - abcxypqr - + abcxypqr + /xyz/auto_callout,no_start_optimize - abcxyz -\= Expect no match + abcxyz +\= Expect no match abc - abcxypqr - + abcxypqr + /(*NO_START_OPT)xyz/auto_callout abcxyz - + /(*NO_AUTO_POSSESS)a+b/B /xyz/auto_callout,no_start_optimize - abcxyz + abcxyz /^"((?(?=[a])[^"])|b)*"$/auto_callout "ab" @@ -2514,133 +2514,133 @@ /^X(?&N)(a)(?|(b)|(q))(c)(d)(?Y)/ XYabcdY - + /Xa{2,4}b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /Xa{2,4}?b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /Xa{2,4}+b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /X\d{2,4}b/ X\=ps X3\=ps X33\=ps X333\=ps X3333\=ps - + /X\d{2,4}?b/ X\=ps X3\=ps X33\=ps X333\=ps X3333\=ps - + /X\d{2,4}+b/ X\=ps X3\=ps X33\=ps X333\=ps X3333\=ps - + /X\D{2,4}b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /X\D{2,4}?b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /X\D{2,4}+b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /X[abc]{2,4}b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /X[abc]{2,4}?b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /X[abc]{2,4}+b/ X\=ps Xa\=ps Xaa\=ps Xaaa\=ps Xaaaa\=ps - + /X[^a]{2,4}b/ X\=ps Xz\=ps Xzz\=ps Xzzz\=ps Xzzzz\=ps - + /X[^a]{2,4}?b/ X\=ps Xz\=ps Xzz\=ps Xzzz\=ps Xzzzz\=ps - + /X[^a]{2,4}+b/ X\=ps Xz\=ps Xzz\=ps Xzzz\=ps Xzzzz\=ps - + /(Y)X\1{2,4}b/ YX\=ps YXY\=ps YXYY\=ps YXYYY\=ps YXYYYY\=ps - + /(Y)X\1{2,4}?b/ YX\=ps YXY\=ps YXYY\=ps YXYYY\=ps YXYYYY\=ps - + /(Y)X\1{2,4}+b/ YX\=ps YXY\=ps YXYY\=ps YXYYY\=ps YXYYYY\=ps - + /\++\KZ|\d+X|9+Y/startchar ++++123999\=ps ++++123999Y\=ps @@ -2650,7 +2650,7 @@ \= Expect no match Z\=ps ZA\=ps - + /Z(?!)/ \= Expect no match Z\=ps @@ -2659,7 +2659,7 @@ /dog(sbody)?/ dogs\=ps dogs\=ph - + /dog(sbody)??/ dogs\=ps dogs\=ph @@ -2667,7 +2667,7 @@ /dog|dogsbody/ dogs\=ps dogs\=ph - + /dogsbody|dog/ dogs\=ps dogs\=ph @@ -2679,14 +2679,14 @@ /abc/ abc\=ps abc\=ph - + /abc\K123/startchar xyzabc123pqr xyzabc12\=ps xyzabc12\=ph - + /(?<=abc)123/ - xyzabc123pqr + xyzabc123pqr xyzabc12\=ps xyzabc12\=ph @@ -2708,11 +2708,11 @@ \= Expect no match abcdef\=notempty xyzabcdef\=notempty - + /^(?:(?=abc)|abc\K)/aftertext,startchar abcdef abcdef\=notempty_atstart -\= Expect no match +\= Expect no match abcdef\=notempty /a?b?/aftertext @@ -2721,79 +2721,79 @@ xyzabc\=notempty xyzabc\=notempty_atstart xyz\=notempty_atstart -\= Expect no match +\= Expect no match xyz\=notempty /^a?b?/aftertext xyz xyzabc -\= Expect no match +\= Expect no match xyzabc\=notempty xyzabc\=notempty_atstart xyz\=notempty_atstart xyz\=notempty - + /^(?a|b\gc)/ aaaa bacxxx - bbaccxxx + bbaccxxx bbbacccxx /^(?a|b\g'name'c)/ aaaa bacxxx - bbaccxxx + bbaccxxx bbbacccxx /^(a|b\g<1>c)/ aaaa bacxxx - bbaccxxx + bbaccxxx bbbacccxx /^(a|b\g'1'c)/ aaaa bacxxx - bbaccxxx + bbaccxxx bbbacccxx /^(a|b\g'-1'c)/ aaaa bacxxx - bbaccxxx + bbaccxxx bbbacccxx /(^(a|b\g<-1>c))/ aaaa bacxxx - bbaccxxx + bbaccxxx bbbacccxx /(?-i:\g)(?i:(?a))/ XaaX - XAAX + XAAX /(?i:\g)(?-i:(?a))/ XaaX -\= Expect no match - XAAX +\= Expect no match + XAAX /(?-i:\g<+1>)(?i:(a))/ XaaX - XAAX + XAAX /(?=(?(?#simplesyntax)\$(?[a-zA-Z_\x{7f}-\x{ff}][a-zA-Z0-9_\x{7f}-\x{ff}]*)(?:\[(?[a-zA-Z0-9_\x{7f}-\x{ff}]+|\$\g)\]|->\g(\(.*?\))?)?|(?#simple syntax withbraces)\$\{(?:\g(?\[(?:\g|'(?:\\.|[^'\\])*'|"(?:\g|\\.|[^"\\])*")\])?|\g|\$\{\g\})\}|(?#complexsyntax)\{(?\$(?\g(\g*|\(.*?\))?)(?:->\g)*|\$\g|\$\{\g\})\}))\{/ /(?a|b|c)\g*/ abc - accccbbb + accccbbb /^X(?7)(a)(?|(b)|(q)(r)(s))(c)(d)(Y)/ XYabcdY /(?<=b(?1)|zzz)(a)/ xbaax - xzzzax + xzzzax /(a)(?<=b\1)/ @@ -2844,7 +2844,7 @@ (?: [0-9a-f]{1,4} | # 1-4 hex digits or (?(1)0 | () ) ) # if null previously matched, fail; else null : # followed by colon - ){1,7} # end item; 1-7 of them required + ){1,7} # end item; 1-7 of them required [0-9a-f]{1,4} $ # final hex number at end of string (?(1)|.) # check that there was an empty component /Iix @@ -2856,25 +2856,25 @@ /(?|(?A)|(?B))/ /(?:a(? (?')|(?")) | - b(? (?')|(?")) ) + b(? (?')|(?")) ) (?('quote')[a-z]+|[0-9]+)/Ix,dupnames a"aaaaa - b"aaaaa -\= Expect no match + b"aaaaa +\= Expect no match b"11111 - a"11111 - + a"11111 + /^(?|(a)(b)(c)(?d)|(?e)) (?('D')X|Y)/IBx,dupnames abcdX eX \= Expect no match abcdY - ey - + ey + /(?a) (b)(c) (?d (?(R&A)$ | (?4)) )/IBx,dupnames abcdd \= Expect no match - abcdde + abcdde /abcd*/ xxxxabcd\=ps @@ -2910,16 +2910,16 @@ /i(?(DEFINE)(?a))/I i - + /()i(?(1)a)/I ia /(?i)a(?-i)b|c/B XabX XAbX - CcC + CcC \= Expect no match - XABX + XABX /(?i)a(?s)b|c/B @@ -2927,20 +2927,20 @@ /^(ab(c\1)d|x){2}$/B xabcxd - + /^(?&t)*+(?(DEFINE)(?.))$/B /^(?&t)*(?(DEFINE)(?.))$/B # This one is here because Perl gives the match as "b" rather than "ab". I # believe this to be a Perl bug. - + /(?>a\Kb)z|(ab)/ - ab\=startchar + ab\=startchar /(?P(?P0|)|(?P>L2)(?P>L1))/ abcd - 0abc + 0abc /abc(*MARK:)pqr/ @@ -2948,7 +2948,7 @@ /abc(*FAIL:123)xyz/ -# This should, and does, fail. In Perl, it does not, which I think is a +# This should, and does, fail. In Perl, it does not, which I think is a # bug because replacing the B in the pattern by (B|D) does make it fail. /A(*COMMIT)B/aftertext,mark @@ -2964,37 +2964,37 @@ /A(*PRUNE)B|A(*PRUNE)C/mark \= Expect no match AC - + # Mark names can be duplicated. Perl doesn't give a mark for this one, # though PCRE2 does. /^A(*:A)B|^X(*:A)Y/mark \= Expect no match XAQQ - -# COMMIT at the start of a pattern should be the same as an anchor. Perl + +# COMMIT at the start of a pattern should be the same as an anchor. Perl # optimizations defeat this. So does the PCRE2 optimization unless we disable # it. /(*COMMIT)ABC/ ABCDEFG - + /(*COMMIT)ABC/no_start_optimize \= Expect no match DEFGABC - + /^(ab (c+(*THEN)cd) | xyz)/x \= Expect no match - abcccd + abcccd /^(ab (c+(*PRUNE)cd) | xyz)/x \= Expect no match - abcccd + abcccd /^(ab (c+(*FAIL)cd) | xyz)/x \= Expect no match - abcccd - + abcccd + # Perl gets some of these wrong /(?>.(*ACCEPT))*?5/ @@ -3013,19 +3013,19 @@ ACBD \= Expect no match A\nB - ACB\n + ACB\n /A\NB./Bs ACBD - ACB\n + ACB\n \= Expect no match - A\nB - + A\nB + /A\NB/newline=crlf A\nB A\rB \= Expect no match - A\r\nB + A\r\nB /\R+b/B @@ -3096,7 +3096,7 @@ /.+/ \= Bad offsets abc\=offset=4 - abc\=offset=-4 + abc\=offset=-4 \= Valid data abc\=offset=0 abc\=offset=1 @@ -3116,24 +3116,24 @@ /(?P(?P=axn)xxx)(?yy)/B -# These tests are here because Perl gets the first one wrong. +# These tests are here because Perl gets the first one wrong. /(\R*)(.)/s \r\n - \r\r\n\n\r - \r\r\n\n\r\n + \r\r\n\n\r + \r\r\n\n\r\n /(\R)*(.)/s \r\n - \r\r\n\n\r - \r\r\n\n\r\n + \r\r\n\n\r + \r\r\n\n\r\n /((?>\r\n|\n|\x0b|\f|\r|\x85)*)(.)/s \r\n - \r\r\n\n\r - \r\r\n\n\r\n + \r\r\n\n\r + \r\r\n\n\r\n -# ------------- +# ------------- /^abc$/B @@ -3141,12 +3141,12 @@ /^(a)*+(\w)/ aaaaX -\= Expect no match +\= Expect no match aaaa /^(?:a)*+(\w)/ aaaaX -\= Expect no match +\= Expect no match aaaa /(a)++1234/IB @@ -3205,39 +3205,39 @@ /(abc)\1+/ -# Perl doesn't get these right IMO (the 3rd is PCRE2-specific) +# Perl doesn't get these right IMO (the 3rd is PCRE2-specific) /(?1)(?:(b(*ACCEPT))){0}/ b /(?1)(?:(b(*ACCEPT))){0}c/ bc -\= Expect no match - b +\= Expect no match + b /(?1)(?:((*ACCEPT))){0}c/ c c\=notempty /^.*?(?(?=a)a|b(*THEN)c)/ -\= Expect no match +\= Expect no match ba /^.*?(?(?=a)a|bc)/ ba /^.*?(?(?=a)a(*THEN)b|c)/ -\= Expect no match +\= Expect no match ac /^.*?(?(?=a)a(*THEN)b)c/ -\= Expect no match +\= Expect no match ac /^.*?(a(*THEN)b)c/ -\= Expect no match +\= Expect no match aabc - + /^.*? (?1) c (?(DEFINE)(a(*THEN)b))/x aabc @@ -3252,11 +3252,11 @@ /(*MARK:A)(*SKIP:B)(C|X)/mark C -\= Expect no match +\= Expect no match D - + /(*:A)A+(*SKIP:A)(B|Z)/mark -\= Expect no match +\= Expect no match AAAC # ---------------------------- @@ -3264,14 +3264,14 @@ "(?=a*(*ACCEPT)b)c" c c\=notempty - + /(?1)c(?(DEFINE)((*ACCEPT)b))/ c c\=notempty - + /(?>(*ACCEPT)b)c/ c -\= Expect no match +\= Expect no match c\=notempty /(?:(?>(a)))+a%/allaftertext @@ -3279,7 +3279,7 @@ /(a)b|ac/allaftertext ac\=ovector=1 - + /(a)(b)x|abc/allaftertext abc\=ovector=2 @@ -3304,7 +3304,7 @@ foobazbarX barfooX bazX - foobarbazX + foobarbazX bazfooX\=ovector=0 bazfooX\=ovector=1 bazfooX\=ovector=2 @@ -3368,17 +3368,17 @@ /^(?>a+)(?>(z+))\w/B aaaazzzzb \= Expect no match - aazz + aazz /(.)(\1|a(?2))/ bab - + /\1|(.)(?R)\1/ cbbbc - + /(.)((?(1)c|a)|a(?2))/ \= Expect no match - baa + baa /(?P(?P=abn)xxx)/B @@ -3419,7 +3419,7 @@ /a[\NB]c/ aNc - + /a[B-\Nc]/ /a[B\Nc]/ @@ -3431,34 +3431,34 @@ # This test, with something more complicated than individual letters, causes # different behaviour in Perl. Perhaps it disables some optimization; no tag is # passed back for the failures, whereas in PCRE2 there is a tag. - + /(A|P)(*:A)(B|P) | (X|P)(X|P)(*:B)(Y|P)/x,mark AABC - XXYZ + XXYZ \= Expect no match - XAQQ - XAQQXZZ - AXQQQ - AXXQQQ + XAQQ + XAQQXZZ + AXQQQ + AXXQQQ # Perl doesn't give marks for these, though it does if the alternatives are -# replaced by single letters. - +# replaced by single letters. + /(b|q)(*:m)f|a(*:n)w/mark - aw -\= Expect no match + aw +\= Expect no match abc /(q|b)(*:m)f|a(*:n)w/mark - aw -\= Expect no match + aw +\= Expect no match abc -# After a partial match, the behaviour is as for a failure. +# After a partial match, the behaviour is as for a failure. /^a(*:X)bcde/mark abc\=ps - + # These are here because Perl doesn't return a mark, except for the first. /(?=(*:x))(q|)/aftertext,mark @@ -3526,22 +3526,22 @@ ababa\=ps ababa\=ph abababx - ababababx + ababababx /^(..)\1{2,3}?x/ aba\=ps ababa\=ps ababa\=ph abababx - ababababx - + ababababx + /^(..)(\1{2,3})ab/ abababab /^\R/ \r\=ps \r\=ph - + /^\R{2,3}x/ \r\=ps \r\=ph @@ -3550,7 +3550,7 @@ \r\r\r\=ps \r\r\r\=ph \r\rx - \r\r\rx + \r\r\rx /^\R{2,3}?x/ \r\=ps @@ -3560,20 +3560,20 @@ \r\r\r\=ps \r\r\r\=ph \r\rx - \r\r\rx - + \r\r\rx + /^\R?x/ \r\=ps \r\=ph x - \rx + \rx /^\R+x/ \r\=ps \r\=ph \r\n\=ps \r\n\=ph - \rx + \rx /^a$/newline=crlf a\r\=ps @@ -3594,7 +3594,7 @@ /./newline=crlf \r\=ps \r\=ph - + /.{2,3}/newline=crlf \r\=ps \r\=ph @@ -3613,9 +3613,9 @@ "AB(C(D))(E(F))?(?(?=\2)(?=\4))" ABCDGHI\=ovector=01 - + # These are all run as real matches in test 1; here we are just checking the -# settings of the anchored and startline bits. +# settings of the anchored and startline bits. /(?>.*?a)(?<=ba)/I @@ -3651,10 +3651,10 @@ /(?:(a)+(?C1)bb|aa(?C2)b)/ aab\=callout_capture - + /(?:(a)++(?C1)bb|aa(?C2)b)/ aab\=callout_capture - + /(?:(?>(a))(?C1)bb|aa(?C2)b)/ aab\=callout_capture @@ -3671,11 +3671,11 @@ /(ab)x|ab/ ab\=ovector=0 ab\=ovector=1 - + /(?<=123)(*MARK:xx)abc/mark xxxx123a\=ph xxxx123a\=ps - + /123\Kabc/startchar xxxx123a\=ph xxxx123a\=ps @@ -3690,22 +3690,22 @@ /aaaaa(*COMMIT)(*PRUNE)b|a+c/ aaaaaac - + # Here are some that Perl treats differently because of the way it handles -# backtracking verbs. +# backtracking verbs. /(?!a(*COMMIT)b)ac|ad/ ac - ad + ad /^(?!a(*THEN)b|ac)../ - ad + ad \= Expect no match ac /^(?=a(*THEN)b|ac)/ ac - + /\A.*?(?:a|b(*THEN)c)/ ba @@ -3716,14 +3716,14 @@ ba /(?:(a(*MARK:X)a+(*SKIP:X)b)){0}(?:(?1)|aac)/ - aac + aac /\A.*?(a|b(*THEN)c)/ ba /^(A(*THEN)B|A(*THEN)D)/ - AD - + AD + /(?!b(*THEN)a)bn|bnn/ bnn @@ -3733,7 +3733,7 @@ /(?=b(*THEN)a|)bn|bnn/ bnn -# This test causes a segfault with Perl 5.18.0 +# This test causes a segfault with Perl 5.18.0 /^(?=(a)){0}b(?1)/ backgammon @@ -3841,13 +3841,13 @@ /[a-c]{0,6}d/IB -# End of special auto-possessive tests +# End of special auto-possessive tests /^A\o{1239}B/ A\123B /^A\oB/ - + /^A\x{zz}B/ /^A\x{12Z/ @@ -3919,13 +3919,13 @@ /[[:<:]]red[[:>:]]/B little red riding hood - a /red/ thing + a /red/ thing red is a colour - put it all on red + put it all on red \= Expect no match no reduction Alfred Winifred - + /[a[:<:]] should give error/ /(?=ab\K)/aftertext @@ -3934,7 +3934,7 @@ /abcd/newline=lf,firstline \= Expect no match xx\nxabcd - + # Test stack guard external calls. /(((a)))/stackguard=1 @@ -3965,25 +3965,25 @@ /A\9B/ -# This one is here because Perl fails to match "12" for this pattern when the $ +# This one is here because Perl fails to match "12" for this pattern when the $ # is present. - + /^(?(?=abc)\w{3}:|\d\d)$/ abc: 12 \= Expect no match 123 - xyz + xyz -# Perl gets this one wrong, giving "a" as the after text for ca and failing to +# Perl gets this one wrong, giving "a" as the after text for ca and failing to # match for cd. /(?(?=ab)ab)/aftertext abxxx ca - cd - -# This should test both paths for processing OP_RECURSE. + cd + +# This should test both paths for processing OP_RECURSE. /(?(R)a+|(?R)b)/ aaaabcde @@ -3995,29 +3995,29 @@ /(*NOTEMPTY)a*?b*?/ ab ba - cb + cb /(*NOTEMPTY_ATSTART)a*?b*?/aftertext ab - cdab + cdab /(?(VERSION>=10.0)yes|no)/I yesno - + /(?(VERSION=8)yes){3}/BI,aftertext yesno /(?(VERSION=8)yes|no){3}/I yesnononoyes \= Expect no match - yesno + yesno /(?:(?abc)|xyz)(?(VERSION)yes|no)/I abcyes xyzno \= Expect no match abcno - xyzyes + xyzyes /(?(VERSION<10)yes|no)/ @@ -4033,11 +4033,11 @@ /(|ab)*?d/I abd - xyd + xyd /(|ab)*?d/I,no_start_optimize abd - xyd + xyd /\k*(?aa)(?bb)/match_unset_backref,dupnames aabb @@ -4097,7 +4097,7 @@ /abc/replace=[9]XYZ 123abc123 - + /abc/replace=xyz 1abc2\=partial_hard @@ -4109,23 +4109,23 @@ /(?<=abc)(|def)/g,replace=<$0> 123abcxyzabcdef789abcpqr - + /./replace=$0 a - + /(.)(.)/replace=$2+$1 abc - + /(?.)(?.)/replace=$B+$A abc - + /(.)(.)/g,replace=$2$1 - abcdefgh - + abcdefgh + /(*:pear)apple|(*:orange)lemon|(*:strawberry)blackberry/g,replace=${*MARK} apple lemon blackberry apple strudel - fruitless + fruitless /(*:pear)apple|(*:orange)lemon|(*:strawberry)blackberry/replace=${*MARK} sauce, apple lemon blackberry @@ -4133,15 +4133,15 @@ /(*:pear)apple|(*:orange)lemon|(*:strawberry)blackberry/g,replace=<$*MARK> apple lemon blackberry apple strudel - fruitless - -/(*:pear)apple/g,replace=${*MARKING} + fruitless + +/(*:pear)apple/g,replace=${*MARKING} apple lemon blackberry /(*:pear)apple/g,replace=${*MARK-time apple lemon blackberry -/(*:pear)apple/g,replace=${*mark} +/(*:pear)apple/g,replace=${*mark} apple lemon blackberry /(*:pear)apple|(*:orange)lemon|(*:strawberry)blackberry/g,replace=<$*MARKET> @@ -4177,10 +4177,10 @@ /(a)(b)|(c)/ XcX\=ovector=2,get=1,get=2,get=3,get=4,getall - + /x(?=ab\K)/ - xab\=get=0 - xab\=copy=0 + xab\=get=0 + xab\=copy=0 xab\=getall /(?a)|(?b)/dupnames @@ -4243,16 +4243,16 @@ 00765 456 \= Expect no match - 356 + 356 '^(a)*+(\w)' g - g\=ovector=1 + g\=ovector=1 '^(?:a)*+(\w)' g - g\=ovector=1 - + g\=ovector=1 + # These two pattern showeds up compile-time bugs "((?2){0,1999}())?" @@ -4293,11 +4293,11 @@ /^(?(?C25)(?=abc)abcd|xyz)/B,callout_info abcdefg - xyz123 + xyz123 /^(?(?C$abc$)(?=abc)abcd|xyz)/B abcdefg - xyz123 + xyz123 /^ab(?C'first')cd(?C"second")ef/ abcdefg @@ -4314,8 +4314,8 @@ /(?(?!)a|b)/ bbb -\= Expect no match - aaa +\= Expect no match + aaa # JIT gives a different error message for the infinite recursion @@ -4349,9 +4349,9 @@ /abc/ \= Expect no match \[9x!xxx(]{9999} - + /(abc)*/ - \[abc]{5} + \[abc]{5} /^/gm \n\n\n @@ -4369,17 +4369,17 @@ /A\8B\9C/ A8B9C - + /(?x:((?'a')) # comment (with parentheses) and | vertical (?-x:#not a comment (?'b')) # this is a comment () (?'c')) # not a comment (?'d')/info /(?|(?'a')(2)(?'b')|(?'a')(?'a')(3))/I,dupnames A23B - B32A + B32A # These are some patterns that used to cause buffer overflows or other errors -# while compiling. +# while compiling. /.((?2)(?R)|\1|$)()/B @@ -4463,7 +4463,7 @@ {4,5a}bc /\x0{ab}/ - \0{ab} + \0{ab} /^(a(b))\1\g1\g{1}\g-1\g{-1}\g{-02}Z/ ababababbbabZXXXX @@ -4505,8 +4505,8 @@ \= Expect no match aacb -/(*MARK:a\zb)z/alt_verbnames - +/(*MARK:a\zb)z/alt_verbnames + /(*:ab\t(d\)c)xxx/ /(*:ab\t(d\)c)xxx/alt_verbnames,mark @@ -4514,28 +4514,28 @@ /(*:A\Qxx)x\EB)x/alt_verbnames,mark x - + /(*:A\ExxxB)x/alt_verbnames,mark - x - + x + /(*: A \ and #comment \ B)x/x,alt_verbnames,mark - x - + x + /(*: A \ and #comment \ B)x/alt_verbnames,mark - x - + x + /(*: A \ and #comment \ B)x/x,mark - x - + x + /(*: A \ and #comment \ B)x/mark - x - + x + /(*:A -B)x/alt_verbnames,mark +B)x/alt_verbnames,mark x /(*:abc\Qpqr)/alt_verbnames @@ -4553,7 +4553,7 @@ B)x/alt_verbnames,mark 1234abc\=offset_limit=7 \= Expect no match 1234abc\=offset_limit=6 - + /A/g,replace=-,use_offset_limit XAXAXAXAXA\=offset_limit=4 @@ -4567,16 +4567,16 @@ B)x/alt_verbnames,mark /abcd/null_context abcd\=null_context -\= Expect error +\= Expect error abcd\=null_context,find_limits - abcd\=allusedtext,startchar + abcd\=allusedtext,startchar /abcd/replace=w\rx\x82y\o{333}z(\Q12\$34$$\x34\E5$$),substitute_extended abcd - + /a(bc)(DE)/replace=a\u$1\U$1\E$1\l$2\L$2\Eab\Uab\LYZ\EDone,substitute_extended abcDE - + /abcd/replace=xy\kz,substitute_extended abcd @@ -4614,8 +4614,8 @@ B)x/alt_verbnames,mark /(?J)(?:(?a)|(?b))/replace=<$A> [a] - [b] -\= Expect error + [b] +\= Expect error (a)\=ovector=1 /(a)|(b)/replace=<$1> @@ -4640,10 +4640,10 @@ B)x/alt_verbnames,mark /(?=a\K)/replace=z BaCaD - + /(?'abcdefghijklmnopqrstuvwxyzABCDEFG'toolong)/ - -/(?'abcdefghijklmnopqrstuvwxyzABCDEF'justright)/ + +/(?'abcdefghijklmnopqrstuvwxyzABCDEF'justright)/ # These two use zero-termination /abcd/max_pattern_length=3 @@ -4766,7 +4766,7 @@ a)"xI /a|(b)c/replace=>$1<,substitute_unset_empty cat - xbcom + xbcom /a|(b)c/ cat\=replace=>$1< @@ -4780,26 +4780,26 @@ a)"xI /a|(?'X'b)c/replace=>$X<,substitute_unset_empty cat - xbcom + xbcom /a|(?'X'b)c/replace=>$Y<,substitute_unset_empty cat - cat\=substitute_unknown_unset - cat\=substitute_unknown_unset,-substitute_unset_empty + cat\=substitute_unknown_unset + cat\=substitute_unknown_unset,-substitute_unset_empty /a|(b)c/replace=>$2<,substitute_unset_empty cat - cat\=substitute_unknown_unset - cat\=substitute_unknown_unset,-substitute_unset_empty + cat\=substitute_unknown_unset + cat\=substitute_unknown_unset,-substitute_unset_empty /()()()/use_offset_limit \=ovector=11000000000 \=callout_fail=11000000000 \=callout_fail=1:11000000000 - \=callout_data=11000000000 - \=callout_data=-11000000000 - \=offset_limit=1100000000000000000000 - \=copy=11000000000 + \=callout_data=11000000000 + \=callout_data=-11000000000 + \=offset_limit=1100000000000000000000 + \=copy=11000000000 /(*MARK:A\x00b)/mark abc @@ -4848,22 +4848,22 @@ a)"xI /([ab])...(?<=\1)z/ a11az - b11bz + b11bz \= Expect no match - b11az - + b11az + /(?|([ab]))...(?<=\1)z/ /([ab])(\1)...(?<=\2)z/ aa11az - -/(a\2)(b\1)(?<=\2)/ - + +/(a\2)(b\1)(?<=\2)/ + /(?[ab])...(?<=\k'A')z/ a11az - b11bz + b11bz \= Expect no match - b11az + b11az /(?[ab])...(?<=\k'A')(?)z/dupnames @@ -4877,8 +4877,8 @@ a)"xI /'ab(?C1)c'/hex,auto_callout abc - -# Perl accepts these, but gives a warning. We can't warn, so give an error. + +# Perl accepts these, but gives a warning. We can't warn, so give an error. /[a-[:digit:]]+/ a-a9-a @@ -4943,7 +4943,7 @@ a)"xI "()X|((((((((()))))))((((())))))\2())((((((\2\2)))\2)(\22((((\2\2)2))\2)))(2\ZZZ)+:)Z^|91ZiZZnter(ZZ |91Z(ZZ ZZ(\r2Z( or#(\Z2(Z\Z(\2\2)2))\2Z)Z(\22Z((\Z2(Z\Z(\2\2)2))\2Z+:)Z|91Z(ZZ ZZ(\r2Z( or#(\Z2(Z\Z((Z*(\2(Z\':))\0)i|||||||||||||||loZ\2\2)2))\2Z)Z(\22Z((\Z2(Z\Z(\2\2)2))\2Z)))int \)\0nte!rnal errpr\2\\21r(2\ZZZ)+:)Z!|91Z(ZZ ZZ(\r2Z( or#(\Z2(Z\Z(\2\2)2))\2Z)Z(\22Z((\Z2(Z\Z(\2\2)2))\2Z)))int \)\0(2\ZZZ)+:)Z^|91ZiZZnter(ZZ |91Z(ZZ ZZ(\r2Z( or#(\Z2(Z\Z(\2\2)2))\2Z)Z(\22Z((\Z2(Z\Z(\2\2)2))\2Z)))int \)\0(2\ZZZ)+:)Z^)))int \)\0(2\ZZZ)+:)Z^|91ZiZZnter(ZZernZal ZZ(\r2Z( or#(\Z2(Z\Z(\2\2)2))\2Z)Z(\22Z((\Z2(Z\Z(\2\2)2))\2Z)))int \))\ZZ(\r2Z( or#(\Z2(Z\Z(\2\2)2))\2Z)Z(\22Z((\Z2(Z\Z(\2\2)))\2))))((((((\2\2))))))"I # This checks that new code for handling groups that may match an empty string -# works on a very large number of alternatives. This pattern used to provoke a +# works on a very large number of alternatives. This pattern used to provoke a # complaint that it was too complicated. /(?:\[A|B|C|D|E|F|G|H|I|J|]{200}Z)/expand @@ -4975,35 +4975,35 @@ a)"xI // \=ovector=7777777777 - -# This is here because Perl matches, even though a COMMIT is encountered -# outside of the recursion. + +# This is here because Perl matches, even though a COMMIT is encountered +# outside of the recursion. /(?1)(A(*COMMIT)|B)D/ BAXBAD - + "(?1){2}(a)"B "(?1){2,4}(a)"B # This test differs from Perl for the first subject. Perl ends up with -# $1 set to 'B'; PCRE2 has it unset (which I think is right). +# $1 set to 'B'; PCRE2 has it unset (which I think is right). /^(?: -(?:A| (?:B|B(*ACCEPT)) (?<=(.)) D) +(?:A| (?:B|B(*ACCEPT)) (?<=(.)) D) (Z) )+$/x AZB - AZBDZ - -# The first of these, when run by Perl, gives the mark 'aa', which is wrong. + AZBDZ + +# The first of these, when run by Perl, gives the mark 'aa', which is wrong. '(?>a(*:aa))b|ac' mark ac '(?:a(*:aa))b|ac' mark ac - + /(R?){65}/ (R?){65} @@ -5023,7 +5023,7 @@ a)"xI /^ (?(DEFINE) (..(*ACCEPT)|...) ) (?1)$/x \= Expect no match abc - + # Perl gives no match for this one /(a(*MARK:m)(*ACCEPT)){0}(?1)/mark @@ -5088,7 +5088,7 @@ a)"xI /^[^a]{3,}?x/i,no_start_optimize,no_auto_possess \= Expect no match bbb - cc + cc /^X\S/no_start_optimize,no_auto_possess \= Expect no match @@ -5145,7 +5145,7 @@ a)"xI /^X\V+?/no_start_optimize,no_auto_possess \= Expect no match X - X\n + X\n /^X\D+?/no_start_optimize,no_auto_possess \= Expect no match @@ -5155,17 +5155,17 @@ a)"xI /^X\S+?/no_start_optimize,no_auto_possess \= Expect no match X - X\n + X\n /^X\W+?/no_start_optimize,no_auto_possess \= Expect no match X - XX + XX /^X.+?Z/no_start_optimize,no_auto_possess \= Expect no match XY\n - + /(*CRLF)^X.+?Z/no_start_optimize,no_auto_possess \= Expect no match XY\r\=ps @@ -5176,20 +5176,20 @@ a)"xI X\n\r\n X\n\rY X\n\nY - X\n\x{0c}Y - + X\n\x{0c}Y + /(*BSR_ANYCRLF)^X\R+?Z/no_start_optimize,no_auto_possess \= Expect no match X\nX X\n\r\n X\n\rY X\n\nY - X\n\x{0c}Y - + X\n\x{0c}Y + /^X\H+?Z/no_start_optimize,no_auto_possess \= Expect no match XY\t - XYY + XYY /^X\h+?Z/no_start_optimize,no_auto_possess \= Expect no match @@ -5199,7 +5199,7 @@ a)"xI /^X\V+?Z/no_start_optimize,no_auto_possess \= Expect no match XY\n - XYY + XYY /^X\v+?Z/no_start_optimize,no_auto_possess \= Expect no match @@ -5209,7 +5209,7 @@ a)"xI /^X\D+?Z/no_start_optimize,no_auto_possess \= Expect no match XY9 - XYY + XYY /^X\d+?Z/no_start_optimize,no_auto_possess \= Expect no match @@ -5219,7 +5219,7 @@ a)"xI /^X\S+?Z/no_start_optimize,no_auto_possess \= Expect no match XY\n - XYY + XYY /^X\s+?Z/no_start_optimize,no_auto_possess \= Expect no match @@ -5229,12 +5229,12 @@ a)"xI /^X\W+?Z/no_start_optimize,no_auto_possess \= Expect no match X.A - X++ + X++ /^X\w+?Z/no_start_optimize,no_auto_possess \= Expect no match Xa. - Xaa + Xaa /^X.{1,3}Z/s,no_start_optimize,no_auto_possess \= Expect no match @@ -5248,12 +5248,12 @@ a)"xI /^X\V+Z/no_start_optimize,no_auto_possess \= Expect no match XY\n - XYY + XYY /^(X(*THEN)Y|AB){0}(?1)/ ABX \= Expect no match - XAB + XAB /^(?!A(?C1)B)C/ ABC\=callout_error=1,no_jit @@ -5332,14 +5332,14 @@ a)"xI /cat|dog/match_word the cat sat -\= Expect no match +\= Expect no match caterpillar snowcat syndicate /(cat)|dog/match_line,literal (cat)|dog -\= Expect no match +\= Expect no match the cat sat caterpillar snowcat @@ -5348,6 +5348,21 @@ a)"xI /a whole line/match_line,multiline Rhubarb \na whole line\n custard \= Expect no match - Not a whole line + Not a whole line -# End of testinput2 +# Perl gets this wrong, failing to capture 'b' in group 1. + +/^(b+|a){1,2}?bc/ + bbc + +# And again here, for the "babc" subject string. + +/^(b*|ba){1,2}?bc/ + babc + bbabc + bababc +\= Expect no match + bababbc + babababc + +# End of testinput2 diff --git a/testdata/testoutput1 b/testdata/testoutput1 index 789fe49..85c1aad 100644 --- a/testdata/testoutput1 +++ b/testdata/testoutput1 @@ -183,27 +183,6 @@ No match abbbbbbbbbbbac No match -/^(b+|a){1,2}?bc/ - bbc - 0: bbc - 1: b - -/^(b*|ba){1,2}?bc/ - babc - 0: babc - 1: ba - bbabc - 0: bbabc - 1: ba - bababc - 0: bababc - 1: ba -\= Expect no match - bababbc -No match - babababc -No match - /^(ba|b*){1,2}?bc/ babc 0: babc diff --git a/testdata/testoutput2 b/testdata/testoutput2 index ef71e50..7e138dc 100644 --- a/testdata/testoutput2 +++ b/testdata/testoutput2 @@ -1,12 +1,12 @@ # This set of tests is not Perl-compatible. It checks on special features # of PCRE2's API, error diagnostics, and the compiled code of some patterns. -# It also checks the non-Perl syntax that PCRE2 supports (Python, .NET, -# Oniguruma). There are also some tests where PCRE2 and Perl differ, -# either because PCRE2 can't be compatible, or there is a possible Perl +# It also checks the non-Perl syntax that PCRE2 supports (Python, .NET, +# Oniguruma). There are also some tests where PCRE2 and Perl differ, +# either because PCRE2 can't be compatible, or there is a possible Perl # bug. # NOTE: This is a non-UTF set of tests. When UTF support is needed, use -# test 5. +# test 5. #forbid_utf #newline_default lf any anycrlf @@ -823,7 +823,7 @@ Subject length lower bound = 4 No match aaaaaa No match - + # Perl does not fail these two for the final subjects. Neither did PCRE until # release 8.01. The problem is in backtracking into a subpattern that contains # a recursive reference to itself. PCRE has now made these into atomic patterns. @@ -4400,7 +4400,7 @@ Subject length lower bound = 2 Callout data = 1 0: ab 1: ab -\= Expect no match +\= Expect no match aaabbb\=callout_data=-1 --->aaabbb 1 ^ ^ b @@ -4844,7 +4844,7 @@ Subject length lower bound = 2 +2 ^ ^ b +3 ^ ^ 0: aaaab -\= Expect no match +\= Expect no match aaaacb --->aaaacb +0 ^ a+ @@ -4912,7 +4912,7 @@ Subject length lower bound = 4 +10 ^ ^ 0: defx 1: def -\= Expect no match +\= Expect no match abcdefzx --->abcdefzx +0 ^ ( @@ -4986,7 +4986,7 @@ Subject length lower bound = 4 +10 ^ ^ 0: defx 1: def -\= Expect no match +\= Expect no match abcdefzx --->abcdefzx +0 ^ ( @@ -5102,7 +5102,7 @@ Capturing subpattern count = 1 Options: auto_callout Starting code units: a b x Subject length lower bound = 2 -\= Expect no match +\= Expect no match Note: that { does NOT introduce a quantifier --->Note: that { does NOT introduce a quantifier +0 ^ ( @@ -5152,7 +5152,7 @@ Capturing subpattern count = 1 Options: auto_callout Starting code units: a b x Subject length lower bound = 2 -\= Expect no match +\= Expect no match Note: that { does NOT introduce a quantifier --->Note: that { does NOT introduce a quantifier +0 ^ ( @@ -5819,7 +5819,7 @@ Subject length lower bound = 2 Number not found for group 'Z' Copy substring 'Z' failed (-49): unknown substring C a1 (2) A (non-unique) - + /(?|(?)(?)(?)|(?)(?)(?))/I,dupnames Capturing subpattern count = 3 Named capturing subpatterns: @@ -6147,7 +6147,7 @@ Subject length lower bound = 3 No match xyz\rabclf No match - + /^abc/Im,newline=cr Capturing subpattern count = 0 Options: multiline @@ -7722,13 +7722,13 @@ No match 0: \x0d\x0afoo \nfoo 0: \x0afoo - + /^$/gm,newline=any abc\r\rxyz 0: - abc\n\rxyz + abc\n\rxyz 0: -\= Expect no match +\= Expect no match abc\r\nxyz No match @@ -7743,7 +7743,7 @@ No match 0+ \x0d\x0a 0: \x0d\x0a 0+ - + /(?m)$/g,newline=any,aftertext abc\r\n\r\n 0: @@ -7763,7 +7763,7 @@ No match /^X/m XABC 0: X -\= Expect no match +\= Expect no match XABC\=notbol No match @@ -7798,9 +7798,9 @@ No match 0: xyabcabc 1: abc \= Expect no match - xyabc + xyabc No match - + /x(?-0)y/ Failed: error 126 at offset 5: a relative value of zero is not allowed @@ -7836,9 +7836,9 @@ Failed: error 115 at offset 5: reference to non-existent subpattern Y 0: Y \= Expect no match - abcY + abcY No match - + /^((?(+1)X|Y)(abc))+/B ------------------------------------------------------------------ Bra @@ -7866,7 +7866,7 @@ No match 1: Xabc 2: abc \= Expect no match - XabcXabc + XabcXabc No match /(?(-1)a)/B @@ -7912,11 +7912,11 @@ Failed: error 115 at offset 6: reference to non-existent subpattern tom-tom 0: tom-tom 1: tom - bon-bon + bon-bon 0: bon-bon 1: bon \= Expect no match - tom-bon + tom-bon No match /\g{A/ @@ -7940,7 +7940,7 @@ Failed: error 142 at offset 4: syntax error in subpattern name (missing terminat >abc< 0: abc 1: abc - >xyz< + >xyz< 0: xyz 1: xyz @@ -7970,7 +7970,7 @@ Failed: error 142 at offset 4: syntax error in subpattern name (missing terminat 1: x 2: abc 3: x - xxyzx + xxyzx 0: xxyzx 1: x 2: xyz @@ -8006,7 +8006,7 @@ Failed: error 142 at offset 4: syntax error in subpattern name (missing terminat 2: abc 3: pqr 4: x - xxyzx + xxyzx 0: xxyzx 1: x 2: xyz @@ -8024,7 +8024,7 @@ Failed: error 142 at offset 4: syntax error in subpattern name (missing terminat \= Expect no match XXXX No match - + /\H+\hY/B ------------------------------------------------------------------ Bra @@ -8034,7 +8034,7 @@ No match Ket End ------------------------------------------------------------------ - XXXX Y + XXXX Y 0: XXXX Y /\H+ Y/B @@ -8261,7 +8261,7 @@ Failed: error 106 at offset 10: missing terminating ] for character class +3 ^ ^ (*FAIL) +3 ^^ (*FAIL) No match - + /a+b?c+(*FAIL)/auto_callout \= Expect no match aaabccc @@ -8325,7 +8325,7 @@ No match +15 ^ ^ (*FAIL) +15 ^ ^ (*FAIL) No match - + /a+b?(*SKIP)c+(*FAIL)/auto_callout \= Expect no match aaabcccaaabccc @@ -8372,7 +8372,7 @@ No match +13 ^ ^ (*FAIL) +13 ^ ^ (*FAIL) No match - + /a(*MARK)b/ Failed: error 166 at offset 7: (*MARK) must have an argument @@ -8397,17 +8397,17 @@ Failed: error 115 at offset 3: reference to non-existent subpattern \= Expect no match \r\nA No match - + /\nA/newline=crlf - \r\nA + \r\nA 0: \x0aA /[\r\n]A/newline=crlf - \r\nA + \r\nA 0: \x0aA /(\r|\n)A/newline=crlf - \r\nA + \r\nA 0: \x0aA 1: \x0a @@ -8418,52 +8418,52 @@ Failed: error 160 at offset 5: (*VERB) not recognized or malformed a\nb 0: a\x0ab \= Expect no match - a\rb + a\rb No match /(*CR)a.b/newline=lf a\nb 0: a\x0ab \= Expect no match - a\rb + a\rb No match /(*LF)a.b/newline=CRLF a\rb 0: a\x0db \= Expect no match - a\nb + a\nb No match /(*CRLF)a.b/ a\rb 0: a\x0db - a\nb + a\nb 0: a\x0ab \= Expect no match - a\r\nb + a\r\nb No match /(*ANYCRLF)a.b/newline=CR \= Expect no match a\rb No match - a\nb + a\nb No match - a\r\nb + a\r\nb No match /(*ANY)a.b/newline=cr \= Expect no match a\rb No match - a\nb + a\nb No match - a\r\nb + a\r\nb No match - a\x85b + a\x85b No match - + /(*ANY).*/g abc\r\ndef 0: abc @@ -8484,21 +8484,21 @@ No match 0: 0: def 0: - + /(*NUL)^.*/ a\nb\x00ccc 0: a\x0ab - + /(*NUL)^.*/s a\nb\x00ccc 0: a\x0ab\x00ccc - + /^x/m,newline=NUL ab\x00xy 0: x - + /'#comment' 0d 0a 00 '^x\' 0a 'y'/x,newline=nul,hex - x\nyz + x\nyz 0: x\x0ay /(*NUL)^X\NY/ @@ -8507,7 +8507,7 @@ No match X\rY 0: X\x0dY \= Expect no match - X\x00Y + X\x00Y No match /a\Rb/I,bsr=anycrlf @@ -8525,7 +8525,7 @@ Subject length lower bound = 3 \= Expect no match a\x85b No match - a\x0bb + a\x0bb No match /a\Rb/I,bsr=unicode @@ -8542,9 +8542,9 @@ Subject length lower bound = 3 0: a\x0d\x0ab a\x85b 0: a\x85b - a\x0bb + a\x0bb 0: a\x0bb - + /a\R?b/I,bsr=anycrlf Capturing subpattern count = 0 \R matches CR, LF, or CRLF @@ -8560,7 +8560,7 @@ Subject length lower bound = 2 \= Expect no match a\x85b No match - a\x0bb + a\x0bb No match /a\R?b/I,bsr=unicode @@ -8577,9 +8577,9 @@ Subject length lower bound = 2 0: a\x0d\x0ab a\x85b 0: a\x85b - a\x0bb + a\x0bb 0: a\x0bb - + /a\R{2,4}b/I,bsr=anycrlf Capturing subpattern count = 0 \R matches CR, LF, or CRLF @@ -8595,7 +8595,7 @@ Subject length lower bound = 4 \= Expect no match a\x85\x85b No match - a\x0b\x0bb + a\x0b\x0bb No match /a\R{2,4}b/I,bsr=unicode @@ -8612,12 +8612,12 @@ Subject length lower bound = 4 0: a\x0d\x0a\x0a\x0d\x0db a\x85\x85b 0: a\x85\x85b - a\x0b\x0bb + a\x0b\x0bb 0: a\x0b\x0bb -\= Expect no match - a\r\r\r\r\rb +\= Expect no match + a\r\r\r\r\rb No match - + /(*BSR_ANYCRLF)a\Rb/I Capturing subpattern count = 0 \R matches CR, LF, or CRLF @@ -8626,7 +8626,7 @@ Last code unit = 'b' Subject length lower bound = 3 a\nb 0: a\x0ab - a\rb + a\rb 0: a\x0db /(*BSR_UNICODE)a\Rb/I @@ -8647,7 +8647,7 @@ Last code unit = 'b' Subject length lower bound = 3 a\nb 0: a\x0ab - a\rb + a\rb 0: a\x0db /(*CRLF)(*BSR_UNICODE)a\Rb/I @@ -8750,10 +8750,10 @@ Failed: error 157 at offset 8: \g is not followed by a braced, angle-bracketed, /^(?+1)(?x|y){0}z/ xzxx 0: xz - yzyy + yzyy 0: yz \= Expect no match - xxz + xxz No match /(\3)(\1)(a)/ @@ -8767,13 +8767,13 @@ No match 1: 2: 3: a - + /TA]/ - The ACTA] comes + The ACTA] comes 0: TA] /TA]/alt_bsux,allow_empty_class,match_unset_backref,dupnames - The ACTA] comes + The ACTA] comes 0: TA] /(?2)[]a()b](abc)/ @@ -8788,7 +8788,7 @@ Failed: error 115 at offset 3: reference to non-existent subpattern abcbabc 0: abcbabc 1: abc -\= Expect no match +\= Expect no match abcXabc No match @@ -8796,7 +8796,7 @@ No match abcXabc 0: abcXabc 1: abc -\= Expect no match +\= Expect no match abcbabc No match @@ -8827,30 +8827,30 @@ No match /a[]+b/alt_bsux,allow_empty_class,match_unset_backref,dupnames \= Expect no match - ab + ab No match /a[]*+b/alt_bsux,allow_empty_class,match_unset_backref,dupnames \= Expect no match - ab + ab No match /a[^]b/alt_bsux,allow_empty_class,match_unset_backref,dupnames aXb 0: aXb - a\nb + a\nb 0: a\x0ab \= Expect no match - ab + ab No match - + /a[^]+b/alt_bsux,allow_empty_class,match_unset_backref,dupnames aXb 0: aXb - a\nX\nXb + a\nX\nXb 0: a\x0aX\x0aXb \= Expect no match - ab + ab No match /a(?!)b/B @@ -8902,7 +8902,7 @@ Subject length lower bound = 0 +12 ^ ) +13 ^ 0: - abc + abc --->abc +0 ^ (? +2 ^ (?= @@ -8923,7 +8923,7 @@ Subject length lower bound = 0 +10 ^^ | +13 ^^ 0: b - + /(?(?=b).*b|^d)/I Capturing subpattern count = 0 Subject length lower bound = 1 @@ -8933,28 +8933,28 @@ Capturing subpattern count = 0 Subject length lower bound = 1 /xyz/auto_callout - xyz + xyz --->xyz +0 ^ x +1 ^^ y +2 ^ ^ z +3 ^ ^ 0: xyz - abcxyz + abcxyz --->abcxyz +0 ^ x +1 ^^ y +2 ^ ^ z +3 ^ ^ 0: xyz -\= Expect no match +\= Expect no match abc No match - abcxypqr + abcxypqr No match - + /xyz/auto_callout,no_start_optimize - abcxyz + abcxyz --->abcxyz +0 ^ x +0 ^ x @@ -8964,7 +8964,7 @@ No match +2 ^ ^ z +3 ^ ^ 0: xyz -\= Expect no match +\= Expect no match abc --->abc +0 ^ x @@ -8972,7 +8972,7 @@ No match +0 ^ x +0 ^ x No match - abcxypqr + abcxypqr --->abcxypqr +0 ^ x +0 ^ x @@ -8986,7 +8986,7 @@ No match +0 ^ x +0 ^ x No match - + /(*NO_START_OPT)xyz/auto_callout abcxyz --->abcxyz @@ -8998,7 +8998,7 @@ No match +17 ^ ^ z +18 ^ ^ 0: xyz - + /(*NO_AUTO_POSSESS)a+b/B ------------------------------------------------------------------ Bra @@ -9009,7 +9009,7 @@ No match ------------------------------------------------------------------ /xyz/auto_callout,no_start_optimize - abcxyz + abcxyz --->abcxyz +0 ^ x +0 ^ x @@ -9067,7 +9067,7 @@ Failed: error 115 at offset 5: reference to non-existent subpattern 3: c 4: d 5: Y - + /Xa{2,4}b/ X\=ps Partial match: X @@ -9079,7 +9079,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /Xa{2,4}?b/ X\=ps Partial match: X @@ -9091,7 +9091,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /Xa{2,4}+b/ X\=ps Partial match: X @@ -9103,7 +9103,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /X\d{2,4}b/ X\=ps Partial match: X @@ -9115,7 +9115,7 @@ Partial match: X33 Partial match: X333 X3333\=ps Partial match: X3333 - + /X\d{2,4}?b/ X\=ps Partial match: X @@ -9127,7 +9127,7 @@ Partial match: X33 Partial match: X333 X3333\=ps Partial match: X3333 - + /X\d{2,4}+b/ X\=ps Partial match: X @@ -9139,7 +9139,7 @@ Partial match: X33 Partial match: X333 X3333\=ps Partial match: X3333 - + /X\D{2,4}b/ X\=ps Partial match: X @@ -9151,7 +9151,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /X\D{2,4}?b/ X\=ps Partial match: X @@ -9163,7 +9163,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /X\D{2,4}+b/ X\=ps Partial match: X @@ -9175,7 +9175,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /X[abc]{2,4}b/ X\=ps Partial match: X @@ -9187,7 +9187,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /X[abc]{2,4}?b/ X\=ps Partial match: X @@ -9199,7 +9199,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /X[abc]{2,4}+b/ X\=ps Partial match: X @@ -9211,7 +9211,7 @@ Partial match: Xaa Partial match: Xaaa Xaaaa\=ps Partial match: Xaaaa - + /X[^a]{2,4}b/ X\=ps Partial match: X @@ -9223,7 +9223,7 @@ Partial match: Xzz Partial match: Xzzz Xzzzz\=ps Partial match: Xzzzz - + /X[^a]{2,4}?b/ X\=ps Partial match: X @@ -9235,7 +9235,7 @@ Partial match: Xzz Partial match: Xzzz Xzzzz\=ps Partial match: Xzzzz - + /X[^a]{2,4}+b/ X\=ps Partial match: X @@ -9247,7 +9247,7 @@ Partial match: Xzz Partial match: Xzzz Xzzzz\=ps Partial match: Xzzzz - + /(Y)X\1{2,4}b/ YX\=ps Partial match: YX @@ -9259,7 +9259,7 @@ Partial match: YXYY Partial match: YXYYY YXYYYY\=ps Partial match: YXYYYY - + /(Y)X\1{2,4}?b/ YX\=ps Partial match: YX @@ -9271,7 +9271,7 @@ Partial match: YXYY Partial match: YXYYY YXYYYY\=ps Partial match: YXYYYY - + /(Y)X\1{2,4}+b/ YX\=ps Partial match: YX @@ -9283,7 +9283,7 @@ Partial match: YXYY Partial match: YXYYY YXYYYY\=ps Partial match: YXYYYY - + /\++\KZ|\d+X|9+Y/startchar ++++123999\=ps Partial match: 123999 @@ -9299,7 +9299,7 @@ Partial match: 123999 No match ZA\=ps No match - + /Z(?!)/ \= Expect no match Z\=ps @@ -9312,7 +9312,7 @@ No match 0: dog dogs\=ph Partial match: dogs - + /dog(sbody)??/ dogs\=ps 0: dog @@ -9324,7 +9324,7 @@ Partial match: dogs 0: dog dogs\=ph 0: dog - + /dogsbody|dog/ dogs\=ps 0: dog @@ -9342,7 +9342,7 @@ Partial match: the cat 0: abc abc\=ph 0: abc - + /abc\K123/startchar xyzabc123pqr 0: abc123 @@ -9351,9 +9351,9 @@ Partial match: the cat Partial match: abc12 xyzabc12\=ph Partial match: abc12 - + /(?<=abc)123/ - xyzabc123pqr + xyzabc123pqr 0: 123 xyzabc12\=ps Partial match: abc12 @@ -9458,7 +9458,7 @@ Partial match: +ab No match xyzabcdef\=notempty No match - + /^(?:(?=abc)|abc\K)/aftertext,startchar abcdef 0: @@ -9467,7 +9467,7 @@ No match 0: abc ^^^ 0+ def -\= Expect no match +\= Expect no match abcdef\=notempty No match @@ -9487,7 +9487,7 @@ No match xyz\=notempty_atstart 0: 0+ yz -\= Expect no match +\= Expect no match xyz\=notempty No match @@ -9498,7 +9498,7 @@ No match xyzabc 0: 0+ xyzabc -\= Expect no match +\= Expect no match xyzabc\=notempty No match xyzabc\=notempty_atstart @@ -9507,7 +9507,7 @@ No match No match xyz\=notempty No match - + /^(?a|b\gc)/ aaaa 0: a @@ -9515,7 +9515,7 @@ No match bacxxx 0: bac 1: bac - bbaccxxx + bbaccxxx 0: bbacc 1: bbacc bbbacccxx @@ -9529,7 +9529,7 @@ No match bacxxx 0: bac 1: bac - bbaccxxx + bbaccxxx 0: bbacc 1: bbacc bbbacccxx @@ -9543,7 +9543,7 @@ No match bacxxx 0: bac 1: bac - bbaccxxx + bbaccxxx 0: bbacc 1: bbacc bbbacccxx @@ -9557,7 +9557,7 @@ No match bacxxx 0: bac 1: bac - bbaccxxx + bbaccxxx 0: bbacc 1: bbacc bbbacccxx @@ -9571,7 +9571,7 @@ No match bacxxx 0: bac 1: bac - bbaccxxx + bbaccxxx 0: bbacc 1: bbacc bbbacccxx @@ -9587,7 +9587,7 @@ No match 0: bac 1: bac 2: bac - bbaccxxx + bbaccxxx 0: bbacc 1: bbacc 2: bbacc @@ -9600,7 +9600,7 @@ No match XaaX 0: aa 1: a - XAAX + XAAX 0: AA 1: A @@ -9608,15 +9608,15 @@ No match XaaX 0: aa 1: a -\= Expect no match - XAAX +\= Expect no match + XAAX No match /(?-i:\g<+1>)(?i:(a))/ XaaX 0: aa 1: a - XAAX + XAAX 0: AA 1: A @@ -9626,7 +9626,7 @@ No match abc 0: abc 1: a - accccbbb + accccbbb 0: accccbbb 1: a @@ -9645,7 +9645,7 @@ No match xbaax 0: a 1: a - xzzzax + xzzzax 0: a 1: a @@ -9762,7 +9762,7 @@ Subject length lower bound = 9 (?: [0-9a-f]{1,4} | # 1-4 hex digits or (?(1)0 | () ) ) # if null previously matched, fail; else null : # followed by colon - ){1,7} # end item; 1-7 of them required + ){1,7} # end item; 1-7 of them required [0-9a-f]{1,4} $ # final hex number at end of string (?(1)|.) # check that there was an empty component /Iix @@ -9792,7 +9792,7 @@ Subject length lower bound = 1 Failed: error 165 at offset 16: different names for subpatterns of the same number are not allowed /(?:a(? (?')|(?")) | - b(? (?')|(?")) ) + b(? (?')|(?")) ) (?('quote')[a-z]+|[0-9]+)/Ix,dupnames Capturing subpattern count = 6 Max back reference = 4 @@ -9811,7 +9811,7 @@ Subject length lower bound = 3 1: " 2: 3: " - b"aaaaa + b"aaaaa 0: b"aaaaa 1: 2: @@ -9819,12 +9819,12 @@ Subject length lower bound = 3 4: " 5: 6: " -\= Expect no match +\= Expect no match b"11111 No match - a"11111 + a"11111 No match - + /^(?|(a)(b)(c)(?d)|(?e)) (?('D')X|Y)/IBx,dupnames ------------------------------------------------------------------ Bra @@ -9877,9 +9877,9 @@ Subject length lower bound = 2 \= Expect no match abcdY No match - ey + ey No match - + /(?a) (b)(c) (?d (?(R&A)$ | (?4)) )/IBx,dupnames ------------------------------------------------------------------ Bra @@ -9920,7 +9920,7 @@ Subject length lower bound = 4 3: c 4: dd \= Expect no match - abcdde + abcdde No match /abcd*/ @@ -9994,7 +9994,7 @@ First code unit = 'i' Subject length lower bound = 1 i 0: i - + /()i(?(1)a)/I Capturing subpattern count = 1 Max back reference = 1 @@ -10018,10 +10018,10 @@ Subject length lower bound = 1 0: ab XAbX 0: Ab - CcC + CcC 0: c \= Expect no match - XABX + XABX No match /(?i)a(?s)b|c/B @@ -10081,7 +10081,7 @@ No match 0: xabcxd 1: abcxd 2: cx - + /^(?&t)*+(?(DEFINE)(?.))$/B ------------------------------------------------------------------ Bra @@ -10122,9 +10122,9 @@ No match # This one is here because Perl gives the match as "b" rather than "ab". I # believe this to be a Perl bug. - + /(?>a\Kb)z|(ab)/ - ab\=startchar + ab\=startchar 0: ab 1: ab @@ -10133,7 +10133,7 @@ No match 0: 1: 2: - 0abc + 0abc 0: 0 1: 0 2: 0 @@ -10147,7 +10147,7 @@ Failed: error 166 at offset 6: (*MARK) must have an argument /abc(*FAIL:123)xyz/ Failed: error 159 at offset 10: an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT) -# This should, and does, fail. In Perl, it does not, which I think is a +# This should, and does, fail. In Perl, it does not, which I think is a # bug because replacing the B in the pattern by (B|D) does make it fail. /A(*COMMIT)B/aftertext,mark @@ -10166,7 +10166,7 @@ No match \= Expect no match AC No match - + # Mark names can be duplicated. Perl doesn't give a mark for this one, # though PCRE2 does. @@ -10174,35 +10174,35 @@ No match \= Expect no match XAQQ No match, mark = A - -# COMMIT at the start of a pattern should be the same as an anchor. Perl + +# COMMIT at the start of a pattern should be the same as an anchor. Perl # optimizations defeat this. So does the PCRE2 optimization unless we disable # it. /(*COMMIT)ABC/ ABCDEFG 0: ABC - + /(*COMMIT)ABC/no_start_optimize \= Expect no match DEFGABC No match - + /^(ab (c+(*THEN)cd) | xyz)/x \= Expect no match - abcccd + abcccd No match /^(ab (c+(*PRUNE)cd) | xyz)/x \= Expect no match - abcccd + abcccd No match /^(ab (c+(*FAIL)cd) | xyz)/x \= Expect no match - abcccd + abcccd No match - + # Perl gets some of these wrong /(?>.(*ACCEPT))*?5/ @@ -10239,7 +10239,7 @@ No match \= Expect no match A\nB No match - ACB\n + ACB\n No match /A\NB./Bs @@ -10254,19 +10254,19 @@ No match ------------------------------------------------------------------ ACBD 0: ACBD - ACB\n + ACB\n 0: ACB\x0a \= Expect no match - A\nB + A\nB No match - + /A\NB/newline=crlf A\nB 0: A\x0aB A\rB 0: A\x0dB \= Expect no match - A\r\nB + A\r\nB No match /\R+b/B @@ -10444,7 +10444,7 @@ No match \= Bad offsets abc\=offset=4 Failed: error -33: bad offset value - abc\=offset=-4 + abc\=offset=-4 ** Invalid value in 'offset=-4' \= Valid data abc\=offset=0 @@ -10507,18 +10507,18 @@ Failed: error 115 at offset 12: reference to non-existent subpattern End ------------------------------------------------------------------ -# These tests are here because Perl gets the first one wrong. +# These tests are here because Perl gets the first one wrong. /(\R*)(.)/s \r\n 0: \x0d 1: 2: \x0d - \r\r\n\n\r + \r\r\n\n\r 0: \x0d\x0d\x0a\x0a\x0d 1: \x0d\x0d\x0a\x0a 2: \x0d - \r\r\n\n\r\n + \r\r\n\n\r\n 0: \x0d\x0d\x0a\x0a\x0d 1: \x0d\x0d\x0a\x0a 2: \x0d @@ -10528,11 +10528,11 @@ Failed: error 115 at offset 12: reference to non-existent subpattern 0: \x0d 1: 2: \x0d - \r\r\n\n\r + \r\r\n\n\r 0: \x0d\x0d\x0a\x0a\x0d 1: \x0a 2: \x0d - \r\r\n\n\r\n + \r\r\n\n\r\n 0: \x0d\x0d\x0a\x0a\x0d 1: \x0a 2: \x0d @@ -10542,16 +10542,16 @@ Failed: error 115 at offset 12: reference to non-existent subpattern 0: \x0d 1: 2: \x0d - \r\r\n\n\r + \r\r\n\n\r 0: \x0d\x0d\x0a\x0a\x0d 1: \x0d\x0d\x0a\x0a 2: \x0d - \r\r\n\n\r\n + \r\r\n\n\r\n 0: \x0d\x0d\x0a\x0a\x0d 1: \x0d\x0d\x0a\x0a 2: \x0d -# ------------- +# ------------- /^abc$/B ------------------------------------------------------------------ @@ -10578,7 +10578,7 @@ Failed: error 115 at offset 12: reference to non-existent subpattern 0: aaaaX 1: a 2: X -\= Expect no match +\= Expect no match aaaa No match @@ -10586,7 +10586,7 @@ No match aaaaX 0: aaaaX 1: X -\= Expect no match +\= Expect no match aaaa No match @@ -10765,7 +10765,7 @@ Subject length lower bound = 1 /(abc)\1+/ -# Perl doesn't get these right IMO (the 3rd is PCRE2-specific) +# Perl doesn't get these right IMO (the 3rd is PCRE2-specific) /(?1)(?:(b(*ACCEPT))){0}/ b @@ -10774,8 +10774,8 @@ Subject length lower bound = 1 /(?1)(?:(b(*ACCEPT))){0}c/ bc 0: bc -\= Expect no match - b +\= Expect no match + b No match /(?1)(?:((*ACCEPT))){0}c/ @@ -10785,7 +10785,7 @@ No match 0: c /^.*?(?(?=a)a|b(*THEN)c)/ -\= Expect no match +\= Expect no match ba No match @@ -10794,20 +10794,20 @@ No match 0: ba /^.*?(?(?=a)a(*THEN)b|c)/ -\= Expect no match +\= Expect no match ac No match /^.*?(?(?=a)a(*THEN)b)c/ -\= Expect no match +\= Expect no match ac No match /^.*?(a(*THEN)b)c/ -\= Expect no match +\= Expect no match aabc No match - + /^.*? (?1) c (?(DEFINE)(a(*THEN)b))/x aabc 0: aabc @@ -10830,12 +10830,12 @@ No match 0: C 1: C MK: A -\= Expect no match +\= Expect no match D No match, mark = A - + /(*:A)A+(*SKIP:A)(B|Z)/mark -\= Expect no match +\= Expect no match AAAC No match, mark = A @@ -10846,17 +10846,17 @@ No match, mark = A 0: c c\=notempty 0: c - + /(?1)c(?(DEFINE)((*ACCEPT)b))/ c 0: c c\=notempty 0: c - + /(?>(*ACCEPT)b)c/ c 0: -\= Expect no match +\= Expect no match c\=notempty No match @@ -10871,7 +10871,7 @@ No match ac\=ovector=1 0: ac 0+ - + /(a)(b)x|abc/allaftertext abc\=ovector=2 0: abc @@ -10944,7 +10944,7 @@ Subject length lower bound = 6 1: 2: 3: baz - foobarbazX + foobarbazX 0: bazX 1: 2: @@ -11260,7 +11260,7 @@ Subject length lower bound = 0 0: aaaazzzzb 1: zzzz \= Expect no match - aazz + aazz No match /(.)(\1|a(?2))/ @@ -11268,15 +11268,15 @@ No match 0: bab 1: b 2: ab - + /\1|(.)(?R)\1/ cbbbc 0: cbbbc 1: c - + /(.)((?(1)c|a)|a(?2))/ \= Expect no match - baa + baa No match /(?P(?P=abn)xxx)/B @@ -11386,7 +11386,7 @@ No match /a[\NB]c/ Failed: error 171 at offset 4: \N is not supported in a class aNc - + /a[B-\Nc]/ Failed: error 150 at offset 6: invalid range in character class @@ -11400,14 +11400,14 @@ Failed: error 171 at offset 5: \N is not supported in a class # This test, with something more complicated than individual letters, causes # different behaviour in Perl. Perhaps it disables some optimization; no tag is # passed back for the failures, whereas in PCRE2 there is a tag. - + /(A|P)(*:A)(B|P) | (X|P)(X|P)(*:B)(Y|P)/x,mark AABC 0: AB 1: A 2: B MK: A - XXYZ + XXYZ 0: XXY 1: 2: @@ -11416,40 +11416,40 @@ MK: A 5: Y MK: B \= Expect no match - XAQQ + XAQQ No match, mark = A - XAQQXZZ + XAQQXZZ No match, mark = A - AXQQQ + AXQQQ No match, mark = A - AXXQQQ + AXXQQQ No match, mark = B # Perl doesn't give marks for these, though it does if the alternatives are -# replaced by single letters. - +# replaced by single letters. + /(b|q)(*:m)f|a(*:n)w/mark - aw + aw 0: aw MK: n -\= Expect no match +\= Expect no match abc No match, mark = m /(q|b)(*:m)f|a(*:n)w/mark - aw + aw 0: aw MK: n -\= Expect no match +\= Expect no match abc No match, mark = m -# After a partial match, the behaviour is as for a failure. +# After a partial match, the behaviour is as for a failure. /^a(*:X)bcde/mark abc\=ps Partial match, mark=X: abc - + # These are here because Perl doesn't return a mark, except for the first. /(?=(*:x))(q|)/aftertext,mark @@ -11579,7 +11579,7 @@ Partial match: ababa abababx 0: abababx 1: ab - ababababx + ababababx 0: ababababx 1: ab @@ -11593,10 +11593,10 @@ Partial match: ababa abababx 0: abababx 1: ab - ababababx + ababababx 0: ababababx 1: ab - + /^(..)(\1{2,3})ab/ abababab 0: abababab @@ -11608,7 +11608,7 @@ Partial match: ababa 0: \x0d \r\=ph Partial match: \x0d - + /^\R{2,3}x/ \r\=ps Partial match: \x0d @@ -11624,7 +11624,7 @@ Partial match: \x0d\x0d\x0d Partial match: \x0d\x0d\x0d \r\rx 0: \x0d\x0dx - \r\r\rx + \r\r\rx 0: \x0d\x0d\x0dx /^\R{2,3}?x/ @@ -11642,9 +11642,9 @@ Partial match: \x0d\x0d\x0d Partial match: \x0d\x0d\x0d \r\rx 0: \x0d\x0dx - \r\r\rx + \r\r\rx 0: \x0d\x0d\x0dx - + /^\R?x/ \r\=ps Partial match: \x0d @@ -11652,7 +11652,7 @@ Partial match: \x0d Partial match: \x0d x 0: x - \rx + \rx 0: \x0dx /^\R+x/ @@ -11664,7 +11664,7 @@ Partial match: \x0d Partial match: \x0d\x0a \r\n\=ph Partial match: \x0d\x0a - \rx + \rx 0: \x0dx /^a$/newline=crlf @@ -11698,7 +11698,7 @@ Partial match: a\x0d 0: \x0d \r\=ph Partial match: \x0d - + /.{2,3}/newline=crlf \r\=ps Partial match: \x0d @@ -11731,9 +11731,9 @@ Partial match: \x0d\x0d ABCDGHI\=ovector=01 Matched, but too many substrings 0: ABCD - + # These are all run as real matches in test 1; here we are just checking the -# settings of the anchored and startline bits. +# settings of the anchored and startline bits. /(?>.*?a)(?<=ba)/I Capturing subpattern count = 0 @@ -11845,7 +11845,7 @@ Callout 2: last capture = 0 --->aab ^ ^ b 0: aab - + /(?:(a)++(?C1)bb|aa(?C2)b)/ aab\=callout_capture Callout 1: last capture = 1 @@ -11856,7 +11856,7 @@ Callout 2: last capture = 0 --->aab ^ ^ b 0: aab - + /(?:(?>(a))(?C1)bb|aa(?C2)b)/ aab\=callout_capture Callout 1: last capture = 1 @@ -11926,7 +11926,7 @@ Callout 2: last capture = 0 0: ab ab\=ovector=1 0: ab - + /(?<=123)(*MARK:xx)abc/mark xxxx123a\=ph Partial match, mark=xx: 123a @@ -11934,7 +11934,7 @@ Partial match, mark=xx: 123a xxxx123a\=ps Partial match, mark=xx: 123a <<< - + /123\Kabc/startchar xxxx123a\=ph Partial match: 123a @@ -11972,18 +11972,18 @@ Partial match: 123a /aaaaa(*COMMIT)(*PRUNE)b|a+c/ aaaaaac 0: aaaac - + # Here are some that Perl treats differently because of the way it handles -# backtracking verbs. +# backtracking verbs. /(?!a(*COMMIT)b)ac|ad/ ac 0: ac - ad + ad 0: ad /^(?!a(*THEN)b|ac)../ - ad + ad 0: ad \= Expect no match ac @@ -11992,7 +11992,7 @@ No match /^(?=a(*THEN)b|ac)/ ac 0: - + /\A.*?(?:a|b(*THEN)c)/ ba 0: ba @@ -12006,7 +12006,7 @@ No match 0: ba /(?:(a(*MARK:X)a+(*SKIP:X)b)){0}(?:(?1)|aac)/ - aac + aac 0: aac /\A.*?(a|b(*THEN)c)/ @@ -12015,10 +12015,10 @@ No match 1: a /^(A(*THEN)B|A(*THEN)D)/ - AD + AD 0: AD 1: AD - + /(?!b(*THEN)a)bn|bnn/ bnn 0: bn @@ -12031,7 +12031,7 @@ No match bnn 0: bn -# This test causes a segfault with Perl 5.18.0 +# This test causes a segfault with Perl 5.18.0 /^(?=(a)){0}b(?1)/ backgammon @@ -13162,7 +13162,7 @@ Starting code units: a b c d Last code unit = 'd' Subject length lower bound = 1 -# End of special auto-possessive tests +# End of special auto-possessive tests /^A\o{1239}B/ Failed: error 164 at offset 8: non-octal character in \o{} (closing brace missing?) @@ -13170,7 +13170,7 @@ Failed: error 164 at offset 8: non-octal character in \o{} (closing brace missin /^A\oB/ Failed: error 155 at offset 4: missing opening brace after \o - + /^A\x{zz}B/ Failed: error 167 at offset 5: non-hex character in \x{} (closing brace missing?) @@ -13332,18 +13332,18 @@ Failed: error 144 at offset 5: group name must start with a non-digit ------------------------------------------------------------------ little red riding hood 0: red - a /red/ thing + a /red/ thing 0: red red is a colour 0: red - put it all on red + put it all on red 0: red \= Expect no match no reduction No match Alfred Winifred No match - + /[a[:<:]] should give error/ Failed: error 130 at offset 4: unknown POSIX class name @@ -13357,7 +13357,7 @@ Start of matched string is beyond its end - displaying from end to start. \= Expect no match xx\nxabcd No match - + # Test stack guard external calls. /(((a)))/stackguard=1 @@ -13411,9 +13411,9 @@ Failed: error 115 at offset 2: reference to non-existent subpattern /A\9B/ Failed: error 115 at offset 2: reference to non-existent subpattern -# This one is here because Perl fails to match "12" for this pattern when the $ +# This one is here because Perl fails to match "12" for this pattern when the $ # is present. - + /^(?(?=abc)\w{3}:|\d\d)$/ abc: 0: abc: @@ -13422,10 +13422,10 @@ Failed: error 115 at offset 2: reference to non-existent subpattern \= Expect no match 123 No match - xyz + xyz No match -# Perl gets this one wrong, giving "a" as the after text for ca and failing to +# Perl gets this one wrong, giving "a" as the after text for ca and failing to # match for cd. /(?(?=ab)ab)/aftertext @@ -13435,11 +13435,11 @@ No match ca 0: 0+ ca - cd + cd 0: 0+ cd - -# This should test both paths for processing OP_RECURSE. + +# This should test both paths for processing OP_RECURSE. /(?(R)a+|(?R)b)/ aaaabcde @@ -13456,14 +13456,14 @@ No match 0: a ba 0: b - cb + cb 0: b /(*NOTEMPTY_ATSTART)a*?b*?/aftertext ab 0: a 0+ b - cdab + cdab 0: 0+ dab @@ -13472,7 +13472,7 @@ Capturing subpattern count = 0 Subject length lower bound = 2 yesno 0: yes - + /(?(VERSION=8)yes){3}/BI,aftertext ------------------------------------------------------------------ Bra @@ -13496,7 +13496,7 @@ Subject length lower bound = 6 yesnononoyes 0: nonono \= Expect no match - yesno + yesno No match /(?:(?abc)|xyz)(?(VERSION)yes|no)/I @@ -13514,7 +13514,7 @@ Subject length lower bound = 5 \= Expect no match abcno No match - xyzyes + xyzyes No match /(?(VERSION<10)yes|no)/ @@ -13548,7 +13548,7 @@ Subject length lower bound = 1 abd 0: abd 1: ab - xyd + xyd 0: d /(|ab)*?d/I,no_start_optimize @@ -13558,7 +13558,7 @@ Subject length lower bound = 0 abd 0: abd 1: ab - xyd + xyd 0: d /\k*(?aa)(?bb)/match_unset_backref,dupnames @@ -13645,7 +13645,7 @@ Failed: error -58 at offset 4 in replacement: expected closing curly bracket in /abc/replace=[9]XYZ 123abc123 Failed: error -48: no more memory - + /abc/replace=xyz 1abc2\=partial_hard Failed: error -34: bad option value @@ -13663,29 +13663,29 @@ Failed: error -34: bad option value /(?<=abc)(|def)/g,replace=<$0> 123abcxyzabcdef789abcpqr 4: 123abc<>xyzabc<>789abc<>pqr - + /./replace=$0 a 1: a - + /(.)(.)/replace=$2+$1 abc 1: b+ac - + /(?.)(?.)/replace=$B+$A abc 1: b+ac - + /(.)(.)/g,replace=$2$1 - abcdefgh + abcdefgh 4: badcfehg - + /(*:pear)apple|(*:orange)lemon|(*:strawberry)blackberry/g,replace=${*MARK} apple lemon blackberry 3: pear orange strawberry apple strudel 1: pear strudel - fruitless + fruitless 0: fruitless /(*:pear)apple|(*:orange)lemon|(*:strawberry)blackberry/replace=${*MARK} sauce, @@ -13697,10 +13697,10 @@ Failed: error -34: bad option value 3: apple strudel 1: strudel - fruitless + fruitless 0: fruitless - -/(*:pear)apple/g,replace=${*MARKING} + +/(*:pear)apple/g,replace=${*MARKING} apple lemon blackberry Failed: error -35 at offset 11 in replacement: invalid replacement string @@ -13708,7 +13708,7 @@ Failed: error -35 at offset 11 in replacement: invalid replacement string apple lemon blackberry Failed: error -58 at offset 7 in replacement: expected closing curly bracket in replacement string -/(*:pear)apple/g,replace=${*mark} +/(*:pear)apple/g,replace=${*mark} apple lemon blackberry Failed: error -35 at offset 8 in replacement: invalid replacement string @@ -13772,13 +13772,13 @@ Get substring 3 failed (-54): requested value is not available Get substring 4 failed (-49): unknown substring 0L c 1L - + /x(?=ab\K)/ - xab\=get=0 + xab\=get=0 Start of matched string is beyond its end - displaying from end to start. 0: ab 0G (0) - xab\=copy=0 + xab\=copy=0 Start of matched string is beyond its end - displaying from end to start. 0: ab 0C (0) @@ -13944,7 +13944,7 @@ Failed: error 109 at offset 7: quantifier does not follow a repeatable item 456 0: 456 \= Expect no match - 356 + 356 No match '^(a)*+(\w)' @@ -13952,7 +13952,7 @@ No match 0: g 1: 2: g - g\=ovector=1 + g\=ovector=1 Matched, but too many substrings 0: g @@ -13960,10 +13960,10 @@ Matched, but too many substrings g 0: g 1: g - g\=ovector=1 + g\=ovector=1 Matched, but too many substrings 0: g - + # These two pattern showeds up compile-time bugs "((?2){0,1999}())?" @@ -14135,7 +14135,7 @@ Callout 25 (?= --->abcdefg 25 ^ (?= 0: abcd - xyz123 + xyz123 --->xyz123 25 ^ (?= 0: xyz @@ -14161,7 +14161,7 @@ Callout (7): $abc$ --->abcdefg ^ (?= 0: abcd - xyz123 + xyz123 Callout (7): $abc$ --->xyz123 ^ (?= @@ -14205,8 +14205,8 @@ Callout (5): 'x\x00z' /(?(?!)a|b)/ bbb 0: b -\= Expect no match - aaa +\= Expect no match + aaa No match # JIT gives a different error message for the infinite recursion @@ -14308,9 +14308,9 @@ Subject length lower bound = 0 \= Expect no match \[9x!xxx(]{9999} No match - + /(abc)*/ - \[abc]{5} + \[abc]{5} 0: abcabcabcabcabc 1: abc @@ -14356,7 +14356,7 @@ Failed: error 115 at offset 2: reference to non-existent subpattern /A\8B\9C/ Failed: error 115 at offset 2: reference to non-existent subpattern A8B9C - + /(?x:((?'a')) # comment (with parentheses) and | vertical (?-x:#not a comment (?'b')) # this is a comment () (?'c')) # not a comment (?'d')/info @@ -14384,14 +14384,14 @@ Subject length lower bound = 1 1: 2: 2 3: - B32A + B32A 0: 3 1: 2: 3: 3 # These are some patterns that used to cause buffer overflows or other errors -# while compiling. +# while compiling. /.((?2)(?R)|\1|$)()/B ------------------------------------------------------------------ @@ -14623,7 +14623,7 @@ Subject length lower bound = 0 0: {4,5a}bc /\x0{ab}/ - \0{ab} + \0{ab} 0: \x00{ab} /^(a(b))\1\g1\g{1}\g-1\g{-1}\g{-02}Z/ @@ -14683,9 +14683,9 @@ No match aacb No match -/(*MARK:a\zb)z/alt_verbnames +/(*MARK:a\zb)z/alt_verbnames Failed: error 140 at offset 10: invalid escape sequence in (*VERB) name - + /(*:ab\t(d\)c)xxx/ Failed: error 122 at offset 12: unmatched closing parenthesis @@ -14698,38 +14698,38 @@ MK: ab\x09(d)c x 0: x MK: Axx)xB - + /(*:A\ExxxB)x/alt_verbnames,mark - x + x 0: x MK: AxxxB - + /(*: A \ and #comment \ B)x/x,alt_verbnames,mark - x + x 0: x MK: A and B - + /(*: A \ and #comment \ B)x/alt_verbnames,mark - x + x 0: x MK: A and #comment\x0a B - + /(*: A \ and #comment \ B)x/x,mark - x + x 0: x MK: A \ and #comment\x0a \ B - + /(*: A \ and #comment \ B)x/mark - x + x 0: x MK: A \ and #comment\x0a \ B - + /(*:A -B)x/alt_verbnames,mark +B)x/alt_verbnames,mark x 0: x MK: A\x0aB @@ -14758,7 +14758,7 @@ No match \= Expect no match 1234abc\=offset_limit=6 No match - + /A/g,replace=-,use_offset_limit XAXAXAXAXA\=offset_limit=4 2: X-X-XAXAXA @@ -14777,20 +14777,20 @@ No match /abcd/null_context abcd\=null_context 0: abcd -\= Expect error +\= Expect error abcd\=null_context,find_limits ** Not allowed together: find_limits null_context - abcd\=allusedtext,startchar + abcd\=allusedtext,startchar ** Not allowed together: allusedtext startchar /abcd/replace=w\rx\x82y\o{333}z(\Q12\$34$$\x34\E5$$),substitute_extended abcd 1: w\x0dx\x82y\xdbz(12\$34$$\x345$) - + /a(bc)(DE)/replace=a\u$1\U$1\E$1\l$2\L$2\Eab\Uab\LYZ\EDone,substitute_extended abcDE 1: aBcBCbcdEdeabAByzDone - + /abcd/replace=xy\kz,substitute_extended abcd Failed: error -57 at offset 4 in replacement: bad escape sequence in replacement string @@ -14844,9 +14844,9 @@ Failed: error -49 at offset 10 in replacement: unknown substring /(?J)(?:(?a)|(?b))/replace=<$A> [a] 1: [] - [b] + [b] 1: [] -\= Expect error +\= Expect error (a)\=ovector=1 Failed: error -54 at offset 3 in replacement: requested value is not available @@ -14890,11 +14890,11 @@ Subject length lower bound = 1 /(?=a\K)/replace=z BaCaD Failed: error -60: match with end before start is not supported - + /(?'abcdefghijklmnopqrstuvwxyzABCDEFG'toolong)/ Failed: error 148 at offset 36: subpattern name is too long (maximum 32 characters) - -/(?'abcdefghijklmnopqrstuvwxyzABCDEF'justright)/ + +/(?'abcdefghijklmnopqrstuvwxyzABCDEF'justright)/ # These two use zero-termination /abcd/max_pattern_length=3 @@ -15246,7 +15246,7 @@ Failed: error 162 at offset 49: subpattern name expected /a|(b)c/replace=>$1<,substitute_unset_empty cat 1: c>b$X<,substitute_unset_empty cat 1: c>b$Y<,substitute_unset_empty cat Failed: error -49 at offset 3 in replacement: unknown substring - cat\=substitute_unknown_unset + cat\=substitute_unknown_unset 1: c>$2<,substitute_unset_empty cat Failed: error -49 at offset 3 in replacement: unknown substring - cat\=substitute_unknown_unset + cat\=substitute_unknown_unset 1: c>[ab])...(?<=\k'A')z/ a11az 0: a11az 1: a - b11bz + b11bz 0: b11bz 1: b \= Expect no match - b11az + b11az No match /(?[ab])...(?<=\k'A')(?)z/dupnames @@ -15439,8 +15439,8 @@ Failed: error 125 at offset 13: lookbehind assertion is not fixed length 1 ^ ^ c +8 ^ ^ 0: abc - -# Perl accepts these, but gives a warning. We can't warn, so give an error. + +# Perl accepts these, but gives a warning. We can't warn, so give an error. /[a-[:digit:]]+/ Failed: error 150 at offset 4: invalid range in character class @@ -15567,7 +15567,7 @@ Contains explicit CR or LF match Subject length lower bound = 1 # This checks that new code for handling groups that may match an empty string -# works on a very large number of alternatives. This pattern used to provoke a +# works on a very large number of alternatives. This pattern used to provoke a # complaint that it was too complicated. /(?:\[A|B|C|D|E|F|G|H|I|J|]{200}Z)/expand @@ -15630,14 +15630,14 @@ Subject length lower bound = 11 // \=ovector=7777777777 ** Invalid value in 'ovector=7777777777' - -# This is here because Perl matches, even though a COMMIT is encountered -# outside of the recursion. + +# This is here because Perl matches, even though a COMMIT is encountered +# outside of the recursion. /(?1)(A(*COMMIT)|B)D/ BAXBAD No match - + "(?1){2}(a)"B ------------------------------------------------------------------ Bra @@ -15673,22 +15673,22 @@ No match ------------------------------------------------------------------ # This test differs from Perl for the first subject. Perl ends up with -# $1 set to 'B'; PCRE2 has it unset (which I think is right). +# $1 set to 'B'; PCRE2 has it unset (which I think is right). /^(?: -(?:A| (?:B|B(*ACCEPT)) (?<=(.)) D) +(?:A| (?:B|B(*ACCEPT)) (?<=(.)) D) (Z) )+$/x AZB 0: AZB 1: 2: Z - AZBDZ + AZBDZ 0: AZBDZ 1: B 2: Z - -# The first of these, when run by Perl, gives the mark 'aa', which is wrong. + +# The first of these, when run by Perl, gives the mark 'aa', which is wrong. '(?>a(*:aa))b|ac' mark ac @@ -15697,7 +15697,7 @@ No match '(?:a(*:aa))b|ac' mark ac 0: ac - + /(R?){65}/ (R?){65} 0: @@ -15735,7 +15735,7 @@ Callout 1: last capture = 1 \= Expect no match abc No match - + # Perl gives no match for this one /(a(*MARK:m)(*ACCEPT)){0}(?1)/mark @@ -15838,7 +15838,7 @@ No match \= Expect no match bbb No match - cc + cc No match /^X\S/no_start_optimize,no_auto_possess @@ -15910,7 +15910,7 @@ No match \= Expect no match X No match - X\n + X\n No match /^X\D+?/no_start_optimize,no_auto_possess @@ -15924,21 +15924,21 @@ No match \= Expect no match X No match - X\n + X\n No match /^X\W+?/no_start_optimize,no_auto_possess \= Expect no match X No match - XX + XX No match /^X.+?Z/no_start_optimize,no_auto_possess \= Expect no match XY\n No match - + /(*CRLF)^X.+?Z/no_start_optimize,no_auto_possess \= Expect no match XY\r\=ps @@ -15954,9 +15954,9 @@ No match No match X\n\nY No match - X\n\x{0c}Y + X\n\x{0c}Y No match - + /(*BSR_ANYCRLF)^X\R+?Z/no_start_optimize,no_auto_possess \= Expect no match X\nX @@ -15967,14 +15967,14 @@ No match No match X\n\nY No match - X\n\x{0c}Y + X\n\x{0c}Y No match - + /^X\H+?Z/no_start_optimize,no_auto_possess \= Expect no match XY\t No match - XYY + XYY No match /^X\h+?Z/no_start_optimize,no_auto_possess @@ -15988,7 +15988,7 @@ No match \= Expect no match XY\n No match - XYY + XYY No match /^X\v+?Z/no_start_optimize,no_auto_possess @@ -16002,7 +16002,7 @@ No match \= Expect no match XY9 No match - XYY + XYY No match /^X\d+?Z/no_start_optimize,no_auto_possess @@ -16016,7 +16016,7 @@ No match \= Expect no match XY\n No match - XYY + XYY No match /^X\s+?Z/no_start_optimize,no_auto_possess @@ -16030,14 +16030,14 @@ No match \= Expect no match X.A No match - X++ + X++ No match /^X\w+?Z/no_start_optimize,no_auto_possess \= Expect no match Xa. No match - Xaa + Xaa No match /^X.{1,3}Z/s,no_start_optimize,no_auto_possess @@ -16056,14 +16056,14 @@ No match \= Expect no match XY\n No match - XYY + XYY No match /^(X(*THEN)Y|AB){0}(?1)/ ABX 0: AB \= Expect no match - XAB + XAB No match /^(?!A(?C1)B)C/ @@ -16272,7 +16272,7 @@ Failed: error 192 at offset 0: invalid option bits with PCRE2_LITERAL /cat|dog/match_word the cat sat 0: cat -\= Expect no match +\= Expect no match caterpillar No match snowcat @@ -16283,7 +16283,7 @@ No match /(cat)|dog/match_line,literal (cat)|dog 0: (cat)|dog -\= Expect no match +\= Expect no match the cat sat No match caterpillar @@ -16297,10 +16297,35 @@ No match Rhubarb \na whole line\n custard 0: a whole line \= Expect no match - Not a whole line + Not a whole line No match -# End of testinput2 +# Perl gets this wrong, failing to capture 'b' in group 1. + +/^(b+|a){1,2}?bc/ + bbc + 0: bbc + 1: b + +# And again here, for the "babc" subject string. + +/^(b*|ba){1,2}?bc/ + babc + 0: babc + 1: ba + bbabc + 0: bbabc + 1: ba + bababc + 0: bababc + 1: ba +\= Expect no match + bababbc +No match + babababc +No match + +# End of testinput2 Error -65: PCRE2_ERROR_BADDATA (unknown error number) Error -62: bad serialized data Error -2: partial match diff --git a/testdata/testoutput8-32-3 b/testdata/testoutput8-32-3 index 83e3086..30667a3 100644 --- a/testdata/testoutput8-32-3 +++ b/testdata/testoutput8-32-3 @@ -853,10 +853,8 @@ Memory allocation (code space): 28 # with link size - hence multiple tests with different values. /(?'ABC'\[[bar](]{792}*THEN:\[A]{255}\[)]{793}/expand,-fullbincode,parens_nest_limit=1000 -Failed: error 186 at offset 5813: regular expression is too complicated /(?'ABC'\[[bar](]{793}*THEN:\[A]{255}\[)]{794}/expand,-fullbincode,parens_nest_limit=1000 -Failed: error 186 at offset 5820: regular expression is too complicated /(?'ABC'\[[bar](]{1793}*THEN:\[A]{255}\[)]{1794}/expand,-fullbincode,parens_nest_limit=2000 Failed: error 186 at offset 12820: regular expression is too complicated diff --git a/testdata/testoutput8-32-4 b/testdata/testoutput8-32-4 index 83e3086..30667a3 100644 --- a/testdata/testoutput8-32-4 +++ b/testdata/testoutput8-32-4 @@ -853,10 +853,8 @@ Memory allocation (code space): 28 # with link size - hence multiple tests with different values. /(?'ABC'\[[bar](]{792}*THEN:\[A]{255}\[)]{793}/expand,-fullbincode,parens_nest_limit=1000 -Failed: error 186 at offset 5813: regular expression is too complicated /(?'ABC'\[[bar](]{793}*THEN:\[A]{255}\[)]{794}/expand,-fullbincode,parens_nest_limit=1000 -Failed: error 186 at offset 5820: regular expression is too complicated /(?'ABC'\[[bar](]{1793}*THEN:\[A]{255}\[)]{1794}/expand,-fullbincode,parens_nest_limit=2000 Failed: error 186 at offset 12820: regular expression is too complicated