There is a complete description of the PCRE2 native API in the diff --git a/doc/html/pcre2_code_copy_with_tables.html b/doc/html/pcre2_code_copy_with_tables.html new file mode 100644 index 0000000..99cb022 --- /dev/null +++ b/doc/html/pcre2_code_copy_with_tables.html @@ -0,0 +1,44 @@ + +
++Return to the PCRE2 index page. +
+
+This page is part of the PCRE2 HTML documentation. It was generated
+automatically from the original man page. If there is any nonsense in it,
+please consult the man page, in case the conversion went wrong.
+
+
+SYNOPSIS
+
+
+#include <pcre2.h> +
++pcre2_code *pcre2_code_copy_with_tables(const pcre2_code *code); +
++This function makes a copy of the memory used for a compiled pattern, excluding +any memory used by the JIT compiler. Without a subsequent call to +pcre2_jit_compile(), the copy can be used only for non-JIT matching. +Unlike pcre2_code_copy(), a separate copy of the character tables is also +made, with the new code pointing to it. This memory will be automatically freed +when pcre2_code_free() is called. The yield of the function is NULL if +code is NULL or if sufficient memory cannot be obtained. +
++There is a complete description of the PCRE2 native API in the +pcre2api +page and a description of the POSIX API in the +pcre2posix +page. +
+Return to the PCRE2 index page. +
diff --git a/doc/html/pcre2_set_max_pattern_length.html b/doc/html/pcre2_set_max_pattern_length.html index 055d1ec..f6e422a 100644 --- a/doc/html/pcre2_set_max_pattern_length.html +++ b/doc/html/pcre2_set_max_pattern_length.html @@ -26,8 +26,11 @@ SYNOPSIS DESCRIPTION-This function sets, in a compile context, the maximum length (in code units) of -the pattern that can be compiled. The result is always zero. +This function sets, in a compile context, the maximum text length (in code +units) of the pattern that can be compiled. The result is always zero. If a +longer pattern is passed to pcre2_compile() there is an immediate error +return. The default is effectively unlimited, being the largest value a +PCRE2_SIZE variable can hold.
There is a complete description of the PCRE2 native API in the
diff --git a/doc/html/pcre2api.html b/doc/html/pcre2api.html
index fa9f342..629f1d0 100644
--- a/doc/html/pcre2api.html
+++ b/doc/html/pcre2api.html
@@ -294,6 +294,9 @@ document for an overview of all the PCRE2 documentation.
pcre2_code *pcre2_code_copy(const pcre2_code *code);
+pcre2_code *pcre2_code_copy_with_tables(const pcre2_code *code);
+
+
int pcre2_get_error_message(int errorcode, PCRE2_UCHAR *buffer,
PCRE2_SIZE bufflen);
@@ -567,8 +570,9 @@ If JIT is being used, but the JIT compilation is not being done immediately,
(perhaps waiting to see if the pattern is used often enough) similar logic is
required. JIT compilation updates a pointer within the compiled code block, so
a thread must gain unique write access to the pointer before calling
-pcre2_jit_compile(). Alternatively, pcre2_code_copy() can be used
-to obtain a private copy of the compiled code.
+pcre2_jit_compile(). Alternatively, pcre2_code_copy() or
+pcre2_code_copy_with_tables() can be used to obtain a private copy of the
+compiled code.
The pcre2_compile() function compiles a pattern into an internal form. @@ -1079,9 +1087,22 @@ if the code has been processed by the JIT compiler (see below), the JIT information cannot be copied (because it is position-dependent). The new copy can initially be used only for non-JIT matching, though it can be -passed to pcre2_jit_compile() if required. The pcre2_code_copy() -function provides a way for individual threads in a multithreaded application -to acquire a private copy of shared compiled code. +passed to pcre2_jit_compile() if required. +
++The pcre2_code_copy() function provides a way for individual threads in a +multithreaded application to acquire a private copy of shared compiled code. +However, it does not make a copy of the character tables used by the compiled +pattern; the new pattern code points to the same tables as the original code. +(See +"Locale Support" +below for details of these character tables.) In many applications the same +tables are used throughout, so this behaviour is appropriate. Nevertheless, +there are occasions when a copy of a compiled pattern and the relevant tables +are needed. The pcre2_code_copy_with_tables() provides this facility. +Copies of both the code and the tables are made, with the new code pointing to +the new tables. The memory for the new tables is automatically freed when +pcre2_code_free() is called for the new copy of the compiled code.
NOTE: When one of the matching functions is called, pointers to the compiled @@ -1119,7 +1140,14 @@ NULL immediately. Otherwise, the variables to which these point are set to an error code and an offset (number of code units) within the pattern, respectively, when pcre2_compile() returns NULL because a compilation error has occurred. The values are not defined when compilation is successful -and pcre2_compile() returns a non-NULL value. +and pcre2_compile() returns a non-NULL value. +
++The value returned in erroroffset is an indication of where in the +pattern the error occurred. It is not necessarily the furthest point in the +pattern that was read. For example, after the error "lookbehind assertion is +not fixed length", the error offset points to the start of the failing +assertion.
The pcre2_get_error_message() function (see "Obtaining a textual error @@ -1215,8 +1243,8 @@ recognized, exactly as in the rest of the pattern. PCRE2_AUTO_CALLOUT If this bit is set, pcre2_compile() automatically inserts callout items, -all with number 255, before each pattern item. For discussion of the callout -facility, see the +all with number 255, before each pattern item, except immediately before or +after a callout in the pattern. For discussion of the callout facility, see the pcre2callout documentation.
@@ -3235,7 +3263,7 @@ Cambridge, England.
REVISION
-Last updated: 17 June 2016 +Last updated: 22 November 2016
Copyright © 1997-2016 University of Cambridge.
diff --git a/doc/html/pcre2build.html b/doc/html/pcre2build.html index 6c8e1de..13ae9cb 100644 --- a/doc/html/pcre2build.html +++ b/doc/html/pcre2build.html @@ -34,9 +34,10 @@ please consult the man page, in case the conversion went wrong.
@@ -376,16 +377,19 @@ they are not.
pcre2grep uses an internal buffer to hold a "window" on the file it is scanning, in order to be able to output "before" and "after" lines when it -finds a match. The size of the buffer is controlled by a parameter whose -default value is 20K. The buffer itself is three times this size, but because -of the way it is used for holding "before" lines, the longest line that is -guaranteed to be processable is the parameter size. You can change the default -parameter value by adding, for example, +finds a match. The starting size of the buffer is controlled by a parameter +whose default value is 20K. The buffer itself is three times this size, but +because of the way it is used for holding "before" lines, the longest line that +is guaranteed to be processable is the parameter size. If a longer line is +encountered, pcre2grep automatically expands the buffer, up to a +specified maximum size, whose default is 1M or the starting size, whichever is +the larger. You can change the default parameter values by adding, for example,
- --with-pcre2grep-bufsize=50K + --with-pcre2grep-bufsize=51200 + --with-pcre2grep-max-bufsize=2097152-to the configure command. The caller of \fPpcre2grep\fP can override this -value by using --buffer-size on the command line. +to the configure command. The caller of \fPpcre2grep\fP can override +these values by using --buffer-size and --max-buffer-size on the command line.
@@ -497,11 +501,32 @@ This cleans all coverage data including the generated coverage report. For more information about code coverage, see the gcov and lcov documentation.
-+There is a special option for use by people who want to run fuzzing tests on +PCRE2: +
+ --enable-fuzz-support ++At present this applies only to the 8-bit library. If set, it causes an extra +library called libpcre2-fuzzsupport.a to be built, but not installed. This +contains a single function called LLVMFuzzerTestOneInput() whose arguments are +a pointer to a string and the length of the string. When called, this function +tries to compile the string as a pattern, and if that succeeds, to match it. +This is done both with no options and with some random options bits that are +generated from the string. Setting --enable-fuzz-support also causes a binary +called pcre2fuzzcheck to be created. This is normally run under valgrind +or used when PCRE2 is compiled with address sanitizing enabled. It calls the +fuzzing function and outputs information about it is doing. The input strings +are specified by arguments: if an argument starts with "=" the rest of it is a +literal input string. Otherwise, it is assumed to be a file name, and the +contents of the file are the test string. + +
pcre2api(3), pcre2-config(3).
-
Philip Hazel
@@ -510,9 +535,9 @@ University Computing Service
Cambridge, England.
-Last updated: 01 April 2016
+Last updated: 01 November 2016
Copyright © 1997-2016 University of Cambridge.
diff --git a/doc/html/pcre2callout.html b/doc/html/pcre2callout.html
index 7e85c9a..8a4fa6c 100644
--- a/doc/html/pcre2callout.html
+++ b/doc/html/pcre2callout.html
@@ -57,11 +57,20 @@ two callout points:
If the PCRE2_AUTO_CALLOUT option bit is set when a pattern is compiled, PCRE2
automatically inserts callouts, all with number 255, before each item in the
-pattern. For example, if PCRE2_AUTO_CALLOUT is used with the pattern
+pattern except for immediately before or after a callout item in the pattern.
+For example, if PCRE2_AUTO_CALLOUT is used with the pattern
+
+ A(?C3)B ++it is processed as if it were +
+ (?C255)A(?C3)B(?C255) ++Here is a more complicated example:
A(\d{2}|--)
-it is processed as if it were
+With PCRE2_AUTO_CALLOUT, this pattern is processed as if it were
--->aaaa
+0 ^ a+
@@ -235,8 +244,8 @@ Fields for numerical callouts
For a numerical callout, callout_string is NULL, and callout_number
contains the number of the callout, in the range 0-255. This is the number
-that follows (?C for manual callouts; it is 255 for automatically generated
-callouts.
+that follows (?C for callouts that part of the pattern; it is 255 for
+automatically generated callouts.
Fields for string callouts
@@ -310,10 +319,15 @@ the next item to be matched.
The next_item_length field contains the length of the next item to be
-matched in the pattern string. When the callout immediately precedes an
-alternation bar, a closing parenthesis, or the end of the pattern, the length
-is zero. When the callout precedes an opening parenthesis, the length is that
-of the entire subpattern.
+processed in the pattern string. When the callout is at the end of the pattern,
+the length is zero. When the callout precedes an opening parenthesis, the
+length includes meta characters that follow the parenthesis. For example, in a
+callout before an assertion such as (?=ab) the length is 3. For an an
+alternation bar or a closing parenthesis, the length is one, unless a closing
+parenthesis is followed by a quantifier, in which case its length is included.
+(This changed in release 10.23. In earlier releases, before an opening
+parenthesis the length was that of the entire subpattern, and before an
+alternation bar or a closing parenthesis the length was zero.)
The pattern_position and next_item_length fields are intended to
@@ -399,9 +413,9 @@ Cambridge, England.
REVISION
-Last updated: 23 March 2015
+Last updated: 29 September 2016
-Copyright © 1997-2015 University of Cambridge.
+Copyright © 1997-2016 University of Cambridge.
Return to the PCRE2 index page.
diff --git a/doc/html/pcre2compat.html b/doc/html/pcre2compat.html
index 3b29e6f..361eb3b 100644
--- a/doc/html/pcre2compat.html
+++ b/doc/html/pcre2compat.html
@@ -107,7 +107,7 @@ processed as anchored at the point where they are tested.
one that is backtracked onto acts. For example, in the pattern
A(*COMMIT)B(*PRUNE)C a failure in B triggers (*COMMIT), but a failure in C
triggers (*PRUNE). Perl's behaviour is more complex; in many cases it is the
-same as PCRE2, but there are examples where it differs.
+same as PCRE2, but there are cases where it differs.
11. Most backtracking verbs in assertions have their normal actions. They are
@@ -123,7 +123,7 @@ the pattern /^(a(b)?)+$/ in Perl leaves $2 unset, but in PCRE2 it is set to
13. PCRE2's handling of duplicate subpattern numbers and duplicate subpattern
names is not as general as Perl's. This is a consequence of the fact the PCRE2
works internally just with numbers, using an external table to translate
-between numbers and names. In particular, a pattern such as (?|(?<a>A)|(?<b)B),
+between numbers and names. In particular, a pattern such as (?|(?<a>A)|(?<b>B),
where the two capturing parentheses have the same number but different names,
is not supported, and causes an error at compile time. If it were allowed, it
would not be possible to distinguish which parentheses matched, because both
@@ -131,10 +131,11 @@ names map to capturing subpattern number 1. To avoid this confusing situation,
an error is given at compile time.
-14. Perl recognizes comments in some places that PCRE2 does not, for example,
-between the ( and ? at the start of a subpattern. If the /x modifier is set,
-Perl allows white space between ( and ? (though current Perls warn that this is
-deprecated) but PCRE2 never does, even if the PCRE2_EXTENDED option is set.
+14. Perl used to recognize comments in some places that PCRE2 does not, for
+example, between the ( and ? at the start of a subpattern. If the /x modifier
+is set, Perl allowed white space between ( and ? though the latest Perls give
+an error (for a while it was just deprecated). There may still be some cases
+where Perl behaves differently.
15. Perl, when in warning mode, gives warnings for character classes such as
@@ -158,45 +159,50 @@ list is with respect to Perl 5.10:
(a) Although lookbehind assertions in PCRE2 must match fixed length strings,
each alternative branch of a lookbehind assertion can match a different length
-of string. Perl requires them all to have the same length.
+of string. Perl requires them all to have the same length.
-(b) If PCRE2_DOLLAR_ENDONLY is set and PCRE2_MULTILINE is not set, the $
+(b) From PCRE2 10.23, back references to groups of fixed length are supported
+in lookbehinds, provided that there is no possibility of referencing a
+non-unique number or name. Perl does not support backreferences in lookbehinds.
+
+
+(c) If PCRE2_DOLLAR_ENDONLY is set and PCRE2_MULTILINE is not set, the $
meta-character matches only at the very end of the string.
-(c) A backslash followed by a letter with no special meaning is faulted. (Perl
+(d) A backslash followed by a letter with no special meaning is faulted. (Perl
can be made to issue a warning.)
-(d) If PCRE2_UNGREEDY is set, the greediness of the repetition quantifiers is
+(e) If PCRE2_UNGREEDY is set, the greediness of the repetition quantifiers is
inverted, that is, by default they are not greedy, but if followed by a
question mark they are.
-(e) PCRE2_ANCHORED can be used at matching time to force a pattern to be tried
+(f) PCRE2_ANCHORED can be used at matching time to force a pattern to be tried
only at the first matching position in the subject string.
-(f) The PCRE2_NOTBOL, PCRE2_NOTEOL, PCRE2_NOTEMPTY, PCRE2_NOTEMPTY_ATSTART, and
+(g) The PCRE2_NOTBOL, PCRE2_NOTEOL, PCRE2_NOTEMPTY, PCRE2_NOTEMPTY_ATSTART, and
PCRE2_NO_AUTO_CAPTURE options have no Perl equivalents.
-(g) The \R escape sequence can be restricted to match only CR, LF, or CRLF
+(h) The \R escape sequence can be restricted to match only CR, LF, or CRLF
by the PCRE2_BSR_ANYCRLF option.
-(h) The callout facility is PCRE2-specific.
+(i) The callout facility is PCRE2-specific.
-(i) The partial matching facility is PCRE2-specific.
+(j) The partial matching facility is PCRE2-specific.
-(j) The alternative matching function (pcre2_dfa_match() matches in a
+(k) The alternative matching function (pcre2_dfa_match() matches in a
different way and is not Perl-compatible.
-(k) PCRE2 recognizes some special sequences such as (*CR) at the start of
+(l) PCRE2 recognizes some special sequences such as (*CR) at the start of
a pattern that set overall options that cannot be changed within the pattern.
@@ -214,9 +220,9 @@ Cambridge, England.
REVISION
-Last updated: 15 March 2015
+Last updated: 18 October 2016
-Copyright © 1997-2015 University of Cambridge.
+Copyright © 1997-2016 University of Cambridge.
Return to the PCRE2 index page.
diff --git a/doc/html/pcre2grep.html b/doc/html/pcre2grep.html
index d02d365..c1a982c 100644
--- a/doc/html/pcre2grep.html
+++ b/doc/html/pcre2grep.html
@@ -80,11 +80,19 @@ span line boundaries. What defines a line boundary is controlled by the
The amount of memory used for buffering files that are being scanned is
-controlled by a parameter that can be set by the --buffer-size option.
-The default value for this parameter is specified when pcre2grep is
-built, with the default default being 20K. A block of memory three times this
-size is used (to allow for buffering "before" and "after" lines). An error
-occurs if a line overflows the buffer.
+controlled by parameters that can be set by the --buffer-size and
+--max-buffer-size options. The first of these sets the size of buffer
+that is obtained at the start of processing. If an input file contains very
+long lines, a larger buffer may be needed; this is handled by automatically
+extending the buffer, up to the limit specified by --max-buffer-size. The
+default values for these parameters are specified when pcre2grep is
+built, with the default defaults being 20K and 1M respectively. An error occurs
+if a line is too long and the buffer can no longer be expanded.
+
+
+The block of memory that is actually used is three times the "buffer size", to
+allow for buffering "before" and "after" lines. If the buffer size is too
+small, fewer than requested "before" and "after" lines may be output.
Patterns can be no longer than 8K or BUFSIZ bytes, whichever is the greater.
@@ -155,12 +163,13 @@ processing of patterns and file names that start with hyphens.
-A number, --after-context=number
-Output number lines of context after each matching line. If file names
-and/or line numbers are being output, a hyphen separator is used instead of a
-colon for the context lines. A line containing "--" is output between each
-group of lines, unless they are in fact contiguous in the input file. The value
-of number is expected to be relatively small. However, pcre2grep
-guarantees to have up to 8K of following text available for context output.
+Output up to number lines of context after each matching line. Fewer
+lines are output if the next match or the end of the file is reached, or if the
+processing buffer size has been set too small. If file names and/or line
+numbers are being output, a hyphen separator is used instead of a colon for the
+context lines. A line containing "--" is output between each group of lines,
+unless they are in fact contiguous in the input file. The value of number
+is expected to be relatively small. When -c is used, -A is ignored.
-a, --text
@@ -169,12 +178,14 @@ Treat binary files as text. This is equivalent to
-B number, --before-context=number
-Output number lines of context before each matching line. If file names
-and/or line numbers are being output, a hyphen separator is used instead of a
-colon for the context lines. A line containing "--" is output between each
-group of lines, unless they are in fact contiguous in the input file. The value
-of number is expected to be relatively small. However, pcre2grep
-guarantees to have up to 8K of preceding text available for context output.
+Output up to number lines of context before each matching line. Fewer
+lines are output if the previous match or the start of the file is within
+number lines, or if the processing buffer size has been set too small. If
+file names and/or line numbers are being output, a hyphen separator is used
+instead of a colon for the context lines. A line containing "--" is output
+between each group of lines, unless they are in fact contiguous in the input
+file. The value of number is expected to be relatively small. When
+-c is used, -B is ignored.
--binary-files=word
@@ -191,8 +202,9 @@ return code.
--buffer-size=number
-Set the parameter that controls how much memory is used for buffering files
-that are being scanned.
+Set the parameter that controls how much memory is obtained at the start of
+processing for buffering files that are being scanned. See also
+--max-buffer-size below.
-C number, --context=number
@@ -202,14 +214,16 @@ This is equivalent to setting both -A and -B to the same value.
-c, --count
Do not output lines from the files that are being scanned; instead output the
-number of matches (or non-matches if -v is used) that would otherwise
-have caused lines to be shown. By default, this count is the same as the number
-of suppressed lines, but if the -M (multiline) option is used (without
--v), there may be more suppressed lines than the number of matches.
+number of lines that would have been shown, either because they matched, or, if
+-v is set, because they failed to match. By default, this count is
+exactly the same as the number of lines that would have been output, but if the
+-M (multiline) option is used (without -v), there may be more
+suppressed lines than the count (that is, the number of matches).
If no lines are selected, the number zero is output. If several files are are
-being scanned, a count is output for each of them. However, if the
+being scanned, a count is output for each of them and the -t option can
+be used to cause a total to be output at the end. However, if the
--files-with-matches option is also used, only those files whose counts
are greater than zero are listed. When -c is used, the -A,
-B, and -C options are ignored.
@@ -232,11 +246,12 @@ just one, in order to colour them all.
The colour that is used can be specified by setting the environment variable
-PCRE2GREP_COLOUR or PCRE2GREP_COLOR. The value of this variable should be a
-string of two numbers, separated by a semicolon. They are copied directly into
-the control string for setting colour on a terminal, so it is your
-responsibility to ensure that they make sense. If neither of the environment
-variables is set, the default is "1;31", which gives red.
+PCRE2GREP_COLOUR or PCRE2GREP_COLOR. If neither of these are set,
+pcre2grep looks for GREP_COLOUR or GREP_COLOR. The value of the variable
+should be a string of two numbers, separated by a semicolon. They are copied
+directly into the control string for setting colour on a terminal, so it is
+your responsibility to ensure that they make sense. If neither of the
+environment variables is set, the default is "1;31", which gives red.
-D action, --devices=action
@@ -321,24 +336,24 @@ files; it does not apply to patterns specified by any of the --include or
-f filename, --file=filename
-Read patterns from the file, one per line, and match them against
-each line of input. What constitutes a newline when reading the file is the
-operating system's default. The --newline option has no effect on this
-option. Trailing white space is removed from each line, and blank lines are
-ignored. An empty file contains no patterns and therefore matches nothing. See
-also the comments about multiple patterns versus a single pattern with
-alternatives in the description of -e above.
+Read patterns from the file, one per line, and match them against each line of
+input. What constitutes a newline when reading the file is the operating
+system's default. The --newline option has no effect on this option.
+Trailing white space is removed from each line, and blank lines are ignored. An
+empty file contains no patterns and therefore matches nothing. See also the
+comments about multiple patterns versus a single pattern with alternatives in
+the description of -e above.
-If this option is given more than once, all the specified files are
-read. A data line is output if any of the patterns match it. A file name can
-be given as "-" to refer to the standard input. When -f is used, patterns
+If this option is given more than once, all the specified files are read. A
+data line is output if any of the patterns match it. A file name can be given
+as "-" to refer to the standard input. When -f is used, patterns
specified on the command line using -e may also be present; they are
tested before the file's patterns. However, no other pattern is taken from the
command line; all arguments are treated as the names of paths to be searched.
---file-list=filename
+--file-list=filename
Read a list of files and/or directories that are to be scanned from the given
file, one per line. Trailing white space is removed from each line, and blank
lines are ignored. These paths are processed before any that are listed on the
@@ -502,22 +517,24 @@ There are no short forms for these options. The default settings are specified
when the PCRE2 library is compiled, with the default default being 10 million.
+\fB--max-buffer-size=number
+This limits the expansion of the processing buffer, whose initial size can be
+set by --buffer-size. The maximum buffer size is silently forced to be no
+smaller than the starting buffer size.
+
+
-M, --multiline
-Allow patterns to match more than one line. When this option is given, patterns
-may usefully contain literal newline characters and internal occurrences of ^
-and $ characters. The output for a successful match may consist of more than
-one line. The first is the line in which the match started, and the last is the
-line in which the match ended. If the matched string ends with a newline
-sequence the output ends at the end of that line.
-
-
-When this option is set, the PCRE2 library is called in "multiline" mode. This
-allows a matched string to extend past the end of a line and continue on one or
-more subsequent lines. However, pcre2grep still processes the input line
-by line. Once a match has been handled, scanning restarts at the beginning of
-the next line, just as it does when -M is not present. This means that it
-is possible for the second or subsequent lines in a multiline match to be
-output again as part of another match.
+Allow patterns to match more than one line. When this option is set, the PCRE2
+library is called in "multiline" mode. This allows a matched string to extend
+past the end of a line and continue on one or more subsequent lines. Patterns
+used with -M may usefully contain literal newline characters and internal
+occurrences of ^ and $ characters. The output for a successful match may
+consist of more than one line. The first line is the line in which the match
+started, and the last line is the line in which the match ended. If the matched
+string ends with a newline sequence, the output ends at the end of that line.
+If -v is set, none of the lines in a multi-line match are output. Once a
+match has been handled, scanning restarts at the beginning of the line after
+the one in which the match ended.
The newline sequence that separates multiple lines must be matched as part of
@@ -533,11 +550,8 @@ well as possibly handling a two-character newline sequence.
There is a limit to the number of lines that can be matched, imposed by the way
-that pcre2grep buffers the input file as it scans it. However,
-pcre2grep ensures that at least 8K characters or the rest of the file
-(whichever is the shorter) are available for forward matching, and similarly
-the previous 8K characters (or all the previous characters, if fewer than 8K)
-are guaranteed to be available for lookbehind assertions. The -M option
+that pcre2grep buffers the input file as it scans it. With a sufficiently
+large processing buffer, this should not be a problem, but the -M option
does not work when input is read line by line (see \fP--line-buffered\fP.)
@@ -585,12 +599,13 @@ It should never be needed in normal use.
Show only the part of the line that matched a pattern instead of the whole
line. In this mode, no context is shown. That is, the -A, -B, and
-C options are ignored. If there is more than one match in a line, each
-of them is shown separately. If -o is combined with -v (invert the
-sense of the match to find non-matching lines), no output is generated, but the
-return code is set appropriately. If the matched portion of the line is empty,
-nothing is output unless the file name or line number are being printed, in
-which case they are shown on an otherwise empty line. This option is mutually
-exclusive with --file-offsets and --line-offsets.
+of them is shown separately, on a separate line of output. If -o is
+combined with -v (invert the sense of the match to find non-matching
+lines), no output is generated, but the return code is set appropriately. If
+the matched portion of the line is empty, nothing is output unless the file
+name or line number are being printed, in which case they are shown on an
+otherwise empty line. This option is mutually exclusive with
+--file-offsets and --line-offsets.
-onumber, --only-matching=number
@@ -604,10 +619,11 @@ capturing parentheses do not exist in the pattern, or were not set in the
match, nothing is output unless the file name or line number are being output.
-If this option is given multiple times, multiple substrings are output, in the
-order the options are given. For example, -o3 -o1 -o3 causes the substrings
-matched by capturing parentheses 3 and 1 and then 3 again to be output. By
-default, there is no separator (but see the next option).
+If this option is given multiple times, multiple substrings are output for each
+match, in the order the options are given, and all on one line. For example,
+-o3 -o1 -o3 causes the substrings matched by capturing parentheses 3 and 1 and
+then 3 again to be output. By default, there is no separator (but see the next
+option).
--om-separator=text
@@ -638,6 +654,18 @@ quietly skipped. However, the return code is still 2, even if matches were
found in other files.
+-t, --total-count
+This option is useful when scanning more than one file. If used on its own,
+-t suppresses all output except for a grand total number of matching
+lines (or non-matching lines if -v is used) in all the files. If -t
+is used with -c, a grand total is output except when the previous output
+is just one line. In other words, it is not output when just one file's count
+is listed. If file names are being output, the grand total is preceded by
+"TOTAL:". Otherwise, it appears as just another number. The -t option is
+ignored when used with -L (list files without matches), because the grand
+total would always be zero.
+
+
-u, --utf-8
Operate in UTF-8 mode. This option is available only if PCRE2 has been compiled
with UTF-8 support. All patterns (including those for any --exclude and
@@ -665,11 +693,12 @@ specified by any of the --include or --exclude options.
-x, --line-regex, --line-regexp
Force the patterns to be anchored (each must start matching at the beginning of
-a line) and in addition, require them to match entire lines. This is equivalent
-to having ^ and $ characters at the start and end of each alternative top-level
-branch in every pattern. This option applies only to the patterns that are
-matched against the contents of files; it does not apply to patterns specified
-by any of the --include or --exclude options.
+a line) and in addition, require them to match entire lines. In multiline mode
+the match may be more than one line. This is equivalent to having \A and \Z
+characters at the start and end of each alternative top-level branch in every
+pattern. This option applies only to the patterns that are matched against the
+contents of files; it does not apply to patterns specified by any of the
+--include or --exclude options.
ENVIRONMENT VARIABLES
@@ -831,7 +860,7 @@ Cambridge, England.
REVISION
-Last updated: 19 June 2016
+Last updated: 31 October 2016
Copyright © 1997-2016 University of Cambridge.
diff --git a/doc/html/pcre2limits.html b/doc/html/pcre2limits.html
index e227a30..c077e6c 100644
--- a/doc/html/pcre2limits.html
+++ b/doc/html/pcre2limits.html
@@ -61,14 +61,10 @@ The maximum length of a lookbehind assertion is 65535 characters.
There is no limit to the number of parenthesized subpatterns, but there can be
no more than 65535 capturing subpatterns. There is, however, a limit to the
depth of nesting of parenthesized subpatterns of all kinds. This is imposed in
-order to limit the amount of system stack used at compile time. The limit can
-be specified when PCRE2 is built; the default is 250.
-
-
-There is a limit to the number of forward references to subsequent subpatterns
-of around 200,000. Repeated forward references with fixed upper limits, for
-example, (?2){0,100} when subpattern number 2 is to the right, are included in
-the count. There is no limit to the number of backward references.
+order to limit the amount of system stack used at compile time. The default
+limit can be specified when PCRE2 is built; the default default is 250. An
+application can change this limit by calling pcre2_set_parens_nest_limit() to
+set the limit in a compile context.
The maximum length of name for a named subpattern is 32 code units, and the
@@ -76,7 +72,12 @@ maximum number of named subpatterns is 10000.
The maximum length of a name in a (*MARK), (*PRUNE), (*SKIP), or (*THEN) verb
-is 255 for the 8-bit library and 65535 for the 16-bit and 32-bit libraries.
+is 255 code units for the 8-bit library and 65535 code units for the 16-bit and
+32-bit libraries.
+
+
+The maximum length of a string argument to a callout is the largest number a
+32-bit unsigned integer can hold.
AUTHOR
@@ -93,9 +94,9 @@ Cambridge, England.
REVISION
-Last updated: 05 November 2015
+Last updated: 26 October 2016
-Copyright © 1997-2015 University of Cambridge.
+Copyright © 1997-2016 University of Cambridge.
Return to the PCRE2 index page.
diff --git a/doc/html/pcre2pattern.html b/doc/html/pcre2pattern.html
index 797690a..8ddb994 100644
--- a/doc/html/pcre2pattern.html
+++ b/doc/html/pcre2pattern.html
@@ -379,32 +379,31 @@ case letter, it is converted to upper case. Then bit 6 of the character (hex
40) is inverted. Thus \cA to \cZ become hex 01 to hex 1A (A is 41, Z is 5A),
but \c{ becomes hex 3B ({ is 7B), and \c; becomes hex 7B (; is 3B). If the
code unit following \c has a value less than 32 or greater than 126, a
-compile-time error occurs. This locks out non-printable ASCII characters in all
-modes.
+compile-time error occurs.
When PCRE2 is compiled in EBCDIC mode, \a, \e, \f, \n, \r, and \t
generate the appropriate EBCDIC code values. The \c escape is processed
as specified for Perl in the perlebcdic document. The only characters
that are allowed after \c are A-Z, a-z, or one of @, [, \, ], ^, _, or ?. Any
-other character provokes a compile-time error. The sequence \@ encodes
-character code 0; the letters (in either case) encode characters 1-26 (hex 01
-to hex 1A); [, \, ], ^, and _ encode characters 27-31 (hex 1B to hex 1F), and
-\? becomes either 255 (hex FF) or 95 (hex 5F).
+other character provokes a compile-time error. The sequence \c@ encodes
+character code 0; after \c the letters (in either case) encode characters 1-26
+(hex 01 to hex 1A); [, \, ], ^, and _ encode characters 27-31 (hex 1B to hex
+1F), and \c? becomes either 255 (hex FF) or 95 (hex 5F).
-Thus, apart from \?, these escapes generate the same character code values as
+Thus, apart from \c?, these escapes generate the same character code values as
they do in an ASCII environment, though the meanings of the values mostly
-differ. For example, \G always generates code value 7, which is BEL in ASCII
+differ. For example, \cG always generates code value 7, which is BEL in ASCII
but DEL in EBCDIC.
-The sequence \? generates DEL (127, hex 7F) in an ASCII environment, but
+The sequence \c? generates DEL (127, hex 7F) in an ASCII environment, but
because 127 is not a control character in EBCDIC, Perl makes it generate the
APC character. Unfortunately, there are several variants of EBCDIC. In most of
them the APC character has the value 255 (hex FF), but in the one Perl calls
POSIX-BC its value is 95 (hex 5F). If certain other characters have POSIX-BC
-values, PCRE2 makes \? generate 95; otherwise it generates 255.
+values, PCRE2 makes \c? generate 95; otherwise it generates 255.
After \0 up to two further octal digits are read. If there are fewer than two
@@ -526,9 +525,9 @@ by code point, as described in the previous section.
Absolute and relative back references
-The sequence \g followed by an unsigned or a negative number, optionally
-enclosed in braces, is an absolute or relative back reference. A named back
-reference can be coded as \g{name}. Back references are discussed
+The sequence \g followed by a signed or unsigned number, optionally enclosed
+in braces, is an absolute or relative back reference. A named back reference
+can be coded as \g{name}. Back references are discussed
later,
following the discussion of
parenthesized subpatterns.
@@ -1326,13 +1325,32 @@ whatever setting of the PCRE2_DOTALL and PCRE2_MULTILINE options is used. A
class such as [^a] always matches one of these characters.
+The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v,
+\V, \w, and \W may appear in a character class, and add the characters that
+they match to the class. For example, [\dABCDEF] matches any hexadecimal
+digit. In UTF modes, the PCRE2_UCP option affects the meanings of \d, \s, \w
+and their upper case partners, just as it does when they appear outside a
+character class, as described in the section entitled
+"Generic character types"
+above. The escape sequence \b has a different meaning inside a character
+class; it matches the backspace character. The sequences \B, \N, \R, and \X
+are not special inside a character class. Like any other unrecognized escape
+sequences, they cause an error.
+
+
The minus (hyphen) character can be used to specify a range of characters in a
character class. For example, [d-m] matches any letter between d and m,
inclusive. If a minus character is required in a class, it must be escaped with
a backslash or appear in a position where it cannot be interpreted as
-indicating a range, typically as the first or last character in the class, or
-immediately after a range. For example, [b-d-z] matches letters in the range b
-to d, a hyphen character, or z.
+indicating a range, typically as the first or last character in the class,
+or immediately after a range. For example, [b-d-z] matches letters in the range
+b to d, a hyphen character, or z.
+
+
+Perl treats a hyphen as a literal if it appears before a POSIX class (see
+below) or a character type escape such as as \d, but gives a warning in its
+warning mode, as this is most likely a user error. As PCRE2 has no facility for
+warning, an error is given in these cases.
It is not possible to have the literal character "]" as the end character of a
@@ -1344,12 +1362,6 @@ followed by two other characters. The octal or hexadecimal representation of
"]" can also be used to end a range.
-An error is generated if a POSIX character class (see below) or an escape
-sequence other than one that defines a single character appears at a point
-where a range ending character is expected. For example, [z-\xff] is valid,
-but [A-\d] and [A-[:digit:]] are not.
-
-
Ranges normally include all code points between the start and end characters,
inclusive. They can also be used for code points specified numerically, for
example [\000-\037]. Ranges can include any characters that are valid for the
@@ -1372,19 +1384,6 @@ tables for a French locale are in use, [\xc8-\xcb] matches accented E
characters in both cases.
-The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v,
-\V, \w, and \W may appear in a character class, and add the characters that
-they match to the class. For example, [\dABCDEF] matches any hexadecimal
-digit. In UTF modes, the PCRE2_UCP option affects the meanings of \d, \s, \w
-and their upper case partners, just as it does when they appear outside a
-character class, as described in the section entitled
-"Generic character types"
-above. The escape sequence \b has a different meaning inside a character
-class; it matches the backspace character. The sequences \B, \N, \R, and \X
-are not special inside a character class. Like any other unrecognized escape
-sequences, they cause an error.
-
-
A circumflex can conveniently be used with the upper case character types to
specify a more restricted set of characters than the matching lower case type.
For example, the class [^\W_] matches any letter or digit, but not underscore,
@@ -1552,13 +1551,8 @@ respectively.
When one of these option changes occurs at top level (that is, not inside
subpattern parentheses), the change applies to the remainder of the pattern
-that follows. If the change is placed right at the start of a pattern, PCRE2
-extracts it into the global options (and it will therefore show up in data
-extracted by the pcre2_pattern_info() function).
-
-
-An option change within a subpattern (see below for a description of
-subpatterns) affects only that part of the subpattern that follows it, so
+that follows. An option change within a subpattern (see below for a description
+of subpatterns) affects only that part of the subpattern that follows it, so
(a(?i)b)c
@@ -2093,9 +2087,9 @@ subpattern is possible using named parentheses (see below).
Another way of avoiding the ambiguity inherent in the use of digits following a
-backslash is to use the \g escape sequence. This escape must be followed by an
-unsigned number or a negative number, optionally enclosed in braces. These
-examples are all identical:
+backslash is to use the \g escape sequence. This escape must be followed by a
+signed or unsigned number, optionally enclosed in braces. These examples are
+all identical:
(ring), \1
(ring), \g1
@@ -2103,8 +2097,7 @@ examples are all identical:
An unsigned number specifies an absolute reference without the ambiguity that
is present in the older syntax. It is also useful when literal digits follow
-the reference. A negative number is a relative reference. Consider this
-example:
+the reference. A signed number is a relative reference. Consider this example:
(abc(def)ghi)\g{-1}
@@ -2115,6 +2108,11 @@ can be helpful in long patterns, and also in patterns that are created by
joining together fragments that contain references within themselves.
+The sequence \g{+1} is a reference to the next capturing subpattern. This kind
+of forward reference can be useful it patterns that repeat. Perl does not
+support the use of + in this way.
+
+
A back reference matches whatever actually matched the capturing subpattern in
the current subject string, rather than anything matching the subpattern
itself (see
@@ -2214,6 +2212,14 @@ capturing is carried out only for positive assertions. (Perl sometimes, but not
always, does do capturing in negative assertions.)
+WARNING: If a positive assertion containing one or more capturing subpatterns
+succeeds, but failure to match later in the pattern causes backtracking over
+this assertion, the captures within the assertion are reset only if no higher
+numbered captures are already set. This is, unfortunately, a fundamental
+limitation of the current implementation; it may get removed in a future
+reworking.
+
+
For compatibility with Perl, most assertion subpatterns may be repeated; though
it makes no sense to assert the same thing several times, the side effect of
capturing parentheses may occasionally be useful. However, an assertion that
@@ -2310,18 +2316,31 @@ match. If there are insufficient characters before the current position, the
assertion fails.
-In a UTF mode, PCRE2 does not allow the \C escape (which matches a single code
-unit even in a UTF mode) to appear in lookbehind assertions, because it makes
-it impossible to calculate the length of the lookbehind. The \X and \R
-escapes, which can match different numbers of code units, are also not
-permitted.
+In UTF-8 and UTF-16 modes, PCRE2 does not allow the \C escape (which matches a
+single code unit even in a UTF mode) to appear in lookbehind assertions,
+because it makes it impossible to calculate the length of the lookbehind. The
+\X and \R escapes, which can match different numbers of code units, are never
+permitted in lookbehinds.
"Subroutine"
calls (see below) such as (?2) or (?&X) are permitted in lookbehinds, as long
-as the subpattern matches a fixed-length string.
-Recursion,
-however, is not supported.
+as the subpattern matches a fixed-length string. However,
+recursion,
+that is, a "subroutine" call into a group that is already active,
+is not supported.
+
+
+Perl does not support back references in lookbehinds. PCRE2 does support them,
+but only if certain conditions are met. The PCRE2_MATCH_UNSET_BACKREF option
+must not be set, there must be no use of (?| in the pattern (it creates
+duplicate subpattern numbers), and if the back reference is by name, the name
+must be unique. Of course, the referenced subpattern must itself be of fixed
+length. The following pattern matches words containing at least two characters
+that begin and end with the same character:
+
+ \b(\w)\w++(?<=\1)
+
Possessive quantifiers can be used in conjunction with lookbehind assertions to
@@ -2459,7 +2478,9 @@ Checking for a used subpattern by name
Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used
subpattern by name. For compatibility with earlier versions of PCRE1, which had
-this facility before Perl, the syntax (?(name)...) is also recognized.
+this facility before Perl, the syntax (?(name)...) is also recognized. Note,
+however, that undelimited names consisting of the letter R followed by digits
+are ambiguous (see the following section).
Rewriting the above example to use a named subpattern gives this:
@@ -2474,30 +2495,52 @@ matched.
Checking for pattern recursion
-If the condition is the string (R), and there is no subpattern with the name R,
-the condition is true if a recursive call to the whole pattern or any
-subpattern has been made. If digits or a name preceded by ampersand follow the
-letter R, for example:
+"Recursion" in this sense refers to any subroutine-like call from one part of
+the pattern to another, whether or not it is actually recursive. See the
+sections entitled
+"Recursive patterns"
+and
+"Subpatterns as subroutines"
+below for details of recursion and subpattern calls.
+
+
+If a condition is the string (R), and there is no subpattern with the name R,
+the condition is true if matching is currently in a recursion or subroutine
+call to the whole pattern or any subpattern. If digits follow the letter R, and
+there is no subpattern with that name, the condition is true if the most recent
+call is into a subpattern with the given number, which must exist somewhere in
+the overall pattern. This is a contrived example that is equivalent to a+b:
- (?(R3)...) or (?(R&name)...)
+ ((?(R1)a+|(?1)b))
-the condition is true if the most recent recursion is into a subpattern whose
-number or name is given. This condition does not check the entire recursion
-stack. If the name used in a condition of this kind is a duplicate, the test is
-applied to all subpatterns of the same name, and is true if any one of them is
-the most recent recursion.
+However, in both cases, if there is a subpattern with a matching name, the
+condition tests for its being set, as described in the section above, instead
+of testing for recursion. For example, creating a group with the name R1 by
+adding (?<R1>) to the above pattern completely changes its meaning.
+
+
+If a name preceded by ampersand follows the letter R, for example:
+
+ (?(R&name)...)
+
+the condition is true if the most recent recursion is into a subpattern of that
+name (which must exist within the pattern).
+
+
+This condition does not check the entire recursion stack. It tests only the
+current level. If the name used in a condition of this kind is a duplicate, the
+test is applied to all subpatterns of the same name, and is true if any one of
+them is the most recent recursion.
At "top level", all these recursion test conditions are false.
-The syntax for recursive patterns
-is described below.
Defining subpatterns for use by reference only
-If the condition is the string (DEFINE), and there is no subpattern with the
-name DEFINE, the condition is always false. In this case, there may be only one
+If the condition is the string (DEFINE), the condition is always false, even if
+there is a group with the name DEFINE. In this case, there may be only one
alternative in the subpattern. It is always skipped if control reaches this
point in the pattern; the idea of DEFINE is that it can be used to define
subroutines that can be referenced from elsewhere. (The use of
@@ -2965,12 +3008,22 @@ depending on whether or not a name is present.
By default, for compatibility with Perl, a name is any sequence of characters
that does not include a closing parenthesis. The name is not processed in
any way, and it is not possible to include a closing parenthesis in the name.
-However, if the PCRE2_ALT_VERBNAMES option is set, normal backslash processing
-is applied to verb names and only an unescaped closing parenthesis terminates
-the name. A closing parenthesis can be included in a name either as \) or
-between \Q and \E. If the PCRE2_EXTENDED option is set, unescaped whitespace
-in verb names is skipped and #-comments are recognized, exactly as in the rest
-of the pattern.
+This can be changed by setting the PCRE2_ALT_VERBNAMES option, but the result
+is no longer Perl-compatible.
+
+
+When PCRE2_ALT_VERBNAMES is set, backslash processing is applied to verb names
+and only an unescaped closing parenthesis terminates the name. However, the
+only backslash items that are permitted are \Q, \E, and sequences such as
+\x{100} that define character code points. Character type escapes such as \d
+are faulted.
+
+
+A closing parenthesis can be included in a name either as \) or between \Q
+and \E. In addition to backslash processing, if the PCRE2_EXTENDED option is
+also set, unescaped whitespace in verb names is skipped, and #-comments are
+recognized, exactly as in the rest of the pattern. PCRE2_EXTENDED does not
+affect verb names unless PCRE2_ALT_VERBNAMES is also set.
The maximum length of a name is 255 in the 8-bit library and 65535 in the
@@ -3393,7 +3446,7 @@ Cambridge, England.
REVISION
-Last updated: 20 June 2016
+Last updated: 23 October 2016
Copyright © 1997-2016 University of Cambridge.
diff --git a/doc/html/pcre2syntax.html b/doc/html/pcre2syntax.html
index 7fdc0dc..8d20353 100644
--- a/doc/html/pcre2syntax.html
+++ b/doc/html/pcre2syntax.html
@@ -492,6 +492,9 @@ Each top-level branch of a look behind must be of a fixed length.
\n reference by number (can be ambiguous)
\gn reference by number
\g{n} reference by number
+ \g+n relative reference by number (PCRE2 extension)
+ \g-n relative reference by number
+ \g{+n} relative reference by number (PCRE2 extension)
\g{-n} relative reference by number
\k<name> reference by name (Perl)
\k'name' reference by name (Perl)
@@ -530,14 +533,17 @@ Each top-level branch of a look behind must be of a fixed length.
(?(-n) relative reference condition
(?(<name>) named reference condition (Perl)
(?('name') named reference condition (Perl)
- (?(name) named reference condition (PCRE2)
+ (?(name) named reference condition (PCRE2, deprecated)
(?(R) overall recursion condition
- (?(Rn) specific group recursion condition
- (?(R&name) specific recursion condition
+ (?(Rn) specific numbered group recursion condition
+ (?(R&name) specific named group recursion condition
(?(DEFINE) define subpattern for reference
(?(VERSION[>]=n.m) test PCRE2 version
(?(assert) assertion condition
-
+
+Note the ambiguity of (?(R) and (?(Rn) which might be named reference
+conditions or recursion tests. Such a condition is interpreted as a reference
+condition if the relevant named group exists.
@@ -589,9 +595,9 @@ Cambridge, England.
-Last updated: 16 October 2015
+Last updated: 28 September 2016
-Copyright © 1997-2015 University of Cambridge.
+Copyright © 1997-2016 University of Cambridge.
Return to the PCRE2 index page. diff --git a/doc/html/pcre2test.html b/doc/html/pcre2test.html index 7509b37..dc1b1dd 100644 --- a/doc/html/pcre2test.html +++ b/doc/html/pcre2test.html @@ -615,6 +615,7 @@ about the pattern: pushcopy push a copy onto the stack stackguard=<number> test the stackguard feature tables=[0|1|2] select internal tables + use_length do not zero-terminate the pattern utf8_input treat input as UTF-8 The effects of these modifiers are described in the following sections. @@ -698,6 +699,18 @@ testing that pcre2_compile() behaves correctly in this case (it uses default values).
+By default, patterns are passed to the compiling functions as zero-terminated +strings. When using the POSIX wrapper API, there is no other option. However, +when using PCRE2's native API, patterns can be passed by length instead of +being zero-terminated. The use_length modifier causes this to happen. +Using a length happens automatically (whether or not use_length is set) +when hex is set, because patterns specified in hexadecimal may contain +binary zeros. +
+@@ -720,10 +733,10 @@ the delimiter within a substring. The hex and expand modifiers are mutually exclusive.
-By default, pcre2test passes patterns as zero-terminated strings to -pcre2_compile(), giving the length as PCRE2_ZERO_TERMINATED. However, for -patterns specified with the hex modifier, the actual length of the -pattern is passed. +The POSIX API cannot be used with patterns specified in hexadecimal because +they may contain binary zeros, which conflicts with regcomp()'s +requirement for a zero-terminated string. Such patterns are always passed to +pcre2_compile() as a string with a length, not as zero-terminated.
-Last updated: 02 August 2016
+Last updated: 04 November 2016
Copyright © 1997-2016 University of Cambridge.
diff --git a/doc/index.html.src b/doc/index.html.src
index 703c298..eebb80b 100644
--- a/doc/index.html.src
+++ b/doc/index.html.src
@@ -94,6 +94,9 @@ in the library.