Documentation correction.
This commit is contained in:
parent
52ba34a73c
commit
8fe95cf804
189
HACKING
189
HACKING
|
@ -1,14 +1,14 @@
|
||||||
Technical Notes about PCRE2
|
Technical Notes about PCRE2
|
||||||
---------------------------
|
---------------------------
|
||||||
|
|
||||||
These are very rough technical notes that record potentially useful information
|
These are very rough technical notes that record potentially useful information
|
||||||
about PCRE2 internals. PCRE2 is a library based on the original PCRE library,
|
about PCRE2 internals. PCRE2 is a library based on the original PCRE library,
|
||||||
but with a revised (and incompatible) API. To avoid confusion, the original
|
but with a revised (and incompatible) API. To avoid confusion, the original
|
||||||
library is referred to as PCRE1 below. For information about testing PCRE2, see
|
library is referred to as PCRE1 below. For information about testing PCRE2, see
|
||||||
the pcre2test documentation and the comment at the head of the RunTest file.
|
the pcre2test documentation and the comment at the head of the RunTest file.
|
||||||
|
|
||||||
PCRE1 releases were up to 8.3x when PCRE2 was developed. The 8.xx series will
|
PCRE1 releases were up to 8.3x when PCRE2 was developed. The 8.xx series will
|
||||||
continue for bugfixes if necessary. PCRE2 releases start at 10.0 to avoid
|
continue for bugfixes if necessary. PCRE2 releases start at 10.0 to avoid
|
||||||
confusion with PCRE1.
|
confusion with PCRE1.
|
||||||
|
|
||||||
|
|
||||||
|
@ -52,7 +52,7 @@ unrelated to those mentioned above), I tried at first to invent an algorithm
|
||||||
that used an amount of store bounded by a multiple of the number of characters
|
that used an amount of store bounded by a multiple of the number of characters
|
||||||
in the pattern, to save on compiling time. However, because of the greater
|
in the pattern, to save on compiling time. However, because of the greater
|
||||||
complexity in Perl regular expressions, I couldn't do this. In any case, a
|
complexity in Perl regular expressions, I couldn't do this. In any case, a
|
||||||
first pass through the pattern is helpful for other reasons.
|
first pass through the pattern is helpful for other reasons.
|
||||||
|
|
||||||
|
|
||||||
Support for 16-bit and 32-bit data strings
|
Support for 16-bit and 32-bit data strings
|
||||||
|
@ -62,7 +62,7 @@ The library can be compiled in any combination of 8-bit, 16-bit or 32-bit
|
||||||
modes, creating up to three different libraries. In the description that
|
modes, creating up to three different libraries. In the description that
|
||||||
follows, the word "short" is used for a 16-bit data quantity, and the phrase
|
follows, the word "short" is used for a 16-bit data quantity, and the phrase
|
||||||
"code unit" is used for a quantity that is a byte in 8-bit mode, a short in
|
"code unit" is used for a quantity that is a byte in 8-bit mode, a short in
|
||||||
16-bit mode and a 32-bit word in 32-bit mode. The names of PCRE2 functions are
|
16-bit mode and a 32-bit word in 32-bit mode. The names of PCRE2 functions are
|
||||||
given in generic form, without a _8, _16, or _32 suffix.
|
given in generic form, without a _8, _16, or _32 suffix.
|
||||||
|
|
||||||
|
|
||||||
|
@ -88,8 +88,8 @@ a "fake" mode that enables it to compute how much memory it would need, while
|
||||||
actually only ever using a few hundred bytes of working memory, and without too
|
actually only ever using a few hundred bytes of working memory, and without too
|
||||||
many tests of the mode that might slow it down. So I refactored the compiling
|
many tests of the mode that might slow it down. So I refactored the compiling
|
||||||
functions to work this way. This got rid of about 600 lines of source. It
|
functions to work this way. This got rid of about 600 lines of source. It
|
||||||
should make future maintenance and development easier. As this was such a major
|
should make future maintenance and development easier. As this was such a major
|
||||||
change, I never released 6.8, instead upping the number to 7.0 (other quite
|
change, I never released 6.8, instead upping the number to 7.0 (other quite
|
||||||
major changes were also present in the 7.0 release).
|
major changes were also present in the 7.0 release).
|
||||||
|
|
||||||
A side effect of this work was that the previous limit of 200 on the nesting
|
A side effect of this work was that the previous limit of 200 on the nesting
|
||||||
|
@ -127,7 +127,7 @@ compiled pattern data as pcre2_match(); however, not all the facilities are
|
||||||
available, and those that are do not always work in quite the same way. See the
|
available, and those that are do not always work in quite the same way. See the
|
||||||
user documentation for details.
|
user documentation for details.
|
||||||
|
|
||||||
The algorithm that is used for pcre2_dfa_match() is not a traditional FSM,
|
The algorithm that is used for pcre2_dfa_match() is not a traditional FSM,
|
||||||
because it may have a number of states active at one time. More work would be
|
because it may have a number of states active at one time. More work would be
|
||||||
needed at compile time to produce a traditional FSM where only one state is
|
needed at compile time to produce a traditional FSM where only one state is
|
||||||
ever active at once. I believe some other regex matchers work this way. JIT
|
ever active at once. I believe some other regex matchers work this way. JIT
|
||||||
|
@ -177,7 +177,7 @@ These items are all just one unit long
|
||||||
OP_ANYBYTE match any single code unit, even in UTF-8/16 mode
|
OP_ANYBYTE match any single code unit, even in UTF-8/16 mode
|
||||||
OP_SOD match start of data: \A
|
OP_SOD match start of data: \A
|
||||||
OP_SOM, start of match (subject + offset): \G
|
OP_SOM, start of match (subject + offset): \G
|
||||||
OP_SET_SOM, set start of match (\K)
|
OP_SET_SOM, set start of match (\K)
|
||||||
OP_CIRC ^ (start of data)
|
OP_CIRC ^ (start of data)
|
||||||
OP_CIRCM ^ multiline mode (start of data or after newline)
|
OP_CIRCM ^ multiline mode (start of data or after newline)
|
||||||
OP_NOT_WORD_BOUNDARY \W
|
OP_NOT_WORD_BOUNDARY \W
|
||||||
|
@ -185,31 +185,31 @@ These items are all just one unit long
|
||||||
OP_NOT_DIGIT \D
|
OP_NOT_DIGIT \D
|
||||||
OP_DIGIT \d
|
OP_DIGIT \d
|
||||||
OP_NOT_HSPACE \H
|
OP_NOT_HSPACE \H
|
||||||
OP_HSPACE \h
|
OP_HSPACE \h
|
||||||
OP_NOT_WHITESPACE \S
|
OP_NOT_WHITESPACE \S
|
||||||
OP_WHITESPACE \s
|
OP_WHITESPACE \s
|
||||||
OP_NOT_VSPACE \V
|
OP_NOT_VSPACE \V
|
||||||
OP_VSPACE \v
|
OP_VSPACE \v
|
||||||
OP_NOT_WORDCHAR \W
|
OP_NOT_WORDCHAR \W
|
||||||
OP_WORDCHAR \w
|
OP_WORDCHAR \w
|
||||||
OP_EODN match end of data or newline at end: \Z
|
OP_EODN match end of data or newline at end: \Z
|
||||||
OP_EOD match end of data: \z
|
OP_EOD match end of data: \z
|
||||||
OP_DOLL $ (end of data, or before final newline)
|
OP_DOLL $ (end of data, or before final newline)
|
||||||
OP_DOLLM $ multiline mode (end of data or before newline)
|
OP_DOLLM $ multiline mode (end of data or before newline)
|
||||||
OP_EXTUNI match an extended Unicode grapheme cluster
|
OP_EXTUNI match an extended Unicode grapheme cluster
|
||||||
OP_ANYNL match any Unicode newline sequence
|
OP_ANYNL match any Unicode newline sequence
|
||||||
|
|
||||||
OP_ASSERT_ACCEPT )
|
OP_ASSERT_ACCEPT )
|
||||||
OP_ACCEPT ) These are Perl 5.10's "backtracking control
|
OP_ACCEPT ) These are Perl 5.10's "backtracking control
|
||||||
OP_COMMIT ) verbs". If OP_ACCEPT is inside capturing
|
OP_COMMIT ) verbs". If OP_ACCEPT is inside capturing
|
||||||
OP_FAIL ) parentheses, it may be preceded by one or more
|
OP_FAIL ) parentheses, it may be preceded by one or more
|
||||||
OP_PRUNE ) OP_CLOSE, each followed by a count that
|
OP_PRUNE ) OP_CLOSE, each followed by a count that
|
||||||
OP_SKIP ) indicates which parentheses must be closed.
|
OP_SKIP ) indicates which parentheses must be closed.
|
||||||
OP_THEN )
|
OP_THEN )
|
||||||
|
|
||||||
OP_ASSERT_ACCEPT is used when (*ACCEPT) is encountered within an assertion.
|
OP_ASSERT_ACCEPT is used when (*ACCEPT) is encountered within an assertion.
|
||||||
This ends the assertion, not the entire pattern match.
|
This ends the assertion, not the entire pattern match.
|
||||||
|
|
||||||
|
|
||||||
Backtracking control verbs with optional data
|
Backtracking control verbs with optional data
|
||||||
---------------------------------------------
|
---------------------------------------------
|
||||||
|
@ -219,12 +219,12 @@ OP_MARK is followed by the mark name, preceded by a length in one code unit,
|
||||||
and followed by a binary zero. For (*PRUNE), (*SKIP), and (*THEN) with
|
and followed by a binary zero. For (*PRUNE), (*SKIP), and (*THEN) with
|
||||||
arguments, the opcodes OP_PRUNE_ARG, OP_SKIP_ARG, and OP_THEN_ARG are used,
|
arguments, the opcodes OP_PRUNE_ARG, OP_SKIP_ARG, and OP_THEN_ARG are used,
|
||||||
with the name following in the same format as OP_MARK.
|
with the name following in the same format as OP_MARK.
|
||||||
|
|
||||||
|
|
||||||
Matching literal characters
|
Matching literal characters
|
||||||
---------------------------
|
---------------------------
|
||||||
|
|
||||||
The OP_CHAR opcode is followed by a single character that is to be matched
|
The OP_CHAR opcode is followed by a single character that is to be matched
|
||||||
casefully. For caseless matching, OP_CHARI is used. In UTF-8 or UTF-16 modes,
|
casefully. For caseless matching, OP_CHARI is used. In UTF-8 or UTF-16 modes,
|
||||||
the character may be more than one code unit long. In UTF-32 mode, characters
|
the character may be more than one code unit long. In UTF-32 mode, characters
|
||||||
are always exactly one code unit long.
|
are always exactly one code unit long.
|
||||||
|
@ -241,32 +241,35 @@ The common repeats (*, +, ?), when applied to a single character, use the
|
||||||
following opcodes, which come in caseful and caseless versions:
|
following opcodes, which come in caseful and caseless versions:
|
||||||
|
|
||||||
Caseful Caseless
|
Caseful Caseless
|
||||||
OP_STAR OP_STARI
|
OP_STAR OP_STARI
|
||||||
OP_MINSTAR OP_MINSTARI
|
OP_MINSTAR OP_MINSTARI
|
||||||
OP_POSSTAR OP_POSSTARI
|
OP_POSSTAR OP_POSSTARI
|
||||||
OP_PLUS OP_PLUSI
|
OP_PLUS OP_PLUSI
|
||||||
OP_MINPLUS OP_MINPLUSI
|
OP_MINPLUS OP_MINPLUSI
|
||||||
OP_POSPLUS OP_POSPLUSI
|
OP_POSPLUS OP_POSPLUSI
|
||||||
OP_QUERY OP_QUERYI
|
OP_QUERY OP_QUERYI
|
||||||
OP_MINQUERY OP_MINQUERYI
|
OP_MINQUERY OP_MINQUERYI
|
||||||
OP_POSQUERY OP_POSQUERYI
|
OP_POSQUERY OP_POSQUERYI
|
||||||
|
|
||||||
Each opcode is followed by the character that is to be repeated. In ASCII or
|
Each opcode is followed by the character that is to be repeated. In ASCII or
|
||||||
UTF-32 modes, these are two-code-unit items; in UTF-8 or UTF-16 modes, the
|
UTF-32 modes, these are two-code-unit items; in UTF-8 or UTF-16 modes, the
|
||||||
length is variable. Those with "MIN" in their names are the minimizing
|
length is variable. Those with "MIN" in their names are the minimizing
|
||||||
versions. Those with "POS" in their names are possessive versions. Other kinds
|
versions. Those with "POS" in their names are possessive versions. Other kinds
|
||||||
of repeat make use of these opcodes:
|
of repeat make use of these opcodes:
|
||||||
|
|
||||||
Caseful Caseless
|
Caseful Caseless
|
||||||
OP_UPTO OP_UPTOI
|
OP_UPTO OP_UPTOI
|
||||||
OP_MINUPTO OP_MINUPTOI
|
OP_MINUPTO OP_MINUPTOI
|
||||||
OP_POSUPTO OP_POSUPTOI
|
OP_POSUPTO OP_POSUPTOI
|
||||||
OP_EXACT OP_EXACTI
|
OP_EXACT OP_EXACTI
|
||||||
|
|
||||||
Each of these is followed by a count and then the repeated character. OP_UPTO
|
Each of these is followed by a count and then the repeated character. The count
|
||||||
matches from 0 to the given number. A repeat with a non-zero minimum and a
|
is two bytes long in 8-bit mode (most significant byte first), or one code unit
|
||||||
fixed maximum is coded as an OP_EXACT followed by an OP_UPTO (or OP_MINUPTO or
|
in 16-bit and 32-bit modes.
|
||||||
OPT_POSUPTO).
|
|
||||||
|
OP_UPTO matches from 0 to the given number. A repeat with a non-zero minimum
|
||||||
|
and a fixed maximum is coded as an OP_EXACT followed by an OP_UPTO (or
|
||||||
|
OP_MINUPTO or OPT_POSUPTO).
|
||||||
|
|
||||||
Another set of matching repeating opcodes (called OP_NOTSTAR, OP_NOTSTARI,
|
Another set of matching repeating opcodes (called OP_NOTSTAR, OP_NOTSTARI,
|
||||||
etc.) are used for repeated, negated, single-character classes such as [^a]*.
|
etc.) are used for repeated, negated, single-character classes such as [^a]*.
|
||||||
|
@ -283,23 +286,23 @@ in the next code unit. The opcodes are:
|
||||||
|
|
||||||
OP_TYPESTAR
|
OP_TYPESTAR
|
||||||
OP_TYPEMINSTAR
|
OP_TYPEMINSTAR
|
||||||
OP_TYPEPOSSTAR
|
OP_TYPEPOSSTAR
|
||||||
OP_TYPEPLUS
|
OP_TYPEPLUS
|
||||||
OP_TYPEMINPLUS
|
OP_TYPEMINPLUS
|
||||||
OP_TYPEPOSPLUS
|
OP_TYPEPOSPLUS
|
||||||
OP_TYPEQUERY
|
OP_TYPEQUERY
|
||||||
OP_TYPEMINQUERY
|
OP_TYPEMINQUERY
|
||||||
OP_TYPEPOSQUERY
|
OP_TYPEPOSQUERY
|
||||||
OP_TYPEUPTO
|
OP_TYPEUPTO
|
||||||
OP_TYPEMINUPTO
|
OP_TYPEMINUPTO
|
||||||
OP_TYPEPOSUPTO
|
OP_TYPEPOSUPTO
|
||||||
OP_TYPEEXACT
|
OP_TYPEEXACT
|
||||||
|
|
||||||
|
|
||||||
Match by Unicode property
|
Match by Unicode property
|
||||||
-------------------------
|
-------------------------
|
||||||
|
|
||||||
OP_PROP and OP_NOTPROP are used for positive and negative matches of a
|
OP_PROP and OP_NOTPROP are used for positive and negative matches of a
|
||||||
character by testing its Unicode property (the \p and \P escape sequences).
|
character by testing its Unicode property (the \p and \P escape sequences).
|
||||||
Each is followed by two code units that encode the desired property as a type
|
Each is followed by two code units that encode the desired property as a type
|
||||||
and a value. The types are a set of #defines of the form PT_xxx, and the values
|
and a value. The types are a set of #defines of the form PT_xxx, and the values
|
||||||
|
@ -317,7 +320,7 @@ Character classes
|
||||||
|
|
||||||
If there is only one character in a class, OP_CHAR or OP_CHARI is used for a
|
If there is only one character in a class, OP_CHAR or OP_CHARI is used for a
|
||||||
positive class, and OP_NOT or OP_NOTI for a negative one (that is, for
|
positive class, and OP_NOT or OP_NOTI for a negative one (that is, for
|
||||||
something like [^a]).
|
something like [^a]).
|
||||||
|
|
||||||
A set of repeating opcodes (called OP_NOTSTAR etc.) are used for repeated,
|
A set of repeating opcodes (called OP_NOTSTAR etc.) are used for repeated,
|
||||||
negated, single-character classes. The normal single-character opcodes
|
negated, single-character classes. The normal single-character opcodes
|
||||||
|
@ -325,35 +328,41 @@ negated, single-character classes. The normal single-character opcodes
|
||||||
|
|
||||||
When there is more than one character in a class, and all the code points are
|
When there is more than one character in a class, and all the code points are
|
||||||
less than 256, OP_CLASS is used for a positive class, and OP_NCLASS for a
|
less than 256, OP_CLASS is used for a positive class, and OP_NCLASS for a
|
||||||
negative one. In either case, the opcode is followed by a 32-byte (16-short,
|
negative one. In either case, the opcode is followed by a 32-byte (16-short,
|
||||||
8-word) bit map containing a 1 bit for every character that is acceptable. The
|
8-word) bit map containing a 1 bit for every character that is acceptable. The
|
||||||
bits are counted from the least significant end of each unit. In caseless mode,
|
bits are counted from the least significant end of each unit. In caseless mode,
|
||||||
bits for both cases are set.
|
bits for both cases are set.
|
||||||
|
|
||||||
The reason for having both OP_CLASS and OP_NCLASS is so that, in UTF-8/16/32
|
The reason for having both OP_CLASS and OP_NCLASS is so that, in UTF-8 and
|
||||||
mode, subject characters with values greater than 255 can be handled correctly.
|
16-bit and 32-bit modes, subject characters with values greater than 255 can be
|
||||||
For OP_CLASS they do not match, whereas for OP_NCLASS they do.
|
handled correctly. For OP_CLASS they do not match, whereas for OP_NCLASS they
|
||||||
|
do.
|
||||||
|
|
||||||
For classes containing characters with values greater than 255 or that contain
|
For classes containing characters with values greater than 255 or that contain
|
||||||
\p or \P, OP_XCLASS is used. It optionally uses a bit map if any code points
|
\p or \P, OP_XCLASS is used. It optionally uses a bit map if any acceptable
|
||||||
are less than 256, followed by a list of pairs (for a range) and single
|
code points are less than 256, followed by a list of pairs (for a range) and
|
||||||
characters. In caseless mode, both cases are explicitly listed.
|
single characters. In caseless mode, both cases are explicitly listed.
|
||||||
|
|
||||||
OP_XCLASS is followed by a code unit containing flag bits: XCL_NOT indicates
|
OP_XCLASS is followed by a LINK_SIZE item containing the total length of the
|
||||||
that this is a negative class, and XCL_MAP indicates that a bit map is present.
|
opcode and its data. This is followed by a code unit containing flag bits:
|
||||||
There follows the bit map, if XCL_MAP is set, and then a sequence of items
|
XCL_NOT indicates that this is a negative class, and XCL_MAP indicates that a
|
||||||
coded as follows:
|
bit map is present. There follows the bit map, if XCL_MAP is set, and then a
|
||||||
|
sequence of items coded as follows:
|
||||||
|
|
||||||
XCL_END marks the end of the list
|
XCL_END marks the end of the list
|
||||||
XCL_SINGLE one character follows
|
XCL_SINGLE one character follows
|
||||||
XCL_RANGE two characters follow
|
XCL_RANGE two characters follow
|
||||||
XCL_PROP a Unicode property (type, value) follows
|
XCL_PROP a Unicode property (type, value) follows
|
||||||
XCL_NOTPROP a Unicode property (type, value) follows
|
XCL_NOTPROP a Unicode property (type, value) follows
|
||||||
|
|
||||||
If a range starts with a code point less than 256 and ends with one greater
|
If a range starts with a code point less than 256 and ends with one greater
|
||||||
than 256, it is split into two ranges, with characters less than 256 being
|
than 256, it is split into two ranges, with characters less than 256 being
|
||||||
indicated in the bit map, and the rest with XCL_RANGE.
|
indicated in the bit map, and the rest with XCL_RANGE.
|
||||||
|
|
||||||
|
When XCL_NOT is set, the bit map, if present, contains bits for characters that
|
||||||
|
are allowed (exactly as for OP_NCLASS), but the list of items that follow it
|
||||||
|
specifies characters and properties that are not allowed.
|
||||||
|
|
||||||
|
|
||||||
Back references
|
Back references
|
||||||
---------------
|
---------------
|
||||||
|
@ -364,7 +373,7 @@ number or by name). When named groups are used, there may be more than one
|
||||||
group with the same name. In this case, a reference to such a group by name
|
group with the same name. In this case, a reference to such a group by name
|
||||||
generates OP_DNREF or OP_DNREFI. These are followed by two counts: the index
|
generates OP_DNREF or OP_DNREFI. These are followed by two counts: the index
|
||||||
(not the byte offset) in the group name table of the first entry for the
|
(not the byte offset) in the group name table of the first entry for the
|
||||||
required name, followed by the number of groups with the same name. The
|
required name, followed by the number of groups with the same name. The
|
||||||
matching code can then search for the first one that is set.
|
matching code can then search for the first one that is set.
|
||||||
|
|
||||||
|
|
||||||
|
@ -378,16 +387,16 @@ opcode to see if it is one of these:
|
||||||
|
|
||||||
OP_CRSTAR
|
OP_CRSTAR
|
||||||
OP_CRMINSTAR
|
OP_CRMINSTAR
|
||||||
OP_CRPOSSTAR
|
OP_CRPOSSTAR
|
||||||
OP_CRPLUS
|
OP_CRPLUS
|
||||||
OP_CRMINPLUS
|
OP_CRMINPLUS
|
||||||
OP_CRPOSPLUS
|
OP_CRPOSPLUS
|
||||||
OP_CRQUERY
|
OP_CRQUERY
|
||||||
OP_CRMINQUERY
|
OP_CRMINQUERY
|
||||||
OP_CRPOSQUERY
|
OP_CRPOSQUERY
|
||||||
OP_CRRANGE
|
OP_CRRANGE
|
||||||
OP_CRMINRANGE
|
OP_CRMINRANGE
|
||||||
OP_CRPOSRANGE
|
OP_CRPOSRANGE
|
||||||
|
|
||||||
All but the last three are single-code-unit items, with no data. The others are
|
All but the last three are single-code-unit items, with no data. The others are
|
||||||
followed by the minimum and maximum repeat counts.
|
followed by the minimum and maximum repeat counts.
|
||||||
|
@ -400,14 +409,14 @@ A pair of non-capturing round brackets is wrapped round each expression at
|
||||||
compile time, so alternation always happens in the context of brackets.
|
compile time, so alternation always happens in the context of brackets.
|
||||||
|
|
||||||
[Note for North Americans: "bracket" to some English speakers, including
|
[Note for North Americans: "bracket" to some English speakers, including
|
||||||
myself, can be round, square, curly, or pointy. Hence this usage rather than
|
myself, can be round, square, curly, or pointy. Hence this usage rather than
|
||||||
"parentheses".]
|
"parentheses".]
|
||||||
|
|
||||||
Non-capturing brackets use the opcode OP_BRA, capturing brackets use OP_CBRA.
|
Non-capturing brackets use the opcode OP_BRA, capturing brackets use OP_CBRA.
|
||||||
A bracket opcode is followed by LINK_SIZE bytes which give the offset to the
|
A bracket opcode is followed by LINK_SIZE bytes which give the offset to the
|
||||||
next alternative OP_ALT or, if there aren't any branches, to the matching
|
next alternative OP_ALT or, if there aren't any branches, to the matching
|
||||||
OP_KET opcode. Each OP_ALT is followed by LINK_SIZE bytes giving the offset to
|
OP_KET opcode. Each OP_ALT is followed by LINK_SIZE bytes giving the offset to
|
||||||
the next one, or to the OP_KET opcode. For capturing brackets, the bracket
|
the next one, or to the OP_KET opcode. For capturing brackets, the bracket
|
||||||
number is a count that immediately follows the offset.
|
number is a count that immediately follows the offset.
|
||||||
|
|
||||||
OP_KET is used for subpatterns that do not repeat indefinitely, and OP_KETRMIN
|
OP_KET is used for subpatterns that do not repeat indefinitely, and OP_KETRMIN
|
||||||
|
@ -419,8 +428,8 @@ bracket opcode.
|
||||||
If a subpattern is quantified such that it is permitted to match zero times, it
|
If a subpattern is quantified such that it is permitted to match zero times, it
|
||||||
is preceded by one of OP_BRAZERO, OP_BRAMINZERO, or OP_SKIPZERO. These are
|
is preceded by one of OP_BRAZERO, OP_BRAMINZERO, or OP_SKIPZERO. These are
|
||||||
single-unit opcodes that tell the matcher that skipping the following
|
single-unit opcodes that tell the matcher that skipping the following
|
||||||
subpattern entirely is a valid branch. In the case of the first two, not
|
subpattern entirely is a valid branch. In the case of the first two, not
|
||||||
skipping the pattern is also valid (greedy and non-greedy). The third is used
|
skipping the pattern is also valid (greedy and non-greedy). The third is used
|
||||||
when a pattern has the quantifier {0,0}. It cannot be entirely discarded,
|
when a pattern has the quantifier {0,0}. It cannot be entirely discarded,
|
||||||
because it may be called as a subroutine from elsewhere in the pattern.
|
because it may be called as a subroutine from elsewhere in the pattern.
|
||||||
|
|
||||||
|
@ -432,11 +441,11 @@ as appropriate.
|
||||||
A subpattern with a bounded maximum repetition is replicated in a nested
|
A subpattern with a bounded maximum repetition is replicated in a nested
|
||||||
fashion up to the maximum number of times, with OP_BRAZERO or OP_BRAMINZERO
|
fashion up to the maximum number of times, with OP_BRAZERO or OP_BRAMINZERO
|
||||||
before each replication after the minimum, so that, for example, (abc){2,5} is
|
before each replication after the minimum, so that, for example, (abc){2,5} is
|
||||||
compiled as (abc)(abc)((abc)((abc)(abc)?)?)?, except that each bracketed group
|
compiled as (abc)(abc)((abc)((abc)(abc)?)?)?, except that each bracketed group
|
||||||
has the same number.
|
has the same number.
|
||||||
|
|
||||||
When a repeated subpattern has an unbounded upper limit, it is checked to see
|
When a repeated subpattern has an unbounded upper limit, it is checked to see
|
||||||
whether it could match an empty string. If this is the case, the opcode in the
|
whether it could match an empty string. If this is the case, the opcode in the
|
||||||
final replication is changed to OP_SBRA or OP_SCBRA. This tells the matcher
|
final replication is changed to OP_SBRA or OP_SCBRA. This tells the matcher
|
||||||
that it needs to check for matching an empty string when it hits OP_KETRMIN or
|
that it needs to check for matching an empty string when it hits OP_KETRMIN or
|
||||||
OP_KETRMAX, and if so, to break the loop.
|
OP_KETRMAX, and if so, to break the loop.
|
||||||
|
@ -447,8 +456,8 @@ Possessive brackets
|
||||||
|
|
||||||
When a repeated group (capturing or non-capturing) is marked as possessive by
|
When a repeated group (capturing or non-capturing) is marked as possessive by
|
||||||
the "+" notation, e.g. (abc)++, different opcodes are used. Their names all
|
the "+" notation, e.g. (abc)++, different opcodes are used. Their names all
|
||||||
have POS on the end, e.g. OP_BRAPOS instead of OP_BRA and OP_SCBRAPOS instead
|
have POS on the end, e.g. OP_BRAPOS instead of OP_BRA and OP_SCBRAPOS instead
|
||||||
of OP_SCBRA. The end of such a group is marked by OP_KETRPOS. If the minimum
|
of OP_SCBRA. The end of such a group is marked by OP_KETRPOS. If the minimum
|
||||||
repetition is zero, the group is preceded by OP_BRAPOSZERO.
|
repetition is zero, the group is preceded by OP_BRAPOSZERO.
|
||||||
|
|
||||||
|
|
||||||
|
@ -456,12 +465,12 @@ Once-only (atomic) groups
|
||||||
-------------------------
|
-------------------------
|
||||||
|
|
||||||
These are just like other subpatterns, but they start with the opcode
|
These are just like other subpatterns, but they start with the opcode
|
||||||
OP_ONCE or OP_ONCE_NC. The former is used when there are no capturing brackets
|
OP_ONCE or OP_ONCE_NC. The former is used when there are no capturing brackets
|
||||||
within the atomic group; the latter when there are. The distinction is needed
|
within the atomic group; the latter when there are. The distinction is needed
|
||||||
for when there is a backtrack to before the group - any captures within the
|
for when there is a backtrack to before the group - any captures within the
|
||||||
group must be reset, so it is necessary to retain backtracking points inside
|
group must be reset, so it is necessary to retain backtracking points inside
|
||||||
the group, even after it is complete, in order to do this. When there are no
|
the group, even after it is complete, in order to do this. When there are no
|
||||||
captures in an atomic group, all the backtracking can be discarded when it is
|
captures in an atomic group, all the backtracking can be discarded when it is
|
||||||
complete. This is more efficient, and also uses less stack.
|
complete. This is more efficient, and also uses less stack.
|
||||||
|
|
||||||
The check for matching an empty string in an unbounded repeat is handled
|
The check for matching an empty string in an unbounded repeat is handled
|
||||||
|
@ -485,7 +494,7 @@ Conditional subpatterns
|
||||||
-----------------------
|
-----------------------
|
||||||
|
|
||||||
These are like other subpatterns, but they start with the opcode OP_COND, or
|
These are like other subpatterns, but they start with the opcode OP_COND, or
|
||||||
OP_SCOND for one that might match an empty string in an unbounded repeat.
|
OP_SCOND for one that might match an empty string in an unbounded repeat.
|
||||||
|
|
||||||
If the condition is a back reference, this is stored at the start of the
|
If the condition is a back reference, this is stored at the start of the
|
||||||
subpattern using the opcode OP_CREF followed by a count containing the
|
subpattern using the opcode OP_CREF followed by a count containing the
|
||||||
|
@ -500,17 +509,17 @@ group x" (coded as "(?(Rx)"), the group number is stored at the start of the
|
||||||
subpattern using the opcode OP_RREF (with a value of RREF_ANY (0xffff) for "the
|
subpattern using the opcode OP_RREF (with a value of RREF_ANY (0xffff) for "the
|
||||||
whole pattern") or OP_DNRREF (with data as for OP_DNCREF).
|
whole pattern") or OP_DNRREF (with data as for OP_DNCREF).
|
||||||
|
|
||||||
For a DEFINE condition, OP_FALSE is used (with no associated data). During
|
For a DEFINE condition, OP_FALSE is used (with no associated data). During
|
||||||
compilation, however, a DEFINE condition is coded as OP_DEFINE so that, when
|
compilation, however, a DEFINE condition is coded as OP_DEFINE so that, when
|
||||||
the conditional group is complete, there can be a check to ensure that it
|
the conditional group is complete, there can be a check to ensure that it
|
||||||
contains only one top-level branch. Once this has happened, the opcode is
|
contains only one top-level branch. Once this has happened, the opcode is
|
||||||
changed to OP_FALSE, so the matcher never sees OP_DEFINE.
|
changed to OP_FALSE, so the matcher never sees OP_DEFINE.
|
||||||
|
|
||||||
There is a special PCRE2-specific condition of the form (VERSION[>]=x.y), which
|
There is a special PCRE2-specific condition of the form (VERSION[>]=x.y), which
|
||||||
tests the PCRE2 version number. This compiles into one of the opcodes OP_TRUE
|
tests the PCRE2 version number. This compiles into one of the opcodes OP_TRUE
|
||||||
or OP_FALSE.
|
or OP_FALSE.
|
||||||
|
|
||||||
If a condition is not a back reference, recursion test, DEFINE, or VERSION, it
|
If a condition is not a back reference, recursion test, DEFINE, or VERSION, it
|
||||||
must start with an assertion, whose opcode immediately follows OP_COND or
|
must start with an assertion, whose opcode immediately follows OP_COND or
|
||||||
OP_SCOND.
|
OP_SCOND.
|
||||||
|
|
||||||
|
@ -540,9 +549,9 @@ information using automatic callouts.
|
||||||
Opcode table checking
|
Opcode table checking
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
The last opcode that is defined in pcre2_internal.h is OP_TABLE_LENGTH. This is
|
The last opcode that is defined in pcre2_internal.h is OP_TABLE_LENGTH. This is
|
||||||
not a real opcode, but is used to check that tables indexed by opcode are the
|
not a real opcode, but is used to check that tables indexed by opcode are the
|
||||||
correct length, in order to catch updating errors.
|
correct length, in order to catch updating errors.
|
||||||
|
|
||||||
Philip Hazel
|
Philip Hazel
|
||||||
August 2014
|
February 2015
|
||||||
|
|
Loading…
Reference in New Issue