Make pcre2_dfa_match() take notice of the match limit, to catch patterns that

use too much resource. This should fix oss-fuzz 1761.

ChangeLog

@@ -173,6 +173,10 @@ one: PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES.
 35. A lookbehind assertion that had a zero-length branch caused undefined 
 behaviour when processed by pcre2_dfa_match(). This is oss-fuzz issue 1859.
 
+36. The match limit value now also applies to pcre2_dfa_match() as there are 
+patterns that can use up a lot of resources without necessarily recursing very 
+deeply. (Compare item 10.23/36.) This should fix oss-fuzz #1761.
+
 
 Version 10.23 14-February-2017
 ------------------------------

doc/html/pcre2_dfa_match.html

@@ -46,8 +46,9 @@ just once (except when processing lookaround assertions). This function is
   <i>wscount</i>      Number of elements in the vector
 </pre>
 For <b>pcre2_dfa_match()</b>, a match context is needed only if you want to set
-up a callout function or specify the recursion depth limit. The <i>length</i>
+up a callout function or specify the match and/or the recursion depth limits.
-and <i>startoffset</i> values are code units, not characters. The options are:
+The <i>length</i> and <i>startoffset</i> values are code units, not characters.
+The options are:
 <pre>
   PCRE2_ANCHORED          Match only at the first position
   PCRE2_ENDANCHORED       Pattern can match only at end of subject

doc/html/pcre2api.html

@@ -954,8 +954,8 @@ time round its main matching loop. If this value reaches the match limit,
 <b>pcre2_match()</b> returns the negative value PCRE2_ERROR_MATCHLIMIT. This has
 the effect of limiting the amount of backtracking that can take place. For
 patterns that are not anchored, the count restarts from zero for each position
-in the subject string. This limit is not relevant to <b>pcre2_dfa_match()</b>,
+in the subject string. This limit also applies to <b>pcre2_dfa_match()</b>,
-which ignores it.
+though the counting is done in a different way.
 </P>
 <P>
 When <b>pcre2_match()</b> is called with a pattern that was successfully
@@ -974,8 +974,8 @@ of the form
   (*LIMIT_MATCH=ddd)
 </pre>
 where ddd is a decimal number. However, such a setting is ignored unless ddd is
-less than the limit set by the caller of <b>pcre2_match()</b> or, if no such
+less than the limit set by the caller of <b>pcre2_match()</b> or
-limit is set, less than the default.
+<b>pcre2_dfa_match()</b> or, if no such limit is set, less than the default.
 <br>
 <br>
 <b>int pcre2_set_depth_limit(pcre2_match_context *<i>mcontext</i>,</b>
@@ -3471,7 +3471,7 @@ Cambridge, England.
 </P>
 <br><a name="SEC42" href="#TOC1">REVISION</a><br>
 <P>
-Last updated: 26 May 2017
+Last updated: 30 May 2017
 <br>
 Copyright &copy; 1997-2017 University of Cambridge.
 <br>

doc/html/pcre2build.html

@@ -260,9 +260,9 @@ setting such as
 <pre>
   --with-match-limit=500000
 </pre>
-to the <b>configure</b> command. This setting has no effect on the
+to the <b>configure</b> command. This setting also applies to the
-<b>pcre2_dfa_match()</b> matching function, but it does also limit JIT matching 
+<b>pcre2_dfa_match()</b> matching function, and to JIT matching (though the
-(though the counting is done differently).
+counting is done differently).
 </P>
 <P>
 The <b>pcre2_match()</b> function starts out using a 20K vector on the system
@@ -554,7 +554,7 @@ Cambridge, England.
 </P>
 <br><a name="SEC25" href="#TOC1">REVISION</a><br>
 <P>
-Last updated: 10 April 2017
+Last updated: 30 May 2017
 <br>
 Copyright &copy; 1997-2017 University of Cambridge.
 <br>

doc/html/pcre2pattern.html

@@ -204,11 +204,11 @@ still recognized for backwards compatibility.
 <P>
 The heap limit applies only when the <b>pcre2_match()</b> interpreter is used
 for matching. It does not apply to JIT or DFA matching. The match limit is used
-(but in a different way) when JIT is being used, but it is not relevant, and is
+(but in a different way) when JIT is being used, or when
-ignored, when matching with <b>pcre2_dfa_match()</b>. The depth limit is ignored 
+<b>pcre2_dfa_match()</b> is called, to limit computing resource usage by those
-by JIT but is relevant for DFA matching, which uses function recursion for
+matching functions. The depth limit is ignored by JIT but is relevant for DFA
-recursions within the pattern. In this case, the depth limit controls the
+matching, which uses function recursion for recursions within the pattern. In
-amount of system stack that is used.
+this case, the depth limit controls the amount of system stack that is used.
 <a name="newlines"></a></P>
 <br><b>
 Newline conventions
@@ -3445,7 +3445,7 @@ Cambridge, England.
 </P>
 <br><a name="SEC30" href="#TOC1">REVISION</a><br>
 <P>
-Last updated: 26 May 2017
+Last updated: 30 May 2017
 <br>
 Copyright &copy; 1997-2017 University of Cambridge.
 <br>

doc/pcre2.txt

@@ -963,8 +963,8 @@ PCRE2 CONTEXTS
        limit, pcre2_match() returns the negative value PCRE2_ERROR_MATCHLIMIT.
        This has the effect of limiting the amount  of  backtracking  that  can
        take place. For patterns that are not anchored, the count restarts from
-       zero for each position in the subject string. This limit is  not  rele-
+       zero for each position in the subject string. This limit  also  applies
-       vant to pcre2_dfa_match(), which ignores it.
+       to pcre2_dfa_match(), though the counting is done in a different way.
 
        When  pcre2_match() is called with a pattern that was successfully pro-
        cessed by pcre2_jit_compile(), the way in which matching is executed is
@@ -981,8 +981,8 @@ PCRE2 CONTEXTS
          (*LIMIT_MATCH=ddd)
 
        where  ddd  is  a  decimal  number.  However, such a setting is ignored
-       unless ddd is less than the limit set by the  caller  of  pcre2_match()
+       unless ddd is less than the limit set by the caller of pcre2_match() or
-       or, if no such limit is set, less than the default.
+       pcre2_dfa_match() or, if no such limit is set, less than the default.
 
        int pcre2_set_depth_limit(pcre2_match_context *mcontext,
          uint32_t value);
@@ -3350,7 +3350,7 @@ AUTHOR
 
 REVISION
 
-       Last updated: 26 May 2017
+       Last updated: 30 May 2017
        Copyright (c) 1997-2017 University of Cambridge.
 ------------------------------------------------------------------------------
  
@@ -3586,9 +3586,9 @@ LIMITING PCRE2 RESOURCE USAGE
 
          --with-match-limit=500000
 
-       to   the   configure  command.  This  setting  has  no  effect  on  the
+       to   the   configure   command.   This  setting  also  applies  to  the
-       pcre2_dfa_match() matching function, but it does also limit JIT  match-
+       pcre2_dfa_match() matching function, and to JIT  matching  (though  the
-       ing (though the counting is done differently).
+       counting is done differently).
 
        The  pcre2_match() function starts out using a 20K vector on the system
        stack to record  backtracking  points.  The  more  nested  backtracking
@@ -3885,7 +3885,7 @@ AUTHOR
 
 REVISION
 
-       Last updated: 10 April 2017
+       Last updated: 30 May 2017
        Copyright (c) 1997-2017 University of Cambridge.
 ------------------------------------------------------------------------------
  
@@ -5751,11 +5751,12 @@ SPECIAL START-OF-PATTERN ITEMS
 
        The heap limit applies only when the pcre2_match() interpreter is  used
        for matching. It does not apply to JIT or DFA matching. The match limit
-       is used (but in a different way) when JIT is being used, but it is  not
+       is used (but in a different way)  when  JIT  is  being  used,  or  when
-       relevant,  and  is  ignored,  when matching with pcre2_dfa_match(). The
+       pcre2_dfa_match() is called, to limit computing resource usage by those
-       depth limit is ignored by JIT but is relevant for DFA  matching,  which
+       matching functions. The depth limit is ignored by JIT but  is  relevant
-       uses  function  recursion  for  recursions  within the pattern. In this
+       for  DFA  matching, which uses function recursion for recursions within
-       case, the depth limit controls the amount of system stack that is used.
+       the pattern. In this case, the depth limit controls the amount of  sys-
+       tem stack that is used.
 
    Newline conventions
 
@@ -8686,7 +8687,7 @@ AUTHOR
 
 REVISION
 
-       Last updated: 26 May 2017
+       Last updated: 30 May 2017
        Copyright (c) 1997-2017 University of Cambridge.
 ------------------------------------------------------------------------------
  

doc/pcre2_dfa_match.3

@@ -1,4 +1,4 @@
-.TH PCRE2_DFA_MATCH 3 "04 April 2017" "PCRE2 10.30"
+.TH PCRE2_DFA_MATCH 3 "30 May 2017" "PCRE2 10.30"
 .SH NAME
 PCRE2 - Perl-compatible regular expressions (revised API)
 .SH SYNOPSIS
@@ -34,8 +34,9 @@ just once (except when processing lookaround assertions). This function is
   \fIwscount\fP      Number of elements in the vector
 .sp
 For \fBpcre2_dfa_match()\fP, a match context is needed only if you want to set
-up a callout function or specify the recursion depth limit. The \fIlength\fP
+up a callout function or specify the match and/or the recursion depth limits.
-and \fIstartoffset\fP values are code units, not characters. The options are:
+The \fIlength\fP and \fIstartoffset\fP values are code units, not characters.
+The options are:
 .sp
   PCRE2_ANCHORED          Match only at the first position
   PCRE2_ENDANCHORED       Pattern can match only at end of subject

doc/pcre2api.3

@@ -1,4 +1,4 @@
-.TH PCRE2API 3 "26 May 2017" "PCRE2 10.30"
+.TH PCRE2API 3 "30 May 2017" "PCRE2 10.30"
 .SH NAME
 PCRE2 - Perl-compatible regular expressions (revised API)
 .sp
@@ -891,8 +891,8 @@ time round its main matching loop. If this value reaches the match limit,
 \fBpcre2_match()\fP returns the negative value PCRE2_ERROR_MATCHLIMIT. This has
 the effect of limiting the amount of backtracking that can take place. For
 patterns that are not anchored, the count restarts from zero for each position
-in the subject string. This limit is not relevant to \fBpcre2_dfa_match()\fP,
+in the subject string. This limit also applies to \fBpcre2_dfa_match()\fP,
-which ignores it.
+though the counting is done in a different way.
 .P
 When \fBpcre2_match()\fP is called with a pattern that was successfully
 processed by \fBpcre2_jit_compile()\fP, the way in which matching is executed
@@ -909,8 +909,8 @@ of the form
   (*LIMIT_MATCH=ddd)
 .sp
 where ddd is a decimal number. However, such a setting is ignored unless ddd is
-less than the limit set by the caller of \fBpcre2_match()\fP or, if no such
+less than the limit set by the caller of \fBpcre2_match()\fP or
-limit is set, less than the default.
+\fBpcre2_dfa_match()\fP or, if no such limit is set, less than the default.
 .sp
 .nf
 .B int pcre2_set_depth_limit(pcre2_match_context *\fImcontext\fP,
@@ -3491,6 +3491,6 @@ Cambridge, England.
 .rs
 .sp
 .nf
-Last updated: 26 May 2017
+Last updated: 30 May 2017
 Copyright (c) 1997-2017 University of Cambridge.
 .fi

doc/pcre2build.3

@@ -1,4 +1,4 @@
-.TH PCRE2BUILD 3 "10 April 2017" "PCRE2 10.30"
+.TH PCRE2BUILD 3 "30 May 2017" "PCRE2 10.30"
 .SH NAME
 PCRE2 - Perl-compatible regular expressions (revised API)
 .
@@ -256,9 +256,9 @@ setting such as
 .sp
   --with-match-limit=500000
 .sp
-to the \fBconfigure\fP command. This setting has no effect on the
+to the \fBconfigure\fP command. This setting also applies to the
-\fBpcre2_dfa_match()\fP matching function, but it does also limit JIT matching 
+\fBpcre2_dfa_match()\fP matching function, and to JIT matching (though the
-(though the counting is done differently).
+counting is done differently).
 .P
 The \fBpcre2_match()\fP function starts out using a 20K vector on the system
 stack to record backtracking points. The more nested backtracking points there
@@ -572,6 +572,6 @@ Cambridge, England.
 .rs
 .sp
 .nf
-Last updated: 10 April 2017
+Last updated: 30 May 2017
 Copyright (c) 1997-2017 University of Cambridge.
 .fi

doc/pcre2pattern.3

@@ -1,4 +1,4 @@
-.TH PCRE2PATTERN 3 "26 May 2017" "PCRE2 10.30"
+.TH PCRE2PATTERN 3 "30 May 2017" "PCRE2 10.30"
 .SH NAME
 PCRE2 - Perl-compatible regular expressions (revised API)
 .SH "PCRE2 REGULAR EXPRESSION DETAILS"
@@ -169,11 +169,11 @@ still recognized for backwards compatibility.
 .P
 The heap limit applies only when the \fBpcre2_match()\fP interpreter is used
 for matching. It does not apply to JIT or DFA matching. The match limit is used
-(but in a different way) when JIT is being used, but it is not relevant, and is
+(but in a different way) when JIT is being used, or when
-ignored, when matching with \fBpcre2_dfa_match()\fP. The depth limit is ignored 
+\fBpcre2_dfa_match()\fP is called, to limit computing resource usage by those
-by JIT but is relevant for DFA matching, which uses function recursion for
+matching functions. The depth limit is ignored by JIT but is relevant for DFA
-recursions within the pattern. In this case, the depth limit controls the
+matching, which uses function recursion for recursions within the pattern. In
-amount of system stack that is used.
+this case, the depth limit controls the amount of system stack that is used.
 .
 .
 .\" HTML <a name="newlines"></a>
@@ -3475,6 +3475,6 @@ Cambridge, England.
 .rs
 .sp
 .nf
-Last updated: 26 May 2017
+Last updated: 30 May 2017
 Copyright (c) 1997-2017 University of Cambridge.
 .fi

src/pcre2_dfa_match.c

@@ -396,6 +396,7 @@ BOOL utf = FALSE;
 
 BOOL reset_could_continue = FALSE;
 
+if (mb->match_call_count++ >= mb->match_limit) return PCRE2_ERROR_MATCHLIMIT;
 if (rlevel++ > mb->match_limit_depth) return PCRE2_ERROR_DEPTHLIMIT;
 offsetcount &= (uint32_t)(-2);  /* Round down */
 
@@ -3218,6 +3219,7 @@ if (mcontext == NULL)
   {
   mb->callout = NULL;
   mb->memctl = re->memctl;
+  mb->match_limit = PRIV(default_match_context).match_limit;
   mb->match_limit_depth = PRIV(default_match_context).depth_limit;
   }
 else
@@ -3231,8 +3233,13 @@ else
   mb->callout = mcontext->callout;
   mb->callout_data = mcontext->callout_data;
   mb->memctl = mcontext->memctl;
+  mb->match_limit = mcontext->match_limit;
   mb->match_limit_depth = mcontext->depth_limit;
   }
+
+if (mb->match_limit > re->limit_match)
+  mb->match_limit = re->limit_match;
+
 if (mb->match_limit_depth > re->limit_depth)
   mb->match_limit_depth = re->limit_depth;
 
@@ -3244,6 +3251,7 @@ mb->end_subject = end_subject;
 mb->start_offset = start_offset;
 mb->moptions = options;
 mb->poptions = re->overall_options;
+mb->match_call_count = 0;
 
 /* Process the \R and newline settings. */
 

src/pcre2_fuzzsupport.c

@@ -178,20 +178,20 @@ for (i = 0; i < 2; i++)
         return 0;
         }
       (void)pcre2_set_match_limit(match_context, 100);
+      (void)pcre2_set_depth_limit(match_context, 100);
       (void)pcre2_set_callout(match_context, callout_function, &callout_count);
       }
 
-    /* Match twice, with and without options, with a depth limit of 100. */
+    /* Match twice, with and without options. */
-
-    (void)pcre2_set_depth_limit(match_context, 100);
 
     for (j = 0; j < 2; j++)
       {
 #ifdef STANDALONE
       printf("Match options %.8x", match_options);
-      printf("%s%s%s%s%s%s%s%s%s\n",
+      printf("%s%s%s%s%s%s%s%s%s%s\n",
         ((match_options & PCRE2_ANCHORED) != 0)? ",anchored" : "",
         ((match_options & PCRE2_ENDANCHORED) != 0)? ",endanchored" : "",
+        ((match_options & PCRE2_NO_JIT) != 0)? ",no_jit" : "",
         ((match_options & PCRE2_NO_UTF_CHECK) != 0)? ",no_utf_check" : "",
         ((match_options & PCRE2_NOTBOL) != 0)? ",notbol" : "",
         ((match_options & PCRE2_NOTEMPTY) != 0)? ",notempty" : "",
@@ -217,9 +217,8 @@ for (i = 0; i < 2; i++)
       match_options = 0;  /* For second time */
       }
 
-    /* Match with DFA twice, with and without options, depth limit of 10. */
+    /* Match with DFA twice, with and without options. */
 
-    (void)pcre2_set_depth_limit(match_context, 10);
     match_options = save_match_options & ~PCRE2_NO_JIT;  /* Not valid for DFA */
 
     for (j = 0; j < 2; j++)

src/pcre2_intmodedep.h

@@ -877,7 +877,9 @@ typedef struct dfa_match_block {
   PCRE2_SPTR last_used_ptr;       /* Latest consulted character */
   const uint8_t *tables;          /* Character tables */
   PCRE2_SIZE start_offset;        /* The start offset value */
+  uint32_t match_limit;           /* As it says */
   uint32_t match_limit_depth;     /* As it says */
+  uint32_t match_call_count;      /* Number of calls of internal function */
   uint32_t moptions;              /* Match options */
   uint32_t poptions;              /* Pattern options */
   uint32_t nltype;                /* Newline type */

src/pcre2test.c

@@ -7054,17 +7054,15 @@ else for (gmatched = 0;; gmatched++)
     {
     capcount = 0;  /* This stops compiler warnings */
 
-    if ((dat_datctl.control & CTL_DFA) == 0)
+    if ((dat_datctl.control & CTL_DFA) == 0 &&
+        (FLD(compiled_code, executable_jit) == NULL ||
+          (dat_datctl.options & PCRE2_NO_JIT) != 0))
       {     
-      if (FLD(compiled_code, executable_jit) == NULL ||
+      (void)check_match_limit(pp, arg_ulen, PCRE2_ERROR_HEAPLIMIT, "heap");
-          (dat_datctl.options & PCRE2_NO_JIT) != 0)
-        {
-        (void)check_match_limit(pp, arg_ulen, PCRE2_ERROR_HEAPLIMIT,
-          "heap");
       } 
+       
     capcount = check_match_limit(pp, arg_ulen, PCRE2_ERROR_MATCHLIMIT,
       "match");
-      }
 
     if (FLD(compiled_code, executable_jit) == NULL ||
         (dat_datctl.options & PCRE2_NO_JIT) != 0 ||

testdata/testinput6

@@ -4941,4 +4941,7 @@
 /(?<=|abc)/endanchored
     abcde\=aftertext
 
+/(*LIMIT_MATCH=100).*(?![|H]?.*(?![|H]?););.*(?![|H]?.*(?![|H]?););\x00\x00\x00\x00\x00\x00\x00(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?![|);)?.*(![|H]?);)?.*(?![|H]?);)?.*(?![|H]?);)?.*(?![|H]););![|H]?););[|H]?);|H]?);)\x00\x00\x00\x00\x00\x00H]?););?![|H]?);)?.*(?![|H]?););[||H]?);)?.*(?![|H]?););[|H]?);(?![|H]?););![|H]?););[|H]?);|H]?);)?.*(?![|H]?););;[\x00\x00\x00\x00\x00\x00\x00![|H]?););![|H]?););[|H]?);|H]?);)?.*(?![|H]?););/no_dotstar_anchor
+.*(?![|H]?.*(?![|H]?););.*(?![|H]?.*(?![|H]?););\x00\x00\x00\x00\x00\x00\x00(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?![|);)?.*(![|H]?);)?.*(?![|H]?);)?.*(?![|H]?);)?.*(?![|H]););![|H]?););[|H]?);|H]?);)\x00\x00\x00\x00\x00\x00H]?););?![|H]?);)?.*(?![|H]?););[||H]?);)?.*(?![|H]?););[|H]?);(?![|H]?););![|H]?););[|H]?);|H]?);)?.*(?![|H]?););;[\x00\x00\x00\x00\x00\x00\x00![|H]?););![|H]?););[|H]?);|H]?);)?.*(?![|H]?););
+
 # End of testinput6

testdata/testoutput6

@@ -7691,6 +7691,7 @@ Failed: error -53: matching depth limit exceeded
 
 /^(a(?2))(b)(?1)/
     abbab\=find_limits 
+Minimum match limit = 4
 Minimum depth limit = 2
  0: abbab
 
@@ -7766,4 +7767,8 @@ No match
  0: 
  0+ 
 
+/(*LIMIT_MATCH=100).*(?![|H]?.*(?![|H]?););.*(?![|H]?.*(?![|H]?););\x00\x00\x00\x00\x00\x00\x00(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?![|);)?.*(![|H]?);)?.*(?![|H]?);)?.*(?![|H]?);)?.*(?![|H]););![|H]?););[|H]?);|H]?);)\x00\x00\x00\x00\x00\x00H]?););?![|H]?);)?.*(?![|H]?););[||H]?);)?.*(?![|H]?););[|H]?);(?![|H]?););![|H]?););[|H]?);|H]?);)?.*(?![|H]?););;[\x00\x00\x00\x00\x00\x00\x00![|H]?););![|H]?););[|H]?);|H]?);)?.*(?![|H]?););/no_dotstar_anchor
+.*(?![|H]?.*(?![|H]?););.*(?![|H]?.*(?![|H]?););\x00\x00\x00\x00\x00\x00\x00(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?!(?![|);)?.*(![|H]?);)?.*(?![|H]?);)?.*(?![|H]?);)?.*(?![|H]););![|H]?););[|H]?);|H]?);)\x00\x00\x00\x00\x00\x00H]?););?![|H]?);)?.*(?![|H]?););[||H]?);)?.*(?![|H]?););[|H]?);(?![|H]?););![|H]?););[|H]?);|H]?);)?.*(?![|H]?););;[\x00\x00\x00\x00\x00\x00\x00![|H]?););![|H]?););[|H]?);|H]?);)?.*(?![|H]?););
+Failed: error -47: match limit exceeded
+
 # End of testinput6