Commit Graph

1053 Commits

Author SHA1 Message Date
Philip Hazel 1c41a5b815 Fix minor issues raised by Clang sanitize 2021-12-14 15:52:24 +00:00
Zoltan Herczeg 4243515033 JIT support for Bidi_Control and Bidi_Class 2021-12-13 07:04:19 +00:00
Philip Hazel 49b29f837d Add short synonyms for Bidi_Control and Bidi_Class 2021-12-10 16:32:10 +00:00
Philip Hazel 0246c6bf64 Add support for Bidi_Control and Bidi_Class properties 2021-12-08 15:34:27 +00:00
Philip Hazel 823d4ac956 Add bidi class and control information to Unicode property data 2021-12-05 18:00:10 +00:00
Philip Hazel 4ef0c51d2b Interpret NULL pointer, zero length as an empty string for subjects and replacements. 2021-11-30 16:34:39 +00:00
Philip Hazel 7ab2769728 Check for NULL replacement in pcre2_substitute() 2021-11-28 17:19:17 +00:00
Philip Hazel 2a294ddadb Add check for NULL subject to POSIX regexec(). 2021-11-28 16:38:36 +00:00
Philip Hazel cb854a912e Add options for NULL pointers to pcre2test. 2021-11-28 16:22:24 +00:00
Carlo Marcelo Arenas Belón ae4e6261e5
match: avoid crash if subject NULL and PCRE2_ZERO_TERMINATED (#53)
* pcre2_match: avoid crash if subject NULL and PCRE2_ZERO_TERMINATED

When length of subject is PCRE2_ZERO_TERMINATED strlen is used
to calculate its size, which will trigger a crash if subject is
also NULL.

Move the NULL check before strlen on it would be used, and make
sure or dependent variables are set after the NULL validation
as well.

While at it, fix a typo in a debug flag in the same file, which
is otherwise unrelated and make sure the full section of constrain
checks can be identified clearly using the leading comment alone.

* pcre2_dfa_match: avoid crash if subject NULL and PCRE2_ZERO_TERMINATED

When length of subject is PCRE2_ZERO_TERMINATED strlen is used
to calculate its size, which will trigger a crash if subject is
also NULL.

Move the NULL check before the detection for subject sizes to
avoid this issue.

* pcre2_substitute: avoid crash if subject or replacement are NULL

The underlying pcre2_match() function will validate the subject if
needed, but will crash when length is PCRE2_ZERO_TERMINATED or if
subject == NULL and pcre2_match() is not being called because
match_data was provided.

The replacement parameter is missing NULL checks, and so currently
allows for an equivalent response to "" if rlength == 0.

Restrict all other cases to avoid strlen(NULL) crashes in the same
way that is done for subject, but also make sure to reject invalid
length values as early as possible.
2021-11-27 16:49:31 +00:00
Carlo Marcelo Arenas Belón 055b7ce4a9
pcre2grep: remove JFRIEDL_DEBUG obsoleted code (#49)
Still uses the already obsoleted PCRE1 API

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
2021-11-27 16:36:17 +00:00
Carlo Marcelo Arenas Belón 587b94277b
doc: formatting/typo fixes to documentation (#47)
* doc: fix incorrect use of JOIN and typo

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>

* doc: reformat of pcre2_substitute to align options

includes some rewording to fit better in an 80 char wide troff output.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>

* doc: update names to pcre2
2021-11-27 16:27:49 +00:00
Zoltan Herczeg d144199dfb
Revert an unintended change in JIT repeat detection. (#58)
Co-authored-by: Zoltan Herczeg <hzmester@freemail.hu>
2021-11-24 16:58:30 +00:00
Carlo Marcelo Arenas Belón eb42305f07
jit: avoid integer wraparound in stack size definition (#42)
pcre2_jit_stack_create() allows the user to indicate how big of a
stack size JIT should be able to allocate and use, using a size_t
variable which should be able to hold bigger values than reasonable.

Internally, the value is rounded to the next 8K, but if the value
is unreasonable large, would overflow and could result in a smaller
than expected stack or a maximun size that is smaller than the
minimum..

Avoid the overflow by checking the value and failing early, and
while at it make the check clearer while documenting the failure
mode.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
2021-11-19 09:23:46 +01:00
Carlo Marcelo Arenas Belón acc520924c
test: avoid failing RunTest if pcre2test -S is not supported (#37)
* test: avoid failing RunTest if pcre2test -S is not supported

If `pcre2test -S` is not supported then then avoid checking for it
in a test.

There is already a conditional check for it to be used when it is
needed and it is available, so adjust that as well.

* pcre2test: update list of platform support for -S

Minix 3 has a BSD userspace and now works fine, but Haiku still
doesn't support stack limits, so update accordingly.
2021-11-09 17:23:02 +00:00
Carlo Marcelo Arenas Belón dae475092d
pcre2grep: avoid portability minefield with buffered fseek(stdin) (#36)
To allow pcre2grep to do an early exit in a resumable way, -m uses
fseek on stdin, which is sadly not supported in several platforms.

Most of the conflicting issues come from the fact that managing the
position while buffering is not trivial, and is therefore an optional
feature[1] of POSIX.1-2017

Workaround this by removing the buffer to stdin, if the -m option is
being used.  There is likely not a significant performance benefit
even for the platforms that support it, but it could be conditionally
added in that case, later.

Fixes: #10

[1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/fseek.html
2021-11-09 17:15:38 +00:00
Carlo Marcelo Arenas Belón 7db8784296
pcre2grep: correctly handle multiple passes (#35)
* tests: use a explicit filehandle to share in testing -m

The way stdin is shared to all participants of a subshell varies
per shell, and at least the standard /bin/sh in Solaris seem to
create a new copy for each command, defeating the purpose of the
test.

Use instead exec to create a filehandle that could then be used
explicitly in the test to confirm that the stream is set.

* pcre2grep: correctly handle multiple passes

When the -m option is used, pcre2grep is meant to exit after enough
matches are found but while leaving the stream pinned to the next position
after the last match.

Unfortunately, it wasn't tracking correctly the beginning of the stream
on subsequent passes, and therefore it will fail to use the right seek
value.

Grab the position of the stream at the beginning and while at it, make
sure that the stream passed hasn't been consumed already.
2021-11-09 16:57:48 +00:00
Philip Hazel 35fee4193b Final file tidies for 10.39. 2021-10-29 17:09:37 +01:00
Carlo Marcelo Arenas Belón 128c50360c
fix building on ancient compilers (#32)
* jit: allow building with ancient MSVC versions

Visual Studio older than 2013, fails to build with JIT enabled,
because it is unable to parse non C89 compatible syntax, with
mixed declarations and code.

While most recent compilers wouldn't even report this as a warning
since it is valid C99, it could be also made visible by adding to
gcc/clang the -Wdeclaration-after-statement flag at build time.

Move the code below the affected definitions.

* pcre2grep: avoid mixing declarations with code

Since d5a61ee8 (Patch to detect (and ignore) symlink loops in
pcre2grep., 2021-08-28), code will fail to build in a strict C89
compiler.

Reformat slightly to make it C89 compatible again.
2021-10-29 16:07:53 +01:00
Philip Hazel bf2c8cc564 Update ChangeLog for GitHub commits and generate HTML docs. 2021-10-29 15:12:56 +01:00
Carlo Marcelo Arenas Belón 3b973ebf4b
inttypes and stdint cleanup (#30)
* cleanup: remove references to no longer used stdint.h

Since 19c50b9d (Unconditionally use inttypes.h instead of trying for
stdint.h (simplification) and remove the now unnecessary inclusion in
pcre2_internal.h., 2018-11-14), stdint.h is no longer used.

Remove checks for it in autotools and CMake and document better the
expected build failures for systems that might have stdint.h (C99)
and not inttypes.h (from POSIX), like old Windows.

* cleanup: remove detection for inttypes.h which is a hard dependency

CMake checks for standard headers are not meant to be used for hard
dependencies, so will prevent a possible fallback to work.

Alternatively, the header could be checked to make the configuration
fail instead of breaking the build, but that was punted, as it was
missing anyway from autotools.
2021-10-29 15:05:19 +01:00
Carlo Marcelo Arenas Belón f5e4e10042
Update to Unicode 14.0.0 (#29) 2021-10-29 14:44:17 +01:00
Carlo Marcelo Arenas Belón d46f1863be
improve on 'auto' mode for DISABLE_PERCENT_ZT (#28)
Visual Studio 2013 includes support for %zu and %td, so let newer
versions of it avoid the fallback, and while at it, make sure that
the first check is for DISABLE_PERCENT_ZT so it will be always
honoured if chosen.

prtdiff_t is signed, so use a signed type instead, and make sure
that an appropiate width is chosen if pointers are 64bit wide.

Remove the need for the size_t cast and instead change the size
of the equivalent format identifier to avoid truncations.
2021-10-29 14:29:47 +01:00
Zoltan Herczeg 51ec2c9893 Fix incorrect detection of alternatives in first character search. 2021-10-05 16:41:13 +00:00
Philip Hazel 0612ed77c2 Update dates and tidy files for 10.38 release. 2021-10-01 16:44:09 +01:00
Philip Hazel 507e4dcf6f Fix incorrect loop counter reset when setting lookbehind lengths (ClusterFuzz
38653).
2021-09-14 16:10:20 +01:00
Zoltan Herczeg dc5f966635 Re-enable an optimization which was unintentionally disabled in 10.35. 2021-09-10 10:00:47 +00:00
Philip Hazel 8f3e11a355 Doc file tidies for 10.38-RC1 2021-08-31 17:14:42 +01:00
Philip Hazel 31a46200fa Add --allow-lookaround-bsk to pcre2grep 2021-08-31 16:24:25 +01:00
Philip Hazel 21c26698b3 Lock out \K in lookaround assertions by default, but provide an option to
re-enable the old behaviour, just in case.
2021-08-30 16:57:44 +01:00
Philip Hazel eea410b33a Improve code for "starts with" optimization in the interpreters. 2021-08-29 17:25:59 +01:00
Philip Hazel d5a61ee891 Patch to detect (and ignore) symlink loops in pcre2grep. 2021-08-28 17:37:33 +01:00
Philip.Hazel d70da76dfb Fix some documentation typos. 2021-06-14 10:53:18 +00:00
Zoltán Herczeg a5389db88d Fix invalid single character repetition issues in JIT. 2021-06-04 12:55:49 +00:00
Zoltán Herczeg 900921f83e Minor improvement for s390x SIMD. 2021-05-30 04:56:42 +00:00
Zoltán Herczeg 1951243b5d JIT compiler update 2021-05-27 08:11:15 +00:00
Philip.Hazel 1c3256349f Final source changes for 10.37 2021-05-26 14:34:14 +00:00
Philip.Hazel cd45050ee4 Final file tidies for 10.37-RC1 2021-04-28 16:44:51 +00:00
Philip.Hazel 080d7789eb Remove real POSIX function names from the pcre2-posix library. 2021-04-27 08:36:47 +00:00
Zoltán Herczeg 38dbea6200 JIT compiler update. 2021-04-26 05:56:04 +00:00
Zoltán Herczeg 0dd0283b17 Prefer single character optimization over bracket repetition in JIT. 2021-02-19 11:58:36 +00:00
Zoltán Herczeg 19a1319c0a Restore single character repetition optimization in JIT. 2021-02-19 09:20:37 +00:00
Philip.Hazel 2c4d3942e4 Fix \K within recursion bug in interpreter. 2021-02-18 09:46:08 +00:00
Zoltán Herczeg b6acebe497 JIT compiler update. 2021-02-12 07:45:12 +00:00
Philip.Hazel 6cb388d55b Update version number for next release. 2021-02-02 09:24:05 +00:00
Philip.Hazel 8144ae04e9 Fix some numerical checking bugs, Bugzilla 2690. 2021-02-01 17:56:12 +00:00
Philip.Hazel c246f53ae1 Get rid of gcc -fanalyzer error (though it was probably a false positive). 2021-01-14 16:56:44 +00:00
Zoltán Herczeg e5e1fab2db Add fast forward requested char simd support for s390x. 2021-01-10 14:51:53 +00:00
Zoltán Herczeg b730793117 Add fast forward char pair simd support for s390x. 2021-01-09 13:41:29 +00:00
Zoltán Herczeg 46158a811f Add fast forward char simd support for s390x. 2021-01-06 07:52:05 +00:00
Philip.Hazel 7eb23f423e Final file tidies for 10.36 2020-12-04 14:30:03 +00:00
Zoltán Herczeg d19789c251 Fix ARM64 compilation warning in JIT. 2020-11-13 08:04:06 +00:00
Philip.Hazel 000bbf2ea7 File tidies for 10.36-RC1 2020-11-06 17:27:35 +00:00
Zoltán Herczeg fb54d81528 JIT compiler update. 2020-11-03 13:20:09 +00:00
Zoltán Herczeg 2451870e3c Fixed a word boundary check bug in JIT when partial matching is enabled. 2020-10-27 08:16:04 +00:00
Zoltán Herczeg 37b76d8609 JIT compiler update. 2020-10-19 06:20:18 +00:00
Philip.Hazel fff544a1e9 Fix potential memory leak in error situation in recent new code. 2020-10-06 08:04:40 +00:00
Philip.Hazel 81da2b97e3 pcre2grep update: -m and $x{..}, $o{..} escapes. Also some doc updates. 2020-10-04 16:34:31 +00:00
Zoltán Herczeg 3bdc76e4f3 Fixed a bug in character set matching when JIT is enabled. 2020-09-19 03:49:32 +00:00
Philip.Hazel f8cbb1f58d Fix Bugzilla #2642: no match bug in 8-bit mode for caseless invalid utf
matching.
2020-09-15 14:36:23 +00:00
Philip.Hazel a2f0fd01c7 Update pcre2test to check delimiters after #perltest and fix some in test 1. 2020-09-14 15:39:39 +00:00
Zoltán Herczeg 384620a172 JIT compiler update. 2020-08-27 06:19:17 +00:00
Zoltán Herczeg 3d317692ac Fix an early fail optimization issue and a buffer overread in JIT. 2020-07-15 04:35:32 +00:00
Philip.Hazel 0ad89ab06d Fix read overflow for invalid VERSION test with one fractional digit at the end
of a pattern. Fixes ClusterFuzz 23779.
2020-06-29 15:35:49 +00:00
Philip.Hazel 3faff02596 Add cast to prevent a compiler warning. 2020-06-05 16:11:01 +00:00
Zoltán Herczeg fda3221597 Guard update after r1260. 2020-06-02 16:54:25 +00:00
Zoltán Herczeg 0652de5597 Add SIMD support for fast forward newline in JIT. 2020-06-02 10:54:37 +00:00
Zoltán Herczeg e0c6029a62 Fix inifinite loop when a single byte newline is searched in JIT. 2020-05-29 14:20:23 +00:00
Philip.Hazel 768c7fe67e Final file tidies for 10.35. 2020-05-09 15:47:41 +00:00
Zoltán Herczeg 018044a54e Force match limit for JIT tests. 2020-05-06 11:18:31 +00:00
Philip.Hazel 56c4bf9095 Check for memfd_create in configuration files. 2020-04-28 15:03:58 +00:00
Philip.Hazel ce558bbff1 Second attempt at getting rid of gcc 10 warning. 2020-04-24 15:36:53 +00:00
Philip.Hazel 5ec5c45423 Added tests for __attribute__((uninitialized)) to both the configure and
CMake build files. Used to disable initialization of the match stack frames
vector (clang has an automatic initialization feature).
2020-04-23 16:50:45 +00:00
Philip.Hazel ca55d0be6b Avoid using [-1] as a suffix in pcre2test as it can provoke a compiler warning. 2020-04-23 15:41:23 +00:00
Philip.Hazel 8b3f8af535 File tidies for 10.35-RC1 release candidate. 2020-04-15 16:34:36 +00:00
Zoltán Herczeg cf670e3bb9 JIT compiler update. 2020-04-14 05:04:32 +00:00
Philip.Hazel c472f3f91a Update to Unicode 13.0.0. 2020-03-25 17:18:33 +00:00
Philip.Hazel f988433788 Fix resource leak in pcre2test introduced by recent patch. 2020-03-24 17:25:58 +00:00
Philip.Hazel 8057c3c8b9 Renamed dftables as pcre2_dftables and enable it to write the tables in binary.
Update documentation about character tables.
2020-03-20 18:09:59 +00:00
Zoltán Herczeg 953d4e9c95 Minor improvements for single character iterators in JIT. 2020-03-10 14:42:41 +00:00
Zoltán Herczeg 0d0d954bbd Fix issues in the early fail optimization. 2020-03-06 09:23:10 +00:00
Zoltán Herczeg 21c40e638b Rework early fail optimization in JIT. 2020-03-05 07:58:49 +00:00
Zoltán Herczeg 106d9d3a25 Improve memory clearing in JIT. 2020-03-02 08:52:01 +00:00
Zoltán Herczeg 325908279e Support more accelerated repeat cases in JIT. 2020-02-27 08:35:14 +00:00
Philip.Hazel 3155a6951f Fix bugs in new UCP casing code for back references and characters with more
than 2 cases.
2020-02-26 16:53:39 +00:00
Zoltán Herczeg 305e273e99 Follow ucp changes in JIT. 2020-02-26 10:18:43 +00:00
Philip.Hazel 68f9c49517 Fix bug introduced in recent UCP changes (writing outside starting code unit
bitmap for non-UTF caseless character U+00DF).
2020-02-25 16:47:36 +00:00
Philip.Hazel 3be538015b Fix bad lookbehind compilation when preceded by a DEFINE group. 2020-02-24 17:29:00 +00:00
Philip.Hazel f50ee03f5d Fix bug in UTF-16 checker returning wrong offset for missing low surrogate. 2020-02-24 15:39:56 +00:00
Zoltán Herczeg a3057bbecd Implement simd support for requested character in JIT. 2020-02-24 05:26:15 +00:00
Philip.Hazel 4a7dfab0ec Unicode upper/lower casing is now used when UCP is set, even if UTF is not set.
This is not yet documented, and it not yet implemented in JIT.
2020-02-23 16:40:05 +00:00
Zoltán Herczeg d0666136c9 JIT compiler update. 2020-02-21 07:44:04 +00:00
Zoltán Herczeg c39fb3a9e1 Remove hackings in JIT. 2020-02-20 08:57:39 +00:00
Zoltán Herczeg c21bd97754 Fix a crash which occurs when the character type of an invalid UTF character is decoded in JIT. 2020-02-20 07:42:47 +00:00
Philip.Hazel a57787b7cd Fix problems with new PCRE2_SUBSTITUTE_MATCHED code. 2020-02-16 17:46:40 +00:00
Zoltán Herczeg 697cf5f602 Fix control verb chain restoration issue in JIT. 2020-02-10 10:18:01 +00:00
Zoltán Herczeg d71dc302a5 Fix compiler warning on ARM64 with JIT. 2020-01-31 10:09:38 +00:00
Zoltán Herczeg ed8a3146b9 JIT compiler update. 2020-01-28 14:13:06 +00:00
Philip.Hazel b040e2e1cd Limit function recursion in pcre2_study to avoid stack overflow issues. 2020-01-27 10:28:19 +00:00
Philip.Hazel 3a6b4948d1 Fix bug in processing (?(DEFINE)...) within lookbehind assertions. 2020-01-26 15:31:27 +00:00