Commit Graph

1122 Commits

Author SHA1 Message Date
Even Rouault 1aa3c60859
Decoding: deal with some SPOT6 images that have tiles with a single tile-part with TPsot == 0 and TNsot == 0, and with missing EOC 2020-11-30 17:53:04 +01:00
yuan 4f487798ba Free p_tcd_marker_info to avoid memory leak 2020-11-26 00:22:49 +08:00
yuan 649298dcf8 Encoder: grow again buffer size in opj_tcd_code_block_enc_allocate_data() (fixes #1283) 2020-11-25 20:41:39 +08:00
Even Rouault 15cf3d9581
Encoder: grow again buffer size in opj_tcd_code_block_enc_allocate_data() (fixes #1283) 2020-11-23 18:14:02 +01:00
Even Rouault eaa098b59b
Encoder: grow buffer size in opj_tcd_code_block_enc_allocate_data() to avoid write heap buffer overflow in opj_mqc_flush (fixes #1283) 2020-11-23 13:49:05 +01:00
Even Rouault 67ec360f48
Fix typo in internal function name 2020-11-23 13:38:27 +01:00
Even Rouault 491299eb07
Merge pull request #1253 from rouault/floating_point_irreversible_encoding
Single-threaded performance improvements in forward DWT for 5-3 and 9-7 (and other improvements)
2020-10-09 13:25:27 +02:00
Pei JIA 65c8f577d2
Bump Java compatibility from 1.5 to 1.6 (#1263) 2020-09-23 11:34:31 +02:00
Even Rouault 0863ccf291
Encoder: avoid uint32 overflow when allocating memory for codestream buffer (fixes #1243) 2020-09-16 11:27:53 +02:00
Even Rouault e8e258ab04
opj_decompress: fix double-free on input directory with mix of valid and invalid images (CVE-2020-15389)
Fixes #1261

Credits to @Ruia-ruia for reporting and analysis.
2020-06-30 21:31:19 +02:00
Even Rouault 172c8ae5cf
Merge pull request #1260 from sebras/fix-issue-1259
openjp2: Plug image leak when failing to allocate codestream index.
2020-06-22 22:35:44 +02:00
Sebastian Rasmussen b028e8d1ce openjp2: Plug image leak when failing to allocate codestream index.
This fixes issue #1259.
2020-06-23 03:42:25 +08:00
Sebastian Rasmussen 79b199a8fe openjp2: Plug memory leak when setting data as TLS fails.
Previously the Tier 1 handle was not freed when setting it as
TLS failed.

This fixes issue #1257.
2020-06-23 03:21:31 +08:00
Sebastian Rasmussen 93b9f7236c openjp2: Error out if failing to create Tier 1 handle.
Previously when the handle failed to be created (e.g. when
opj_calloc returned NULL due to low memory), the code still
assumed that the t1 handle pointer was valid and dereferenced
NULL, causing a crash. After this commit OpenJPEG will instead
error out under this condition.

This fixes issue #1255.
2020-06-23 02:32:31 +08:00
szukw000 25fb144c42
Testing for invalid values of width, height, numcomps (#1254) 2020-06-10 17:40:50 +02:00
Even Rouault 1c5627ee74
T1 encoder: speed-up by aggressive inlining and more cache friendly data organization
~ 9% speed improvement seen on 10980x10980 uint16 image, T36JTT_20160914T074612_B02.tif
opj_compress time from 17.2s to 15.8s
2020-05-24 15:38:48 +02:00
Even Rouault 1e931fdb36
Forward DWT 9-7: major speed up by vectorizing vertical pass
`bench_dwt -I -encode` times goes from 8.6s to 2.1s
2020-05-23 01:01:05 +02:00
Even Rouault a38e970fa5
Forward DWT 5-3: major speed up by vectorizing vertical pass
`bench_dwt -encode` times goes from 7.9s to 1.7s
2020-05-23 01:01:05 +02:00
Even Rouault e69fa09f60
Forward DWT: small code refactoring to allow future improvements for the vertical pass 2020-05-22 16:01:45 +02:00
Even Rouault 33d3d0de07
dwt.c: remove unused typedef 2020-05-22 15:06:29 +02:00
Even Rouault 97b384aecd
Forward DWT 5x3: performance improvements in horizontal pass, and modest in vertical pass 2020-05-22 15:03:40 +02:00
Even Rouault bd5f5ee7de
Forward DWT: small code refactoring to allow future improvements for the horizontal pass 2020-05-22 15:02:33 +02:00
Even Rouault 45a35223b7
Speed-up 9x7 IDWD by ~30% with OPJ_NUM_THREADS=2
"bench_dwt -I" time goes from 2.2s to 1.5s
2020-05-21 17:21:55 +02:00
Even Rouault 272b3e0fb2
Remove useless + 5U margin in opj_dwt_decode_tile_97()
Nothing in code analysis nor test suite shows that this margin is
needed.
It dates back to commit dbeebe72b9
where vector 9x7 decoding was introduced.
2020-05-21 15:42:51 +02:00
Even Rouault 47943daa15
Speed-up 9x7 IDWD by ~20%
"bench_dwt -I" time goes from 2.8s to 2.2s
2020-05-21 15:42:51 +02:00
Even Rouault 0c09062464
bench_dwt.c: add a -I switch to test irreversible FWDT/IDWT 2020-05-20 23:20:48 +02:00
Even Rouault 19ef7f26c4
Merge pull request #1211 from sebras/master
Add check to validate SGcod/SPcoc/SPcod parameter values.
2020-05-20 21:10:55 +02:00
Even Rouault adccbc8336
Irreversible decoding: partially revert previous commit, to fix failures in test suite 2020-05-20 20:31:28 +02:00
Even Rouault 3cd1305596
Irreversible compression/decompression DWT: use 1/K constant as per standard
The previous constant opj_c13318 was mysteriously equal to 2/K , and in
the DWT, we had to divide K and opj_c13318 by 2... The issue was that the
band->stepsize computation in tcd.c didn't take into account the log2gain of
the band.

The effect of this change is expected to be mostly equivalent to the previous
situation, except some difference in rounding. But it leads to a dramatic
reduction of the mean square error and peak error in the irreversible encoding
of issue141.tif !
2020-05-20 20:31:28 +02:00
Even Rouault f38c069547
Irreversible decoding: align code more closely to the standard by avoid messing up with stepsize (no functional change) 2020-05-20 20:31:28 +02:00
Even Rouault e46e300de5
opj_dwt_encode_1_real(): avoid many bound comparisons, similarly to decoding side 2020-05-20 20:31:28 +02:00
Even Rouault 4ab2ed0907
opj_j2k_setup_encoder(): add validation of tile width and height to avoid potential division by zero 2020-05-20 20:31:28 +02:00
Even Rouault c6a413a423
opj_mct_encode_real(): add SSE optimization 2020-05-20 20:31:28 +02:00
Even Rouault 3d35d0f3af
tcd.c: add comment 2020-05-20 20:31:28 +02:00
Even Rouault 00cff6f5c0
Encoder: use floating-point operations for irreversible transformation 2020-05-20 20:31:28 +02:00
Even Rouault 99107d5e46
dwt.c: change sign of constants to match standard and compensate (no functional change) 2020-05-20 20:31:28 +02:00
Even Rouault 07d1f775a1
Add multithreaded support in the DWT encoder.
Update the bench_dwt utility to have a -decode/-encode switch

Measured performance gains for DWT encoder on a
Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz (4 cores, hyper threaded)

Encoding time:
$ ./bin/bench_dwt -encode -num_threads 1
time for dwt_encode: total = 8.348 s, wallclock = 8.352 s

$ ./bin/bench_dwt -encode -num_threads 2
time for dwt_encode: total = 9.776 s, wallclock = 4.904 s

$ ./bin/bench_dwt -encode -num_threads 4
time for dwt_encode: total = 13.188 s, wallclock = 3.310 s

$ ./bin/bench_dwt -encode -num_threads 8
time for dwt_encode: total = 30.024 s, wallclock = 4.064 s

Scaling is probably limited by memory access patterns causing
memory access to be the bottleneck.
The slightly worse results with threads==8 than with thread==4
is due to hyperthreading being not appropriate here.
2020-05-20 20:30:21 +02:00
Even Rouault 97eb7e0bf1
Add multithreading support in the T1 (entropy phase) encoder
- API wise, opj_codec_set_threads() can be used on the encoding side
- opj_compress has a -threads switch similar to opj_uncompress
2020-05-20 20:30:21 +02:00
Even Rouault 4edb8c8337
Add support for generation of PLT markers in encoder
* -PLT switch added to opj_compress
* Add a opj_encoder_set_extra_options() function that
  accepts a PLT=YES option, and could be expanded later
  for other uses.

-------

Testing with a Sentinel2 10m band, T36JTT_20160914T074612_B02.jp2,
coming from S2A_MSIL1C_20160914T074612_N0204_R135_T36JTT_20160914T081456.SAFE

Decompress it to TIFF:
```
opj_uncompress -i T36JTT_20160914T074612_B02.jp2 -o T36JTT_20160914T074612_B02.tif
```

Recompress it with similar parameters as original:
```
opj_compress -n 5 -c [256,256],[256,256],[256,256],[256,256],[256,256] -t 1024,1024 -PLT -i T36JTT_20160914T074612_B02.tif -o T36JTT_20160914T074612_B02_PLT.jp2
```

Dump codestream detail with GDAL dump_jp2.py utility (https://github.com/OSGeo/gdal/blob/master/gdal/swig/python/samples/dump_jp2.py)
```
python dump_jp2.py T36JTT_20160914T074612_B02.jp2 > /tmp/dump_sentinel2_ori.txt
python dump_jp2.py T36JTT_20160914T074612_B02_PLT.jp2 > /tmp/dump_sentinel2_openjpeg_plt.txt
```

The diff between both show very similar structure, and identical number of packets in PLT markers

Now testing with Kakadu (KDU803_Demo_Apps_for_Linux-x86-64_200210)

Full file decompression:
```
kdu_expand -i T36JTT_20160914T074612_B02_PLT.jp2 -o tmp.tif

Consumed 121 tile-part(s) from a total of 121 tile(s).
Consumed 80,318,806 codestream bytes (excluding any file format) = 5.329697
bits/pel.
Processed using the multi-threaded environment, with
    8 parallel threads of execution
```

Partial decompresson (presumably using PLT markers):
```
kdu_expand -i T36JTT_20160914T074612_B02.jp2 -o tmp.pgm -region "{0.5,0.5},{0.01,0.01}"
kdu_expand -i T36JTT_20160914T074612_B02_PLT.jp2 -o tmp2.pgm  -region "{0.5,0.5},{0.01,0.01}"
diff tmp.pgm tmp2.pgm && echo "same !"
```

-------

Funded by ESA for S2-MPC project
2020-04-21 15:55:44 +02:00
Even Rouault 64689d05df
struct opj_j2k: remove unused fields, and add some documentation 2020-04-18 18:25:44 +02:00
Even Rouault 774889a328
Merge pull request #1244 from rouault/fix_pi_warnings
Fix warnings about signed/unsigned casts in pi.c
2020-04-17 00:39:46 +02:00
szukw000 b6b7e96b0c
color_apply_icc_profile: add checks on the number of components (#1236) 2020-04-17 00:37:33 +02:00
Eduardo Barretto 040e142288
jp3d/jpwl/mj2/jpip: Fix resource leaks (#1226)
This issues were found by cppcheck and coverity.
2020-04-17 00:09:40 +02:00
Even Rouault 271a71ef0f
Fix warnings about signed/unsigned casts in pi.c 2020-04-16 23:34:10 +02:00
Even Rouault 221a801a97
Rename mis-named function opj_tcd_get_encoded_tile_size() to opj_tcd_get_encoder_input_buffer_size() 2020-04-16 20:33:22 +02:00
Even Rouault 1c54024165
opj_decompress: add sanity checks to avoid segfault in case of decoding error
Prevent crashes like:
opj_decompress -i 0722_5-1_2019.jp2 -o out.ppm -r 4 -t 0

where 0722_5-1_2019.jp2 is
https://drive.google.com/file/d/1ZxOUZg2-FKjYwa257VFLMpTXRWxEoP0a/view?usp=sharing
2020-04-01 21:11:36 +02:00
Even Rouault 563ecfb55c
opj_compress: improve help message regarding new IMF switch 2020-02-13 09:59:36 +01:00
Even Rouault 84f3bebbff
Implement writing of IMF profiles
Add -IMF switch to opj_compress as well
2020-02-12 15:55:25 +01:00
Even Rouault fffe32adcb
openjpeg.h: fix values of OPJ_PROFILE_IMF_ constants 2020-02-12 15:55:02 +01:00
Even Rouault 05f9b91e60
opj_tcd_init_tile(): avoid integer overflow
That could lead to later assertion failures.

Fixes #1231 / CVE-2020-8112
2020-01-30 00:59:57 +01:00
Even Rouault 024b840739
opj_j2k_update_image_dimensions(): reject images whose coordinates are beyond INT_MAX (fixes #1228) 2020-01-11 01:51:58 +01:00
Even Rouault ac3737372a
Merge pull request #1217 from rouault/fix_ossfuzz_18979
pi.c: avoid integer overflow, resulting in later invalid access to memory in opj_t2_decode_packets()
2019-11-17 13:08:41 +01:00
Robert Ancell 9701b3305d JPWL: convert: Fix buffer overflow reading an image file less than four characters (#1196)
Fixes #1068
2019-11-17 03:09:59 +01:00
Even Rouault 4cb1f66304
pi.c: avoid integer overflow, resulting in later invalid access to memory in opj_t2_decode_packets(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18979 2019-11-17 01:18:26 +01:00
Even Rouault 5875a6b446
opj_tcd_mct_decode()/opj_mct_decode()/opj_mct_encode_real()/opj_mct_decode_real(): proper deal with a number of samples larger than 4 billion (refs #1151) 2019-10-03 11:04:30 +02:00
Sebastian Rasmussen f3ee448815 openjp2/j2k: Validate all SGcod/SPcod/SPcoc parameter values.
Previously the multiple component transformation SGcod(C)
and wavelet transformation SPcod(H)/SPcoc(E) parameter
values were never checked, allowing for out of range values.

The lack of validation allowed the bit stream provided in
issue #1158 through. After this commit an error message
points to the marker segments' parameters as being out of
range.

input/nonregression/edf_c2_20.jp2 contains an SPcod(H) value
of 17, but according to Table A-20 of the specification only
values 0 and 1 are valid. input/nonregression/issue826.jp2
contains a SGcod(B) value of 2, but according to Table A-17
of the specification only values 0 and 1 are valid.
input/nonregression/oss-fuzz2785.jp2 contains a SGcod(B)
value of 32, but it is likewise limited to 0 or 1. These test
cases have been updated to consistently fail to parse the
headers since they contain out of bounds values.

This fixes issue #1210.
2019-09-04 05:14:41 +02:00
Sebastian Rasmussen d801bd4e62 openjp2/j2k: Make comments adhere to specification.
The function is used to read both SPcod and SPcoc, so all
comments should refer to both marker segments' parameter names.
2019-09-04 05:14:30 +02:00
Even Rouault e66125fe26
Merge pull request #1164 from sebras/master
openjp2/j2k: Report error if all wanted components are not decoded.
2019-09-03 17:03:54 +02:00
Even Rouault 9b7620ee7a
Merge pull request #1185 from Young-X/fix
Fix several potential vulnerabilities
2019-04-26 19:52:52 +02:00
Even Rouault a94cfbd533
Change opj_j2k_check_poc_val() to take into account tile number 2019-04-25 15:06:45 +02:00
Even Rouault 6423163141
Fix POC in multi-tile scenarios: avoid almost endless loop when a tile has no POC settings 2019-04-25 14:40:56 +02:00
Even Rouault 23883458b9
opj_j2k_check_poc_val(): prevent potential write outside of allocated array 2019-04-25 14:40:56 +02:00
Even Rouault 6589c609f6
opj_j2k_check_poc_val(): fix starting index for checking layer dimension
The standard mandates that the layer index always starts at zero for every
progression.
2019-04-25 14:40:55 +02:00
Even Rouault 1e3a57563d
compression: emit POC marker when only one single POC is requested (fixes #1191) 2019-04-25 14:40:55 +02:00
Even Rouault 5dd75f62e2
j2k.c: use correct naming convention for total_data_size variable 2019-04-23 16:52:21 +02:00
Young Xiao 3aef207f90 bmp_read_rle4_data(): avoid potential infinite loop 2019-04-15 16:10:18 +08:00
Young Xiao 21399f6b7d convertbmp: detect invalid file dimensions early
width/length dimensions read from bmp headers are not necessarily
valid. For instance they may have been maliciously set to very large
values with the intention to cause DoS (large memory allocation, stack
overflow). In these cases we want to detect the invalid size as early
as possible.

This commit introduces a counter which verifies that the number of
written bytes corresponds to the advertized width/length.

See commit 8ee335227b for details.

Signed-off-by: Young Xiao <YangX92@hotmail.com>
2019-04-15 16:10:18 +08:00
Even Rouault a1d32a596a
opj_t1_encode_cblks: fix UBSAN signed integer overflow
Fixes #1053 / CVE-2018-5727

Note: I don't consider this issue to be a security vulnerability, in
practice.
At least with gcc or clang compilers on x86_64 which generate the same
assembly code with or without that fix.
2019-03-29 11:17:39 +01:00
Even Rouault 25b815dc46
Revert "[JPWL] tgatoimage(): avoid excessive memory allocation attempt,"
This reverts commit 05be308446.

This commit doesn't compile due to missing OPJ_UINT64 type
2019-03-29 10:44:35 +01:00
Even Rouault e1740e7ce7
Revert "[MJ2] Avoid index out of bounds access to pi->include[]"
This reverts commit c277159986.

The commit didn't compile. include_size is not defined in openmj2
2019-03-29 10:40:58 +01:00
Sebastian Rasmussen b2751967ec openjp2/j2k: Report error if all wanted components are not decoded.
Previously the caller had to check whether each component data had
been decoded. This means duplicating the checking in every user of
openjpeg which is unnecessary. If the caller wantes to decode all
or a set of, or a specific component then openjpeg ought to error
out if it was unable to do so.

Fixes #1158.
2019-02-21 16:48:02 +08:00
Hugo Lefeuvre 8ee335227b convertbmp: detect invalid file dimensions early
width/length dimensions read from bmp headers are not necessarily
valid. For instance they may have been maliciously set to very large
values with the intention to cause DoS (large memory allocation, stack
overflow). In these cases we want to detect the invalid size as early
as possible.

This commit introduces a counter which verifies that the number of
written bytes corresponds to the advertized width/length.

Fixes #1059 (CVE-2018-6616).
2018-12-14 05:10:35 +01:00
Even Rouault e7640f58f1
Merge pull request #1168 from Young-X/fix_dev
Fix multiple potential vulnerabilities and bugs
2018-12-07 21:27:38 +01:00
Young Xiao 05be308446 [JPWL] tgatoimage(): avoid excessive memory allocation attempt,
and fixes unaligned load

Signed-off-by: Young Xiao <YangX92@hotmail.com>
2018-11-28 14:44:06 +08:00
Young_X bd88611ed9 [JP3D] To avoid divisions by zero / undefined behaviour on shift (CVE-2018-14423
Signed-off-by: Young_X <YangX92@hotmail.com>
2018-11-28 14:39:15 +08:00
Young_X ce9583d1d7 [JPWL] opj_compress: reorder checks related to code block dimensions to avoid potential int overflow
Signed-off-by: Young_X <YangX92@hotmail.com>
2018-11-28 14:39:14 +08:00
Young_X c58df14990 [OPENJP2] change the way to compute *p_tx0, *p_tx1, *p_ty0, *p_ty1 in function
opj_get_encoding_parameters

Signed-off-by: Young_X <YangX92@hotmail.com>
2018-11-28 14:39:14 +08:00
Young_X c277159986 [MJ2] Avoid index out of bounds access to pi->include[]
Signed-off-by: Young_X <YangX92@hotmail.com>
2018-11-28 14:39:14 +08:00
Even Rouault 2e5ab1d998
color_apply_icc_profile: avoid potential heap buffer overflow
Derived from a patch by Thuan Pham
2018-11-27 23:31:30 +01:00
Young_X 46822d0edd [JPWL] imagetotga(): fix read heap buffer overflow if numcomps < 3 (#987)
Signed-off-by: Young_X <YangX92@hotmail.com>
2018-11-23 17:08:57 +08:00
Young_X 619e1b086e [JPWL] fix CVE-2018-16375
Signed-off-by: Young_X <YangX92@hotmail.com>
2018-11-23 17:08:56 +08:00
Young_X c5bd64ea14 [MJ2] To avoid divisions by zero / undefined behaviour on shift
Signed-off-by: Young_X <YangX92@hotmail.com>
2018-11-23 14:47:36 +08:00
Even Rouault 92023cd6c3
Merge pull request #1160 from hlef/master
jp3d/jpwl convert: fix write stack buffer overflow
2018-11-16 09:42:19 +01:00
ichlubna c196b23b90 openjp3d: Int overflow fixed (#1159)
When compressing a lot of slices (starting from 44 FullHD slices with 3 8bit components in our experiments) the rate values are high enough to cause an int overflow that leads to negative lengths and wrong results. The cast happens too late.
2018-11-16 09:40:31 +01:00
Hugo Lefeuvre cab352e249 jp2: convert: fix null pointer dereference
Tile components in a JP2 image might have null data pointer by defining a
zero component size (for example using large horizontal or vertical
sampling periods). This null data pointer leads to null image component
data pointer, causing crash when dereferenced without != null check in
imagetopnm.

Add != null check.

This commit addresses #1152 (CVE-2018-18088).
2018-11-07 18:53:18 +01:00
Hugo Lefeuvre 0bc90e4062 jp3d/jpwl convert: fix write stack buffer overflow
Missing buffer length formatter in fscanf call might lead to write
stack buffer overflow.

fixes #1044 (CVE-2017-17480)
2018-11-01 09:05:26 +01:00
Stefan Weil 948332e6ed Fix some potential overflow issues (#1161)
* Fix some potential overflow issues

Put sizeof to the beginning of the multiplication to enforce that
size_t instead of smaller integer types is used for the calculation.

This fixes warnings from LGTM:

    Multiplication result may overflow 'unsigned int'
    before it is converted to 'unsigned long'.

It also allows removing some type casts.

Signed-off-by: Stefan Weil <sw@weilnetz.de>

* Fix code indentation

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-31 20:44:30 +01:00
Nikola Forró 943db0f1c2 Fix several memory and resource leaks
Signed-off-by: Nikola Forró <nforro@redhat.com>
2018-10-31 16:16:22 +01:00
Even Rouault cd900d9661
opj_thread_pool_setup(): fix infinite waiting if a thread creation failed 2018-10-18 11:45:45 +02:00
Even Rouault 0e6a5553cf
Merge pull request #1148 from hlef/master
CVE-2018-5785: fix issues with zero bitmasks
2018-09-22 23:54:12 +02:00
Even Rouault 8fc09e50e5
opj_jp2_apply_pclr(): remove useless assert that can trigger on some files (fixes #1125) 2018-09-22 23:47:56 +02:00
Even Rouault aaf48ee6ba
Merge branch 'pr1095' 2018-09-22 23:12:50 +02:00
Karol Babioch cc3824767b
opj_mj2_extract: Check provided output prefix for length
This uses snprintf() with correct buffer length instead of sprintf(), which
prevents a buffer overflow when providing a long output prefix. Furthermore
the program exits with an error when the provided output prefix is too long.

Fixes #1088.
2018-09-22 23:12:39 +02:00
Even Rouault ee827ad3f3
Merge branch 'pr1107' 2018-09-22 23:05:54 +02:00
szukw000 1eb9a57ac1
opj_mj2_extract: Avoid segfault for long filenames 2018-09-22 23:05:38 +02:00
Even Rouault 5d94bcd89c
Merge pull request #1136 from reverson/master
Cast on uint ceildiv
2018-09-22 22:59:36 +02:00
Even Rouault b54c06fb35
Merge pull request #1119 from stweil/ssize_t
Use local type declaration for POSIX standard type only for MS compiler
2018-09-22 22:59:17 +02:00
Even Rouault 17bbb0e23f
Merge pull request #1128 from stweil/typos
Fix some typos in code comments and documentation
2018-09-22 22:55:33 +02:00
Even Rouault c6ee006250
Merge pull request #1141 from szukw000/changes-in-pnmtoimage
Changes in pnmtoimage if image data are missing
2018-09-22 22:47:27 +02:00
Hugo Lefeuvre ca16fe5501 convertbmp: fix issues with zero bitmasks
In the case where a BMP file declares compression 3 (BI_BITFIELDS)
with header size <= 56, all bitmask values keep their initialization
value 0. This may lead to various undefined behavior later e.g. when
doing 1 << (l_comp->prec - 1).

This issue does not affect files with bit count 16 because of a check
added in 16240e2 which sets default values to the color masks if they
are all 0.

This commit adds similar checks for the 32 bit case.

Also, if a BMP file declares compression 3 with header size >= 56 and
intentional 0 bitmasks, the same issue will be triggered in both the
16 and 32 bit count case.

This commit adds checks to bmp_read_info_header() rejecting BMP files
with "intentional" 0 bitmasks. These checks might be removed in the
future when proper handling of zero bitmasks will be available in
openjpeg2.

fixes #1057 (CVE-2018-5785)
2018-09-22 14:51:50 -04:00