Commit Graph

163 Commits

Author SHA1 Message Date
Even Rouault 491299eb07
Merge pull request #1253 from rouault/floating_point_irreversible_encoding
Single-threaded performance improvements in forward DWT for 5-3 and 9-7 (and other improvements)
2020-10-09 13:25:27 +02:00
Even Rouault 19ef7f26c4
Merge pull request #1211 from sebras/master
Add check to validate SGcod/SPcoc/SPcod parameter values.
2020-05-20 21:10:55 +02:00
Even Rouault 3cd1305596
Irreversible compression/decompression DWT: use 1/K constant as per standard
The previous constant opj_c13318 was mysteriously equal to 2/K , and in
the DWT, we had to divide K and opj_c13318 by 2... The issue was that the
band->stepsize computation in tcd.c didn't take into account the log2gain of
the band.

The effect of this change is expected to be mostly equivalent to the previous
situation, except some difference in rounding. But it leads to a dramatic
reduction of the mean square error and peak error in the irreversible encoding
of issue141.tif !
2020-05-20 20:31:28 +02:00
Even Rouault fe4c15f12c
Testing: revise testing of lossy encoding by comparing PEAK and MSE with original image 2020-05-20 20:31:28 +02:00
Even Rouault 4edb8c8337
Add support for generation of PLT markers in encoder
* -PLT switch added to opj_compress
* Add a opj_encoder_set_extra_options() function that
  accepts a PLT=YES option, and could be expanded later
  for other uses.

-------

Testing with a Sentinel2 10m band, T36JTT_20160914T074612_B02.jp2,
coming from S2A_MSIL1C_20160914T074612_N0204_R135_T36JTT_20160914T081456.SAFE

Decompress it to TIFF:
```
opj_uncompress -i T36JTT_20160914T074612_B02.jp2 -o T36JTT_20160914T074612_B02.tif
```

Recompress it with similar parameters as original:
```
opj_compress -n 5 -c [256,256],[256,256],[256,256],[256,256],[256,256] -t 1024,1024 -PLT -i T36JTT_20160914T074612_B02.tif -o T36JTT_20160914T074612_B02_PLT.jp2
```

Dump codestream detail with GDAL dump_jp2.py utility (https://github.com/OSGeo/gdal/blob/master/gdal/swig/python/samples/dump_jp2.py)
```
python dump_jp2.py T36JTT_20160914T074612_B02.jp2 > /tmp/dump_sentinel2_ori.txt
python dump_jp2.py T36JTT_20160914T074612_B02_PLT.jp2 > /tmp/dump_sentinel2_openjpeg_plt.txt
```

The diff between both show very similar structure, and identical number of packets in PLT markers

Now testing with Kakadu (KDU803_Demo_Apps_for_Linux-x86-64_200210)

Full file decompression:
```
kdu_expand -i T36JTT_20160914T074612_B02_PLT.jp2 -o tmp.tif

Consumed 121 tile-part(s) from a total of 121 tile(s).
Consumed 80,318,806 codestream bytes (excluding any file format) = 5.329697
bits/pel.
Processed using the multi-threaded environment, with
    8 parallel threads of execution
```

Partial decompresson (presumably using PLT markers):
```
kdu_expand -i T36JTT_20160914T074612_B02.jp2 -o tmp.pgm -region "{0.5,0.5},{0.01,0.01}"
kdu_expand -i T36JTT_20160914T074612_B02_PLT.jp2 -o tmp2.pgm  -region "{0.5,0.5},{0.01,0.01}"
diff tmp.pgm tmp2.pgm && echo "same !"
```

-------

Funded by ESA for S2-MPC project
2020-04-21 15:55:44 +02:00
Even Rouault 84f3bebbff
Implement writing of IMF profiles
Add -IMF switch to opj_compress as well
2020-02-12 15:55:25 +01:00
Even Rouault b5cb419faf
tests: add alternate checksums for libtiff 4.1
Fixes #1233

libtiff 4.1 slightly modifies the way it generates files. So
add the new expected md5sum.

Not super elegant solution admitedly.
2020-02-07 22:05:55 +01:00
Sebastian Rasmussen f3ee448815 openjp2/j2k: Validate all SGcod/SPcod/SPcoc parameter values.
Previously the multiple component transformation SGcod(C)
and wavelet transformation SPcod(H)/SPcoc(E) parameter
values were never checked, allowing for out of range values.

The lack of validation allowed the bit stream provided in
issue #1158 through. After this commit an error message
points to the marker segments' parameters as being out of
range.

input/nonregression/edf_c2_20.jp2 contains an SPcod(H) value
of 17, but according to Table A-20 of the specification only
values 0 and 1 are valid. input/nonregression/issue826.jp2
contains a SGcod(B) value of 2, but according to Table A-17
of the specification only values 0 and 1 are valid.
input/nonregression/oss-fuzz2785.jp2 contains a SGcod(B)
value of 32, but it is likewise limited to 0 or 1. These test
cases have been updated to consistently fail to parse the
headers since they contain out of bounds values.

This fixes issue #1210.
2019-09-04 05:14:41 +02:00
Even Rouault bdec5ae272
Add test for previous commit 2019-04-25 14:40:56 +02:00
Even Rouault b86717fdd3
Add test for previous commit 2019-04-25 14:40:56 +02:00
Even Rouault 1e3a57563d
compression: emit POC marker when only one single POC is requested (fixes #1191) 2019-04-25 14:40:55 +02:00
Even Rouault 4170681661
Add test cases for https://github.com/uclouvain/openjpeg/issues/1120 and https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2785 2018-06-20 15:28:53 +02:00
Even Rouault 98cfdd3134 opj_j2k_read_cod: remove check for 'No more than one COD marker per tile' (fixes #1043)
This check was added per daed8cc919
to fix https://github.com/uclouvain/openjpeg/issues/476 , but it does not seem
to be necessary with latest master (issue476.jp2 doesn't cause memory issues),
and breaks reading legit files.
2017-11-30 14:48:34 +01:00
Even Rouault 7e2b6bebff Add capability to decode only a subset of all components of an image.
This adds a opj_set_decoded_components(opj_codec_t *p_codec,
OPJ_UINT32 numcomps, const OPJ_UINT32* comps_indices) function,
and equivalent "opj_decompress -c compno[,compno]*" option.

When specified, neither the MCT transform nor JP2 channel transformations
will be applied.

Tests added for various combinations of whole image vs tiled-based decoding,
full or reduced resolution, use of decode area or not.
2017-09-19 17:06:19 +02:00
Even Rouault 4776b15fa7 Add test for fix of a55c024fc6 2017-08-28 18:08:13 +02:00
Even Rouault 79aa0b54dd issue104_jpxstream.jp2 decompresses just fine (and like Kakadu). Update md5refs.txt and remove from knownfailures 2017-08-24 14:31:13 +02:00
Even Rouault c059f431e6 Remove broken.jpc test from known failures, by avoiding too long processing time 2017-08-24 14:09:34 +02:00
Even Rouault fdcd4e6365 Remove -NR-DEC-p1_06.j2k-156-decode and NR-DEC-p1_06.j2k-164-decode from knownfailures
and make them explicit failures. The result images are empty at the requested resolution
2017-08-24 13:23:25 +02:00
Even Rouault afb308b9cc Encoder: grow buffer size in opj_tcd_code_block_enc_allocate_data() to avoid write heap buffer overflow in opj_mqc_flush (#982) 2017-08-14 17:20:37 +02:00
Even Rouault ac375ac9f5 Partial revert BPC related check of #975 (#979)
PR #975 introduced a check that rejects images that have different bit depth/sign
per compoment in SIZ marker if the JP2 IHDR box has BPC != 255
This didn't work properly if decoding a .j2k file since the new bit added in
opj_cp_t wasn't initialized to the right value.
For clarity, tThis new bit has also been renamed to allow_different_bit_depth_sign

But looking closer at the code, it seems we were already tolerant to inconsistencies.
For example we parsed a JP2 BPCC box even if BPC != 255 (just a warning is emitted)
So failing hard in opj_j2k_read_siz() wouldn't be very inconsistent, and that
alone cannot protect against other issues, so just emit a warning if BPC != 255
and the SIZ marker contains different bit depth/sign per component.

Note: we could also check that the content of JP2 BPCC box is consistant with the one
of the SIZ marker.
2017-08-09 11:34:08 +02:00
Even Rouault d6fa300997 Avoids undefined shift behaviour in m_dc_level_shift computation
Fixes warning found on clusterfuzz-testcase-minimized-5146316340461568
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2495

Credit to OSS Fuzz
2017-07-27 18:10:03 +02:00
Even Rouault 73d1510d47 Encoder: fix packet writing of empty sub-bands (#891, #892)
There are situations where, given a tile size, at a resolution level,
there are sub-bands with x0==x1 or y0==y1, that consequently don't have any
valid codeblocks, but the other sub-bands may be non-empty.
Given that we recycle the memory from one tile to another one, those
ghost codeblocks might be non-0 and thus candidate for packet inclusion.
2017-06-12 18:37:50 +02:00
Even Rouault 81c5311758 T1: fix BYPASS/LAZY, TERMALL/RESTART and PTERM/ERTERM encoding modes. (#674)
There were a number of defects regarding when and how the termination of
passes had to done and the computation of their rate.
2017-06-09 10:49:03 +02:00
Even Rouault 2d2c368b19 Tests: test opj_compress in VSC mode (related to #172) 2017-05-23 14:31:39 +02:00
Matthieu Darbois fac916f72a Fix PNM file reading (#847)
Malformed PNM file could cause a crash in opj_compress.
Checks were added to prevent this.

Fixes #843
Updates #440
2016-09-22 00:30:34 +02:00
Matthieu Darbois 805972f4c8 Add test for issue 820 (#829)
Update uclouvain/openjpeg#820
2016-09-11 19:47:32 +02:00
Matthieu Darbois 45ed017b1b Add test for issue 826 (#827)
Update uclouvain/openjpeg#826

Also correct names for data of issue uclouvain/openjpeg#823
2016-09-08 23:56:06 +02:00
Matthieu Darbois e078172b1c Add sanity check for tile coordinates (#823)
Coordinates are casted from OPJ_UINT32 to OPJ_INT32
Add sanity check for negative values and upper bound becoming lower
than lower bound.
See also
https://pdfium.googlesource.com/pdfium/+/b6befb2ed2485a3805cddea86dc7574510178ea9
2016-09-08 00:24:15 +02:00
Matthieu Darbois acfb307df4 Add test for PR 818 (#822)
update uclouvain/openjpeg#818
2016-09-07 00:00:21 +02:00
mayeut 6609719b40 Correct expected result for test of issue 495 2016-05-08 20:26:12 +02:00
mayeut 1a8318f6c2 Fix Out-of-Bounds Access in function opj_tgt_reset
Fix uclouvain/openjpeg#775
2016-05-08 20:10:13 +02:00
mayeut 162f6199c0 Fix Heap Buffer Overflow in function color_cmyk_to_rgb
Fix uclouvain/openjpeg#774
2016-05-08 19:18:05 +02:00
mayeut 8f9cc62b3f Fix division by zero
Fix uclouvain/openjpeg#733
2016-05-08 18:40:12 +02:00
Julien Malik 17a0a8a195 Use lowercase for cmake commands consistenly
Inspired from https://github.com/InsightSoftwareConsortium/ITK/blob/master/Utilities/Maintenance/HowToCreateTheCMakeCaseConversion.txt
This needs vim 7.3 and fails with vim 7.4

This also fixes a number of :
- missing empty line at end of files
- useless space at end of lines
2016-05-02 16:05:41 +02:00
mayeut 4f11e89c80 Add tests for recent issues
Update uclouvain/openjpeg#725
Update uclouvain/openjpeg#726
2016-04-30 18:51:36 +02:00
Matthieu Darbois 9a20f8e8d1 Update lcms (#544)
Update to mm2/Little-CMS@0e8234e090
2016-04-30 17:58:04 +02:00
Antonin Descampe 72deb588cb Merge pull request #706 from mayeut/issue135
Fix issue 135
The fix is legal regarding the standard but I did not manage to find out if it covers a bug in opj_t2_read_packet_data or if the file is corrupted
2016-04-30 13:51:01 +02:00
Matthieu Darbois e166e4a209 Check SSIZ is valid in opj_j2k_read_siz (#762) 2016-04-28 19:20:14 +02:00
mayeut 20789fed4e Fix issue 135
dwt_interleave_h.gsr105.jp2 now has the same output as kakadu
issue399 is corrupted. Only the corrupted part changes.

Update known failures for x86 MD5

NR-DEC-kodak_2layers_lrcp.j2c-31-decode-md5
NR-DEC-kodak_2layers_lrcp.j2c-32-decode-md5
NR-DEC-issue135.j2k-68-decode-md5
2016-04-27 23:15:47 +02:00
mayeut 74e814358c Allow to write 3/5/7/9/11/13/15 bpp TIF files
Update uclouvain/openjpeg#729
2016-04-23 04:29:03 +02:00
mayeut 54393d9e38 Allow to read 3/5/7/9/11/13/15 bpp TIF files 2016-04-23 04:24:04 +02:00
mayeut fb4be3894e Fix undefined size jp2 box handling
Update #653
2015-11-07 01:35:43 +01:00
Stefan Weil 99fc1ab306 Fix typos in comments and string
Most typos were found by codespell.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2015-09-29 05:55:43 +02:00
mayeut 4cde646cd7 Add test for uclouvain/openjpeg#571 2015-09-10 01:24:46 +02:00
mayeut 5f02757eef Correct lossless issue on linux x86
Update uclouvain/openjpeg#571
2015-09-10 00:39:23 +02:00
mayeut c5dd6035c9 Correct CMake version requirements
Update uclouvain/openjpeg#488
2015-09-01 20:43:43 +02:00
mayeut 2eb97a851d Update tests 2015-08-23 12:32:21 +02:00
mayeut a521472427 Add tests for CMYK/esYCC/CIELab 2015-08-21 21:41:50 +02:00
mayeut dc34a6cc8c Add test for #388 2015-07-31 01:48:51 +02:00
mayeut ae7953849b Add jpylyzer tests for JP2 compression
Unify syntax
2015-07-30 01:33:04 +02:00