walkero
/
openjpeg
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity
2,700 Commits 24 Branches 23 Tags 132 MiB
Commit Graph

2700 Commits

Author SHA1 Message Date
Even Rouault 793edc38e4 Decrease memory consumption for whole image single tile decoding. 
We can use the same buffer for the tile decoding and the final image, and
save the intermediate buffer to transfer between those.

Effect on the decoding of MAPA (9944 x 13498 x 3 components of size byte)

Peak memory from 4.5 GB to 2.7 GB

Now:
n5: 2699708767 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
 n1: 1610689344 0x4E77E07: opj_aligned_malloc (opj_malloc.c:61) <-- final image
  n1: 1610689344 0x4E7195B: opj_alloc_tile_component_data (tcd.c:676)
   n1: 1610689344 0x4E722D2: opj_tcd_init_decode_tile (tcd.c:816)
    n1: 1610689344 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
     n1: 1610689344 0x4E4C742: opj_j2k_decode_tiles (j2k.c:10324)
      n1: 1610689344 0x4E4E20E: opj_j2k_decode (j2k.c:7826)
       n1: 1610689344 0x4E52E42: opj_jp2_decode (jp2.c:1564)
        n0: 1610689344 0x40369E: main (opj_decompress.c:1459)
 n1: 815554560 0x4E72231: opj_tcd_init_decode_tile (tcd.c:1217) <-- working memory for code blocks: 9944*13498/64/64*8192*3
  n1: 815554560 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
   n1: 815554560 0x4E4C742: opj_j2k_decode_tiles (j2k.c:10324)
    n1: 815554560 0x4E4E20E: opj_j2k_decode (j2k.c:7826)
     n1: 815554560 0x4E52E42: opj_jp2_decode (jp2.c:1564)
      n0: 815554560 0x40369E: main (opj_decompress.c:1459)
 n1: 219758391 0x4E4C0BF: opj_j2k_read_tile_header (j2k.c:4661) <-- ingestion of code stream
  n1: 219758391 0x4E4C742: opj_j2k_decode_tiles (j2k.c:10324)
   n1: 219758391 0x4E4E20E: opj_j2k_decode (j2k.c:7826)
    n1: 219758391 0x4E52E42: opj_jp2_decode (jp2.c:1564)
     n0: 219758391 0x40369E: main (opj_decompress.c:1459)
 n1: 39822000 0x4E7224F: opj_tcd_init_decode_tile (tcd.c:1224) <-- OPJ_J2K_DEFAULT_NB_SEGS*sizeof(opj_tcd_seg_t) per codeblock
  n1: 39822000 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
   n1: 39822000 0x4E4C742: opj_j2k_decode_tiles (j2k.c:10324)
    n1: 39822000 0x4E4E20E: opj_j2k_decode (j2k.c:7826)
     n1: 39822000 0x4E52E42: opj_jp2_decode (jp2.c:1564)
      n0: 39822000 0x40369E: main (opj_decompress.c:1459)
 n0: 13884472 in 49 places, all below massif's threshold (1.00%)

Before:
n5: 4493329848 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
 n2: 1610709160 0x4E77C87: opj_aligned_malloc (opj_malloc.c:61)
  n1: 1610689344 0x4E717DB: opj_alloc_tile_component_data (tcd.c:676)
   n1: 1610689344 0x4E72152: opj_tcd_init_decode_tile (tcd.c:816)
    n1: 1610689344 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
     n1: 1610689344 0x4E4C64A: opj_j2k_decode_tiles (j2k.c:10318)
      n1: 1610689344 0x4E4E08E: opj_j2k_decode (j2k.c:7826)
       n1: 1610689344 0x4E52CC2: opj_jp2_decode (jp2.c:1564)
        n0: 1610689344 0x40369E: main (opj_decompress.c:1459)
  n0: 19816 in 2 places, all below massif's threshold (1.00%)
 n1: 1610689344 0x4E43F36: opj_j2k_update_image_data.isra.7 (j2k.c:8743)
  n1: 1610689344 0x4E4C5C1: opj_j2k_decode_tiles (j2k.c:10358)
   n1: 1610689344 0x4E4E08E: opj_j2k_decode (j2k.c:7826)
    n1: 1610689344 0x4E52CC2: opj_jp2_decode (jp2.c:1564)
     n0: 1610689344 0x40369E: main (opj_decompress.c:1459)
 n1: 815554560 0x4E720B1: opj_tcd_init_decode_tile (tcd.c:1217)
  n1: 815554560 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
   n1: 815554560 0x4E4C64A: opj_j2k_decode_tiles (j2k.c:10318)
    n1: 815554560 0x4E4E08E: opj_j2k_decode (j2k.c:7826)
     n1: 815554560 0x4E52CC2: opj_jp2_decode (jp2.c:1564)
      n0: 815554560 0x40369E: main (opj_decompress.c:1459)
 n1: 402672336 0x4E4C545: opj_j2k_decode_tiles (j2k.c:10336)
  n1: 402672336 0x4E4E08E: opj_j2k_decode (j2k.c:7826)
   n1: 402672336 0x4E52CC2: opj_jp2_decode (jp2.c:1564)
    n0: 402672336 0x40369E: main (opj_decompress.c:1459)
 n0: 53704448 in 58 places, all below massif's threshold (1.00%)
 2017-08-07 18:18:53 +02:00
szukw000 bc3cb74100 Changes for converttif.c to fix tsize_t 2017-08-07 16:44:28 +02:00
Even Rouault 2fbd4bb0b9 opj_j2k_read_sot(): check current TPSot number regarding previous (non-zero) TNsot to avoid opj_j2k_merge_ppt() to be called several times. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2851. Credit to OSS Fuzz 2017-08-04 18:02:10 +02:00
szukw000 57e36dbfeb First change on changes-for-afl-tests 2017-08-02 17:27:08 +02:00
Antonin Descampe 155fc2e279 Update abi-check to take into account new defaults for "add" git command 2017-08-02 17:07:29 +02:00
Antonin Descampe eb90d8ec93 WIP: fix abi-check and automatic upload 2017-08-02 16:50:11 +02:00
Even Rouault 48125b0d12 src/bin/jpwl/convert.c: add missing fclose() in error code path (suggested by maddin200, #976) 2017-07-31 17:35:10 +02:00
szukw000 00f45684a8 Catch images broken by AFL 2017-07-31 13:58:08 +02:00
Even Rouault 13cde9fa37 src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
Even Rouault 9a6d41d22b opj_event_msg(): force zero termination of buffer 2017-07-30 19:27:01 +02:00
Even Rouault b716f86163 Fix breakage of 22bf99ce02 2017-07-30 19:26:47 +02:00
Even Rouault 22bf99ce02 Test return value of opj_j2k_setup_decoding_tile() (commit ec31fa0c7f by ak-dxdy, #561) 2017-07-30 19:07:16 +02:00
Even Rouault ffa9a4f658 Fix warnings in USE_JPIP compilation mode 2017-07-30 18:46:34 +02:00
Even Rouault c22cbd8bdf Avoid heap buffer overflow in function pnmtoimage of convert.c, and unsigned integer overflow in opj_image_create() (CVE-2016-9118, #861) 2017-07-30 18:43:25 +02:00
Even Rouault 83342f2aaf Fix Doxygen warnings (patch derived from Winfried's doxygen-dif.txt.zip, #849) 2017-07-30 18:18:59 +02:00
Even Rouault 4748318136 j2k.c: remove hardcoded constants related to m_state, and useless FIXME 2017-07-30 17:26:03 +02:00
Even Rouault e23e0c94d0 Avoid p_stream->m_user_data_length >= (OPJ_UINT64)p_stream->m_byte_offset assertion in opj_stream_get_number_byte_left(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2786. Credit to OSS Fuzz 2017-07-30 16:48:15 +02:00
Even Rouault 1ed8d67797 opj_j2k_set_decode_area: replace assertions by runtime checks. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2795. Credit to OSS Fuzz 2017-07-30 15:35:47 +02:00
Even Rouault 68832af20e opj_tcd_dc_level_shift_decode: avoid int32 overflow when prec == 31. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2799. Credit to OSS Fuzz 2017-07-30 15:22:24 +02:00
Even Rouault 517bf6fd86 src/bin/jpwl/convert.c: fix memleak (fix suggested by maddin200, #631) 2017-07-29 21:11:23 +02:00
Even Rouault 51eb86d8f7 Fix warnings in pi.c raised by VS11 analyze (#190) 2017-07-29 19:43:23 +02:00
Even Rouault 397f62c0a8 Fix write heap buffer overflow in opj_mqc_byteout(). Discovered by Ke Liu of Tencent's Xuanwu LAB (#835) 2017-07-29 19:13:49 +02:00
Even Rouault 11445eddad opj_pi_update_decode_poc(): limit layno1 to the number of layers (CVE-2016-1626 and CVE-2016-1628, #850) 
This has been recently fixed in a less elegant way per
80818c39f5
 2017-07-29 19:03:13 +02:00
Even Rouault 3fbe713690 opj_tcd_get_decoded_tile_size(): fix potential UINT32 overflow (#854, CVE-2016-5152) 
Fix derived from https://pdfium.googlesource.com/pdfium.git/+/d8cc503575463ff3d81b22dad292665f2c88911e/third_party/libopenjpeg20/0018-tcd_get_decoded_tile_size.patch
 2017-07-29 18:38:16 +02:00
Even Rouault 5a3e7aaf33 color_cielab_to_rgb(): reject images with components of different dimensions to void read heap buffer overflow (#909) 2017-07-29 17:56:12 +02:00
Even Rouault 784d4d47e9 Fix breakage of 2fa0fc61f2 (#970) 2017-07-29 17:51:10 +02:00
Even Rouault 2fa0fc61f2 imagetopnm(): make sure the alpha component has same dimension as other components to avoid read heap buffer overflow (#970) 2017-07-29 17:28:55 +02:00
Even Rouault db9ef99f6d opj_t1_decode_cblk(): avoid undefined shift behaviour. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2487. Credit to OSS Fuzz 2017-07-29 16:34:35 +02:00
Even Rouault f6551f822f opj_t1_clbl_decode_processor(): avoid undefined behaviour if roishift >= 31. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2506. Credit to OSS Fuzz 2017-07-29 16:29:11 +02:00
Even Rouault 9906fbf737 Avoid assertion in opj_j2k_merge_ppt() in case premature EOC is encountered in opj_j2k_read_tile_header(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2785. Credit to OSS Fuzz 2017-07-29 16:22:36 +02:00
Even Rouault 71b4f5b124 opj_pi_next_pcrl(): avoid undefined shift behaviour. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2787. Credit to OSS Fuzz 2017-07-29 15:52:11 +02:00
Even Rouault d6654d906c opj_int_ceildiv(): fix int32 overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2494. Credit to OSS Fuzz 2017-07-28 22:15:47 +02:00
Even Rouault 361c4506fd opj_tcd_dc_level_shift_decode(): avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2516. Credit to OSS Fuzz 2017-07-28 22:06:26 +02:00
Even Rouault 7bdbe490cb Fix null pointer dereference in opj_jp2_apply_pclr(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2558. Credit to OSS Fuzz 2017-07-28 21:55:22 +02:00
Even Rouault 16aeb9282f Fix null pointer dereference in opj_j2k_add_mct() (#895) 
Fixes openjeg-crashes-2017-07-27/issue879-poc1.j2k of #895
 2017-07-28 21:39:30 +02:00
Even Rouault c5bf5ef4d6 Avoid use-after-free when a MCT marker is found after a MCC one (#895) 
Fixes openjeg-crashes-2017-07-27/issue880-poc2.j2k of #895
 2017-07-28 21:29:55 +02:00
Even Rouault e03e947466 Avoid undefined shift behaviour if bit depth == 32 (#895) 
Fixes openjeg-crashes-2017-07-27/id:000000,sig:11,src:003798,op:ext_AO,pos:128.jp2
 2017-07-27 22:29:17 +02:00
Even Rouault 820fcfe8bb opj_j2k_update_image_data / opj_tcd_update_tile_data: fix unaligned load/store (#895) 
When components don't have the same width, unaligned load/store are possible.

Fixes openjeg-crashes-2017-07-27/id:000000,sig:11,src:001342,op:flip4,pos:162.jp2 of #895
 2017-07-27 19:35:35 +02:00
Even Rouault 6c4e5bacb9 opj_pi_next_rpcl / opj_pi_next_pcrl / opj_pi_next_cprl: avoid int overflow (#895) 
Fixes int overflow on openjeg-crashes-2017-07-27/id:000000,sig:08,src:000879,op:flip2,pos:128.jp2
 2017-07-27 19:22:14 +02:00
Even Rouault 178194c093 opj_jp2_check_color(): replace assertion regarding mtyp by runtime check (#672, #895) 
Fixes test case openjeg-crashes-2017-07-27/id:000000,sig:06,src:000001,op:flip1,pos:808.jp2
of #895
 2017-07-27 18:52:35 +02:00
Even Rouault d6fa300997 Avoids undefined shift behaviour in m_dc_level_shift computation 
Fixes warning found on clusterfuzz-testcase-minimized-5146316340461568
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2495

Credit to OSS Fuzz
 2017-07-27 18:10:03 +02:00
Even Rouault a88cbb6a0b Fix various undefined shift behaviour in pi.c 
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2496
Credit to OSS Fuzz
 2017-07-26 22:53:59 +02:00
Even Rouault 6c5fe9407b Avoid potential undefined shift behaviour in opj_bio_read() from opj_t2_read_packet_header() 
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2485
Credit to OSS Fuzz
 2017-07-26 22:22:44 +02:00
Even Rouault dbf527bf2a Merge pull request #800 from rouault/tier1_optimizations_multithreading_pterm_check 
Implement predictive termination check
 2017-07-26 22:05:32 +02:00
Even Rouault 94c4b7300c T1 decoder: check code stream errors when predictable termination is enabled and emit a warning when errors are found 2017-07-26 21:43:32 +02:00
Even Rouault 5e795d90a1 Spelling fixes (patch by ka7, #890, rebased on top of master) 2017-07-26 21:06:38 +02:00
Even Rouault 2be20ce7d9 Reformat src/bin/wx/OPJViewer/source/OPJThreads.cpp src/bin/wx/OPJViewer/source/imagjpeg2000.cpp wrapping/java/openjp2/JavaOpenJPEG.c 2017-07-26 21:04:01 +02:00
Even Rouault 94cc97c58a opj_decompress: fix null pointer dereference on comps[].data on id_000167,sig_11,src_006079,op_havoc,rep_4 (#939) 2017-07-26 20:13:09 +02:00
Even Rouault 8d2e69e37d Fix assertion / memory leak in opj_j2k_merge_ppt() on corrupted images (#939) 
Fixes issue on id:000020,sig:06,src:001958,op:flip4,pos:149 that has two
SOT markers for the same tile with the same tile part number, causing
opj_j2k_merge_ppt() to be called several times.
 2017-07-26 19:49:38 +02:00
Even Rouault 5c5319984b Avoid division by zero in opj_pi_next_rpcl, opj_pi_next_pcrl, opj_pi_next_cprl (#938) 
Fixes crash on id_000004,sig_06,src_000679,op_arith8,pos_49,val_-17
 2017-07-26 18:05:56 +02:00