walkero
/
openjpeg
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity
openjpeg/src/lib/openjp2
History
Even Rouault 793edc38e4 Decrease memory consumption for whole image single tile decoding. 
We can use the same buffer for the tile decoding and the final image, and
save the intermediate buffer to transfer between those.

Effect on the decoding of MAPA (9944 x 13498 x 3 components of size byte)

Peak memory from 4.5 GB to 2.7 GB

Now:
n5: 2699708767 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
 n1: 1610689344 0x4E77E07: opj_aligned_malloc (opj_malloc.c:61) <-- final image
  n1: 1610689344 0x4E7195B: opj_alloc_tile_component_data (tcd.c:676)
   n1: 1610689344 0x4E722D2: opj_tcd_init_decode_tile (tcd.c:816)
    n1: 1610689344 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
     n1: 1610689344 0x4E4C742: opj_j2k_decode_tiles (j2k.c:10324)
      n1: 1610689344 0x4E4E20E: opj_j2k_decode (j2k.c:7826)
       n1: 1610689344 0x4E52E42: opj_jp2_decode (jp2.c:1564)
        n0: 1610689344 0x40369E: main (opj_decompress.c:1459)
 n1: 815554560 0x4E72231: opj_tcd_init_decode_tile (tcd.c:1217) <-- working memory for code blocks: 9944*13498/64/64*8192*3
  n1: 815554560 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
   n1: 815554560 0x4E4C742: opj_j2k_decode_tiles (j2k.c:10324)
    n1: 815554560 0x4E4E20E: opj_j2k_decode (j2k.c:7826)
     n1: 815554560 0x4E52E42: opj_jp2_decode (jp2.c:1564)
      n0: 815554560 0x40369E: main (opj_decompress.c:1459)
 n1: 219758391 0x4E4C0BF: opj_j2k_read_tile_header (j2k.c:4661) <-- ingestion of code stream
  n1: 219758391 0x4E4C742: opj_j2k_decode_tiles (j2k.c:10324)
   n1: 219758391 0x4E4E20E: opj_j2k_decode (j2k.c:7826)
    n1: 219758391 0x4E52E42: opj_jp2_decode (jp2.c:1564)
     n0: 219758391 0x40369E: main (opj_decompress.c:1459)
 n1: 39822000 0x4E7224F: opj_tcd_init_decode_tile (tcd.c:1224) <-- OPJ_J2K_DEFAULT_NB_SEGS*sizeof(opj_tcd_seg_t) per codeblock
  n1: 39822000 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
   n1: 39822000 0x4E4C742: opj_j2k_decode_tiles (j2k.c:10324)
    n1: 39822000 0x4E4E20E: opj_j2k_decode (j2k.c:7826)
     n1: 39822000 0x4E52E42: opj_jp2_decode (jp2.c:1564)
      n0: 39822000 0x40369E: main (opj_decompress.c:1459)
 n0: 13884472 in 49 places, all below massif's threshold (1.00%)

Before:
n5: 4493329848 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
 n2: 1610709160 0x4E77C87: opj_aligned_malloc (opj_malloc.c:61)
  n1: 1610689344 0x4E717DB: opj_alloc_tile_component_data (tcd.c:676)
   n1: 1610689344 0x4E72152: opj_tcd_init_decode_tile (tcd.c:816)
    n1: 1610689344 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
     n1: 1610689344 0x4E4C64A: opj_j2k_decode_tiles (j2k.c:10318)
      n1: 1610689344 0x4E4E08E: opj_j2k_decode (j2k.c:7826)
       n1: 1610689344 0x4E52CC2: opj_jp2_decode (jp2.c:1564)
        n0: 1610689344 0x40369E: main (opj_decompress.c:1459)
  n0: 19816 in 2 places, all below massif's threshold (1.00%)
 n1: 1610689344 0x4E43F36: opj_j2k_update_image_data.isra.7 (j2k.c:8743)
  n1: 1610689344 0x4E4C5C1: opj_j2k_decode_tiles (j2k.c:10358)
   n1: 1610689344 0x4E4E08E: opj_j2k_decode (j2k.c:7826)
    n1: 1610689344 0x4E52CC2: opj_jp2_decode (jp2.c:1564)
     n0: 1610689344 0x40369E: main (opj_decompress.c:1459)
 n1: 815554560 0x4E720B1: opj_tcd_init_decode_tile (tcd.c:1217)
  n1: 815554560 0x4E4BCF1: opj_j2k_read_tile_header (j2k.c:8597)
   n1: 815554560 0x4E4C64A: opj_j2k_decode_tiles (j2k.c:10318)
    n1: 815554560 0x4E4E08E: opj_j2k_decode (j2k.c:7826)
     n1: 815554560 0x4E52CC2: opj_jp2_decode (jp2.c:1564)
      n0: 815554560 0x40369E: main (opj_decompress.c:1459)
 n1: 402672336 0x4E4C545: opj_j2k_decode_tiles (j2k.c:10336)
  n1: 402672336 0x4E4E08E: opj_j2k_decode (j2k.c:7826)
   n1: 402672336 0x4E52CC2: opj_jp2_decode (jp2.c:1564)
    n0: 402672336 0x40369E: main (opj_decompress.c:1459)
 n0: 53704448 in 58 places, all below massif's threshold (1.00%)
 		2017-08-07 18:18:53 +02:00
..
CMakeLists.txt install static libraries 2017-07-13 11:34:15 +02:00
bench_dwt.c Add bench_dwt program (compiled only if BUILD_BENCH_DWT=ON) 2017-06-20 17:56:19 +02:00
bio.c Remove OPJ_NOSANITIZE in opj_bio_read() and opj_bio_write() (#761) 2017-06-17 19:15:00 +02:00
bio.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
cidx_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
cidx_manager.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
cio.c Avoid p_stream->m_user_data_length >= (OPJ_UINT64)p_stream->m_byte_offset assertion in opj_stream_get_number_byte_left(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2786. Credit to OSS Fuzz 2017-07-30 16:48:15 +02:00
cio.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
dwt.c Comment fix 2017-07-06 12:11:37 +02:00
dwt.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
event.c opj_event_msg(): force zero termination of buffer 2017-07-30 19:27:01 +02:00
event.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
function_list.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
function_list.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
image.c Avoid heap buffer overflow in function pnmtoimage of convert.c, and unsigned integer overflow in opj_image_create() (CVE-2016-9118, #861) 2017-07-30 18:43:25 +02:00
image.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
indexbox_manager.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
invert.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
invert.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
j2k.c Decrease memory consumption for whole image single tile decoding. 2017-08-07 18:18:53 +02:00
j2k.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
jp2.c Fix warnings in USE_JPIP compilation mode 2017-07-30 18:46:34 +02:00
jp2.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
libopenjp2.pc.cmake.in [trunk] add missing private lib. Update issue 223 2014-02-24 17:22:48 +00:00
mct.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
mct.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
mqc.c T1 decoder: check code stream errors when predictable termination is enabled and emit a warning when errors are found 2017-07-26 21:43:32 +02:00
mqc.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
mqc_inl.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
openjpeg.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
openjpeg.h Fix Doxygen warnings (patch derived from Winfried's doxygen-dif.txt.zip, #849) 2017-07-30 18:18:59 +02:00
opj_clock.c Fix mingw related warnings 2017-06-21 12:54:40 +02:00
opj_clock.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
opj_codec.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
opj_common.h MQC/RAW decoder: use an artificial 0xFF 0xFF terminating marker. 2017-06-02 18:24:07 +02:00
opj_config.h.cmake.in [trunk] Finalize API for openjpeg 2.1 2014-04-28 09:17:31 +00:00
opj_config_private.h.cmake.in Fix support of posix_memalign for Linux 2016-01-06 18:06:45 +01:00
opj_includes.h Enable __SSE__ / __SSE2__ with Visual Studio 2017-06-20 18:24:21 +02:00
opj_intmath.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
opj_inttypes.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
opj_malloc.c IDWT 5x3: generalize SSE2 version for AVX2 2017-06-21 12:12:58 +02:00
opj_malloc.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
opj_stdint.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
phix_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
pi.c Fix warnings in pi.c raised by VS11 analyze (#190) 2017-07-29 19:43:23 +02:00
pi.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
ppix_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
t1.c Fix Doxygen warnings (patch derived from Winfried's doxygen-dif.txt.zip, #849) 2017-07-30 18:18:59 +02:00
t1.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
t1_generate_luts.c t1_generate_luts.c: fix compiler warnings 2017-06-02 18:22:59 +02:00
t1_luts.h T1: Transpose coder optimizations to decoder, and cleanup code 2017-06-02 18:20:35 +02:00
t2.c Fix Doxygen warnings (patch derived from Winfried's doxygen-dif.txt.zip, #849) 2017-07-30 18:18:59 +02:00
t2.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
tcd.c opj_tcd_dc_level_shift_decode: avoid int32 overflow when prec == 31. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2799. Credit to OSS Fuzz 2017-07-30 15:22:24 +02:00
tcd.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
tgt.c Fix Doxygen warnings (patch derived from Winfried's doxygen-dif.txt.zip, #849) 2017-07-30 18:18:59 +02:00
tgt.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
thix_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
thread.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
thread.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
tls_keys.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
tpix_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00