openjpeg/src/lib/openjp2
Even Rouault ca34d13e76 Decoding: do not allocate memory for the codestream of each codeblock
Currently we allocate at least 8192 bytes for each codeblock, and copy
the relevant parts of the codestream in that per-codeblock buffer as we
decode packets.
As the whole codestream for the tile is ingested in memory and alive
during the decoding, we can directly point to it instead of copying. But
to do that, we need an intermediate concept, a 'chunk' of code-stream segment,
given that segments may be made of data at different places in the code-stream
when quality layers are used.

With that change, the decoding of MAPA_005.jp2 goes down from the previous
improvement of 2.7 GB down to 1.9 GB.

New profile:

n4: 1885648469 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
 n1: 1610689344 0x4E78287: opj_aligned_malloc (opj_malloc.c:61)
  n1: 1610689344 0x4E71D7B: opj_alloc_tile_component_data (tcd.c:676)
   n1: 1610689344 0x4E7272C: opj_tcd_init_decode_tile (tcd.c:816)
    n1: 1610689344 0x4E4BDD9: opj_j2k_read_tile_header (j2k.c:8618)
     n1: 1610689344 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
      n1: 1610689344 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
       n1: 1610689344 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
        n0: 1610689344 0x40374E: main (opj_decompress.c:1459)
 n1: 219232541 0x4E4BBF0: opj_j2k_read_tile_header (j2k.c:4685)
  n1: 219232541 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
   n1: 219232541 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
    n1: 219232541 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
     n0: 219232541 0x40374E: main (opj_decompress.c:1459)
 n1: 39822000 0x4E727A9: opj_tcd_init_decode_tile (tcd.c:1219)
  n1: 39822000 0x4E4BDD9: opj_j2k_read_tile_header (j2k.c:8618)
   n1: 39822000 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
    n1: 39822000 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
     n1: 39822000 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
      n0: 39822000 0x40374E: main (opj_decompress.c:1459)
 n0: 15904584 in 52 places, all below massif's threshold (1.00%)
2017-08-07 18:32:52 +02:00
..
CMakeLists.txt install static libraries 2017-07-13 11:34:15 +02:00
bench_dwt.c Add bench_dwt program (compiled only if BUILD_BENCH_DWT=ON) 2017-06-20 17:56:19 +02:00
bio.c Remove OPJ_NOSANITIZE in opj_bio_read() and opj_bio_write() (#761) 2017-06-17 19:15:00 +02:00
bio.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
cidx_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
cidx_manager.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
cio.c Avoid p_stream->m_user_data_length >= (OPJ_UINT64)p_stream->m_byte_offset assertion in opj_stream_get_number_byte_left(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2786. Credit to OSS Fuzz 2017-07-30 16:48:15 +02:00
cio.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
dwt.c Comment fix 2017-07-06 12:11:37 +02:00
dwt.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
event.c opj_event_msg(): force zero termination of buffer 2017-07-30 19:27:01 +02:00
event.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
function_list.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
function_list.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
image.c Add opj_image_data_alloc() / opj_image_data_free() 2017-08-07 18:32:52 +02:00
image.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
indexbox_manager.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
invert.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
invert.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
j2k.c Decoding: do not allocate memory for the codestream of each codeblock 2017-08-07 18:32:52 +02:00
j2k.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
jp2.c opj_jp2_apply_pclr() also needs to use opj_image_data_alloc/opj_image_data_free 2017-08-07 18:32:52 +02:00
jp2.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
libopenjp2.pc.cmake.in [trunk] add missing private lib. Update issue 223 2014-02-24 17:22:48 +00:00
mct.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
mct.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
mqc.c T1 decoder: check code stream errors when predictable termination is enabled and emit a warning when errors are found 2017-07-26 21:43:32 +02:00
mqc.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
mqc_inl.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
openjpeg.c Complementary fix to previous commit 2017-08-07 18:32:52 +02:00
openjpeg.h Add opj_image_data_alloc() / opj_image_data_free() 2017-08-07 18:32:52 +02:00
opj_clock.c Fix mingw related warnings 2017-06-21 12:54:40 +02:00
opj_clock.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
opj_codec.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
opj_common.h Decoding: do not allocate memory for the codestream of each codeblock 2017-08-07 18:32:52 +02:00
opj_config.h.cmake.in [trunk] Finalize API for openjpeg 2.1 2014-04-28 09:17:31 +00:00
opj_config_private.h.cmake.in Fix support of posix_memalign for Linux 2016-01-06 18:06:45 +01:00
opj_includes.h Enable __SSE__ / __SSE2__ with Visual Studio 2017-06-20 18:24:21 +02:00
opj_intmath.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
opj_inttypes.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
opj_malloc.c IDWT 5x3: generalize SSE2 version for AVX2 2017-06-21 12:12:58 +02:00
opj_malloc.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
opj_stdint.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
phix_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
pi.c Fix warnings in pi.c raised by VS11 analyze (#190) 2017-07-29 19:43:23 +02:00
pi.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
ppix_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
t1.c Decoding: do not allocate memory for the codestream of each codeblock 2017-08-07 18:32:52 +02:00
t1.h Decoding: do not allocate memory for the codestream of each codeblock 2017-08-07 18:32:52 +02:00
t1_generate_luts.c t1_generate_luts.c: fix compiler warnings 2017-06-02 18:22:59 +02:00
t1_luts.h T1: Transpose coder optimizations to decoder, and cleanup code 2017-06-02 18:20:35 +02:00
t2.c Decoding: do not allocate memory for the codestream of each codeblock 2017-08-07 18:32:52 +02:00
t2.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
tcd.c Decoding: do not allocate memory for the codestream of each codeblock 2017-08-07 18:32:52 +02:00
tcd.h Decoding: do not allocate memory for the codestream of each codeblock 2017-08-07 18:32:52 +02:00
tgt.c Fix Doxygen warnings (patch derived from Winfried's doxygen-dif.txt.zip, #849) 2017-07-30 18:18:59 +02:00
tgt.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
thix_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
thread.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00
thread.h Reformat: apply reformattin on .h files (#128) 2017-05-15 12:21:30 +02:00
tls_keys.h src/lib/openjp2/*.h: use OPJ_ prefix for inclusion guards instead of reserved __ (#587) 2017-07-30 19:46:52 +02:00
tpix_manager.c Reformat whole codebase with astyle.options (#128) 2017-05-09 20:46:20 +02:00