JPEG2000 Is Slow … Or Is It?

5

Throughout this year I’ve been contacted by all sorts of organizations who make the following claims,

  • The JPEG2000’s your SDK creates are terrible! It crashes my application
  • Large JPEG2000’s (>1 gigapixel) are an extremely bad idea because it crashes my application!
  • JPEG2000 is just bad, because someone said so on the internet

The problem when people make these claims is that it’s never backed by details, often just third-hand information. As you will see, it’s actually quite simple to get to the bottom of these issues as it always boils down to two fundamental questions,

What type of JPEG2000 do you have and what toolkit are you using?

First things first, JPEG2000 is hard. JPEG2000 is very complex. JPEG2000 gives you access to hundreds of encoding options. You can take one JPEG2000 image and transcode it to hundreds of different “types” of files (I refer to these permutations as profiles) while still representing the same data and still conforming to the specification. Many encoding options can vastly impact decoding speed (CPU time, Memory usage) and filesize while not altering image quality at all. Many options sound fantastic when reading the ISO/IEC 15444-1 specification, but do not materialize in real world usage scenarios. The final complication is that using large datasets—typically found in the geospatial domain—only magnifies these differences and causes significant problems in some decoders.

Sounds familiar right, except for the selling bit

Hang on Chris, there are JP2 encoding options? Absolutely! Each encoder such as Hexagon Geospatial’s own ECW JPEG2000 SDK sets appropriate defaults to yield the best balance of encoding & decoding speed. The problem I often see is that customers may start tweaking various encoding options without understanding their impact downstream. Or they may test, but only do so using one SDK, on one platform/OS, which gives limited perspective.

Another important point to clarify is that JPEG2000 is an open-standard with both open-source and proprietary implementations. Users often conflate this to mean “JPEG2000 is open-source” which does not really make any sense.

As expected, the open-standard status has created many implementations however each varies in sometimes not-so-subtle ways. The compliance test suite referred to in Table C.1 below is very narrow and once again does not adequately cover very large images. So to further complicate matters, you can both comply with the decoding requirements outlined by the specification yet still fail to decode compliant, large geospatial-sized images. Fun, right? This also explains why there a market-specific JPEG2000 toolkits available. Hexagon’s ECWJP2 SDK and Lizardtech DSDK is clearly aimed at the geospatial market, Comprimato originally focused on Cinematic uses, Aware JP2 and Luratech JP2 on Medical imaging, OpenJPEG as a general purpose open-source toolkit and of course Kakadu which is one of the most robust implementations with all the bells-and-whistles (there’s also others).

JPEG2000 ISO Conformance Test

With all this in mind, I transcoded a customer supplied JP2 which was deemed to be “slow” and created 21x alternate files with slightly differing encoding parameters. Here’s the original with the metadata output from gdal_translate -mdd all and our ECWJP2 driver:

Note: These encoding options are only a subset

04_072_original.jp2

15759 x 15756 px x 3 UInt8 bands
CODE_BLOCK_SIZE_X=32
CODE_BLOCK_SIZE_Y=32
GML_JP2_DATA=FALSE
PRECINCT_SIZE_X=32,32,32,64,128
PRECINCT_SIZE_Y=32,32,32,64,128
PRECISION=8,8,8
PROFILE=2
PROGRESSION_ORDER=LRCP
QUALITY_LAYERS=5
RESOLUTION_LEVELS=5
TILES_X=124
TILES_Y=124
TILE_HEIGHT=128
TILE_WIDTH=128
TRANSFORMATION_TYPE=5×3
USE_EPH=FALSE
USE_SOP=FALSE

Transcoding this file into the default profile that JP2ECW creates yields,

04_072_ECWJP2SDK5_default.jp2

15759 x 15756 px x 3 UInt8 bands
CODE_BLOCK_SIZE_X=64
CODE_BLOCK_SIZE_Y=64
GML_JP2_DATA=FALSE
PRECINCT_SIZE_X=128,128,128,128,128,128,128,128
PRECINCT_SIZE_Y=128,128,128,128,128,128,128,128
PRECISION=8,8,8
PROFILE=0
PROGRESSION_ORDER=RPCL
QUALITY_LAYERS=1
RESOLUTION_LEVELS=8
TILES_X=1
TILES_Y=1
TILE_HEIGHT=15756
TILE_WIDTH=15759
TRANSFORMATION_TYPE=5×3
USE_EPH=TRUE
USE_SOP=FALSE

I then went ahead and transcoded another 20x files using different sizes and combinations of precincts, tile size, quality layers, resolution levels, code-blocks, eph/sop markers, progression orders etc. Now, I have a good cross-section of different JP2 profiles. I am in a great position to see how these parameters can effect different toolkits.

I also included 3x other real-world datasets as a reference,

EJPE and NPJE are raw JPEG2000 codestreams representing the NGA and NATO Military encoding profiles for JPEG2000 embedded within NITF. Using the codestreams directly ensures timing is only for JP2 decoding rather than any overhead in the GDAL NITF driver itself.

EPJE:
5654 x 11677px x 8 UInt16 bands
CODE_BLOCK_SIZE_X=64
CODE_BLOCK_SIZE_Y=64
GML_JP2_DATA=FALSE
PRECINCT_SIZE_X=32768,32768,32768,32768,32768
PRECINCT_SIZE_Y=32768,32768,32768,32768,32768
PRECISION=11,11,11,11,11,11,11,11
PROFILE=1
PROGRESSION_ORDER=RLCP
QUALITY_LAYERS=20
RESOLUTION_LEVELS=5
TILES_X=6
TILES_Y=12
TILE_HEIGHT=1024
TILE_WIDTH=1024
TRANSFORMATION_TYPE=5×3
USE_EPH=FALSE
USE_SOP=FALSE

NPJE:
5654 x 11677px x 8 UInt16 bands
CODE_BLOCK_SIZE_X=64
CODE_BLOCK_SIZE_Y=64
GML_JP2_DATA=FALSE
PRECINCT_SIZE_X=32768,32768,32768,32768,32768
PRECINCT_SIZE_Y=32768,32768,32768,32768,32768
PRECISION=11,11,11,11,11,11,11,11
PROFILE=1
PROGRESSION_ORDER=LRCP
QUALITY_LAYERS=19
RESOLUTION_LEVELS=5
TILES_X=6
TILES_Y=12
TILE_HEIGHT=1024
TILE_WIDTH=1024
TRANSFORMATION_TYPE=9×7
USE_EPH=FALSE
USE_SOP=FALSE

IMG_PHR1A_PMS_201202250025599_ORT_IPU_20120523_2858-001_R1C1.jp2 which is from Airbus Defense & Space‘s Pleiades sample imagery.

32768 x 16384 px x 4 UInt16 bands
CODE_BLOCK_SIZE_X=64
CODE_BLOCK_SIZE_Y=64
GML_JP2_DATA=TRUE
PRECINCT_SIZE_X=64,128,128,256,256
PRECINCT_SIZE_Y=64,128,128,256,256
PRECISION=12,12,12,12
PROFILE=2
PROGRESSION_ORDER=RPCL
QUALITY_LAYERS=5
RESOLUTION_LEVELS=5
TILES_X=16
TILES_Y=8
TILE_HEIGHT=2048
TILE_WIDTH=2048
TRANSFORMATION_TYPE=5×3
USE_EPH=FALSE
USE_SOP=FALSE

Build / Hardware

Intel Core i7-5600U @ 2.6Ghz ( VirtualBox v5.0.1 )
8gb RAM
CentOS v7.1.1503 (Kernel v3.10.0-229.20.1.el7_x86_64)
GCC v4.8.3
GDAL v2.0.1 release with default configure options and additional drivers,

JPEG2000 Test Script

Dependencies:

  • GDAL utilities
  • Imagemagick identify utility
    • Provides convenient hash algorithm of image data only, discounting header/metadata differences
  • GNU core-utils timeout utility for killing processes that take longer than 120 secs… you’ll quickly see why this is needed

The script is very simple, especially for anyone who are well versed in the GDAL utilities. There are 3 simple tests performed on every JP2 found in the current working directory,

  1. Image subset – extract a 800×800 px image from the source input at native resolution
  2. Image thumbnail – generate a 500×500 px overview or thumbnail from the 15759×15756 source
  3. Pixel value – extract pixel values from position 2056,2056
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#!/bin/bash

# Setup

export LD_LIBRARY_PATH=./MrSID_DSDK-9.1.0.4045-linux.x86-64.gcc44/Raster_DSDK/lib/:./ERDAS-ECW_JPEG_2000_SDK-5.2.1/Desktop_Read-Only/lib/x64/release:$LD_LIBRARY_PATH
export GDAL_DATA=/usr/local/share/gdal
# setup caches to 500mb
export GDAL_CACHEMAX=500
export NCSCFG_CACHE_MAXMEM_64=500000000

out="./output-$(date +%Y-%m-%d-%H%M)"
mkdir -p $out

JP2="*.jp2 *.JP2 *.jpc"
# Drivers available
drvlist=( JP2MrSID JP2ECW JP2OpenJPEG JPEG2000 )
# Drivers to test
totest=( JP2MrSID JP2ECW JP2OpenJPEG )

for driver in ${totest[@]}; do
        gdalskip=(${drvlist[@]})
        # Remove current driver from GDAL_SKIP
        drivers=(${gdalskip[@]/$driver/})
        driverslist="${drivers[@]}"

        echo "--- $driver ---"
        export GDAL_SKIP=$driverslist
        echo "* GDAL_SKIP=$GDAL_SKIP"
        # Adjust timeout where appropriate
        for i in $JP2; do
                echo "Processing ${i##*/}.." | tee -a $out/$driver.log
                echo "* Image subset (1/3):" | tee -a $out/$driver.log
                timeout --preserve-status -s 9 60 /usr/bin/time -v -o $out/$driver.log -a gdal_translate $i $out/$driver-test1-${i##*/}.tif -b 1 -b 2 -b 3 -srcwin 2000 5000 800 800 >> $out/$driver.log
                identify -quiet -format "Hash: %#" $out/$driver-test1-${i##*/}.tif >> $out/$driver.log
                echo "* Image thumbnail (2/3):" | tee -a $out/$driver.log
                timeout --preserve-status -s 9 60 /usr/bin/time -v -o $out/$driver.log -a gdal_translate $i $out/$driver-test2-${i##*/}.tif -b 1 -b 2 -b 3 -outsize 500 0 >> $out/$driver.log
                identify -quiet -format "Hash: %#" $out/$driver-test2-${i##*/}.tif >> $out/$driver.log
                echo "* Pixel value at point (3/3):" | tee -a $out/$driver.log
                timeout --preserve-status -s 9 60 /usr/bin/time -v -o $out/$driver.log -a gdallocationinfo $i 2056 2056 >> $out/$driver.log
        done
done

Tests in Progress

OpenJPEG manually killed during a test run, memory was later increased to 8gb total

OpenJPEG manually killed after using 4gb RAM

Log Output

For JP2MrSID Driver on Test #1. usr/bin/time outputs variety of process metrics, where I manually stored the user time, CPU %, file system inputs and file hash in the spreadsheet of results below. (Clearly outputting csv would have been far more convenient at generating reports.)

Processing 04_072_CPRL_P128_PLT_5QL.jp2..
* Image subset (1/3):
Input file size is 15759, 15756
0...10...20...30...40...50...60...70...80...90...100 - done.
	Command being timed: "gdal_translate 04_072_CPRL_P128_PLT_5QL.jp2 ./output-2015-12-21-1037/JP2MrSID-test1-04_072_CPRL_P128_PLT_5QL.jp2.tif -b 1 -b 2 -b 3 -srcwin 2000 5000 800 800"
	User time (seconds): 0.52
	System time (seconds): 0.10
	Percent of CPU this job got: 31%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.97
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 20128
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 16
	Minor (reclaiming a frame) page faults: 5864
	Voluntary context switches: 167
	Involuntary context switches: 67
	Swaps: 0
	File system inputs: 89032
	File system outputs: 3760
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
Hash: 86c00f8ae9ff5f4f1bc19cd2aa209a45edcc663922470672ad1405d6b51ed509

Results

JPEG2000-test-results1

Test #1 – Image subset results

JPEG2000-test-results2

Test #2 – Image thumbnail results

JPEG2000-test-results3

Test #3 – Pixel value results

Source: GDAL-JPEG2000-decoding-results

Results Summary

  • JPEG2000/Jasper is borderline useless, however this is expected given development has stalled for some time, yet people continue using it. In almost all cases the Driver would spin indefinitely decoding input. Out of 75 tests it completed only 7 and those took a significant amount of time to complete.
  • JP2OpenJPEG is preferred by many users due to its BSD license and active development however it still failed 19 tests. Of the test results it did pass, memory requirements were the highest (by significant amounts) and it also remained 10x slower (or more) than the two proprietary drivers tested below.
  • JP2MrSID showed 100% completion rate and very low memory usage throughout
  • JP2ECW failed 2 tests however in all other results provided the fastest results, on average +12%

Random Comments & Observations

Because I know you will ask…

    • Kakadu remains untested as I do not have access to v6 or v7 sources
    • Running an identical test on Windows might prove interesting
    • All three tests are intentionally basic, yet exercise fundamental driver requirements. Expanding the tests to include multi-threaded reading would further separate the good from the bad. (If someone suggests a simple approach I’d be happy generating the results!)
    • Test 1 file hash results show JP2MrSID & JP2ECW matching for all comparison images. JP2OpenJPEG showed two mismatches for the first two files. In both cases the generated TIF files appears truncated or corrupted.
    • Test 2 thumbnail file hashes are not intended to match as each driver will implement slightly different resampling algorithms as demonstrated by the Absolute Error difference images between the decoder outputs.
diff-jp2ecw-vs-jp2mrsid-RPCL1024-fuzz10

JP2MrSID vs JP2ECW

diff-jp2ecw-vs-jp2openjpeg-RPCL1024-fuzz10

JP2ECW vs OpenJPEG

  • Test 1 hash results for the 3x real world images shows differences however I believe its related to uint16. Still trying to narrow down the cause.
  • The two failures on JP2ECW were Linux specific and has already been resolved in the v5.3 release as well as further speed improvements.
  • Testing larger JP2’s in terms of dimensions may separate JP2ECW and JP2MrSID further in terms of performance
  • Ever wonder why your Linux Desktop (Gnome etc.) would always crash opening a Folder with JP2’s in it? That would likely be OpenJPEG crashing or running out memory generating the thumbnail images. For unknown reasons most distro’s still ship OpenJPEG v1.5 which is far worse than v2.1
  • Sorry, but I cant provide the test data however please do test on your own datasets using the same methodology!

I am well aware this post will no doubt inspire others to improve OpenJPEG performance. My main hope is that through highlighting these faults in the decoders using realistic imagery for Geospatial use, the community will stop tarnishing the whole format with the same brush. Based on my results here and all customer complaints I have been involved with the last few years, it’s the open-source decoders who are to blame 95% of the time (5% admittedly are people using ECWJP2 v3.3 from 2006).

So is JPEG2000 slow? It can be, but the answer is far more nuanced than yes or no. In many ways, it’s the same issue as people creating large GeoTIFF’s with no pyramids, no tiling but on a much larger scale.

Remember the strength of JPEG2000 being an open-standard is choice! Shop around. Compile your own results. Don’t blindly trust some guy on the internet, remember?

Happy New Year!

Share.

About Author

Chris Tweedie

Chris Tweedie directs all stages of the product lifecycle for ERDAS APOLLO and ECW technologies. He has been with Hexagon Geospatial for 6 years in Technical Sales and Product Management roles for what is now the Provider Suite. Not one to shy away from getting his hands dirty, he enjoys squeezing the most value out of the products built in the Australian development office including ERDAS APOLLO, ECW/JP2 SDK, GeoCompressor and ECW Plugins.

5 Comments

  1. Interesting article. Curious about the numbers you are getting, for example I see that the OpenJPEG
    CPU was running at up to 180% in your first test.

    • Chris Tweedie

      Hi Aaron,

      GNU Time man page shows

      %P Percentage of the CPU that this job got, computed as (%U + %S) / %E.

      Where U and S are user/kernel mode time. I cant explain why it was consistently over 100% but i can confirm the reporting was consistent over numerous test runs. Grab the script and give it a whirl yourself.

      One thing i should have clarified above was the VM was allocated only 2x cores. So that may be another explanation if gnu time behaves like top reporting with multiple cores

        • Chris Tweedie

          Hi Aaron, Sorry but I have no desire (or time) to collate all the results again for 32-bit! 🙂

          You can of course run the same script on that dataset with kdu. If you do, please post back your results

          • Thanks, Chris.

            By the way, there are some big changes coming to OpenJPEG for geo-spatial workflows. Memory usage will be dropping and performance increasing by a significant amount in the next few library releases. Plus new features such as precinct-level decode.

            There is more discussion of this on the GDAL news group if you’re interested.

            Cheers,
            Aaron

Leave A Reply