Throughout this year I’ve been contacted by all sorts of organizations who make the following claims,
- The JPEG2000’s your SDK creates are terrible! It crashes my application
- Large JPEG2000’s (>1 gigapixel) are an extremely bad idea because it crashes my application!
- JPEG2000 is just bad, because someone said so on the internet
The problem when people make these claims is that it’s never backed by details, often just third-hand information. As you will see, it’s actually quite simple to get to the bottom of these issues as it always boils down to two fundamental questions,
What type of JPEG2000 do you have and what toolkit are you using?
First things first, JPEG2000 is hard. JPEG2000 is very complex. JPEG2000 gives you access to hundreds of encoding options. You can take one JPEG2000 image and transcode it to hundreds of different “types” of files (I refer to these permutations as profiles) while still representing the same data and still conforming to the specification. Many encoding options can vastly impact decoding speed (CPU time, Memory usage) and filesize while not altering image quality at all. Many options sound fantastic when reading the ISO/IEC 15444-1 specification, but do not materialize in real world usage scenarios. The final complication is that using large datasets—typically found in the geospatial domain—only magnifies these differences and causes significant problems in some decoders.
Hang on Chris, there are JP2 encoding options? Absolutely! Each encoder such as Hexagon Geospatial’s own ECW JPEG2000 SDK sets appropriate defaults to yield the best balance of encoding & decoding speed. The problem I often see is that customers may start tweaking various encoding options without understanding their impact downstream. Or they may test, but only do so using one SDK, on one platform/OS, which gives limited perspective.
Another important point to clarify is that JPEG2000 is an open-standard with both open-source and proprietary implementations. Users often conflate this to mean “JPEG2000 is open-source” which does not really make any sense.
As expected, the open-standard status has created many implementations however each varies in sometimes not-so-subtle ways. The compliance test suite referred to in Table C.1 below is very narrow and once again does not adequately cover very large images. So to further complicate matters, you can both comply with the decoding requirements outlined by the specification yet still fail to decode compliant, large geospatial-sized images. Fun, right? This also explains why there a market-specific JPEG2000 toolkits available. Hexagon’s ECWJP2 SDK and Lizardtech DSDK is clearly aimed at the geospatial market, Comprimato originally focused on Cinematic uses, Aware JP2 and Luratech JP2 on Medical imaging, OpenJPEG as a general purpose open-source toolkit and of course Kakadu which is one of the most robust implementations with all the bells-and-whistles (there’s also others).
With all this in mind, I transcoded a customer supplied JP2 which was deemed to be “slow” and created 21x alternate files with slightly differing encoding parameters. Here’s the original with the metadata output from gdal_translate -mdd all and our ECWJP2 driver:
Note: These encoding options are only a subset
15759 x 15756 px x 3 UInt8 bands
Transcoding this file into the default profile that JP2ECW creates yields,
15759 x 15756 px x 3 UInt8 bands
I then went ahead and transcoded another 20x files using different sizes and combinations of precincts, tile size, quality layers, resolution levels, code-blocks, eph/sop markers, progression orders etc. Now, I have a good cross-section of different JP2 profiles. I am in a great position to see how these parameters can effect different toolkits.
I also included 3x other real-world datasets as a reference,
EJPE and NPJE are raw JPEG2000 codestreams representing the NGA and NATO Military encoding profiles for JPEG2000 embedded within NITF. Using the codestreams directly ensures timing is only for JP2 decoding rather than any overhead in the GDAL NITF driver itself.
5654 x 11677px x 8 UInt16 bands
5654 x 11677px x 8 UInt16 bands
IMG_PHR1A_PMS_201202250025599_ORT_IPU_20120523_2858-001_R1C1.jp2 which is from Airbus Defense & Space‘s Pleiades sample imagery.
32768 x 16384 px x 4 UInt16 bands
Build / Hardware
JPEG2000 Test Script
- GDAL utilities
- Imagemagick identify utility
- Provides convenient hash algorithm of image data only, discounting header/metadata differences
- The compare utility is also highly recommended for visual diff’ing.
- GNU core-utils timeout utility for killing processes that take longer than 120 secs… you’ll quickly see why this is needed
The script is very simple, especially for anyone who are well versed in the GDAL utilities. There are 3 simple tests performed on every JP2 found in the current working directory,
- Image subset – extract a 800×800 px image from the source input at native resolution
- Image thumbnail – generate a 500×500 px overview or thumbnail from the 15759×15756 source
- Pixel value – extract pixel values from position 2056,2056
Tests in Progress
For JP2MrSID Driver on Test #1. usr/bin/time outputs variety of process metrics, where I manually stored the user time, CPU %, file system inputs and file hash in the spreadsheet of results below. (Clearly outputting csv would have been far more convenient at generating reports.)
Processing 04_072_CPRL_P128_PLT_5QL.jp2.. * Image subset (1/3): Input file size is 15759, 15756 0...10...20...30...40...50...60...70...80...90...100 - done. Command being timed: "gdal_translate 04_072_CPRL_P128_PLT_5QL.jp2 ./output-2015-12-21-1037/JP2MrSID-test1-04_072_CPRL_P128_PLT_5QL.jp2.tif -b 1 -b 2 -b 3 -srcwin 2000 5000 800 800" User time (seconds): 0.52 System time (seconds): 0.10 Percent of CPU this job got: 31% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.97 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 20128 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 16 Minor (reclaiming a frame) page faults: 5864 Voluntary context switches: 167 Involuntary context switches: 67 Swaps: 0 File system inputs: 89032 File system outputs: 3760 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 Hash: 86c00f8ae9ff5f4f1bc19cd2aa209a45edcc663922470672ad1405d6b51ed509
- JPEG2000/Jasper is borderline useless, however this is expected given development has stalled for some time, yet people continue using it. In almost all cases the Driver would spin indefinitely decoding input. Out of 75 tests it completed only 7 and those took a significant amount of time to complete.
- JP2OpenJPEG is preferred by many users due to its BSD license and active development however it still failed 19 tests. Of the test results it did pass, memory requirements were the highest (by significant amounts) and it also remained 10x slower (or more) than the two proprietary drivers tested below.
- JP2MrSID showed 100% completion rate and very low memory usage throughout
- JP2ECW failed 2 tests however in all other results provided the fastest results, on average +12%
Random Comments & Observations
Because I know you will ask…
- Kakadu remains untested as I do not have access to v6 or v7 sources
- Running an identical test on Windows might prove interesting
- All three tests are intentionally basic, yet exercise fundamental driver requirements. Expanding the tests to include multi-threaded reading would further separate the good from the bad. (If someone suggests a simple approach I’d be happy generating the results!)
- Test 1 file hash results show JP2MrSID & JP2ECW matching for all comparison images. JP2OpenJPEG showed two mismatches for the first two files. In both cases the generated TIF files appears truncated or corrupted.
- Test 2 thumbnail file hashes are not intended to match as each driver will implement slightly different resampling algorithms as demonstrated by the Absolute Error difference images between the decoder outputs.
- Test 1 hash results for the 3x real world images shows differences however I believe its related to uint16. Still trying to narrow down the cause.
- The two failures on JP2ECW were Linux specific and has already been resolved in the v5.3 release as well as further speed improvements.
- Testing larger JP2’s in terms of dimensions may separate JP2ECW and JP2MrSID further in terms of performance
- Ever wonder why your Linux Desktop (Gnome etc.) would always crash opening a Folder with JP2’s in it? That would likely be OpenJPEG crashing or running out memory generating the thumbnail images. For unknown reasons most distro’s still ship OpenJPEG v1.5 which is far worse than v2.1
- Sorry, but I cant provide the test data however please do test on your own datasets using the same methodology!
I am well aware this post will no doubt inspire others to improve OpenJPEG performance. My main hope is that through highlighting these faults in the decoders using realistic imagery for Geospatial use, the community will stop tarnishing the whole format with the same brush. Based on my results here and all customer complaints I have been involved with the last few years, it’s the open-source decoders who are to blame 95% of the time (5% admittedly are people using ECWJP2 v3.3 from 2006).
So is JPEG2000 slow? It can be, but the answer is far more nuanced than yes or no. In many ways, it’s the same issue as people creating large GeoTIFF’s with no pyramids, no tiling but on a much larger scale.
Remember the strength of JPEG2000 being an open-standard is choice! Shop around. Compile your own results. Don’t blindly trust some guy on the internet, remember?
Happy New Year!