Help! My ECW Has Speckled Edges!

They appear out of nowhere. They set up camp and act like they own the place. How did they get there, and who invited them? I’m not talking about your nosy neighbor, I’m talking about a different problem: those speckled edges. Also known as transparency remnants, removing collars, nodata transparency, edge patterns and I’ve even heard a customer refer to it as the “cow pattern” but let’s just stick with calling it the “speckled edge” for now as all GIS.SE questions here are talking about the same thing.

It’s a problem that continues to haunt users loading ECW files in a variety of applications. These ugly artifacts appear along the edge of the dataset and is particularly noticeable when you have gone through the painstaking process of mosaicking or trying to view underlying datasets:

ecw artifacts

Via GIS Stackexchange, display in Esri ArcGIS for Desktop

The problem with trying to solve these questions on various online communities is that they are confusing two completely separate concepts:

  • nodata or null pixel transparency
  • transparency / alpha / opacity masks (also called Opacity bands).

They are very different but continue to be used interchangeably which leads to at least part of the confusion.

So what can you do..?

Recommendation 1: Setting nodata transparent values is never recommended for ECW or JP2.

ecw artifacts lost shadows

Non-application specific, this time in GeoMedia 2014 but shows unintended consequences losing shadowed areas

Applications such as GDAL created nearblack and GeoMedia, QGIS, ArcGIS, FME supports multiple nodata values, but it’s arguably a workaround that will never work consistently because the degree of colour shifting in the background (white or black) will never be the same and you risk losing areas within your image such as shadowed areas.

It’s at best a quick fix but as soon as you compress to ECW, say after mosaicking, you will once again have the speckly edges return because it’s a symptom of the compression. The other problem is interoperability.  Whilst most applications support knocking out one nodata value—such as 0,0,0—not all support removing a range. This leads to an inconsistent user experience when viewing ECW’s in different applications.

no opacity

ECW with no opacity band, black background.

nodata transparency

Nodata transparency enabled in QGIS 2.2 for 0,0,0

Ecw with nodata

ECW display with nodata enabled and white background. Ugh

ecw with 5 nearblack values

ECW display with 5 near black nodata values. Double ugh!

Now I have hopefully convinced you that knocking out specific or a range of pixels is not a great scalable idea. The portable solution is to somehow persist this information back into the file. But how?

Recommendation #2:  Always compress using a transparency/opacity/alpha mask.

Using the ECWJP2 SDK terminology, a transparency mask is implemented as an opacity band called AllOpacity. Like other formats this mask controls whether underlying RGB bands will be transparent or opaque and nothing in between. This completely removes any compression artifacts and solves our other problem of portability since this information is stored within the ECW file. The last question then is application support which leads me to the other common problem that customers run into. Lack of a time travelling device!

ECW was designed during the 1990s and predates any use of Opacity bands in the industry. This means the first version of the publicly available decoding SDK, v3, has no support. Period. Around 2008 we added support for Opacity bands in the v4 SDK by appending the information in a backwards-compatible way and was first released in ERDAS IMAGINE 2010. But this turned out to be a double edged sword as it ensured everyone in the industry could continue reading ECW files but it was never clear to users why one application could view the alpha mask (v4) while others couldn’t (v3) since it really comes down to the version of the SDK being used internally by each application. Unfortunately, that is not something that Hexagon Geospatial can control (thus my desire for time travel).

Version First Released Last Released Opacity Support
ECW JP2 SDK v3.x 2004 2008  Not supported
ECW JP2 SDK v4.x 2009 2012 ECW v2 (Appended) supported
ECW JP2 SDK v5.x 2013 2015* ECW v2 (Appended) & ECW v3 (Interleaved)  supported

Our latest release of the v5 ECWJP2 SDK has full support for Opacity bands in ECW v2 and the new ECW v3 format. With all the other benefits of v5 such as extensive platform support and performance improvements, if you are still running into this speckled edge problem now is the time to upgrade or ask your application/data provider to get with the program! It hasn’t been an ECW format limitation for many years now.

Some examples of what an embedded ECW Opacity channel looks like in various applications are below.

band4alpha channel

Band4 interpreted as Alpha in QGIS 2.2

alpha in esri

Band_4 as Alpha band in Esri ArcGIS for Desktop 10.2

alpha in IMAGINE

Opacity layer = yes in ERDAS IMAGINE 2014 Image Metadata (auto-applied)

gdal alpha

GDALinfo 1.10 reporting

As a user of Application X, how are you supposed to know whether it supports reading/writing ECW Opacity bands?

Well if you are so inclined, you can browse to the Application binary location (typically /bin) and locate a file named either NCSEcw.dll, NCSEcw4.dll or NCSEcw4_RO.dll and view the file properties. Since v4, Hexagon Geospatial only provides signed, fully versioned DLL’s so for example Esri ArcGIS for Desktop 10.3 ships v5.0.1 by default, Globalmapper 16 v5.1.1, QGIS 2.6 v5.0.1 and ERDAS IMAGINE 2015 v5.2.0. Of course just because an application ships a version greater than v4.x still might require work at the application level to enable it but you can be 100% sure its not a limitation of the ECW format.

Pssst… since you’ve read this far down, the latest ECWJP2 SDK v5.2 release previewed here is now available!

 

  • Registration for HxGN LIVE 2018 Is Open

    Why register later when you can do it today? Register now to join us at next year's conference!

  • Recent Posts