Image Processing

Much is made today, by manufacturers and users alike, of the image quality attributes of acquisition devices in projection radiography systems. Metrics such as Detective Quantum Efficiency have been developed to measure this quality on a quantitative, absolute scale. However, as the "raw" image quality differences between image acquisition devices decrease, and as these devices begin to approach their theoretical limits, other elements in the medical imaging chain will take on increased importance.

Image processing is one such element. In fact, it is a critical element. Image processing can take the output of a marginally acceptable image acquisition system, and make it qualitatively suitable for diagnostic purposes. On the other hand, image processing can also render useless the output of an excellent image acquisition device. Image processing must be done right in order for the complete imaging system to be clinically useful, but "right" can have different meanings for different applications.

It is foolhardy to view or interpret a diagnostic image without an awareness and understanding of the image processing techniques that were used to produce it. This does not mean that one must become an imaging scientist or algorithm developer to view or interpret digital medical images. However, being able to recognize and appreciate the sometimes subtle effects of image processing can help the viewer to separate anatomy and physiology from art and artifact.

Medical image processing can be classified loosely into three generations. The first generation goes all the way back to the early days of screen/film (S/F) imaging. Each film type had built into it a certain characteristic response curve (called the D-logE curve, since it is a plot of optical Density versus the logarithm of Exposure; also known as the H&D curve after the two researchers, Ferdinand Hurter and Vero Charles Driffield, who first introduced it in the late 1800s).

This generally S-shaped curve describes how x-ray exposure (and changes in x-ray exposure, sometimes called subject, or radiation contrast) is converted into optical density (and changes in optical density, sometimes called radiographic contrast) on the film. The shape of the curve, which also depends on chemical processing conditions, provides the visual "look" of that particular film type. For example, high-contrast films have a steep, narrow H&D curve, while "latitude," or low-contrast films have wider curves with lower slope. The right film depends on the needs of the application, and on the diagnostic preferences of the viewers.

With the dawn of the digital age in medical imaging (the late 1970s and early 1980s), the 1st generation expanded to include computer-based image processing techniques. In addition to a greater degree of control of signal amplitude (for example, using histograms and window width and level adjustments), image processing started using the spatial frequencies of image signals as a variable. For example, higher spatial frequencies could be used to adjust the visibility of small structures and the visibility of image noise, while lower spatial frequencies contributed more to the appearance of larger structures.

Edge enhancement (the most familiar algorithm in this class being unsharp masking ) and noise reduction are examples of the new digital tools that were now part of the 1st-generation image processing arsenal. Processing techniques could be tuned by the manufacturer (or by the user!) to produce the preferred "look" for each body part/projection or application. These relatively simple, proven techniques are still used today in some systems.

The desire for even greater control of image appearance led to the 2nd generation of image processing techniques. Algorithms in the 2nd generation are more complex and sophisticated than those of the earlier years. An example of a 2nd-generation technique is multiscale processing, in which the original image is decomposed into a collection (up to 12 in some systems) spatial-frequency ranges. Each range can be processed differently, providing users with ability to customize the output "look" for each body part, projection, or special application. The 2nd generation requires a good deal of user input and iterative on-site visual optimization in order to determine the many processing parameters for each image type. Only in this way can the performance of the image processing be optimized for each user or site.

This exquisite control of image display properties has become the hallmark of modern digital imaging systems, but it brings with it a "dark side," namely, the need to know exactly what you are doing. As noted, it is very easy to make relevant anatomy, physiology and pathology disappear with image processing. It is much more difficult to figure out how to enhance just the right structures to provide the optimum diagnostic presentation to the viewer. Medical imaging manufacturers continue to spend considerable time, effort and money developing robust image processing algorithms that provide a net benefit in the applications in which they are used.

The 3rd generation of image processing is just starting. One major goal of this generation is to eliminate the need for extensive user input and interaction, one of the main drawbacks of the earlier generations. The final image should just "appear" on the output device, with the attributes needed to optimize the diagnostic interpretation. This simple concept requires an even greater degree of complexity and sophistication in the processing algorithms. Some new algorithms can already operate autonomously (i.e., without user intervention), creating not only optimized images for display, but, as a by-product, a smoother, more efficient workflow, which enables users to focus more on patients than on images. This 3rd generation of image processing must be "intelligent" enough to analyze the content of each input image, and decide how best to present the clinically relevant details. Features such as automatic body-part recognition, automatic collimator recognition, and application-specific processing-parameter setting fall into the category of 3rd-generation image processing.

Below is a set of images that illustrate the various generations of image processing. Note the differences in detail visibility, grayscale reproduction and artifacts between the generations.

Image Sets

Chest

Images of the human thorax pose one of the most difficult image processing challenges. Chest images tend to have a large exposure dynamic range, clinically relevant details in both light and dark areas of the image, and a large variety of possible subtle and obvious pathologies, all of which must be properly displayed, and none of which are usually known a priori.

Meeting these diverse requirements has been particularly difficult for S/F systems. Over the past century, film manufacturers have spent considerable resources trying to design the perfect S/F system for chest imaging. Digital imaging systems, with their innate separation of acquisition, processing and display, provide opportunities for a cleaner solution. In particular, their wider acquisition latitude can capture more information than can the S/F system, which must simultaneously act as capture medium (wide latitude important) and display medium (contrast important).

Image processing can then take the acquired data and attempt to present them for optimal interpretation. The accompanying images show four different reduced-resolution, processed versions of the same original CR chest image, along with small sections of each image at their native resolution.

Chest SF Small

1st Generation - S/F "Look"

1st Generation (digital) - Unsharp Masking (Edge Enhancement - program boosts contrast of higher spatial frequencies relative to lower spatial frequencies, according to frequency threshold selected by the user/manufacturer for the application)

2nd Generation - Multiscale algorithm for contrast adjustment (program adjusts local contrast in multiple spatial frequency ranges, as determined by input parameters selected by the user/manufacturer)

3rd Generation - Intelligent multiscale algorithm (no user intervention needed - program analyzes image in multiple spatial frequency ranges and optimizes display for both soft tissues and skeletal details automatically)

Knee

Imaging of the extremities requires not only high resolution for bony structures, but also the ability to view subtle contrast changes in soft tissue all the way to the skin line. Due to their relatively narrow exposure latitude, this is difficult with most S/F systems. When the bone has optimal contrast, the soft tissues near the skin line are generally too dark, necessitating the use of a "hot light."

Digital imaging systems, with their independent acquisition, processing and display functions, offer a potential solution. The ability to adjust image processing to the characteristics of the input image provides increased flexibility in the final display for interpretation. The accompanying images show four different reduced-resolution, processed versions of the same original CR knee image, along with small sections of each image at their native resolution.

1st Generation - S/F "Look"

1st Generation (digital) - Unsharp Masking (Edge Enhancement - program boosts contrast of higher spatial frequencies relative to lower spatial frequencies, according to frequency threshold selected by the user/manufacturer for the application)

2nd Generation - Multiscale algorithm for contrast adjustment (program adjusts local contrast in multiple spatial frequency ranges, as determined by input parameters selected by the user/manufacturer)

3rd Generation - Intelligent multiscale algorithm (no user intervention needed - program analyzes image in multiple spatial frequency ranges and optimizes display for both soft tissues and skeletal details automatically)

Ankle

The superposition of bony structures (low x-ray transmission) can make it difficult to visualize soft tissues on S/F. Digital systems, with their wider exposure latitude and image processing, can help. The accompanying images show four different reduced-resolution, processed versions of the same original CR ankle image, along with small sections of each image at their native resolution.

1st Generation - S/F "Look"

1st Generation (digital) - Unsharp Masking (Edge Enhancement - program boosts contrast of higher spatial frequencies relative to lower spatial frequencies, according to frequency threshold selected by the user/manufacturer for the application)

2nd Generation - Multiscale algorithm for contrast adjustment (program adjusts local contrast in multiple spatial frequency ranges, as determined by input parameters selected by the user/manufacturer)

3rd Generation - Intelligent multiscale algorithm (no user intervention needed - program analyzes image in multiple spatial frequency ranges and optimizes display for both soft tissues and skeletal details automatically)

Skull

The skull also presents a large dynamic range for display - much of the soft tissue around the skull, such as the nose, is usually difficult to see without special processing (even a "hot light" may not be able to reveal this area of high relative exposure in a S/F image). The accompanying images show four different reduced-resolution, processed versions of the same original CR skull image, along with small sections of each image at their native resolution.

1st Generation - S/F "Look"

1st Generation (digital) - Unsharp Masking (Edge Enhancement - program boosts contrast of higher spatial frequencies relative to lower spatial frequencies, according to frequency threshold selected by the user/manufacturer for the application)

2nd Generation - Multiscale algorithm for contrast adjustment (program adjusts local contrast in multiple spatial frequency ranges, as determined by input parameters selected by the user/manufacturer)

3rd Generation - Intelligent multiscale algorithm (no user intervention needed - program analyzes image in multiple spatial frequency ranges and optimizes display for both soft tissues and skeletal details automatically)

Paediatric Spine (Lat)

The lateral spine is a very difficult exam to capture, due to its very wide dynamic range. In S/F systems, soft tissue is frequently "burned out" near the skin line due to the range of x-ray exposures involved, and the primary need to visualize the bony details. In addition, the (dark) overlapping lungs behind the hemi-diaphragms can make details in the spine difficult to see. The accompanying images show four different reduced-resolution, processed versions of the same original CR paediatric spine image, along with small sections of each image at their native resolution.

1st Generation - S/F "Look"

1st Generation (digital)- Unsharp Masking (Edge Enhancement - program boosts contrast of higher spatial frequencies relative to lower spatial frequencies, according to frequency threshold selected by the user/manufacturer for the application)

2nd Generation - Multiscale algorithm for contrast adjustment (program adjusts local contrast in multiple spatial frequency ranges, as determined by input parameters selected by the user/manufacturer)

3rd Generation - Intelligent multiscale algorithm (no user intervention needed - program analyzes image in multiple spatial frequency ranges and optimizes display for both soft tissues and skeletal details automatically)

Chest Phantom - Dose Series

There is an interaction between dose and image processing. At lower dose levels, the signal-to-noise ratio (SNR) of images degrades (they look noisier). Increasing the contrast of such images, either globally or in multiple spatial-frequency bands, can also increase the noise impression, and decrease the subjective image quality. Thus, the ability to do image processing is often limited at lower dose.

Unfortunately, many image processing algorithms treat all images the same way, and, thus, do a poorer job on lower dose images. Algorithms that measure local image SNR can do a better job of adjusting the extent of processing so as to avoid degradation in subjective image quality. Modern (3rd generation, and some 2nd generation) programs can adapt their processing to the local (or global) SNR of the image being processed.

The accompanying images show a chest phantom exposed at three different dose levels, 0.5 mAs, 4 mAs, and 32 mAs, all at the same kV (125 kV). Each dose level is processed with a 1st-, 2nd-, and 3rd-generation processing algorithm (a S/F "look"(top), a multiscale technique requiring user input of parameters (middle), and an intelligent multiscale technique that sets its own parameters based on image analysis (bottom)).

As expected, the noise appearance improves as the dose increases (the SNR is increasing). However, the 1st-generation, S/F-like display produces essentially the same radiographic contrast at the three dose levels, which is consistent with the fixed shape of its characteristic response. The 2nd-generation algorithm also maintains roughly the same detail contrast as the dose increases. The 3rd-generation technique is able to improve detail contrast as the dose increases, adjusting its processing to the improved SNR. In fact, the detail visibility of the 3rd generation technique at lower dose levels can be better than that of the 2nd- or 1st generation techniques at higher doses. This raises the possibility of reducing patient dose during acquisition, and allowing the image processing technique to compensate intelligently for the poorer SNR, while still delivering a diagnostically useful output image.

Analog (i.e., film-based) methods of edge enhancement, such as unsharp masking were already in use in the 1930s in halftoning applications for printing. Their use in medical image processing did not come until half a century later.