Imaging Challenges, Part Deux

StevenSimske — Sat, 20 Sep 2008 11:30:00 GMT

In a blog post earlier on this busy blogging week (hard to tell I'm spending a lot of time rotting in airports/hotels, no?), I introduced some of the difficulties in image clustering, or aggregation. This post introduces some of the broad approaches used to solve such imaging challenges.

Broadly, there are at least three classes of image analysis technologies used to compare photos (more broadly termed "images" to include scanners, cameras, inspection systems, video, and all other forms of "image capture"):

1. Machine vision/pattern recognition

2. Segmentation-based approach

3. Image modeling

There is some overlap among these three approaches, but they are distinct enough to proceed.

1. Machine vision/pattern recognition: This approach typically uses correlation (a statistical measure of image similarity) to compare two images. Correlation can be used to compare texture, frequency, color, shape and/or other content in the two images compared. The use of frequency-based (looking for how an image varies spatially in one or more directions) comparisons allows images of different scale (size of features) to be readily compared by simply scaling the frequency outputs relative to one another. Best matches provide the scale differences and subsequent alignment of the relatively scaled images. Machine vision systems usually are initialized through "training" the system by capturing one or more images of a calibrating (or "ground truth") feature to which the other image(s) is(are) compared. Clearly, such pattern recognition or "machine vision" based systems are especially amenable to inspection, wherein many images are to be compared to the calibration image. This type of comparison is especially effective for comparing Pictures #1 and #2 in the "Imaging Challenges" post of two days past.

2. Segmentation-based approach. Image "segmentation" is the process by which an image is divided into regions, called segments, which can thereafter be used as individual images for comparison. Therefore, this approach is recursive inasmuch as it affords refined segmentation, or sub-segmentation, as further information needs to be extracted. Images which contain steganographic (hidden) information such as digital watermarks, fiducial marks, and the like, are effectively analyzed by these approaches. Note that this approach is also taken on the "storage" side for many compression approches, such as tiled JPEGs. This segmentation-based approach can be used to cluster Picture #3 in the "Imaging Challenges" post with Pictures #1 and #2.

3. Image modeling. An image model is a description of salient features for the image analysis algorithm to find in the image. This may be conveyed through a template (description of layout), a feature set, or other means. An "image model" assumes that the image processing system is capable of "image understanding", meaning it is capable of accurately deciding whether or not an image contains a match to the model defined. Such a system relies on powerful statistical classifiers to decide on a "yes" or "no" for the match. Clearly, the goal is to have 0% false positives (no regions identified as matching that actually are not matches) and 0% false negatives (no regions that actually are matches but are missed, or unidentified). In reality, though, the system is tuned to provide the best overall accuracy based on the task at hand. If the cost of a false positive is much higher than the cost of a false negative, then the system should be tuned to favor false negatives (miss some matches) but have very high confidence in the matches. This is usually the case for the reading of security-related information [in future blogs, I will discuss classification in greater detail]. Note that, if each GPS location has a "model" for the images that can be captured (such as a 3-D panoramic), there is the possibility that Picture #4 in the "Imaging Challenges" post could be aggregated with Pictures #1-#3 (even though the content in the Picture has no overlap with the other 3). Currently, however, except in expensive systems for high-security locations, these models do not yet exist. But, they will in the future (think about 3-D video games, for example, in which a 3-D panoramic model is created for various "rooms" in the game).

-Steve

Security Printing and Imaging : authentication, image understanding

Imaging Challenges, Part Deux