Higher-Order Active Contour Energies for Gap Closure

One of the main difficulties in extracting line networks from images, and in particular road networks from remote sensing images, is the existence of interruptions in the data caused, for example, by occlusions. These can lead to gaps in the extracted network that do not correspond to gaps in the real network. In this paper, we describe a higher-order active contour energy that in addition to favouring network-like regions, includes a prior term penalizing networks containing ‘nearby opposing extremities’, thereby making gaps in the extracted network less likely. The new energy term causes such extremities to attract one another during gradient descent. They thus move towards one another and join, closing the gap. To minimize the energy, we develop specific techniques to handle the high-order derivatives that appear in the gradient descent equation. We present the results of automatic extraction of networks from real remote-sensing images, showing the ability of the model to overcome interruptions.


Introduction
The huge growth in the amount of digital imaging data of various types available in many fields, including remote sensing, medicine, and biology, makes the construction of systems capable of automatically extracting information of semantic value from this data a necessity.While each application comes with its own semantics, prior knowledge, and other specificities that mean it must in principle be treated anew, in practice there are frequently commonalities between different applications that make a somewhat more generic approach feasible.Line networks are an example.They represent information of semantic value in many applications: vascular networks in medicine; filamentary structures in biological images; road and river networks in remote sensing and cartography; and galactic filaments in astronomy; while possessing many properties in common across these applications.The extraction of their properties from imaging data (most often, the identification of the region in the image domain corresponding to the network) is thus of some importance, but semi-automatic extraction remains a time consuming and expensive task.Research concerned with these problems has therefore begun to focus on the development of efficient methods for the automatic extraction of line networks.In this paper, we particularly focus on the extraction of line networks from remote-sensing images, but the models described should be equally useful for imagery from other applications, including medical and biological images.
In order to be able to extract networks, we must first be able to model them well, that is, be able implicitly or explicitly to put an image-dependent probability distribution P(R|I ) on the space of regions in the image domain whose mass is concentrated on the region corresponding to the network in I .Such a probability can of course be decomposed into two pieces: a prior probability on the space of regions given that the region corresponds to a network, P(R), and a likelihood describing the images to be expected given that the region R in the image domain corresponds to a network, P(I |R).The construction of distributions P(R) and P(I |R) that generate a posterior probability P(R|I ) concentrated on the network in a given image is not easy.Generic priors that are not concentrated on network-shaped regions combined with likelihoods based on local image measurements are not sufficient to concentrate P(R|I ) on the network sought, because local image measurements typically assume similar values for many regions that do not correspond to the network, and the prior is incapable of distinguishing between them.To improve the situation, one can thus advance in two directions.The first is to design likelihoods that capture some of the dependencies amongst the image values associated with the network, the use of line detection filters being the most common example.Even this does not suffice, though, since the measurements are still relatively local, and may be similar for many structures that do not form part of a network.The second is to design priors that are concentrated on network-shaped regions.Constructing such priors is also non-trivial, however, first because the space of regions is an infinite-dimensional nonlinear space, and second because networks form a subset of this space that is difficult to characterize.Networks possess strongly constrained geometric properties (e.g., narrow arms with roughly parallel sides), but arbitrary topology.They cannot be defined, for example, as variations around a mean shape.
Rochery et al. [17] (for more detail see [21]) have proposed a method for the quasi-automatic 1 extraction of line networks based on advances in both these directions.These advances make use of a new generation of active contour models, introduced by Rochery et al. [17,21], and named 'higher-order active contours' (HOACs). 2 While classical active contours use only boundary length and interior area (and perhaps boundary curvature) as prior knowledge, HOACs allow the incorporation of non-trivial prior knowledge about region geometry, and the relation between region geometry and the data, via nonlocal interactions between tuples of contour points.They are also intrinsically Euclidean invariant.They differ from most other methods for incorporating prior geometric knowledge into active contours [2,3,5,11,15] in not being based upon perturbations of a reference region or regions.This allows them to model regions consisting of an arbitrary number of connected components, for example, in which the morphology of each component and the inter-component interactions are controlled.Using 1 By this we mean that no human initialization is required, but that the model possesses parameters that cannot at present be set automatically. 2 Nain et al. [13] use an energy of the same form, although expressed in terms of region integrals, to segment vessels in medical images.
this new framework, Rochery et al. [17,21] proposed a model that goes a long way towards capturing the prior geometric knowledge we have of network regions, as well as the complex dependencies between image values associated with networks.The prior model has as low-energy configurations, regions composed of arms of roughly constant width that join together at junctions.The likelihood model predicts not just high image gradients along the edges of the network, but incorporates longer-range dependencies that predict that image gradients along one side of a network arm will be parallel, while image gradients on opposite sides of a network arm will be anti-parallel.
Thanks to this prior knowledge, the model produces good results using gradient descent to minimize the contour energy, starting from a generic initialization that renders the method quasi-automatic.The primary failure mode of the method is the presence of 'gaps' in the extracted networks caused by 'interruptions' in the image of the road.These interruptions are caused by various types of 'geometric noise': in the case of road networks, for example, trees and buildings close to the network that change its appearance either via occlusion or because of cast shadows.The method fails to close these gaps for three reasons, two related to the model, and one to the algorithm: (1) the prior knowledge concerning the geometry of the network (P(R)) does not distinguish between two distant arms that each comes to an end, and two arms that form a gap, once the extremities are more than a few pixels apart.Thus the model does not capture our prior knowledge that road networks, for example, usually do not possess such gaps; (2) the prior knowledge concerning the image to be expected from a given network (P(I |R)) does not allow for the possibility that there will be interruptions in the observed road; (3) the gradient descent algorithm may be unable to close the gap even if the configuration with the gap closed has lower energy than the configuration with the gap present, due to the shape of the energy surface between the two configurations.
[18] made a preliminary attempt to address the gap closure problem.They introduced a 'gap closure' force making nearby opposing network extremities attract one other, thus closing gaps between them.The force was introduced directly to the gradient descent equation.While the results obtained with this force are similar in quality to those obtained via the new work in this paper, the force was not a total functional derivative, i.e., it could not be obtained from an energy.This complicates analysis, and more seriously means that convergence is not guaranteed.It is the