Computing layered surface representations: An algorithm for detecting and separating transparent overlays

The biological visual system possesses the ability to compute layered surface representations, in which one surface is represented as being viewed through another. This ability is remarkable because, in scenes involving transparency, the link between surface topology and image topology is greatly complicated by the collapse of the photometric contributions of two distinct surfaces onto image intensity. Previous analysis of transparency has focused largely on the role of different kinds of junctions. Although junctions are important, they are not sufficient to predict layered surface structure. We present an algorithm that propagates local junction information by searching for chains of polarity-preserving junctions with consistent 'sidedness,' and then propagates the transparency labeling into interior regions. The algorithm outputs a layered representation specifying (i) the distinct surfaces, (ii) their depth ordering, and (iii) their surface attributes. We demonstrate the results of the algorithm on a number of images - both synthetic and real. We end by considering implications for related domains, such as shading.