Glass patterns

Figure 1: Glass patterns. A, A set of random dots created by Leon Glass in 1969 by randomly dropping paint on paper. B, That same set of random dots with a transparency of itself copied, rotated, and overlaid.

Glass patterns are formed from the superimposition of two random dot patterns: an original and a second pattern generated following a linear or nonlinear transformation of the original. Though each set is random, a variety of different spatial patterns such as circles, spirals, hyperbolae, can be generated by introducing correlations between the two sets of dots. Figure 1A is a random dot pattern formed by randomly dropping paint on paper, and Figure 1B is the image formed from the superimposition of Figure 1A upon itself following a rotation (Glass, 1969). In this image, for each dot there is a corresponding "partner" dot that lies along the circumference of a circle centered at the point of rotation of the two images. The visual system is able to decode these correlations, thereby perceiving the underlying global transformation.

[edit] Variants

By carrying out other transformations, different geometries can be readily generated. First consider linear transformations given by

\[x'= ax \cos \theta - b y \sin \theta\]
\[y'= ax \sin \theta + b y \cos \theta\]

where \((x,y)\) are the coordinates of the original dot and \((x',y')\) are the coordinates of the transformed dot following a scaling of the \(x\)-coordinate by an amount \(a\ ,\) a scaling of the \(y\)-coordinate by an amount \(b\ ,\) and a rotation of the image about the origin by an angle \(\theta\) (Glass and Perez, 1973).

For the linear transformations, the geometries of the perception can be determined from the eigenvalues of the linear transformation. The eigenvalues are given by

\[\lambda_\pm = \frac{(a+b) \cos \theta \pm \sqrt{(a-b)^2 - (a+b)^2 \sin^2 \theta}}{2}\]

If the eigenvalues are pure imaginary numbers, then the transformation corresponds to a rotation; if the eigenvalues are complex numbers then the image is a spiral, corresponding to a focus; it the eigenvalues are real with the same sign, then the image corresponds to a node in nonlinear dynamics; and if the eigenvalues are real with different signs, then the resulting hyperbolic image corresponds to a saddle point. If the transformation transports each dot too far away from its original position, then we can no longer perceive the underlying geometry.

Figure 2: Example Glass patterns. A, hyperbolic or "saddle" pattern, a=0.95 / b=1.05 / \(\theta\)=0. B, spiral pattern, a=1.05 / b=1.05 / \(\theta\)=5. C, concentric or circular pattern, a=1 / b=1 / \(\theta\)=5. D, radial or "starburst" pattern, a=1.05 / b=1.05 / \(\theta\)=0

These transformations can be implemented easily with modern computer graphics applications (see External Links below). Figure 2 shows several images generated by using this transform on 1000 randomly positioned dots, generating hyperbolic (Figure 2A), spiral (Figure 2B), concentric (FIgure 2C), and radial (Figure 2D) geometries.

There are a large number of other ways in which the images can be modified to make perception of patterns difficult (Glass and Switkes, 1973; Wilson and Wilkinson, 1998; Barlow and Berry, 2010). Some of the modifications are: parts of the image can be eliminated, each of a correlated pair of dots can be generated with different contrast or different color with respect to the background, a random term can be added to the transformation, and extraneous pairs of dots that do not conform to the transformation can be added. A theoretical understanding of the mechanisms underlying the perception of these images should be able to predict the various ranges of parameters that would lead to perception of the pattern in each of these variants.

[edit] Physiological Basis of Perception

Glass patterns contain both local correlations and global structure which can be manipulated independently. As noted in the original description of these patterns, the spatial autocorrelation function of the images captures the local structure. The local cues present in Glass patterns are quite weak because each pair of dots is embedded in a noisy and random background comprising other dot pairs. Nevertheless, a strong percept of global form arises from these sparse local cues. This suggests two stages of processing: an initial stage in which the local cues are detected, and a second stage of processing where the local cues must be integrated across the entire image to achieve a global percept.

Most physiological interpretations of Glass pattern perception have been based on Hubel and Wiesel's (1962) early description of the structure and response properties of neurons in visual cortex. They found a class of neurons called "simple cells" that were sensitive to the orientation of bars of light presented in a particular region of visual space (the "receptive field"). Furthermore, they exhibited "on" and "off" subregions of their receptive field (sensitive to increments and decrements of light, respectively). Another class of cells, "complex cells", were also responsive to elongated stimuli of a fixed orientation but with somewhat larger receptive fields that respond to the information therein in more complicated ways. They also identified a columnar structure in visual cortex, in which nearby neurons tended to prefer similar stimulus orientation, and this preference shifted smoothly across cortical space. Hubel and Wiesel proposed a hierarchical structure in which the complex cells received input from simple cells, although continuing research into the circuitry of visual cortical neurons and their inputs has revealed that the connections are far more complex than can be described by that simple wiring description (see Priebe and Ferster, 2008 for review).

Glass (1969) proposed that this anatomical structure would be able to carry out computation of the local autocorrelation function. Two correlated dots falling in the excitatory receptive field of a simple cell would lead to excitation of that simple cell. Further, correlated dots nearby would fall in the receptive fields of simple cells in the same column, whereas uncorrelated dots nearby would fall in the receptive fields of cells in other columns. The resulting strong intracolumnar excitation of the complex cells could provide a substrate for detection of the local autocorrelation functions.

Figure 3: Three example Glass patterns are shown in the top row. A, A linear or translational pattern. B, A concentric pattern. C, A radial pattern. In the bottom row, small portions of each pattern (indicated by the gray box) have been enlarged. Despite the distinct global forms, the local structure for all three patterns appears similar to a human observer and to an example V1 receptive field (shown as the red and green Gabor).

This model appears to be consistent with subsequent physiological studies of electrical activity of neurons in the primary visual cortex (also known as area V1) of macaque monkeys. Smith et al. (2002) recorded action potentials from single neurons in response to dot patterns generated by superimposing a random set of dots on itself following a translation. They found that the receptive field properties of the neurons, as measured with oriented stimuli, were predictive of their preferences for Glass patterns. There was also good agreement between the experimentally recorded activity and a theoretical model of simple cells based on Hubel and Wiesel's original proposals. This model, in which pairs of dots stimulate V1 receptive fields based on their orientation and separation, successfully predicts a number of features of Glass pattern perception. For instance, consider the case of opposite polarity Glass patterns (where each dot pair is of different color or contrast). Here, a concentric pattern often evokes a kind of roughly orthogonal "radial" percept. This can be explained from that simple model: two dots of opposite polarity most effectively stimulate a neuron when they are orthogonal to the lobes of the receptive field and are separated by the spatial period of the cell (Smith et al., 2002). More research will be necessary to distinguish between a model in which V1 neurons perform a simple filtering of the visual stimulus vs. an autocorrelation. In addition, the neurons in early visual cortex do not seem capable of signaling the global form information present in Glass pattern stimuli (Smith et al., 2002; 2007), leaving the integration stage of Glass pattern perception to higher areas in visual cortex (Figure 3). There is some evidence that visual area V4 may serve as at least a portion of this integration stage. Neurons in area V4 are selective for concentric, radial and cross-shaped structures (Gallant et al., 1993, 1996; Kobatake and Tanaka, 1994), exactly the kind of global form sensitivity that would support Glass pattern perception.

Psychophysical experiments support the importance of autocorrelation in the processing of Glass patterns (Barlow and Berry, 2010). In support of the notion that the column may be playing a crucial role in visual perception, Zucker (2002) proposed that excitatory interactions between individual neurons in a given "clique" of cells, all of which have similar orientation specificity and are located in a given column, might be playing an important role in contour detection. In this formulation, a "clique" of cells carries out the averaging operations that are necessary to compute the local autocorrelations. An alternative, although not necessarily mutually exclusive, approach to understanding Glass pattern perception has come in the form of a neural network model of boundary and form perception. Grossberg and colleagues (Grossberg and Mingolla, 1985; Gove et al., 1995) incorporated top-down feedback, bottom-up feed forward interactions, and multiple levels of the visual hierarchy (thalamus, V1, V2 and V4) into their model. It is capable of providing explanations of and predictions for a large number of visual effects and illusions including Glass patterns.

Regardless of the initial mechanism for computing the local image statistics of a Glass pattern (local autocorrelations or another method), it is still necessary to integrate information from different regions of the image to form the global percept. Psychophysical studies carried out by Wilson and Wilkinson pose sharp questions about the nature of the intercolumnar information processing. By partially removing some regions of the correlated dot images, they determined that patterns with concentric structure (e.g., see Figure 1 from Wilson et al., 1997) were easier to perceive than the other types of correlated dot images, followed by radial and then translational patterns. Since the local information was the same in the various images, the differences in ability to perceive the images would be due to the integration steps. These findings were disputed by Dakin and Bex (2002), who argued that the shape of the window in which the dot images were displayed played an important role in determining the detection thresholds (see Wilson and Wilkinson, 2003 and Dakin and Bex, 2003 for further discussion). However, a number of subsequent studies have found differential sensitivity to global forms both in human psychophysics (Seu and Ferrera, 2001; Badcock et al., 2005; Palomares et al., 2010; Khuu et al., 2011) and scalp potentials (Pei et al., 2005), supporting Wilson and Wilkinson's initial findings. Wilson et al. (1997) proposed a two-stage model to account for their results in which an initial oriented filtering stage (representing V1) is following by rectification and a second stage of filtering and integration. Based on their psychophysical findings, Wilson and Wikinson suggested that neurons in area V4 are a likely candidate for this integration process (Wilson et al., 1997; Wilson and Wilkinson, 1998). This model provides a plausible explanation for the means by which global form information is pooled and also can explain the variation in threshold among different patterns.

Because of its ability to measure responses simultaneously across the visual hierarchy, functional magnetic resonance imaging (fMRI) is an appealing means to explore the stages of Glass pattern processing. Using fMRI in human observers, Ostwald et al. (2008) found support for idea that Glass patterns are processed in two stages. They showed sensitivity to global form present in Glass patterns is mediated by higher visual areas that pool the local orientation signals provided by early visual areas. However, there is some evidence that sensitivity to curvature and global form present in Glass patterns exists as early as primary visual cortex (Mannion et al., 2009; Mannion et al., 2010; Mannion and Clifford, 2011). Nonetheless, even if neurons in V1 and other early areas have some selectivity for global form characteristics, substantial integration by higher cortical areas is necessary to support the full perception of Glass patterns.

For the situation in which the dots are relatively sparse and small, thinking about an image as the union of multiple pairs of dots is probably a good way to think about the structure of the image. However, as the density and sizes of the dots increase, Glass patterns have streaky appearances in which the streaks are oriented along a path revealing the underlying structure of the field. If the pattern is not completely additive (that is, all the dots are full contrast such that overlapping dots do not sum to create a region of higher contrast where they overlap) then a simple intuition about Glass pattern detection can break down. In dynamic Glass patterns, the random positions of the dots are updated temporally to produce a rapid sequence of patterns following the same dot pairing rule. These types of patterns contain no coherent motion signal, but do suggest a "path" of motion based on the streaks produced by the dots. These motion cues are strong enough to influence the motion perception of human and monkey psychophysical observers as well as the responses of individual neurons in macaque visual cortex (Krekelberg et al., 2003) and the fMRI BOLD signal in human visual cortex (Krekelberg et al., 2005) in areas that are specialized for processing motion.

[edit] Future Challenges

The visual system is amazingly adept at perceiving structure in the visual field and interpreting it accurately in a very short time. Although the structure of Glass patterns has a certain elegance and simplicity that invites analysis, it is not clear how the outlined mechanisms would be more generally applicable to interpret more complex images. Moreover, the above discussion of the computations carried out in the perception of these images leaves out features that we know are important for perception, including eye movements and feedback interactions between different regions of the visual system. Although various models suggest that the columns are the locus for the computation of local spatial autocorrelation functions, physiological evidence for this is still lacking. Furthermore, while higher visual cortical areas have been suggested as the locus of integration of Glass pattern form cues, this has not been explored at the single neuron level.

[edit] History

Glass completed his PhD at the University of Chicago in 1968 using spatio-temporal correlation functions to study "Theory of Atomic Motions in Simple Liquids". He then embarked on postdoctoral study, supported by an NIH Fellowship, at the Department of Machine Intelligence and Perception at the University of Edinburgh with a goal of applying methods used in his doctorate to study visual perception. Shortly after his arrival in Edinburgh, Glass' supervisor, H. C. Longuet-Higgins returned from a trip in which he learned of an interesting experiment done by Erich Harth (Harth, Beek, Pertile, Young, 1970). Take a blank piece of paper. Place this on a Xerox machine and make a copy of it. Now make a copy of the copy. This procedure is then iterated, always making a copy of the most recent copy (see Fig. 9 in Harth et al., 1970).

During this procedure, small imperfections in the paper and dust on the optics of the Xerox machine introduced noise that eventually led to an image with many dots that changed little under future iterations. (Current copiers use different technologies so the experiment is no longer easily reproduced). In order to analyze the local inhibitory fields leading to the interesting spatial structures, Glass made a transparency of the dots to project on a wall, with a view of determining the spatial autocorrelation function of the dot patterns. In the course of doing this Glass noted circular images by superimposing the transparency of the dots upon the Xerox copy of the dots with a slight rotation (as in Figure 1B). Longuet-Higgins urged Glass to write up the results which appeared in Nature in 1969. The original paper described the effect and also sketched out a possible role for columnar organization of the visual cortex in the visual processing of these images (Glass, 1969). Over the next decade, Glass and colleagues wrote two additional papers describing further effects that could be generated by manipulations of dot images (Glass and Perez, 1973, Glass and Switkes, 1976). However, it was not until 1982, when David Marr called these images Glass patterns in his classic text in visual perception (Marr, 1982), that interest in the images began to build. To date, these patterns have been employed in dozens of visual psychophysics and electrophysiology experiments, proving particularly useful in the study of form perception.

[edit] References

• Badcock DR, Clifford CWG, Khuu SK (2005) Interactions between luminance and contrast signals in global form detection. Vision Research 45: 881-889.
• Barlow H, Berry DL (2010) Cross-and auto-correlation in early vision. Proceedings of the Royal Society B: Biological Sciences 278: 2069-2075.
• Dakin SC, Bex PJ (2002) Summation of concentric orientation structure: seeing the Glass or the window? Vision Research 42: 2013-2020.
• Dakin SC, Bex PJ (2003) Response to Wilson & Wilkinson: Evidence for global processing but no evidence for specialised detectors in the visual processing of Glass patterns. Vision Research 43: 565-566.
• Gallant JL, Braun J, Van Essen DC (1993) Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex. Science 259: 100-103.
• Gallant JL, Connor CE, Rakshit S, Lewis JW, Van Essen DC (1996) Neural responses to polar, hyperbolic and Cartesian gratings in area V4 of the macaque monkey. Journal of Neurophysiology 76: 2718-2739.
• Glass L (1969) Moiré effect from random dots. Nature 223: 578-580.
• Glass L, Perez R (1973) Perception of random dot interference patterns. Nature 246: 360-362.
• Glass L, Switkes E (1976) Pattern recognition in humans: Correlations which cannot be perceived. Perception 1: 67-72.
• Glass L (2002) Looking at dots. Mathematics Intelligencer 24, Number 4: 37-43.
• Gove A, Grossberg S, Mingolla E (1995) Brightness perception, illusory contours, and corticogenicuate feedback. Visual Neuroscience 12: 1027-1052.
• Grossberg S, Mingolla E (1985) Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations. Perception and Psychophysics 38: 141-171.
• Harth E, Beek B, Pertile G, Young F (1970) Signal stabilization and noise suppression in neural systems. Kybernetik 7: 112-122.
• Hubel, DH, Wiesel, TN. (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology (London) 160: 106-154.
• Khuu SK, Moreland A, Phu J (2011) The role of shape-from-shading information in the perception of local and global form in Glass patterns. Journal of Vision 11(7):20: 1-13
• Kobatake E, Tanaka K (1994) Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. Journal of Neurophysiology 71: 856-867.
• Krekelberg B, Dannenberg S, Hoffman KP, Bremmer F, Ross J (2003) Neural correlates of implied motion. Nature 424: 674-677.
• Krekelberg B, Vatakis A, Kourtzi Z (2005) Implied motion from form in the human visual cortex. Journal of Neurophysiology 94: 4373-4386.
• Mannion DJ, McDonald JS, Clifford CWG (2009) Discrimination of the local orientation structure of spiral Glass patterns early in human visual cortex. NeuroImage 46: 511-515.
• Mannion DJ, McDonald JS, Clifford CWG (2010) The influence of global form on local orientation anisotropies in human visual cortex. NeuroImage 52: 600-605.
• Mannion DJ, Clifford CWG (2011) Cortical and behavioral sensitivity to eccentric polar form. Journal of Vision 11(6):17: 1-9.
• Marr D (1982) Vision, Freeman, San Francisco.
• Ostwald D, Lam JM, Li S, Kourtzi Z (2008) Neural coding of global form in the human visual cortex. Journal of Neurophysiology 99: 2456-2469.
• Palomares M, Petter M, Vildavski V, Hou C, Norcia A (2010) Connecting the dots: How local structure affects global integration in infants. Journal of Cognitive Neuroscience 22: 1557-1569.
• Pei F, Pettet MW, Vildaviski VY, Norcia AM (2005) Event related potentials show configural specificity of global form processing. Neuroreport 16: 1427-1430.
• Priebe NJ, Ferster D (2008) Inhibition, spike threshold, and stimulus selectivity in primary visual cortex. Neuron 57: 482-497.
• Seu L, Ferrera VP (2001) Detection thresholds for spiral Glass patterns. Vision Research 41: 3785-3790.
• Smith MA, Bair W, Movshon JA (2002) Signals in macaque striate cortex neurons that support the perception of Glass patterns. Journal of Neuroscience 22: 8334-8345.
• Smith MA, Kohn A, Movshon JA (2007) Glass pattern responses in macaque V2 neurons. Journal of Vision 7: 1-15.
• Wilson HR, Wilkinson F, Asaad W (1997) Concentric orientation summation in human vision. Vision Research 37: 2325-2330.
• Wilson HR, Wilkinson F (1998) Detection of global structure in Glass patterns: Implications for form vision. Vision Research 38: 2933-2947.
• Wilson HR, Wilkinson F (2003) Further evidence for global orientation processing in circular Glass patterns. Vision Research 43: 563-564.
• Zucker SW (2002) Which computation runs in visual cortical columns? In: Problems in Systems Neuroscience, JL van Hemmen and TJ Sejnowski (eds.) Oxford University Press.

[edit] External Links

• Java applet demonstration of Glass patterns from Leon Glass' website, created by Alex Gorelick and Michael Rozenbojm at McGill University, July 2011.