In response to the overwhelming amount of stimuli taken in by the retina, the early stages of visual processing are centered around grouping, clustering, segmentation, and finding patterns. This initial process of turning an array of individual signals into packets of information to be further processed at later stages of vision is commonly referred to as preattentive processing, and refers to only the information that can be gleaned from a single fixation of the eye. Since eye movements take approximately 200-250 msec.s to execute, anything processed under that threshold is considered preattentive. “Examples of preattentive features include colour, orientation, intensity, size, shape, curvature, and line length” (Healey, Booth & Enns, 1993, p. 1).
In order to create designs that can be understood quickly and effortlessly, it is important to understand how the perceptual system is neurologically tuned to create patterns, in order to coordinate rather than compete with these processes. Preattentive principles can be leveraged to improve the efficiency and accuracy of common visual tasks including boundary detection and grouping, target detection, and estimation (Healey, 1999, p. 209).
To illustrate some of these grouping principles, evaluation of the preattentive attributes of a sound mixing board will be used as an example. A sound board is an example of a product that looks incredibly complex to the uninitiated; however, a basic understanding of its function can render it significantly more simple to navigate. While preattentive groupings are of course no replacement for domain knowledge, effective employment of proximity, similarity, and other principles may increase its learnability and reduce cognitive load even for experienced users.
The common principles that seem to govern grouping of visual elements were first codified by Wertheimer and other Gestaltists in the early 20th century. The patterns that they observed included grouping by similarity of color, shape, size, or orientation, proximity, continuity, closure, symmetry, and common fate (moving the in the same direction). More recently, Palmer and colleagues have identified additional powerful grouping principles of common region (1992) and element connectedness (Palmer & Rock, 1994), which holds that elements within an enclosure or linked by lines establishes a preferred grouping, even when in conflict with proximity or similarity cues. Although these principles have been acknowledged and studied since their inception, “the neural correlates that govern them have yet to be entirely clarified” (Sasaki, 2007, p. 132). Increasingly sophisticated understanding of the neuroscience of vision and “recent neuroimaging techniques…have revealed brain activity related to Gestalt laws and perceptual organization” (Sasaki, 2007, p. 132).
In the process of identifying objects from an array of visual stimuli, proximity is a natural and essential, yet not sufficient, first step in object discrimination (Loffler, 2008, p. 2106). In lower visual areas, neurons are organized retinotopically, preserving this important location information (Stone, 2010, p. 103). Investigation of the precise role of attention in early visual processing has led Trick & Enns (1997) to differentiate ‘clustering’ of nearby elements, a pre-attentive process, from complete shape-formation, which may rely on top-down feedback, which their research indicated are separable operations.
The identification of enclosure and boundaries is another key step in this process.Edge detection, which results from combinations of zero-crossings from excitation to inhibition in retinal ganglion cells (Marr & Hildreth, 1980), is essential to establishing both shapes and boundaries. Palmer & Rock characterize edge detection as “an early step in perceptual organization, one that begins the process of partitioning the image into connected regions of uniform sensory properties,” (1994, p. 33). Elder & Zucker suggest that textures can create regions not due specifically to areas of textural differences but rather the distinct boundary created where two textures meet, due to the fact that “the ‘filling-in’ of region information follows the neural grouping of the bounding contour by between 50 and 150 msec.” making texture segmentation a subset of bounding contours (1998, p. 151).
Similarity is another fundamental grouping principle, although recent research indicates that it may involve slightly higher areas of the visual cortex (Sasaki, 2007, p. 132). Grouping by color, size, luminance, and orientation are all supported by specifically tuned cells along the visual pathway. Cells that transmit visual information from the retina along the optic tract to the lateral geniculate nuclei and into the visual cortex are “each tuned to a specific pattern,” and “a neuron can detect the presence of its preferred pattern in the midst of a complex retinal image” (Stone, 2012, p. 33-34). Livingstone & Hubel describe the differences between the magnocellular and parvocellular pathways in the lateral geniculate nuclei, which “differ physiologically in four major ways-color, acuity, speed, and contrast sensitivity” (1988, p. 741). The magno pathway is sensitive to achromatic contrast and contributes to motion detection and analysis, while the parvo pathway is involved in encoding static form and color (Stone, 2012, p. 87-88). This differentiation continues into the visual cortex.
In addition to these main principles, a number of others have also been observed. For example, Wolfe & Friedman-Hill (1992) found that a symmetrical background texture aids in target recognition while symmetry between distractors and the target hinders recognition. Some aspects of depth can be processed preattentively, including stereopsis and elements of occlusion and positioning (Livingstone & Hubel, 1988, p. 746). Other recognized factors include motion, curvature, and shadow among others (see Healey, Booth, & Enns, 1996, p. 112).
Luna & Montoro (2011) differentiate between intrinsic grouping principles, such as shape, position, or color, and extrinsic principles, such as common region or connectedness, as by lines or abutment. They found that when intrinsic and extrinsic principles are used redundantly, the grouping effect is reinforced, but when they are in conflict, grouping becomes unclear or fluctuates (p. 1475). Similarly, preattentive just-noticeable-differences (JND), including color and orientation, are coarser than the JND of foveal, attentive vision (Wolfe, 1994, p. 207). Gori & Spillman (2009) also found that “the threshold for perceiving stimuli as irregularly spaced or dissimilar in size or brightness is much smaller than the threshold for grouping” (p. 1194). Therefore, in design terms, just because a subtle difference can be perceived when examining two elements does not necessarily mean that it will be a useful differentiation during preattentive processing.
Theories of Visual Search
A number of theories of visual processing have been proposed to provide a working model of vision and visual search to bridge the gap between known neurological processes and observable phenomena. Perhaps the most influential is the Feature Integration Theory (FIT), first proposed by Treisman & Galade in 1980. The basis of FIT is that there are certain elements of a visual scene that are processed pre-attentively. These preattentive elements were organized into ‘dimensions’, like color or orientation, which corresponded to proposed separate perceptual subsystems, and individual ‘features’ or particular values in a dimension (red, vertical), a terminology which has endured (1980, p. 89). These features will tend to ‘pop out’ such that they may be searched in parallel. Parallel searches are characterized by a very fast response time which does not increase as the number of distractors increases. When the target of a search is characterized by more than one feature, referred to as a conjunction however the search is likely to be processed serially and requires directed attention in which each item is evaluated for a match one after another (Treisman & Galade, 1980; Quinlan, 2003). Although FIT and its later incarnations have received criticism on a number of levels, it has had an enduring impact.
Duncan & Humphreys’ (1989) similarity theory of visual search built on FIT but instead of upholding a distinction between parallel and serial search or feature vs. conjunction targets, posited a continuum of search difficulty as function of the similarity between targets and non-targets (greater similarity increases RT) and well as similarity among non-targets (greater similarity decreases RT). This too has been complicated by more recent research, but it does provide a solid common sense guidance for understanding what features are most likely to stand out and how to manipulate their salience. A slightly different conception is provided by Huang & Paschler’s Boolean map theory (2007). This model stipulates that “the visual system can select one feature at one time (feature-by- feature selection), and the process yields a map of the spatial distribution of the feature (obligatory encoding of location) with the option of combining this map with a preexisting map (Boolean operations)” (p. 603). Other models include Wolfe’s theory of guided search (1994) and Itti, Kock & Neibur’s model of saliency-based visual attention (1998).
Conceptions of preattentive visual processing have traditionally been associated exclusively with feedforward, or bottom-up, information processing. Recent research indicates that this is most likely a somewhat artificial distinction (Khorsand, Moore & Soltani, 2015), yet it remains helpful for basic understanding and design applications. For example, in regard to accurate estimation of small numerical values without serial counting, this process (‘subitizing’) has been assumed to be preattentive due to its conformance to parallel search patterns in testing. Recent research suggests that this process may indeed depend on attentional resources (Hesse, Schmitt, Klingenhoefer & Bremmer, 2017), however for design purposes, all that is necessary to know is that this process happens very quickly, accurately, and effortlessly.
The example considered here is a Behringer Europower PMP4000 16-channel mixer, however the basic layout and many of the design components of this model are common to a variety of similar products. The organization and design of the numerous components provide a range of preattentive grouping cues, drawing on shape, color, proximity, orientation, common region, and others, some helpful, and others less so. At the highest level, similarity in shape and size help to group like with like according to function and affordance (fig. 1), from XLR (a) and ¼” jack inputs (b) to knobs (c) and sliders (d).
In some places, however, such groupings compete with one another. The spacing of the knobs in the center console (e) taken alone would result in a vertical grouping due to a greater distance between the columns compared to the rows. The color-coding of individual knobs, as well as the variation in the intensity of the background color, however, suggest horizontal groupings of differing sizes. On this level too, in some places, these two factors coordinate, as in the section of blue knobs, while in others they compete, as where the pink vs. orange knobs may suggest segregation but the common area established by the dark gray background supports unity. These competing grouping indicators may reflect the complex relationships among these elements, but in general the conflict will weaken preattentive processing cues, reducing efficiency in comprehension.
In fact, the overall horizontal orientation of this layout, while not without logic, significantly impairs understanding of the basic system structure in terms of physical architecture as well as user control. The audio channels are actually organized vertically, from an individual audio input connected at the top of the board, the signal from which then travels down through various modifications, before reaching the final volume fader at the bottom the board. A vertical orientation is also the preferred grouping in the user manual, likely for this reason (fig. 2). While there is some suggestion of this underlying structure in the boundaries created by the fine white vertical lines in addition to proximity, the visual weight is not sufficient to compete with the strongly horizontal color and shape groupings. This is a basic understanding that is fundamental to console operation, and yet it is mostly obscured the preattentive elements of the design.
Color is used effectively (fig. 3) to signal the relationship between the individual and master controls in pink (Monitor 1 & 2; f & l) and orange (FX; g & k). The same is true for the pink monitor controls between the main bank and the corresponding controls in auxiliary stereo controls to the right (i). An undesirable correspondence exists however between the bottom row of black knobs (L-R Balance/Pan) and the black knobs to the right (Level). Due to their identical shape and color, as well as similar positioning and small size, they are likely to be confused.
Enclosure is used throughout to indicate related sets of controls. An example of this can be seen in the inputs at the top right (fig. 4). Here the vertical orientation suggested by proximity is reinforced by line enclosure. Although the lines used to create common regions do not connect, the principle of continuation suggests that these will be perceived as complete bounding boxes. This is beneficial to the left of the image area (m) as it provides separation for the individual channels. On the right (n), conversely, the columns are pairs of inputs with each row acting as a separate unit (electronically), again introducing conflict between the spacing and other cues. Instead of equal spacing between the three right-most columns, a smaller gap between the first two and a larger gap before the third would clarify their functional relationship preattentively.
The relatively light weight of these lines means that they are best used redundantly rather than to drive preattentive grouping. Palmer & Rock’s principle of element connectedness comes into play to link the “voice canceller” button to the input to which it can be applied (p). The line that connects the two elements provides a clear indication of their relationship even in the absence of similarity or any particular proximity clue when examined closely, but it is possible that it would not be a strong enough signal to act preattentively.
In the center of this section (fig. 4), alignment creates another false impression in areas 9/10 and 11/12 (o). In reality, the top white & red inputs and the bottom two silver inputs in each “column” serve the same purpose but simply provide alternative sockets for two different types of connectors. While this equivalence is disguised by the offset positioning of these pairs of inputs to the left, on the right, the top four white & red inputs are indeed a separate system from the inputs below. In styling these two sections the same way in the pursuit of symmetry (another preattentively processed feature but misused here) and perhaps to make room for the voice canceller button, the appearance of similarity belies the very different relationships in these two sets of connectors. The only indication of this difference is the horizontal separator to the right, but again, its relative strength, especially at a distance or in the low-light conditions often associated with the context of use, it is unlikely to predominate.
The exact nature of the visual pathways in the brain the best way to represent them in a userful, cohesive model of visual processing are very much ongoing questions. While modern vision science has increasingly blurred the line between pre- and post-attentive processes in early perception, the basic insights into how visual elements are grouped and organized at this nearly instantaneous and subconscious level commonly termed preattentive provide valuable guidance for interface design and information visualization. Understanding the biological evidence for why certain design principles and best practices, as well as quantitative guidance from research in terms of perceptible differences, can help designers make informed and effective choices.
Deller, M., Ebert, A., Bender, M., Agne, S., & Barthel, H. (2007). Preattentive visualization of information relevance. Proceedings of the International Workshop on Human-Centered Multimedia, 47-56. doi:10.1145/1290128.1290137
Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review, 96(3), 433-458. doi:10.1037/0033-295X.96.3.433
Elder, J. H., & Zucker, S. W. (1998). Evidence for boundary-specific grouping. Vision research, 38(1), 143-152.
Gori, S., & Spillmann, L. (2010). Detection vs. grouping thresholds for elements differing in spacing, size and luminance: An alternative approach towards the psychophysics of Gestalten. Vision Research, 50(12), 1194-1202.
Healey, C. G. (1999). Perceptual colors and textures for scientific visualization. University of California, Berkeley. Retrieved from http://www.ics.uci.edu/~majumder/vispercep/percepcolforviz.pdf
Healey, C. G., Booth, K. S. & Enns, J. T. (1993) Harnessing preattentive processes for multivariate data visualization. Proceedings Graphics Interface ’93, 1-10.
Healey, C. G., Booth, K. S., & Enns, J. T. (1996). High-speed visual estimation using preattentive processing. ACM Transactions on Computer-Human Interaction (TOCHI), 3(2), 107-135.
Hesse, P. N., Schmitt, C., Klingenhoefer, S., & Bremmer, F. (2017). Preattentive processing of numerical visual information. Frontiers in Human Neuroscience, 11, 1-14. http://doi.org/10.3389/fnhum.2017.00070
Houtkamp, R., & Roelfsema, P. R. (2010). Parallel and serial grouping of image elements in visual perception. Journal Of Experimental Psychology: Human Perception And Performance, 36(6), 1443-1459. doi:10.1037/a0020248
Huang, L. & Pashler, H. (2007). A boolean map theory of visual attention. Psychological Review, 114(3), 599-631.
Itti, L., Koch, C. & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254-1259.
Kafalignonul, H., Breitmeyer, B. G. & Ogmen, H. (2015). Feedforward and feedback processes in vision. Frontiers in Psychology, 6, 1-3.
Khorsand, P., Moore, T., & Soltani, A. (2015). Combined contributions of feedforward and feedback inputs to bottom-up attention. Frontiers in Psychology, 6, 1-11.
Livingstone, M. S. & Hubel, D. H. (1988). Segregation of Form, Color, Movement, and Depth: Anatomy, Physiology, and Perception. Science, 240, 740-749.
Loffler, G. (2008). Perception of contours and shapes: Low and intermediate stage mechanisms. Vision Research 48(20), 2106–2127.
Luna, D. & Montoro, P.R. (2011). Interactions between intrinsic principles of similarity and proximity and extrinsic principle of common region in visual perception. Perception, 40(12), 1467-1477.
Marr, D., & Hildreth, E. (1980). Theory of edge detection. Proceedings of the Royal Society of London B: Biological Sciences, 207(1167), 187-217.
Palmer, S. E. Common region: A new principle of perceptual grouping. Cognitive Psychology, 24(3), 436-447.
Palmer, S. & Rock, I. (1994). Rethinking perceptual organization: The role of uniform connectedness. Psychonomic Bulletin & Review, 1(1), 29-55.
Quinlan, P. T. (2003). Visual feature integration theory: Past, present, and future. Psychological Bulletin, 129(5), 643-673. doi:10.1037/0033-2909.129.5.643
Sasaki, Y. (2007). Processing local signals into global patterns.Current Opinion in Neurobiology, 17, 132-139.
Stone, J. V. (2012). Vision and the brain: How we perceive the world. Cambridge, MA: MIT Press.
Treisman, A. & Galade, G. (1980). A Feature-integration theory of attention. Cognitive Psychology, 12, 97-136.
Trick, L. M. & Enns, J. T. (1997). Clusters precede shapes in perceptual organization. Psychological Science, 8(2), 124-129.
Wolfe, J. M. (1994). Guided Search 2.0 A revised model of visual search. Psychonomic Bulletin & Review, 1(2), 202-238.
Wolfe, J. M. & Friedman-Hill, S. R. (1992). On the role of symmetry in visual search. Psychological Science, 3(3), 194-198.