Social Research: An International Quarterly

________

i begin with those phenomena that are instances of not seeing what is there to be seen, which occur for many different reasons. One familiar instance is camouflage. A camouflaged object is, in principle, perfectly visible—it is above the visual threshold by all physical criteria for seeing; being large enough, of high enough contrast, and close enough to be seen, among other criteria—but it is hiding in plain sight (see figure 1).

What renders the camouflaged object invisible are the rules by which our perceptual system operates. These rules of perceptual organization, all of which were subsumed under the overarching principle of prägnanz, translated as "simplicity" and now thought of as algorithms, were first articulated by the Gestalt psychologists. They were intended to explain why we see organized scenes containing objects and their backgrounds that appear to be consistent with what we know is actually there, despite the "mosaic of stimulation" that activates our retinas by the light reflected into our eyes. The Gestalt psychologists argued that the retinal stimulation had no intrinsic organization and could lead to an almost infinite number of percepts [End Page 194] (see Wertheimer [1923] 1938). They proposed that the brain automatically imposes organization upon the retinal stimulation according to rules that yield the simplest possible perception given the input. Without these imposed rules of grouping, the retinal input would be perceived only as an uninterpretable mosaic pattern of light, dark, and colored points.

The grouping rule of particular importance in creating the perceived invisibility of camouflaged objects is that of similarity, which dictates that similar units of retinal stimulation are automatically grouped into objects by the perceptual system. When you first look at figure 1, you are likely to see brownish blotches of color on a white

Click for larger view
View full resolution

Figure 1.

Pinto horses camouflaged against snowy background. Pintos, © Bev Doolittle, The Greenwich Workshop, Inc. Used with permission.

[End Page 195] background rather than two pinto horses standing in a snowy, rocky field. This is because the brownish blotches in the image tend to be grouped together, as do the white areas, thereby hiding the figures of the horses, which are otherwise not defined by any visible contours. It is interesting to note that once you manage to see the two horses, your visual system will have created illusory contours that seem to outline the figures but actually do not exist in the image itself, a clear case of seeing something that is not there. Ever after, when you see this particular image, you will see the horses, not the blotches.

Grouping by similarity is a key factor underlying the camouflage that occurs with regularity in the natural world. When the coloring and patterning of creatures who are hunted are the same as or similar to their surroundings, they are extremely difficult for predators to see because the prey animals' patterns are grouped with their backgrounds and thus they become invisible (see figures 2 and 3, as

Click for larger view
View full resolution

Figure 2.

Camouflaged pygmy seahorse with coral. Photograph by Steve Childs. Used with permission. https://www.flickr.com/photos/steve_childs/.

[End Page 196] well as https://www.wherecoolthingshappen.com/wp-content/uploads/2014/04/gecko.jpg). However, as soon as an animal moves, the gig is up. It is immediately visible, standing out from its background; that is because another principle of grouping is now operating, the rule of common fate, which dictates that things that move together are grouped together and so are suddenly seen for what they are.

Camouflage is, of course, a device widely used in warfare, in the design of both clothing and weaponry. It is also used for more playful purposes. Older readers may remember with fondness the recurring "Finding Nina" cartoons by Al Hirschfeld that appeared on

Click for larger view
View full resolution

Figure 3.

Brimstone butterfly camouflaged as a leaf. Photograph by Steve Childs. Used with permission. https://www.flickr.com/photos/steve_childs/.

[End Page 197] the front page of the Theater section of the Sunday New York Times, where we eagerly searched for the camouflaged name of his daughter, Nina (see figure 4). Similarly, those who grew up somewhat later may remember the "Where's Waldo" books of their childhood. These are also cases of camouflage, of something hiding in plain sight, and of our not seeing what is there to be seen. In the "Nina" cartoons, some of the lines composing the name are perceived as part of other lines in the cartoon, and thus by virtue of the Gestalt principle of good continuation become invisible. This rule mandates that lines or edges that are smooth continuations, i.e., that flow in the same direction, are grouped as a single unit.

Click for larger view
View full resolution

Figure 4.

"Audrey Hepburn, Breakfast at Tiffany's" with hidden "Nina." © Al Hirschfeld. Reproduced by arrangement with Hirschfeld's exclusive representative, the Margo Feiden Galleries Ltd., New York. www.alhirschfeld.com.

[End Page 198]

Another phenomenon in which something we are looking at can be made invisible occurs through a process known as masking. If an image of a scene is shown to us briefly and then immediately followed by another pattern that covers the entire area where the scene had been, or, more simply, a colored disc is flashed and followed immediately by a differently colored surrounding ring, neither the scene nor the disc will be visible, although both would be if they were flashed for the same brief moment without the "mask" or if the "mask" was delayed. This is generally believed to be the result of the usurping or co-opting of visual processing operations by the second pattern, which shuts down the processing of the first stimulus, causing it to be effectively invisible. Visual masking functions to protect the visual system from information overload, which can be a very serious problem given the overwhelming amount of retinal stimulation our visual systems must handle at every instant.

The masking effect is not due to the attributes of the particular scene or colored disc, but rather to the ways in which our visual system operates. It is interesting and even instructive to note, however, that if your own name is flashed and followed by a masking stimulus, the mask is far less likely to be effective and your name is far more likely to be visible to you. Another name is as easy to mask as most other things, which illustrates a significant feature of our visual systems: the importance of meaning to our perceptual systems. Not surprisingly, highly meaningful things capture our attention and we see what we are attending to. Your own name will pop out in a sea of other names or be heard by you if spoken in a crowded, noisy room, whereas someone else's name may not. In fact, your own name is one of the few things that will resist inattentional blindness—a phenomenon we discuss below.

The concept of crowding also belies our belief that we have a clear, rich, and detailed view of our visual environments. Crowding occurs when we are paying attention to something but not explicitly fixating on it because it is off to the side and surrounded by other things—not an unusual occurrence in our normally cluttered visual [End Page 199] environments—and is experienced, for example, when we notice but do not focus on objects along the side of the road while we are driving. It also occurs when we are reading, so that words elsewhere on this page than where we are focused appear blurred. This phenomenon can be illustrated very simply. Figure 5 contains an array in which the word "cat" is central and the letter X is off to the left, surrounded by a series of the letter O. If you look at the word "cat" and do not turn your eyes toward the X but nevertheless pay attention to it, you will not be able to identify it as an X and most likely will perceive it as bits of fragmented blotches. In contrast, if it were alone and in the same position, you would have no difficulty identifying it.

Click for larger view
View full resolution

Figure 5.

Array demonstrating the phenomenon of crowding.

I will mention one more case, which can only be demonstrated in a vision laboratory but is nevertheless interesting and an important aspect of our perceptual experience. It is a phenomenon known as binocular rivalry, which occurs when the two eyes receive quite different images. It should be noted that our eyes always receive slightly different images when we are viewing a three-dimensional scene because the two eyes are about 2.5 centimeters apart in the head and thus view the scene before us from slightly different vantage points. In fact, this difference in the images to the left and right eyes is the basis of stereoscopic vision, which is one of the sources of our ability to see depth even though the images on our retinas are two-dimensional. The visual system interprets the small width differences between the images as signifying depth, so instead of seeing double images, we see objects at different depths. Binocular rivalry occurs when there is an extreme difference between the images to the two eyes, not just small width differences. For example, if one eye sees the [End Page 200] image of a car while the other sees the image of a horse, the images will be rivalrous and our perception will alternate between them. If one eye receives an image of a face while the other receives an image of a car, we are highly likely to see only the face for very long periods of time and to see the car briefly because faces are highly meaningful objects for us. Their dominance in binocular rivalry once again illustrates the power of meaning to affect what we see. In all cases of binocular rivalry, we do not see at any given time all that is there on our retinas to be seen.

________

it turns out that objects that are perfectly visible—not masked, not camouflaged, and in our direct view, seen with both eyes—are not perceived if we are not paying attention to them, a phenomenon called inattentional blindness, which Irvin Rock and I named in 1998. There are many, many occurrences that qualify as inattentional blindness, including the way magicians have used the phenomenon since well before it was named and studied (Mack and Rock 1998).

The now most-famous demonstration of this phenomenon is familiarly referred to as the "invisible gorilla" video (Simon and Charbris 1999; see https://www.youtube.com/watch?v=vJG698U2Mvo). Observers who watch the video for the first time are asked to keep track of the number of times players wearing white or black uniforms pass basketballs among themselves. It is a relatively demanding task. During the video, a person dressed in a gorilla suit walks through the basketball court, stops at its center, pounds her chest, and then walks off the court and out of the video. Approximately half the viewers who do not know anything about this video prior to doing the task fail to see the gorilla even though eye movement records show that they may have been looking directly at it. Moreover, they evince great surprise when they watch the video again without simultaneously doing the counting task and now see what they failed to see the first time because they were not attending to it. Their surprise reflects our shared erroneous belief: that we see what is there to be seen. [End Page 201]

The phenomenon of inattentional blindness (Mack and Rock 1998) has been heavily explored in many different laboratories, but everyone experiences it in their daily lives. It occurs innocuously when you have an intense conversation with someone and fail to be aware of people walking by. It occurs much more dangerously when you text while driving or while walking across the street—or even when you talk on a cell phone while trying to do these things—since doing so makes it highly likely that you will not see the approaching gorilla or, as the case may be, an oncoming car or a child running into the street to retrieve a ball.

While there are very few objects or events that seem to be able to counteract inattentional blindness, we know that highly meaningful ones usually will. In particular, as we saw above in the discussion of crowding, we know that if your name is shown or said while your attention is fully focused elsewhere, you are highly likely to see or hear it, a phenomenon known as "the cocktail party effect" (Mack and Rock 1998).

Another kind of inattentional blindness—more troubling than most, since it is far more difficult to overcome—is symptomatic of the clinical disorder known as visual or unilateral neglect. This is a disorder that often follows a stroke, occurring most often when the right parietal area of the cortex—a part of the brain intimately involved with paying attention—is damaged. Patients suffering visual neglect fail to perceive things that are on their left despite the fact that there has been no damage at all to the visual system, which continues to operate normally. (Bear in mind that objects on our left are processed by the right side of the visual cortex, while objects on our right are processed by the left side.) These patients fail to shave the left side of their faces or eat the food on the left side of their plates. If asked to copy an outline drawing of a clock face, they only draw its right half. There is general agreement that visual neglect arises because the underlying cortical lesion leaves patients unable to disengage their attention from objects on the right, so they end up being inattentionally blind to things on the left (see, for example, Palmer 1999, 563–65). [End Page 202]

Change blindness, another heavily researched perceptual phenomenon, is related but not identical to inattentional blindness, and is powerful evidence of how little we consciously see of what we look at (Rensink, O'Regan, and Clark 1997). As its name suggests, we are often blind to much of what is in front of us because we can only attend to a very few objects at any one time. If you are shown a photograph of a scene, for example a beach, and then you are quickly shown the same scene again except for one object, for example a sand pail removed from the scene or a mast of a sailboat added, you are likely not to see the change even if you are searching for one, and it may take many views before you do. In fact, you will only pick up the change if you happened to attend to the sand pail in the first scene, encode it into memory, look for it in the second scene, and compare what you remember of the first scene with the second one. (Examples of change blindness can be viewed at https://www.youtube.com/watch?v=qN7s9E6M4RQ and https://www.gocognitive.net/demo/change-blindness.)

There are two more cases of perceptual invisibility related to inattentional blindness that merit mention. One is called choice blindness, which was originally documented by Lars Hall and Peter Johansson (Johansson et al. 2005) in an experiment where participants were shown pairs of pictures of reasonably similar faces of, for example, young women, and asked to choose the one they preferred. Subsequently they were shown the chosen picture and asked why they preferred it. After a few trials, the experimenter deftly swapped a chosen picture for its unchosen mate. The majority of the participants were completely unaware of the swap and blithely went on to explain why they preferred the picture they were now looking at but had rejected when asked to choose—hence the name choice blindness. This phenomenon is especially interesting because here the participants are explicitly asked to look at both photographs and chose the face they prefer. Nevertheless, they are blind to the swap (video example: https://channel.nationalgeographic.com/brain-games/episodes/you-decide/). [End Page 203]

Misdirection is a magician's trick of the trade. The reason many magic tricks work is because the magician directs your attention to something other than the sleight of hand. For example, if a magician is doing something tricky with her right hand, she may make some large, attention-capturing gesture with her left hand, causing you to pay attention to it so you fail to see what she is doing with her other hand. Misdirection is a case of inattentional blindness with a different name. It is purposefully used by the skilled magician who, by means of it, keeps you from seeing what she does not want you to see by getting you to attend to something else.

All these phenomena, which have their roots in the fact that we are literally blind to things that we are not paying attention to, tell us something important about our perceptual system, which has the daunting task of processing the huge amount of stimulation it is constantly receiving. They tell us that one way the system deals with this constant bombardment is by filtering out so much and fully processing only a fraction of the input it receives, and what is actually processed sufficiently to reach awareness is, for the most part, what is important to us, preventing the rest from cluttering our consciousness. It seems to make some intuitive sense that we only consciously see what we are attending to and are functionally blind to the rest. Paradoxically, but nonetheless true, invisibility seems to be almost as much a feature of our perceptual experience as visibility.

________

i turn now to cases that might be considered correlative to those of functional perceptual blindness. These are cases in which we see something other than what is there. There are far too many cases to describe them all, so I have chosen only a very few to represent the many and have deliberately omitted any discussion of perceptual illusions that are more than likely familiar to the reader, such as the Müller-Lyer illusion (see figure 6).

Perhaps one of the most compelling sorts of seeing what is not there to be seen is the phenomenon of subjective or illusory contours. Remarkably, when we perceive illusory contours, we also perceive [End Page 204]

Click for larger view
View full resolution

Figure 6.

Müller-Lyer illusion.

Click for larger view
View full resolution

Figure 7.

Kanizsa triangle showing subjective contours.

differences in brightness and depth that are not present. Figure 7, known as the Kanizsa illusory triangle (Kanizsa 1979), named for the person who first created it, is one of the most famous examples of this phenomenon. When looking at this figure we see a white triangle in front of a black-outlined triangle. The white triangle appears whiter than the white background and seems to be covering three black circular discs. In fact, there are no actual contours that define this white triangle. There are no circular black discs but rather Pac-Man-like shapes, and the whiteness of the triangle is, of course, identical to the background. Neither the white triangle nor the brightness difference is present in the figure itself; both are created by the perceptual system.

There are many other instances where we see contours that do not actually exist, as in figure 8, which illustrates a phenomenon known as scission. Most observers describe this figure as either a bow-like shape in front of a horizontal narrow triangle or as a triangle in front of the bow-like figure. What is important for our purposes is that in order to perceive either version, our perceptual system must create contours where none exist. [End Page 205]

Part of the explanation of illusory contours is again found in how the visual system operates by imposing organization on the input it receives. In the cases of the Kanizsa triangle and scission, it is the Gestalt grouping rule of good continuation that, as pointed out earlier, mandates that lines that are smooth continuations of each other will be seen as belonging together and therefore as one line. Thus, in figure 8 we see the overlapping bow-like figure and narrow horizontal triangle, and in the Kanizsa triangle we see the black discs occluded by a black outline triangle. In both cases we fail to see what is really there and we also see something that is not there.

Synesthesia is the term for a phenomenon that occurs in a subset of the population, in which stimulation of one of the senses elicits a sensation in another. A synesthete might see red every time she looks at the letter B or see green every time she hears or sees the number 2. These cases of seeing, hearing, or even tasting something that is not there have been explained as being the result of idiosyncratic neural connections that exist in the brains of synesthetes.

Click for larger view
View full resolution

Figure 8.

Scission.

[End Page 206]

Perhaps the most interesting case of seeing what is not there occurs when our expectations lead us to see something when there is nothing to be seen. This may be illustrated by recent research in my own lab (Mack et al. 2016), in which we consistently presented visual arrays like the one shown in figure 9 and asked participants to report either whether there was an odd circle among the four or what letters appeared in the letter matrix. They did not know until immediately after the display disappeared which of these tasks they would be asked to do. In 100 trials, they were asked to report the letters on only 10 trials, which of course caused them to pay more attention to the circles, about which they were asked on the other 90 trials. On trial 101 we introduced a change. This time only the circles were present, not the letters, but we asked them to report the letters. What we found was that more than half of the participants actually made up letters and expressed genuine surprise when they were told afterward that there had been no letters at all. (We can compare their surprise with that of the viewers who failed to see the gorilla amid the thrown balls in the video described earlier.) It seems clear that the expectation of an array containing both circles and letters, which had been built up over many trials, led the majority of the participants to imagine or even hallucinate letters where there were none.

The power of our expectations to lead us to see what is not there has been known for a long time. One of the earliest laboratory demonstrations of this was reported by Jerome Bruner and his colleague Leo Postman in an article published in 1949. They described experiments in which observers were shown playing cards, each card presented only very briefly, and asked to identify them. Some of the cards were trick cards in which the usual color was reversed—for example, a red six of spades or a black three of hearts. Bruner and Postman found that "faced with a red six of spades, for example, a subject may report with considerable assurance, 'the six of spades' or the 'three of hearts.' … In both instances the perceptual resultant conforms with past expectations about the 'normal' nature of playing cards" (1949, 213). (A demonstration of this experiment can be seen at https://www.youtube.com/watch?v=yFYBY_YUH5I.) [End Page 207]

Click for larger view
View full resolution

Figure 9.

Array for experiment demonstrating the importance of expectation in visual perception.

A different set of recent experiments in our lab (Mack et al. 2017) provides us with another example of the power of expectation to determine what we see, leading us again to see what is not there. In these experiments, observers were shown a series of scenes that either did or did not contain something unusual or weird. In one of these experiments we simply asked participants to tell us whether or not the scene was weird. We found that even when the observers were shown scenes that contained something very weird, for example a girl licking a light bulb rather than an ice cream cone, they virtually never perceived the incongruity. Instead, they normalized their perceptions, showing clearly that their expectations led them to see what was not there (see https://mackperceptionlab.files.wordpress.com/2016/10/stimuli-exp-1-2-4.pdf). Magicians also take advantage of the phenomenon of seeing what is expected rather than what is actually there (see, for example, https://www.youtube.com/watch?v=mc0gQcP20pg). [End Page 208]

________

each of the many examples described here tells us that despite our deep-seated belief that what we see is what is there to be seen, there are an enormous number of instances in which things that are clearly visible are nevertheless invisible to us. Conversely, we see more than what is there when our visual systems create something from nothing or transform what is actually there into something that is not there.

Coming full circle to this issue of Social Research on "Invisibility," the inattentional blindness that is researched in cognitive-science labs can account for much of the social invisibility that plagues our society, from the homeless man lying on the street to the African American woman standing nearby, a phenomenon made vivid in Ralph Ellison's Invisible Man. Surely, the Black Lives Matter movement is an attempt to counteract social invisibility by forcing us to pay attention to—and therefore see—the brutality visited on young black men by police. Our shameful inattentional blindness leads to our failing to see those we deem less than equal to us and not worthy of our attention.

The contributors to this special issue have explored invisibility in some of its scientific, philosophical, economic, mythological, and religious manifestations. It seems of crucial importance not to leave out sociopolitical invisibility, which was in fact one of the motivating reasons for organizing the issue. It is essential to recognize that actually seeing requires actually paying attention, and that people and problems that have long been invisible to us suddenly become potently and painfully visible when we give them our attention.