Pattern 002 Perception

Environments are information before they are anything else. Even with the conventional conception of the senses as partitions of distinct sensory inputs, the world is not identical with what human bodies are, on average, built to register. We see a narrow band of the available wavelengths, hear a narrow band of the available frequencies, register a small fraction of the volatile compounds moving through a given space as smells, and detect exactly one stretch of non-visible radiation, the infrared, which arrives as warmth on our skin. The rest of the spectrum passes through us unannounced, and the parts of it that do us real damage manage to do so without ever presenting themselves to perception at all, which is why a sunburn shows up hours after the beach day that caused it.

We know the rest is there because we have built instruments that convert these phenomena into ones we can perceive. In 1800 William Herschel spread sunlight through a prism and walked a thermometer across the resulting band, and found the temperature still climbing past the red, in the place where there was nothing left to see.

The narrowness, it is worth saying, is not arbitrary. The bands we see sit more or less where the sun's energy is concentrated after it has been filtered by the atmosphere. What we call the world is therefore already a compressed representation, selected for over eons, and fitted rather precisely to one planet (Earth) and one animal (us).

Even within the narrow bands we do have access to, the compression continues, and it is a good deal more aggressive than it feels from the inside. In 1998, psychologists Daniel Simons and Daniel Levin conducted study in which an experimenter stopped a stranger on the campus of Cornell University to ask for directions; during the exchange, two workmen would pretend to carry a door between them, and swap the initial experimenter with a new one, completing the sentence where the first left off. Usually the stranger carries on giving directions, without noticing that the person they are talking to has been swapped for somebody else.

The leading account of this selectivity is that our brains maintain a model of what is likely to be out there and uses the incoming signal mainly to correct that model, so that what arrives in awareness is closer to a well-disciplined inference than to a registered record. The fun word that gets thrown around for this is hallucination, though it is generally more disciplined and data-derived than pure fantasy. The model is learned through continuous interaction with the world and through the mere fact of being in the world at all, and it is answerable at every moment to what the world does next, which is the only reason it works as well as it does. Put simply, we act on a structured representation of the world not on raw sensory inputs.

The low-hanging fruit for this is the example of the "checker shadow illusion." In 1995 Edward Adelson created a picture of a checkerboard with a green cylinder standing upright on its surface, casting a shadow across one corner of the board. A dark square outside the shadow and a light square inside it are printed in exactly the same shade of gray, but no matter how hard you try the square outside of the shadow will always appear darker than the square inside of it. You can prove it to yourself by covering everything except the two squares, at which point they collapse into the same color. As soon as your brain is compelled to understand the image as a whole, it actually enforces a coherence that does not actually present.

That we are interacting with a structured representation of the world is not only an insight of contemporary neuroscience but one already well known in the eighteenth century. Kant drew the distinction between phenomena, meaning things as they appear once our senses and mental constructs have had their way with them, and noumena, the inaccessible realm of the "Ding an sich," or the "thing-in-itself."

Whereas earlier philosophical traditions from Plato to Descartes theorized innate forms or content to the structuring of sensory information, Kant theorized that the mechanism for structuring sensory information into formal properties was innate. Plato's forms were already known to the soul and had only to be remembered, and Descartes' ideas came preloaded with the equipment. Kant's proposal is stranger, since what he installs has no content whatsoever and stays empty until the world gives it something to work on. Space and time are his clearest case: not properties of the world so much as the structure of experience as such. He is describing what has to hold for experience to be possible at all rather than making a claim about how infants are wired, though it is the second reading that has proved the more productive one since.

To illustrate the point it is helpful to think about the faculty of language. Nobody is born speaking German, and no particular grammar comes preinstalled. What appears to be innate is a capacity with a shape, and the shape shows itself in the fact that the several thousand languages currently in use are not spread evenly across the space of possible communication systems but cluster in a comparatively narrow part of it. Chomsky's name for the shape is universal grammar: the structure of existing languages indexes something about the physical structure of the capacities that produce them, in roughly the way a footprint in the sand indexes a foot.

Looked at in a certain way, every individual mind is in some sense a mechanism within a larger inter-subjective structure that is language as such. Ferdinand de Saussure gave the distinction its usable form, separating the utterances people actually make from the system those utterances draw on, a system that no single speaker holds in their mind in its full form and that none of them can revise on their own authority. While language is indexed in the world as neurological wetware, written texts, sound pressure fluctuations, visual symbols, and so on, it is not explicitly any one of these things, existing both as and in excess of them.

It is enough, here, to say simply that while we experience the machinery of thought, self, and subjectivity as immaterial, it is in fact a component of larger physical structures, and that the harder you press on the distinction between the interior experience of a self and the biophysical machinations of the Earth, the less stable that distinction becomes. More to the point, 'you' and 'I' are not in strict terms the bounded entities that we experience, and our conception of ourselves as such is a property of the useful hallucinations that structure experience in the first place.

The point is not necessarily to make a case for the un-reality of experience. There is, after all, no way to be outside of the hallucination your mind is constructing for you, and again, it would be wrong to call it "false" as its value is borne out by you being alive in the world at all. The point is to make a specific kind of argument about other categories of experience that feel equally natural and equally given: the culture you belong to, the nation you reside in, the racial category ascribed to you. These are hallucinations in the non-pejorative sense as well. Nobody sat you down and taught them to you; you learned them by being among people who already learned them, and they are answerable to the world in that they do a decent job of predicting how it will treat you. They arrive as observations rather than what they are, inferences, partly because they are stored in the environment itself, where you meet them already in place and in the arrangement of things rather than as something you had to work out. Just as language is indexed and reproduced through text, speech, and brains, identitarian categories are indexed and reproduced through people, images, and the environments they move through.