One day – a day now many decades in the past – a fire brigade in the American Midwest is called to a fire. The firefighters enter the kitchen where the fire is and pour water on the flames, but it seems as strong as ever. They repeat the exercise, again, to little effect. Then the lieutenant in charge of the crew begins to get a funny feeling — what he will describe later, to an interviewer, as his ESP. Based on this feeling, and this feeling alone, he pulls his men out of the building. As soon as they are out, the floor collapses. The fire had been fed from the basement below.

First told by Gary Klein in his 1998 book Sources of Power, the fire lieutenant’s sixth sense has been celebrated in academic journals and popular books ever since. Firefighters are people who run towards, not away, from fires, but the lieutenant’s instinct told him to run. Fire, by its very nature, is idiosyncratic and variable, which means that a lieutenant who only fought fires that followed a prescribed and predictable course would be of little use.

In the middle of this chaotic scene, it was the lieutenant’s experience, not deductive reason, that allowed him to see how this fire was something else altogether. In retrospect, we can understand the clues that were there to be found: the kitchen was hot, too hot for the size of fire they could see; the room was quiet, too quiet for the amount of heat they could feel. That the unquenchable flames came from a muffled source below now seems a matter of reason, not magic.

This deductive work, slow and careful, can take place in retrospect. It was pattern recognition, the sense that what was happening around him did not quite fit, that saved the lieutenant’s crew. The fire’s dark pattern told him to run.

To a cognitive scientist, we are all firefighters. A lawyer reading a contract will throw up her hands well before he can articulate exactly how the client will lose his suit. A teacher tingles when the bubbling noise of the classroom shifts from un-tutorable anarchy to a restlessness to learn. A biochemist looking at the printout from an experiment knows her machine is failing before the fault light comes on.

To be good at your job, in other words, means to have a feel for the relevant patterns. In his book Bounce, journalist Matthew Syed describes how tennis players receiving a serve watch the angles of opponent’s torso for clues to where the ball will go – an argument, perhaps, for returning to the Ancient Greek practice of competing naked. He reports on studies of hospital nurses in neonatal wards that find them able to spot infections before the canonical symptoms appear. Even things as subtle as shifts in personality that, with enough experience, trigger in the nurse a sense of a peaky child, of something not quite right.

Once you see a pattern, you may be able to describe it and make it a matter of reasons and rules, codify it, teach it. The “McMansions” thrown up by suddenly-wealthy clients in the suburbs are full of architectural dark patterns, and the charm of a website such as “McMansion Hell” is that it gives us the reasons and rules behind why we find these houses so offensive.

More seriously, the architect Christopher Alexander sought to enumerate the “pattern language” of buildings in a series of books in the 1970s. More than lists of rules, they are stories in which patterns play a starring role. Stories, for example, about how the cloister around a courtyard provides a moment of transition from inside to out, turning the open space from a threat to an invitation. And counterstories about how, in the absence of a pattern such as this, other patterns take over: the courtyard decays, empties out, feels dead – a familiar sight on an American college campus that, in expanding too quickly, has miscopied or misremembered the pattern from an half-remembered model.

Alexander was unusual in his rich use of story and detail. That’s because architecture is an elite profession, and because the more elite the profession, the more abstract the patterns and the language used to describe them. It’s a false syllogism – the patterns of a building fire are certainly no less complex than those in a lawsuit – but the fallacy persists. Under the grip of a half-remembered philosophy, we sometimes feel that the details, the substance, of an experience are too particular, too dirty, too mundane, to be meaningful.

We try, for example, to refine the complexity and dirt of war into the game of Chess. To the ambitious parents wanting to advance their child’s career, Chess appears to be more than just a game: in its distance from the actual facts of war, it appears as an ur-game, a game not of strategies, but of Strategy itself. Next to chess, the roleplaying and storytelling of a Dungeons and Dragons adventure seems like a feeble distraction. The more abstract and general the activity appears, the more powerful its methods must be.

Yet the facts do not bear this intuition out. Consider the two boards below. Both have the same number of pieces. To a novice player, the two are equally difficult to describe or remember: on average, after being shown either board for a few seconds, he can correctly place just a few pieces. The same, it turns out, is true for a master player when faced with the board on the left – but not, it turns out, with the board on the right. Given the board on the right, a master player can replicate it exactly. (When I presented a Chess-minded colleague of mine with the second board, he remembered it not only the next day, but the next week, and expects to remember it when I returned to London the following year.)

The difference that gives my colleague the advantage is simple: the configuration of the board on the left could never have arisen in ordinary play. The board on the right, by contrast, fits many of the standard patterns of arrangement and conflict that an advanced player is likely to have encountered before (this board comes in particular from a very unusual game, known as Garry Kasparov vs. the World, which is a story for another time).

What these results – and others like it – tell us is how the specific nature of our experiences persist even when we struggle to rid our practice of what we think of as the mundane and specific. Chess players live in a world no more abstract than that of firefighters: they watch, and learn, the patterns to hand.

And yet despite evidence like this, the dream of universal patterns persists. If Chess is too rooted in its patterns and rules of play, we say, let us go beyond it!

If you had been given such a mandate, a mandate to invent a game of pure and unconstrained pattern-making that exceeded Chess in every way, you could do little better than the game of Go: a grid of 19 vertical, 19 horizontal lines on which players place, in alternation, black and white counters.

In contrast to Chess, it is simple to state the rules of Go. Begin with the children’s game Othello, in which you capture a line of your opponent’s counters by placing yours on either end. Go is similar, except that instead of capturing lines, you capture areas, a feature suggested by the Chinese word for Go, weiqi: “encircling game”. Apart from the capture rule, the only other is that of “ko”, which simply says that, within a game, a board position may never be repeated.

Go being played by Japanese-American farmers in the US during the depression

As in physics, the transition from one dimension to two changes everything.  Othello is a forgettable game, but the game of Go has lasted thousands of years, and given rise to a vast literature and culture of play. The simplicity of the rules means that, in contrast to Chess, the number of valid moves is much higher.

This lifting of constraints makes Go a fearsome task for a computer. Chess computers play largely by intelligent analysis of if-this-then-that: if I do this, your best response is that, to which I should do this, or perhaps this, to which you might do this… This branching set of possibilities is called a game tree, but Go’s unstructured nature makes these trees explode, branching in ways that are practically unprunable.

Machines did, finally, beat a master player at Go, in Seoul, Korea just this year. The program, AlphaGo, drew in large part upon recent developments in image recognition. These algorithms use the same basic principles behind the functioning of the human eye to break up a visual scene into its basic components of lines and shapes, and then learn how these components combine to form objects: cats, tables, people. AlphaGo used them to watch and learn the underlying patterns from databases of human games – and then to play against itself with these patterns in novel combinations, learning from the new ones that arose to become even better. AlphaGo, it might be said, not only learned Go, but evolved a new culture of play within its own electronic brain.

As a consequence, its game was unusual indeed. “The sadness and beauty of watching Google’s AI play Go”, was the title of a Wired Magazine article by Cade Metz, that captured the uncanny feelings the game generated among commentators and expert players, who were stunned by the alien and inhuman nature of the plays the program made.

This surprise came to a head in the second game of the contest. In response to AlphaGo’s now-famous thirty-seventh move, the machine’s human opponent Lee Sidol literally got up and left the room. The move was so surprising, so unusual, that commentators could barely find the words to describe it, to classify it with the familiar terms of the trade. As Metz described in his Wired piece, the machine itself acknowledged this in an unexpected fashion. Its internal code, containing a model of human play, automatically computed the chance of a human player making this move: one in ten-thousand.

At times, in other words, AlphaGo and Lee Sedol were barely playing the same game at all: the two players were inside entirely distinct pattern worlds that happened to meet on a single board. The incomprehension of these masters proved, again, the same rootedness of experience that even our most abstract patterns seem to share — with the strange twist that one of those experiences was completely contained within the mind of a machine.

And yet – and yet? Beyond the most abstract of architectural histories, the most unconstrained of games, there lies, like a limit-point, the world of mathematics. Surely here must be a world of patterns that have some generality, some reality, beyond experience? We have all, at some time or another, been asked by a teacher to complete a series like this:

1, 3, 5, 7, 9, …

The pattern suggests itself: the odd numbers, in order. We complete it with 11. Or one like this:

1, 4, 9, 16, 25, …

The squares; the next in the series is 36. But look again, and consider the pattern made by the *difference* between successive terms: 3, 5, 7, 9. One completes the same pattern, now, in a different way. Instead of “squaring the next integer”, it becomes “add two to the amount you added to get the number before”. It was the recognition of this second, equivalent description that was at the heart of the first computer: Charles Babbage’s appropriately-titled “difference engine”. The key to simple patterns like this is the Newtonian calculus, and once you see that, then it is as if an undifferentiated sea of sequences precipitates out – patterns like puddles that merge in heavy rain.

Consider, however, this series, for which the Newtonian calculus provides little assistance:

1, 11, 21, 1211, 111221, 312211, …

What is the next term? Not an easy problem, but a clue may be provided by its name: the “Look and Say” sequence. If you have not encountered it yet, it is worth trying to crack the code before moving on.

Welcome back. The rule is simple, though hard to see: to build the series, one goes back and forth between a number and its digit-by-digit description. If we “describe” 1 we might say it is “one ‘1’”; therefore, 11. But “11” is just “two ‘1’s”, or 21. 21 is “one ‘2’ and one ‘1’”, or 1211. And so forth: the descriptions ramify outwards. The mathematician John Conway was the first to describe and analyse the sequence in detail, and one of his first tasks was to analyse how quickly, on average, the length of the numbers grow.

The alternation of Conway’s sequence between numbers and their descriptions seems like a trick, something unfair or silly, and when I present the sequence in the lecture hall many students laugh — they get the joke. But while Conway’s sequence is funny, it is most certainly not a trick. The alternation between numbers-as-numbers and numbers-as-descriptions is at the heart of some of the deepest theorems of mathematics, including Godel’s famous “incompleteness” theorem, that demonstrates the existence of “unprovable truths”, mathematical statements that can be seen to be true by an observer, but cannot be proven to be true within the system itself. When Godel sat down to demonstrate this strange result, he first showed how to encode statements about numbers as numbers themselves, literally “programming” with integers as if mathematics itself, the world of Platonic forms, was a computer system.

Even if it doesn’t rise to such dizzying heights, Conway’s Look and Say series has remarkable structure. One more step in the sequence above gives us 13112221; a step after that gives us 1113213211. The first five digits of this number, 11132, it so turns out, evolve independently from the rest of the sequence. You can, in other words, get the next term after 1113213211 by evolving 11132  forward in isolation (to get 311312), and the other set, 13211, forward, independently (to get 131221), and can continue to do so, each string growing, and, indeed, decomposing in turn. These two strings of digits, 11132 and 13211, behave as “atoms”, that decay and mutate independently.

Conway found that there were 92 such “atoms”. Which, as he pointed out, is just as many as there are naturally-occurring elements – Carbon, Oxygen, Helium, Uranium, etc. – in the periodic table. Fortunately, the fact that there are just as many Conway atoms as in the real world is accidental. If such a strange meta-mathematical game had predicted the deep structure of chemistry, I believe most physicists would have literally gone mad.

Reading Conway’s analysis of the sequence is like watching a highly intelligent player play himself in Chess. Is all mathematics like this? Or does it, unlike everything else we’ve seen, exceed the experiences of the people who create it? Chess and Go, Lawyering and Firefighting and Nursing, may be forever tied to the experiences of their domain, but is this true for mathematics? Can mathematics go beyond the journals in which the mathematicians publish?

The very strange (and yet very well known) answer is that it does. While Conway’s sequence doesn’t predict the periodic table, physicists long ago worked out the sequences that do. It is the “spherical harmonics”, rather than the look-and-say, that predicts the structure of electron orbits, and thereby the patterns of chemical bonding among the elements – why a metal like Sodium will react with Chlorine gas to make table salt. More advanced mathematics does the same for the nucleus of the atom, predicting the patterns of decay and radioactivity as elements transmutate.

That this works is the everyday, commonplace miracle. Galileo was one of the first to see it, writing that the book of nature was written in the language of mathematics. The success of physics in the 19th and 20th Centuries took Galileo’s vision mainstream. An article from the 1960, by the physicist Eugene Wigner, described this odd feeling as the “unreasonable effectiveness of mathematics”. Chess tells us nothing about the outcome of a naval battle, and a firefighter’s experiences give him no leverage in a neonatal care ward. Mathematics seems the only exception to the iron rule of experience.

The question for the 21st Century, perhaps the only fundamental question for the human species, is how far this effectiveness will spread. While mathematics appears to apply remarkably well to the prediction of patterns in the physical world, it has had much less success in predicting the behavior of living systems, social behavior, or the mental world of an individual.

Most of our attempts to extrapolate the tools that worked so well in physics have failed utterly. Our best attempts to model these more complex systems put us back, ironically enough, in the realm of experience-based pattern learning. As with the AlphaGo machine, we train our computers on large numbers of examples – the stock market, or traffic in a city – and try to get them to extrapolate the rules. Our computer programs mimic human thought. Idealized thought, perhaps, as in the case of so-called “Bayesian” statistics, and massively sped up by server-farms and repurposed game consoles — but human thought nonetheless.

We are a long way, then, from the magic of the physical sciences, where our tools allow us to predict the novel behavior and structure of elements that have never been seen before. A computer can train on examples from the plays of Shakespeare, and predict that readers will classify King Lear as a tragedy; but it is impossible – as of yet – for that same machine to imagine the play Shakespeare might have written had he written more, or under different circumstances.

The dream of extending mathematics beyond the material world has its adherents, and science fiction is full of stories of machines, or geniuses, that find the laws beneath the patterns. My own work began – as it did for many physicists who turned to the human sciences – with this hope.

Yet I find myself increasingly interested in the residue that this task leaves behind. The patterns our mechanical pattern-finders do not match; the outcomes they do not predict; the patterns in the world the human mind must discover for itself.

I see this, for example, in recent work on the nature of mathematics itself. I was fascinated by the human impulse to play, which I believed I saw not just in children, but in the student, and even professor, of mathematics. Wanting a concrete example, I took a particularly mathematical branch of physics, string theory, and looked at the tens of thousands of papers that had been written in the field since its invention.

When I turned the standard pattern matching, pattern finding, tools loose on this archive of papers I was pleased — but not unduly shocked — by the ease with which the machine could classify them into groups and categories, identifying collaborations, influences, themes, and hunches that I knew the string theorists cared about.

More interesting, however, were the papers that confused the machine – the ones it flagged as unusual, unexpected, unclassified, the “thirty-seventh” moves of string theory. Some were clearly nuts – crank articles that (to be polite) were off in the margins, or the wilds, of pure speculation. But others I recognized when I opened them: some of the greatest, most subtle, and most influential ideas in the field.

We increasingly allow machines to monitor us, and serve us what we need; Amazon predicts the books I will like, Yelp the restaurants; governments, perhaps, the probability of deviance and the trouble I might cause. Are we simply that predictable? I find the results of my most recent investigations, the search for the residues and thirty-seventh moves, the things machines cannot predict, a cause for hope. When we look to those unusual places, I find, sometimes, the words, and actions, that bring our lives the most meaning.

………………..

http://santafe.edu/~simon

This article here is based on a talk at the Barbican’s Science on the Screen, organized by the London Mathematical Laboratory.

Brain, Complexity, Education, Mind, Science