Abstraction, distraction, automation

originally published 2017

Composition in Line by Piet Mondrian, 1917 [public domain image]

In 1917, Piet Mondrian and colleagues launched the journal De Stijl, to document and promote their increasingly abstract art, design and architecture. They believed in a quasi-religious philosophy which held that humanity progresses from physical or natural states of being to increasingly spiritual or abstract ones. Art was seen as a means for facilitating this evolution, as a direct aesthetic expression of the universal. And as artists, they saw themselves as among an elite few who could convey divine wisdom [1].

A century later, computational abstraction has helped humanity move away from earthly matters — literally and otherwise — led by a scientific, not spiritual, elite. I propose that we might look for the universal by digging up the roots of abstraction (both artistic and automated) in nature, and connecting them in aesthetic experience.

From roots to roofs

Artistic abstraction — the use of simple shapes and colors — was held by Mondrian to represent the spiritual or universal. He therefore sought to take art into a realm of pure intellectual abstraction, as opposed to the more emotional, individualistic depiction of the natural world. Indeed, he came to see his art as so highly evolved that it tended toward non-art: it was the beginning of the end of art itself. [2]

By 1917, Mondrian was thus moving away from painting trees and landscapes to the rooftops of Paris and the street grids of Manhattan, drawing inspiration along the way from architecture and music. He saw New York City as abstract life given form: “In the metropolis, beauty is expressed more mathematically” [2]. But while architecture surpassed the naturalism of painting and sculpture, it was not the ideal art, for Mondrian felt that with three dimensions, architecture impaired the perception of relationships, and it was the latter that would reveal the universal. Two dimensions were the best way of expressing binary relations. [3]

Enter the computer

Fast-forward 20 years, and Marshall McLuhan declared, “An abstract painting represents a direct manifestation of creative thought processes as they might appear in computer designs” [4].

At the same time, engineer A. Michael Noll began experimenting with a room-sized IBM computer to produce visual output, and noticed that the patterns he was generating resembled abstract art. He then reproduced Mondrian’s 1917 Composition in Line using the computer, and asked 100 people to compare photocopied versions of his and Mondrian’s compositions.

This was a version of Alan Turing’s test to detect whether an unseen conversation partner was human or machine; for Turing, the computer was a universal machine (or “abstract machine” in theoretical computer science) that could emulate any other calculating machine. Noll found that less than a third of people were able to identify Mondrian’s original composition, whereas about two-thirds preferred Noll’s own computer-generated version. [5]

So what? Had computers finally brought about the end of art that Mondrian claimed to have started? Had we become so freed from earthly concerns that we could ascend into the sort of spiritual enlightenment that Mondrian envisioned? After all, the same IBM 7094 that Noll used in his experiment had just helped launch the first American into space.

Back to Earth

While computers represent complete abstraction, aside from Turing’s thought experiment, they are built from natural materials. And all information has a material basis [6].

Similarly, Mondrian’s paintings are more than mere geometry. His paintings were never rigorously planned, but done intuitively and improvisationally. No painting was ever considered finished: Mondrian constantly re-worked, sometimes revising a painting for years before finally deciding to simply move a single line an inch or so. [2]

In person, you can see where he rubbed things out, painted over sections. There are visible brushstrokes, and many shades of white, grey and black, despite his desire to go for “pure” primary colors. The wood frames are interesting, and often integral. These are material objects with depth and dimension, not flat pictures.

Indeed, Mondrian called his art “Abstract-Real” because it represented the real. In fact, his 1917 painting copied by Noll came not from divine inspiration, but was simply a more abstract version of his 1915 painting Pier and Ocean. (In his sketches, he even drew in the pier.) By subtracting elements progressively and rendering the picture in horizontal and vertical lines, he shifted our perspective to what his contemporary, Wasily Kandinsky, called “the in-between”. The artist sees the spiritual in the material, according to Kandinsky, while science sees only the observed. [7]

Noll’s experiment wasn’t a fair comparison anyway. By presenting each work as a Xerox copy, he scientifically controlled for variables, but in so doing, ignored the canvas and paint and crossing-out that gave materiality to the Mondrian.

Reality reduction

Computational abstraction, then, is grounded in science, not spirituality. By means of immaterial mathematics and geometry, it creates increasingly realistic models (abstractions) of natural phenomena. Scientists (computer and otherwise) aim to reduce reality into a few equations, or lines of code [8].

“Computational thinking,” we are told, is a fundamental skill that everyone should learn and that will revolutionize our health, economy and social relations. Key to this is abstraction, defined as recognizing and generalizing patterns from specific instances to create parameters. [9]

Mondrian similarly aimed to generalize worldly phenomena (such as the rhythm of the sea), through abstraction of form and subtraction of detail. But he was not interested in patterns or parameters. Each of his outputs was unique.

This distinction widens when we bring in the other pillar of computational thinking, automation. Computational thinking is about the automation of abstractions. This means layering up abstractions (computer code, in this context), using machines to interpret these abstractions, to enable us to build large, complex systems. [10]

Backlash to the future

Automation is getting some bad press lately. There is a growing anxiety, even grudging acceptance, that computers are taking over the world. Some feel we have already lost control of our systems of food, finance, social interaction and economic life to increasing automation and complexity [11]. There is a growing backlash against a technological elite that views people as data and uses computers for control.

Others, however, look forward to merging with machines, seeing mutual and even spiritual benefits [12]. Indeed, the physical world is now viewed as engaged in various forms of computation [13][14]; some see the entire universe as a computer [15]. If both we and computers are made from natural materials, and if the whole universe is a computer, then maybe machines are simply the next phase of our evolution — hosts to our intelligence, culture, spirituality.

But is computational thinking be seen as a solution to the problems created by computation? Is anyone promoting artistic abstraction as a solution to society’s ills?

Look in between

Maybe there is a middle ground. “Physical computing” moves computational practice off the screen and into the real world — specifically into the hands of artists. This often involves using very simple computing machines (microcontrollers) to interface the real world with the digital. It involves thinking computationally, yes, in order to translate messy real world interactions into simple instructions for and conversations with comparatively stupid machines. Igoe and O’Sullivan [16] prompt artists to consider “levels of abstraction (and distraction)” in using computational thinking and making in their practice. But they demote the computer to mere device, in favor of human interactions, encouraging breaking, hacking and subversion.

Scaling down has led to scaling up. The increasing miniaturisation of computing technology has enabled microprocessors to be embedded into cars, refrigerators, body parts. They are no longer as simple as the ones artists typically use, and they silently communicate with other devices and with their corporate masters. With such a computing infrastructure, we have, in a way, moved back to Noll’s time of room-sized computers: you were literally inside them [17]. Today, however, that computing infrastructure is planetary.

Mondrian was right in equating architecture with mathematics, and he was right that architecture obscures our perception of relationships. When you’re inside the machine, you can’t see the forest or the trees.

So too Igoe and O’Sullivan are right in stressing levels of abstraction, for we need a systems perspective to fight distraction. If the whole universe computes, might we find the universal by taking a computational view of the natural? If abstraction enables us to see the in-between, a systems view of abstraction focuses on the relations between things. Armed with the knowledge of what Mondrian’s Composition in Line depicts, we can connect things up to see the sea. Our challenge is to de-abstract and de-compute the world, to focus on material and human relations, by harnessing artistic, critical computational thinking. Fight computational distraction, embrace artistic abstraction.

Notes

1. Holtzman, H. and James, M.S. (1993) The New Art, the New Life: The Collected Writings of Piet Mondrian. (New York: Da Capo)
2. Bois, Y-A, Joosten, J., Zander Rudenstein, A. and Janssen, H. (1994) Piet Mondrian: 1872–1944 (exhibition catalog). (New York: Leonardo Arte, with the National Gallery of Art and the Museum of Modern Art)
3. Bois, Y.A. (1987) Mondrian and the theory of architecture. Assemblage 4 (Oct. 1987), pp. 102–130.
4. McLuhan, M. (1964) Understanding Media. (New York: McGraw-Hill)
5. Taylor, G. (2012) Routing Mondrian: The A. Michael Noll experiment. Media-N 8(2) (Fall 2012) http://median.newmediacaucus.org/routing-mondrian-the-a-michael-noll-experiment/ (Accessed 5 Apr 2016)
6. Haigh, T. (2013) Software and souls; programs and packages. Communications of the ACM 56(9): 31–34.
7. Kandinsky, W. (1947) Point and Line to Plane. New York: Guggenheim Foundation.
8. Wolfram, S. (2002) A New Kind of Science. (Cambridge, MA: Wolfram Research)
9. Wing, J. (2014) Computational thinking benefits society. Social Issues in Computing 10 Jan 2014. http://socialissues.cs.toronto.edu/2014/01/computational-thinking/(Accessed 6 Apr 2016)
10. Wing, J. (2008) Computational thinking and thinking about computing. Phil. Trans. R. Soc. A ,366, 3717–3725. doi:10.1098/rsta.2008.0118 (Accessed 6 Apr 2016)
11. Hodson, H. (2015) No one in control: The algorithms that run our lives. New Scientist 3007 (7 Feb 2015) https://www.newscientist.com/article/mg22530070.300-no-one-in-control-the-algorithms-that-run-our-lives/?full=true&print=true(Accessed 6 Apr 2016)
12. Kurzweil, R. (1999) The Age of Spiritual Machines. (Viking Press)
13. Van de Velde, W. (2003) The World as Computer. Proceedings of the Smart Objects Conference, Grenoble.
14. Ledford. H. (2013) Plants perform molecular maths. Nature 24 June 2013. http://www.nature.com/news/plants-perform-molecular-maths-1.13251 (Accessed 6 Apr 2016)
15. Lloyd, S. (2010) The computational universe. In Davies, P. and Gregersen, N.(Eds.) Information and the Nature of Reality. (Cambridge Univ. Press)
16. Igoe, T. and O’Sullivan, D. (2004) Physical Computing: Sensing and Controlling the Physical World with Computers. (Thomson/PremierPress).
17. Bogost, I. (2017) You are already living inside a computer. The Atlantic Sept 14, 2017. https://www.theatlantic.com/technology/archive/2017/09/you-are-already-living-inside-a-computer/539193/ (Accessed 21 Oct 2017)