Drawn by Abstraction

Beginning

Recently, an ex-Apple designer and the head of OpenAI announced that they are collaborating to develop tangible AI products. In their announcement post, they mentioned that, “Computers are now seeing…”(2025). I have a problem with that and everyone should too. This is not a problem of the present times as it has a long history. It started with humans calling it "perception" (1961), later "vision" (1978), and then the more recent association with "gaze" (2024). This problem is rooted in the human tendency to anthropomorphize everything across culture and history, meaning associating human characteristics or behaviors to non-human entities or concepts. So they invariably did the same for machines too, borrowing terms from human cognition. All the previously used terms also carry implications of intention and awareness which machines inherently lack.

But, the primary concern stems from the human focus on “product” or “result” in these systems rather than the “process” across the computational history. This led to an interpretability divide between the processes machines employ to recognize the environment and the human understanding of those processes.

Diagram and Diagrammatization

Despite being separate fields, computation, cognition, and design share similar representational conventions, all working with text and images. They use two types of conventions: arbitrary (like the text "tree" with no inherent connection to its referent) and homomorphic (like photographs of a tree that preserve structural relationships to the original). Diagrams form a middle part of this continuum becoming mixed forms which has both these characteristics.

Even though machine implements everything in zeros and ones, their representations are diagrammatic. Simply put, in machine’s context, what it does is “machine recognition,” how it does is “diagrammatization” and what it leads to is “machine abstraction.” Diagrammatization then refers to how systems handle these representational relationships in order to access value for processing.

Dream of the Plotter

The first original autonomous artistic images were created using a flat bed plotter, making them not only the technological predecessors to modern AI systems but also as mediators of creativity enabling material thinking.

Do you know that the most widely used test image in image processing research comes from the Playboy magazine? or that the machine was trained to recognize sentiments through scraped online user movie reviews from Rotten Tomatoes? There is fascination in every corner of the machines’ history, so is the obscureness. Throughout computational history, such datasets and algorithms were curated and processed, with their intermediate steps drawn through pen plotter.

Processing and Selecting

The curation process is foundational, as it captures a rapidly evolving technological timeline that might otherwise be missed as innovations continuously advance. By deliberately pausing to document these elements, this process creates an opportunity for audiences to examine what has been overlooked, under-appreciated, or simply left behind in the collective urge to rush forward. This retrospection allows to better understand the technological trajectory while simultaneously creating grounding of how one has arrived at the current state. It might also provide insights into what developments may lie ahead.

This selection deliberately spans both classical and modern computational contexts, with 2012 serving as the pivotal demarcation point, a year when a revolutionary research paper fundamentally transformed computational approaches worldwide through its neural network architecture, AlexNet.

Forms of Knowing
 

The common understanding of research typically centers on systematic empirical investigation, yet artistic research occupies a more nuanced position in knowledge production. The act of copying or reproducing reveals new elements previously unnoticed, suggesting that repetition itself can be epistemic. When machines engage in a similar process of reproduction, entirely new forms of knowledge emerge through human observation.

Unlike the conventional approach where algorithms process inputs towards definitive outputs, this framework deliberately subverts the conventional algorithmic approach by prioritizing the intermediate processing steps rather than the final result. The processing deliberately crosses the temporal boundaries by running classical datasets through modern algorithms and, conversely, applying classic algorithms to modern datasets.

The artistic research systematically yielded 44 pen-plotted plates across 12 series, conceived as an exhibition, each revealing different facets of the machine abstraction processes of images and texts. Complementing the exhibition, a comprehensive catalogue was produced to document the research, providing the audience with essential context detailing the process from initial conception to the final realization.

Insights and Discussions

It was revealed that diagrammatization has evolved yet remained central to machines’ interpretation of visual and textual information, with significant variations in the abstraction level across different generations. The visualization through pen plotting creates tangible manifestations of abstract computational methods that enable understanding and critique of machines by presenting the underlying processes in a perceptible form that spans the historical development of machine abstraction.

For machines, diagrams serve as a unifying framework that both feeds and extracts textual and visual information, acting as a synthesizing bridge. Rather than being intermediaries, machines transform both text and images into diagrammatic structures as a form of abstraction to derive value through relationships. 

If any representation of a relationship is a diagram, what are the fundamental differences between machine and human diagrammatization?