Motivation (2022)
Science and technology directly change artistic practice and expression.
Photography for example, by making reproduction of specific scenery trivial, had myriad effects. It (among other trends) relieved artistic practice of the need for a reproductive purpose, leading artists to explore media, practice, subject, and technique in novel ways evident since the late 1800s in the explosion of impressionism, abstraction, and conceptualized art; it eventually provided a grounding source of subject matter for new types of works and practice ("hyperrealism" a la Chuck Close, Gerhard Richter, and now the comical CJ Hendry); and it's simple existence created an entirely new media for expansive work (besides the democratized use of the everyday, consider Jeff Wall's or Cindy Shermans's compositions).
There are simply too many cases of science and technology influencing what we see and to easily associate in depth here:
- Andy Warhol's and Roy Lichtenstein's rich but methodologically "cheap" pop art,
- Bridget Riley's perceptually confounding op art,
- Richard Serra's monumental and tenuous steel sculptures from shipworking steel plate,
- On Kawara's bleak, repetitive acknowledgements of existence,
- Nam June Paik's videoized experiences,
- James Turrel's delicately engineered light sculptures,
AI/ML capabilities like Dall-E 2 (as well as the underlying NLP technology) similarly promise to restructure artistic practice among many other aspects of modern life. However it is unclear how, and our society needs creative practitioners to explore what themes in subject, media, exhibition, and interaction teach us about ourselves in the presence of machines that can so adeptly generate language and (digital) visuals. This process has already started of course. I like the evocative and CGI-heavy video installations by Refik Anadol.
Goals
I am interested in studies with AI technology with a more traditional, and ideally more reflectively intellectual focus. I am curious, broadly speaking, what AI tech like Dall-E 2 "understands" from art history about art, and how that might influence traditional human expressive practice. As a painter, I both receive personal value from the process of "doing work" and see clearly the impact of "human-scale" physical work on those I share my paintings with. There is something indelable about human practice and engagement with the physical world that, to me, deeply informs meaning in art: A painting I make as a gift for my son is as much a gift of my experience, thoughts, effort, and time as it is a simplistic visual on a wall. Obviously, a tool like Dall-E 2 operates purely on digital patterns, not on our physical experience. Also art often deals, faithfully and productively, with the taboo and even sometimes the obscene, in conflict with in-built public safety measures in Dall-E 2.
Art is often by nature uniquely experiential, while generative digital visuals are fundamentally fungible. Art is emotional in both execution and observation, often with the otherwise indescribable being the core focus of both processes. Art is often by nature a confrontation with the status quo using experiences or languages that are uncomfortable, but thereby rooted in intellectually observing or addressing perceived failings of the social strata it comes from and speaks to. How do these processes proceed in the presence of computer programs that can generate imagery on demand?
Studies in Progress
My initial studies appear to revolve a bit around how subject matter can be generated with AI tooling. One traditional starting point for artists is simply to copy masters, and learn from history. So what would an artist learn from Dall-E 2's "understanding" of artistic works and artists oevre? What works would thus be produced? This idea also aligns with "practical" efforts to productize AI, which seem to often concern knowledge generation and productive summarization.
The below images are samples of Dall-E 2 responses to simplistic prompts like "A <artist> painting", with reasonably well known and influential artists as denoted in the caption.
This isn't entirely a trivial exercise. On the one hand, the prompt is simplistic and could be exercising the memorization potential of a SOTA model. To be clear, that the prompt can generate even reasonable images at all is rather remarkable; but it is a perfectly feasible "traditional" search function because matching images can be readily looked up explicitly. Despite some light criticism below, I emphasize that it's simply amazing Dall-E 2 can be reasonably generative and present examples even remotely connected to artists work.
How "good" the examples above are is debatable, beyond the ubiquitous debatability of quality in art. Consider first the O'Keefe in the third column of the third row. This is a plausibly realistic reproduction of an O'Keefe painting. Dall-E 2 "understands" O'Keefe had some significant and famous focus on detailed views into the reproductive organs of flowers, and used gradiated color schemes. The Monet (row 1, column 3) and the Khalo (row 2, column 2) are similar strong examples if a bit compositionally weak. Broadly speaking Dall-E 2 "understands" the major stylistic elements of these artists' works, even if it doesn't present a strong response. For example, the Albers (row 1, column 1) seems to reflect color and geometry, but with vibrant colors and a geometric structure that would probably be seen as quite uncharacteristically "uncontrolled" for Albers. The Kruger (row 2, column 1) understands "red" and "text", but seem to miss the complexity and advertising influence (in both directions) of Kruger's work. The Richter (row 3, column 4), seems to know Richter painted abstracts and some monotone seascapes/landscapes but completely misses the vibrancy, layering, and technical virtuousity of Richter paintings.
To return to the discussion above the examples, the goal of a "copy study" is not simply to make examples. The goal is to undertake a practical, and physical, human-computer interaction with the tool. Part of this exercise is subject generation, while another is subject utilization. The image below compares another variant snapshot of the generated Albers painting with a painted version (up to edits) of the same.
One project would be to embark on a full series of paintings -- "masters studies" -- using Dall-E 2 source material as above. Unlike the study above these will likely be square canvases reflecting the uniform pixel boundaries of the examples, with the embedded buttons but likely without the color coding. Including some form of visual acknowledgement of the source subject matter is an important visual element carrying the source and schema of the works forward in any representation of them.
What such studies will accomplish is, by nature and on purpose, unclear. Or rather the immediate purpose is simply to do the works, communicate them, and participate in the resultant learning process.
As mentioned above, simply asking for an example of a particular artist's work is simplistic in the right lens. Models like Dall-E 2 are advertised as engines for the creation of new visual subject matter that probably does not exist in the "corpus" of training data. The example below shows 56 examples of a particularly productive take on a generative task related to one very famous painting: Vermeer's "Girl with a Pearl Earring". Added is a modern flavor by asking for an emo girl with a pearl earring, while making no reference to the Vermeer itself.
As it stands, the image above is designed as a printable work of it's own right. The use of multiple images emphasizes the "generative" and "scalable" nature of AI generated visuals, while allowing the viewer to interpret themes (up to the artist's own embedded selection bias) in the images Dall-E 2 believes suitable for the prompt. Interesting features, to us, are as follows: While the prompt itself could suggest any framing of a "girl", the responses indeed are similar in character to the framing (portrait-ish) of the Vermeer without any explicit mention of it. Many cases even include a sideways facial glance that is very reminiscent of the painting's subject. There are of course pearl earrings, but almost not exclusively having pearl necklaces and pearl headbands (or a blend of the two). Dall-E 2 also seems to occasionally inject a head/hair covering as seen in the painting, but also unlike the complete covering in the painting, perhaps "perceiving" hairstyle to be a key feature of "emo" styling.
Many of these images could be worthy subject matter for a painter sufficiently adept at recreating them. Masters like Gerhard Richter have exhibited works along similar lines. I am currently partial to potential similar explorations, at some possibly even much larger scale, of printed works derived from many responses to prompts that hint at the impact of artistic work on visual representation. Here the "physicality" would need to be the immersiveness: could an entire room be tiled in such images, at say 3.35 in. squared scale (a Dall-E 2 image, at 300dpi)? Could non-repeating wallpaper with unique generative example images derived as above, possibly with human input to train aesthetic "selection bias", be made? Such experiments might effectively emote the technology's reach beyond effective "human scale", while playing on their derivation from deeply human content.
Motivation (2026)
AI can wholly "make artistic looking things" now. Dall-E (2), among surely others, was just the beginning, and frankly I've found myself feeling it was much more interesting when it was worse. At least per image.
This is very weird and, I think, misunderstood. Specifically, the "image generation" part is not where art can be found it. There are perhaps some corners of the literal visual content that have some artistic quality to them, unfortunately, as there is a valid thread that algorithmic image generation is built on theft. For my part, at least, focusing on singular algorithmically generated static digital images misses the core features and purpose of the technology. Generative models are built for scale, for volume, for velocity, accessibly but potentially without singular value.
This is what theunending.ai is trying to be about. "Relentless image generation." You can learn more there.