Report 7: Aesthetic Explorations in AI

Post Reply
Posts: 160
Joined: Wed Sep 22, 2010 12:26 pm

Report 7: Aesthetic Explorations in AI

Post by glegrady » Mon Oct 05, 2020 2:01 pm

MAT594GL Techniques, History & Aesthetics of the Computational Photographic Image ... 0f594.html

Please provide a response to any of the material covered in this week's two presentations by clicking on "Post Reply". Consider this to be a journal to be viewed by class members. The idea is to share thoughts, other information through links, anything that may be of interest to you and the topic at hand.

Report for this topic is due by December 1, 2020 but each of your submissions can be updated throughout the length of the course.
George Legrady

Posts: 8
Joined: Sun Oct 04, 2020 11:57 am

Re: Report 7: Aesthetic Explorations in AI

Post by chadress » Tue Dec 08, 2020 8:58 am

*** This post discusses material presented in the last two weeks of class ***

Screen Shot 2020-12-08 at 8.33.57 AM.png
DeepFake photograph with perfectly symmetrical eyes. Source: NYTimes.

The Real Real: The Shattering of the Photograph and the Reconstruction of Truth.

From the beginning of this class I have discussed the documentary function of the photograph as a socially-mediated, negotiable construct. Recent technological advancements are adding significant degrees of complexity to this ongoing negotiation. Artificial intelligence, specifically Generative Adversarial Networks (GAN’s) and other forms of neural networks have the ability to create photographs from an algorithm. And in doing so have further eroded the photograph as a historical representative of truth, shattered documentary norms, and precipitated new modes of establishing reality.

GAN’s are trained initially with actual photographs, and once fully educated, are able to produce photographs indistinguishable from those made by their human counterparts. A lens is no longer needed. Ditto the camera, ditto the photographer. Though the warning bells are still a faint call in the distance, this tide-shift of technology will precede a tsunami of disorientation.

“Deepfakes Are Going To Wreak Havoc On Society. We Are Not Prepared,” reads the headline of a recent article in Forbes. “In the months and years ahead, deepfakes threaten to grow from an Internet oddity to a widely destructive political and social force.” Over 70 years ago Walter Benjamin proposed that a lack of authenticity in reproducible art resulted in a politicization of artistic production. He could not have foreseen to what degree the evolution of technology would prove him right.

The power of neural networks are not yet fully understood, yet the products they generate are already available for sale. According to the New York Times, websites such as Generated.Photos allows anyone, anywhere, to purchase a “unique, worry-free” fake person for $2.99 or 1,000 people for $1,000. Other websites give the images away for free.

Photographs of people are not the only things being created from machine-based artificial intelligence. At, you can create your own artificial world (or social media account) with photographs of cats, dogs, and even rental apartments which do not have a reality-based counterpart. Pre-trained GAN’s exist for such things as ceramics, tennis shoes, and even seashells. For a price, websites such as allow customers to train their own GAN, meaning, in theory, anyone can generate a fake photograph of anything.

How will GAN technology affect our traditional relationship with the photograph as a refrent of reality? To what degree will fake photographs flood our lives, which are increasingly led in the virtual space of the internet? What happens when seeing is no longer believing? Hal Foster, in the essay Smashed Screens, suggests an “epistemological recalibration” is called for. The “artifice” must be embraced, “but not to demystify or to disrupt the real so much as to make the real real again, which is to say, effective again, felt again, as such.” But how do we make the real “real again”, when AI threatens to undermine our understanding of what is real in the first place?

Foster sees a shift in the “framing of the real” in the past decade, one which results from the “nihilistic tendencies” of postructuralist theory and postmodern art. It’s critique of authority on ideological grounds was “soon understood to erode the very ability to claim a truth or to posit a reality at all.” This shattering of the real has arguably led to our current post-truth crisis. In this world, nothing and anything is true, just as everything can be false. Fake news and alternative facts are equally embraced and amplified by algorithms and artificial intelligence designed to perpetuate bias. In response, documentary practice is slowly pivoting from the absolutes of deconstruction to the fragile nuance of reconstruction. “This shift”, Foster says, “was a response to the near monopoly, on the part of corporations and governments, over what counts as real in the first instance.” This power allowed for mass censorship to privilege one side over the other, resulting in strands of reality both seen and unseen (or known and unknown as Donald Rumsfeld put it.) According to Foster, traumatic, criminal and/or catastrophic events ranging from secret wars and environmental disasters to drone strikes and detention centers “can be partially or totally blocked from view.” Integrity is expensive in this post-truth world, and few in power wish to pay.

Making ‘the real’ real again therefore requires a new approach to documentary practice. “It becomes imperative, then,” Foster tells us, “to reconstruct these events as cogently as possible by means of media both new and old.” Forensic Architecture is what Eyal Weizman proposes as one possible approach to this problem. Weizman suggests a shift from “individual testimony” aimed at “empathy with victims” to a “process of materialization and mediatization” aimed at a general politics of human rights advocacy. Foster tells us that “such forensic practice salvages, assembles, and sequences fragmentary representations in order both to image and to narrate disputed events; these scripts can then be offered as evidence in courts of law as well as in courts of opinion.” In other words, the photographic image will have to be cast as a part of a wider narrative structure, it’s role as irrefutable evidence is now forever refutable. The tendency of photographs to deceive, whether through objective artifice, or artificial intelligence, should be understood foremost as its defining feature, it’s most basic nature.

This new understanding of the photograph’s innate nature does not preclude its ability to represent reality. Rather it’s our basic understanding of reality which must shift. Artists such as Harun Faroki, Hito Steyerl and Trevor Paglen practice a form of Weizman’s Forensic Architecture by assembling the truth through narratives built upon “fragmentary representations” which include photographs and videos. However, as Foster explains, these narrative constructions are “concerned less to expose a given reality behind representation than to reconstruct an occluded reality, or point to an absented one, by means of representation.”

The real real is not something we should take as granted, as we once did with the truth-value of the photograph. It is not something to be demystified or deconstructed as Foster tells us, rather instead it must be activated and reconstructed through new documentary approaches, and it’s veracity questioned through our courts of opinion and law. In this light, the photograph, and it’s truth-value, remains a negotiable construct. This holds true whether it was made by a camera, or by an algorithm.

Links: ... faces.html ... 93241f7494 ... fter-farce

Posts: 8
Joined: Thu Oct 22, 2020 7:10 pm

Re: Report 7: Aesthetic Explorations in AI

Post by ehrenzeller » Wed Dec 09, 2020 10:11 am

In discussing some of the high prices AI-generated works have fetched at auction, the issue of duplication—individuals using the same code and same input in order to achieve the same results—was a major point of interest. Why pay $50,000+ for a work you could create yourself? Then again, why pay $150,000 for a banana taped to the wall? Undoubtedly this has led artists like Klingemann to sell their code/algorithms in addition to the pieces at auction.

The question of ownership is also worth exploring in this new frontier. If the artist does not write the code/program themself should they be given credit for the piece? What if they don’t write the code and also have not created the input themselves? At SCOPE Festival in Miami, pieces created by Art & AI Lab at Rutgers were exhibited with no human artist’s name attached. Coming from the fine art world, this has interesting potential to democratize art and shift the focus back to the piece instead of the artist. There’s no question the artist, their persona and lived experience are given equal weight to one’s creations in today’s art landscape, where artist talks frequently offer little-to-no discussion of their creative process and instead focus on their personal journeys. This particular sub-culture would certainly distinguish itself should it eliminate the human ego.

From a purely aesthetic standpoint, I was drawn to Klingemann’s astatic Memories of Passersby I, the work of digital sculptor, Scott Eaton ( and Sofia Crespo’s Neural Zoo (

Regarding deepfakes and their societal implications: after reading the Forbes article attached to the syllabus and further reflection, I believe the deepfake revolution may be more akin to the advent of Adobe Photoshop. Certainly, things have become more complex, but if this is what it takes for society to finally internalize and adopt the mantra “Don’t believe everything you see” (an offshoot of the “Don’t believe everything you read”, back when reading was still a commonplace practice) we may ultimately be in a better place than we are right now. In a previous journal entry, I mentioned digital watermarking; since then, I found this article about AI watermarks “outsmarting” deepfakes, focusing on the NYU Tandon School of Engineering’s recent work.

Posts: 15
Joined: Wed Jan 11, 2017 10:42 am

Re: Report 7: Aesthetic Explorations in AI

Post by merttoka » Sat Dec 12, 2020 11:39 am

Originally intended for problems that are hard to solve using algorithmic methods, artificial intelligence provided a new lens for computer scientists to tackle harder problems with the help of real-world data. It was almost inevitable for these systems to be employed by artists for aesthetic explorations due to their feature detection and pattern recognition capabilities. One such artist who adopted the use of ML algorithms is Refik Anadol. He has been working with data for a long time, and he finds new ways of incorporating machine intelligence with artistic explorations. His projection mapping on Walt Disney Concert Hall processes tons of data points (memories) of WDCH archives using deep learning methods and presents the "consciousness" of the concert hall to the audience. Another artist that stood out to me in the lecture notes was Sougwen Chung. She is not only using AI algorithms as a tool but also as a collaborator. When she is drawing on a piece of canvas, robot arms contribute to her drawings autonomously.

Posts: 9
Joined: Sun Oct 04, 2020 11:54 am

Re: Report 7: Aesthetic Explorations in AI

Post by k_parker » Tue Dec 15, 2020 9:24 am

What are the social implications of the contemporary photographic image? I was certainly disappointed when I uploaded my face into Face++ Face Comparison and was only able to compare my face to the stock photo of a presumably older gentleman in the free version. It turns out that, as a woman in my mid 20’s, it is very unlikely that my face is theirs. For a better test, I will use the facial comparison technology I have readily on my devices. The facial categorization on my iphone 10 has a particularly bizarre time categorizing my twin sister and I: it appears that there are 3 separate categories of our faces with facial expression being the most disguisable tool for categorization. Google photo has only one category for our faces and we are lumped together as one mega person with rapidly changing hair color and the ability to exist in two separate places at once. However, I cannot help but think that this is only a short term problem.

As facial recognition becomes better and better and more fine tuned to see beyond my sister and I’s similar, though notably distinguishable, faces and expressions how can this software be applied in a practical way? Old photographs. Much of my current studio research is on the ability for the photograph to alter memories of experiences and places. As a twin, up until about age 9, my sister and I looked so similar that when looking at photographs even my mother is unable to identify which is which. As a result, I can never assume that any image of me is actually me and any memory I have of that event has not been superimposed on by a false assumption of identity. Consequently, my memory of before I can remember lived events takes on a dual embodiment- existing as neither one nor the other, and always either. Advanced facial recognition could be a solution to this problem.

However, I can see a world where my relatively unique situation of memory/dual identity becomes universal. With the rapidly increasing use of deep fakes, what will be the long term psychological impact of the photographic image when we can logically assume that any photograph/video of us has a chance to be falsified. We may get to a point where we can never readily assume the identity of our own bodies in representation. How will this impact our memories? I can only assume that the photograph will have less and less of an impact on how we remember situations and places. Memories may have to fall back to their gradual slippage. Or perhaps our brain’s ability to override previous information to make sense of new visual information will continue and there will just be doubt… This is a popular study showing the brain’s ability to override information and assume bodily identity over objects.

Posts: 8
Joined: Sun Oct 04, 2020 12:15 pm

Re: Report 7: Aesthetic Explorations in AI

Post by wqiu » Wed Dec 16, 2020 1:25 am

I would like to share my thoughts on Dr. Ahmed Elgammal research of creative GAN, i.e. CAN.
(6.28 MiB) Downloaded 27 times
Overall, I think it is very inspiring and critically innovative.

The research trained an GAN with universal architecture, however, two changes have been made to improve its artistic creativity.

1) dataset. GAN produces a new example that shares visual similarities to most images in the dataset. To create art, rather than faces or bedrooms, CAN uses Wiki Art dataset, rather than ImageNet for the training.
CAN Dataset.png
2) objective. The ordinary GAN's objective is to generate example to looks as similar as possible to the images in the training set. However, for art creation, this would make the model repetitively create artworks that looks like the existing artworks. The paper modeled the artistic innovation as there points:
1) generate novel works,
2) the novel work should not too novel, i.e., it should not be too far away from the distribution or it will generate too much arousal, thereby activating the aversion system and falling into the negative hedonic range according to the Wundt curve,
3) the generated work should increase the stylistic ambiguity.
The main contribution of this work is the notion of modeling creativity. The concept of "arousal", which is borrowed from Psychological research, is proved to be very important for people to recognize certain things as art. It is also discussed in Can Computers Create Art? and Visual Indeterminacy in GAN Artby Aaron Hertzmann.

As "the most significant arousal-raising properties for aesthetics are novelty, surprisingness, complexity, ambiguity, and puzzlingness", they change the generative model objective in order to produce images with appropriate amount of arousal-raising properties. In the end, they model the creativity score of a generated work as the combination of two scores. The first one is the judgement on if this work is an "art" or not. The second is stylistic ambiguity, i.e. how difficultly the artwork can be categorized into any categories of visual art. The reasoning behind is, a novel art should looks like an art, which can be done by comparing if the generated example looks close enough to the artwork used to train the model. Meanwhile, it has to be novel, so it should not fall into any existing category of art. Instead, it should be something new by itself. They use mathematical equations to implement this reasoning, and the final results are conniving enough for art exhibition audiences.

In conclusion, although novelty is not the only standard to determine if some art is creative or even determine if something is art, it is an important but difficult property to measure. This work modeled novelty critically and produced great results. I believe further development can come out following this direction.
CAN Results.png

Posts: 12
Joined: Fri Jan 19, 2018 11:09 am

Re: Report 7: Aesthetic Explorations in AI

Post by zhangweidilydia » Wed Dec 16, 2020 2:23 am

I would love to share some artworks related to aesthetic exploration in A.I

- unknown territories: searching for islands by Vinzenz Aubry / Merani Schilcher ... schilcher/
The work aims at a haptic search engine not limited by prior knowledge based on language needed when looking for information online. A camera captures the outlines of the sand mass shaped by the user’s hands and matches it in real time with 75,000 satellite images of islands all over the planet to find an island of a similar shape. A search engine that responds to human’s compulsion to discover previously unknown places
Screen Shot 2020-12-16 at 2.22.03 AM.png
- Sofia Crespo
Sofia Crespo is a generative artist working with neural networks and machine learning with a huge interest in biology-inspired technologies. One of her main focuses is the way organic life uses artificial mechanisms to simulate itself and evolve, this implying the idea that technologies are a biased product of the organic life that created them and not a completely separated object. On the side, she is also hugely concerned with the dynamic change in the role of the artists working with machine learning techniques.
Screen Shot 2020-12-16 at 2.02.44 AM.png
-Mario Klingemann
Mario Klingemann is considered a pioneer in the field of neural networks, computer learning and artificial intelligence art. He spoke to Goethe-Institut about using AI creatively and the role of technology in a modern interpretation of Kulturtechnik.

Posts: 14
Joined: Mon Apr 16, 2018 10:23 am

Re: Report 7: Aesthetic Explorations in AI

Post by yichenli » Wed Dec 16, 2020 6:43 pm

For this report, I would like to compare Warriors (2020) by James Coupe and Draw me like one of your French girls (Chop Suey) (2019) by Matthew Chan. (I omitted images from Warriors since it was mentioned in class.)
Screenshot_2020-12-16 MATTHEW XIN CHAN.png
Both works use DeepFake to change the scenes in well-known movies. The original Warriors (1979) film is centered on a New York gang that was falsely blamed for murdering a gang leader, therefore chased by both the police and other gangs. The movie touches on identity and policing, which is one reason for the pushback against facial recognition. As the visitor walks into the installation, their faces are classified, then imposed on the actors' bodies in the movie based on the perceived similarities between them. The categories into which the visitor is classified is reminiscent of how "race and gender determine how 'identities, rather than persons, interact with the public sphere.'" (qtd. in Mattern 2018) I thought the installation was very successful in its similarity with how facial recognition may be used-- for social sorting.
Different from James Coupe, who is a veteran artist and professor, Matthew Chan, the artist who made Draw me like one of your French girls (Chop Suey) (2019) is a younger artist. I am aware of the differences in resources and experience between these two artists, but I chose Chan's work because of its contrast with Warriors (2020) . Chan uses Deepfake to replace Leonardo DiCaprio's face in an iconic scene from Titanic with his own. Rather than the transcription of the audio track, the subtitles are replaced with a chop suey recipe. Chan's work examines "the lack of cultural representation in film and its influence on people," a topic that is increasingly gathering people's attention. His face is not only a stand-in for his racial and gender identity but also himself, which coincides with one argument for increased diversity in movies -- diverse actors are needed so that younger people of certain ethnicity(s) can see "themselves" in that role.
Matthew Chan's website:
Mattern 2018: ... codespace/

Post Reply