Using AI to Generate Profile Pictures

First and foremost, each artwork is new and unique. The project is in collaboration with the Artists' Estate, which includes the due transfer of Copyright for these works to the project (this is huge!). We start from a set of around 50 'Avatars' - which are art works that the artist created in the years 1948-1951.

The usage of the word 'Avatars' did not come from the artist or the artist's estate, but from us as curators. We took note of the NFT Avatar craze that happened in the 2020's and beyond - digital profile pictures that focused on a characteristic or a trait - our point being that such experiments have already been done and much earlier in fine art [1]. For the definition of Avatars please refer to the images below from the Gobardhan Ash Retrospective exhibition. Note that not all of the art works created by the artist are 'avatars' or 'profile pictures' - they can be landscapes while some others are hazy and we have poor images of them from older catalogs and therefore are not usable. We have had to rely on a limited set of images.

Irani Avatar

Note the caricature, the traits, the stereotyping - the chiseled nose and the chin and the classic (Parsi) hat. You know you are looking at a quintessential Parsi from Mumbai in the above image - and therefore an Avatar! [6]

What is an Avatar

THE ORIGINAL ARTWORKS - THE 'AVATAR' LIKE WORKS

Exhibition View

Avatar Exhibition View

The entire exhibition can be seen elsewhere [3,4].

Earlier Sales

The original paper on GAN's was published in 2014 by Goodfellow et al. The computing power was still not there. It was only around 2017 that we began to see some interesting experiments using GAN's.

It should be noted that Neural Networks or Convolutions are not new. The author himself having coded up convolutions in 1997 [1] and having attended a class on Neural Nets & Fuzzy Logic by Lofti Zadeh - where the avant-garde use-case was to identify when clothes in a washing machine were clean enough! The advent of GPU's changed everything as Neural Nets are inherently "parallelizable". 

AI Generated Nude Portrait #7 Frame #64 - Sold for 842,116 USD Sothebys London 2022 (Image Courtesy: Sothebys)

Portrait of Edmond de Belamy by Robbie Barrat

AI Generated Portrait of Edmond de Belamy - Sold for 432,500 USD Christies 2018 (Image Courtesy: Christies)

Generative Adversarial Networks

These have evolved! since 2018 and in fact, the AI tech is moving so fast that six months seems to be five years. GPU's are bursting exponentially, state of the art is continuously changing. Though the focus of a lot of these technological battles are specifically targeted at LLM's where there seems to be a desire to have as much data coupled with as much computing as possible. Most GAN's have been trained with close to 100,000 images possibly supervised. This also highlights a problem, what if we have fewer data points, and much fewer images than what is expected in a GAN. Style transfer does not really work for art - the images created are too far from the artist's own creation. Using FFHQ or Huggingfaces pickle files and then attempting a style transfer is not appropriate. As curators, we have a detailed knowledge/understanding of the artworks that we are training from. We have been able to "solve" the problem by repetitive augmentation, by standardization, with the knowledge of the compositional pieces, with the knowledge of what's relevant, the ethos of the artist, by breaking and joining, by mixing and matching the compositions. We feel this is a superior way of creating profile picture projects. We realise that this is a "categorization" approach to AI

To give you an idea of the complexity, please see below the FID graph which converges close to 10 before bouncing up never to seemingly fall back no matter how much we try. The FID score achieved is "not bad" given that the best achieved by GAN's being trained from the FFHQ database with much higher resolution images reaches close to an FID of 5 ; however, these are trained with 70,000 plus images. Note we (at Prinseps) are dealing with fine art, not very high-resolution, and in this case with a rich set of colours. We feel the results are quite encouraging. More importantly, as an experiment in modern art, we find the generated images to be both modernist and appropriate and certainly very different than the cartoonish images that proliferate the NFT / Ordinal space that certainly don't qualify as modernist. The AI engine is creating legitimate newer artworks trained in the style of the artist. 

Some of the images may be close (in the neural net), and ideally, we should use a measure to select images that are the furthest from each other. But this is not an easy problem. There is a concept of an Euclidian distance; however, that is suited for black and white pixels. Gobardhan was a master colorist - he was able to achieve texturing in a two dimensional space by using water-colour what would probably have been more easily done in oils. He was able to create depth that would generally only have been possible in an impasto art work executed in oils. We are enthused that the AI engine is able to learn and generate the richness of the colours and the depth in execution. This is certainly encouraging for future projects. The color palette in these works is high and they cannot be minimized to black and white for a euclidean distance calculation.  Given an initial image, it is somewhat of a reverse optimization to find a "far" image. This is still up for research.

It was not easy, we have failed on many attempts to produce the following images with exponentially high and ridiculous frechet scores.

GAN Failures

To finally achieve success

FID vs Ticks

The GAN-generated images display the latent walk. We focus on a feature - a specific background. And compare the difference just of the backgrounds and then the difference of the difference to show the "walk". 
Latent Feature Walk

Latent Feature Walk

Differences Latent Vector Feature Walk

The same "walk" is visible in the faces. We focus on a specific face type.

Latent Walk - Impressionistic Faces

Latent Walk - ImpressionismLatent Walk - Impressionsim

We believe this is cutting-edge. But more importantly legitimate modern art with newer art works created in the artist's style. Some of the artworks are below. We do hope you find them aesthetically appealing.

References

1 - Modernism - "The First of tthe Avatars"

2 -  "Analytical Models for Vehicle/Gap Distribution on Automated Highway Systems", H.-S.J. Tsao, R.W. Hall, and Indrajit Chatterjee, Transportation Science, Vol. 31, No. 1., pp. 18-33, 1997.

3 - Gobardhan Ash Retrospective Exhibition @ Kolkata Center For Creativity

4 - Interviews from the Retrospective Exhibition

5 - The Revolutionary Wall of Artificial Intelligence

6 - Decoding Gobardhan Ash's Avatars

p.s. The Italian Consul General in Kolkata who visited the exhibition had a great idea. As the images have been derived from art works with titles/roles/traits. Maybe the same information can be used to find an appropriate name for the image. In technical terms essentially means a Language Model add on to the GAN (easier said than done though) [5]. Stay tuned!

Any questions?