Learning to Capture the Concept - Where do I come from? Where do I go?

An AI-generated installation in the form of a waterfall overflowing with an abundance of garbage. Created from datasets generated by the performers documenting the equation between them and their garbage, this work hopes to encourage thought on consumption and the garbage amassed on a daily basis.

Artists: Malavika PC, Papia Chakraborty , Asli Dinc

Supported By: Pritha Kundu

Our initial idea was to have three parts to our installation:?

Part 1: A great garbage waterfall?

Part 2: Performative gestures of discarding garbage, and?

Part 3: Textual gameplay with the AI we trained to talk to the audience regarding garbage and its disposal.?

All of this eventually combined into a motion-censored interaction with an audience that could turn a potentially overwhelming massive garbage waterfall into a clear and clean flowing water body. During our mentor sessions, it was brought to our attention that our project had too many ideas and that it could board well for us to find the core of our idea, stick with it, and grow it out.

Through the references given to us during our mentor session by Jake Elwes and Madhu Nataraj we discovered that the process of documentation itself could come from the ethics and sentiment of a performer’s core of paying attention to the material, time and body they deal with – in our case, garbage and AI. Here are some of the key artists and references suggested by our mentors that left a strong impact on us:

Trevor Paglen

• ImageNet Roulette: This project gave us an insight into wondering about the kinds of “schooling” that an AI model needs, and what kinds of datasets it can comprehend. The machine is a scientific invention, thus its classifications and categorizations are ipso facto scientific — can we stop there, or must one acknowledge that we project our ingrained human training onto the machine, which has little control over what it is fed?
• From ‘Apple’ to ‘Anomaly’: About 30,000 individually printed photographs made up the work, which was meant to be a form of extended homage to Magritte’s ‘The Treachery of Images’ for the era of machine learning.?Taking a close look at a widely used dataset for training AI —?ImageNet —?it displayed the hazardous links between images and labels, provoking us to rethink how we create meaning through our datasets.

Kate Crawford and Vladan Joler

• Anatomy of an AI: ?Artificial intelligence (AI) may seem far away and abstract, but it already permeates every aspect of our daily life. ‘Anatomy of an AI’ carefully compiles and condenses this enormous volume of information into a detailed high-resolution graphic by analysing the massive networks that support the “birth, life, and death” of a single Amazon Echo smart speaker. This data visualisation helped us understand the enormous amount of resources that go into the creation, distribution, and disposal of the speaker, breaking down the otherwise strange concept of AI into something we are more familiar with.

Tehching Hsieh

• One Year Performance: The works of this American-Vietnamese performer were brought to our attention in order to introduce us to the depth of performance analysis and the importance of performance-related documentation. His works are not particularly AI-related. On the other hand, it has a lot to do with the decision to dedicate oneself to timed documentation of the experiments that the artist decided to undergo just as an artistic exercise. Our mentor, Jake Elwes, introduced us to the world of Hsieh by using the specific example of ‘One Year Performance’ (1980-1981), in which he set up shop in one location, punched a clock every hour, and took a photo of himself on each punch.

Gaining insights from these artworks, we discussed and decided that it was the question of the garbage that concerned us most, so we unanimously decided to set aside the performative gestures and gameplay aspects of our proposal aside. Thus began our deep dive into documenting our garbage.?

Our Learnings

It has been an interesting journey for us so far on quite a few levels. First and foremost our biggest challenge has been to keep in touch and coordinate our meetings between the four of us across the globe. Initially, given each coordinator’s life and the great fear of dealing with a project that we playfully pitched for that we needed to execute now, we spent a better part of our initial learnings showing up on calls when we said we would. Even though it seems simplistic to just show up, it was harder than we expected it to be. We are relieved to say we got better at organising ourselves come the second month of the project. At this time we have hired a photographer, Ankit Banerjee, to train the three of us to photograph the kind of images we need for the AI to train so that all our images look cohesive under one dataset.?

Most of the pictures focused on beautifully composing the garbage and taking graphically interesting pictures of it. Given Malavika’s artistic interest, she was making the documentation more aesthetic than it was meant to be. Asli then pointed out that shadows confuse the AI!

This too might seem simple to an AI expert but to Malavika and Papia it came as quite a learning curve. Sometimes we imagine Asli Dinc nodding her head wondering what could do with our novice skills; however, with patience and some humour, we pull through.

Learning to Document

At the start, we shot the images in our own style. It took us some time to narrow down the type of image we wanted, and more importantly, to understand and acknowledge why an image needs to be a certain way and what that means for training the AI.?

For example, initially, when Malavika set up a sample DIY photo booth and went through meticulously recording about 3 weeks’ worth of her garbage, Asli Dinc sweetly but surely shot it all down. Malavika being a visual artist got excited with the shadows that the garbage was casting on the floor and the backdrop. Most of the pictures focused on beautifully composing the garbage and taking graphically interesting pictures of it. Given her artistic interest, she was making the documentation more aesthetic than it was meant to be. Asli then pointed out that shadows confuse the AI! So, this meant we needed to shoot the garbage item with full exposure so that the object can be separated easily.

Papia is new to the camera. With good advice from Ankit, Papia purchased a second-hand Lumix Panasonic digital camera for her process documentation. He then gave her exercises and she set out to start her documentation from the outdoors. She learned how to control the settings in her camera, how to manage light, how to compose the garbage and how to avoid shadows of herself falling on the garbage she was shooting. Upon seeing this set as well, Asli gave us more pointers. She told us to avoid shiny surfaces as the shine confuses the AI too. She also advised us that this was the chance for us to shoot mass garbage. We were also each shooting our images in different ratios of rectangles. One can only imagine the many colours that Asli was turning upon looking at our multiple amateur attempts in the beginning.?

Every few days we took pictures of our personal garbage, which we saved item by item. The preset setting of the camera and the booth allows for all the images from Papia, Asli and Malavika to be similar and makes it appear as if it comes from one voice.?

By this point we had learnt that shooting at 2040 pixels x 2040 pixels, at 300 dpi, RGB, and in .jpeg format was ideal for our dataset. We were on the path to creating two types of datasets:?

1. Single Images
2. This is the set that is shot indoors in an infinity booth with a mutually agreed upon preset light and camera setting (as advised by Ankit and Asli Dinc). Every few days we took pictures of our personal garbage — which we saved item by item. The preset setting of the camera and the booth allows for all the images from Papia, Asli and Malavika to be similar and makes it appear as if it comes from one voice.?
3. Combination Images
4. This is purely an outdoor exercise, where we take images of the garbage we find around us. This allowed us to study our surroundings and understand the states of garbage from 3 different places in our world.

We decided to each deliver a minimum of 1000 images each towards our final dataset using specific settings on our camera to garner the best image to train our AI model. Our camera settings were 25 mm lens, F-5.6 aperture, 1/80th shutter speed, 250-400 ISO depending on the camera. Depending on the kind of training needed for the AI, environment and required outcome, we learnt that the settings are subject to change.?Following is how we expected our final images to look. Yes, we finally nailed it!

To continue reading, please go to the link