How to reduce impact of AI on Global Warming?

We all know global warming is real and threatening our present and future.

But ever wondered how is AI impacting global warming?

How does AI impact global warming?

To build any effective AI solution, we need

a) tons of data

b) features/feature vectors from the tons of data

c) GPU/TPU processing power to crunch the features

d) Training and validation

e) Operationalization with champion and challenger models

As an example, as published by Gartner, the number of features has been exponentially increasing over a period of time. The most heavily used GPT 3 uses 175 billion!!

As a result the AI solutions are becoming power hungry.

Mathematically, an adult human brain can process 56 billion features and consumes 12 watts of energy. GPT3 with 175 billion consumes 3,000,000 watts of energy.

Despite most data centers / cloud stores turning to green energy, by 2030 AI alone will consume 3.5% of world's electricity.

What can we software professionals do to turn the tide?

Even blockchain solutions were scoffed upon by climatologists, but the community is steadily working towards it. Last week 'The Merge' happened successfully.

The Merge refers to the original Ethereum Mainnet merging with a separate proof-of-stake blockchain called the Beacon Chain, now existing as one chain. The Merge reduced Ethereum's energy consumption by ~99.95%.

It is phenomenally inspiring for AI practitioners in my opinion to take the cue.

Let me make some double-click suggestions:

• Abdicate accuracy for MVP

None of the AI solutions proclaim 100% accuracy, anyways it will get categorized as overfitting. So why to keep on training to achieve the non-existent Holy Grail?

The story is similar for validation, which 'typically' follows as a waterfall after training. Can validation be interspersed with training to achieve the speed and right accuracy?

• Incremental Learning

Classical batch machine learning needs all data to be simultaneously accessed but seldom meets the requirements to handle the large volume, leading to accumulation of unprocessed data. This method also misses out on integrating new information into already constructed models as it regularly reconstructs new models from scratch. This is not only very time consuming but also leads to potentially outdated models.

We can change this to sequential data processing in a streaming manner. This does not only allow us to use information as soon as it is available leading to all-time up to date models but also reduces the costs for data storage and maintenance. It is almost like training and validation are done in an agile way.

Incremental learning does have few challenges like model is constructed without complete retraining and only a limited number of p training examples are maintained

• Active Learning

This learning is based on the hypothesis that learning algorithm has the liberty to choose data to learn from. Rather than exposing the whole ton of data and do a supervised/unsupervised learning, through 'relevant' queries the focused region of interest in the subset of data is established.

• Transfer Learning

This learning I had heavily used in past. With the ubiquity of pre-trained models (esp CNN models from ResNet, or available open source) we can plug pre-existing pre-trained model into our data store.

• Federated Learning

Next level of crowdsourcing is utilized in Federated Learning. It requires each 'client/developer' to train the model locally on their device/server. The data used to train the model does not leave the device/server.

The models(weights, biases etc) are then sent to central server, which averages out the model parameters to create a new master model. This is iterative in nature to achieve desired accuracy levels.

This is super useful in case of privacy requirements of data.

• Neural Architecture Search

Neural Architecture Search is a classical concept of meta learning. It unleashes the ability of AI to make other AI models better.

It essentially takes the process of a human manually tweaking a neural network and learning what works well, and automates this task to discover more complex architectures.

• Small and Wide data

Gartner predicts that emerging “small and wide data” approaches will enable more robust analytics and AI, reducing organizations’ dependency on big data. Wide data allows analysts to examine and combine a variety of small and large, unstructured and structured data, while small data is focused on applying analytical techniques that look for useful information within small, individual sets of data.

Conclusion

I believe the whole topic is a part of Green AI, which lies under the umbrella of Responsible AI along with Ethical AI and Compliance.

Practitioners should go beyond ‘conventional’ (deep) machine learning to make AI more energy efficient. Energy usage should be a key AI metric to look out for.