The war for large language models
Last week, the buzz was around large language models. A day after?DeepMind came out with Gopher, a 280 billion parameter transformer language model, Google introduced the Generalist Language Model (GLaM)?– a trillion weight model that uses sparsity.
The?full version of GLaM has 1.2T total parameters?across 64 experts per mixture of experts (MoE) layer with 32 MoE layers in total but only activates a subnetwork of 97B (8% of 1.2T) parameters per token prediction during inference.
As if this was not enough, in another innovation in the large language model generation space,?LG AI Research has revealed its new artificial intelligence language model “Exaone”, with capabilities of tuning 300 billion different parameters or variables.
1
Andrej Karpathy Believes AI Models Will Consolidate
Andrej Karpathy, recently wrote a very compelling Twitter thread, making an?argument about the consolidation of AI model architecture. In the last few years, the neural network architecture across areas of applications have begun to look similar.
An example is Transformers – traditionally linked to building language models, their scope is more far-reaching. Transformers were introduced for processing language models – leading to the launch of models like BERT, XLNet, and GPT. However, transformers can be used for more diverse applications.
2
Our Discussion with Elad Ziklik Of Oracle
Recently, for an episode of Simulated Reality, a video podcast series by Analytics India Magazine, we had?Elad Ziklik, VP, Product Management for AI Services and Data Science at Oracle, in the house.
During the session, Elad delved deep into some critical aspects of artificial intelligence and how organisations can leverage its full potential, the challenges involved, and Oracle’s key AI capabilities and strategies.
3
Goa Institute of Management's PGDM in Big Data Analytics.
The?two-years residential Postgraduate Diploma in Management programme?has an intake capacity of 120 students. The main intention of the programme is to create future-ready and data-fluent professionals equipped with capabilities for data-driven decision-making.
The programme structure is divided into a 40 to 60 ratio of business knowledge and Big Data Analytics related experience.
4
Council Posts
AIM Leaders Council is an invitation-only forum of senior executives in the Data Science and Analytics industry. To check if you are eligible for a membership, please fill the form?here.
+?Data Engineering Advancements By 2025?- Anish Agarwal
+?Envisioning The Future of Deepfakes?- Aishwarya Srinivasan
+?Building Multilingual Chatbots In India—Overcoming Cultural Challenges?- Ankush Sabharwal
5
Featured Video:?Xenobots | The Living Robots - That Reproduce
领英推荐
6
Hands-on Guides for ML Developers
7
Data Science Hiring Process At Khatabook
At?Khatabook, the analytics and data science team consists of 30 members. Analytics and data science is a centralised function in Khatabook. For instance, Khatabook has five products, and each product has a data science leader and a team of 4-5 data analysts/scientists.
8
The Personal Data Protection Bill
The?Indian Personal Data Protection Bill?is expected to be taken up for a much-awaited discussion in the winter session of the Parliament, after almost two years of deliberations. But a few of the provisions have come under great scrutiny by dissenters and opposition parties across the country, leaving the future of the PDP bill a question mark.
9
Player Of Games
After back to back innovations in the gaming space, DeepMind has taken a step further and created a system called?Player of Games (PoG), whose structure and mechanism it has released in a research paper.
What makes Player of Games stand out is that it can perform well at both perfect and imperfect information games.
10
BOTTOM OF THE NEWS
Here's what all happened last week.