Getting Started with LLMs: A Quick Guide to Resources and Opportunities

Subscribe to my Medium and Substack Newsletter

Large language models (LLMs) and Generative AI are undoubtedly transforming our lives. As an ML practitioner, I've been frequently asked for advice on how to get started with LLMs for those with little to no prior NLP experience. The rapid growth of LLMs can be overwhelming, but their potential to change our lives is undeniable.

As technology rapidly evolves, the ethical challenges surrounding LLMs grow at a similar pace. It's crucial for users to grasp the technology's foundations and understand its limitations to maintain control, rather than being controlled by it.

?? Web3+AI

I would like to think of LLMs as "Democratized AI" because they rely on data generated by everyone. This data can come from various sources, such as social media posts, code snippets, product reviews, online Q&A communities, petitions, or even pet photos.

With the democratization of data, ML is shifting from being centralized (limited to engineers, scientists, and researchers) to decentralized, where everyone can utilize it—similar to the concept of Web3. We've entered an era where machine learning can be harnessed by individuals everywhere, leading to groundbreaking opportunities.

?? Recommended Resources:

While working on writing my tech book, which includes a chapter focusing on NLP, I've discovered many valuable resources for anyone looking to learn more about LLMs:

Overview

1. A Comprehensive Review of ChatGPT: This paper is an excellent starting point for anyone new to the LLM field (e.g., ChatGPT) or seeking a comprehensive review:

• History of ChatGPT and OpenAI
• GPT models (core technology, model history, versions)
• Applications (scientific writing, education, medical, etc.)
• Challenges (technical limitations, misuses, ethics, regulations)

2. Other survey papers:

• A Survey of Large Language Models: LLM overview.
• Transformer Models: An Introduction and Catalog: It gives a very comprehensive overview of transformer-related models. One thing I like the most about this paper is that it gives very good visualization graphs and tables illustrating how different LLMs compare.
• A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT. This paper discusses LLM and other generative AI models, such as diffusion models.

3. A very high-level introduction to NLP: A Complete Guide to Natural Language Processing

Practical

Interested in training or testing your own LLMs?

1. Hugging Face Model Hub: Access pre-trained LLMs and tools for fine-tuning. Hugging Face is a popular platform that offers pre-trained LLMs and tools to fine-tune models for specific tasks. Their Model Hub includes documentation and resources to help you get started.
2. GLUE Benchmark: A collection of natural language processing (NLP) tasks designed to evaluate the performance of models.
3. Building LLM Applications for Production by Chip Huyen
4. Which GPU(s) to Get for Deep Learning by Tim Dettmers

Deep Dive

1. Fundamental papers:

• "Attention is All You Need": This groundbreaking paper by Vaswani et al. introduces the Transformer architecture, which forms the basis for GPT models and other LLMs.
• The Illustrated Transformer: A visual guide by Jay Alammar, this resource helps you understand the Transformer architecture in depth.
• "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding": This paper by Devlin et al. introduces BERT, another influential LLM that laid the groundwork for many subsequent models. The BERT model set the new benchmark for almost all NLP tasks when the paper was published.
• "Language Models are Few-Shot Learners": This paper by Brown et al. presents GPT-3, a state-of-the-art LLM, and demonstrates its impressive few-shot learning capabilities.

2. Paper reading series by Mu Li (for Mandarin speakers).

Ride the Tide

1. Stay informed: AlphaSignal.ai --- Weekly summary of the top research papers, repos, and tweets identified by AI models.
2. Create your own startup: Consider joining Y Combinator.

?? AI Startups and Collaboration

This is an exciting era for AI startups, and we're eager to explore potential collaborations. If you're an LLM expert or ready to apply state-of-the-art Web3/NLP techniques for real-world impact, let's chat!

?? Share Your Resources

What other resources have you found helpful in the LLM field, or what topics would you like to know more about? Let me know and I'll add more to the list.

(Thanks for reading this far! Feel free to repost/share this article, with proper credit noting the author and source.)