Pratik Ratadiya

Many of ML practitioners might have heard the term "Extending context length" for LLMs. In case if you want to dive deeper into this topic along with its code implementation, give this blog a read. I have described about Position Interpolation, NTK aware and Dynamic NTK method, which are widely used for extending context length.

68

What if your queries required access to multiple sub-related data from different documents, for an LLM to parse and make sense of before returning a coherent and accurate output? We often ask complicated questions, right? 🔹Well to solve this problem, these researchers released Multi-Head RAG (MRAG) which leverages activations of Transformer’s multi-head attention layer, instead of the decoder layer, as keys for fetching multi-aspect documents. They are taking advantage of the fact that multi-head attention can learn to capture different data aspects. 🔹The interesting part is that MRAG is both cost-effective and energy-efficient. In other words, it does not require additional LLM queries, multiple model instances, increased storage, or multiple inference passes over the embedding model. 🔹Moreover, MRAG can be seamlessly integrated with existing RAG frameworks and benchmarking tools like RAGAS as well as different classes of data stores. 🔹The researchers provide a comprehensive evaluation methodology, including specific metrics, synthetic datasets, and real world use cases to demonstrate MRAG’s effectiveness. 🔸Do check it out! #genai #technology #artificialintelligence Click #linkedangle and follow for more content Set the 🔔 notification on my page, don’t miss a post!

181

10 条评论

Ran a quick test on the new #OpenAI #o1mini to solve IIT-JEE Advanced 2024 maths questions. It accurately solved 13 out of 17 questions with an impressive accuracy of ~76%. Quick thoughts: 1. The hidden chain of thought is quite detailed and extensive. Breaks the problem, solves and verifies and iterates multiple times. This might increase the output token cost by a big margin. 2. Seems there are tools integrated for calculations. Much needed as #gpt4o does a bad job at it. 3. Latency is quite high. Some questions took more than 50 seconds to get answered. This needs to come down for production user-facing use cases. 4. Hallucination still exists. Check the 1st question in the link, it had the correct step-by-step reasoning but hallucinated at generating the final answer. 5. Overdoing self-thought. Saw a couple of instances where it did generate correct answers in the process but eventually made wrong answers. Would give a clearer picture once the rate limits are increased and benchmarking is done on a bigger sample. Let me know any interesting findings and thoughts 😀 https://lnkd.in/giSCezyG #GenAI #OpenAI #PhysicsWallah

139

5 条评论

Day 5 of learning in public 🚀 Traditional fine-tuning methods for large language models often face challenges like high computational requirements, catastrophic forgetting, and single-task specialization. Enter LoRA (Low-Rank Adaptation), a game-changing technique that addresses these limitations, enabling efficient and practical fine-tuning of large language models. LoRA introduces a novel approach to fine-tuning by adding a small number of task-specific weights to the pre-trained model instead of updating all the weights. Here's how it works: 1. Low-Rank Matrices: For each layer in the pre-trained model, LoRA introduces a pair of low-rank matrices: one for the weights and one for the biases. 2. Adaptation: During fine-tuning, LoRA updates these low-rank matrices instead of modifying the pre-trained weights directly. 3. Composition: At inference time, the low-rank matrices are composed with the pre-trained weights to produce the final, adapted weights for the fine-tuned model. The low-rank nature of these matrices is the key to LoRA's efficiency, dramatically reducing the memory and computational requirements for fine-tuning. This makes it possible to adapt large language models even on consumer-grade hardware like CPUs or edge devices. But LoRA's advantages don't stop there. It can match or even outperform the performance of full fine-tuning while using far fewer updatable parameters. Additionally, the LoRA updates are reversible, allowing the model to revert to its original state, and composable, enabling a single pre-trained model to be fine-tuned for multiple tasks simultaneously. To illustrate the power of LoRA, let's dive into a practical example. Here, I'll demonstrate how to implement LoRA from scratch and fine-tune the DistilBERT model using LoRA on the IMDB dataset for sentiment analysis. https://lnkd.in/g-Tz2NU7 As you can see, LoRA is a remarkable technique that addresses the limitations of traditional fine-tuning methods, making it more efficient, practical, and versatile. By introducing low-rank, task-specific updates, LoRA enables efficient adaptation while preserving the model's original knowledge and allowing for multi-task specialization. As the demand for deploying large language models on resource-constrained devices continues to grow, techniques like LoRA will make these powerful models more accessible and practical.

4

Brain Teaser #10: Suppose we are fine-tuning a large language model using Low-Rank Adaptation (LoRA). We possess two low-rank update matrices, δWA​ and δWB​​, both associated with the rank "r". Considering our objective to preserve the model's current knowledge while incorporating new data from diverse features, what "mathematical rationale" should guide the selection of one of the update matrices (δWA​ or δWB) for low-rank adaptation? Hint: Consider the specific characteristics or properties of the matrices that can help in developing mathematical reasoning. Here is my response: The norm ratio, calculated as the ratio of the nuclear norm to the Frobenius norm of the update matrix, is an essential metric for assessing how updates impact a model. It provides insights into the model's stability and adaptability. Here’s a brief overview: https://lnkd.in/ekB_q9XX #MachineLearning #DeepLearning #NaturalLanguageProcessing #LoRA #FineTuning #AI #DataScience #ModelOptimization #NeuralNetworks #ArtificialIntelligence #TechInnovation #DataDriven #KnowledgeIntegration #Research #LLMs

30

4 条评论

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

Day 13 of #30DaysOfAI 🚀 Getting structured data from Large Language Models (LLMs) has always been a challenge. While LLMs typically return markdown content by default, asking for structured formats like JSON often leads to errors or inconsistencies. That’s why OpenAI’s Structured Outputs is a game-changer! 🎉 With Structured Outputs, you can provide a JSON Schema to define exactly how you want the data to be structured. This ensures the LLM consistently generates the right format every time, making it far more reliable for applications that depend on structured data. Here’s an example: Imagine you’re building an invoice management application. The app scans an invoice, extracts relevant information, and then returns it as a structured JSON format. With Structured Outputs, you can precisely define the type of data to extract—ensuring accuracy and making it easy to store or categorize invoices. Want to learn more? Check out this guide on Structured Outputs by Shreya Khandelwal to see how you can start using this feature effectively. How would you use Structured Outputs in your projects? Let’s discuss this in the comments!

1

Here’s the #1 mistake EB-1A aspirants in tech make. ⚠️ They go outside-in versus inside-out. What does that even mean? It means they look externally for how other people built EB-1A profiles as a guide to how they should build their profile. I get it - if you don’t know where to start, makes sense to do what other successful applicants do, right? Wrong. This approach is breeding a generation of EB-1A’s that all look the same. 🐑 The whole point of the EB-1A process is to show why YOU are UNIQUELY recognized as among the best in your field. 💯 So how does it help to get all the same judging, awards, and media that everyone else is getting? ✅ You are supposed to stand out from the crowd, and show the impact of the work YOU are doing within the EB-1A framework. Plus, in tech, everyone has a different story to tell. How does it help to use someone else’s approach to tell your story? These EB-1A’s come across as fake - and you know who hates fakers more than anyone else? 👮‍♂️ USCIS. And when they see these applications they will profile you as a faker and completely disregard the parts that are deserving. We know because we (and many of our lawyer colleagues) are now being asked to help scores of tech workers who got their EB-1A’s denied after filling up their profiles with unimpressive awards and marketing exposes masquerading as media articles in 3rd rate publications. Question: since when do awesome tech employees ever get recognition of their expertise in the press? They don’t. Their employers do. So it looks weird that a generation of employees are now suddenly getting press coverage. ✅ Instead of an outside-in approach, go inside-out. First identify your unique impact proposition. What is your narrative? How are you uniquely amazing? Because if you are a senior employee at an innovative company, chances are you doing some incredible, impactful EB-1A-worthy work! Then, build your case from there in a way that honors that narrative. This is the formula we use in our course, and then in our law firm, that leads to a ~99% approval rate. Don’t be tempted to fake it! #immigration #eb1a #globalmobility -------- Do you work in high-tech, are stuck in long green card backlogs, and are looking to level up to an EB-1A? Our next attorney-led live EB-1A profile-building cohort begins on June 8. Apply now to secure your spot! The link is in the comments below.

55

6 条评论

Not following textbooks has helped me write seemingly novel proofs. Whenever a text introduces me to a theorem, I write down its statement and close the text. I then reason from the fundamentals over several hours or days. Outcome: I have proved 100+ theorems from the K-10 curriculum and my proofs for about 10% of them turned out to be different from those in the textbook. If you want to develop as a mathematical thinker, try not to follow proofs. Spend time thinking about the problem or learning the fundamentals if needed. The goal is understanding, not completion. Real math is not a timed exam. It involves experiments, failure, reading, discussions, and more experiments. #mathematics #learning #doing

43

12 条评论

India is not just catching up in the AI race; we're innovating and leading in ways that are uniquely ours. The Economic Times recently highlighted how our approach to Generative AI goes beyond just Large Language Models (LLMs), addressing the specific challenges and opportunities within our diverse market. Rahul Agarwalla, Managing Partner at SenseAI Ventures, shared his invaluable insights into this AI revolution. His leadership and vision are truly driving innovation in the field, and it's an honour to be part of this journey with him. For anyone interested in the developments in AI and how India is making its mark on the global stage, this is a must-read.

46

3 条评论

The possible downgrading risks for NVIDIA (or any similar high-growth stock) typically stem from a mix of internal and external factors. Here are the key risks: 1. Valuation Concerns High P/E Ratio: NVIDIA's P/E ratio is significantly above industry averages, raising concerns that the stock is overvalued. DCF Valuation Gap: If actual growth doesn’t meet expectations, analysts might downgrade their price targets. 2. Macroeconomic Factors Interest Rate Hikes: Rising interest rates can hurt growth stocks as future cash flows are discounted more heavily. Recession Risks: Economic slowdowns could reduce demand for NVIDIA’s products in gaming, AI, and data centers. 3. Industry-Specific Risks Semiconductor Cycles: The semiconductor industry is highly cyclical, and any downturn could lead to reduced revenue. Competition: Intense competition from AMD, Intel, and emerging players in AI chip markets could erode market share. Supply Chain Challenges: Global supply chain issues, including chip shortages, could hurt production and profitability. 4. Regulatory Risks Geopolitical Tensions: U.S.-China relations could impact NVIDIA’s revenue from China, a significant market for its products. Government Regulations: Stricter export controls or antitrust regulations might restrict growth opportunities. 5. Technological Risks Innovation Pressure: Failure to stay ahead in AI, GPUs, or other high-tech sectors could lead to downgrades. Customer Concentration: Heavy reliance on a few large customers (e.g., cloud service providers) exposes NVIDIA to risk if these relationships weaken. 6. Execution Risks Overestimated AI Growth: While AI is a strong growth driver, over-optimistic forecasts could lead to disappointment. Mergers and Acquisitions: Failed or costly acquisitions (e.g., ARM deal termination) can impact investor sentiment. 7. Market Sentiment Profit-Taking: If NVIDIA’s stock has experienced a significant rally, any bad news could trigger profit-taking. Short-Term Earnings Miss: Missing quarterly earnings expectations can lead to immediate downgrades. Mitigation for Investors Diversify portfolios to balance sector risks. Stay updated on earnings reports and industry trends. Monitor macroeconomic indicators and geopolitical developments. Considering these risks, NVIDIA's long-term potential remains strong, but investors should prepare for possible downgrades during periods of market volatility or sector corrections. NVIDIA

2

1 条评论

Going from PoC to Production for LLMs? 🤔 In the age of GPTs, chat interfaces are ubiquitous. But what if we add a voice/conversational layer on top? Imagine a C-suite member checking sales on the go or a new employee understanding leave policies through a natural voice interface. Intriguing, right? This got me thinking: how do we move from a PoC running on a local machine to a full-fledged production application? Here's what I found out: 1️⃣ Data Management and Knowledge Distillation: Handle and version your knowledge base (PDFs, Images, Excel, ER Diagrams). Keep data up-to-date and manage new information efficiently. How do you ensure information in the data is not outdated? What’s the best way to encode and chunk this data? The questions arise! 2️⃣ Access control and Citations: How do you ensure people cannot access data they are not supposed to have access to? How do you retrieve specific documents to ensure better answers (LLM models work the best when provided with specific information) and lower bills (Each LLM service charges by the number of tokens)? How do you remove any Personally Identifiable Information (PII)? What kind of caching strategies to use? (Pareto principle, 80% of user queries would be for 20% of the questions that get repeated) 3️⃣ Security and Reproducibility: Protect against bad actors and prompt injection. Create secure networks. How do you safeguard your application? It’s a growing field with evolving challenges. 4️⃣ Heart of the Operations (Sort of, I still think your knowledge base is the heart): Now that you have designed and taken care of the above aspects, there is still more. What size of the model to use? What environment to host it in? how to scale and replicate? (If you have a fine-tuned model). 5️⃣ Generating responses: How do you ensure the results generated are accurate and correct? While it is an active area of research, the only effective solution at this point is humans in the loop. Productionizing LLMs is complex, with more questions than answers. I’d love to hear your experiences and insights. What challenges have you faced, and what solutions have you found? If you can think of important things that I may have missed out, feel free to comment below! Ps. I will be writing and building more on this idea in the coming days, stay tuned for more :) #LLM #AI #MachineLearning #DataScience #VoiceInterface

12

1 条评论

https://lnkd.in/esZgE24d Seven years ago I gave this talk at IIT Kanpur. Back then it seemed inevitable that we would have self-driving cars by now, and some predictions I made were wrong. Overall, though, I think this talk has aged reasonably well. Little did I know back then that the reference to the 2016 MaxSAT I gave would spark the creation of a new company, InsideOpt. The Seeker Solver that we sell is based on the technology outlined, and it is changing the optimization landscape. On the social implications, we since edited a special issue of the AI Magazine on the social disruption that AI brings. More work is needed. We need to have a wide societal discourse on how we are going to share the increasing wealth that AI brings to the world. #insideopt.com

8

Nice to have 'Top New Release' status on Amazon for the latest book (my 25th) that explores an alternative paradigm for quantum computation. Illustrations (111) by Dr. Narendra Madhav Joshi. My take is that we owe it to ourselves at this early stage of the industry's lifecycle to explore alternative approaches. Here is a brief overview: "This book attempts to reveal something of the vastly different and fundamentally creative quality of computing that must accompany any computation involving the quantum-levels. To linearly project digital computing laurels manifest as increasing speeds and the ability to process vaster amounts of information, as the inevitable trajectory of quantum computing is perhaps, in the aphorism attributed to the Buddha, to look only at the finger and to miss the moon and the sky that it is pointing to. The cover figure highlights all that is being missed, summarized as an egg-like structure synonymous with the term ‘Hiranyagarbha’ in Sanskrit. This depicts the womb of creation abundant with many layers and patterns reflecting luminescence, due to the constant superposition and entanglement inherent to the quantum computation that derives from a light-centered interpretation of quantum dynamics." The book is available here: https://lnkd.in/gXu4H9uW #quantumcomputation #quantumtech #technologyleadership #cosmology #systemsthinking

16

5 条评论

LIQUIDMIND®.ai द्रवमनः कृत्रिमबुद्धिः (https://liquidmind.ai) is looking for rockstar Engineering interns to join our 0->1 startup. The ideal candidate: Folks pursuing their final year or pre-final year BTech (AI/ML/Computer and information sciences/Electronics/Electrical Engineering or other affiliated engineering disciplines) Which cohort will I be applying for? Our October 2024 - January 2025 winter cohort. How can I apply? Please apply today if you think you are a good fit for this role to Meet Bhatt मीत भट्ट or DM me. You can also email a copy of your profile to support@liquidmind.ai When is the last date to submit my application? 7 October 2024 Who are we? We are a fast growth AI first Platform that caters to needs of MSME's in making their payments journey friction free across B2B segment) and also cater to Manufacturing exporters/traders in Trade Finance making their export journey friction free. Our LIQUIDMIND®.ai द्रवमनः कृत्रिमबुद्धिः (https://liquidmind.ai) team: We have a very talented team of product engineers a/c front end/backend/full stack/ML devs + product managers, designers and sales/marketing folks already working with us from A+ institutes a/c Bharat. What technologies do engineers at LIQUIDMIND®.ai द्रवमनः कृत्रिमबुद्धिः (https://liquidmind.ai)use? Our engineers use/deploy the latest of technologies across Cloud computing, AI/ML models and front end / backend technologies. We believe from engineering our products from the ground up with what is right for our target segment and the best technologies we can deploy easily.

10

LIQUIDMIND®.ai द्रवमनः कृत्रिमबुद्धिः (https://liquidmind.ai) is looking for rockstar Engineering interns to join our 0->1 startup. The ideal candidate: Folks pursuing their final year or pre-final year BTech (AI/ML/Computer and information sciences/Electronics/Electrical Engineering or other affiliated engineering disciplines) Which cohort will I be applying for? Our Dec 2024 - February 2025 winter cohort. How can I apply? Please apply today if you think you are a good fit for this role to Meet Bhatt मीत भट्ट or DM me. You can also email a copy of your profile to support@liquidmind.ai When is the last date to submit my application? 31 October 2024 Who are we? We are a fast growth AI first Platform that caters to needs of MSME's in making their payments journey friction free across B2B segment) and also cater to Manufacturing exporters/traders in Trade Finance making their export journey friction free. Our LIQUIDMIND®.ai द्रवमनः कृत्रिमबुद्धिः (https://liquidmind.ai) team: We have a very talented team of product engineers a/c front end/backend/full stack/ML devs + product managers, designers and sales/marketing folks already working with us from A+ institutes a/c Bharat. What technologies do engineers at LIQUIDMIND®.ai द्रवमनः कृत्रिमबुद्धिः (https://liquidmind.ai) use? Our engineers use/deploy the latest of technologies across Cloud computing, AI/ML models and front end / backend technologies. We believe from engineering our products from the ground up with what is right for our target segment and the best technologies we can deploy easily.

2

1 条评论

Here's why prompt engineering is one of today's most valuable skills. VentureBeat recently published an article by Deven Panchal of AT&T Labs. I applaud Deven for his vision. He says, "A well-crafted prompt can significantly impact the quality and relevance of the output of LLMs." So many are refusing to accept this. Mainly because prompting uses natural language. They are misguided, assuming AI can expertly prompt for them. Even if it can someday (which I doubt), it can’t today. The closest thing is SmartGPT 2.0, which still can’t compete with an expert prompt engineer. I recommend learning and practicing prompt engineering. 🟢 If you don't know where to learn, comment "START" and I'll point you in the right direction. --------------------------------------------- See comments for a link to the full article. 🔗 Follow 🔔Mark Fulton for more on #AI #Prompts #AIAgents

27

18 条评论

Another major use case of batch LLM inference that people are over looking is asynchronous evals (LLM as a judge/evaluation models). Tons of companies we’ve seen are already starting to spend massive compute allocation on LLM as a judge systems and they will only increase as the need for trust and production volumes increase. We’ve heard anywhere from 5-40% of total inference spend (and growing). Such a large spend on just evals is probably unsustainable, so it will be increasingly important for a provider to offer cheap inference to service and win such batch workload tasks. My hypothesis: the emergence of a specialized inference platform just for LLM as a judge, with inference products specifically built around these workloads (or as an add on for existing inference providers) Lmk what you all think, happy Tuesday everyone 😁

21