Markus Leippold的动态

Transparency is key in sustainable finance! Discover #ParseStudio and unlock the full potential of your PDFs to bring clarity to your analysis. Markus Leippold, Imene K., Saeid A. Vaghefi and Chiara Colesanti Senni ??

Currently reading: “From Local to Global: A Graph RAG Approach to Query-Focused Summarization”. This insightful paper introduces a novel method that combines Graph-based knowledge extraction with Retrieval-Augmented Generation (RAG) to enhance complex question answering over large datasets. Some key highlights: 1. Graph Indexing: The paper outlines how the system constructs a graph of entities and relationships from text, enabling a structured understanding of data. 2. Community Summarization: The system groups related entities into “communities” and generates summaries at various levels (root, high, and intermediate). This approach helps answer global, multi-hop questions that require deeper insights. 3. Query-Focused Summarization: The graph summaries allow for more comprehensive and diverse answers, ideal for broad or complex queries. 4. Efficiency and Scalability: This method proves to be scalable for larger datasets and outperforms traditional text-based methods in providing relevant, efficient answers. If you’re interested, you can find it here: https://lnkd.in/dsPB63h9

Building Multimodal RAG Application #5: Multimodal Retrieval from Vector Stores ? Link: https://lnkd.in/dSz6SXZ9 Multimodal RAG combines textual and visual data to enrich the retrieval process, enhancing large language models’ ability to generate more contextually accurate and detailed responses by accessing multiple data types. This article, the fifth in an ongoing series on building Multimodal Retrieval-Augmented Generation (RAG) applications, dives into the essentials of setting up multimodal retrieval using vector stores. Starting with environment setup, this guide covers installing and configuring the LanceDB vector database, a robust solution for managing and querying multimodal data. Next, it demonstrates how to ingest both text and image data into LanceDB using LangChain, a popular framework for managing LLM workflows. The article concludes with a practical walkthrough of performing multimodal retrieval, enabling efficient searches across both text and image data, which can significantly enhance RAG applications by leveraging rich, diverse information sources. ? Do not miss the 60% discount on To Data & Beyond yearly subscription: https://lnkd.in/gmwFUqYr ? Check out To Data & Beyond's new ebook: LLM Roadmap from Beginner to Advanced Level: https://lnkd.in/dp7_CssN

#RAG approaches are constantly evolving. Let's talk about the secured & governed approach 'T-RAG'. Tree-RAG (T-RAG), uses a tree structure to represent entity hierarchies within the organization. This is used to generate a textual description to augment the context when responding to user queries pertaining to entities within the organization’s hierarchy. Evaluations show that this combination performs better than a simple RAG or finetuning implementation. Workflow of Tree-RAG (T-RAG): For a given user query, they search the vector database for relevant document chunks to be used as context. Additionally, if the query mentions entities from the organization, information about them is extracted from the entities tree and added to the context. They finetuned the Llama-2 7B model on an instruction dataset generated from the organization’s document. They use the finetuned model for response generation. Their system differs from the typical RAG application in the Query process. Instead of using an existing pre-trained LLM, they use a finetuned version of the LLM for answer generation; they finetuned the LLM model on an instruction dataset of questions and answers generated based on the organization’s document. A feature of T-RAG is the inclusion of an entities tree in addition to the vector database for context retrieval. The entities tree holds information about entities in the organization and their location within the hierarchy. Each node in this tree represents an entity with the parent node indicating the group it belongs to. During retrieval, they use the entities tree to further augment the context retrieved by the vector database. Know more in-depth about T-RAG in this paper: https://lnkd.in/gtDgVSPi ----------------------------------------------------------------- BTW, learn many GenAI concepts in my e-book 'GenAI for Everyone'. Download the 'GenAI for Everyone' E-Book: https://lnkd.in/gy7F3tvB

?? New in MEAP! ?? Knowledge Graph-Enhanced RAG, by Tomaz Bratanic and Oskar Hane. ?? Have you ever considered incorporating structured data into your #RAG applications? While most people focus solely on unstructured data as the primary source for RAG, you can also utilize #structureddata. Our latest MEAP release, "Knowledge Graph-Enhanced RAG," provides practical techniques to help you build RAG using both unstructured and structured data. This book is a practical guide to empowering #LLMs with RAG. Throughout the book, you will learn how and why storing data as a graph can enhance RAG accuracy, performance, and traceability. You will also learn to build a vector similarity search retrieval tool, create an Agentic RAG application, extract information from text to form a #KnowledgeGraph, evaluate performance and accuracy, and much more. ?? Take action now! Save 50% with code libratanic350. ?? Get into the book: https://mng.bz/4JNB ?? Check out this video to find out: https://mng.bz/v87x #GraphRAG #ManningBooks #LearnwithManning

?? ?????????????????????????? ?????????????????? ????????????????????????: ?? ????????????????'?? ?????????? : ??Have you ever flipped a coin and thought about the mathematics behind it? The Bernoulli Distribution is the simplest probability distribution that models scenarios with two possible outcomes—success or failure, yes or no, win or lose. ?? ?? ????????????????????: ??The Bernoulli distribution is a fundamental concept in probability and statistics. ?? It describes a single trial with only two possible outcomes: success (usually represented by 1) or failure (represented by 0). ?? ?????? ?????????????????????????????? 1?? Two Outcomes Only: Success (1) or Failure (0). 2?? Probability of Success: Represented p, where 0≤??≤1. 3?? Independent Trials: Each trial is independent of the others. 4?? Mean and Variance: Directly determined by the probability of success (p). ?? ????????-?????????? ?????????????? : ??Coin Toss: Flipping a coin where the outcome is either heads (success) or tails (failure). ?? ??Quality Control: Checking if a product passes (1) or fails (0) in an inspection. ?? ?? ?????? ?????????? ?????????? ????? ?? Understanding the Bernoulli distribution is essential because it forms the foundation of many other statistical concepts, like the Binomial Distribution. It's a must-know for anyone diving into probability, data science, or machine learning! ?? Thanks to my mentor Krish Naik ??Repost if you agree! ??Follow Rohini Singh for more! ??Tag someone who needs this message today. #Probability #BernoulliDistribution #DataScience #StatisticsMadeSimple #LearningJourney #MachineLearning #DataVisualization #LinkedIn #LinkedInlearning #ComputerScience

?? Delighted to unveil the result of months of dedication and innovation! Presenting our Graph-based Text Classification Project! ?? What's the project about? This endeavor transcends traditional vector-based techniques, delving into the realm of graph theory. Leveraging the NetworkX library, we constructed directed graphs from preprocessed documents, capturing crucial word sequence and relationships. Our ace in the hole? Maximal Common Subgraphs (MCS) for feature extraction, unveiling pivotal patterns for precise classification. ?? Key Takeaways: - Embracing graph-based methodologies opened up novel avenues for document analysis. - Overcoming methodological hurdles, such as navigating web scraping intricacies and optimizing algorithms, proved pivotal in refining our techniques. - Persistence paid dividends! Despite the inherent complexity of implementing MCS, our unwavering dedication led to the development of a robust classification model. ?? Why it's Significant: Our project not only pushes the boundaries of text classification but also emphasizes the importance of exploring unconventional methodologies to derive insightful conclusions from data. Achieving an 80% accuracy rate with just 15% training and 85% testing data surpasses the performance of vector-based approaches, which yielded only 59% accuracy with the same training data. ?? Acknowledgments: A heartfelt appreciation to our devoted team, whose relentless effort and Haider warrich collaboration made this project a reality! Special recognition goes to # for their invaluable contributions and to Waqas Ali for his guidance. #DataScience #TextClassification #GraphTheory #DataAnalytics #Teamwork

To RAG or Not to RAG? https://lnkd.in/guqjWNbq While Retrieval Augmented Generation (RAG) has proven to be a game-changer for many practical applications, it is not a cure-all, particularly when it comes to "reasoning-intensive" tasks such as coding and math. These tasks require a depth of reasoning and context that keyword-based search queries struggle to capture, making RAG less effective in these scenarios. Consider a simple example: given a document containing biographies of various book writers, a RAG system can efficiently retrieve specific information about a single author (or a few, normally 3-5 relevant chunks are pulled out) and answer to related questions. However, if you ask a more complex, summarization type of question, such as how many of those authors are from the US, RAG falls short. This type of query requires a full context analysis, which is significantly more expensive—approximately $1 per 1M tokens for processing a few thousand pages through a Large Language Model (LLM). So how to solve this problem? Take a shot!

Something I’ve found interesting in my Large Language Model (LLM) engineering journey is working out when (and when not) to use function calling - a simple but powerful technique that allows LLMs to perform tasks outside of text generation. Although this technique is fun to implement as it appears to give the model a kind of autonomy, in my experience so far I have had more success using a multi-step prompting, walk-around approach. My first prompt requires a response in JSON with the function my program needs to call along with the required inputs. My program performs the function and the output is used to generate the model’s final response for the user. In practice this is pretty much what function calling is doing anyway [https://lnkd.in/dAt2V7xM]. In my opinion this work-around technique gives more control over the actions taken by your program and can act as a guardrail for unexpected behaviour. My research into this seems to back this up. When looking at the Berkeley Function Calling Leaderboard [https://lnkd.in/dG3CnQPB] and keeping the model of interest constant, ‘walk-around’ prompting performs similarly and often better than the model's native support for function calling. I’m certain that there are scenarios where using native function calling would give better results. I am interested to hear which approach others have had more success with. #DataScience #Analytics #BusinessIntelligence #LargeLanguageModels

Deepseek R1 is an amazing paper packed with insights: 1. Zero: They train a model to start using only simple reward based GRPO - simple rewards: adherence to format, compiler-evals for code and final answer for math problems - this model suffers language mixing and understandability, but it great at reasoning like with AIME benchmark 2. Cold start: Get reasoning prompts to generate data from Zero and filter it using the ground truth and DeepSeek-v3 as judge - focus on reasoning, writing and role-play in one stage - factual answers, helpfulness and safety in another stage: how they collect this is based on DeepSeek-v3; needs more inspection from my end 3. Smaller models: Benefit from distillation and also would benefit from RL post-distillation but not explored Good things about the paper: 1. Increased CoT length vs multilingual-ness & control using summary 2. What all has been separated into different stages and why? 3. How to think about training data? Bad things about the paper: 1. Some benchmarks could have been better, but limited by access to OpenAI APIs 2. Do they mention which prompts were used for SFT data generation? ~ maybe again need to read the DeepSeek v3 paper 3. Do they mention how they distill to smaller models? Potential Ideas: 1. What are some other qualities of a good reward, other than adherence to instructions? 2. How can we generate longer chains for SFT data?