Spring AI and KeywordMetadataEnricher killer

Andre Kapp Andre Kapp

Andre Kapp

Senior Java Back-End Software Engineering Specialist

发布日期: 2024年10月22日
+ 关注

A Big lesson learned when uploading big PDF documents to a local RAG solution. Running on an old Intel-based MacBook Pro I had the following piece of code to extract keywords: 

          var enricher = new KeywordMetadataEnricher(chatModel, keywordsCount);
          List<Document> enrichedDocs = enricher.apply(documents);

This file upload process splits the document into chunks. The bigger the PDF, the more chunks you have. Enriching this list of documents, streams through the list and makes a call to the LLM to extract keywords. These are added to the document.

This process is a CPU killer! Processing 400 chunks is a night's work, and you need some serious cooling under the Mac!

Still trying to figure out how to get this to behave a bit better!

要查看或添加评论，请登录

Andre Kapp的更多文章

  • Spring AI and custom JSON data
    2024年10月19日

    Spring AI and custom JSON data

    Most of the Spring AI examples use "Dad Jokes" to demonstrate the capabilities of integrating with LLMs. I have nothing…

社区洞察

其他会员也浏览了