Hyperfast Contextual Custom LLM with Agents, Multitokens, Explainable AI, and Distillation

Access full document, here.

New additions to my ground-breaking xLLM system include multi-token distillation when processing prompts, agents to meet user intent, more NLP, and a command prompt menu accepting both standard prompts and various actions. A Web API to test it, will be available by next week. The picture below shows the command prompt menu. For instance, the -p option allows you to fine-tune in real time. IDs are indexes referring to text entities in the corpus, used to help you get the source with full content, for any item returned in the search results.

I also added several illustrations, featuring xLLM in action with a full session and sample commands to fine-tune in real-time. All the code, input sources (anonymized corporate corpus from fortune 100 company), contextual backend tables including embeddings, are on GitHub.

My system has zero weight, no transformer, no training, and no neural network. It relies on explainable AI, does not require training, is fully reproducible, and fits in memory. Yet your prompts can retrieve relevant full text entities from the corpus with no latency — including URLs, categories, titles, email addresses, and so on — thanks to well-designed architecture. It does not use any Python library nor API call to external apps. This is a work in progress, more coming soon.

Read more, get the code, the anonymized corporate corpus used as input (fortune 100 client), the full paper (42 pages with code and demo) and everything for free, here. This sub-LLM is part of a multi-LLM system, governed by an LLM router. Everything is on GitHub, accessible from the paper in one click.

The backend tables (one set per sub-LLM) are stored as nested hashes, in memory. If instead you are interested in more traditional systems based for instance on vector databases, see the incredible performance offered by my partner, here.