Nous Research

It's always rewarding to see our work at Nous encourage people to build AI more openly! Ankit Pal at Open Life Science AI mentioned he was inspired by us to create the Medical Reasoning Tasks List, helping to advance AI in healthcare across diverse tasks: https://lnkd.in/gf-vhUt7 When we share our efforts in public forums, we often don't realize the positive impact they have across various industries. Freer information flow, fact-checking and democratic innovation propels our society forward. As Louis Pasteur said, "...knowledge belongs to humanity, and is the torch which illuminates the world."

Could the next big open source model?be built by a global, distributed network of independent builders? We think it will be and are laying the groundwork for it to become a reality with DisTrO! Check out the AI + a16z podcast with Anjney Midha from a16z and Jeffrey Quesnelle and Bowen Peng from Nous Research as they dive into the technical breakthroughs behind DisTrO and explore how it could democratize AI development. Thanks for having us on! https://lnkd.in/gyjKjmyK a16z Andreessen Horowitz

Nous Research is hiring a Web Developer! For General Web Development Projects, Pages, and Payment Gateways. You will work on all web dev projects within Nous Research, including our primary website, merch store, and custom web applications that include use of AI API's, payment gateways, and custom applications for AI interfaces. We offer good wages, and a collaborative environment with intensely curious AI researchers. Interested candidates can email [email protected] with their resume. For more details, send us a direct message here. We look forward to hearing from you!

Introducing a new open dataset release, Hermes Function Calling V1, the datamix that gave Hermes 2 Pro its tool use and structured output capabilities. HuggingFace Repo: https://lnkd.in/ex_smRtv The dataset includes single and multi-turn Function Calling and Structured JSON Output datasets, and an updated version of Glaive AI's function calling dataset, perfect for training LLMs to be better agents! Also check out our Hermes Function Calling repo for more information on the format and how to use models trained with this data here: https://lnkd.in/eqphH6fA And for help or questions, join our Discord! Here: https://lnkd.in/gnp7JnuQ This work is a culmination of the contributions of interstellar ninja, Teknium, Glaive AI, Theodore Galanos and many others who provided assistance along the way.

What if you could use all the computing power in the world to train a shared, open source AI model? The preliminary report: https://lnkd.in/eXpXUv-M Nous Research is proud to release a preliminary report on DisTrO (Distributed Training Over-the-Internet) a family of architecture-agnostic and network-agnostic distributed optimizers that reduces the inter-GPU communication requirements by 1000x to 10,000x without relying on amortized analysis, and matches AdamW+All-Reduce in convergence rates. This enables low-latency training of large neural networks on slow internet bandwidths with heterogeneous networking hardware. DisTrO can increase the resilience and robustness of training LLMs by minimizing dependency on a single entity for computation. DisTrO is one step towards a more secure and equitable environment for all participants involved in building LLMs. Without relying on a single company to manage and control the training process, researchers and institutions can have more freedom to collaborate and experiment with new techniques, algorithms, and models. This increased competition fosters innovation, drives progress, and ultimately benefits society as a whole. This research is thanks to the hard work of Bowen Peng, Jeffrey Quesnelle, Ari Lotter and Umer Adil. We invite researchers interested in exploring this area to join us in our quest!

?? Meet Hermes 3: The First Full-Parameter Fine-Tuning of Llama 3.1! ??♂? ?? We’re excited to introduce Hermes 3, the first full-parameter fine-tune of Llama 3.1 405B, brought to you by Nous Research and trained on a Lambda 1-Click-Cluster. From strategic decision-making to complex creative writing, role-playing, and Agent building—Hermes 3 excels across the board. Try your prompts immediately on https://lambda.chat/ or use our Chat Completions API - for FREE.

The first Llama 3.1 405B fine-tuning was just completed by Nous Research on Lambda 1-Click Cluster. The outcome is Hermes 3, an unlocked and uncensored instruct model, which a/ follows user directions better and b/ scores as good or better than base model on benchmarks, even after quantization. That enables it to be served on a single 8x H100s node - for which we have on-demand availability and an amazing $2.99 / GPU / Hour price. All can prompt Hermes 3 for free on the https://lnkd.in/dZAMPWcb All can leverage the Lambda Completions API for free: https://lnkd.in/dCM_YEkQ We have a blog post out to bring our story together: https://lnkd.in/dvAf6RTT

Last week, we released our first step in validating and independently confirming the claims of the Bitnet paper, a 1B model trained on the first 60B tokens of the Dolma dataset. Comparisons made on the Weights & Biases charts below are between the Bitnet implementation and a full FP16 run (all hyperparameters equivalent). Model: https://lnkd.in/evCdAgKH Weights and Biases: https://lnkd.in/etWtQ6n3

5 months ago, Andreessen Horowitz（A16Z）announced its support for the Open Source AI Initiative, which includes Nous Research and a friend of mine. Half a month ago, Nous Research just announced the closing of $5.2 million seed financing round. Today, Nous Research?are announcing the latest project, an effort to provide a new evaluation system for open source models. Traditional benchmarking leans heavily on public datasets which can be easy to game and often lead to superficial score improvements that mask true model capabilities (or lack thereof), misdirecting efforts towards meaningless score chasing. To achieve this, Nous Research built a subnet on Bittensor, a decentralized network for AI projects. This system allows people to submit your finetuned model, and then validators will evaluate all models submitted against freshly generated data from the Cortex subnet. The Cortex subnet is a dynamic source of fresh synthetic data, continuously generated by GPT-4, which allows for fresh and unpredictable high quality data to test models against, and ensures a fair and accurate consensus on which model most closely mirrors GPT-4's performance. This new system aims to celebrate and reward creators of public, open source models that genuinely meet user needs, through the incentive structures built into Bittensor. Learn more on the following pages: Nous Subnet Leaderboard： https://lnkd.in/gHjM6Equ Nous Subnet Repository：https://lnkd.in/gSzcWVTH Bittensor：https://bittensor.com Cortex Subnet Repository :https://lnkd.in/guBmfuNu #ai #artificialintelligence #opensource #gpt #decentralized