Quotient AI

It’s incredible to see developers starting to organically pick up Quotient AI and share their work through cool demos. As a founder, nothing prepares you for the awesome feeling when people **in a another country** are excited about your product. Hai N. — I’m grateful we’re part of the same community. The feedback you share with us is invaluable and makes our product better and better everyday. Excited for what’s to come! Keep an eye out for more ships in the next couple of weeks ??

I had an amazing time with my good friend Jason (Choonghwan (Jason) Lee) last weekend at the Duke University AI Hackathon! While many of the teams at the event were busy building great AI-powered tools, chatbots and agents, we noticed a systemic lack of focus on quality testing, especially for enterprise AI solutions. From car dealership bots selling $50,000 cars for $1, to AI customer service agents using profanity, many chatbots out there get deployed too early without proper stress testing. To combat this, we built Evalon, an automated platform for large scale testing of chatbots. Evalon generates business specific tests in combination with synthetic user personas to find unique edge cases for the chatbot. We also implemented LLM as a judge methods to evaluate those tests, allowing us to get detailed reasoning on how the chatbot can better perform. We won the best LLM evaluations special prize thanks to the team at Quotient AI! Check out our full-stack web app and devpost submission to learn more! I had a blast building out the product and learning how to leverage llm's in a unique way. Devpost: https://lnkd.in/e2-evath Website: https://evalon.vercel.app

We took the new @anthropic models out for a spin

Apple's AI summaries stumble when summarizing humor and sarcasm. We challenged our AI Engineer James Liounis to improve their email summarization prompt ??

We were excited to sponsor the Duke University AI Hackathon last weekend!

We're working on intelligent prompt engineering. ?? PromptLab + ?? IQ Rapidly prototype with your favorite models. Evaluate. Build production-ready AI products. Out in beta. (Grab your laptop for this one!) ?? on ?? more coming soon ??

?? No tricks, only treats! ?? Join us for a fun night of AI projects with AI Tinkerers London. We're teaming up with Monzo Bank and Zilliz to bring engineers, AI enthusiasts, and tinkerers together for a hands-on meetup focused on everything AI. Whether you're working on the next breakthrough or just curious, you'll have the chance to share ideas, network, and enjoy pizza, swag, and refreshments! Interested in Speaking? Got an exciting project or experiment in AI? We’d love to hear about it! No need for polished slides—just bring your work-in-progress ideas and connect with a community of experts from across the AI space. Sign up in comments ??

Quotient AI's new SMELL framework, now AI developers can better align LLM evaluators with human feedback. ?????????????????????????? ?????????? ?????????????????? ? Subject-Matter Expert Language Liaison for AI evaluation ? Combines human expertise with LLM capabilities ? Generates structured feedback, rubrics, and evaluation metrics ?????? ???????????????? ? Integrates domain-specific knowledge into AI assessment ? Leverages LLMs to process and analyze vast amounts of data ? Bridges the gap between expert input and AI capabilities ???????????? ?????? ???????????????????????? ? Enhances enterprise AI development, especially in customer support ? Improves AI performance through targeted feedback ? Offers scalable evaluation across various domains and use cases Quotient AI's SMELL framework is revolutionizing how we approach AI evaluation, ensuring more human-aligned and reliable AI systems.

John Berryman and I recorded a podcast.. and we took ourselves very seriously! Listen to the first episode of Two-Shot to learn more about prompt engineering, LLM evals, and our takes on the is-RAG-new-or-is-it-just-search debate. Check out the full episode of Two-Sho with {John Berryman} {Principal Consultant, Arcturus Labs} {ex-GitHub Copilot} on all the podcast places: - YouTube: https://lnkd.in/gNjq4KGg - Apple: https://lnkd.in/gHxV7jhu - Spotify: https://lnkd.in/g7vFPND9 More episodes are coming with some very cool people! Each episode invites a guest to a straight-to-the-point, no-softballs, deep(er) dive about the areas they're most passionate about. DMs are open for speaker & topic proposals.

Everyone I talk to is at least considering developing with open LLMs, but they often have to "show that it works" with a proprietary model first. This is changing fast. I'm very confident that open LLMs are going to play a huge part in AI development moving forward. There's a ton of very compelling tooling being developed that's lowering the adoption barrier for open LLMs and making the open vs. proprietary decision more of a tossup. I write about this shift and the tools I expect to be part of the permanent core dev stack: Together AI, Fireworks AI, Groq, OpenPipe, Martian, unstructured.io, LangChain, LlamaIndex, Pinecone and Qdrant. Check out the article here: https://lnkd.in/ev2vRXc6