Openlayer

?? Announcing LLM evals and real-time monitoring with Openlayer ?? Today, we are excited to release two major new features that have been highly requested, and some more on top! Join our Discord community for more updates and be a part of our exciting development journey! ?? https://lnkd.in/gZwskTnN ?? New features 1?? Evals for LLMs ?? You can now use Openlayer to evaluate your LLMs with the most comprehensive suite of tests on the market. Integrate your prompts/models/datasets effortlessly with a single line of code or directly from our platform, and instantly check for issues in your inputs and outputs such as PII leakage, invalid JSON schemas, malformed code syntax, and so much more. 2?? Real-time monitoring ?? Connect your production or test data and measure performance over time, for LLMs and all other task types supported by Openlayer. Hook into your inferences to log every data point, or upload predictions and ground truths in batch. Immediately get alerts when your metrics and tests start to fail. 3?? New metrics and tests ?? Openlayer now supports a much wider range of metrics and tests that can test the performance and quality of your models and data in development and production. Measure the runtime performance of your LLM by setting guardrails on latency and token usage. Guard against hallucinations using GPT-based evaluations like relevancy or harmfulness. Set expectations on your model inputs and outputs to ensure they are follow the correct schema or format. 4?? Slack & email notifications ???Stay in the loop with personal and team-wide Slack and email notifications. Get alerts when your models and data fail tests in development or production. Get updates on workspace activity in Openlayer, such as comments or changes to model deployment statuses. And the best part? You can experience all of these new powerful features first-hand today for free. Sign up at app.openlayer.com and get exploring! #launch #ai #ml #artificialintelligence #machinelearning

If only they had used Openlayer... ?? https://lnkd.in/efkJ_-JN

At Openlayer, we are committed to maintaining the highest standards of security and privacy to protect your data. When using Openlayer, all you need to focus on are evaluations, and we'll make sure your data is protected with SOC 2, GDPR compliance, role-based access, SSO, air-gapped deployments, encryption, and automated backups across multi-region hosting. Learn more at https://lnkd.in/e4uBVpcp

Happy Halloween! Here's a treat - our October changelog! We've added support for SAML SSO, an improved test diagnosis page, some design refreshes, and more. Read the full list of updates here: https://lnkd.in/epUcBUiC

We've integrated our Changelog into our docs! Monthly updates of everything we've shipped, fixed, and improved will resume now. Stay tuned this week for the October changelog. https://lnkd.in/ecGscMQZ

We just made it even easier to diagnose issues with your test results on Openlayer ? For every test you've created on Openlayer, you can now see: - new list view along with the graph view of test results over time - an overview of recent issues and aggregate counts for each test status type - error messages when a test is skipped or errored Try it out on https://app.openlayer.com

?? We've added support for login with SAML SSO! Your team's Openlayer workspace is now more secure than ever.

We recently rolled out a re-design of our blog. See it live at the link below, stick around and read a post or two ? https://lnkd.in/eSbWzGBV

Hurb, the largest travel agency in Brazil, has partnered with Openlayer to accelerate the development of their AI initiatives. Read up on how they use our platform to continuously evaluate and monitor models shipping to their 20+ million users. "With this partnership, we will significantly accelerate the development of our machine learning models, maintaining the confidence that all updates will go through a rigorous validation process" - Vinicius Mello https://lnkd.in/egWYsyvm