Helicone (YC W23)

Today, we're publicly releasing all of our LLM usage statistics. 1.5B+ requests, 1108B+ tokens, 16+ TB of data, now available for your curiosity or research. All anonymized. Explore one of the largest public AI conversation datasets ever: https://lnkd.in/eeD7Xg75

We just improved the Properties pages with new metrics and better user experience! ?? Properties lets you add custom metadata to LLM requests for advanced segmentation and analysis. Tag requests with session IDs, conversation context, or application data to gain deeper insights into your AI application performance. Check out the Properties tab in Helicone.

Helicone (YC W23) is growing fast and we need help!! ?? We're hiring a DevRel Engineer to join our founding team. You'll work directly with me and Cole Gottdank to build our growing developer community, create content and help shape the future of AI observability. This is an incredible opportunity for someone passionate about AI and open source to make a real impact. If this sounds like you or someone you know, please apply here (IN PERSON - SF): https://lnkd.in/gWMqWjRb (P.S. feel free to message me or Cole if you have any questions!)

Today, we are introducing our new Generate API. ?? Now you can deploy your Editor prompts effortlessly with a light and modern package. Take the prompt ID in the editor and deploy it everywhere. Supports all the Helicone features natively, while we keep it updated in the Editor. See documentation in the comments.

?? Excited to partner with Helicone (YC W23) - Top 3 @ProductHunt Open-Source '24! Now you can monitor your LLM apps while using Novita's 200+ LLM APIs for DeepSeek, Llama, Mistral & more: ?? Monitor & debug LLM requests in real-time ?? Test prompts without code changes ?? Track costs & performance ??? Catch regressions pre-deployment

?? It's an exciting time in AI Research! OpenAI's new Deep Research tool is turning heads—and for good reason. It's designed for users who need in-depth analysis of complex topics, yielding quite impressive results. ?? Meanwhile, free alternatives like Perplexity’s Deep Research and Open Deep Research are gaining traction as a response to OpenAI's $200/month price tag. Our latest blog dives into key capabilities of OpenAI Deep Research, and how it compares to more budget-friendly alternatives (Google, Perplexity, other open-source research tools). ?? Link in comment. #DeepResearch #LLM #AIDeveloper #AIObservability

?? Grok 3 just dropped and it's making big claims about being the "Smartest AI in the world". Here's how it compares with top models right now: ?? ???????????????? ?????????????????? Early users found Grok 3’s “Thinking” mode solves problems better than many competitors. ?? ?????????? Grok 3 performed well on structured logic problems with proper chains of thought. ?? ???????? ???????????? ???????? Found high-quality information on recent events, similar in depth and quality to Perplexity's Deep Research, but not at the level of OpenAI's. ?? ???????????? ?????????????????????? Early user found Grok 3?struggled with complex coding. GPT-4o and Claude provided better solutions. ?? ???????? & ???????????????? ?????????? While strong in structured problem-solving, it failed Andrej Karpathy’s?Unicode emoji mystery challenge, whereas DeepSeek's R1 performed better. ?? ?????????? & ???????????????????? The model lacks any advanced abilities for humor. When asked for jokes, it repeatedly gave variations of the same puns, similar to older LLMs. ?? ????????-???????????????? ???????????? Early users found Grok 3?hallucinating citations?and even inventing fake URLs, similar to problems seen in other LLMs. Overall, Grok 3 has been impressive but not perfect—and still lags behind OpenAI’s o3 in benchmarks. Detailed comparison in the comments.?? #xAI #LLM #AIMonitoring

?? Product Launch Alert?- AutoEval is Now Live on Helicone?? Model Evaluation Meets AI Observability ?? LastMile AI has partnered with Helicone (YC W23) to bring native AutoEval metrics directly into Helicone’s AI observability platform. If you’re using Helicone to monitor AI applications, track experiments, and optimize LLM usage, you can now run evaluations, compare model performance, and analyze drift—all in one place. What This Means for You: ??Seamless model evaluation: Track model accuracy, latency, and reliability without switching tools ??Real-time cost insights: Monitor API usage and optimize spending across LLMs ??Better decision-making: Identify the best-performing models based on real-world production data Get Started Today ?? Read the integration guide (Link in the comments??) ?? Try it out on Helicone (YC W23)'s platform

?? Switching to DeepSeek without proper testing can be risky. Here's a step-by-step guide to help you switch your production app to DeepSeek safely. https://lnkd.in/gnWM9fTN #DeepSeek #Helicone #LLM #PromptEngineering #AI