Lightup Data

?? The rumors are true: Lightup Data Quality for Unstructured Data is coming... ?? Be the first to see the exclusive beta preview, join the waitlist today, linked in comments below. Stay tuned for more exciting updates as we continue to innovate with AI in 2025! #Lightup #DataQuality #AI #BetaPreview #UnstructuredDataQuality #UnstructuredData

??? Given Large Language Models (LLM) are only as good as the data they’re trained on, #DataInconsistency in the #UnstructuredData feeding those models is the not-so-silent killer of LLM accuracy. ?? Conflicting information in unstructured data, such as documents, creates confusion for LLMs. Why? LLMs can’t decide which information is correct.? Naturally, this leads to faulty results and unreliable outputs —? a significant challenge for organizations relying on LLM-based #InformationRetrieval for tasks. We put that to the test in our latest blog by senior staff engineer Sarabjot Singh, where he used 4 leading LLMs to simulate a practical business scenario of extracting information from unstructured data in a mock financial report. ?? *SPOILER ALERT*? All 4 LLMs ???????????? due to inconsistent data. (Guess what? We ???????? that would happen, which is why ????'???? ???????? ???? ?????????? ????.) ?? See the test, prompt, and results in Sarabjot's blog, linked in the comments below. ?? P.S. At the end of the blog, sign up to join the waitlist for our exclusive beta preview of ???????? ?????????????? ?????? ???????????????????????? ????????, coming soon... #LLM #Lightup #AI #UnstructuredDataQuality #BetaPreview

?? New Feature Alert: Zero-Config for Auto Metrics! ?? ?? We’re excited to announce a new automated feature that'll skyrocket your Lightup productivity: ????????-????????????. ?? Instead of manually configuring metadata Auto Metrics and monitors for tables, Zero-Config does it for you,???????????????????????????.?By default, Zero-Config is enabled in Lightup, so you’ll get metadata insights from day one — without any extra clicks or configurations. ?? ?? Read new feature announcement, linked in comments. ?? #Innovation #Automation #Lightup #NewFeatureRelease #ZeroConfig #DataQuality

This was a really fun conversation with Malcolm Hawker! The data stack is profoundly getting redefined for unstructured data and LLM-based apps on top of it. And that changes everything from the way you build data products, the way you monitor them, and the way you govern your unstructured data. When we had this conversation late last year, we left it at "data foundations need to be rethought" but we couldn't really articulate the solution. And a lot of it still remains unknown today. That said, we are seeing some really promising results with a whole new way of data quality and data observability for unstructured data. I alluded to it last week. There's more we're going to talk about really soon. This is a very fertile area to innovate in. The good news is that there are many low hanging fruits. Stay tuned for more! For now, give this episode a listen. Or find the transcript on our website (link in the comments).

?? In our latest fireside chat, Malcolm Hawker, CDO of Profisee, joins Lightup CEO and Founder Manu Bansal to unpack a question posed in Malcolm’s LinkedIn post: “Why should data teams be skeptical if they hear a 'focus on foundations' is needed to enable GenAI?” ?? For this “poke-the-bear-worthy” fireside chat challenging some of the core foundations of data, hear Malcolm and Manu engage in a lively conversation with probing questions and surprising answers, covering:? ?? The paradox of operationalizing and getting value out of AI.? ?? The multiple sides of dark data and the implications of lighting it up.?? ?? The unspoken truth about AI governance frameworks. ?? Watch the full fire side chat and see Malcolm's original post that sparked this chat, linked in comments below. ?? #FiresideChat #AI #Lightup #DataQuality #AIGovernance

The messaging around data quality and data observability used to be confusing up until last year but it has really straightened out this year. As customers researched the market and took a hard look at their own requirements, Gartner also got it straight (credit to Melody Chien). I really like where we ended up: Data Quality and Observability (DQO) is one (big) category with monitoring of data quality — monitoring the data content — being the first and the most important pillar among other needs. We don't have to fight over DQ vs. DO any more. It's (now) all part of the same equation. From a product point of view, we are seeing much-needed convergence too. When this all started 5 years ago, we saw two broad camps. The "data quality" camp was focused on scanning data. The "data observability" camp was mainly interested in presenting metadata. Thankfully, this artificial divide has also been bridged. Most products now provide both the abilities. The biggest difference in positioning and product design now appears to be the target user persona(s) and the design philosophy. Some are targeting engineers while others are targeting business users. Some are designing blackboxes while others are designing glassboxes. Some are designing autopilots while others are designing copilots. Some are enabling centralization while others are designing for democratization. It would be interesting to see how this category evolves in 2025, especially as the race to become AI-first picks pace. Exciting times ahead! A link to my full blog post is in the comments. #dataquality #dataobservability cc: Lightup Data

??Happy Holidays to our Lightup community! ?? Wishing our employees, customers, partners, advisors, and investors a safe, healthy, and joyful holiday season filled with cheerful connections and heartfelt memories! ?? In the spirit of giving, may we all remember that the best-quality present is the gift of your presence. ?? Cheers to new and exciting opportunities in 2025! To your success, Lightup

?? As we look ahead to 2025, we paused to look back at how the landscape of #DataQuality and #DataObservability has evolved so far and why that matters. In 2023, Manu Bansal wrote a blog to clarify the differences between Data Quality Monitoring and Data Observability technology solutions. ?? Fast forward to 2024, where Gartner recognized the convergence of these two technologies with a unified category: ???????? ?????????????? ?????? ??????????????????????????. This change reflects what we've been seeing all along — organizations want the best of both worlds. ?? As this blended category evolves, one thing is clear: Ensuring high-quality data content is more critical than ever, which is why Data Quality and Data Observability technologies have become increasingly intertwined. ? Check out Manu's latest blog to find out what this means for the future of Lightup in his blog, ???????????????????? ???????? ?????????????? ?????? ???????? ??????????????????????????: ????????’?? (??)?????????????? ?????? ?????? ???????? ?????????????? ( ?? link in comments). #DataQualityAndObservability #Lightup #TechTrends #FutureOutlook

We’re excited to announce the beta preview release of Genie, Lightup’s new GenAI copilot assistant (link in comments)! ? Genie is designed to help automate tasks, answer questions, and provide guidance for configurations and processes — all through a simple, natural language chat experience. ? 2025 is going to be the year of automation.?There's a lot more to talk about when it comes to automating data quality. We'll continue this conversation in the new year. ? For now, if you want a sneak peak into the Genie, join the waitlist (link in comments). #dataquality #dataobservability

?? <???????? ????????> ?? We’re excited to announce the beta preview release of Genie, Lightup’s new GenAI copilot assistant.? You asked for more AI, and we’re on it! ?? Introducing Genie, Lightup’s copilot assistant designed to help automate tasks, answer questions, and provide guidance for configurations and processes — all through a simple, natural language chat experience. We can’t wait to show you what Genie can do! ?? We’re now offering early access to this exciting new feature through an exclusive beta preview. ? Get a sneak peek at the future of Lightup productivity, join the waitlist today and be among the first to ask Genie questions! ?? Read our latest blog announcement to join the Genie waitlist, linked in comments. Stay tuned for more Genie updates… #DataQuality #GenAI #AI #Genie #BetaPreview?#Lightup #CopilotAssistant