Seattle Data Guy

BigQuery continuous queries?operate continuously processing SQL statements, allowing companies to analyze, transform, and replicate data in real time as new events arrive in BigQuery. They use familiar SQL syntax to define the data analysis and can handle large volumes of data efficiently. Companies can synchronize their data immediately as it reaches BigQuery and push that data via?Estuary?to the downstream applications and tools after enriching the data as they need it. This unlocks real-time use cases powered by data in BigQuery, such as immediate personalization, anomaly detection, real-time analytics, and reverse ETL, etc. By Daniel Palma Jobin George Rob Meyer https://lnkd.in/gbJ_5mTP

I don't know who needs to hear this today, but no one will take ownership of your career. If you're a data engineer or analyst and you're not careful you'll quickly become the go-to-person for ad-hoc data pulls. Just a service desk for the marketing, HR, sales or other department. Don't get me wrong, it feels great to feel needed. I remember being the person that could quickly write any query needed for a data set for another team. But completing 1000 Jira tasks doesn't mean you were impactful. It can be easy to get caught up in the motion of work without ever asking if the work is worth doing. As the saying goes, don't mistake motion for progress, especially if your goal is to grow. In this article I want to discuss why it's important to find projects that move the needle and how to do so. https://lnkd.in/gYAi_Eky

If you're looking for a Lunch and Learn to join Mage is running one in less than 20 minutes! So if you want to learn about "The Multi-Tenant Foundation for Data Mesh Excellence in Mage Pro" check it out. Tommy Dang will be leading the discussion! https://lnkd.in/gMwcBf-C

Going from a data engineer or analyst to leading a data team is hard. You have to pick up a whole new skill set fast, learn a whole new set of terminology and not to mention you likely spend a lot less time coding and writing SQL. Just ask Alex Freberg. Not to mention you're often pressured to deliver results quickly, and have stakeholders who like using the the words "This should be quick" and "just" a lot... This is why I have been talking to dozens of data leaders over the past few months to understand how they made the transition and what skills they felt were crucial to being a successful data leader. But I'd also love to hear from you. What skills, articles or books would you recommend to new heads of data out there? Also, if you'd like to learn more, then you can check out this video. https://lnkd.in/gdipVDnv

I'll be the MCee at the upcoming Shift Left Data Conference next Thursday. You can expect some bad rhythm guitar playing and out-of-tune singing. But if you can manage to overcome the cringe, you might enjoy the wisdom from all the folks talks! The lineup is absolutely spectacular!

You might not know this, but besides creating content, I am a data engineering and infrastructure consultant. That's my actual job. I've shared content about it off and on, and tomorrow, I'll partner with OpenTeams and Inessa Pawson to share more about my journey. So, if you'd like to learn about "Going Solo" from me, Benjamin Rogojan, you can sign up below! Don't miss out! Register: https://lnkd.in/detvSZgB

When you first start out as a data engineer or data analyst your focus is often, well, the data. You spend countless hours learning SQL, Python, and the latest tools. Nights are dedicated to refining data models and how other companies build data pipelines and perform analyses. But eventually, something shifts. A light bulb goes off. You start asking yourself:?What’s the point? You’ve built dozens—maybe even hundreds—of dashboards. You’ve developed complex?pipelines?designed to make data accessible for analysts.?But does anyone actually use them? What value is driven by my work? As the unsettling thought creeps in, you might start questioning your purpose.?I’m just completing tasks my manager assigns—isn’t that my job? Shouldn’t they be the one deciding what’s important? Some people get stuck there, deferring entirely to their managers to set the agenda. But others feel compelled to dig deeper, asking a more meaningful question: What work truly creates impact? The difference between staying stuck and making progress is?being a “needle mover.”?If you simply complete tasks without understanding their value, you’ll struggle to advance in your career or create meaningful change for your organization. As a?data leader,?your responsibility is clear: identify and pursue projects that move the needle. I am planning to run two more on April 8th and 9th, you can sign up here if you're interested, you can learn more about it here -> https://lnkd.in/gc3U5D7e

For the last few years, I have spoken with both data and business leaders. Two conversations stuck out, in which the leaders expressed frustration with data teams failing to deliver business value. These conversations, unfortunately, were far from a one-offs. I saw it firsthand when I was hired to replace a consultant who had not delivered business value fast enough. But here is the truth? Many data teams aren’t set up for success. To make matters worse, I find that many data engineers, analysts, and scientists get thrown into the role of Head of Data or perhaps director and don’t get much in the way of coaching or advice. There is just the expectation that we will do a good job because we were a good individual contributor. Then, they’re faced with challenges like: - Picking the right tools while navigating aggressive sales tactics - Juggling ad-hoc business requests with long-term strategic planning - Dealing with a lot of hype around AI and real-time - Building trust with stakeholders - Balancing their technical expertise while also having to lead - And so many other challenges That’s why I’ve made it my mission this year to focus on how data teams can operate more effectively. Not just on the technology, but on leadership, strategy, and execution. I've already run several webinars with great speakers like Tessa Xie Shachar Meir Lauren Lin Jeff Skoldberg Josue A. Bogran and so many more. I am planning to run two more on April 8th and 9th, you can sign up here if you're interested, you can learn more about it here -> https://lnkd.in/g4UMk839 Also tag anyone you think should go!

If you were recently put in charge of a data team and your first thought are wondering what shiny new data infrastructure tools and solutions will you get to use. You’re headed in the wrong direction. Don’t get me wrong, we all love shiny toys. But they can also be a distraction to what the business needs. Sometimes, it’s the businesses themselves that distract us and use the terms they’ve heard or read about and demand that we deliver possible solutions that aren’t needed or that the business is not ready for. I do believe part of the data team's role is to help consult in these situations. Don’t just be a task taker, be a strategic player. If you want to be part of a strategy, you have to act on it. To be clear, you’ll still face businesses and CEOs that will continue to treat you like a task taker. Sometimes this can be fixed by taking ownership and leading with limited power, and other times you’ll be trusted right away. If you're looking for help to improve your standings in the business or are wanting help picking the right data tool, feel free to set-up a free consultation with me - https://lnkd.in/gTWgrnAm

Data and analytics teams are often responsible for several key pillars in a company. This can pose a challenge when you’re the head of data and analytics and you need to pick your next data project. Where do you even start when you’re constantly bombarded with requests from your management team. In this article I want to discuss how you can start selecting your key projects to work on as a head of data analytics. First, I started by breaking down your projects into 3 key types of projects. Analytics, automation and data products. Below we will discuss what each of these projects are and some considerations you should have when taking these projects on. https://lnkd.in/gPA2tKZQ