We've only just begun.

Last week we were crowned winners of ‘Optimizing for Speed Without Sacrificing Stability’?by The Google Cloud DevOps Awards. An amazing achievement in a short time, made possible by many inside our advanced analytics & data science team, and outside. We’ve partnered with several teams across the company to activate our data products, that are solving problems at scale, with our services being hit thousands and thousands of times a day.

The journey so far has been a pretty wild one and I wouldn’t change it for the world. I wanted to celebrate ?? our success, share a little more about our team and where we’ve come in a short time. And talking to the title of this post, I’ve thrown it back to the Carpenters, because, we really have?only just begun…?

It ain’t over till it’s?over?

Our work is never “done” and it won’t ever be. The last twelve months have only scratched the surface, yet we’ve got so much to be proud of and so many lessons learnt. We started to properly work in Google Cloud Platform about nine months ago, and in that time, we’ve shipped, learnt, trained, iterated, pivoted many times. Think of that saying ‘eat, sleep, rave, repeat’,?it’s a constant and we’re here for it.?

The thing that I love (and find equally as scary) about our work, is that whatever we build, we operate it. This is quite different to where we’ve been in this company before, and where many large companies are today. Little things that have a big impact is alerting in Slack. We’ve got alert channels, where we get pretty much real-time notifications so we can see things happening as and when, and then jump on it if we need to.

On a personal level, when I joined it was one of the biggest culture shocks for me — there were so many handshakes, forms to complete, people to seek approval, boxes to be ticked, suppliers and the good old midnight onwards release windows. As I reflect, I see it from many angles. Investment, legacy behaviours, lack of understanding, silos. Hallelujah for the major shift, through the commitment of our leadership team. I was ready to move on when I returned from maternity leave, but this last twelve months has validated the reasons why I chose to stick around and dive head first into this team, on this mission.

I want to get away, I want to fly?away?

To give you a sense of what we’ve been doing, it’s been a little bit like flying a plane — I’m not a pilot, but I’m sure we can all relate.

Last year we began our migration to Google Cloud Platform (GCP), partnering with existing data teams, working in many data warehouses to make data available in ‘one safe place’, in our case, BigQuery. Alongside making the data available and usable, we’ve been building models and creating the infrastructure and pipes to connect to different systems to activate our data products — and a ton of other stuff.?

So, in the words of Lenny, ‘I want to get away, I want to fly away’.?You’re aboard the GCP 737 flying to a beautiful destination at 40,000 feet, cruising at 5000mph. Fasten your seatbelts and hold tight. I did say 5,000 and not 500. There’s no way we could wait for everything to be perfectly in place to start creating value and sure, that’s meant we’ve created a level of technical debt, but as we’ve scaled our products, we’ve scaled our people in those teams and set ourselves up in a neat way. It’s not easy, but we constantly come back to ‘improve daily work’ from the DevOps legend, Gene.?

We’ve had to ruthlessly prioritise what data comes into our world in what order, based on choosing a series of problems to solve early on to get the platform and pipes up and running, deliver value early on, gain momentum and gain our credibility. Whilst we’ve been in build mode, this has sometimes meant taking a manual copy of data we need for a set of time to allow us to press ahead, and by the time we come to launch the thing we’ve build, we have the ‘proper’ data in place with the required refresh frequency in place, e.g. daily, hourly, real-time.?

Picture this, we’re flying the plane whilst putting the wings on at the same time. We’ve been through turbulence, we’ve failed to take off, and yet, we’ve landed the plane many times, took off again, changed the fuel and all made it out alive. This flying malarkey is not easy. But it’s the most rewarding thing when you’re surrounded by brilliant team members, willing to do things differently and write a better future for our people and our customers.

On this journey, we’re able to create value in all corners of the business, and personally, that’s what makes me so excited about working in here. How often do you get this chance in your career, at this scale?

Searchin, searchin (for so?long)

You can blame the aviation talk on the Boeing 737 I came off the week I was writing this. As I was drafting this next part, it got pretty wild referring to seats and rows, so I decided to go for a store-like analogy.?

Data can be so difficult access, which means people will find other ways, and you end up with no single source of truth, no ‘one safe place’ for data. When you’re on a mission to help make better decisions, driven by data (like we are), you only achieve ‘data democratization in every corner of the company’ when everyone has a common place to go for data, with the right access. A big part of our team is on modelling the data, so it’s not just available in any old structure, but meaningful.?

I was introduced to the ‘IKEA concept’?(Curzon, 2021) for the modern data warehouse, and it got me thinking about what many companies data warehouse/s actually look like, whilst they’re still not taking data management and tender loving care seriously. I’d like to think I’ve chimed a new one, the ‘Aldi middle aisle’ (Hemming, 2021)?of a data warehouse.?

With the Aldi middle aisle concept, it’s pure chaos. There’s absolutely no order to it. You didn’t walk in with a problem to solve, you didn’t even know this stuff existed, and you certainly didn’t have need for it. Yet you walked out with a new doormat, sealable lunch containers in four sizes, and some hot sauce. You don’t know where you’ll put this stuff, why you got this stuff, or how you’ll use it. The doormat will find a home, the lunch containers, you’ll probably use one of them, and the hot sauce, well that’ll just sit in the cupboard.

Now imagine IKEA. With the IKEA concept, you head into the data warehouse with a problem you’re solving for. The data warehouse in our case is BigQuery. You see everything in sections, it’s wonderful. Everything has a title, a description, sizes etc. You can make a note of its name, where it is and grab it when you’re ready. Then on the way to access what you need, you’re introduced to things you never knew you needed but you know will help either this problem, or one that’s in discovery. You start to innovate. If you’ve shopped at IKEA, I know you’re hearing me.?

Tell me you didn’t come out with something you didn’t know you needed, but it looks fabulous? Tell me, that when you went back again, you didn’t understand the label on the product? Course you did. It’s the most perfect ‘meta-data’ you could wish for. It’s repeatable. It’s memorable. This stuff takes time, management and TLC. Don’t underestimate the effort people in these roles go to, to make your life easier.?

Beautiful people

To make great great work happen, the core ingredient is people. Sure, the tools and platform are fundamental, working in Google Cloud and with all the other awesome tools such as dbt , we’ve created something special here. But, ultimately, a bunch of shiny tools and platforms are nothing without the people.?

As we’ve scaled, we’ve added more people into a variety of roles. In the early days, we might have had nobody in certain roles, or a few, and we’re still hiring now. We work in cross-functional teams where you’ll typically see a data product manager, data analyst, data scientist, data engineers, ML engineers, software engineers working together, daily, supported by analytics engineering an example. Whilst we’ve all got a job to do, sometimes we need to be where we’re needed most and that’s what all our teams I’ve worked across have been great at it, and it’s a big reason why we’ve been so successful. Fortunately for me, being in product, I get to work with tons of people across our team, and outside our team too, so I see it all.

It’s not about individuals, it’s about our team solving problems together. We work towards a common goal, we’re open with one another and we work effectively. There’s always something to improve, of course, but we get the work done and have fun.?

I can tell you it’s certainly not agile theatre here. It’s methods, tools, and approaches that work for us, and we always look for better ways. Yes, we use JIRA for our backlog and Kanban, and Confluence for our documentation, Lucid for our flows, system designs, ideation sessions. We have daily stand ups, retrospectives when we have a notable thing we shipped. In the data product teams, we have consistency in some things, and slight differences in others, because it what works best for our people and the context we’re working in. Not all work is equal.?

Being in this environment, where Outlook defaults to 30 minute meetings, participant lists are big, in some of our data products teams, we’ve stopped meetings on Tuesdays and Fridays, and after 12 noon Mondays, Wednesdays and Fridays — otherwise, the focus, flow and joy is disrupted, and people context switch, get pulled out of their zone and it’s mega disruptive. If we get crazy asks from people the team is protected, and if it’s an emergency sure we break the rules, but only when absolutely necessary.?

Being predominantly remote, here’s some of the things we’ve been doing this last year whilst building a new team:

• We use Slack for bants, shoutouts, data product teams, common interests, functional teams, product/service alerts, support etc.
• We use Teams…?
• We come together for wider team time optionally on a Thursday in a given location.
• We have data product team working sessions regularly, and time to time in person.
• We do lots of cross-data product team sharing — we’re building an ecosystem of things and we can learn from each other!
• We have functional team meetings weekly, and regularly in person. In my weekly team meeting, we use Trello for the regular agenda and we rate the meeting ‘Come Dine with Me’ style out of 10… classic product managers.
• We have remote monthly socials — quiz master game is strong here.
• We have a town hall every 2 weeks.
• We have a team-wide show and tell every 2 weeks — we seem to have a theme, national [insert thing] day. Recently it was all about umbrellas.?
• We have an annual (we’ve had one as we’re only one) Data Jam.

A company-wide effort

There’s more thank yous across our wider team, the experts in the company we collaborated with such as our lovely friends in the wider Change team such as Architecture, Security, Data Estate, Delivery, Digital Product, through to wider company teams such as operations and commercial.

We make less handshakes than I’ve ever experienced in this company. We build something, we run it. Something goes wrong, it’s on us. It’s making our alert ?? channels in Slack light up and within minutes we’re on it.?

Some Q&A from this journey so far??

• Has it been easy??Not one bit?
• Has it been fun??FUN!
• Have I learnt anything??So much
• Does it feel like I was pushing water up a hill at times??Yes it has done, less and less so
• Does the team get frustrated??Yes and it’s our job to overcome those?
• Do we make things perfect??No,?perfection simply doesn’t exist (Hawkins)
• Do we always get it right??No we don’t
• Do we always aim to make it better??Yes we do
• Have we said no to things that just exist to exist??Yes
• Have we felt resistance??Of course, it’s the nature of our world
• Have we felt openness??Absolutely!
• Would I do it all over again??100% yes I would?

Things you can do?next?

Even in 2022, when the word?data?is probably one of the most used in conversations by people have at all levels in a company; I don’t think enough people have an appreciation of the work that goes into handling and working with it. One of our goals is to achieve data-democratisation in every corner of the company. What does that mean? Everyone in the company, regardless of their technical knowledge, can work with and be confident in talking about data. With this, you get people making informed decisions.

In my mind, this is, away with the traditional training courses like ‘give a great PowerPoint presentation’ — sure that’s handy, but make course more useful for everyone’s work.

Get basic knowledge on data (that’s broad — let’s say experimentation, using data the right way and how data is made usable) — if you’re in a HR function, revisit your learning materials and induction processes. Data should be at the heart of everyone’s basic knowledge in 2022. I heard Booking do this super well as part of onboarding!?

Appreciate your?beautiful people?in data — I’ve only mentioned certain aspects of our work, be kind the next time you just want that ‘quick thing’ — you don’t know if the data is available, if it’s usable, if it’s sitting in the old data warehouse and could take 14 hours to run, or if it’s a case of joining 14 tables ??

We’ve grown, we’re still growing— if you like the sound of this, and you’re interested in joining us change the fabric of this company through data, drop me a direct message or a comment, or check the jobs over here in LinkedIn.

Side note, did you get the song/artist for each section ?? — I went from the Carpenters to Lenny Kravitz (twice) to Luther Vandross/Change to Chris Brown.

Closing thoughts

Why is being the best at ‘Optimizing for Speed Without Sacrificing Stability’ significant for us?

We’re operating in a complex system.?

Large companies are notoriously slow to agree on change, and even slower to deliver change. Having been part of teams changing the fabric of this company, with insight & data turning into ideas and then into production in hours/days/weeks instead of months, even years, this is everything.?

We have lots of customers inside and outside. Stability is experience. If your experiences aren’t stable, it results in a poor experience. Something we struggle to forget and overcome. Having services up and running that are being hit thousands and thousands of times of day, with support of our team is an awesome feeling.?