Making the scary leap to management

“My default reaction was no. But when I thought about it, the reason I'm saying no is because it scares me, I think this is going to be hard, and that's probably the exact right reason I should try.”

Matt Boyle, an engineering manager at Cloudflare, joins the show to talk about coming on board the tech giant during the pandemic, how that influences his management style, and the vital importance of regular retrospectives.

Below is a transcript of this conversation, edited for clarity and brevity. You can subscribe to PriorityZero on Spotify, Apple, Pocket Casts, and YouTube.

How did you get into software engineering??

I did a very typical thing. A computer science degree at the University of Birmingham. I loved computer science, and as part of that I did a year in industry where I worked for General Electric in a project management role.

I got really lucky, and they made me an offer to go back on a graduate scheme called the IT leadership program. On that you did six-month rotations in four different parts of the business. I don't actually think there was a better way I could have started my career. I did some software engineering, and some leadership and management stuff too. I think that made me a lot more balanced.

I got to see not only how big companies write code and manage deployments, but also how they manage big, multi-million dollar projects and migrations. I absolutely loved my time there, but I really wanted to go to a much smaller company and experience what they had to offer. I joined a company called Crowdcube with an engineering team of around 12.

Did you find yourself gravitating to one size of company over the other?

I think everybody should work in both. You'll learn things from big companies that you can apply to small companies, and having experience in a small company and understanding what is possible is really valuable for applying to big companies, too.?

For example, if you work at a really small company, the distance between you and production is probably 30 seconds, maybe less. You get to a big company, maybe, and it's taking a couple of hours to get to production, maybe days, weeks, maybe you have no access to the path to production at all. Being able to ask “why” is valuable, and if you get the answer that it's not possible, you can say, "actually it is, because I've done it."?

Having that healthy tension and experience levels between start-ups and big companies, the middle of those two things is where productivity really is. You are going to get constraints as you get bigger. There are going to be certifications you want to get, compliance things you have to ensure happens, but you should always keep that thought in your head that it really is possible to move much faster than this. The people who I think do that best have experience of doing both, because they understand why the constraints are there, but they also understand what's possible.

Yeah, you can either decide to feel stifled by that gap, or you can be curious about why and then discover, once you get context, that it's there for really good reasons. Is that your experience?

Yeah, for the most part, but there have definitely been situations where it's not been there for good reason, and just someone being willing to step up and ask why we do things like this, and digging and digging until they can figure out why that decision was made or why we're doing it that way, can be really valuable. You can save a ton of cycle time by being willing to do that. I really value those people who are willing to dig a little bit deeper and figure out why it is this way.

I think there are so many companies that are going through that process right now and trying to figure out where they can make things a bit more efficient. Is that where you spend a lot of your time thinking?

Definitely and I think you've called out a really important point there, which is that this is dynamic. You might make a decision in January that's not valid in December anymore. As the company grows, shrinks, priorities change, you should reevaluate your processes as well as your team compositions and your priorities. You see companies often rethink their priorities, but I'm not sure they think about the way they work as much as they probably should.

What led you to Cloudflare?

I wanted to focus more on back-end engineering. I had spent a lot of my free time learning the programming language Go and wanted to be able to bring that to work. So that was kind of my focus: I want to use Go at work, and focus more on the backend because I was doing a lot more full stack at Crowdcube. I ended up joining a fintech called Curve.?

I think there were about 30 of us. I stayed there for about two years, and by the time I left, we were a couple of 100 people. I got to see us scale up. I also got the opportunity as a tech lead to take on some more people management responsibilities, which was a useful way to learn a little bit about that space, what I enjoyed and what I didn't, and how to balance my time between those things. It was a really great place to work, and I learned a lot, and I got to spend a lot of time writing Go, which is what I wanted to do.?

As the pandemic hit, I felt like it was time for me to do something a little bit different. Cloudflare has always been a company that I wanted to join. I loved their blog and as I interview people now, it consistently emerges as the number one reason candidates want to join Cloudflare. It continues to be a great channel for us to find talent, and I was no different. I applied, didn't ever expect to really hear anything back, and went through the interview process and got to join, which was amazing. It was a dream come true for me.?

I joined Cloudflare just as we went into lockdown in the UK. So I went from being an in-office worker at one company to a remote worker at a different one. Yeah, and Cloudflare itself was still figuring out how to do remote work, so it was definitely a transition. I found it really tough. It was also the first time that I'd joined a team that wasn't all in one place. Most of my colleagues were going to be in the US, and figuring out how to navigate that and work together was certainly challenging, but it went really well for me.?

I really enjoyed my time with the team I was on, and after about a year, I was promoted to tech lead. Our team grew, and my manager was promoted to director of multiple teams. Then, I took the managerial role of the team I was the IC on, which is still the team I manage today.

That’s a journey we hear really often, that IC to tech lead to management. What did you learn along the way, especially in that middle phase?

I think the more time you spend being an individual contributor, and as you start to transition into that technical lead role, you do a little bit of management stuff, and you start to realize that whichever way you look at this, everything you're doing is ultimately about people. You can swap programming languages, you can make that code a little bit more efficient, but ultimately everything comes down to having a team that works really well together and feels like they trust each other. Also working in an environment where people feel they can speak to their colleagues about issues and ideas. I naturally started to discover that as much as I always loved writing code, I also loved talking to people, I loved helping people, I loved unblocking people. In the role I am in I also get to do a bit of product management as well. I like figuring out what's the most impactful thing we can work on.

I'm not convinced I would have pursued an engineering manager role myself, but it was one of those opportunities that arose. My manager said, 'I think you'd be good at this.' My default reaction was no. But when I thought about it, I realized that saying no scares me. I think this is going to be hard, and that's probably the exact right reason I should try.

What scared you about it?

It's really hard to measure your success as an engineering manager. You can feel like you're doing all the right things and getting good feedback, but what if someone is not telling you what they're really thinking? There are things you can put in place to hopefully account for that, too, like anonymous surveys, but it's really hard to have data that clearly outlines your performance.?

It also changes the dynamics of the team. All these people who are my colleagues and now my reports. It can be quite an isolating position. I was quite concerned about the dynamic shift; maybe that my colleagues wouldn't respect me as a manager, and I wouldn't be able to do a good job. All these things were fears of mine.?

Also, as an IC, it feels very easy to measure success. Deliver this feature by this day. Do this ticket by this day. Review this PR and deliver it whenever. It’s very quantitative. You can't really do that as an engineering manager. You can't measure your success by code anymore, and you've got to start thinking a little bit differently.

How did you figure out how to bridge that gap?

I think one of the most valuable things any team can do is to have a retrospective regularly. I think if you had to cancel everything else, that's the meeting to have. If you have no stand ups, you have no other interactions, have a retrospective every couple of weeks.?

You might find at the start that it doesn't feel that valuable because it takes a really long time to build trust that people can say things like, “this isn't working. Can we change it?” Even if people raise small issues or small actions, as a manager, your job is to go and chase those down and make impact. As you start to make changes, people will realize they can have impact and change things. Then people will start having deeper conversations. From those, you can start to uncover what's really bothering the team.

What have you learned about running good retrospectives?

I think the most important thing is to be really aggressive about time boxing. It's very easy to just spend all of your time in one area and not actually get to any concrete action items, and then you leave dissatisfied. Being able to say, we're going to spend 15 minutes in total; there's an open period where people can put down all their issues. I'll spend a quick two minutes grouping them. You can vote on a theme, and then we're going to talk about each theme for exactly five minutes, and then we have to make two action items for each of the themes that we discussed. Just being very rigorous with time management is the way you've got to drive outcomes. If you don't get to concrete actions, it isn't valuable.??

What have you learned about managing teams that are a bit more distributed than maybe most?

It's hard. There are no good answers. I think the retrospective is the way to make it work. You have to find what's not working in your team composition and make changes to make it work. You also have to make space for people to raise issues and concerns, and then you have to really take action on those things.?

The other one if you are in a lot of time zones, you're probably going to have to move to a model where asynchronous working is your default. I don't just mean raising pull requests and leaving time to review them. I mean things that maybe would be a meeting becoming a long-form written Wiki article or a very long email or something that people can digest in their own time and review it and then get back to you with feedback. It means that the decision making loop can feel slower, but in the long run, if you can adopt that model, you'll be very successful with teams across time zones.

Shifting from a speaking culture to a writing culture?

Yeah, and we're still working on this. There are still things that we have synchronous meetings for where I would wonder if it could work as a written document. If you start to go down that transitionary path, you'll realize it's probably the right way to go to allow everybody to have the work-life balance they want, and also ensure that you're documenting all of the decisions that you're making, as well as allowing diverse views into all of those processes and projects that you're working on. I'm reaching a point where I think it has to be the way.?

The challenge is that it's not going to suit everybody. As we've moved from an in-person culture to a remote culture, a lot of people will find it hard to move to that way of thinking and working. Some people may never get there. They may prefer to work in an office. I think we need to be honest about acknowledging that. There's going to be people who like working in very different ways. In the past, it was very much everyone goes to the office, if you don't like it, tough. That's what we do.?

Some breakout companies, like Automattic and GitLab, were seen as trailblazers. However, I'd say remote working is the default for many people now, at least for three days a week or so. We should acknowledge those who operate optimally in the office and avoid assuming that remote work suits everyone, as it may not.?

Where do you fall personally??

Plan in person and execute remotely. That's where I would love to be.?

This can't always happen, especially when you are as distributed as we are, getting everyone in the same room is really, really challenging. But if there were no budget constraints and everybody was available to travel, I would get everyone in the room maybe for a week every quarter. We would do a ton of planning and team building, and spend time together. Once we figured out exactly what we wanted to do, we'd go away and we'd execute remotely.

What were the other areas of management that you found challenging?

I've had maybe five or six managers in my career, maybe more, so I know how my 1:1s look. You don't get to see anyone else's 1:1s, right? You don't know what sort of questions people are going to ask. You don't know what their concerns are going to be. You don't really know how they like to structure things, or if they're going to structure them well at all. You can obviously ask people about their preferences, but if you do, you get some really interesting answers.?

What's the best answer you've got to that question??

To provide a structure, and the one that I really like is progress, plans, problems. You put those into a document, and you ask folks to fill that in every week. I found that it brought structure to pretty much everybody's 1:1s. Then, what I did for certain people who clearly knew what they wanted out of their 1:1 was to go off-piste. If someone's coming to 1:1s without questions or concerns, adding that structure helps. We also make a point of hiring interns and graduates at Cloudflare. Having a standard way to structure these meetings is really powerful.

What's your priority zero?

In November, Cloudflare had a really major incident. We were down for a couple of days. There's a very public blog post about this. If you search for code orange, you'll find it. A very short summary of it is that one of our data centers went offline, and we were massively over indexed on that data center. Ever since then, we've had a huge emphasis on making sure everything is highly available and that our infrastructure is solid.

That's the promise you give to customers, too??

100%. But my focus is on internal teams, and we need to give them the same promise. They can't give that promise to customers if we don't give it to them. So, my focus has been on making sure all of our developer tooling will be available, whatever happens. If someone takes a data center off the internet, we have redundancy everywhere.?

My goal is always that an engineer at Cloudflare shouldn't even know that anything has happened. If a data center has gone offline, they may see a quick blip where we bring something else online, or some cutover happens, but they don't need to change any configuration. That's been my priority zero since November: how do we get to a place where every single tool at Cloudflare, especially the ones that are needed for the path to production, is highly available to all engineers at all times?

Where do you start with that??

It all starts with retrospectives again. We had two different forms of retrospective for this. Typically, when I've attended retrospectives in the past, they're all very insular. It's just your team and you figuring out how you could have done better in the way you were working and maybe the projects you were working on.?

This time we looked outwardly to our customers, which is engineering in our case, and asked what we could have done better for them. There are some really obvious wins there. There was tons of really interesting output from that, too, around an educational gap. "I thought this tool did this, but it turned out it actually worked a little bit more this way." That's like a softer thing we need to work on, rather than just making sure everything is in all the different data centers and whatnot.

Figuring out what would have the biggest impact that we should be working on immediately, and then just building a backlog around this sort of hardening, as we called it, making sure everything was in a really good spot so that if that incident ever happened again, we would feel really good that we would be in a much better spot to deal with it. That incident did happen again, in February, and we were in a great spot to deal with it. I don't even think there was any sort of real impact on customers.

What would need to happen for this to no longer be priority zero?

I'll always try to think of this as priority zero. I think it's very easy to take your eye off the ball in terms of highly available infrastructure and making sure all of your tooling is available at all times for everybody. I never want to get to a place where it's not a priority. I think being able to measure what great looks like is really powerful.?

Something we do at Cloudflare is Chaos Monkey to inject chaos into your infrastructure and see what happens. We do that regularly, so we're always learning and finding opportunities to improve. And honestly, if we find anything that is a failure mode that we weren't ready for, it goes straight to the top of our backlog. We always try to keep that P0 mentality.

Matt’s recommendation: The Acquired Podcast