Book Review & Insights: The Phoenix Project A novel about IT, DevOps and Helping Your Business Win
I got the book "The Phoenix Project" - a novel about IT, DevOps, and Helping Your Business Win, written by Gene Kim, Kevin Behr, and George Spafford, as a gift. The CEO of the IT company I am currently working at gave it to me the moment he saw me looking at the book with my eyes wide open. Usually, I do have a sharp sixth sense for knowing what is worth holding on to, and considering the fact that the CEO is quite a bright man, I was very curious about what was brought to his attention.
Indeed, the book is a valuable asset to everyone who wants to develop their professional life. Let’s face it - every company is a technology company, whether you admit it or not. As Christopher Little, a software executive and one of the earliest chroniclers of DevOps said: “Every company is a technology company, regardless of what business they think they’re in. A bank is just an IT company with a banking license”.
“The Phoenix Project” is a book about a new IT initiative, code-named “Phoenix project”, which is critical to the success of the Parts Unlimited company. The project is over the budget and extensively over the deadline. That is why the CEO of PU wants the IT manager to report directly to him and fix the fiasco which was being played around the different departments. With the help of one of the board members, which later on becomes the mentor of the IT manager, the company is saved. The mentor shows and makes the IT manager follow the three principles underpinning DevOps:
- First principle: systems thinking. The principles of flow. The focus is on all business value streams that are enabled and run by IT. Or in other words said: the flow begins when clear requirements are set (by a Product manager)—> the requirements are executed by the developers —> transitioned to the IT operations (Project managers) and then delivered to the end clients. “W. Edwards Deming called this “appreciation for the system”.”
- Second principle: Amplifying feedback loops. This includes understanding and supporting all customers, internal and external, and amplifying all feedback loops, and applying knowledge when needed;
- Third principle: creating a culture that fosters two things - continual learning and continual experimentation.
Lately, a lot has been going on in the firm where I work as a Production Manager - a lot of reductions, a lot of infrastructural amendments which, I think, surely are for the better of the company. There is a hunger for change. In all departments, on all levels of the company, since no one knows and has the drive to do something toward this change. In the book “the change” is described as follow:
“Any activity that is physical, logical, or virtual to applications, databases, operating systems, networks, or hardware that could impact services being delivered.”
Well, change is inevitable, in one way or another, and I would say - it is always good if we are able to lead it. Let's start step by step and look over the first principle in detail.
Principle one: Systems thinking
There is a chronic conflict in every IT organization that there is an inherent conflict between Development and IT operations which creates a downward spiral, resulting in ever-slower time to market for new products and features, reduced quality, increased outages and worst of all - an ever-increasing amount of technical debt. IT organizations are responsible for many things, but the two goals that should be followed simultaneously are as follows:
- Respond to the rapidly changing competitive landscape;
- Provide stable, reliable, and secure service to the customers;
Configured this way, Development and IT Operations have diametrically opposed goals and incentives. Dr. Eliyah M Goldratt, one of the founders of the manufacturing management movement, called these types of configurations “the core, chronic conflict” - when organizational measurements and incentives across different silos prevent the achievement of global, organizational goals. Dr. Eliyahu M. Goldratt, who also created the Theory of Constraints, showed us how any improvements made anywhere besides the bottleneck are an illusion. Erratic but true. Any improvement made after the bottleneck is useless because it will always remain starved, waiting for work from the bottleneck. And any improvements made before the bottleneck merely results in more inventory up at the bottleneck.
The most important job is to be ensured the fast, predictable, and uninterrupted flow of planned work that delivers value to the business while minimizing the impact and disruption of unplanned work, so it can be provided stable, predictable and secure IT service. It must be ensured that the most constrained resources are doing only the work that serves the goal of the entire system, not just one silo.
The constraint in the book was one of the developers called Brent. He was taking crucial parts in every project that must be delivered and since he was the single senior developer for a long time, he was the only one who knows how to fix issues once they appear. He was the bottleneck of the company. According to Dave Lutz, there are two types of Brents: hoarders and sharers. He writes:
“.. I’ve also come across otherwise smart [people] who are of the mistaken belief that if they hold on to a task, something only they know how to do, it’ll ensure job security. These people are knowledge Hoarders. “
Almost four years ago when I started my professional development, I admit - I was a hoarder. The moment I learned something I was so afraid of sharing it with the others around since I thought that if the time of the upper management comes to choose someone to cut off, this could have been me. This could have been a failure I would not be able to accept or even face. All these fears were so embedded in me from my childhood that if someone drops me off a project it would be a total loss. It came from the fact that my father is a military person to whom not being the best in whatever I was doing was absolutely NOT an option. This “guilt” in me not being able to meet all expectations and plans stayed in me for so long. Thankfully not anymore. I am a total sharer now.
Thankfully I am now surrounded by people who believe in me unapologetically. My family, my mentor and my closest friends are the ones who made me realize that indeed if I fail in something would be a total loss. But not for me. For me would be even a gift so I would be able to improve and learn from my mistakes. And as my high school teacher once advised me “if you fail – stand up, turn around, shake off your dust and continue foreword with your head held high!”.
Being a hoarder doesn’t work. Everyone is replaceable. No matter how talented they are. Sure, it may take longer at first to find out how to do that specific task, but it will happen without them. In the “Phoenix Project”, Brent was a sharer whose contribution was an inevitable part of the success of the company. He believed job security comes from sharing knowledge and capability increases.
Second principle: Amplifying feedback loops
The second principle as described in the book is creating the right to left feedback loops. The goal of almost any process improvement initiative is to shorten and amplify feedback loops so necessary corrections can be continually made.
"..How do the our competitors keep bringing such incredible new capabilities to market while we remain stuck in the mud?”
Yes, the marketing experts are like art or music majors, not people with a technology background, (not that I am) they would publicly promise the impossible and IT would figure out how to deliver. However, with the proper way of receiving and giving relevant feedback, these "empty promises" can be fulfilled and the real customers’ needs can be met.
The promises cannot be thrown away freely from the window ledge with the clients expected to follow. Something very important must be underlined here: the majority of the marketing projects can’t be done without IT. High touch marketing requires high tech.
In the book, the mentor of the IT manager, called Eric, described the relationship between a CEO and CIO (which was described not as Chief Information Officer, but more likely as that your "Career Is Over”) as a dysfunctional marriage. That "both sides feel powerless and held hostage by the other." However, in order for a marriage to be successful, continuous feedback from the very beginning from both sides should be given and taken. Despite the hard times, despite the disappointments and the different opinions. This is how trust and loyalty are built. And believe me, there is nothing more important than loyalty. Loyalty matters.
A great team does not mean that it has the smartest people. What makes teams great is that everyone trusts one another. It can be a powerful thing if that magic dynamic exists. I recalled a program to which I was enrolled. On the 1st of February 2019, I and a bunch of other Managers were certified for successfully undertaking a Management and Leadership Program organized by the company we are currently working in. The program lasted four months and was featured by @intunitycoaches and more specifically Plamen Petrov.
He used great approaches with us: a lot of games were played, a lot of discussions and passionate fights we all went through, however at the end of every session we had we were left with food for thought and actually “hungry” for more self-improvements. Because let’s be honest - at the end of the day if you do not have self-awareness and you do not know what you want to achieve in the long-term, no matter what kind of genius and talented people you have in your team, it is doomed to fail.
The picture is taken from my Instagram post here
At the beginning of the last session, @Plamen asked us to pick up one random photo from two desks full of chaotically placed images. The photo I choose was the above one. Running and dirt. Why? It does purely describe my professional background so far. And it perfectly defines my e-v-e-r-y-d-a-y challenges - running a marathon while being as dirty as hell.
While looking at the photo I’ve concluded two things: I am taking from this course note that I always need to ask myself (when it comes to making decisions):
- Are the other opinions I am listening to actual facts or just opinions?
- If something is not OK - how am I helping the issue to exist?
Once I answer these two questions, I can continue running - to the next goal, to the next meeting, to the next conversation with and for my people. And even enjoying the fact that most of the times I am in the dirt.. but hey! Dirt is healthy. “Dirt” is the work, the grind, and the hustle. So let’s be it. In his book “Five Dysfunctions of a Team”, Patrick Lencioni writes that in order to have mutual trust, you need to be vulnerable.
What I observe in my daily routine and it is deeply discussed in the book is that more processes are needed, as well as better support from the top management - including IT processes tooling and training. Everyone thinks that the real way to get work done is to just do it. That makes the job impossible. Because here is what happens: first, you take an urgent data-driven project, where the live date cannot be delayed because of external commitments made to the business, stakeholders or to the customers. Then, a bunch of developers is added who use up all the time in the schedule, leaving no time for testing or operations deployment. And because no one is willing to slip the deployment date, every one after Development has to take outrageous and unacceptable shortcuts to hit the date. Making changes without waiting for authorization because of deadline pressure. The results are never pretty. Usually, the software product is so unstable and unusable that even the people who were screaming for it end up saying that it’s not worth shipping.
Four are the categories of work: business projects, internal projects, changes, and unplanned work. Unplanned work is what prevents you from doing the actual work. Like matter and antimatter, in the presence of unplanned work, all planned work ignites with incandescent fury, incinerating everything around it. Unlike the other categories of work, unplanned work is recovery work, which almost always takes you away from your goals. That’s why it is so important to know where your unplanned work is coming from. (Kanban Board)
"Every work center is made up for four things: the machine, the man, the method, and the measures. And because of the ever-improving production monitoring of the infrastructure and applications and the constant feedback received more often than not, we should know about the incidents before the business does."
Being able to take needless work out of the system is more important than being able to put more work in the system. To do that, you need to know what matters to the achievement of the business objectives, whether it is the project, operations, strategy, compliance with laws and regulations, security or whatever. That's how the feedback is so crucial for every stage of every project.
Creating constant feedback loops from IT operations back into Development, designing quality into the product at the earliest stages. Much faster feedback is needed. Sensei Mike Rother says that it almost does not matter what you improve, as long as you’re improving something. Why? Because if you are not improving, entropy guarantees that you are actually getting worse, which ensures that there is no path to zero errors, zero work-related accidents, and zero loss. This is called the Improvement Kata. He uses the word “kata” because he understands that repetition creates habits, and habits are what enable mastery. Whether you are talking about sports training, learning a musical instrument, or whatever - nothing means more to mastery than practice and drills. Studies have shown that practicing five minutes daily is better than practicing once a week for three hours. And if you want to create a genuine culture of improvement, you must create those habits.
Third principle: creating a culture that fosters two things - continual learning and continual experimentation;
Or in other words - becoming grasshoppers. In these competitive times, the name of the game is a quick time to market and fail fast. As Sensei Goldratt would say:
".. you’ve deployed an amazing technology, but because you haven’t changed the way you work, you haven’t actually diminished a limitation."
The flow of work should ideally go in one direction only: forward. When it is seen some work going backward, the only word coming to the mind is “waste”. It might be because of defects, lack of specification, or rework. Regardless, it’s something that should be fixed. The goal is single-piece flow.
While finance goals are important, they are not the most important. Finance can hit all our objectives, and the company still can fail. After all, the best accounts receivables team on the planet can’t save a company if the teams are in the wrong market with the wrong product strategy with an R&D team that cannot deliver. In order to be kept up with the customer demand, which includes the upstream comrades in Development, it needs to be created what Humble and Farley called “a deployment pipeline”. That’s the entire value stream from code check-in to production. That’s not an art, that’s production. Business agility is not just about raw speed. It’s about how good you are at detecting and responding to changes in the market and being able to take larger and more calculated risks. A new culture needs to be created that reinforces the value of taking risks and learning from failure and the need for repetition and practice to create mastery.
IT is not merely a department. Instead, it’s pervasive, like electricity. It’s a skill like being able to read or do the math. Understanding what technology can and cannot do has become a core competency that every part of this business must-have. Because let's face it:
"..Any COO who does not intimately understand the IT systems that actually run the business is just an empty suit, relaying on someone else to do their job."
As stated in the book, "... a dysfunctional marriage assumes that the business and IT are two separate entities. IT should either be embedded in business operations or into the business. There you go. No tension. No marriage, and maybe no IT department either. Maybe everyone is a form of DevOps, and it’s something much more than that. It’s Product Management, Development, IT Operations, and even Information Security all working together and supporting one another".
#PhoenixProject #LetsAllBeBrents #DevOps #IT #Operations #ProductManagers #ProductOwners #ProductionManagers #ProjectManagers #CEO #CIO #COO