Parallel Universes: The Shared Visions of Science Fiction Writing and Software Development Management

It's been a while since I've been thinking about this rather unusual topic. When my debut book is released on Amazon in English, I thought I'd celebrate by sharing how I see the book writing process and how it might be similar to software development project management methods I use on a daily basis.

But where the idea is coming from?

Some of you might know that, on top of my day job, which is all about project, program or team management, I’m also a book writer. I’ve already had the pleasure of publishing two science fiction books: “The Controller”, which is available in both English and Polish, and the second part of the trilogy, called “The Banished”. I really hope it will be available in English in the not-too-distant future. While I know I have to write the third part of the “The Sphere of Worlds” series, I took a little break (about two years now, so maybe not that little) and I'm currently finishing my third book, which is completely unrelated to the series.

I think the above has a lot in common with project management. I know that writers are often seen as artists, not tied to any formal approaches. Creators who build the worlds, characters and stories should probably not be chained within more or less formal methodologies. Not quite... To write a book, you still need a solid toolkit and hard skills. If that weren't true, then all the writing coaches or trainers would have no work! Still, though, most people see writing as something creative, and they don't want it to put into strong frames of methods or guidelines.

I have to admit that initially, that was also my impression. However, after two books published, I started to see that it's not the whole truth. I can say even more – the more I write, the more similarities I see in this process with the project management processes or even software development methods. And this article is going to be about that: how the novel creation process can be compared to different approaches in software development. I know it might be a bit controversial, but I’d love to hear your views on that in the comments! I think we can have quite an interesting discussion here.

But before we go straight to the methodologies or methods let me start with…

…different ways of creating a story.

Both from my own experience as a writer, observations as a reader, as well as several talks with other fellow-writers I had I can see that the creation process authors are using is located somewhere on the axis spread across to different approaches.

One of the examples of the first one (let’s maybe call it “right” – as a side, not whether it is correct or not) might be J.R.R. Tolkien and his approach of creating the world, characters and all the artifacts before he even started writing any story there. Many of you might know that this artist before even creating the main plot of “Lord of the Rings” or “Hobbit” has planned the whole world in a really small details – including even creation of a real language that Elves are using. Only after setting up such a stage he felt secure enough to prepare the plan of the adventures that his (again, very detaily designed) characters will have. And only after having that, he started real writing.

I'd love to hear your thoughts on this, PMs! Does this sound familiar? Isn't it like what you do when you're leading a project using classical (waterfall) methods? You secure the resources, plan the actions in great detail and only start when you're absolutely sure you can deliver what you were asked to in the beginning. Then, you prepare the Project Chapter that will draw the deliverables and surrounding environment in great detail. Once you've got the green light, you go into the planning phase. Just like Tolkien, preparing the “Silmarillion” that describes the world, you are creating tools such as Risk Register, RACI, Risk Response Plans and so on so they can be later used like Elves language or whole mythology of Middle-Earth.

I also want to mention that there's another side to the axis? (let me call it “left” – again not meaning, that it is somehow worse). Let's explore this using an example of another great fantasy writer, Andrzej Sapkowski. In Poland, where I live (and he actually lives geographically something like 1km from my place ??), most people probably know him as the creator of Wied?min (The Witcher). For anyone who believes that the story of a male equivalent of a witch is from the game series, please believe me when I say that the stories are far older than the great products of CDP (or even CDP itself). Why am I mentioning him here?

I'm not sure if you know, but the very first idea for The Witcher was introduced as a short story in a Polish magazine. The author wrote it as a short story for a contest and once it appeared, it was so catchy that he started writing another, and another stories or books!

I don't know about you, but for me, that looks quite familiar to one of the approaches we can see in project management. The first story that was published in a magazine was a PoC (Proof of Concept). Once it was really catching, the "product" started to grow. But it still grew in a very iterative way. Sapkowski, until a certain point in time, was creating the new stories like we do it in Agile mode. Every story or novel was an equivalent of the sprint, and after it was completed, the scope of the next one was determined. It sounds like Agile, doesn't it? I hope you agree with me!

Then we have other authors who are somewhere on the axis, with some leaning more to the right and some leaning more to the left. It's the same with projects. There aren't many projects that are pure Agile or pure Waterfall. They all fall somewhere on the spectrum.

But which approach is better?

As you may have noticed, I deliberately used two authors creating stories of the same genre. I could have used Clancy instead of Tolkien or maybe a criminal books author instead of Sapkowski (although I must admit, I am having trouble finding similarly good examples). Based on the examples I used, I believe we can say that using Agile or Waterfall approaches in book writing does not determine whether the final product will be a success. But is it really like this? And is it easily and straightforward transferable into project management?

I am certain that if we get a real master in Waterfall and a real master in Agile and ask them to deliver the same project, they will both succeed. However, as with real masters in literacy, such as Tolkien and Sapkowski, if someone is a twenty-dan black belt with a golden dragons master in a certain methodology, they will be able to make use of it regardless of the project they are granted. How many such masters do we have?

My experience in project and program management is longer and more extensive than in writing books. As an author, I am still learning and looking for the best approach to make both the process and the final product better and better. I can say with confidence that while my first two books were written in a more traditional Waterfall approach, in the case of my newest book, I have taken a much more Agile approach. I’ll tell you how it looked.

The "Sphere of Worlds" series began with "The Controller" and "The Banished." I was the debut author and wanted to know whether I should continue writing and whether my work will be well received.

Even if it was a PoC, it was still very detailed planned. I started with the story, then went deeper into the different threads, ending with each chapter description. This is the classical Product Breakdown Structure that you might be familiar with from Prince2 methodologies (or Work Breakdown Structure in PMI).

I also prepared a set of supporting resources, such as geopolitics of the world the story was to happen or very detailed research of different organisations, military units, even weapon types.

I gathered all the resources required for the story to happen – the characters and key plots – and here the first downside of this approach in case of book writing appears. I treated the characters similarly to how waterfall methodologies treats people working on the project.

In many cases, they are perceived more like Excel cells, with their characteristics limited to the actual needs of the project. This includes their skills, level of seniority and availability. It does not matter on how the skills were acquired or how these skills certain person would like to make use of. In a formal project, especially one developed by shared or external resources, this approach might be sufficient. However, in a book, it makes the characters look "flat" since you are not exploring their motivation, feelings and what is behind certain actions. This was noticed by the reviewers, who often point out that "The Controller" is much more about the story and it is sometimes hard to attach to the characters. Some people like it, others find it more like a reportage, providing facts about the situation, not necessarily focusing on the people the situation concerns.

The same thing can be seen in projects led using “hard” waterfall methodologies. A PM that is only looking at deliverables, not people that are delivering them, is perceived as a boss, kind of a “higher entity: rather than a leader, still in some situations might be successful with very specific kind of project. However, the team working on a project will most likely not be happy, and certainly will not be able to function as a real team. Instead, they will be a bunch of resources gathered to deliver certain goals. While this approach may seem brutal, there are projects in which it works.

After reading the reviews, I realized that in the next book ("The Banished") I needed to take a closer look at my resources. I spent much more time understanding their motivations and added more information about their background. This resulted in better reviews, especially from readers that would like to follow more the characters, than just a story. Still, the general perception is that in both books, the story is the main focus, while the characters are just tools to deliver the final effect.

I believe I managed to move on to the Sapkowski side, but not enough. Hopefully in the third part, I will find the correct place on the axis.

"The Banished" has one more new thing: something similar to change management process in project management. This book has one thread that wasn't planned in advance. It was an idea that came to my mind and I decided to modify the original plan by adding it.? It was a challenging task, especially since the entire story and all the plots had already been created. I wanted to ensure that the new plot would not be "external" to the existing ones. I believe I succeeded, although I have not been informed which thread I am referring to. Still however since “The Banished” is more on a waterfall side, implementing new “functionality” into detaily planned story was a challenge and I actually cannot imagine how much more of such could be done without ruining the consistency of whole story.

Nevertheless, these reviews were one of the reasons why I decided to take a break from that story and start a new one. But that was not the only reason. I wanted to try writing the whole novel using the "left" way – Sapkowski’s way.

Also, the general scope was much different from what we have in "Sphere of Worlds." The number of threads is much smaller, the story is more streamlined, and it happens in a limited area – on a military spaceship. I had an idea, but I stopped myself (which wasn’t very easy, I have to admit) from creating a detailed summary. I decided to write a book in an Agile mode, moving from chapter to chapter and scene to scene, letting the story and characters (which were the only thing documented in advance) drive me. I knew what I wanted to accomplish, but I was prepared to create the path as I went along. The results?

I am still writing (almost two years, compared to three months for "The Controller" and five months for "The Banished"), but I am happy with what I have achieved. When I sent just a part to my publisher, he said he likes it a lot. He particularly liked the way the characters are much more real, and that he can now attach to them, try to understand them, and believe the actions they are taking are backed by their previous experience. At the same time, he said the story looks good, and he cannot wait until he sees how it develops.

Sounds great, even might be perceived that Agile approach, where I kind of went with the flow works. Even more – I was able to deliver the value much quicker, like we have in case of Agile-driven projects… I remember in case of “The Controller” and “The Banished” I needed to send the full book to the publisher so he could say it is good to be published – here he said that even after a small sample… And initially indeed it looked that way – until I needed to finally start closing the plots… Then it appeared some of them were created quickly, some of them does not tie together and actually my greatly created characters are in the position the only thing they could do is putting the bullet in their head – since there is hardly a way they can do anything on the situation I put them in. So, I have nice 400+ pages of the story that does not have the end and creating a real, great ending is starting to become hardly possible.

This situation is all too familiar. I've seen it countless times. A Product Owner is provided with a functionality he gets excited about during the initial sprints. This opens a Pandora's box of new ideas and ways to develop the product further. The team members (characters) proceed according to these new ideas, putting themselves in less and less optimal technical situations. They want to keep the PO happy by making quick changes, but they don't think about the future. They can ignore this for a while, but it will eventually become more and more painful. There will come a point when you have to face these problems and start fixing them.

I found myself in the same situation in the newest book. At a certain point, I had to stop, step back, and prepare a detailed summary of what had already happened. I had to fully understand the interconnections and approach the topic of "technology debt" I created earlier. I needed to put away the Agile rule of "code over documentation," since the "code," in this case, the paragraphs and chapters, were far from self-descriptive (or easily digestible in a limited time). I needed a detailed plan to take a look at it and fully understand what I actually did. I needed a proper architecture design, non-functional requirements and solid plan for what and how I want to achieve. Only after doing this, I was able to figure out how I can give these characters at least a chance to survive, with some changes to what has been written earlier.

So, does that mean we cannot adopt PM principles to writing books?

I suppose in previous, longer than I planned in the very beginning (again a bit more Agile than Waterfall approach) paragraphs I was able to show that creating a story is very similar to creating a software product. If you would agree to that, I suppose you could also share my opinion, that similarly to the project management methods, book writing methods develops, while the decision on which we will choose, or rather how much of Agile and how much of Waterfall bits you will take to your toolkit is very much dependent on the environment you are acting on.

While in software development this environment is covering the organization you work for, the type of the contract you have, the product you need to deliver and the maturity of the team, in the author’s world this is all about type of the book and your maturity in the area. For sure going to any end of the axis, unless your name is not Chuck Norris might be not the best path to follow. Of course, we can find among authors people that indeed holds that high rank – mentioned earlier Tolkien, Sapkowski, Clancy, King, as well as the others. But they most likely will not even read this article (at least two of them might have objective problems in reading what is on Internet), so I would assume that putting yourself somewhere on the axis makes a bit more sense.

Same thing you have with software development. If you are a real guru in Agile, you might be able to successfully lead any given project in Agile and avoid problems with technical debt, unclear architecture or further scalability. Being most experienced within Waterfall you will be able to lead successful delivery without making a team unhappy by feeling as simple resources. But I still believe the more chance you will still have by going somewhere on the axis – for some individuals and projects it might be better to stay on the right side of it, for other on the left.

For me, writing a science fiction books with very high pressure on realism I believe staying a bit more on the right side might be the best. Not too much so I will not treat my characters as simple resources, but not going too much to the left so I will not loose the need for solid research and maintaining the action consistent with how the world is working. Maybe if I would (someday?) decide to go to the Might’n’Magic world, I suppose I would move a bit to left, but still not very near to the end – so I will not have to create completely fantasy spells that will rescue my characters from impossible situations.