The Progression of Software Development: From Tasks to Value

Have you skipped steps on the way to working on sexier tech topics?

In the subjective realm of software development, it is vital to comprehend how every small element impacts the overall success and value of a product or organization. Without this clarity, teams and departments face missed deadlines, increased costs, and strained relationships between stakeholders and developers.

Some call it a “dev tax”, chalking the problem up to a baked in inefficiency or translation problem. Others will micro-manage the process with a fine tooth comb thinking they will find some piece of evidence which shows ‘someone is doing it wrong, and I have the answer’. Both the indifference and obsessive tracking of every little detail will ruin the social structure of the group, reduce the return of value hence averting continued funding, and often put peoples’ livelihoods in jeopardy.

There are dozens of process efficiency frameworks out there, many tie back to a version of “Agile” from the famous Toyota lean car making roots. Today many of these frameworks have evolved into cult followings which may allow for followers to simply follow the formula without understanding the first principles behind them. This article is an attempt to leave the dogma at the door and exclusively refer to the underlying principles.

This progression demonstrates how foundational elements influence high-level outcomes. The article will discuss each layer and its contribution to the whole. Each layer is essential and must be built upon in sequence. Recognizing the importance of each piece improves team efficiency and effectiveness. Understanding these layers is crucial for success and value in software development practices. Not every single concept of software development will be reflected here. Rather these are the key pieces which a manager or leader of these teams need to support the team and nourish the process across many teams at scale.

The Foundation: Unit of Work

At the very base is the concept of the "Unit of Work." This foundational element speaks to the critical nature of sizing work into small, similarly sized pieces to enhance predictability and reduce cognitive load. In software development, a unit of work can be a task, a user story, or any other small segment of the scope that can be completed within a defined time frame, ideally a day. The importance of breaking work into these manageable units cannot be overstated. It allows for better estimation, tracking, and management of the full scope. Predictable units of work lead to predictable feature outcomes, forming the bedrock upon which all other metrics and KPIs are built.

There is no tool which will solve for this because every team and each member on that team is unique. Each team uses the technology they have selected slightly differently from another team. What will solve for this is good old fashion discussion of the scope of work with all the right people (who are “all the right people is a topic for another day”). Based on the collective understanding of the work, the team breaks it into smaller pieces which the average team member can complete in roughly a day.

This is the single highest leverage puzzle piece, in my humble opinion, because it forces many key things to happen at the very beginning of building anything. If you take away only one thing from this, break down work into manageable pieces.

Layer One: Issue Tracking and Management

Moving up the pyramid, the first horizontal layer consists of detailed issue tracking and management elements:

- Issue Statuses: Monitoring the current status of issues (open, in progress, resolved) helps in understanding progress and identifying bottlenecks.

- Issue Actual Start - End/Resolved: Tracking when issues start and end provides insights into cycle time and helps in improving process efficiency.

- Issue Sub-tasks: Breaking down issues into sub-tasks ensures that all aspects of an issue are addressed, promoting thoroughness and completeness. This is often used when mentoring less experienced team members to create practice for thinking through and breaking down their own work.

- Issue Links: Understanding the relationships between issues helps in managing dependencies and impacts across the project.

- Issue Time Tracked: Tracking the time spent on issues is essential for assessing effort and improving future estimations. While this is not required, for all teams, forever, it is a very effective objective way to show where time was spent.

- Issue Estimates: Estimations for issues help in planning and resource allocation, contributing to more predictable outcomes. Estimates may be in time or the popular “story points”.

These elements are crucial for the day-to-day management of software projects. They provide granular visibility into the progress and challenges faced during development. Teams that effectively utilize these tracking and management elements often experience smoother workflows, reduced bottlenecks, and improved collaboration among team members.

Layer Two: Team and Scope Management

The second layer focuses on team dynamics and scope management:

- Team Velocity: Measures the amount of work a team can complete in a given period. It is a key metric, based on historical data from Layer One, for predicting future performance and planning sprints.

- Cycle Time: Measures the time taken from the initiation of a task to its completion. This is similar to Velocity but does not measure over a particular length of time. Shorter times are better, though with diminishing returns. This is based on Layer One Actual Start and End Dates.

- Team Capacity Planning: Ensures that the team has the right amount of work allocated, preventing burnout and underutilization.

- Scope Dependencies: Identifying and managing dependencies, based on Layer One Links, between different parts of the scope helps in avoiding delays and ensuring smooth progress.

- Scope Actual Cost: Tracking the actual cost associated with completing scope items helps in budget management and financial planning. This is derived from Layer One time logs or story points completed, where story points have a rough cost based on employee salaries for a time period.

- Scope Estimate Accuracy: Ensuring that scope estimates are reasonably accurate is vital for effective planning and resource allocation. While perfect accuracy is highly unlikely, over time the team should find a consistent sweet spot of 15%+/-. This is based on Layer One Estimates and Actual Dates.

These elements help in managing the resources and scope of the project, ensuring that the team can work efficiently and effectively towards project goals.

Layer Three: Process and Profitability

The third layer encompasses broader process and profitability metrics:

- Lead Time: Measures the average time it takes a team to have availability to start a work item. Shorter times are better, though with diminishing returns. This is based on Layer Two Team Velocity as well as the team’s backlog of work.

- Roadmap: A clear roadmap provides a strategic direction for the project, aligning all stakeholders towards common goals. This is aligned to Layer Two Team Velocity & Lead Time.

- Aggregated Costs: Understanding the total costs involved in the project helps in financial planning and ensuring profitability. This is simply Layer Two Actual Cost at scale.

- Profitability: Ultimately, all efforts in a software project aim towards achieving profitability or value creation. This metric indicates whether the project is financially viable. This will be different to every organization, however in short, ‘is there enough value created here to continue investing money?’.

These elements provide a broader perspective on the project's efficiency and financial health, linking operational metrics to financial outcomes.

Layer Four: Strategic Improvements

The fourth layer highlights strategic improvement areas. Each of these examine the previous layers to tease out where there is wasted time, lack of clarity, and tooling optimizations.

- Operational Improvements: Continuous improvement of operational processes ensures that the team can work more efficiently and effectively.

- Personnel Improvements: Investing in personnel development and well-being leads to a more motivated and skilled team.

- Cashflow Improvements: Managing cash flow effectively ensures that the project remains financially sustainable.

These elements focus on long-term improvements and strategic investments that enhance the overall capability and sustainability of the team(s) output. Often at this layer, more is not better. Better is more.

Layer Five: Strategic Planning and Growth

At the top of the operational section, we have high-level strategic metrics:

- Strategic Planning: Involves setting long-term goals and determining the actions needed to achieve them. It provides direction and purpose for the project.

- YOY Growth: Year-over-year growth measures the project's progress and success over time, indicating its long-term viability and success.

These strategic elements ensure that the project is not just focused on short-term gains but is also positioned for long-term success and growth.

The Apex: Valuation

Finally, at the very top is Valuation. This metric represents the culmination of all efforts and investments. A high valuation indicates that the project or company has successfully managed its operations, resources, and finances, creating significant value. The valuation is a reflection of the trust and confidence that stakeholders have in the project's potential for future success.

Unit of Work supports Issue Tracking, which is the basis for Team Performance, where you then show Consistently Creating Value, yet only then can you Reinvest Strategically, and Examine Growth to realize a Valuation.

This layered approach shows how foundational elements in software development, like the unit of work, build upon each other to impact higher-level outcomes, culminating in the valuation of the project or company. By understanding and optimizing each layer, from granular issue tracking to strategic planning, software organizations can enhance their efficiency, effectiveness, and ultimately, their value creation. This holistic approach ensures that every aspect of the development process contributes positively to the overall success and valuation of the organization.

There are many many other factors which play into this examination like size of the organization, employees vs contractors, internal enterprise applications vs monetized SaaS applications, the advent of AI as coding assistants, and all the various compliance requirements some teams face, just to name a few.

If you struggle with delivering on your roadmap, or your costs are volatile, look at the team performance. If the Velocity or Cycle Time isn’t relatively consistent, there is an opportunity to look at the issue tracking. If there isn’t data to support estimates being used day in and day out, there is a strong chance there isn’t clarity around the Unit of Work.

So… this is an opportunity to look at your organization and be brutally honest.

Have you skipped any steps?

Maybe not completely skipped, but neglected?

How do the challenges you currently face tie back to the underlying principles of Layer One or "The Unit of Work"?

Luckily it’s not too late. Once you can see the system as a whole, you can follow the problem you are experiencing back down to its roots and at the very least arrive at a better question to work on.

It’s also perfectly normal to ask for help. You may be really savvy in one area, but inexperienced in another. Find a guide who doesn’t worship dogma and simply wants to set you up to succeed. I have nothing to sell you, but I can show you what has worked well for many other companies as well as the lessons they learned along the way.