4 Places Agile Project Teams Spend Their Time

One of the things that I'm very focused on in my role at AWS is staying on the tools. By this I mean continually building. I spoke about evolving projects, versus building once and declaring the project done, in a recent blog post called ‘Are you sure you are setting up your IT project teams for success?’ This post examines the advantages of always-on project teams above finite resources attached to a particular project for a particular length of time, which is how organisations usually do it.?

Now I want to dig deeper. If we accept that maintaining and gradually improving something is better than second guessing every last requirement you are likely to have in the future, let's take a look at some places where your project team’s time might be best spent.

1. Updating inputs from external sources.?

External sources can very easily change, within any app. Sometimes you simply need more as time goes by. Sometimes you need different ones. That’s why I like to use an API gateway, above system calls. System calls are platform dependent, whereas APIs offer a degree of abstraction that lets the project team focus on the “contract with other systems” independent of the details of the underlying system. If you were to build your project using system calls for external data inputs, you would probably find yourselves creating separate program logic for different platforms. Then when you do need to expand or change data sources, you are settling in for the long haul.?

However, one of the nice features of the Amazon API Gateway is that it supports app evolution and improvement over time, with the ability to define validation rules for inputs. For example, when adding fields to your build from a new source, the gateway checks with you: ‘Should that be a string field or a number field? And if it's a number field, does it need to be a range?’ What this does for the builder - even when they need to ingest pretty significant amounts of data and send it to be processed in different ways - is allow them to do it with as little componentry as possible. And as simply as possible. In other words it removes what we call ‘undifferentiated heavy lifting’ from making system changes, making monthly or weekly improvements par for the course.

I always think of the apocryphal claim of the Michelangelo quote, where he was asked ‘How did you make the statue of David?’ Apparently he responded, ‘It's easy, I just chip away the stone that doesn't look like David.’ In the case of APIs, the idea is they allow you to build as little as possible to reveal sophisticated functionality.?

2. Deploying apps and mobile backend services

Something that might change for your customers or citizens over time is their mobile app needs. And you might not see that change coming. Look no further than COVID-19 to know how quickly and unpredictably change is sometimes required in apps and mobile services. State governments have been fast to respond to citizens’ needs in rapidly creating critical apps to inform and guide them during COVID. When the State of Victoria made the sighting of vaccination certificates a condition of entry to certain venues, Service Victoria rapidly built a function into its mobile app allowing people to store Commonwealth-supplied certificates there. More than 2 million people added their certificate in the first 10 days; This represents great agility and proving it’s entirely possible to get going really fast with solid back end capabilities at a very low barrier to entry.?

What would it mean to your project team to have the same agility and flexibility to revisit the functionality of live apps at a moment’s notice? You’ll be pleased to hear it’s not that hard. Typically, front end systems need a pretty sophisticated collection of back end services to round out their functionality, including storage, databases and user management (sign-up, sign-in, identity, resource access control, and security), messaging, and user engagement. Creating these backend services is often perceived as? time consuming and error prone, which manifests in reluctance on the part of project teams to revisit mobile apps.?

At AWS we address this reluctance with technology. For example, AWS Amplify is a set of purpose-built tools that lets frontend web and mobile developers build full-stack applications very quickly, on AWS. In fact, you can configure a web or mobile app backend, connect your app in minutes, visually build a web frontend UI, and easily manage app content, all with no cloud expertise.

3. Moving data between databases?

Another challenge that prevents organisations from revisiting products is a perception that moving data between databases is complicated and resource intensive. Common reasons for database migration are to improve security and compliance, reduce cost, improve performance, and meet unforeseen scalability requirements by moving to a more suitable technology.

AWS offers a schema conversion tool that lets project teams export from one database engine to another, for example SQL server into Amazon Redshift. So if you've got a data warehouse that you're currently running on Microsoft SQL Server and you need to move it to Amazon Redshift, which is a massively parallel processing database that operates in the cloud? in a high scale, high performance and low cost model, you can do that using this tool. It allows you to convert the data types of the tables, databases, and any other components and do a trial run before going live, and that makes the migration process much easier. It also substantially reduces the risk, as schema conversion is often the hardest part of migration.?

The key point here is automation. The goal in transitioning from periodic projects with large, upfront investments to a model where apps are continually optimised to deliver maximum value all of the time, is to find the tools to automate difficult processes. If you're building any sort of repeatable infrastructure or application deployment, AWS CloudFormation is another such tool, because you can essentially document that deployment as a template and deploy it repeatedly. You can also parameterise it, and use it to spin up multiple environments, which is a boon to productivity and repeatability.?

4. Application scaling for performance and cost optimisation.

There is one more important consideration in making the switch from set and forget to always-on projects, and that is to enable auto scaling? to accommodate fluctuations in demand. AWS Auto Scaling monitors your applications and automatically adjusts capacity to maintain steady, predictable performance at the lowest possible cost. When demand drops, AWS Auto Scaling automatically removes any excess resource capacity, and since you only pay for what you use, you avoid overspending.?

The concept of AutoScaling applies to things like compute and storage. If you are using serverless technologies like Lambda and? DynamoDB it is even easier!

But it is not just about scaling up for performance, it is scaling down to maximise cost efficiency. Why run more than required and pay for things you don’t need? It makes no sense - so spending time on optimisation is a gift that keeps on giving!

Thinking long term about components

When you are building new applications to meet current business challenges, it pays to think about the components that you are using and how they may evolve and change over time. This helps you reframe the way you build systems, with a view to staying current. Agile teams focus on things that genuinely make updating and improving these components repeatedly over time easier to do, with less moving parts, at a more effective cost. When you structure your project teams to refactor and regularly improve on their products, you are better equipped to meet the rapidly changing needs of your customers.?

Keep on building!?

Listen to this related podcast: on “Removing What is Not Needed” from your architecture.