Starting a new software project can be exciting and challenging, but also daunting and complex. How do you plan, design, code, test, and deploy your software effectively and efficiently? In this article, we will cover some of the essential steps and best practices that can help you kick off your software project successfully.
The first step is to define the scope of your software project. What is the problem you are trying to solve, and what are the goals and requirements of your software? Who are the users and stakeholders, and what are their needs and expectations? How will you measure the success and quality of your software? These questions will help you clarify the scope and boundaries of your software project, and avoid scope creep and unrealistic expectations later on.
I believe the most important thing at the beginning of any project is to ask "What problem are we trying to solve?" It is even stated in the "Defining the Scope" section, but it truly should be a separate line item. Let the strengths of each individual involved (engineers, architects, designers, product managers, testers, and stakeholders) be a part of defining the problem, then everyone is working toward the same goal and can act accordingly to create a great Product. The Power of Product is the Team!
Always start the new software project by creating a walking skeleton. Decide which infrastructure is needed for the project (i.e., database, email server, app server(s), authentication method, etc.) Add all those components to the system and then make sure to configure so that they can handshake and talk to each other. You now have a walking skeleton. This skeleton has zero functionality, but is up and is walking. From that moment on, you will not have any unpleasant surprises when deploying to production. Next, benchmark your "Time to Hello World" app. Make sure it can be deployed in a matter of minutes.
Requirement gathering is the most important part. When building software first we should understand customer needs. After analysing those we can start building a great product. Also, maintains solid documentation from start to end.
Ask the stakeholders what is the expected end date. If they give you an exact date get on the next bus out of town; you have been setup to fail.
Optimize hardware expenses Accelerate time-to-market for new features Enhance cost-effectiveness Simplify software maintenance Decrease overall weight Implement a scalable platform
The next step is to choose the methodology that suits your software project. A methodology is a set of principles and practices that guide how you manage, organize, and execute your software project. There are different types of methodologies, such as waterfall, agile, scrum, kanban, and others, each with its own advantages and disadvantages. Depending on the size, complexity, and nature of your software project, you should select the methodology that best fits your needs and preferences.
Not a single methodology fits any software project all the time. This makes it very hard to pick one approach for a single project. Sometimes waterfall fits perfectly, other times an agile approach might be a good fit. The best thing to do would be to define a single approach that can handle all situations. We use our own approach that we call 'the vortex framework'. This is a combination of waterfall, iterative and agile that looks at a value based delivery. Each deliverable goes into a fixed phased circle and continuous the cycle until it is finished for delivery. It can go through the cycles as many times as needed. Each cycle lessens the remaining work until it's done. This works for complete systems, as well for single features.
Depending on the problem definition, identification of the target group and the type of the solution defines which methodology to use. If you would like to start from complete scratch I suggest to start with Agile Manifesto as the ground rule and builds up experience and needs based on following the decrees in the Agile Manifesto.
Regardless of the methodology chosen, you will need to create a documented plan or schedule to explain what will be done when and by whom. Initially, this can be high level like phases or number of sprints of fixed duration. Ultimately, you will need a more detailed plan. Clients and users will ask when they are getting their stuff (that they paid for) and project team member will want to know when they need to be available to do their work on the project (so they can plan their other work and time off).
For the personal software project enthusiast, identifying the right set of guiding principles and practices - a methodology - is fundamental. The methodology you choose serves as your compass, providing direction on how to manage, organize, and implement your project. When it comes to managing personal projects, simplicity can be a virtue. Rather than implementing complex tracking systems, consider user-friendly tools like Trello for task management or GitHub for version control. Even a well-structured spreadsheet or regular journal entries can help you monitor progress and stay organized. Remember, the goal is to keep your focus on your passion, not to get overwhelmed with management details.
Gauging the Complex Complexity. Defining or gauging the complexity of a problem is very complex in itself. One of the preliminary steps people follow to define the complexity is to account for the unknowns. If it's something which have been accomplished in the past then that can be used as a reference but if it's something that has not been done before then the unknown becomes immeasurable. Complexity sometimes may not lie with the problem statement itself but the lack of prior experience, tools or technology to solve it. For an instance, the problem of traveling to Mars is a pretty clear problem statement but the complexity lies in the capability to achieve it.
The third step is to design the architecture of your software project. The architecture is the high-level structure and organization of your software, which defines how the components, modules, layers, and interfaces interact and communicate with each other. The architecture also determines the technologies, frameworks, tools, and standards that you will use to develop your software. Designing a good architecture is crucial for ensuring the scalability, performance, security, and maintainability of your software.
Along with architecture, having a support document explaining architecture is like adding a cherry on top of the cake. Many times, we don't have detailed steps explaining architecture, which may result in not executing the things as per the expectation. Adding a detailed step wise support document can help both technical and non-technical stakeholders understand the system in a better way.
Good application architecture is designed for change. This does not only include extendibility and modifiability, but also replaceability and removability. Some key characteristics: 1) Strong modularity Enforce boundaries with module access control (explicit imports / exports of components). 2) Business oriented decomposition Enhance understandability and scalability by using business oriented modules like products, orders, etc.. Avoid a technical decomposition (controllers, etc.). 3) Independence of frameworks Assure replaceability of frameworks by encapsulating them in wrapper components that can be safely used by (business) modules. This allows for building a (multi-team) monolith that can be split in (micro)services when needed.
Setting up the architecture is a critical step when it comes to personal software projects. Good architecture design ensures your software is scalable, efficient, secure, and maintainable. For personal projects, it's essential to balance these factors with simplicity and your comfort level with the chosen technologies.
Gather data on development timelines, defect rates, resource utilization, and customer feedback from past projects. For a detailed Project Metrics Report.
Architecture should focus on “what”, not “how”. Aim for loose specification of implementation details. You need a queue, great, but it doesn’t matter what tool or service will be used to provide the queuing functionality. The exception from the looseness goal might be a situation of a very constrained environment where you know that implementing team has an expert level knowledge of a single multi purpose platform (like a cloud) and you can rely on this expertise to achieve more challenging timelines - keep in mind the vendor lock-in this decision might create on the architecture level.
The fourth step is to code the functionality of your software project. This is where you implement the logic, algorithms, and features of your software, using the programming languages, libraries, and tools that you have chosen. Coding is the core activity of software development, and it requires a lot of creativity, skill, and discipline. You should follow the coding standards, conventions, and best practices that are relevant to your software project, and use version control systems, such as Git, to manage your code.
Understandability is key in quality software. My golden rule is to treat code as documentation. Clean code rules that I apply in this context are: * Use descriptive and meaningful naming (involve others for non-trivial things); * Be consistent (keep a shared terminology list); * Avoid magic numbers (use definitions); * Use comments only for providing context (explain why, not how). It's normal to get blinded by your own code, so code reviews are of the utmost importance!
Agile development helps to improve communication between the development team and the product owner. Because of this, we can have a greater focus on collaboration and feedback.
Embracing clean coding practices is vital for producing high-quality, readable, and maintainable code. Following coding principles like SOLID, DRY and YAGNI promotes clarity, reduces duplication and improves overall codebase health. Clean code is easier to understand, modify and extend.
Choose a suitable software architecture pattern. A software architecture pattern is a general and reusable solution to a common problem in software design. It provides a high-level structure and guidelines for your software system, such as how to decompose it into components, how to communicate between them, how to distribute them across different nodes or layers, etc. There are many software architecture patterns that you can choose from, depending on your project’s characteristics and needs. You can also combine or customize different patterns to create your own hybrid solution. Document and communicate your software architecture. Evaluate and validate your software architecture.
Coding and testing are not to be seen as one-time actions. Coding and testing is to be co-developed, initially to achieve the simplest of functionality, then expand their scope piece by piece, going back to the stakeholders and getting additional feedback. As they see the system growing in front of them, they are going to change their mind about what needs to be done. It's thus important to have a process where the team works on as few functionalities as possible at any moment, completing and delivering them, thus changes to stuff not yet developed will not affect timeline (unless it causes refactoring type of ripple-effect - which sometimes is a price that has to be paid).
The fifth step is to test the quality of your software project. Testing is the process of verifying and validating that your software meets the specifications, requirements, and expectations that you have defined. Testing also helps you identify and fix any bugs, errors, or defects that may affect the functionality, usability, or reliability of your software. There are different types of testing, such as unit testing, integration testing, system testing, acceptance testing, and others, each with its own purpose and scope.
I consider every automated test as a part of the application's shadow. When the application changes, its shadow has to change with it. To avoid unnecessary overhead I use the following question: are we out of business if this goes wrong? If the answer is yes, test it to the bone. For the rest, a linear scale can be used for defining the effort. If it hurts a lot, test a lot. If it doesn't hurt, don't test.
User Acceptance Test (UAT) also plays crucial role in validating the solution. This involves few selected users testing the solution in simulated or actual environment. Acceptance is taken before the solution is rolled out for all the users. Validation by real end-users helps ensure that the software meets their expectations
Don’t let perfection be the enemy of progress. While ensuring your software is bug-free and meets your outlined specifications is important, speed also matters. Getting your product out to your customers swiftly for real-world testing is invaluable. Feedback from actual users often uncovers insights that traditional testing can miss. Plus, the sooner you can bring your product to market, the sooner you can start generating revenue. Remember, every step taken should be a step closer to earning that first dollar.
Testing is not a step. Or at least it should not be only a step. Testing should be continous. "test the quality of your software project" Did a human write this? Did the author mean "test the quality of your software"? However, there is truth spoken in error, and there is potential value in assessing the quality of the project. This is where retrospectives come into play, at least as far as the project team are concerned. Other stakeholders may also wish to assess the quality of the project - is it financially viable? is it a zombie project? (I have a story about one of those...)
Testing the quality is an integral part of software development, encompassing several levels of testing. Firstly, unit testing ensures the consistency of introduced interfaces. Next, smoke testing provides quick feedback on freshly delivered code, typically examining the correctness of coded state machines. Following smoke testing, integration testing verifies the solution's ability to execute its main function. Subsequently, the software solution should pass through a quality gate that assesses its behavior in various edge case scenarios. Lastly, security testing has become a necessary step for highly available software solutions.
The sixth and final step is to deploy the software project. Deploying is the process of delivering and releasing your software to the intended users or customers, either on-premise or in the cloud. Deploying also involves configuring, installing, updating, and monitoring your software, as well as providing support and maintenance. Deploying can be done manually or automatically, using tools such as Jenkins, Docker, Kubernetes, and others, depending on the complexity and frequency of your software releases.
In my opinion, in personal projects, where the goal is idea validation, you may not need a highly scalable architecture from the get-go. Opting for third-party services might be a smarter move, even if they come at a higher cost. The objective here is to get your software into users' hands quickly and efficiently, allowing you to gauge its viability in the marketplace.
It is beyond the scope of this post to discuss the cost and benefits of the different deployment methods available to you. No matter what you choose, use an industry-standard method to deploy your application. The only code you should have that is proprietary is the code you are selling. The benefits: 1. Someone else has already worked out the kinks in the deployment process. 2. Got a question? Use your favorite search engine and find the answer. 3. There are great tutorials like the ones here on LinkedIn to learn the best practices of your architecture/ deployment methods of choice. 4. Hiring a new engineer? If they don't know your deployment process they can learn it fast.
CODE REVIEWS 1.) Code reviews should enforce the business' software development SOPs, not infringe upon a software developer's individualism. 2.) Software should never go to production without having a second set of eyes reviewing what has changed. However, that second set of eyes should have a clear understanding of what is being reviewed. If a developer isn't sure about what the modifications or new applications are supposed to accomplish, and the code reviewer doesn't understand either... that's a recipe for disaster. 3.) Code reviews should not be personal. Leave opinions as to how it should have been coded out of the equation. The code reviewer should ensure that SOPs are followed and that the code accomplishes what was expected.
Assume you will publish your software from day one and treat all changes as if they were publicly visible. That should affect your approach to security, test coverage, documentation, readability, accessibility, setting clear expectations. After all if you write software for user base of one (you) is it really worthwhile to spend your time on it? Publicizing also means deployment (or distribution). Make your software ready (and easy) to deploy from day 1, use broadly adopted standards, consider popular target platforms, i18n, accessibility. Specify your deployment (or installation) guide assuming your target audience is someone with no knowledge about your software but with understanding of common practices and tools.
DevOps capability needs to be considered at the inception of project, if you have it use it. Also consideration of how you deploy subsequent updates in an evergreen model. Use of SOC capabilities in Cloud, prevent unwanted intrusion. Employ best practice in terms of elasticity, balancing the cost element Monitor, Monitor, Monitor - Use of AI tools for remediation should also be considered. (Log aggregators, Dynatrace etc...)
1. Understand the customer 2. Understand the experiences required by customer 3. Understand the purpose and specific goals of the project 4. Try to define a set of Must Haves and Must Nots around the project 5. Now get a PRD from the customer/sponsor or get it aligned with the customer 6. Based on the PRD, align on cost, quality, schedule with the sponsor. 7. From now on, it's all standard software project management practices based on the methodology.
As an engineering leader starting a new software project, I recommend the following key steps to consider: 1. Defining clear project objectives and requirements, involving stakeholders and aligning their expectations to set the project on the right path from the beginning. 2. Building a capable and motivated team, fostering effective communication, and implementing an agile project management approach to adapt to changes efficiently, which can contribute to a successful software project delivery.
Spend enough time in Requirement analysis. Be thorough with use cases. Ensure Use case diagrams are created, reviewed by stakeholders. More you spend time in understanding requirements, less you spend time in architecture, design, coding, testing, etc... Also think about documentation, pricing, maintenance efforts.
I usually engage my shaman and schedule a ayahuasca retreat for the team as part of the project kickoff. Scope and boundaries are usually crystal clear after this.
A key part of a software project startup is identifying the stakeholders, both internal and external, and determining their role and influence. Will they encourage the project success or introduce obstacles. You also need determine how to communicate with these stakeholders and manage their expectation. Even if you deliver the perfect software, if the stakeholders don't agree that is satisfies their requirements or expectation, the project will not be considered to be successful.