How do you design and document algorithms for maximum efficiency?

I prefer to define a "problem" as a feature which your software solution is expected to deliver that can be used by the business team. When we talk about a problem wrt the context of a product, we need to define the scope of our expected solution, which includes a happy path, technical validations (duplicates, schema checks ...), business validations (idempotency, cross field validations ...) and edge case scenarios. Sequence diagrams generally help in visualising the control flow, which should ideally contain the different user personas involved, the service(s) involved and a high level view of the technology infrastructure. A "Given-When-Then" syntax also helps in figuring out the possible expected behaviours of our proposed solution.

To write a good document of a solution(in this case an algorithm) you have to lead the readers through the problem which made you create the solution. Without that context, the information becomes half baked. 1. So first you need to state the problem 2. Other algorithms you considered. 3. What were the pros and cons of those and why it dint feel "enough" 4. Your algorithm with flow diagram and explanation 5. Pros and cons of the algo 6. How it fits well into the problem statement. 7. What are the bottlenecks still, if any in the algorithm. 8. Miscellaneous 9. FAQs.

Visualizing the problem statement is a great way to really enter into the problem. Most complex algorithms tend to get tricky as the number of dimensions in the data begin expanding. A great tool can most of the times be a whiteboard or big sheet of paper and a colleague with an open ear to bounce your ideas of. Sometimes just saying the problem statement and your expected solution out loud lets you have a clearer picture

The biggest drawback at beginners is to define problem. Especially developers try to write down code immediately and try to see solution. They define problem and prepare documentation after it works. This is due to lack of confidence and not to know how to define problem before solution. The first job that I give to juniors is to right down problem as requirement and then draw a flow-chart. They always have difficulty in doing this because they never do this in schools. Then, we call somebody else if they could understand the problem from written flowchart. After they finished coding we return to flowchart to review and check over-written or not covered parts.

Asking questions can help you clarify the problem statement, including edge cases. Making assumptions while writing them down is okay to ensure everyone is paying attention to them. It is important to remember that every system has its own limitations and data. So, it is important to design an algorithm for the specific system you are building; you can draw from past systems you developed; just make sure to adjust it to this one. This is when problem definition is critical, and asking clarifying questions can help you bring clarity.

RFC ( Request for Comments) is a great document to articulate your solution and focus you on the right approach. This is a known best practice for designing components/solutions. In the document, you will write the: problem definition, constraints (time, space limitations), tradeoffs between speed, memory and scalability, data structures, data access patterns, design patterns for execution speed, memory usage, readability, and maintainability. Other components such as edge cases handling, error handling, and people collaborating on the solution, including code review and comments. Lastly, the RFC approach enables you to share your rational design decisions by having a clear document on which people can comment and collaborate.

At this stage I'd rather advice to start with code snippets and toy models. This is the safest and quickest way to understand the ways to approach the right solution

Its good to start with brute force approach, that is a working solution. Once that is done, we can examine scope of reduction in space/time complexity, scope of application of an algorithmic pattern etc, and then move towards an optimized solution.

Try not to overengineer the solution by going beyond the problem definition, which might result in an overly complex or inefficient solution.

Just skip this step and start experimenting. I found that a lot of 'optimal' solutions from literature are hard to parallelise. So, if you have a multi-core environment it often helps to look at the more simple existing solutions and see if you can apply parallelisation.

When starting coding, think through how to break your code into manageable components and outline the steps your algorithms take; remember to adhere to code standards and conventions; here, you can use a linter to assist you during coding. Use meaningful names, code modularity, and build tests to verify each component in isolation and together. Remember to seek peer review of your code, which will help you catch errors and promote knowledge sharing within the team. Every feedback/constructive criticism is an opportunity to learn and improve.

A good way to design code is to use pseudocode and visual diagramming such as flowcharts. These can integrated into supporting documentation or better still, the coding notebook environment then be peer reviewed for accuracy of logic. State any assumtions and potential biases and validate the results of independent testing prior to any further iterations and version control (e.g. via a GitHub repository).

Keep testability in mind. Nothing breaks the spirit more than having to test through a long pipeline just to test the last stepm, especially if it involves reading/writing database, external systems or other side effects. The more complicated the program, the harder it is to test it thoroughly. A good trick of the trade is testing different parts in isolation, and quickly. In order to do that, every function should be "pure", which means it causes zero side effects and output is fully defined by its parameters. That way one can run hundreds of tests very quickly and get immediate feedback if something broke. This is especially important for "what comes after". e.g. when you have finished the progam and someone else needs to update it.

Using assertions often helps to enforce the integrity and maintainability of your code. They can highlight logical errors and prevent breaking the algorithm by inconsistent updates.

I always suggest developers that to write their code, algorithm as their colleagues could understand and able to continue their work. I always ask, will you able to debug your code two months later? Can any tester able to test your code without looking any additional documentation? Sympathy with other developers and testers are key here.

Documentation is not just a formality; it's a vital part of your algorithm's lifecycle. If you created an RFC, you are halfway there. Remember to detail the input and output description, assumptions, and limitations, complexity, security aspects, maintainability, production observability/alerts system, and CI/CD process for your algorithm. Also - version control your document; as your code version evolves, you would like to know which document is for which, so remember to use that. Also, you can use automated documentation tools such as Javadoc, Doxygen, etc, to save time and reduce the risk of inconsistencies between code and document.

Nothing specific to algorithms here. If you applied an existing solution, but wrote your own implementation properly refer to the original problem/solution. I am not a fan of documentation that is automatically generated from code. If useful documentation can be generated it means that your original code is already readable and the 'documentation' is just duplication. Only comment the things you cannot communicate via correctly naming, typing and designing your code. That said, algorithms usually contain non-obvious optimisations and sometimes it is inconvenient to abstract such an optimisation behind a function with a name that clearly describes what it does. Use comments in these rare cases.

I would suggest starting the documentation process earlier, probably with the problem definition, and gradually improving it to reflect each step. Waiting until the last moment to start the documentation process might result in missing key design information. Try to use documentation comments in the code so that API documentation can be easily autogenerated.

There at least to given references to requirement documents or problem definition within code unless there is rules for documentations in the company. The biggest improvement in documentation is keeping change logs and versioning your code.

Many times, you will need to refactor your algorithm following new learning from production or benchmarking. Be open to that and think about simplifying complex logic, removing redundancies, and cleaning your code!

Reviewing can be done in two different manner. The first one is technical design review, done by a team members from same and other projects, lead engineers if possible by analyzing functionality with other functions or system altogether. The latter is for optimizing.

The vast majority of algorithm implementation are already going to be optimized in a library of the language that you're using. In addition, many solutions have an obvious algorithm to use. In industry, the engineer usually isn't looking for a speedup on the level of time complexity but rather factors like 2x or 10x. If the algorithm is for an industry application, optimizing in this manner is going to be the most important step in ensuring success.

By understanding the scope of the product and the target audience, you can get a better understanding of the problem that the algorithm is trying to solve. This will help you to identify the most effective solution. It is also important to start with a basic solution and then improve it over time. This will help you to identify the issues that you need to address in order to improve efficiency.

Learning by example is a great way to understand standards for documentation. Be it in an organisation trying to follow their standards, or establishing your own standard of documentation.

Pick your battles intelligently. In vast majority of cases the performance is dominated by IO (input and output), such as reading the database or making calls over the network. In information systems, the cases where the code itself must run fast are not as common (unless you're working on embedded systems where raw performance is crucial, even in AI one must often optimize IO above all else). Modern software also contains a lot of code. All that must be easily understandable, testable (think how to check one small piece at a time?) and changeable. This maintainability and readability is often far more important than small optimizations.