What are the best practices for managing large-scale algorithm development projects in OR?

Focus should be on understanding the problem and the business issues. So it the problem that will define the algorithm development and not that develop a an algorithm and try to fit a problem to that. If we understand the business issues appropriately, then we may find some of the business issues can be solved by optimization, some by forecasting. So we should not say it is optimization problem, forecasting, but this business issues can be effectively addressed by optimization problem. It is possible that some of the other business issues can be addressed by forecasting or Markovian model. In my opinion understanding the business issues will define the problem and objective.

In managing large-scale algorithm projects in Operations Research (OR), it's crucial to embrace innovative approaches. Instead of solely focusing on traditional practices, consider incorporating cutting-edge techniques such as machine learning, data mining, and optimization algorithms. Leverage cloud computing for scalability and harness the power of big data analytics to gain valuable insights. Foster a culture of experimentation and encourage team members to think outside the box. Embrace agile methodologies to promote flexibility and adaptability throughout the project lifecycle. By embracing innovation, you can unlock new possibilities and achieve groundbreaking results in your algorithm development projects.

Essa forma de definir um problema n?o é exclusiva da PO, é um processo que deve ser aplicado a qualquer modelo que envolva solu??o de problemas.

This step by far is the most important one. It requires to properly understand the business need & translating it into a mathematical model and constraints. Not only you would need to have proper understanding of the business, you are required to have skills mathematics. This step is what differentiate a highly experienced OR expert from someone who just started. Doesn’t matter how many OR solver or optimization library you know. If you are solving the wrong problem that has no value to the business. Once the objective functions are defined, there are many people who can use software to find the solution using a library. But not everyone is experienced enough to actually come up with the proper model for the problem.

The first step is understand what is the business issue you need to solve, and so define the project objective. If you know what is the problem you can choose the metrics to measure performance related to the project objective.

Planning the project also allows the team to determine the critical path, longest stretch of time to complete dependent tasks. Typically, data collection tends to be on the critical path. What if the data is readily available? At that point data collection may no longer be on the critical path. However, the team cannot make such determination without planning the project and resources

Planing is of important in every project, knowing the data, tools, equipment, time etc needed to solve the problem that is initially well define.

Interessante passo a passo sobre pesquisa e projeto de algoritmos. Iniciando pelos objetivos, diagnóstico, desenvolvimento, comunica??o, e aprendizado.

Project planning should address delivering at least a rough solution prototype to the client as soon as possible. Clients are typically not good at formulating the requirements to the OR solution, and user iterating with prototype is the most reliable source of real business requirements.

Defining the problem is critical, but solving it requires ethical alignment—especially in large-scale AI projects. In building AI-powered systems for disability outcomes, I’ve learned that clarity on objectives must align with ethics. Key practices include: Ethical Design: Following frameworks like UN and UNESCO to avoid bias and ensure equity. Purposeful Scalability: Scaling systems to serve underserved populations, not just privileged ones. Transparent Evaluation: Measuring not just performance, but fairness and impact. Clarity is vital, but aligning objectives with ethics ensures real, equitable solutions

One advice in designing the algorithm: start small. Come up with a formulation, then come up with a tiny instance of the problem and see if it works. Very often you'll see a failure - thanks to the small problem size, it will also be easy to see its reason and come up with remedies.

In my experience, a mathematical formulation of the problem space with variables, constraints and objective function should be first thought through and documented before initiating development. Reviewing the layman version of it with the stakeholders to validate business objectives, assumptions and levers/parameters would ensure the business processes involved have been modeled correctly to avoid any surprises down the line.

In my experience it is a good practice to prioritize Explainability and Interpretability: Design algorithms that are not just "black boxes" but offer insights into their decision-making processes. This builds trust with users, facilitates debugging and improvement, and allows for human oversight where necessary.

Two thumbs up for highlighting uncertainty here! This is the key issue that OR typically glosses over. At least since the advent of InsideOpt Seeker there is no more excuse for not actively incorporating data uncertainty in the OR model. It is incredibly important to do so to avoid that the OR model creates plans that do not execute well as soon as the real data deviates slightly from the estimates. Brittle plans cost serious money.

get a sense of size you are confronted with. you ll be a the fork point: you want to do this or raise the white flag. the sooner you know, the healthier it is. before that, often times, the problem statement is vague and obscure at best (RFPs....). formulation cuts the chase and IS necessary : it does map out the data you are told to have, or promised to be given, so formulating consumes the data you are told to be given into a story to be solved. if only for that, that clarifies what is it you are about to embark in. when you bypass the moment of fear (this is too big to solve), well, you go about scraping it first, what would happen if some decisions were made in a first stage first, and there you go... things start loosening up....

- Go beyond traditional metrics: Don't solely rely on accuracy or cost savings. Develop and use metrics that align with the project's specific objectives and consider other critical factors like user satisfaction, fairness, and environmental impact. - Stress test and conduct sensitivity analysis: Explore the algorithm's performance under extreme conditions and different data scenarios. This identifies potential weaknesses and vulnerabilities before real-world deployment.

This is the engine room, this is where you know if the algorithm fit well to the problem, there is no overfitting or under fitting. Is very important for an algorithm to be test to know it effectiveness.

- In addition to full documentation of the project/algorithm, it’s also good to document lessons learned and hurdles encountered throughout the project. Doing so provides insights how to assess future or similar projects, including finding alternatives to make any models run faster, smoother, or more efficient.

Each scientific, numerical, and computational software solutions need at least 3 sets of documentation: (1) User Guide: how to use the current software solution, what sort of problem can you solve with it? What sort of problem you cannot? What are the expectation and assumptions for the model and the inputs. Avoid using any math in this document unless absolutely necessary and or if it is a simple one, may be unit conversion. (2) Developer Guide: talk about how the code is structured! how to modify it? How to add to it! How to debug (3) Technical Documentation: you can let yourself go deep on the mathematics of the algorithm; Don’t hold back; go into details of how mathematically everything is computed.

1. Need to speak to end users: With respect to developing algorithms for Operations, it is important to write a document that can be followed by people who do not have the similar technical vocabulary as people who develops, and maintains the algorithm. 2. How to understand the results?: End user of an algorithm rarely cares about methods, models used at backend. They are more interested how to read and comprehend the results. This calls for a nuanced understanding of customer perspective and actual customer behavior. 3. Murphy’s law wins: “Anything that goes wrong will go wrong” We can write pages to make sure users don’t misunderstand. They will misunderstand, and will find a use case not in objectives. Being humble goes a long way.

Present the algorithm as a tool, not a solution. Emphasize that the algorithm is a powerful tool that aids decision-making but ultimately empowers users to make informed choices based on their context and values.

The monitoring should include an assessment of the actually incurred operational results vs. the "optimized" results on paper. When the operational plan was developed, there was an objective function and a metric to assess the quality of the plan. This assessment crucially must be compared with the actual results by executing on these plans in real life. If there are deviations this might show that we are optimizing the wrong objective or that we are not handling the uncertainty in our forecasts sufficiently well.

The business needs constantly changes; you would need to adapt your method to stay relevant to the business. Moreover, the results of your earlier models bring insights and allows the biz customer and you to understand the problem better. Things that you might have overlooked, needs to be addressed. Also during the course of using the model with different inputs, you become aware of many degenerative cases that your algorithm doesn’t handle properly; you would need to fix those as well.

Economic Lens: Track cost-effectiveness and return on investment: Evaluate the algorithm's economic impact and justify its continued development and maintenance. Explore market dynamics and user preferences: Adapt the algorithm to evolving market needs and user behavior to maintain competitiveness. Seek strategic partnerships and collaborations: Leverage external expertise and resources to accelerate optimization and innovation.