How do you use evidence in your algorithm research?

1 Evidence types

Evidence in algorithm research can be broadly classified into three types: theoretical, empirical, and practical. Theoretical evidence is based on mathematical proofs, formal models, or logical arguments that show the correctness, complexity, optimality, or other properties of your algorithm. Empirical evidence is based on experiments, simulations, or benchmarks that show the performance, scalability, robustness, or other aspects of your algorithm. Practical evidence is based on applications, case studies, or user feedback that show the relevance, usefulness, or impact of your algorithm.

2 Evidence sources

Evidence sources are the data, tools, or methods that you use to obtain and present your evidence. Depending on your research question and goals, you may need to use different sources of evidence, or combine them in a coherent and consistent way. For example, you may need to use existing data sets, generate synthetic data, or collect real-world data to test your algorithm. You may need to use standard software libraries, implement your own code, or use third-party tools to run your experiments. You may need to use statistical tests, graphical plots, or qualitative analysis to report your results.

3 Evidence quality

Evidence quality is the degree of confidence, validity, and reliability that can be assigned to evidence. It is influenced by the design, execution, and analysis of experiments, the selection, representation, and interpretation of data, and the comparison, discussion, and generalization of findings. To ensure high-quality evidence, best practices should be followed. These include defining clear and specific research questions and hypotheses; using appropriate metrics and indicators; selecting sufficient samples and repetitions; using fair baselines and competitors; employing rigorous methods and procedures; reasoning and arguing objectively; employing clear language and notation; and being honest and ethical in conduct and reporting.

4 Evidence communication

Evidence communication is the process of conveying your evidence to your target audience, such as your peers, reviewers, or users. It involves choosing the most suitable and effective formats, channels, and styles to present your evidence in a clear, concise, and convincing way. For example, you may need to use tables, figures, or charts to illustrate your data and results. You may need to use pseudocode, algorithms, or code snippets to describe your methods and procedures. You may need to use abstracts, introductions, or summaries to highlight your main contributions and findings.

5 Evidence evaluation

Evidence evaluation is the process of assessing the strengths, weaknesses, and implications of your evidence, as well as the feedback and criticism that you receive from others. It involves reviewing, revising, and refining your evidence, as well as acknowledging, addressing, and resolving any limitations, errors, or gaps. For example, you may need to use peer review, replication, or validation to verify and improve your evidence. You may need to use comments, questions, or suggestions to identify and fix any issues or flaws. You may need to use discussion, conclusion, or future work to draw and suggest any implications or extensions.

6 Evidence integration

Evidence integration is the process of synthesizing and combining your evidence with the existing knowledge and literature in your field. It involves relating, comparing, and contrasting your evidence with the previous and current work of others, as well as identifying and filling any gaps or opportunities. For example, you may need to use literature review, citation, or referencing to situate and contextualize your evidence. You may need to use analysis, critique, or synthesis to evaluate and integrate your evidence. You may need to use contribution, novelty, or significance to demonstrate and justify your evidence.

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?