How can you determine the reliability of decision support tools?

1 Check the source

The first step is to check the source of the DST, which means finding out who developed it, what their credentials and reputation are, and what their purpose and motivation are. You want to use DSTs that are created by reputable and trustworthy organizations, such as professional associations, academic institutions, or government agencies. You also want to avoid DSTs that are sponsored by commercial entities, biased groups, or hidden agendas, as they may have ulterior motives or conflicts of interest. You can check the source by looking at the website, the about page, the contact information, and the references and citations of the DST.

2 Evaluate the design

The second step is to evaluate the design of the DST, which means examining how it works, what it does, and how it does it. You want to use DSTs that are transparent, user-friendly, and adaptable. You also want to avoid DSTs that are vague, complicated, or rigid. You can evaluate the design by looking at the features, the functions, and the methods of the DST. For example, you can check if the DST has clear instructions, feedback, and explanations; if it allows you to customize the inputs, outputs, and parameters; and if it uses reliable and valid data, algorithms, and models.

3 Test the performance

The third step is to test the performance of the DST, which means trying it out, using it, and assessing it. You want to use DSTs that are accurate, consistent, and relevant. You also want to avoid DSTs that are erroneous, inconsistent, or irrelevant. You can test the performance by looking at the results, the outcomes, and the impacts of the DST. For example, you can check if the DST produces correct and reliable answers, recommendations, and solutions; if it generates similar and stable results under different conditions and scenarios; and if it addresses your needs, goals, and expectations.

4 Compare the alternatives

The fourth step is to compare the alternatives, which means looking at other DSTs, options, and sources of information and advice. You want to use DSTs that are complementary, compatible, and corroborative. You also want to avoid DSTs that are contradictory, incompatible, or isolated. You can compare the alternatives by looking at the similarities, the differences, and the agreements of the DSTs. For example, you can check if the DST matches or supplements other DSTs, tools, or methods; if it aligns or conflicts with other sources of information, evidence, or expertise; and if it confirms or challenges your intuition, judgment, or experience.

5 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?