What do you do if your algorithm-based employee performance evaluation is yielding inaccurate results?

If your employee performance evaluation is yielding inaccurate results, it's crucial to investigate why. Look into the evaluation criteria, the process, and any potential biases or errors. Then, consider implementing changes to improve accuracy, such as clarifying expectations, providing better training for evaluators, or using multiple sources of feedback. Transparency and open communication with employees about the evaluation process can also help address any concerns and rebuild trust.

Borrowing SIPOC terminology from Six Sigma - scrutinizing Suppliers, Inputs, Process, Outputs and Customers individually can yield results in the identification and further analysis of the underlying problem.

Make a thorough review of the algorithm's inputs and logic which involves data check, algorithm review, feedback Incorporation, iterative adjustments and communicate any changes or improvements to employees and stakeholders.

Can algorithms find your soulmate? Similarly algorithms can not improve your performance. The error is in our thinking, not in code or data. We need to fine-tune our cognitive model.

Whenever we get inaccurate results from the model, identifying errors in our processes and data is the key to solving the issue at hand. We first have to see what kind of data we're handling and if there are any missing data(NaNs) or if the data is skewed or biased and how many examples are contained in each of the classes. This identification can be done programmatically as well using some tools and techniques like pandas, etc., since manually it will become very tough and tedious.

In my experience, the kind of metrics we would define for a problem is very important. Suppose, for example, if we're working on an segmentation problem and the selected metric is mAP(Mean Average Precision) instead of IOU(Intersection over Union), then the total evaluation would fail since the mAP is a good metric for object detection kind of problems. Similarly, in this case, using the most aptly describable metric for this particular problem would be vital. It is also good to experiment with various similar kind of metrics and benchmarks to get a clear picture of how the model is behaving currently.

Algorithm Tuning and Fine-tuning is of paramount importance. As the data changes, the current best performing model might not be well equipped to handle the newer kinds of data. In such cases, fine-tuning of the model will be required using the proper hyper parameters like setting the learning rate, experimenting with other optimizers, weight decay value and scheduling the training so as to stop it at the right epoch and save it properly. Even data augmentation, can be a very effective technique to adapt to the changing circumstances and data.

In my experience, the incorporation of a good feedback loop plays a significant role in making the model production ready. Having a system which is capable of identifying the problems and report them becomes very useful. The feedback loop can be of two types, one automated pipeline like CI/CD pipelines which would continuously operate in the background and log the data from the model. The other one would be to have a separate dataset altogether to test the model's actual performance on unseen data.

Algorithms is not the alternative of Human communication. Communication can do wonder and I experience everyday. Belive in human not in bots. They are only tools to assist you.

People often use their gut feel and dump algorithms. Today AI is a great help to modify the algorithm and weed out any inaccuracy. After you get the result of the algorithm, you may use your judgment. But be very careful in doing that.

In my experience, no matter what the task or challenge at hand is, ultimately the supervision of a human is of utmost importance. If the human is having a good knowledge of the problem that he is solving, then that becomes the single most advantageous situation since he would be able to select the right kind of parameters, model architecture, dataset and evaluation metrics. He would also know how to augment the dataset in order to increase the size of the dataset, thereby increasing the model's performance. In case of persistent biases in the model, he would be able to rightly detect those and correct them at the earliest.

If you need algorithms to measure performance of your employee, then be sure that your organisation is going through a tough time. Employee is not machine, what you can measure via algorithms. It is all about trust and relationship. Invest in relationship not in algorithms.