Categorical Accuracy for one-hot labels: Tensorflow/Keras to PyTorch

You have most likely found yourself in situations where you are building a model in PyTorch but the academic paper is in Tensorflow/Keras.

Not all code migrations are straight or there is an equivalent function. This is the case of "Categorical Accuracy for one-hot labels". Let's first see what one-hot is.

One-hot labels

Let's suppose your training dataset includes a set of people photos and their sex. The metadata is a csv (comma-separated values) file with the following structure:

Dataset sample is a total of 8 rows where prediction output would be Sex column:

Sex prediction expected values would be 1 (female) or 0 (male). However your designed classification model (for example a convolutional network CNN) has 2 output classes, therefore you need to transform the metadata. This is where 'one-hot labels' comes to play.

From the above, all males (0) were transformed into [1, 0] and therefore all females (1) were transformed to [0, 1]. Now with the expected output transformed you can choose Categorical Accuracy as the metric for training.

Categorical Accuracy

Calculate the percentage of values predicted by your model (y_pred) that match the expected values (y_true) for each unique label. This in other words translates into the following.

From the previous result following the Categorical Accuracy calculation we have 5 correct and 3 incorrect predictions.

For example, from first row [-0.0181, 0.1989] we have 0.1989 which is greater than -0.0181 therefore the model predicts the class or image is closer to being female (column 2) and this match with our ground truth (y_true) which for the first row the expected value is [0, 1]

Finally we calculate Categorical Accuracy by comparing y_pred (model prediction) and y_true (expected prediction).

For this first iteration the categorical accuracy value would be 0.625 (5 out of 8 correct predictions)

Categorical Accuracy: Tensorflow/Keras

The output value is as expected: 0.625

Categorical Accuracy: Pytorch

Because there is not equivalent function, we need to code it our self.

The output value is as expected: 0.625

Conclusions

You can access the code for this notebook on GitHub: https://github.com/karelbecerra/ai-ml-dl-samples/tree/main/pytorch/categorical-accuracy-one-hot-labels