By default the confusion matrix is estimated from the training set, this means from all images that are not flagged as a test.



In order to properly evaluate any classification solution, we need good performance on the independent test set. The reason is that we need confirmation of generalization capability of our classifier on example unseen in training. In perClass, testing is defined on the level of images, images flagged as a test are not used for training the classifier.


NOTE: In order to take any change in image flags into account, we need to retrain the model!


To estimate the test set confusion matrix, we switch to the Test set tab . In our example, we will observe that the confusion matrix is not complete. This is because not all classes were represented (labeled) in our test scans.



After we add relevant labeling to the two test scans, we need to switch to the Training set confusion matrix and then back to the Test set matrix to update it:



Note that we do not observe the same misclassification of "leaves" into "virus" as in the training set. In addition, we can see higher errors in two new fields: misclassifying some "stem" as "leaves" and "background" as "virus".


It is very difficult from the performance estimates only to judge to what extent are these relevant errors. We need to understand what pixels these misclassications represent in the image. This is greatly simplified by perClass Mira Errors tool and the Current image confusion matrix.