Machine learning models are trained on annotated data. In perClass Mira, the concept of testing is strictly referring to evaluation of model performance on unseen samples. We would like to stress that data used for testing should never be comprising the same or very similar physical objects as the ones used for model training. 


In perClass Mira, images can be flagged for testing. This means, that any subsequent model retraining will not use these images for any of the training steps.


The software also provides extensive support for cross-validation used in model comparison. Cross-validation splits the data set into training and test parts multiple times. Each time, a model is built and the performance estimated. Eventually, we end up with a mean and standard deviation of model performance. This simplifies comparison of different models based on statistical significance.


Cross-validation is supported both over images and over groups defined by file names (for example over days of scanning, varieties, replicas, and others).


NOTE: Flagging an image does not change existing models directly. The user needs to explicitly retrain a model or perform a new model search in order for the new image flags to take effect.