Machine learning models are trained on annotated data. In perClass Mira, the concept of testing is strictly refering to evaluation of model performance on unseen examples. We would like to stress, that data used for testing should never be comprising the same of very similar physical objects as the ones used for model training.
In perClass Mira, images can be flagged for testing. This means, that any subsequent model retrainning will not use these images for any of the steps.
The software also provides extensive support for cross-validation used in model comparison. Cross-validation splits the data set into trainng and test parts multiple times. Each time, a model is built and the performance estimated. Eventually, we end up with a mean and standard deviation of model performance. This simplifies comparison of different models based on statistical significance.
Cross-validation is supported both over images and also over groups defined by file names (for example over days of scanning, varieties, scanning with replicas and others)
NOTE that flagging an image only does not change already existing models. The user needs to explicitly retrain a model or perform new model search in order for the new image falgs to take effect.