Keywords: detectors, cascade of classifiers
Problem: How to build a classifier hierarchy?
Solution: This simple example illustrates a classifier hierarchy where a detector is followed by a classifier
Hierarchical classifiers combine several separately trained classifiers by decision-level rules. A commonly used example is a detector-classifier cascade where only the data samples identified as targets by the detector are processed by the classifier. This may be useful when we want to protect our classes from outliers. Instead of building a discriminant that would classify all possible values of a new sample vector, we define the region in the feature space where the data might be using a detector. The discriminant will be trained only in this area.
Given a two class problem, a Gaussian model is trained on the all data, the reject
option requires that the estimated Gaussian rejects 10\% of the data.
>> load fruit; a=a(:,:,[1 2])
'Fruit set' 200 by 2 sddata, 2 classes: 'apple'(100) 'banana'(100)
>> pd=sddetector
(a,'fruit',sdgauss
,'reject',0.1)
sequential pipeline 2x1 'Gaussian model+Decision'
1 Gaussian model 2x1 one class, 1 component (sdp_normal)
2 Decision 1x1 thresholding on fruit at op 1 (sdp_decide)
Note that since the target class fruit
was not present in the original data, all data points are used. Next, we need to train a discriminant between both classes. Here, for example, a mixture of Gaussian with 3 components per class:
>> p=sdmixture
(a,'n',3);
>> pd2=p*sddecide
sequential pipeline 2x1 'Mixture of Gaussians+Decision'
1 Mixture of Gaussians 2x2 2 classes, 6 components (sdp_normal)
2 Decision 2x1 weighting, 2 classes, 1 ops at op 1 (sdp_decide)
Now we can construct the cascade pipeline. We first provide the top-level classifier executed on all data samples (for us, the detector pd). Then, we specify what decision passes to the next stage (here fruit
) and which classifier will be executed on such samples. Finally, we visualize the cascade decisions:
>> pc=sdcascade
(pd,'fruit',pd2)
2-stage cascade pipeline 2x1 (sdp_cascade)
>> sdscatter
(a,pc)
Note the warning message reminding us that cascades are built from classifiers returning decisions, not from soft outputs.
The Knowledge Base article Detector classifier cascade with ROC analysis provides a more advanced example of a cascade. At each stage, the classifiers operating point is set using the ROC analysis.
More complicated hierarchical classifiers can be constructed by adding further pairs of decisions and trained classifiers using sdcascade
.