Keywords: detectors, cascade of classifiers

Hierarchical classifiers combine several separately trained classifiers by decision-level rules. A commonly used example is a detector-classifier cascade where only the data samples identified as targets by the detector are processed by the classifier. This may be useful when we want to protect our classes from outliers. Instead of building a discriminant that would classify all possible values of a new sample vector, we define the region in the feature space where the data might be using a detector. The discriminant will be trained only in this area.

Given a two class problem, a Gaussian model is trained on the all data, the reject option requires that the estimated Gaussian rejects 10\% of the data.

>> load fruit; a=a(:,:,[1 2])

'Fruit set' 200 by 2 sddata, 2 classes: 'apple'(100) 'banana'(100)

>> pd=sddetector(a,'fruit',sdgauss,'reject',0.1)
sequential pipeline     2x1 'Gaussian model+Decision'
 1  Gaussian model          2x1  one class, 1 component (sdp_normal)
 2  Decision                1x1  thresholding on fruit at op 1 (sdp_decide)   

Note that since the target class fruit was not present in the original data, all data points are used. Next, we need to train a discriminant between both classes. Here, for example, a mixture of Gaussian with 3 components per class:

>> p=sdmixture(a,'n',3); 
>> pd2=p*sddecide
sequential pipeline     2x1 'Mixture of Gaussians+Decision'
 1  Mixture of Gaussians    2x2  2 classes, 6 components (sdp_normal)
 2  Decision                2x1  weighting, 2 classes, 1 ops at op 1 (sdp_decide)

Now we can construct the cascade pipeline. We first provide the top-level classifier executed on all data samples (for us, the detector pd). Then, we specify what decision passes to the next stage (here fruit) and which classifier will be executed on such samples. Finally, we visualize the cascade decisions:

>> pc=sdcascade(pd,'fruit',pd2) 
2-stage cascade pipeline 2x1   (sdp_cascade)    
>> sdscatter(a,pc) 

Note the warning message reminding us that cascades are built from classifiers returning decisions, not from soft outputs.

The Knowledge Base article Detector classifier cascade with ROC analysis provides a more advanced example of a cascade. At each stage, the classifiers operating point is set using the ROC analysis.

More complicated hierarchical classifiers can be constructed by adding further pairs of decisions and trained classifiers using sdcascade.