Cheat-sheet- B.1. Introduction
- B.2. Installation
- B.3. Nut classifier FREE demo example
- B.4. Activating perClass Demo
- B.4.1. Activating perClass Runtime
- B.4.2. Activating perClass Toolbox
- B.5. Building nut classifier
- B.6. Training Peanut/Amandel classifier
- B.7. Exporting perClass classifier for exacution from Cubert Utils
- B.8. Where to go from here?
B.1. Introduction ↩
This tutorial explains how to build classification solutions with perClass software package to develop classification solutions integrated with Cubert spectral cameras.
B.2. Installation ↩
Cubert Utilities 2.0.4 (Fuchsia) contains perClass 5.1 installation.
Cubert Utilities 2.0.6 contains perClass 5.2 installation.
During installation, make sure "perClass Toolbox and SDK" and "DEMO classifier Plug-ins (perClass)" options are selected.
B.3. Nut classifier FREE demo example ↩
perClass bundled with Cubert utils contains a free demo example classifier for nuts (amandels, peanuts).
Download the nut data set from
http://perclass.com/files/cubert/perclass_nut_data_set_jun17.zip(45MB)Extract files to your local hard drive
Start CubertUtilsCore.exe and CubertUtilsTouch.exe interface
Go to off-line image interface
- Select path where the nut data set was extracted (by clicking the "Change local directory" link):
- You should see several gray-level PAN images of nuts:
- We will now apply the pre-trained nut classifier that can distinguish peanuts and amandels. Click on the first one containing both peanuts and amandels and select the "Nuts" plugin in the view mode:
- The classifier will be applied to the hyperspectral image recording with 138 channels. Amandels get classified in red and peanuts in green.
- Returning back the multiple-image view, we can see decisions overlaid on all images. Note, that the classifier was trained only on peanuts, amandels and background. Therefore, other types of objects such as hand and other nuts (pine seeds, walnuts) are misclassidied as one of the known categories.
B.4. Activating perClass Demo ↩
In order to develop your own custom classifier with perClass, a demo license needs to be activated. There are two components of the software,namely perClass Toolbox for Matlab and perClass Runtime integrated in Cubert Utils. Therefore, two activation keys are needed.
B.4.1. Activating perClass Runtime ↩
perClass Runtime is activated from within Cubert Utils Touch interface. Go to settings and others (three-dot symbol). Click on "Activate perClass" link.
Fill-in the activation code "RUNTIME-XXXX-XXXX-XXXX-XXXX".
B.4.2. Activating perClass Toolbox ↩
perClass Toolbox is activated from within Matlab environment.
Start Matlab
Add perClass Toolbox on Matlab path using "Set Path" button. The perClass installation directory is under CubertFuchsia directory, e.g. at
c:\Program Files\CubertFuchsia\perClass\perClass_Pro_5.1_29may17\perclass.
Activation is initiated by the sdactivate command. Provide activation key formated as "TOOLBOX-XXXX-XXXX-XXXX-XXXX"
>> sdactivate('TOOLBOX-1699-2295-1553-5145')
perClass Demo 5.1 (30-May-2017), Copyright (C) 2007-2017, PR Sys Design, All rights reserved
Demo license. For evaluation purposes only.
Customer: () Issued: 21-sep-2017
Toolbox with DB,imaging,export: The license expires on 24-sep-2017.
Activation successful, the license file written to 'C:\Program Files\CubertFuchsia\perClass\perClass_Pro_5.1_29may17\perclass\perclass.lic'.
ans =
1
B.5. Building nut classifier ↩
In this example, we build a custom nut classifier classifying peanuts and almonds. The data set contains the following images:
image index classes
----------------------------------------
5, 7 peanuts + almonds
9 peanuts only
29 almonds only
19 hand
47,54 pine seeds only
70,71 walnuts only
79, 80 almonds (normal + open)
Load a hyperspectral image. We will first load the 002_Auto_009_snapshot_cube.tiff image containing only peanuts.
>> im9=sdimage('perclass_nut_data_set_jun17\v002\000k\002_Auto_009_snapshot_cube.tiff','sddata')
2500 by 138 sddata, class: 'unknown'
>> sdimage(im9)
ans =
1
The sdimage command with 'sddata' option returns a data set containing 2500 image pixels, each with 138 wavelengths. In machine learning terminology, pixels are sampels of our data set and wavelengths are features. Each data sample in perClass has a class label, by default 'unknown'.
Second command opens visualization figure showing image content. By clicking up/down cursor, we can change image band. Note, that some bands show high-values for few (outlier) pixels. While for genuine data, values range in low thousands, for these pixels, some wavelength values are over 60000.
We will first remove these outlier pixels. We could, for example, project
the data in suitable subspace and hand-draw polygon classifier enclosing
the good data domain. There is, however, an easier way removing these
outliers using sdbox command. We may provide it with a range of valid
data values. As this solution is simpler and more robust, we will adopt it
in this example.
Note, that we need sdbox command from perClass 5.2. If your perClass
version is 5.1 (which can be checked with sdversion command), you need to
download sdbox.m and sdbox.p files from http://perclass.com/cubert/ and copy
them to perclass directory. Exact location of your perclass directory is
displayed with sdversion command. Older sdbox files can be overwritten.
We will use sdbox to accept only data in the data range 0 to 50000:
>> p=sdbox(im9,'bounds',[0 50000])
sequential pipeline 138x1 'Box classifier+Decision'
1 Box classifier 138x1
2 Decision 1x1 threshold on 'inside'
If sdbox command with 'bounds' option returns 'unknown option bounds' error, you use older version from perClass 5.1. Make sure the files you downloaded above were correctly copied to the perclass directory.
The sdbox command returns a "pipeline" which can be applied to new data by multiplication. Result is a decision 'inside' or 'outside':
>> dec=im9*p
sdlab with 2500 entries, 2 groups: 'inside'(2457) 'outside'(43)
We only preserve pixels that fall inside the above-defined range:
>> im9( dec=='inside' )
2457 by 138 sddata, class: 'unknown'
Visualizing the image content without outliers:
>> sdimage( im9( dec=='inside' ) )
ans =
4
Now, we need to label objects and background. One way would be to use label painting feature. Faster alternative is to apply data clustering and discover underlying clusters.
To initiate k-means clustering procedure from image figure, press 'k'. A dialog will open asking for number of clusters. In our case, we're interested in two, assuming there is a clear separation between background and object spectra. The clustering result is shown on the following screenshot:
We have enabled image legend (by pressing 'l' key or clicking legend button on the toolbar). Note, that the names of clusters are automatically generated C1 and C2. We need to interpret which one is background and which one represents peanuts. Note also, that re-running the clustering may result in "flipped" naming due to random initialization. If you wish to re-run the clustering, first move to the original 'class' labels by perssing '1' or via "Image/Use labels/1 class" manu as the clustering result forms a new 'cluster' label set. Re-running clustering on 2-class 'cluster' labels would apply k-means to each class separately and produce four cluster result.
In order to define understandable names for the clusters, move mouse pointer to some background pixel and press 'r' to rename the cluster name into 'back' (Tip: in order to operate the Matlab default dialog without mouse, use the TAB key to move from the edit field to the OK button and press ENTER)
Similarly, rename the cluster representing objects in the image into 'peanut'. We may now save the data set into Matlab workspace by pressing 's' key (or "Image/Create data set in workspace" menu) and filling variable name (e.g. 'A').
Creating data set A in the workspace. 2457 by 138 sddata, 2 'cluster' groups: 'back'(1961) 'peanut'(496)
We will now load an image with almonds, remove outliers, cluster and define 'back' and 'almond' classes. Before saving the image data set to Matlab workspace variable 'B', make sure that the color or 'almond' class is not red. You may, for example, move mouse to a 'almond' pixel, press 'c' and select the color from a dialog or press '.' (dot) to randomly change color of class under cursor. The colors defined in the image data set will be propagated to the trained classifier and later to visualization in Cubert Utils.
>> im29=sdimage('perclass_nut_data_set_jun17\v002\000k\002_Auto_029_snapshot_cube.tiff','sddata')
2500 by 138 sddata, class: 'unknown'
>> sdimage( im29( im29*p=='inside' ) )
ans =
6
Creating data set B in the workspace.
2428 by 138 sddata, 2 'cluster' groups: 'back'(2017) 'almond'(411)
B.6. Training Peanut/Amandel classifier ↩
We will join data sets A and B to form a full training set tr:
>> tr=[A;B]
4885 by 138 sddata, 3 'cluster' groups: 'back'(3979) 'peanut'(495) 'almond'(411)
Note that vertical cat is used (denoted by semi-colon), because we join another set of samples in the same 138D feature-space. The resulting data set contains 4885 samples each labeled into one of the three classes, namely 'back', 'peanut' or 'almond'.
In order to build a classifier, we first perform PCA (Principal Component Analysis) to reduce input 138D representation into a lower, e.g. 20D subspace. Hyperspectral data can be typically compressed using PCA to few important dimensions without significant loss of information.
>> p1=sdpca(tr,20)
PCA pipeline 138x20 100% of variance
The sdpca command returns a pipeline that starts from 138D space and
generated a new 20D data set. We may apply this pipeline to any 138D
data set:
>> tr2=tr*p1
4885 by 20 sddata, 3 'cluster' groups: 'back'(3979) 'peanut'(495) 'almond'(411)
We may visualize tr2 as any other data set using interactive scatter plot:
>> sdscatter(tr2)
ans =
1
When opening scatter plot on our data set, some classes may have identical markers. It is useful to show leggend ('l' key or toolbar button). Class marker may be changed by right-click and "change marker" command in context menu. Into a dialog box, we may then fill-in the desired matlab marker such as 'r.' for red dot. Another usful option is to switch on the distribution plots by pressing 'd' or via toolbar.
We can see that the three classes in our data set have a uni-modal shape. Therefore, we will use Gaussian classifier:
>> pg=sdgauss(tr2)
sequential pipeline 20x1 'Gaussian model+Decision'
1 Gaussian model 20x3 full cov.mat.
2 Decision 3x1 weighting, 3 classes
This will train a Gaussian model on each of the classes and add a default decision step.
In order to create a pipeline that can be applied to raw spectra with 138 wavelengths, we need to join the PCA and classification steps:
>> P1=p1*pg
sequential pipeline 138x1 'PCA+Gaussian model+Decision'
1 PCA 138x20 100%% of variance
2 Gaussian model 20x3 full cov.mat.
3 Decision 3x1 weighting, 3 classes
Now, we may want to visualize classifier decision on an unseen image. Let us use the image 5 that contains both peanuts and almonds:
>> im5=sdimage('perclass_nut_data_set_jun17\v002\000k\002_Auto_005_snapshot_cube.tiff','sddata')
2500 by 138 sddata, class: 'unknown'
We show it without outliers:
>> sdimage( im5( im5*p=='inside' ) )
ans =
2
In the image Figure, select "Image/Apply classifier..." command from menu or press 'd'. A dialog will open where we can specify a pipeline name ('P1' in our case).
B.7. Exporting perClass classifier for exacution from Cubert Utils ↩
In order to execute the classifier, we trained, outside Matlab in Cubert
Utils, we need to export it. This is accomplished with sdexport command
and 'Cubert' option. Cubert utils stores user-defined plugins in
Bin\user\plugin directory (.e.g `c:\Program
Files\CubertFuchsia\Bin\user\plugin`).
We will first change name of P1 pipeline to a descriptive string and
then export it as Cubert plugin:
>> P1=setname(P1,'Almond/Peanut')
sequential pipeline 138x1 'Almond/Peanut'
1 PCA 138x20 100%% of variance
2 Gaussian model 20x3 full cov.mat.
3 Decision 3x1 weighting, 3 classes
>> sdexport(P1,'c:\Program Files\CubertFuchsia\Bin\user\plugin\clf1.xml','cubert')
Exporting pipeline to Cuber plugin file..ok
This pipeline requires perClass runtime version 4.0 (11-jul-2013) or higher.
We need to close and re-open Cubert Utils Touch as a list of user plugins is refreshed when starting. When we move to the list of images and select View mode, a new 'clf1' classifier option will appear. Selecting it, we will see almonds detected with a color we selected earlier in image figure:
We may note, that the split open almonds in images 79 and 80 are misclassified as peanuts.
Let us improve our classifier to accomodate split almonds. We will load the image 79, remove outliers:
>> im79=sdimage('C:\Users\pavel\Desktop\Cubert_share\v002\000k\002_Auto_079_snapshot_cube.tiff','sddata')
2500 by 138 sddata, class: 'unknown'
>> sdimage( im79( im79*p=='inside' ) )
ans =
3
We cluster the image to three clusters and label 'almond-open' separately. Resulting data set is saved into variable 'C':
Creating data set C in the workspace.
2475 by 138 sddata, 3 'cluster' groups: 'back'(1936) 'almond'(355) 'almond-split'(184)
>> tr3=[A;B;C]
7360 by 138 sddata, 4 'cluster' groups: 'back'(5915) 'peanut'(495) 'almond'(766) 'almond-split'(184)
>> p3=sdpca(tr3,20)
PCA pipeline 138x20 100% of variance
In addition to PCA, we use here also LDA (linear discriminant analysis also known as Fisher projection) to derive low-dimensional class-separating subspace:
>> p4=sdlda(tr3*p3)
LDA pipeline 20x3
Again training Gaussian model:
>> pg=sdgauss(tr3*p3*p4)
sequential pipeline 3x1 'Gaussian model+Decision'
1 Gaussian model 3x4 full cov.mat.
2 Decision 4x1 weighting, 4 classes
>> P2=p3*p4*pg
sequential pipeline 138x1 'PCA+LDA+Gaussian model+Decision'
1 PCA 138x20 100%% of variance
2 LDA 20x3
3 Gaussian model 3x4 full cov.mat.
4 Decision 4x1 weighting, 4 classes
>> P2=setname(P2,'Almond/Almond-Open/Peanut')
sequential pipeline 138x1 'Almond/Almond-Open/Peanut'
1 PCA 138x20 100%% of variance
2 LDA 20x3
3 Gaussian model 3x4 full cov.mat.
4 Decision 4x1 weighting, 4 classes
>> sdexport(P2,'c:\Program Files\CubertFuchsia\Bin\user\plugin\clf2.xml','cubert')
Exporting pipeline to Cuber plugin file..ok
This pipeline requires perClass runtime version 4.0 (11-jul-2013) or higher.
We close and re-open Cubert Utils Touch and use 'clf2' plugin. The split almonds are now highlighted in yellow.
B.8. Where to go from here? ↩
We have seen how to use perClass to define labels and build classifiers and then export them to Cubert Utils.
- You may experiment with the nut data set to include other classes (pine seeds, walnuts)
- You may wish to use another model. For example, Support Vector Classifier (sdsvc), neural net (sdneural), random forest (sdrandforest) etc.
Example of building SVC classifier on PCA-projeted spectra:
>> tr3
7360 by 138 sddata, 4 'cluster' groups: 'back'(5915) 'peanut'(495) 'almond'(766) 'almond-split'(184)
>> ps=sdsvc(tr3*p1,'rbf','C',10,'one-against-one')
>> P4=p1*ps
You may want to include rejection decition discarding all data unseen in training
>> P5=sdreject(P3,tr3,'reject',0.01)
etc.