Cheat-sheetSDCROSSVAL Perform cross-validation S=SDCROSSVAL(ALG,DATA,options) [S,RES,E]=SDCROSSVAL(ALG,DATA,options) Custom crossvalidation (no testing, arbitrary output) [RES,E]=SDCROSSVAL(ALG,DATA,'notest',options) INPUTS ALG untrained algorithm or pipeline returning decisions DATA dataset to perform cross-validation on OPTIONS 'method' cross-val method: rotation (def),randomization,leave-one-out 'folds' number of folds to perform (default: 10) 'seed' random seed (default: no seed set) 'ops',PD set of operating points to estimate ROC with variances 'prox' If DATA is a square proximity matrix, 'prox' option makes sure the test set will only be represented by training prototypes (features). This is important to avoid positive bias. 'notest' No testing is performed. ALG may return anything. 'measures',M Cell array with measure names and parameters. OUTPUT S String summarizing the results (mean/std for each perf.measure) RES Structure with estimated performances per fold E Evaluation object storing per-fold trained algorithms DESCRIPTION SDCROSSVAL performs N-fold cross-validation of untrained algorithm, pipeline or mapping ALG. Rotation, randomization and leave-one-out schemes are supported. In the default rotation mode, the DATA is split into N folds (splitting each class separately). Repeatedly, N-1 subsets are used to train ALG and the performance is estimated on the Nth subset. Randomization splits DATA using RANDSUBSET method. By default 50% of samples are used for training, the rest for testing. The numerical value following the 'random' option is passed to RANDSUBSET. The leave-one-out is run over samples by default but may be executed on all unique categories of a sample property using the 'over' option. This allows us to cross-validate algorithm over patients or objects. If 'ops' option is used with a operating points defined in SDROC object or via SDDECIDE function, SDCROSSVAL estimates ROC with variances at these op.points. EXAMPLES Rotation over 20 folds S=SDCROSSVAL(ALG,DATA,'folds',20) Rotation computing user-specified performance measures S=SDCROSSVAL(ALG,DATA,'measures',{'TPr','apple','precision','apple'}) Randomization, use 80% of DATA for training S=SDCROSSVAL(ALG,DATA,'method','rand',0.8) Randomization, use 100 samples per class for training S=SDCROSSVAL(ALG,DATA,'method','rand',100) Leave-one-out S=SDCROSSVAL(ALG,DATA,'method','loo') Leave-one-out over patients labels S=SDCROSSVAL(ALG,DATA,'method','loo','over','patient') READ MORE http://perclass.com/doc/guide/evaluation.html#intro
sdcrossval is referenced in examples: