heri-eval - evaluate classification algorithm
heri-eval [OPTIONS] dataset [--
SVM_TRAIN_OPTIONS]
heri-eval runs training algorithm on dataset and
then evaluate it using testing set, specified by option -e. If option
-n was applied, cross-validation is used for evaluation, training and
testing on different folds are run in parallel, thus utilizing available
CPUs. If -r is used, the dataset is splitted into training and
testing datasets randomly with the specified ratio, and then holdout is
run.
- -h, --help
- Display help information.
- -f
- Enable output of per-fold statistics. See -Mf.
- -n N
- Enable T*N-fold cross-validation mode and set the number of folds
to N.
- -r ratio
- Split the dataset into training and testing parts with the specified ratio
of their sizes (in percents).
- -t T
- Enable T*N-fold cross-validation mode and set the number of runs to
T which 1 by default.
- -e
testing_dataset
- Enable hold-out mode and set the testing dataset.
- -T threshold
- Set the minimum threshold for making a classification decision. If this
flag is applied, micro-average precision, recall, and F1 are calculated
instead of accuracy.
- -o filename
- Save predictions from testing sets to the specified file.
Format: outcome_class prediction_class [score]
- -O filename
- Save incorrectly classified objects to the specified file.
Format: #object_number: outcome_class prediction_class
[score])
- -m filename
- Save confusion matrix to the specified file.
Format: frequency : outcome_class prediction_class
- -p opts
- Pass the specified opts to heri-stat(1).
- -s opts
- Pass the specified opts to heri-split(1).
- -M chars
- Sets the output mode where chars are: t -- output total statistics, f --
output per-fold statistics, c -- output cross-fold statistics. The default
is "-M tc".
- -S seed
- Pass the specified seed to heri-split(1).
- -K
- Keep temporary directory after exiting.
- -D
- Turn on the debugging mode, implies -K.
heri-eval -e testing_set.libsvm training_set.libsvm -- -s 0 -t 0
export SVM_TRAIN_CMD='liblinear-train'
export SVM_PREDICT_CMD='liblinear-predict'
heri-eval -p '-mr' -n 5 training_set.libsvm -- -s 4 -q
heri-eval -p '-mr' -n 5 training_set.libsvm -- -s 4 -q
export SVM_TRAIN_CMD='scikit_rf-train --estimators=400'
export SVM_PREDICT_CMD='scikit_rf-predict'
heri-eval -p '-c' -Mt -t 50 -r 70 dataset.libsvm
- SVM_TRAIN_CMD
- Training utility, e.g., liblinear-train (the default is svm-train).
- SVM_PREDICT_CMD
- Predicting utility, e.g., liblinear-predict (the default is
svm-predict).
- SVM_HERI_STAT_CMD
- Utility for calculating statistics (the default is
heri-stat(1)).
- SVM_HERI_STAT_ADDONS_CMD
- Utility for calculating additional statistics (the default is
heri-stat-addons(1)).
- SVM_HERI_SPLIT_CMD
- Utility for splitting the dataset (the default is
heri-split(1)).
- TMPDIR
- Temporary directory (the default is /tmp).
<http://github.com/cheusov/herisvm>