Table 2 Prediction accuracy of the electronic nose data using machine learning algorithms
Model and parameters | Validation | Accuracy (95% CI) | Sensitivity | Specificity | PPV | NPV | Kappa | AUC (95% CI) |
|---|---|---|---|---|---|---|---|---|
k-nearest neighbors (k = 5) | Internal | 0.87 (0.79–0.93) | 0.88 | 0.87 | 0.86 | 0.88 | 0.74 | 0.94 (0.89–0.99) |
External | 0.94 (0.87–0.98) | 0.88 | 1.00 | 1.00 | 0.90 | 0.88 | 0.95 (0.90–1.00) | |
Decision tree (trials = 20, winnow = TRU, model = tree) | Internal | 0.73 (0.63–0.82) | 0.90 | 0.58 | 0.66 | 0.86 | 0.47 | 0.91 (0.84–0.98) |
External | 0.91 (0.84–0.96) | 0.92 | 0.90 | 0.90 | 0.92 | 0.82 | 0.94 (0.90–1.00) | |
Neural network (size = 1, decay = 0) | Internal | 0.89 (0.81–0.94) | 0.77 | 1.00 | 1.00 | 0.83 | 0.78 | 0.92 (0.86–0.97) |
External | 0.94 (0.87–0.98) | 0.88 | 1.00 | 1.00 | 0.90 | 0.88 | 0.94 (0.89–0.99) | |
Support vector machines (linear kernel) (C = 1) | Internal | 0.92 (0.85–0.96) | 0.83 | 1.00 | 1.00 | 0.87 | 0.84 | 0.94 (0.89–0.99) |
External | 0.92 (0.85–0.96) | 0.83 | 1.00 | 1.00 | 0.87 | 0.84 | 0.94 (0.89–0.99) | |
Support vector machines (radial kernel) (C = 1) | Internal | 0.95 (0.89–0.98) | 0.90 | 1.00 | 1.00 | 0.91 | 0.90 | 0.94 (0.89–0.99) |
External | 0.95 (0.89–0.98) | 0.90 | 1.00 | 1.00 | 0.91 | 0.90 | 0.94 (0.89–0.99) | |
Support vector machines (polynomial kernel) (degree = 3, scale = 0.1, C = 1) | Internal | 0.92 (0.85–0.96) | 0.83 | 1.00 | 1.00 | 0.87 | 0.84 | 0.94 (0.89–0.99) |
External | 0.94 (0.87–0.98) | 0.88 | 1.00 | 1.00 | 0.90 | 0.88 | 0.94 (0.89–0.99) | |
Random forest (mtry = 2) | Internal | 0.91 (0.84–0.96) | 0.90 | 0.92 | 0.91 | 0.91 | 0.82 | 0.92 (0.86–0.99) |
External | 0.79 (0.70–0.87) | 0.85 | 0.73 | 0.75 | 0.84 | 0.58 | 0.94 (0.90–0.99) |