Comparative Evaluation of Performance Measures for Human Driving Skills

Abstract

In this paper, we evaluate the adequacy of several performance measures for the evaluation of driving skills between different drivers. This work was motivated by the need for a training system that captures the driving skills of an expert driver and transfers the skills to novice drivers using a haptic-enabled driving simulator. The performance measures examined include traditional task performance measures, e.g., the mean position error, and a stochastic distance between a pair of hidden Markov models (HMMs), each of which is trained for an individual driver. The emphasis of the latter is on the differences between the stochastic somatosensory processes of human driving skills. For the evaluation, we developed a driving simulator and carried out an experiment that collected the driving data of an expert driver whose data were used as a reference for comparison and of many other subjects. The performance measures were computed from the experimental data, and they were compared to each other. We also collected the subjective judgement scores of the driver's skills made by a highly-experienced external evaluator, and these subjective scores were compared with the objective performance measures. Analysis results showed that the HMM-based distance metric had a moderately high correlation between the subjective scores and it was also consistent with the other task performance measures, indicating the adequacy of the HMM-based metric as an objective performance measure for driving skill learning. The findings of this work can contribute to developing a driving simulator for training with an objective assessment function of driving skills.