MDAD: A Multimodal and Multiview in-Vehicle Driver Action Dataset

“Driver’s distraction is deadly!”. Due to its crucial role in saving lives, driver action recognition is an important and trending topic in the field of computer vision. However, a very limited number of public datasets are available to validate proposed methods. This paper introduces a new public, well structured and extensive dataset, named Multiview and multimodal in-vehicle Driver Action Dataset (MDAD). MDAD consists of two temporally synchronised data modalities from side and frontal views. These modalities include RGB and depth data from different Kinect cameras. Many subjects with various body sizes, gender and ages are asked to perform 16 in-vehicle actions in several weather conditions. Each subject drives the vehicle on multiple trip routes in Sousse, Tunisia, at different times to describe a large range of head rotations, changes in lighting conditions and some occlusions. Our recorded dataset provides researchers with a testbed to develop new algorithms across multiple modalities and views under different illumination conditions. To demonstrate the utility of our dataset, we analyze driver action recognition results from each modality and every view independently, and then we combine modalities and views. This public dataset is of benefit to research activities for humans driver action analysis.