Human Activity Recognition using Binary Motion Image and Deep Learning

Abstract

View based recognition methods use visual templates for recognition and hence do not extract complex features from the image. Instead they retain the entire raw image as a single feature in high dimensional space. These example images or templates are learnt under different poses and illumination conditions for recognition. With this in mind, we build on the idea of 2-D representation of action video sequence by combining the image sequences into a single image called Binary Motion Image (BMI) to perform human activity recognition. For classification, we employ Convolutional Neural Networks which inherently provide slight invariance to translational and rotational shifts, partial occlusions as well as background noise. We test our method on Weizmann dataset focusing on actions that look similar like run, walk, jump, side and skip actions. We also extended our method to 3-D depth maps using MSR Action3D dataset by extracting three BMI projections namely the front view, the side view and the top view. From the results obtained, we believe that BMI is sufficient for activity recognition and has shown to be invariant to speed of the action performed in addition to the aforementioned variations. � 2015 Published by Elsevier B.V.