Human Pose Estimation in Space and Time Using 3D CNN

TitleHuman Pose Estimation in Space and Time Using 3D CNN
Publication TypeBook Chapter
Year of Publication2016
AuthorsGrinciunaite, A, Gudi, AA, Tasli, HE, den Uyl, M
EditorHua, G, Jégou, H
Book TitleComputer Vision – ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III
PublisherSpringer International Publishing
ISBN Number978-3-319-49409-8

This paper explores the capabilities of convolutional neural networks to deal with a task that is easily manageable for humans: perceiving 3D pose of a human body from varying angles. However, in our approach, we are restricted to using a monocular vision system. For this purpose, we apply a convolutional neural network approach on RGB videos and extend it to three dimensional convolutions. This is done via encoding the time dimension in videos as the 3rd dimension in convolutional space, and directly regressing to human body joint positions in 3D coordinate space. This research shows the ability of such a network to achieve state-of-the-art performance on the selected Human3.6M dataset, thus demonstrating the possibility of successfully representing temporal data with an additional dimension in the convolutional operation.