Human Motion Prediction by 2D Human Pose Estimation using OpenPose

EasyChair Preprint 2580

Abstract

The prediction of human motion became an important issue, considering that it can be utilized to solve plenty of problems for autonomous systems. In the case of human-machine interaction such as an autonomous car or robot that works in a human living environment need to predict the human's future movement for its moving trajectories. Some of the previous researches used Kinect camera which has a depth sensor that the camera used to detect the pose of a human body. However, in this research we start with using the RGB camera as the other option that we can rely on. We set a goal to predict 1 second ahead of the motion which includes simple motions such as hand gesture and walking movement. We used OpenPose library from OpenCV to extract features of a human body pose including 14 points. YOLOv3 is used to crop the main feature in the frames before OpenPose process the frame. We input distance and direction which are calculated from the features by comparing two consecutive frames into the Recurrent Neural Network Long Short-Term Memory (RNN-LSTM) model. As the result, the human movement was predicted with 98% of accuracy. The evaluation criteria for acceptable distance was within 1.8% of diagonal frame length. We confirmed the validity of the RGB based method in the simple human motion case from the result, and we conclude that this is an important step to realize the prediction of more complex human motion.

Keyphrases: OpenPose, RNN-LSTM, YOLOv3, deep learning, human motion prediction

@booklet{EasyChair:2580,
  author    = {Andi Prademon Yunus and Nobu C. Shirai and Kento Morita and Tetsushi Wakabayashi},
  title     = {Human Motion Prediction by 2D Human Pose Estimation using OpenPose},
  howpublished = {EasyChair Preprint 2580},
  year      = {EasyChair, 2020}}