One step ahead: Detecting unusual human motions

QualityMinds GmbH provides services in quality assurance and testing for software and machine learning systems. They specialize in software engineering, requirements engineering, machine learning, and AI testing, including testing machine learning models for autonomous driving.

As part of a project with QualityMinds, our students had the opportunity to explore the fascinating world of human motion prediction. Human motion prediction involves forecasting future human movements based on a time sequence of a given body position and recent motions. QualityMinds uses advanced Deep Neural Networks, such as Graph Convolution Networks, to predict a person's actions, looking up to one second into the future. However, some actions have been difficult to predict accurately, which led to the development of this project.

The primary goal was to quantify anomalies in human motions, as these unusual movements pose challenges for the prediction models. To achieve this, Alaa, Vincent, and Jonas utilized the Human 3.6M public dataset, which contains 3.6 million human motion action sequences.

They then applied four distinct outlier detection models, each offering a unique perspective on identifying outlying motions. To validate their findings, the students compared the prediction errors from the human motion prediction models with the identified outliers. Through this comparison, they demonstrated a direct connection between outliers and failed motion predictions.

Our project produced three main results, each aimed at enhancing QualityMinds' motion prediction capabilities:

• Outlier Detection App: An interactive tool for flexible analysis of outlier sequences.
• Outlier Validation App: A tool designed to find correlations between a motion sequence's anomaly degree (precision score) and its prediction error.
• Kinematic Comparison Toolkit: A toolkit to compare and visualize inliers and outliers for specific actions, such as walking or eating, based on kinematic key characteristics like joint velocity and acceleration.

In conclusion, our collaboration with QualityMinds has provided them with tools to improve human motion prediction for autonomous driving and other applications. By incorporating information about identified outliers, QualityMinds can enhance the accuracy of motion prediction models used in human-robot interaction and autonomous systems, ensuring a safer and more efficient future for all. Moving forward, the team aims to expand these insights by generalizing them to other public datasets.

Alaa, Vincent, and Jonas are proud to have been part of this project and are excited to see the impact of their work in the field of autonomous technology.